(defun wrandf (xs weights)
    (loop with r = (random (apply #'+ weights))
          for x in xs
          for w in weights
          for cw = w then (+ cw w)
          when (> cw r) return x))

  ; assumes pair
  (defmacro wrand (&rest args)
    `(funcall
       (wrandf
         (list ,@(mapcar (lambda (x) `(lambda () ,(cadr x))) (pair args)))
         (list ,@(mapcar #'car (pair args))))))

http://arclanguage.org/item?id=3471

The major difference between my version and yours is that you require each constructor to belong to one "union" type, whereas in my version each constructor forms its own type. The latter approach seems more Arc-like to me. It's simpler, and in a dynamic language I don't think you should be forced to stuff constructors into fixed categories. (I'm not saying that such categories are useless, just that variant tags shouldn't be inextricably tied to them.)

You may want to look at OCaml's polymorphic variants, which are closer to what I had in mind for Arc:

http://caml.inria.fr/pub/docs/manual-ocaml/manual006.html#ht...

The auto-binding of names to fields seems cool, but I'd need to practice it to see if it was a good strategy overall. You've already managed to shadow car and cdr, which could cause a lot of code to fail mysteriously (think macroexpansions, e.g. each). Also, if the user binds the names explicitly, you get destructuring for free.

The clearest advantage of distinct types is in things like binary trees:

  (tunion binary-tree
    branch (left  binary-tree?)
           (right binary-tree?)
    leaf   (value object))

  (vtype branch left right)
  (vtype leaf   value)

  (tunion my-type
    my-type (foo pred)
            (bar int?))

  (mac vtype (type . arguments)
    `(tunion ,type ,type ,@arguments))

  (vtype variant1 (x pred))
  (vtype variant2 (a int?) (b object))
  
  (tcase var
    (variant1
      variant1 (do-something var))
    (variant2
      variant2 (do-something-else var))
    else
      (fail))

  (vcase var
    variant1 (do-something var)
    variant2 (do-something-else var)
             (fail))

I feel like there ought to be a better way to do this--something that's less complex but handles more cases. In particular I'd like to be able to alternate OO and pattern-matching styles on the same datatypes. However, I haven't been able to come up with a solution that really satisfies me.

Regarding (car nil) and (cdr nil), these return nil because it's sometimes convenient. For instance, here's a function that collects every other item in a list:

  (def foo (xs)
    (if xs
      (cons car.xs (foo cddr.xs))))

  (def foo (xs)
    (if xs
      (cons car.xs (if cdr.xs (foo cddr.xs)))))

  (defho while ([test] . [f])
    ...)

  (defho awhen (x . [f it])
    ...)

Then perhaps you can define 'defho in terms of 'mac and 'w/hof, maybe? The Arc solution: throw more macros at it.

Regarding (x . [f it]) :

In canonical arc2, [f it] is (fn (_) (f it)), so (x . [f it]) becomes (x fn (_) (f it)) . In Anarki, [f it] is (make-br-fn (f it)), so (x . [f it]) becomes (x make-br-fn (f it))

This makes the use of [] problematical.

'w/hof is trivially implementable, 'defho much less so due to the syntax.

w/hof is also more general, at the expense of being more talkative.

In fact it might be better to do things this way:

  (mac while (cond . body)
    (w/hof (cond (fn () ,cond)
            body (fn () ,@body))
      ((afn ()
         (when (cond)
           (body)
           (self))))))

  (mac aif (cond then (o else))
    (w/hof (cond ,cond
            then (fn (it) ,then)
            else (fn () ,else))
      (if cond
          (then cond)
          (else))))

My examples are from the core language, obviously, but the suggestion would benefit anyone wanting to extend the language with their own macros.

Further, using arc.arc as a basis is pointless; the reason those macros are there is because it's a waste of time to rewrite them. Most of the macros I've written in Arc depend on new syntax, not just rearranging expressions: look at p-m: and w/html , which I doubt are possible in ordinary HOF style.

So yes: while it certainly looks interesting, it doesn't seem to leverage the "common enough" theme quite enough.

Why not just leave unhygienic macros for the anaphora?

> Another possible solution would be to just have a very concise syntax for function literals.

No, really, macros, being essentially compilers, give you enough power to build everything you'd ever want into Lisp, including "hygienic" macros, and even to build facilities to write hof-like things in less painful ways.

Obviously I like unhygienic macros when they are necessary. The problem is, I keep writing higher-order functions and getting tired of typing the "fn" over and over, and then I have to convert the function to a macro, making it twice as long and hard to read. Hygienic macros help, but they still are not as easy to write as plain functions.

I know I'm not the only person who has this problem, but maybe I'm the only one who realizes I have this problem. I see people on the Internet raving about the AMAZING POWER of macros, and most of their examples are just higher-order functions with some small cosmetic changes. Most of the macros in Arc are of the same kind.

I'm not denying that macros are powerful. I just think there is a gross inefficiency in using them where you shouldn't have to, just because of minor syntactic concerns.

I wanted to solve this with a short, clean syntax for function literals, but I haven't been able to come up with one and neither, apparently, has anyone else. So instead I decided to try something that would generate macros out of HOFs.

Now, maybe that's a bad idea; instead, we could redefine the brackets, so that a pipe divides args and body, and if it doesn't have a pipe, it defaults to binding _ ? I don't know how hard that would be, or some variation on the concept, but it would be a bit shorter than (fn (args) (body)), if you don't want to type that all the time.

  [params || body]

  (make-br-fn (params || body))

  (let old (rep make-br-fn)
    (= make-br-fn
       (annotate 'mac
         (fn (l)
           (if (some '|| l)
               (do
                 your-stuff)
               (old l))))))

Other possibilities, in no particular order:

  [ # ]
  [ - ]
  [ = ]
  [ -> ]
  [ : ]
  [ => ]
  [ > ]
  [ ~ ]
  [ % ]
  [ ! ]
  [ $ ]
  [ ^ ]
  [ & ]
  [ * ]
  [ @ ]
  [ + ]
  [ | ]
  [ || ]
  [ ? ]

# can't be redefined

Let's try some mockups:

  [ a b c :
    (/ (+ (- b) (sqrt (+ (* b b) (* 4 a c))))
       (* 2 a))]

  [: (thunk this)]

  [ a b c ->
    (/ (+ (- b) (sqrt (+ (* b b) (* 4 a c))))
       (* 2 a))]

  [-> (thunk this)]

  [ a b c =>
    (/ (+ (- b) (sqrt (+ (* b b) (* 4 a c))))
       (* 2 a))]

  [=> (thunk this)]

  [foo [bar]]

  (make-br-fn (foo (make-br-fn (bar))))

  (given ; this gives us access to the old
         ; implementation of [] syntax; it
         ; is used when we don't find the
         ; separator
         old (rep make-br-fn)
         ; use a variable to easily change
         ; the separator
         separator ': ;for experimentation
    (= make-br-fn
       ; a macro is just a function that has
       ; been tagged (or annotated) with the
       ; symbol 'mac
       (annotate 'mac
         ; the reader reads [...] as
         ; (make-br-fn (...))
         (fn (rest)
               ; find the separator
           (if (some separator rest)
               ; note the use of the s-variant givens
               ; the "s" at the end of the name of givens
               ; means that the variables are specifically
               ; bound in order, and that succeeding variables
               ; may refer to earlier ones
               (givens ; scans through the list, returning
                       ; an index for use with split
                       ; (no built-in function does this)
                       scan
                       (fn (l)
                         ((afn (l i)
                            (if (caris l separator)
                                i
                                (self (cdr l) (+ i 1))))
                          l 0))
                       ; now do the scan
                       i (scan rest)
                       ; this part destructures a two-element
                       ; list
                       (params body)
                         ; used to get around a bug in split
                         (if (isnt i 0)
                             (split rest i)
                             (list nil rest))
                 ; it just becomes an ordinary function
                              ; body includes the separator,
                              ; so we also cut it out
                 `(fn ,params ,@(cut body 1)))
               ; pass it to the old version of make-br-fn
               ; if a separator was not found
               (old rest))))))

  (= foo [ i :
           [ : i]])

  ((foo 42))

  (read-accept-bar-quote #f)

  {| body}

  (afn ()
    (your-code))
  =>
  (let self nil
    (= self
      (fn ()
        (your-code))))

  (aif x
    (your-code))
  =>
  (let it x
    (if it
      (your-code)))

  (n-of n (readb stream))

  (def partition (f xs)
    (if (alist xs)
      (let f (testify f)
        ((afn (xs a b)
           (if (no xs)
             (list (rev a) (rev b))
             (f (car xs))
               (self (cdr xs) (cons (car xs) a) b)
               (self (cdr xs) a (cons (car xs) b))))
         xs nil nil))
      (map [coerce _ 'string] (partition f (coerce xs 'cons)))))

  (mac accums (accfns . body)
    (let gaccs (map [uniq] accfns)
      `(withs ,(mappend (fn (gacc accfn)
                          (list gacc 'nil accfn `[push _ ,gacc]))
                        gaccs accfns)
         ,@body
         (list ,@(map [list 'rev _] gaccs)))))

  (def partition (f xs)
    (if (alist xs)
      (let f (testify f)
        (accums (a b) (each x xs ((if (f x) a b) x))))
      (map [coerce _ 'string] (partition f (coerce xs 'cons)))))

accums seems good... I suggest pushing it on nex-3's arc-wiki.

What I would like to see is a proper order-preserving accum. Here's a try at a "reasonably efficient" implementation:

  (mac accumf (accfn . body)
    " Collects or accumulates the values given to all calls to `accfn' within
      `body' and returns a list of those values.  The returned list
      is in the same order as calls to `accfn'.
      See also [[accum]] [[summing]] "
    (w/uniq (hd tl)
      `(let (,hd ,tl ,accfn) nil
            (= ,accfn
              (fn (s)
                (if ,hd
                    (do (= ,hd (cons s nil)) (= ,tl ,hd))
                    (do (= (cdr ,tl) (cons s nil)) (= ,tl (cdr ,tl))))))
            ,@body
            hd))))

  arc> (macex '(compose no litmatch))
  (fn gs1639 (no (apply litmatch gs1639)))
  arc> (macex '(complement litmatch))
  (fn gs1641 (no (apply litmatch gs1641)))
  arc> ((compose no litmatch) "a" "aston")
  nil
  arc> ((complement litmatch) "a" "aston")
  Error: "vector-ref: expects type <non-negative exact integer> as 2nd argument,
  given: \"a\"; other arguments were: #3(tagged mac #<procedure>)"

  (apply or args)

  (eval (cons or args))

Consider again the very similar case of Arc tables being unable to differentiate "key not found" from "key associated with nil". I was halfway thru a brilliant retort to pg on this with an example from my own real live working code (where I used CL's ability to differentiate the cases) when I realized Doh! I totally did not need the capability (and man was I happy I realized that before trying to school pg <g>).

Besides the kludginess of mem, consider how easy it is to screw up with find. How do you verify that a list doesn't contain any nils? Without a type system to enforce such invariants you get the worst kind of bugs: those that are manifest only once in a blue moon and present no obvious diagnosis.

That similar problems arise with other uses of nil suggests to me that we're missing an abstraction.

   (all idfn lst)

Meanwhile, are you sure (member nil lst) is a kluge? It sure looks readable, and it sure does work. What exactly bothers you about that?

Meanwhile, a use case means your are not just running code samples you typed in at the repl, it means you are trying to solve a problem using a computer and something came up. So here we are after iterating over the members of the library and we built up a list of overdue books per member -- no, that would not have nils, that would have an assoc entry (bob . nil). Hmmm... can you help me here, I cannot think of one.

You have not responded to the very similar example of tables. Correct, one cannot distinguish the absence of a key from a key added with the value nil. A use case is not a couple of lines of code proving that, a use case is a lookup on drivers license number during a traffic stop and...?

My point you may have missed is that without a use case not perfectly well expressed with either of the two solutions I offered above, the fundamental weakness is in offering a theoretical functional gap in a language as if it were a gap in a language.

Recall that pg is already on record as wanting things that are problems in practice, that this is a hacker's language, not a strait jacket language like C++ or Java with all sorts of theoretical angst over untyped variables.

   (= (cdr prior-kill-me) (cdr kill-me))

   (= (car kill-me) nil)

    data Elem = Elem String deriving (Show, Eq)
    instance Ord Elem where
        Elem x < Elem y = length x < length y
    
    tree1 = insert (Elem "Some string") Empty
    tree2 = remove (Elem "Some string") tree1