Ah, interesting! Here's some background that might help understand why having them act like tables might help. `len` wasn't the first function that I found didn't work with templates; it was just the simplest. I was working on adding teardown functionality to unit-test.arc, and I wanted to look at some of the suites that were created, to see if I was adding tests properly. As the templates end up pretty big (one suite with two tests is about 25 lines), and it was late at night, I figured I'd make it simple on myself, and just look at the keys of the template.

This had nothing to do with the desired "production" code, but only with REPL-hacking introspection.

   (keys rep.tem)

edit: I was also assuming templates type-hinted field values but I don't think they do. Maybe they should?

I was very proud in Wart that multi-branch ifs were a library feature. And check out this quote in a blog post about Forth:

    : (   41 word drop ; immediate
    ( That was the definition for the comment word. )
    ( Now we can add comments to what we are doing! )

---

I disagree with your vision for templates. If you just want something that behaves like tables, why not just use tables? A helper that fills in default values would be pretty easy to write.

Yeah, I've seen projects that show off the power of a language by doing "x in < 100 lines" that just don't count a remote API call with half a million LOC running on a server or something ;) But with language features like macros and templates that have become ubiquitous, I feel like it's kind of cheating not to just fold them into arc proper.

But that's just me... one thing I've learned being here is that I seem to flow against the culture more than with it, so I can just agree to disagree.

> If you just want something that behaves like tables, why not just use tables?

They are tables, that's what's frustrating. They're tables with metadata. From what I can tell reading earler posts about templates, they used to be something that behaves like tables. Interop between forum data and Racket (and any code where tables are expected) is awkward because that incompatibility has to be worked around, resulting in extra code and extra complexity. Templates need a separate API despite having the same behavior as tables.

>If so, you need some way to distinguish tables that were generated by templates. Which implies something that manages them throughout their life cycle.

Fair enough. But why is it necessary to change their type to do so? Why not make this a feature of tables as a whole, if it's useful? Or tag tables in a way that doesn't change their type, if that's possible?

Tables and objects feel like separate concepts, and they have complementary strengths and weaknesses, and one doesn't subsume the other. To me it seems obvious that if we want to have both, we need them to have different types.

For example, sometimes you want the 'dynamic' ability to set arbitrary keys of metadata on a thing. Sometimes you want the same operation to be an error, by providing a schema. How would a single type do both? No language does so, to my knowledge.

Things should have the same type when they have compatible behavior. When they are incompatible, they shouldn't.

Supporting helpers like len and keys may well still make sense. And as the original story did, this is easy to do.

But in general, having incompatible types easily share functions without sharing too much is an open problem: https://en.wikipedia.org/wiki/Expression_problem A language can easily add a method to many types, or add a new type to many methods. But we don't yet know how to achieve both sides.

And honestly, I think the expression problem isn't important. It doesn't take too much code per method/type. And making it easier just encourages large, bloated codebases.

> ...given how often I and other people seem to run into issues with it.

One thing that might be useful here is a list of issues people have encountered with templates. Maybe we should create a wiki page on GitHub and add to it every time an issue comes up. Then we can have a big-picture view of them and a sense of how many are things people need to learn about Arc, and how many are bugs to be fixed.

I'm trying to follow, but I think you and I must have different understandings of the expression problem. That article lists several known solutions to the expression problem. The solution Anarki uses is `defextend`.

What do you mean by "sharing too much"?

I don't think that aspect of the expression problem is solvable in a language design. Instead, it's an ongoing conversation in the community. Sometimes the intent of one feature and the intent of another feature interact, leading people to do a nonzero amount of work to figure out the intent of the two features put together. That work is an essential part of what the community is trying to accomplish together, so it's a cost that can't be eliminated. The intent has to be reflected in the code somewhere, so there will be a nonzero amount of code that serves feature-coordinating purposes.

I haven't read any more than a few papers on it, and maybe only one of those in depth (which I'll mention below). Mostly I'm going by forum threads, wiki articles, and the design choices certain languages make (like Inform's multimethods and Haskell's type classes).

As far as I understand the history, Philip Wadler's work basically defined the strict parameters of the expression problem and explored solutions for it. Separate compilation and the avoidance of dynamic casts were big deals for Wadler for some reason.

That work was focused on Java, where it's easy to define new classes that implement existing interfaces but impossible to implement new interfaces on existing classes.

The solution I'm most familiar with for Java-style languages is the use of object algebras, as described in Oliveira and Cook's "Extensibility for the Masses: Practical Extensibility with Object Algebras" (https://www.cs.utexas.edu/~wcook/Drafts/2012/ecoop2012.pdf). In this approach, when you extend the system with a new type, you define a new interface with a generic type parameter and a factory method for building that type, and you have that interface inherit all the existing factory methods. So you don't have to solve the unsolvable task of implementing a new interface for an existing class, because you're representing your types as type parameters and methods, not simply as classes.

So I think the main subject of research was how best to represent an extensible program's types and functions in a language like Java where the most obvious choices weren't expressive enough. I think it's more of a "how do we allow extensions to be made at all" problem than a "how do we make all the extensions maintainable" problem.

Cheating how?

It's totally fine to move something into arc.arc if you want to do that. It's always felt like a non-existent distinction in my mind whether something is under arc.arc or libs/. Is Anarki all language or all standard library? Depends on how you look at it. Why does it matter?

> But that's just me... one thing I've learned being here is that I seem to flow against the culture more than with it, so I can just agree to disagree.

Arc3.2:

    arc> (withs nil 3)
    3

    arc> (withs nil 3)
    Can't take car of nil
      context...:
       /usr/share/racket/collects/racket/private/kw.rkt:594:14:  withs
       /home/zck/programs/arc/anarki/ac.rkt:647:0: ac-call
       /home/zck/programs/arc/anarki/ac.rkt:1398:4
       eval-one-top12

You can explicitly pass no arguments. In Anarki:

    arc> (withs () 3)
    3

Edit: I feared that rolling back may also be non-trivial, but there were only some minor conflicts. Do a `git pull`! All tests are passing, and the bugs here should be fixed now.

I tried to make some changes to () instead of nil, and I was not a big fan of how it looked. I found it very unusual that unless quoted, parentheses mean function application. Letting () be the way to write the empty list (and I believe it worked differently quoted from unquoted, but I'm not sure offhand) completely breaks my mental model of how Lisps are parsed.

It's failing because `(type (inst 'foo))` is different in Anarki than Arc. It's a simple change to make it work; I just want to do two things before I stop looking at it:

I somehow forgot that unit-test.arc has its own tests. Could you post the instructions for running the tests in the Readme? That would also have the salubrious side effect of showing people a way to run a bunch of existing tests.

If we do, then I don't know if it matters if we run unit-test.arc tests -- it's just one Arc library, presumably of several.

It's a little different than other libraries because it's what we use for Anarki unit tests.

I will admit to not being super sure what the real differences between nil and '() are. Presumably it's more than "what is the human-readable representation of the value that terminates a list/is the false value". But I'm not sure what. Also, is there a difference between the quoted and unquoted version? It feels odd to write () in a repl unquoted -- usually, I expect parens to mean a function or macro call.

Prior discussions I've found:

A <p> tag does have a body. HTML like this:

  <body>
    <p>First paragraph
    <p>Second paragraph
  </body>

  <body>
    <p>First paragraph
    </p><p>Second paragraph
  </p></body>

  <body>
    <p />First paragraph
    <p />Second paragraph
  </body>

It looks like the HTML specification defines this as a "non-void-html-element-start-tag-with-trailing-solidus parse error." The spec says that in this case, "The parser behaves as if the U+002F (/) is not present," but also that "[browsers] may abort the parser at the first parse error that they encounter for which they do not wish to apply the rules described in this specification."

I don't know of any browsers that abort the parsing altogether, so it's still reliable to write the HTML that way.

However, the similarity to XML is actively misleading in this case. When you process that document as HTML, you still get structure like this:

  <body>
    <p>First paragraph
    </p><p>Second paragraph
  </p></body>

  <body>
    <p></p>First paragraph
    <p></p>Second paragraph
  </body>

  <body>
    <p>First paragraph</p>
    <p>Second paragraph</p>
  </body>

I think modern HTML does have a reliable common subset with XML. Modern HTML treats <br></br> and <p /> as parse errors, but it treats <br /> and <p></p> as valid. To write HTML/XML polyglot content, you just need to pay attention to whether you're dealing with a void element like "br" or a non-void element like "p".

Incidentally, why use an HTML/XML polyglot at all? There are at least a few situations where it can make sense:

- You're serving it as HTML, but (at least someday) you might want to use an XML-processing tool on it or serve it as XHTML.

- You're trying to serve it as XHTML, but you're worried you'll mess up your server configuration and serve it as HTML by mistake.

- You're confident you can serve it as XHTML today, but you have a backup plan to serve it as HTML if needed. In particular, you're afraid someday your XHTML will be invalid due to a bug in your code, a bug in a browser, an intentional spec violation in a browser (e.g. for security or user privacy), or a backwards-incompatible change in the spec. The XHTML spec dictates that an invalid page won't be displayed at all, so if you end up with invalid XHTML for any of those reasons, your site will be rather unusable until you can implement a fix. If that happens at a time you're not ready to drop everything and look at the bug in depth, then you can make a pretty quick switch to serving it as HTML, and most of the page will display again.

Because of the brittle handling of errors, XHTML still hasn't really gotten off the ground. So it seems like the primary value of the HTML/XML polyglot is to serve a document as HTML but use XML-processing tools on it behind the scenes.

---

A side note...

The closing </p> can be omitted if the next tag is one of 25 different tags (https://developer.mozilla.org/en-US/docs/Web/HTML/Element/p, see "tag omission").

1. Write my entries in an org file that contains all the entries.

2. Narrow to the subtree (C-x n s), and export to an HTML buffer (C-x C-e h H)

3. Manually copy the relevant part of the overly-large HTML file into a new file (blog-entry-name.html). This is only the content of a page; it does not include any headers, footers, navbar stuff, or the html wrapper around the body.

4. Insert by hand a serialized arc template, containing three keys: a url slug, the title of the page, and the publication date.

5. Update the frontpage of my site to link to the new page. This file is similarly formatted: an arc template, then html content.

6. Update a file that indicates which pages should go in the sidebar of my site.

7. Run an arc command to generate the entire static site.

8. Check it out locally, then rsync the content to my nearlyfreespeech.net server.

Obvious places for improvement are 3, 5, and 6.

https://gist.github.com/zck/7d563d3ee1b7baf5bf67558aece6c861

If I may kick off a tangent, this is the part of "Worse is better" that tends to be forgotten/deemphasized in Pitman's formulation[1]. C and Unix succeeded because they focused on keeping the implementation simple and accessible for many years. (They eventually forgot that lesson, of course, and have been coasting on the initial momentum for a very long time.)

In fact, reading the story about the "PC loser-ing problem", I realized that I was so conditioned by the Unix solution that I had never even _considered_ the former as a possibility. I do sometimes wonder how many amazingly good ideas we've lost, that would now actually be much simpler than the stack we have, but we're just used to it.

I think the concept could be better generalized by rephrasing it as "cheaper is better" though. Technically it's not "worse", it just has a different set of values. Obviously, users value it more, or they wouldn't adopt it. I see it as closely related to ideas like "compatibility is key", "customer is king", and "money is power", each of which builds on the following.

Customers adopt products that have the best cost-benefit ratio. It doesn't matter if the fancy "good" solution is 10% better (from 90% to 100%) if it also costs 2x as much. Maintenance of the ideal solution may actually be cheaper, but it's really hard to estimate maintenance in advance, especially in design fields like software development. Once the "cheap" solution is adopted, future adoption and upgrades are even cheaper compared to switching to the "good" solution, because the user is already invested, and has built a network of integrations that would be very hard to replicate.

The network effect and basic epidemiology probably provide good explanations for the rapid victory of "cheap" solutions—they spread faster because they are easier to "get", and that amplifies the infection rate to new nodes. Anyone can understand why to adopt something cheap. It takes a lot of effort to learn and understand the technical advantages of a superior system. Given the work involved in properly evaluating competing options to discover technically superior solutions, I think it's safe to assume that the percentage of potential customers that just pick the cheapest one that works, or that is already adopted by the largest number of other users, will always be higher than those who actually compare all the options to pick a better solution.

So "worse" solutions actually are "better", because they're cheaper to adopt. This is especially visible when you look at history and see how many times the systems focusing on backwards compatibility won out over those that merely tried to be "new." Compatibility reduces the cost of adoption. It's that simple.

Does that mean that we're doomed to a "race to the bottom"? I don't think so. In fact, I think with some care new solutions can be designed that are sufficiently better/faster/cheaper that they do disrupt the existing ecosystem. It happens all the time. We see Facebook beating Myspace, all the various chat programs killing XMPP, Slack starting to eat IRC, etc. Most of those did it by making adoption easier for new users. The secret is that a new system doesn't have to replace the existing system, just be easy to adopt. Lots of people use multiple chat programs at the same time. The Lean Startup book[0] was written by an entrepreneur working on a chat system, who initially thought that to make adoption easy he had to integrate with existing systems. What they learned was that people didn't mind adding it to their list of chat systems, and actually liked the ability to meet new networks of friends.

I've been very intrigued recently by a lot of early internet protocols, like IRC, SMTP, NNTP, etc. which are very clean and simple. So easy to use that you can literally connect to an SMTP server via telnet and send an email by hand with just a few simple text commands. I've seen people mention gopher a few times recently (the core doesn't change very fast, but people like to implement custom clients), and even HTTP is pretty simple. I think there's a lot to be said for simple, text-based protocols, because they're easy to understand and implement something that connects to them. I almost think a good test for how complicated an interface is, is how easy it would be to implement in arc, which has very little library support for most of these things. It turns out to be quite easy to build an IRC bot with arc[1].

It is interesting to me that arc may not be very widely adopted, but it is probably one of the few programming languages that has almost as many implementations as it has community members. If we made it just a little bit easier to pick up and start using (particularly in production), the community would probably grow a little faster.

I think there's a lot of opportunity now and in the near future for reintroducing simple foundations, perhaps slightly extended, but mostly made more accessible for new users. Our technology stack has gotten so tall and complicated in the name of shortcuts and simplicity, that a lot of efficiency can be gained by cutting out a few layers. Once people start targeting certain abstraction boundaries, like WASM + WASI, it should be pretty easy to replace everything under that boundary with a much simpler system. A lot of the disadvantages of "good" systems, like microkernels vs monolithic ones, are now so completely outweighed by the rest of the environment that it should be pretty straightforward to build an OS with much better security much closer to the metal than what we have now with 2+ layers of VM sandboxing.

_____

[0] http://theleanstartup.com/

But you should elaborate on your last 2 paragraphs. I'm not sure I buy either that Arc adoption can pick up or that the mainstream tech stack will ever cut out layers.

My synthesis of "Worse is better" for myself (with Mu[1] and SubX[2]):

a) I don't think of evolution as "bad". Building something incompatible is indeed maladaptive. I'm clear-eyed about that.

b) Mu doesn't try to come up with the perfect architecture that doesn't need to evolve. Instead it tries to identify and eliminate every source of friction for future rewrites.

c) My goal isn't to go mainstream. I'd be happy to just have some minor Arc-level adoption. I think it's better to have a small number of people who actually understand the goal (an implementation that's friendly to outsiders) than to have a lot of adoption that causes Mu to forget its roots. My real goal is to build something that outlasts the mainstream stack (the way mammals outlasted the dinosaurs). That doesn't feel as difficult. It's clear the mainstream has a lot of baggage bogging it down. It'll eventually run out of steam. But probably not in my lifetime.

Anyways, I hope in a year or so to give Mu an Arc-like high-level language. It won't improve Arc's adoption, but hopefully it will help promulgate the spirit of this forum: to keep the implementation transparent, and to be friendly to newcomers without burning ourselves out.

[1] https://github.com/akkartik/mu/blob/master/Readme.md

> 2. Browse and install packages.

I install packages (although I think the only ones I use are ones I've created) by downloading the file, then calling `(load "/path/to/file")`.

> 3. Write code with syntax highlighting.

I use Emacs. I use the same syntax highlighting as Common Lisp with this line in my Emacs config: https://bitbucket.org/zck/.emacs.d/src/de1b94f07f5d89fb88e1b....