Packrat Parsing
In the 80s I wrote an image processing language with extensible syntax using Earley’s Algorithm. It worked well enough, but it required expert knowledge to understand what was happening when syntax from disparate modules interacted.
I’ve recently learned of Parsing Expression Grammars (PEGs) and Packrat Parsing. I think this could be well suited to an extensible language. This is a linear time algorithm which can easily take unfactored BNF style productions as its language definition. It is a memory hog, a reasonable size module might take 50M to parse, but I think the simplicity of syntax specification will more than out weigh the memory foot print. (That image processing language ran on a Vax11-750 with 8M of RAM shared with 10 people. This is a technique which is only now becoming feasible with larger memory sizes.)
I am currently prototyping a packrat parser for an extensible language. The prototype is in Javascript and runs inside a browser with fields for the grammar and the input string which makes interaction trivial.
It is not on a publicly accessible machine. I may put a snapshot of it in a public place if anyone asks.