Summary
I propose the term "superlanguages" to describe languages with extensible syntax, designed for the implementation of domain specific embedded languages and to support language oriented programming, because they effectively provide a superset of the syntax and semantics of other languages.
Advertisement
The following is a revised version of a recent blog entry on my home page that I thought might interest Artima.com readers.
A lot of the work on LOP has been focused on language toolkits (e.g. libraries, editors, workbenches). A closely related idea is Charles Simonyi's concept of Intentional Software (http://www.intentsoft.com/technology/IS_OOPSLA_2006_paper.pdf). To be perfectly honest I am skeptical whether intentional programming is truly distinct from language oriented programming. In the software engineering community a very similar approach called "Language Driven Development" has been gaining traction as described in the online book Applied Metamodelling : A Foundation for Language Driven Development, there is some recent discussion of the book at Lambda-the-Ultimate.org.
The idea of language oriented programming frequently resurfaces in the collective programmer conciousness. For example a recent article by Markus Völter, Architecture as Language: A story, posted on InfoQ, proposes the language oriented programming approach to model-driven architecture without naming it or referring to any tools. This spontaneous recreation of an idea for me is an indicator that it has real substance.
I am not a big fan of language workbenches or the general principle of relying on tool support for my development processes. I was an old-school Atari 400 hacker and I learned to program using nothing but an onion-skin keyboard, and the simplest of text editors. I feel that if you can't do it in text, then forget about it (hence my recent energies devoted to creating a textual mapping of xUML). This is why languages that can be dynamically extended to include the syntax and semantics of radically different programming languages, such as XMF and Kathadin, are so interesting to me. You can work with them from the simplest of command lines.
I have proposed the term "superlanguages" to describe these kinds of languages because they effectively provide a superset of the syntax and semantics of other languages. This term appealed to the creators of XMF who are using it to describe XMF. I might have preferred to use the term metalanguage, because these are languages for defining languages, but this term is already overloaded. ML as in the ML family of languages (e.g. ML, SML, CAML, OCAML) stands for metalanguage. In the language theory community a metalanguage is any language used to define the semantics of another language, but usually not the syntax. An interesting area of research might be the usage of superlanguages as formal descriptions of both the syntax and semantics of a programming language, but I digress.
The idea of a superlanguage is different from languages with a simple metaprogramming protocol such as Common Lisp, Ruby, and Python. For example you can't write the following C-style DSP within Lisp with seriously hacking the reader macros (essentially requiring you to write your own C parser):
int fib(int x) {
if (x <= 1)
return 1;
else
return fib(x - 1) + fib(x - 2);
}
Implementing a C like language (and then jumping out of it in the middle!) was part of a series of challenges I gave the XMF language creators (including defining a Lisp like languages and a PHP-like language). This was for me to see if they are really doing something interesting (as opposed to reinventing Lisp), and it turned out that they were. The XMF team seem to relish new challenges fro defining domain-specific embedded languages. You can see an example of XMF moving in and out of a C-style embedded langauge at http://www.ceteva.com/XMF/Snippets/C/C.html. What perhaps excited me most about XMF, compared to Katahdin which apepars to be at least temporarily on hiatus, is that it has been used in commercial development. It only recently went open-source, and helps demonstrate that there are practical advantages to language oriented programming in general.
> An interesting area of research might be the usage of > superlanguages as formal descriptions of both the syntax > and semantics of a programming language, but I digress.
They do something similar (at least with the semantics), in "Concepts, Techniques and Models of Computer Programming" (http://www.amazon.com/dp/0262220695), where they use a "kernel language", where various abstraction modules are added, to implement/describe the computation model (or semantics, if you like) of various existing languages.
This approach - decoupling the abstractions from the language implementations - enables you to unify the various languages by describing their abstractions in a coherent framework.
> Thanks for an article with a lot of interesting links. > I'll be sure to check them out, as this is something that > interests me greatly.
My pleasure, I'm glad you liked it.
> As I've said in earlier similar discussions, I think this > - languages where you are able to create new languages - > may be the next big thing. > > The XL language (http://xlr.sourceforge.net/) is also > about this, and this is called "concept programming" there.
Great link, thanks! I followed a link on that site to find an old DDJ article about Moka which appears to be a Java preprocessor for language oriented programming. It is quite interesting.
It seems the term "concept" and "concept oriented programming" in one that has multiple meanings attached to it over the past few years:
> > The XL language (http://xlr.sourceforge.net/) is also > > about this, and this is called "concept programming" > there. > > Great link, thanks! I followed a link on that site to find > an old DDJ article about > [a href="http://www.ddj.com/java/184404696"]Moka[/a] which > appears to be a Java preprocessor for language oriented > programming. It is quite interesting.
Right. There are also mentionings of Mozart (http://mozart-dev.sf.net/) (confusingly, there's another project with the same name (http://www.mozart-oz.org/), around), and according to the author, XL is pretty much the successor of Mozart, no longer using Mozart as an intermediate language, having a much simpler internal structure of the programs (a programmer-accessible AST) than Mozart, and is actually written in XL, itself (bootstrapped).
I've not had a chance to look that much into XL, myself, but I intend to (there's also a mailing list there, where one may discuss the language. Unfortunately, at the moment, the list is spammy).
Yes. Just like "Language-Oriented Programming", "Intentional Programming", "Concept Programming", it seems that not only are there many names for this - there are also several meanings for some of the same names... :)
When I heard about XL and "concept programming", it was in particular what is described in the last link above (the Stroustrup/Dos Reis article), I was thinking of, which I found a little confusing. And that, uhm, concept, also has several names, such as "constrained generics". That's one of the most exciting things I think is in the coming C++0x, but it's rather different from LOP, etc.
> I also think this is the next big thing. I have high > hopes for IntentionalSoft's work....now if they just > release a demo.
Indeed. I'm also following that, as one of the more promising developments in this area, ever since I read about IP in the book "Generative Programming".
> I also think this is the next big thing. I have high > hopes for IntentionalSoft's work....now if they just > release a demo.
They had already a full featured demo when Charles Simonyi held a keynote on ECOOP 2001. So stay tuned, superlanguages are the next-big-thing for a decade and I suspect they will be the next-big-thing in the next decade as well.
> > I also think this is the next big thing. I have high > > hopes for IntentionalSoft's work....now if they just > > release a demo. > > They had already a full featured demo when Charles Simonyi > held a keynote on ECOOP 2001.
I'm talking about something out in the wild.
> So stay tuned, > superlanguages are the next-big-thing for a decade and I > suspect they will be the next-big-thing in the next decade > as well.
I suspect you're wrong. I think the difference is tooling.
> I suspect you're wrong. I think the difference is tooling.
In a sense I feel guilty of being immersed in the "superlanguage" or LOP business myself by writing EasyExtend which is a framework and not a language in itself. In my experience system integration is a real challenge, not throwing together two or more syntactical conventions. I needed several releases to get even console support and import conventions approximately correct. From this point of view I wasn't much impressed about Katahdin merging Python and Fortran into something that is poor from both perspectives.
Skinnable syntax is a feature language design novices seem to dream of and it is the first one that gets dropped because it causes more hassles than being useful. Part of it is tooling. Making languages dependent on development tools is an idea developers seem to scare more than they feel the benefits of defining program structures using cartoons. It is not a failed idea that can be proven to be incorrect though. So it stays alive as the hottest property of the Next-Big-Language.
When I first started learning Common Lisp is when I learned of DSLs. It seems that there are a great deal of draw backs in defining DSLs. The major con is that the writers of the DSL are seriously the only few who know the language. In Common Lisp most DSLs are a collection of functions and macros which are still clearly Common Lisp and still are difficult on the eyes. A Super Language enables one to easily write code well outside the realm of the implementation language where neither the semantics or syntax is familiar. We have ANSI, OSI and other such standards for a reason correct? The ideas presented in this article can obviously apply to defining Domain Specific Data representations (XML comes to mind) but not language itself. What if natural languages had such a concept. There would be a different language for every topic we wished to speak about and then even multiple languages describing the same topic.
Language weaving: Instruct me in English... Romance me in French!
Now that's no to say that Super Languages are a bad Idea. There is a lot of excitement going on about the idea of language weaving and interoperability. If I defined Common-Lisp, C, Python, and Ruby all in one language to be transformed into that language before compilation we could achieve some serious language cross interoperability. Recently, CLPython comes to mind (no foreign call overhead there).
Existing languages, while not DSL, are better for certain tasks than others (as well as have numerous libraries for these tasks). You could have mixed language algorithms and file taking advantage of each languages strength and weaknesses, best example of this is in-line assembly how about some A.I. snippests done up Prolog and some control code in Lisp with some GUI stuff in C.
> What if natural languages had such a > concept. There would be a different language for every > topic we wished to speak about and then even multiple > languages describing the same topic.
Inventing notational conventions and jargon is familiar to any scientist. Each discipline invents new syntax, diagram styles etc. In mathematics each sub discipline uses own notations. This isn't considered harmful because mathematicians use a somewhat redundant presentation style and introduce basic concepts over and over again in an introductory chapter or an appendix of their books.
Part of this culture is also alive in computing where XML documents contain references to DTDs.