The Artima Developer Community
Sponsored Link

Weblogs Forum
Superlanguages: Syntactic and Semantic Supersets of other Languages

13 replies on 1 page. Most recent reply: Mar 21, 2008 3:20 AM by Kay Schluehr

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 13 replies on 1 page
Christopher Diggins

Posts: 1215
Nickname: cdiggins
Registered: Feb, 2004

Superlanguages: Syntactic and Semantic Supersets of other Languages (View in Weblogs)
Posted: Mar 12, 2008 1:56 PM
Reply to this message Reply
Summary
I propose the term "superlanguages" to describe languages with extensible syntax, designed for the implementation of domain specific embedded languages and to support language oriented programming, because they effectively provide a superset of the syntax and semantics of other languages.
Advertisement
The following is a revised version of a recent blog entry on my home page that I thought might interest Artima.com readers.


Language oriented programming (LOP), is an idea that has been around for at least 40 years within the Lisp/Scheme community. I believe it was first given the name in 1994 by Martin Ward. The earliest article with the term "Language Oriented Programming" I can find is this article by the same name. A more recent version of Ward's article that was updated in 2003 can be found at http://www.cse.dmu.ac.uk/~mward/martin/papers/middle-out-t.pdf. LOP has been getting some attention lately due in large part to Martin Fowler's essay on Language Workbenches: A Killer App for Domain Specific Languages.

A lot of the work on LOP has been focused on language toolkits (e.g. libraries, editors, workbenches). A closely related idea is Charles Simonyi's concept of Intentional Software (http://www.intentsoft.com/technology/IS_OOPSLA_2006_paper.pdf). To be perfectly honest I am skeptical whether intentional programming is truly distinct from language oriented programming. In the software engineering community a very similar approach called "Language Driven Development" has been gaining traction as described in the online book Applied Metamodelling : A Foundation for Language Driven Development, there is some recent discussion of the book at Lambda-the-Ultimate.org.

The idea of language oriented programming frequently resurfaces in the collective programmer conciousness. For example a recent article by Markus Völter, Architecture as Language: A story, posted on InfoQ, proposes the language oriented programming approach to model-driven architecture without naming it or referring to any tools. This spontaneous recreation of an idea for me is an indicator that it has real substance.

I am not a big fan of language workbenches or the general principle of relying on tool support for my development processes. I was an old-school Atari 400 hacker and I learned to program using nothing but an onion-skin keyboard, and the simplest of text editors. I feel that if you can't do it in text, then forget about it (hence my recent energies devoted to creating a textual mapping of xUML). This is why languages that can be dynamically extended to include the syntax and semantics of radically different programming languages, such as XMF and Kathadin, are so interesting to me. You can work with them from the simplest of command lines.

I have proposed the term "superlanguages" to describe these kinds of languages because they effectively provide a superset of the syntax and semantics of other languages. This term appealed to the creators of XMF who are using it to describe XMF. I might have preferred to use the term metalanguage, because these are languages for defining languages, but this term is already overloaded. ML as in the ML family of languages (e.g. ML, SML, CAML, OCAML) stands for metalanguage. In the language theory community a metalanguage is any language used to define the semantics of another language, but usually not the syntax. An interesting area of research might be the usage of superlanguages as formal descriptions of both the syntax and semantics of a programming language, but I digress.

The idea of a superlanguage is different from languages with a simple metaprogramming protocol such as Common Lisp, Ruby, and Python. For example you can't write the following C-style DSP within Lisp with seriously hacking the reader macros (essentially requiring you to write your own C parser):

  int fib(int x) {
     if (x <= 1) 
        return 1; 
     else 
        return fib(x - 1) + fib(x - 2);
  }

Implementing a C like language (and then jumping out of it in the middle!) was part of a series of challenges I gave the XMF language creators (including defining a Lisp like languages and a PHP-like language). This was for me to see if they are really doing something interesting (as opposed to reinventing Lisp), and it turned out that they were. The XMF team seem to relish new challenges fro defining domain-specific embedded languages. You can see an example of XMF moving in and out of a C-style embedded langauge at http://www.ceteva.com/XMF/Snippets/C/C.html. What perhaps excited me most about XMF, compared to Katahdin which apepars to be at least temporarily on hiatus, is that it has been used in commercial development. It only recently went open-source, and helps demonstrate that there are practical advantages to language oriented programming in general.


Terje Slettebø

Posts: 205
Nickname: tslettebo
Registered: Jun, 2004

Re: Superlanguages: Syntactic and Semantic Supersets of other Languages Posted: Mar 13, 2008 8:32 AM
Reply to this message Reply
Thanks for an article with a lot of interesting links. I'll be sure to check them out, as this is something that interests me greatly.

As I've said in earlier similar discussions, I think this - languages where you are able to create new languages - may be the next big thing.

The XL language (http://xlr.sourceforge.net/) is also about this, and this is called "concept programming" there.

Terje Slettebø

Posts: 205
Nickname: tslettebo
Registered: Jun, 2004

Re: Superlanguages: Syntactic and Semantic Supersets of other Languages Posted: Mar 13, 2008 9:44 AM
Reply to this message Reply
Just to add some more:

> An interesting area of research might be the usage of
> superlanguages as formal descriptions of both the syntax
> and semantics of a programming language, but I digress.

They do something similar (at least with the semantics), in "Concepts, Techniques and Models of Computer Programming" (http://www.amazon.com/dp/0262220695), where they use a "kernel language", where various abstraction modules are added, to implement/describe the computation model (or semantics, if you like) of various existing languages.

This approach - decoupling the abstractions from the language implementations - enables you to unify the various languages by describing their abstractions in a coherent framework.

Christopher Diggins

Posts: 1215
Nickname: cdiggins
Registered: Feb, 2004

Re: Superlanguages: Syntactic and Semantic Supersets of other Languages Posted: Mar 13, 2008 1:16 PM
Reply to this message Reply
Hi Terje,

> Thanks for an article with a lot of interesting links.
> I'll be sure to check them out, as this is something that
> interests me greatly.

My pleasure, I'm glad you liked it.

> As I've said in earlier similar discussions, I think this
> - languages where you are able to create new languages -
> may be the next big thing.
>
> The XL language (http://xlr.sourceforge.net/) is also
> about this, and this is called "concept programming" there.

Great link, thanks! I followed a link on that site to find an old DDJ article about Moka which appears to be a Java preprocessor for language oriented programming. It is quite interesting.

It seems the term "concept" and "concept oriented programming" in one that has multiple meanings attached to it over the past few years:

* http://www.ddj.com/184410968 (1999)
* http://conceptoriented.com/papers/CopInformalIntroduction.html (2007)
* http://www.research.att.com/~bs/popl06.pdf (2006)

Dick Ford

Posts: 149
Nickname: roybatty
Registered: Sep, 2003

Re: Superlanguages: Syntactic and Semantic Supersets of other Languages Posted: Mar 13, 2008 1:32 PM
Reply to this message Reply
I also think this is the next big thing. I have high hopes for IntentionalSoft's work....now if they just release a demo.

Terje Slettebø

Posts: 205
Nickname: tslettebo
Registered: Jun, 2004

Re: Superlanguages: Syntactic and Semantic Supersets of other Languages Posted: Mar 13, 2008 10:22 PM
Reply to this message Reply
> > The XL language (http://xlr.sourceforge.net/) is also
> > about this, and this is called "concept programming"
> there.
>
> Great link, thanks! I followed a link on that site to find
> an old DDJ article about
> [a href="http://www.ddj.com/java/184404696"]Moka[/a] which
> appears to be a Java preprocessor for language oriented
> programming. It is quite interesting.

Right. There are also mentionings of Mozart (http://mozart-dev.sf.net/) (confusingly, there's another project with the same name (http://www.mozart-oz.org/), around), and according to the author, XL is pretty much the successor of Mozart, no longer using Mozart as an intermediate language, having a much simpler internal structure of the programs (a programmer-accessible AST) than Mozart, and is actually written in XL, itself (bootstrapped).

I've not had a chance to look that much into XL, myself, but I intend to (there's also a mailing list there, where one may discuss the language. Unfortunately, at the moment, the list is spammy).

> It seems the term "concept" and "concept oriented
> programming" in one that has multiple meanings attached to
> it over the past few years:
>
> * http://www.ddj.com/184410968 (1999)
> *
> http://conceptoriented.com/papers/CopInformalIntroduction.h
> tml (2007)
> * http://www.research.att.com/~bs/popl06.pdf (2006)

Yes. Just like "Language-Oriented Programming", "Intentional Programming", "Concept Programming", it seems that not only are there many names for this - there are also several meanings for some of the same names... :)

When I heard about XL and "concept programming", it was in particular what is described in the last link above (the Stroustrup/Dos Reis article), I was thinking of, which I found a little confusing. And that, uhm, concept, also has several names, such as "constrained generics". That's one of the most exciting things I think is in the coming C++0x, but it's rather different from LOP, etc.

Terje Slettebø

Posts: 205
Nickname: tslettebo
Registered: Jun, 2004

Re: Superlanguages: Syntactic and Semantic Supersets of other Languages Posted: Mar 13, 2008 11:44 PM
Reply to this message Reply
> I also think this is the next big thing. I have high
> hopes for IntentionalSoft's work....now if they just
> release a demo.

Indeed. I'm also following that, as one of the more promising developments in this area, ever since I read about IP in the book "Generative Programming".

A while ago, I came across a video, where they demonstrated the software (when it was a research project at Microsoft), and it blew me away... Unfortunately, the original video (http://www.cse.unsw.edu.au/~cs3141/ip.asf) is no longer available, but I found excerpts from it at YouTube (http://www.youtube.com/watch?v=tSnnfUj1XCQ and http://www.youtube.com/watch?v=ZZDwB4-DPXE).

Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Superlanguages: Syntactic and Semantic Supersets of other Languages Posted: Mar 15, 2008 4:52 AM
Reply to this message Reply
> I also think this is the next big thing. I have high
> hopes for IntentionalSoft's work....now if they just
> release a demo.

They had already a full featured demo when Charles Simonyi held a keynote on ECOOP 2001. So stay tuned, superlanguages are the next-big-thing for a decade and I suspect they will be the next-big-thing in the next decade as well.

Dick Ford

Posts: 149
Nickname: roybatty
Registered: Sep, 2003

Re: Superlanguages: Syntactic and Semantic Supersets of other Languages Posted: Mar 15, 2008 6:28 PM
Reply to this message Reply
> > I also think this is the next big thing. I have high
> > hopes for IntentionalSoft's work....now if they just
> > release a demo.
>
> They had already a full featured demo when Charles Simonyi
> held a keynote on ECOOP 2001.

I'm talking about something out in the wild.

> So stay tuned,
> superlanguages are the next-big-thing for a decade and I
> suspect they will be the next-big-thing in the next decade
> as well.

I suspect you're wrong. I think the difference is tooling.

Christopher Diggins

Posts: 1215
Nickname: cdiggins
Registered: Feb, 2004

Re: Superlanguages: Syntactic and Semantic Supersets of other Languages Posted: Mar 17, 2008 11:21 AM
Reply to this message Reply
I have written a follow-up article to this one: What are Superlanguages good for at DobbsCodeTalk.com

It is worth noting that Christophe de Dinechin, the inventor of XL, responded to this blog entry on his own blog.

Raoul Duke

Posts: 127
Nickname: raoulduke
Registered: Apr, 2006

Re: Superlanguages: Syntactic and Semantic Supersets of other Languages Posted: Mar 17, 2008 8:35 PM
Reply to this message Reply
XL sure looks neat. Somewhat amazing that the person making it is also capable of writing good documentation.

Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Superlanguages: Syntactic and Semantic Supersets of other Languages Posted: Mar 18, 2008 2:47 AM
Reply to this message Reply
> I suspect you're wrong. I think the difference is tooling.

In a sense I feel guilty of being immersed in the "superlanguage" or LOP business myself by writing EasyExtend which is a framework and not a language in itself. In my experience system integration is a real challenge, not throwing together two or more syntactical conventions. I needed several releases to get even console support and import conventions approximately correct. From this point of view I wasn't much impressed about Katahdin merging Python and Fortran into something that is poor from both perspectives.

Skinnable syntax is a feature language design novices seem to dream of and it is the first one that gets dropped because it causes more hassles than being useful. Part of it is tooling. Making languages dependent on development tools is an idea developers seem to scare more than they feel the benefits of defining program structures using cartoons. It is not a failed idea that can be proven to be incorrect though. So it stays alive as the hottest property of the Next-Big-Language.

eric jones

Posts: 1
Nickname: xplo43
Registered: Mar, 2008

Re: Superlanguages: Syntactic and Semantic Supersets of other Languages Posted: Mar 20, 2008 5:03 PM
Reply to this message Reply
When I first started learning Common Lisp is when I learned of DSLs. It seems that there are a great deal of draw backs in defining DSLs. The major con is that the writers of the DSL are seriously the only few who know the language. In Common Lisp most DSLs are a collection of functions and macros which are still clearly Common Lisp and still are difficult on the eyes. A Super Language enables one to easily write code well outside the realm of the implementation language where neither the semantics or syntax is familiar. We have ANSI, OSI and other such standards for a reason correct? The ideas presented in this article can obviously apply to defining Domain Specific Data representations (XML comes to mind) but not language itself. What if natural languages had such a concept. There would be a different language for every topic we wished to speak about and then even multiple languages describing the same topic.

Language weaving: Instruct me in English... Romance me in French!

Now that's no to say that Super Languages are a bad Idea. There is a lot of excitement going on about the idea of language weaving and interoperability. If I defined Common-Lisp, C, Python, and Ruby all in one language to be transformed into that language before compilation we could achieve some serious language cross interoperability. Recently, CLPython comes to mind (no foreign call overhead there).

Existing languages, while not DSL, are better for certain tasks than others (as well as have numerous libraries for these tasks). You could have mixed language algorithms and file taking advantage of each languages strength and weaknesses, best example of this is in-line assembly how about some A.I. snippests done up Prolog and some control code in Lisp with some GUI stuff in C.

I'll stop now.

Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Superlanguages: Syntactic and Semantic Supersets of other Languages Posted: Mar 21, 2008 3:20 AM
Reply to this message Reply
> What if natural languages had such a
> concept. There would be a different language for every
> topic we wished to speak about and then even multiple
> languages describing the same topic.

Inventing notational conventions and jargon is familiar to any scientist. Each discipline invents new syntax, diagram styles etc. In mathematics each sub discipline uses own notations. This isn't considered harmful because mathematicians use a somewhat redundant presentation style and introduce basic concepts over and over again in an introductory chapter or an appendix of their books.

Part of this culture is also alive in computing where XML documents contain references to DTDs.

Flat View: This topic has 13 replies on 1 page
Topic: Superlanguages: Syntactic and Semantic Supersets of other Languages Previous Topic   Next Topic Topic: You’re Just My Type


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us