After a short introduction about the relevance of macros as tools to design abstractions, including full programming languages, I show some common patterns of Scheme
macrology: recursive macros, accumulators, and the usage of literals to incorporate helpers in macros.
Back to macros
Macros are the reason why I first became interested in Scheme, five or
six years ago. At the time - as at any time - there was a bunch of
people trolling in comp.lang.python, arguing for the addition of macros
to the language. Of course most Pythonistas opposed the proposal.
At the time I had no idea of the advantages/disadvantages of
macros and I felt quite ignorant and powerless to argue. I
never liked to feel ignorant, so I decided to learn macros, especially
Scheme macros, because they are the state of the art for what concerns
Nowadays I have some arguments to back up the position against macros.
I have two main objections, one technical (less important) and one
political (more important).
The technical reason is that I do not believe in macros for languages
without S-expressions. There are plenty of examples of macro systems
without S-expressions - for instance Dylan or PLOT in the Lisp
world and Logix and MetaPython in the Python world, but none of
them ever convinced me. Scheme macros are much better because of
the homoiconicity of the language ("homoiconicity" is just a big word for
the code is data concept). [Notice that technically Scheme macros work on syntax
objects and not directly on S-expressions like traditional Lisp
macros, but this is a subtle point I will discuss when talking about
hygiene; I can skip it for the moment being.]
I have already stated in episode 12 my political objection, i.e. my
belief that macros have a high cost in terms of complication of the
language (look how complicated the R6RS module system is!). Moreover,
code based on macros tends to be too clever, difficult to debug, and
sometime idiosyncratic; I do not want to maintain code such kind of code
in a typical enterprise context, with programmers of any kind of competence.
Sometimes I wish that even Python was a simpler language!
There is a difference between simpler and dumber, of course.
I am not implying that every enterprise should
adopt only enterprise-oriented languages; as a matter of fact
various cutting edge enterprises are taking advantage of
non-conventional and/or research-oriented languages, but I see them as
exceptions to the general rule.
My opinion is based on the fact that on my daily work (I use Python
exclusively there) I have never felt the need for macros. For
instance, I had occasion to write both small declarative languages and
small command-oriented languages, but they were so simple that I had
no need for Scheme macros. Actually, judging from my past
experience, I think extremely unlikely that I will ever need something
as sophisticated as Scheme macros in my daily work. The one thing
that I miss in Python which Scheme has is pattern matching, not
Having said that, I do not think that macros are worthless, and actually
I think they are extremely useful and important in another domain,
i.e. in the domain of design and research about programming
languages. Scheme is certainly not the only language where you can experiment
with language design, it is just the best language for this kind of
tasks, at least in my humble opinion.
For instance, a few months ago I have described
an experiment I did with the Python meta object protocol, in
order to change how the object system work, and replace
multiple inheritance with traits. Even if in Python it is possible to
customize the object system, I do not thing the approach is optimal,
because changing the semantics without changing the syntax
does not feel right. In Scheme I could have implemented the same
with a custom syntax and in a somewhat less magical way.
I am interested with this
kind of experiments, even if I will never use them in
production code, and I use Scheme in preference for such purposes.
The major interest of Scheme macros for me lies in the fact that they
enable every programmer to write her own programming language.
I think this is a valuable thing. Anybody who has got opinions
about language design, or about how an object system should should work, or
questions like "what would a language look like if it had feature X?",
can solve his doubts by implementing the feature with macros.
Notice that I recognize that perhaps not everybody should design its
own programming language, and that certainly not everybody should
distribute its own personal language. Nevertheless, I think
everybody can have opinions about language design. Experimenting
with macrology can help to put to test such opinions and to learn
The easiest approach is to start from a Domain Specific
Language (DSL), which does not need to be a fully grown programming
language. For instance, in the Python world
everybody is implementing his own templating language to generate web
pages. In my opinion, this a good thing per se, the problem is that
everybody is distributing his own language so that there is a bit of
Even for what concerns fully grown programming languages we see nowadays
an explosion of new languages, especially for the Java and
the .NET platforms, since it is relatively easy to implement a new
language there. However, it still takes a substantial amount of work.
On the other hand, writing a custom language embedded in Scheme by
means of macros is much easier. I see Scheme as an excellent platform
for implementing languages and experimenting with new ideas.
Sometimes Python is accused of having too many web frameworks. And
it's true, there are a lot. That said, I think writing a framework is
a useful exercise. It doesn’t let you skip over too much without
understanding it. It removes the magic. So even if you go on to use
another existing framework (which I'd probably advise you do), you'
ll be able to understand it better if you've written something like it
on your own.
You can the replace the words "web framework" with "programming
language" and the quote still makes sense. You should read my
Adventures in this spirit: the ambition of the series is to give to
the readers the technical competence to write small Scheme-embedded
languages by means of macros. Even if you are not going to design your
own language, macros will help you to understand how languages work.
Personally I am interested in the technical competence, I do not want
to write a new language. There are already lots of languages out
there, and writing a real language is a lot of grunt work, because it
means writing debugging tools, good error messages, wondering about
portability, interacting with an user community, et cetera et cetera.
The goal of learning macros well enough to implement a programming
language is an ambitious one; it is not something I can attain in one
episode of the Adventures, nor in six. However, one episode is enough
to explain at least one useful technique which is commonly used in
Scheme macrology and which is good to know in order to reach our final
goal, in time.
The technique I will discuss in this episode is writing recursive
macros with accumulators. In Scheme it is common to introduce an
auxiliary variable to store a value which is passed in a loop - we
discussed it in episode 6 when talking about tail call optimization:
the same trick can be used in macros, at expand-time instead that at
In order to give an example I will define a macro cond minus
(cond-) which works like cond, but with less parenthesis.
Here is an example:
The code above should be clear. The auxiliary macro cond-aux
is recursive: it works by collecting the arguments x1,x2,...,xn
in the accumulator (acc...). If the number of arguments is even,
at some point we end up having collected all the arguments in the
accumulator, which is then expanded into a standard conditional; if
the number of arguments is even, at some point we end up having
collected all the arguments except one, and a "Mismatchedpairs"
exception is raised. The user-visible macro cond- just calls
cond-aux by setting the initial value of the accumulator to ().
The entire expansion and error checking is made at compile time.
Here is an example of usage:
> (let ((n 1))
(cond- (= n 1) ; missing a clause
(= n 2) 'two
(= n 3) 'three
1. &who: cond-
2. &message: "Mismatched pairs"
form: (((= n 1) (= n 2)) ('two (= n 3)) ('three else) 'unknown)
I have nothing against auxiliary macros, however sometimes you may
want to keep all the code in a single macro. This is useful if you are
debugging a macro since an auxiliary macro is usually not
exported. The trick is to introduce a literal to defined the helper
macro inside the main macro. Here is how it would work in this example:
Marshall's essay is quite nontrivial, and it is intended for expert
Scheme programmers. On the other hand, it is child play compared to
Petrofsky's essay, which is intended for foolish Scheme wizards ;)