The Artima Developer Community
Sponsored Link

Weblogs Forum
The Sum of Ant

11 replies on 1 page. Most recent reply: Sep 8, 2005 8:57 AM by Walter Higgins

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 11 replies on 1 page
Ken Arnold

Posts: 27
Nickname: arnold
Registered: Apr, 2003

The Sum of Ant (View in Weblogs)
Posted: Jul 13, 2003 10:35 PM
Reply to this message Reply
Summary
...Wherein we examine what ant fundamentally lacks, and possible ways to address the important problem of build control...
Advertisement

In a previous blog on today.java.net I said some fairly strong things about ant, and I think I ought to say something more, both good and bad. This seems like the place to do it.

First, let us give ant its due. Ant is designed to be a portable way to replace make. Ant was to be platform independent, so the obvious implementation choice was Java, which is fine, mostly. And the obvious data format for the project description -- saying what needs to be built -- was XML, which is not fine, as we shall see.

Ant performs this task with some power, if little aplomb. One can build ant files that will compile Java, C, and many other languages; perform CVS and FTP tasks; build jar files and EJB deployments, and so forth. Ant can run the same configuration without modification, no matter the platform. This is no mean feat. There is a reason why it has been winning many awards.

But when I say it is "with little aplomb", I mean it. Nobody can accuse XML of being an easy read, of course, but if that were ant's worst problem, well, ant is hardly alone in this sin. XML abuse is rampant, and ant is far, far from the worst offender. Given the limitations of XML syntax, ant does pretty well. To see it really suck, try reading Graydon Hoare's essay on XML abuse. (Ignore the odd section headings and stick to the content.)

The real problem is the one I talked about in my brief java.net blog:

Ant is nothing more than the sum of its parts
By this I mean that ant has not learned the basic power of composition, building things out smaller parts. This was the great insight that our Unix forbearers bequeathed us toolsmiths, and it's pretty sad to see it forgotten. In Unix this is done with pipes -- the output of one program can become the input of another. Along with conventions about program output, this allows you to build up something that is more than the sum of its parts. A utility that finds file in the file system can generate a list of files for any purpose: removal, editing, rebuilding, copying, printing, ...

Ant has many kinds of tasks. These are portable because they are written in Java, and these task implementations can rely upon operating system abstractions in the underlying ant platform.

But they almost never work together. There are some common tools for building up lists of files, but if you want a list of files from any other source, good luck. This is why there are two different tasks (one optional) that calculate list of Java class dependencies, and they are incompatible. And if you want a list of Java class dependencies for a task that one of these two tasks can't handle, good luck.

A common quote in our biz is the observation that a good software tool will do things the author never expected1. I doubt that writers of ant tasks are very often surprised.

In fact, I will assume that the value of composable tools is so obvious, I will not belabor it further. Unless you ask...

My experience is that the primary legibility problem with Makefiles are strings and variables. The string operations are primitive, ugly, and limited. So to do anything at all interesting requires cryptic and hobbled work. After that, the next worst problem is the lack of flow control. Loops are put into the shell commands because make doesn't have loops, or (shudder) people use recursion as a replacement. Ugh.

The primary portability problem is obvious: Different system have different commands, and make just executes commands.

So where would one go to leverage existing tools that already know how to do these things well? Scripting languages, of course. Perl, python/jython, tcl, and so on live, breath, eat, and excrete strings for a living, and they are programming languages with real logic. They are ported to every platform that matters (more than Java runs on).

All they are missing are tools to express dependencies and how to apply them to building things. One of the responses to my post pointed me at scons, which does this in Python. It seems just the right direction, although I haven't had a chance to do more than play with scons. I don't claim it is the right answer, but it sure is moving down the right path.

Jonathan Simon's blog entry about how to use ant tasks from jython points towards some hope that we can reuse all the work these folks put into ant tasks. This has often been non-trivial effort, and it would be good to keep this work viable.

One could pick any language, of course, and choosing a winner here would antagonize folks who would just prove they could do it better in the "right" language. I have my preferred scripting languages, but compared to scripting in XML itself (which is where the ant folks seem to be going), no major possibility is worse.

I intend to poke harder at scons, and if it looks good, dive in fully to find out. You'll be seeing a blog here about scons and the lessons to learn from it.


[1] Google has failed me. Can the death of the internet be far behind? I cannot find an attribution for this quote, despite the obvious set of words and the fact that it's a computer quote that I've heard a billion times. [Note: URL for Graydon Hoare's XML writeup has been updated; 7/1/2005]


Frank Mitchell

Posts: 37
Nickname: fmitchell
Registered: Jan, 2003

Re: The Sum of Ant Posted: Jul 13, 2003 11:17 PM
Reply to this message Reply
At my old job we ended up with Ant files so large we split them into multiple physical files, and used XML Entities to include the subfiles in a main file. A lot of the targets were copy-and-paste because we couldn't think of a good way to factor out the differences. Some targets existed solely to check timestamps and skip other targets. Imagine maintaining all that ... in XML.

A better approach might have been to factor the many targets into wholly independent Ant files. Even better, a lot of platform-specific targets should have been ported to Makefiles, shell scripts, or .BAT files.

Also, take a look at Rake <http://jimweirich.umlcoop.net/software/rake/>, a simple attempt to use Ruby as a build tool. It's pretty basic, but I think Ruby blocks make "Rakefiles" more readable.

Frank Mitchell

Posts: 37
Nickname: fmitchell
Registered: Jan, 2003

Re: The Sum of Ant Posted: Jul 13, 2003 11:30 PM
Reply to this message Reply
URL should be http://jimweirich.umlcoop.net/software/rake/

Joseph Grace

Posts: 6
Nickname: occam
Registered: Mar, 2003

Also see a-a-p syntax. Posted: Jul 15, 2003 3:42 PM
Reply to this message Reply
Another candidate along the same lines is a-a-p by Bram Moolenar of vim (vi-alike editor) fame. It just went 1.0 this month the Friday before O'Reilly OSCon. Bram gave a session on a-a-p there which I was unable to attend. I've requested he post his slides for the talk at

http://conferences.oreillynet.com/pub/w/23/oscon_grid.html

with O'Reilly but no luck yet.

The syntax is much more akin to make (simple colon separated dependencies), and python is great. I was hoping he would do it in java/jython to take advantage of the ant recipes. However, perhaps ant could adopt the a-a-p syntax and provide a compatibility the other way.

Also, it occurs to me that perl6 (if it ever comes to be) could well self-contain make functionality with its new grammar construct. I'm not a perl hack (unfortunately) but the grammar keyword and support suggests to me that perl6 will be great for creating new mini-languages. The grammar regex support is tremendous, and perl6 would probably deal with any make-like syntax handily. Just a thought for the future.

= Joe =

Terence Parr

Posts: 1
Nickname: parrt
Registered: Jul, 2003

Re: The Sum of Ant Posted: Jul 16, 2003 1:15 PM
Reply to this message Reply
Howdy. In line with the thoughts on not using XML for human interfaces, you might enjoy my "Humans should not have to grok XML" article I did back in 2001:

http://www-106.ibm.com/developerworks/xml/library/x-sbxml.html?dwzone=xml

Regards,
Terence

Bruce Eckel

Posts: 875
Nickname: beckel
Registered: Jun, 2003

Re: The Sum of Ant Posted: Sep 21, 2003 4:27 PM
Reply to this message Reply
A brilliant observation. I was always a bit afraid to criticize Ant since I thought perhaps I just didn't fully understand the issues, but now it seems that I do.

Readability of anything in programming is important, and yet is often lost "because we have to blah blah." When moving databases around, XML is great because it's far more readable than proprietary database formats. But to then turn around and say "XML for everything!" repeats the usual silly mistake we all make periodically, of finding the one true answer.

I've spent years with 'make', and have pushed the boundaries and found at least some of the limitations. Make is nice because it's structured and succinct; it seems to keep the amount of verbiage down to a minimum, something I immediately noticed that Ant does not do (Ant is verbose for the sake of supporting XML). But there are places where make quickly breaks down -- sometimes the automated creation of lists, sometimes complex dependencies. This has caused me hours of frustration trying to get make to do things that it very probably just can't do.

I believe problems like this have caused a rather large number of projects in the last couple of years, to build the "ultimate" build tool. Some of those tools seem to have produced interesting ideas, but I think what I want is something that (A) is simple and obvious to use (I don't want to relive the 'make' learning curve) (B) is succinct and structured. I should be able to accrete my build description without tons of redundant verbiage. (C) doesn't limit me because the designer ran out of energy.

I imagine the best solution to this problem is "The simplest thing that could possibly work." 'Make' is a language, and it falls into the Unix trap of everybody making up their own languages. I don't want to have to learn a language every time I pick up a new tool. Perl is valuable because it combines all the various Unix tools into a single unified language, although it unfortunately also combines the cruft from those Unix tools.

So instead of someone inventing yet another new build language for me to learn, a language which will inevitably be limited because the author will not go to the trouble of creating a general purpose language, I think the best solution would be to create some kind of framework on top of my favorite language (Python). Perhaps it would even have the basic syntax of make, but the point would be that the simple things would be taken care of using the fundamental structure of the thing (that is, I don't have to write a bunch of Python just to get my build configured). But if I need to do something more complex, like create and manipulate a list of files, I can easily drop into the full power of Python just by writing a Python expression (perhaps escaped as is done with JSPs in web pages).

So an example implementation of this system might use the fundamental makefile structure (so I don't have to re-learn that) for basic dependencies, but as soon as you want to do anything fancy you just write Python code, rather than trying to use the weird and whacky make doodads. This approach would also have the benefit of allowing easier translation of existing makefiles, and possibly even conversion of makefile-building tools.

Gregg Wonderly

Posts: 317
Nickname: greggwon
Registered: Apr, 2003

Re: The Sum of Ant Posted: Sep 23, 2003 2:22 PM
Reply to this message Reply
I have complained about all kinds of ant things to many groups of people. There are a lot of people that are just standing in line behind the mantra like a bunch of lemmings it would seem.

It is truely frustrating how many times the same lessons, and important lessons at that, go unlearned...

It's much better when everything is a file...

In XML, I've always thought like the following would be more readily useful. I don't love XML, but I do find some good uses for it.


<task name="my name">
<jar output="myjar.jar">
<input task="clientDeps"/>
</jar>

</task>

<task name="clientDeps">
<depend>
<classpath task="classpathTask"/>
<hide>
<input task="hidden.list"/>
</hide>
<top name="my.package.top.class"/>
</depend>
</task>

<task name="classpathTask">
<path>
<file name="jar1.jar"/>
<file name="jar2.jar"/>
<file name="jar3.jar"/>
<dir name="${build.output}"/>
</path>
</task>

<task name="hidden.list"/>
<packages>
<jar file="${JAVA_HOME}/lib/rt.jar"/>
</packages>
<packages>
<jar file="${JINI}/lib/jsk-platform.jar"/>
</packages>
</task>

Dave Ford

Posts: 6
Nickname: dford
Registered: Jun, 2003

Re: The Sum of Ant Posted: Dec 17, 2003 11:39 PM
Reply to this message Reply
I use Java for all of my builds "scripts". It works great!! Since Ant tasks are written in Java, they are easily leveraged. Also, most of my build scripts are reusable between projects. This is easy if you structure all of your projects in a similar way.

Dave Ford

George Dinwiddie

Posts: 412
Nickname: gdinwiddie
Registered: Nov, 2004

Re: The Sum of Ant Posted: Nov 16, 2004 2:44 PM
Reply to this message Reply
> A common quote in our biz is the observation that a good software
> tool will do things the author never expected. I doubt that writers
> of ant tasks are very often surprised.
>
> Google has failed me. Can the death of
> the internet be far behind? I cannot find an attribution for this
> quote, despite the obvious set of words and the fact that it's a
> computer quote that I've heard a billion times.

I don't have the book at hand to verify, but I believe this is from The UNIX Programming Environment by Brian W. Kernighan, Rob Pike. If not, it's from the Bell Labs Journal devoted to the announcement of unix.

Brad Appleton

Posts: 3
Nickname: bradapp
Registered: Mar, 2003

Re: The Sum of Ant Posted: Mar 2, 2005 2:22 AM
Reply to this message Reply
Have you looked at the Groovy scripting language (http://groovy.codehaus.org/) which is a Java-ized, JVM & Java bytecode compatible scripting language? Groovy basically tries to take Ruby (which itself tries to take the best of Perl, Python and Smalltalk) and give it a Java-like syntax.

Take a peek at Andrew Glover's article from the "Practically Groovy" series entitled "Ant Scripting with Groovy: Combine Ant and Groovy for more expressive and controllable builds" at http://www-128.ibm.com/developerworks/library/j-pg12144.html

Also see article on "Build Scripts with Groovy and Ant" at http://www.javaworld.com/javaworld/jw-10-2004/jw-1004-groovy.html

and another article "Scripting a Groovy Ant" at http://www.fawcette.com/javapro/2004_05/online/kjones_05_19_04/

David Benjamin

Posts: 1
Nickname: dbjag
Registered: Jul, 2005

Re: The Sum of Ant Posted: Jul 21, 2005 11:16 PM
Reply to this message Reply
One of the things that make Ant nice to use is that its fairly declarative. Like an XSLT stylesheet for you source code to transform it into a built result (easier to read than XSLT, though not by much). Yes, it kind of sucks as a scripting language, and yes, XML in general sucks as a scripting medium.

But build scripts are often rules. A big part of Make wasn't as much running lists of commands after each target, it was defining how to turn a *.c into a *.o, and a *.o into a *.a, etc. A big part of build scripts are defining these rules. And ant not only has a number of these rules defined for easy use, it has enough re-usable objects in it so that writing you r own task can be pretty easy.

But I can say this, but at the same time I've written "simple" ant scripts that have swollen to 4,000 lines. With build scripts, while the majority of work can get done with a few clean lines, the exceptions and fiddly bits make everything more complicated. Thats when you really need a scripting language, and thats when ant begins to chafe.

Ant slowly evolves, the addition of macros has given some hope of slimming down the monolithic ant files, and include files keep us from using horrible ENTITY includes. But this is probably leading to a nightmare of hard to navigate and debug includes of includes, like CPP macros run amock, or worse, imake.

What I'm trying to say is that a build scripts are different, and so they need different languages. But then they aren't different enough, so they need regular scripting languages...damn, I have to go. The builds broken again.

Walter Higgins

Posts: 2
Nickname: tent8f
Registered: Jul, 2003

Re: The Sum of Ant Posted: Sep 8, 2005 8:57 AM
Reply to this message Reply
Take a look at TinyMake, it's a perl-based tool for building software.
dependencies are declared using standard perl code...

file { sh "touch $target" } "master.log" => ["system.log","app.log"];

http://search.cpan.org/~walterh/TinyMake003/TinyMake.pm

Walter

Flat View: This topic has 11 replies on 1 page
Topic: Another HeronFront release Previous Topic   Next Topic Topic: Scheduling and Parkinson's Law


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us