The Artima Developer Community
Sponsored Link

Weblogs Forum
Ruby vs Python for Code Generation and Flippin' Templates

16 replies on 2 pages. Most recent reply: Feb 19, 2008 12:51 PM by mike bayer

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 16 replies on 2 pages [ 1 2 | » ]
Andy Dent

Posts: 165
Nickname: andydent
Registered: Nov, 2005

Ruby vs Python for Code Generation and Flippin' Templates (View in Weblogs)
Posted: Feb 14, 2008 10:29 PM
Reply to this message Reply
Summary
Ruby is the cool new kid of on the block and has some significant cred in code-generation circles. Python has whitespace sensitivities that make it a poorer fit for templating solutions. This sounds like a no-brainer decision but is the real argument about which language the market wants to see used?
Advertisement

With my return to running my own business as a full-time activity and freedom to indulge entrepeneurial instincts, one of the projects I'll be putting more time into is AppMaker. AppMaker is a long-standing Mac product for application generation to which I acquired the rights when the original author, Spec Bowers, retired. Progress has been stalled for much of the time of OS/X with Apple's closed attitude on Cocoa - a contributing factor to Spec's retirement.

Given the length of time that has elapsed since there was a new release of AppMaker and the fact that very few users seemed to customise templates significantly, I feel some freedom to vary the template approach. For a start, I'm planning to replace the somewhat-idiosyncratic OO language with Python or Ruby. There's no justification for maintaining a complex language, with an old code base, when the world has moved on to create outstanding scripting and text manipulation languages. The startup cost for any new AppMaker user wanting to customise their templates will be drastically reduced if they are based on a standard language.

I was very impressed with Jack Herrington's Code Generation in Action which includes a taxonomy of code generators, and Ruby examples. His book, and the growth of Ruby in the somewhat code-generationy Rails framework, have put me in the position of having to choose - would you use Ruby or Python for a code generator language?

The AppMaker generators best fit what he calls a Partial Class Generator where a definition file (the AppMaker Project) is combined with a set of templates (the selected language/framework) to generate the code. The previous market for the product included a strong educational segment as the code generated was deliberately made representative of best practice for a given framework.

Tagged vs Mode-Flipping Templates

Many people would be familiar with JSP templates as an example of the Partial Class Generator approach. However, the AppMaker templates have somewhat of a twist and I'm trying to decide if it is a style worth preserving. Here's a reminder of the traditional bounded template that Herrington shows or JSP conditional logic:

Herrington Example Template

public class <%= class_name %>Base {       
<% fields.each { |field| %>                              
  protected String _<%= field %>;<% } %>                 
  public <%= class_name %>Base()                         
  { <% fields.each { |field| %>                          
    _<%= field %> = new String();<% } %>                 
  }                                                      
<% fields.each { |field| %>                              
  public String get<%= field.capitalize %>() { return _<%= field %>; } 
  public void set<%= field.capitalize %>( String value ) { _<%= field %> = 
value; } 
<% } %> 
} 

JSP style Templates

 <% if ("someValue".equals(request.getParameter("param1"))) { %>
        Generate this template text if param1 equals someValue
    <% } else { %>
        Otherwise generate this template text
    <% } %>

AppMaker's template language

The AppMaker proprietary template language flips between a literal text state and a code state, using the percentage sign as the escape character to flip states. This results in templates that have less noise and probably would cause less confusion than using angle-bracketed tags like JSP. Of course, they are no longer viewable by a simple HTML browser which can ignore the JSP tags but the domains we're targetting for such generation are more likely to be generating traditional source code into multiple files such as shown (with elisions...) below.

 procedure Window.genWindow
   import   appName: string
   declare cmdList:  list of Command
   getViewCommands (/* cmdList */)
   declare dlgList:  list of Dialog
   getDialogList (/* cmdList, dlgList */)
   import   firstWindow:   bool  // param
   declare  windName:      string = window.name
...
   declare  hdrFile:    file
   filename = windName + ".h"
   hdrFile.create (name = filename/*, creator, fileType*/)
   hdrFile.open ()
   hdrFile.genText ()
%
// %filename%

#pragma once

#include "AMWindow.h"
%genHdrItemInclude ()%

class %dataName%;

class %WindName% : public AMWindow {
public:
         %WindName% (
               AMAEObject*    inSuperObject);
   virtual ~%WindName% ();

public:
   static void    Create      (AMDoc*        inDoc,
                         %dataName%*      inData);

public:
   virtual void   Open     (AMDoc*        inDoc);
   virtual void   Close ();
...
%
for each cmd in cmdList
   cmd.declareDoCommand ()
end for
%
protected:
   virtual void   ActivateData ();
   %for each WindowItem
      windowItem.genAllAuxWinDecl ()
   end for%

public:
   %dataName%*    mData;

%for each windowItem%
   %windowItem.genAllWindowField ()%
%end for%
};
%
   hdrFile.close ()

//-----
   declare  srcFile: file

   filename = windName + ".cp"
   srcFile.create (name = filename/*, creator, fileType*/)
   srcFile.open ()
   srcFile.genText ()
%
// %filename%

#include "%windname%.h"

#include <Types.h>
...
%
for each dlg in dlgList
   //declare   itsClassName:  string
   dlg.getClassName ()
%
#include "%itsClassName%.h"
%
end for
%
%genIncludeData ()%
%genSrcItemInclude ()%
%genCommandIncludes (/* cmdList */)%

%If lang = "MPW"%
   #pragma segment %windname%

%end if%

void  %WindName%::Create (
   AMDoc*         inDoc,
   %dataName%*    inData)
{
   %WindName%*    winObj = new %WindName% (inDoc);

   if (winObj != nil) {
      winObj->Open (inDoc);
      winObj->ConnectToData (inData);
      ShowWindow (winObj->mWindow);
   }
}

%WindName%::%WindName% (
   AMAEObject*    inSuperObject)
   : AMWindow (inSuperObject)
{
   mData = nil;
}

%WindName%::~%WindName% ()
{
   mData->RemoveResponder (this);
}


}

void  %WindName%::Close ()
{
   delete this;
}
...
%
   srcFile.close ()
end genWindow


A Kong

Posts: 1
Nickname: ak2000
Registered: Feb, 2008

Re: Ruby vs Python for Code Generation and Flippin' Templates Posted: Feb 15, 2008 2:50 AM
Reply to this message Reply
I do not think the white space in python is a very valid argument against Python's usefulness in code generation. I used python cheetab, a template engine, a lot for code generation, and it works like a charm.

Eli Courtwright

Posts: 14
Nickname: eliandrewc
Registered: Jan, 2006

Re: Ruby vs Python for Code Generation and Flippin' Templates Posted: Feb 15, 2008 10:29 AM
Reply to this message Reply
Agreed; I've used Cheetah Templates and can attest that whitespace is not a problem at all. However, I've also used PSP, where it can be quite annoying. Fortunately, with the multitude of Python web frameworks these days, I'll probably never have to use PSP again.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Ruby vs Python for Code Generation and Flippin' Templates Posted: Feb 15, 2008 1:37 PM
Reply to this message Reply
Mr. Dent is fond of claiming that syntactically significant whitespace is a problem but is apparently reluctant to back it up.

Larry Bugbee

Posts: 13
Nickname: bugbee
Registered: Dec, 2004

Re: Ruby vs Python for Code Generation and Flippin' Templates Posted: Feb 15, 2008 4:50 PM
Reply to this message Reply
I too have used Python for dynamic code generation without any ill effects. I suspect those that claim a problem don't know how or haven't tried.

Johannes Brodwall

Posts: 19
Nickname: jhannes
Registered: Jun, 2003

Re: Ruby vs Python for Code Generation and Flippin' Templates Posted: Feb 15, 2008 6:34 PM
Reply to this message Reply
Err... Cheetah doesn't use Python language. I don't think anyone denies that a Python based engine can drive a templating language. But I have yet to see that the templating language is Python, in the way that JSP scriptlets contain real Java code and ERb templates contain real Ruby code. The obvious reason would seem to be that the indentation of the generated text (HTML, say) would easily interfere with the Python structure.

Whether having the template language be in the same syntax as the host is a good idea or not is of course debatable. JSP has not been astonishingly successful in a technical sense. I think ERb has a significant better result.

If there is a Python templateting language where the template syntax is Python code, I'd love to learn about it, though.

Andy Dent

Posts: 165
Nickname: andydent
Registered: Nov, 2005

Re: Ruby vs Python for Code Generation and Flippin' Templates Posted: Feb 15, 2008 8:30 PM
Reply to this message Reply
> Mr. Dent is fond of claiming that syntactically
> significant whitespace is a problem but is apparently
> reluctant to back it up.

S'funny - two posts in a row and I'm now fond of claiming. I'm actually a fan of Python as a programming language, especially because of how well the whitespace works.

However, I am very much aware of the problems with whitespace in mixed Python programming environments (painful experience) and with copying and pasting code from email and web pages. I can't back this up with examples because they are all transitory experiences within a programming team I no longer belong to and wouldn't have the right to disclose anyway.

When it comes to markup languages, as reference my recent doubts over YAML, I stand by my experience. Significant whitespace in casually edited files with insufficient redundancy is a solution that doesn't scale. If you want to look into some of the complex XML-based interoperability projects from which I derive these opinions, have a waltz through https://www.seegrid.csiro.au/.

Andy Dent

Posts: 165
Nickname: andydent
Registered: Nov, 2005

Re: Ruby vs Python for Code Generation and Flippin' Templates Posted: Feb 15, 2008 8:46 PM
Reply to this message Reply
Johannes wrote
> ... I have yet to see that the
> templating language is Python, in the way that JSP
> scriptlets contain real Java code and ERb templates
> contain real Ruby code. The obvious reason would seem to
> be that the indentation of the generated text (HTML, say)
> would easily interfere with the Python structure.

Absolutely. You have understood and explained the issue, clarifying what I didn't. Thanks.

> If there is a Python templateting language where the
> template syntax is Python code, I'd love to learn about
> it, though.

Python Server Pages http://www.ciobriefings.com/psp/ is an example I found when researching, where Python is embedded as the template language:

<% %> or <%= %> used to bracket Python code

<%=self.standard_html_header(self)%>
<h1>Links</h1>
< ;ul>
<%
for id,ob in self.objectItems(['Python Server Page']):
title = ob.title
if title!='': print '<li><a href="%s">%s</a>' % (ob.absolute_url(),title)
%>
</ul>
<%=self.standard_html_footer(sel f)%>

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Ruby vs Python for Code Generation and Flippin' Templates Posted: Feb 15, 2008 10:44 PM
Reply to this message Reply
> > Mr. Dent is fond of claiming that syntactically
> > significant whitespace is a problem but is apparently
> > reluctant to back it up.
>
> S'funny - two posts in a row and I'm now fond of
> claiming
. I'm actually a fan of Python as a
> programming language, especially because of how well the
> whitespace works.

I find it a little annoying to directly question someone about an assertion, receive no response and then see a similar assertion posted by that same person days later. But, I do regret the tone of my post above all the same.

> However, I am very much aware of the problems with
> whitespace in mixed Python programming environments
> (painful experience) and with copying and pasting code
> from email and web pages. I can't back this up with
> examples because they are all transitory experiences
> within a programming team I no longer belong to and
> wouldn't have the right to disclose anyway.

I don't need proof. I just think you could explain exactly how things go wrong and what issues were caused. Then we could discuss whether other constraints would resolve those issues. Until I know for sure what the issues are, however, it's difficult to come up with solutions.

I can understand the issue with email but I don't find that to be a very compelling reason.

> When it comes to markup languages, as reference my recent
> doubts over YAML, I stand by my experience. Significant
> whitespace in casually edited files with insufficient
> redundancy is a solution that doesn't scale.

Honestly, I don't really have a strong prejudice for significant whitespace. What I really care about is required consistent indentation. But it's frustrating to keep hearing about how significant whitespace is an issue without a clear explanation. My personal experience is that it's never been an issue. There are a lot of ways to destroy source. That's why source control exists. I'm not swayed that significant whitespace is so fragile that it should not be used as a technique.

> If you want
> to look into some of the complex XML-based
> interoperability projects from which I derive these
> opinions, have a waltz through
> https://www.seegrid.csiro.au/.

I'm not sure I understand how XML is relevant to the issue at hand. Perhaps you have a more direct link.

John M Camara

Posts: 15
Nickname: camara
Registered: Dec, 2004

Re: Ruby vs Python for Code Generation and Flippin' Templates Posted: Feb 16, 2008 2:21 AM
Reply to this message Reply
For code generation I used to prefer Cheetah but now I use Mako [1]. It's syntax is simpler and it's coding syntax matches Python except for the '%' marker and the end of code markers. It's just as fast and capable as Cheetah and it's documentation is very clear and covers all features.

Mako Supports
* inheritable templates
* name spaces
* Unicode
* internationalization
* caching
* etc

Here is a simple example from the projects home page

<%inherit file="base.html"/>
<%
rows = [[v for v in range(0,10)] for row in range(0,10)]
%>
<table>
% for row in rows:
${makerow(row)}
% endfor
</table>

<%def name="makerow(row)">
<tr>
% for name in row:
<td>${name}</td>\
% endfor
</tr>
</%def>


[1] http://www.makotemplates.org/

Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Ruby vs Python for Code Generation and Flippin' Templates Posted: Feb 16, 2008 2:50 PM
Reply to this message Reply
I wonder why all those template languages look like language design accidents or first year student experiments? Maybe the ugliness is intended and instead of gluing two languages together so accurately that no one notices the glue the glue becomes part of the visual design and the look shall stick to all the funny % characters because they spot the interesting things.

Moreover '{%' or '<%' like parens are idiomatic for template languages because no tasteful language designer would ever use them in their language design: ugliness as a simple disambiguation strategy.

mike bayer

Posts: 22
Nickname: zzzeek
Registered: Jan, 2005

Re: Ruby vs Python for Code Generation and Flippin' Templates Posted: Feb 16, 2008 3:03 PM
Reply to this message Reply
> I wonder why all those template languages look like
> language design accidents or first year student
> experiments? Maybe the ugliness is intended and instead of
> gluing two languages together so accurately that no one
> notices the glue the glue becomes part of the visual
> design and the look shall stick to all the funny %
> characters because they spot the interesting things.
>
> Moreover '{%' or '<%' like parens are idiomatic for
> template languages because no tasteful language designer
> would ever use them in their language design: ugliness as
> a simple disambiguation strategy.

while I'm personally partial to the % based approach (I wrote Mako), I think you're looking for http://genshi.edgewall.org , another great approach.

in short i'm a little surprised that the original post here did not mention any of the currently popular Python templating languages...if PSP, which is not widely used at all, is all that could be found, I fear we aren't advocating for Python strongly enough.

Andy Dent

Posts: 165
Nickname: andydent
Registered: Nov, 2005

Re: Ruby vs Python for Code Generation and Flippin' Templates Posted: Feb 16, 2008 8:22 PM
Reply to this message Reply
James Watson wrote:
> > However, I am very much aware of the problems with
> > whitespace in mixed Python programming environments
> > (painful experience) and with copying and pasting code
> > from email and web pages.
>
> I don't need proof. I just think you could explain
> exactly how things go wrong and what issues were caused.

Sorry, I thought the Python issues were sufficiently well-known I didn't need to elaborate. The problems generally come from mixing tabs and spaces, tools which react differently to the tab key and people moving between tools and sharing code across environments.

A common recommendation for Python programmers is to disallow tab characters in source and only allow for an agreed number of spaces per indent. The typical ways in which an indent is inserted or adjusted in front of code are:

- manual insertion of X number of spaces
- pressing the tab key to insert an "indent" (often translated as X spaces)
- pressing shift-tab to remove one indent level
- pressing other key combinations to control indent/outdent
- hitting return at the end of a line and having the auto-indent position your next line

Most source editors allow you to set the tab key or their indent/outdent facility to thus use that number of spaces. I've encountered some situations, particularly where keys other than tab are used for indent/outdent, where the tab key will still insert tabs. If I'm correct in recalling this, one particularly subtle behaviour I've seen is that the tab key inserts spaces (adjusting indent) when it is at the start of the line or if the entire line or lines are selected but will still insert plain tabs when the insertion point is within a series of spaces.

Whatever the cause, I've definitely seen umpteen examples of source code that have ended up with mixed spaces and tabs doing the indent. Python copes to some degree but can be unpredictable and often complain about such source.

Depending on the email clients used, quoting is sometimes shown as indentation and tabs are translated into varying numbers of spaces. I'm talking about people on Mac, Windows and Linux clients here.

IBM's DeveloperWorks, if I recall correctly, has often published web articles where the Python source example fragments have been mangled by their whitespace processing and wrapped unmercifully. Usually the source is available as an attached zip file but copying and pasting quick examples in such circumstances requires reading the source and working out the apparent logic and thus where indents lie.

In a language like Python with syntactical clues and being able to interpret the logic of the source, an experienced programmer can usually deduce the correct indent levels and logic nesting. The start of class and function definitions are unambiguous so the main point of concern is the local nesting into loops and conditionals.

> But it's frustrating to
> keep hearing about how significant whitespace is an issue
> without a clear explanation. My personal experience is
> that it's never been an issue. There are a lot of ways to
> destroy source. That's why source control exists.

I have seen confusion over diffs between versions because the diff tool had been configured to ignore leading whitespace, in the interests of reducing noise levels when comparing C++ code, in ignorance of the effects on Python source comparison.

I'm not talking so much about destroying source as getting it wrong to start with when transferring from other sources into your program. Hopefully the above scenarios clarify my point.

> I'm not swayed that significant whitespace is so fragile that
> it should not be used as a technique.
As I said in my previous post, when it comes to Python code I'm a fan of the language and I think the huge benefits of the significant whitespace override the occasional hiccups in a team. If you have a mixed environment it is important to be wary of the whitespace issues. PyLint has default configurations to pick up mixed spaces and tabs.

When it comes to storing structured data, the reason I recommend XML (or any other format with named closing structures) is that complex data often lacks the redundancy in content that clues us to indentation levels being incorrect and thus nesting being wrong. If you have nested levels of items with arbitrary property lists, how do you tell if a given key-value pair belongs to a parent structure other than solely by the indent level?

BTW thanks for prompting me to go cross-check the YAML Wikipedia article as I picked up a mistaken assertion in there that XML lacked a reference facility (IDREF has been built in since v1.0) http://en.wikipedia.org/wiki/YAML#Non_Hierarchical_Data_Models and other anti-XML bias.

Andy Dent

Posts: 165
Nickname: andydent
Registered: Nov, 2005

Re: Ruby vs Python for Code Generation and Flippin' Templates Posted: Feb 16, 2008 8:32 PM
Reply to this message Reply
Kay Schluehr wrote:
> Moreover '{%' or '<%' like parens are idiomatic for
> template languages because no tasteful language designer
> would ever use them in their language design: ugliness as
> a simple disambiguation strategy.

Giggling thanks, Kay.

I think the vast majority of template languages are invented as an attempt to redo JSP or end up looking like that through the same thinking process - make the tags look like elements so people used to hand-editing HTML will understand them.

Whilst I appreciate its power, I've never enjoyed programming in XSLT and shudder at some of the other attempts to write programming languages in XML.

One of the distinguishing characteristics I think of using a template language like AppMaker's, to generate programs, is the interweaving of comparatively large amounts of template source with the literal text. This is very easy to edit with its simple flipping modes compared to the tags for bounding embedded stuff approach of most of the web-oriented template languages.

The desire to replace the homegrown OO language in the templates comes from a maintenance need and a marketing suspicion that a more recognisable scripting language would attract more people to modify templates.

Andy Dent

Posts: 165
Nickname: andydent
Registered: Nov, 2005

Re: Ruby vs Python for Code Generation and Flippin' Templates Posted: Feb 16, 2008 8:50 PM
Reply to this message Reply
Mike Bayer wrote
> i'm a little surprised that the original post
> here did not mention any of the currently popular Python
> templating languages.

I mentioned PSP because I had a link to it in my notes from research about three years ago and it was representative of the kind of templating syntax out there. I didn't mean to imply any dominance or endorsement. Thanks for the pointer to Genshi and I particularly appreciated its plain text mode:

Dear $name,

These are some of my favorite fruits:
#for fruit in fruits
* $fruit
#end
I appreciated even more the reference on the Genshi pages to http://tomayko.com/weblog/2004/11/30/pythonic-xml-based-templating-language and I know exactly how he feels about XSLT is the strange lover you can't seem to leave behind :-).

His blog also led to interesting posts on template languages http://www.loudthinking.com/arc/000405.html and http://tomayko.com/weblog/2004/12/02/a-note-on-template-design which clarified for me the issue that good design in writing templates for web output probably leads to significant differences from a code generation templates. Specifically, web templates are presentation documents that should not include business logic.

Flat View: This topic has 16 replies on 2 pages [ 1  2 | » ]
Topic: Ruby vs Python for Code Generation and Flippin' Templates Previous Topic   Next Topic Topic: Ode to the Python Quick Reference (Ruby Needs One!)


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us