The Artima Developer Community
Sponsored Link

Weblogs Forum
Code Generation: The Real Lesson of Rails

42 replies on 3 pages. Most recent reply: Apr 6, 2007 12:21 PM by robert young

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 42 replies on 3 pages [ « | 1 2 3 ]
Ivan Lazarte

Posts: 91
Nickname: ilazarte
Registered: Mar, 2006

Re: Code Generation: The Real Lesson of Rails Posted: Mar 23, 2006 3:16 PM
Reply to this message Reply
Advertisement
Why does everyone try to impose objects on a relational model? There are several ways to get around the configuration
and code generation temptations when dealing with databases.

These are some sql solutions:

1. Use a Hashmap to add and delete "column like" data manually per row in resultset.

2. Use Commons BeanUtils Dynamic Bean facilities to map from result sets.
http://jakarta.apache.org/commons/beanutils/commons-beanutils-1.7.0/docs/api/org/apache/commons/beanutils/package-summary.html

3. Use Commons DBUtils QueryUtility callbacks with a predefined bean. It can map resultsets to lists of hashmaps, or a list of
of beans you might use...
http://jakarta.apache.org/commons/dbutils/apidocs/org/apache/commons/dbutils/handlers/BeanListHandler.html

Personally, I'm never for magic, so I'll take sql any day over a magically available database.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Code Generation: The Real Lesson of Rails Posted: Mar 23, 2006 8:36 PM
Reply to this message Reply
> I have heard said "code generation is a sign that your
> language lacks a high enough abstraction level".
>
I would say it is a sign that your language lacks a high enough abstraction level for the particular domain for which you're generating code, and that's usually not a bad thing. Our code generators very definitely raise the level of abstraction of our programming, because we program in our DSL, which is a domain specific language--a language that is very specific to the class of problem we are repetitively trying to solve.

> I suspect there will come a day where the best of both
> worlds will be available: we will be able to point a tool
> at a schema and there will be "invisible" generated code
> ala-active record, and yet the types will be statically
> checked and the IDE will allow us to access them as if
> they were statically defined. I see a lot of use for code
> generation today, but I hope that IDE/language builders
> realize that each time people resort to code generation it
> is a strong indicator that improvement is needed in the
> language/toolchain.
>
That sounds very close to what we have right now with our Java code generators, except the generated code isn't invisible. You don't have to look at the code if you don't want, I suppose, but I like the fact that you can. You can look at the code and/or the JavaDoc. It makes it less like magic.

Peter LeClair

Posts: 1
Nickname: wpl
Registered: Jun, 2005

Re: Code Generation: The Real Lesson of Rails Posted: Mar 23, 2006 9:25 PM
Reply to this message Reply
Bill, I like this article, and I am glad to see some discussion about code generation. I believe this approach to coding could be exploited a great deal more than I have seen it.

I developed a code generator to get rid of the tedium and typo's in writing client/server apps. Focus of the generator is not on how easily the programmer can produce code, or how the project can be standardised, but on how quickly and accurately the design specifications can be instantiated. As a result, the code generator avoids some of the problems noted in the various responses.
1- The code is generated from a set of project specifications, so logic is not restricted to interpretations of database metadata.
2- The generator writes source code for an entire project, from html forms/servlets through business process managers to sql interface, so there are no gaps, no "90% problem". The generator produces source code, so alteration is possible. And yes, if it needs tweaks, it is the generator that gets tweaked.
3- The generator does not use frameworks or external libraries, only a servlet container (eg Tomcat). If you are coding the whole project automatically, why use third-party software? Data processing is visible, and delivery installation is simple. (And programmers don't need to learn frameworks and tool libraries...)
4- The modules for the different tiers are standardised, so it is easy to see what is going on (or not going on, but after several iterations we have pretty well got it right!)

The generation process is simple: enter the specs, push the generate button. Generating a typical business project takes minutes, and produces both Java source and classes. (The tendency is to evolve projects in small increments.) The resulting code is simple as dirt, just Java on Tomcat. This is the way I wanted to code when I was doing it by hand: compact, legible, bug-free and fast!

Scala Enthusiast

Posts: 7
Nickname: sashao
Registered: Jun, 2005

Re: Code Generation: The Real Lesson of Rails Posted: Mar 26, 2006 4:39 PM
Reply to this message Reply
>>> sometimes you can't get there by abstracting via OO mechanisms, at least not without sacrificing other good things such as static type checking, readability, and speed of development. It is hard for me to figure out how to describe it, but there are situations in which you are repeatedly doing the same class of thing, but for which you couldn't make a class.<<<

OK, I understand the static typing part: if you have _lots _of business logic, then static typing can help you catch some of the errors in compile-time you would need to otherwise catch with tests using Ruby. Please note though that object-model the ORM cooks for you would be usually "flatter" and less usable for static checking -- how many times have you seen ZIPcode modelled by ORM as an object instead of a string? If what you do most of the time is shipping tuples to and from the database doing occasional runtime validation, then there is not much advantage in static typing (performance, maybe, but this is a topic for another discussion).

Readability -- still not sure how much readability (in man-months :-) you gain by using user.getName() as opposed to user.get("name") or user.get(NAME) when what you really want is user.name (Ruby/Python) or user.Name (C#). IOW, how much effort and inconvenience should you be willing to spend on "beautification" here?

The most intriguing is however the speed of development part. Granted, generating repeated statements is faster than writing them by hand, but if we compare "dynamic objects" generated by the ORM during runtime based on DB schema (the "Rails way") to their staticly generated counterparts -- what is exactly the gain in speed of development? In my book, the latter looses because there is an additional step to be invoked.

One of the time-saving features I like very much in Rails is immediate feedback. I was blown away by this -- literally: modify the code--CTRL+S--reload the page--see. Feels almost like compiling Java in Eclipse. Does it work the same way in the static Java code generation scenario? What has been your experience so far?

Jules Jacobs

Posts: 119
Nickname: jules2
Registered: Mar, 2006

Re: Code Generation: The Real Lesson of Rails Posted: Mar 28, 2006 6:16 AM
Reply to this message Reply
Lisp has another form of code generation. It sits between dynamic and static code generation.

Static code generation: code it generated at development time.
Dynamic: generated at runtime.

Lisp macros: generated at compile time.

If you do it that way, you can have static typing and generated documentation + hidden code.

damien morton

Posts: 15
Nickname: dmost
Registered: Jan, 2004

Re: Code Generation: The Real Lesson of Rails Posted: Mar 28, 2006 8:03 AM
Reply to this message Reply
Ya know, ive seen the code generation angle pushed on multiple fronts over the years, and its an excellent idea, best realized in a compiler.

Once you step outside the rigid confines of a formal programming language, code generation largely falters.

Oh, well, you do find crusty old Lisp programmers doing magic with defmacro, but its best described as a dangerous art, and kept away from minors.

I mean, lets face it, how many people have you worked with whose hand made code you trust, let alone trusting their code generators.

One of the problems that Bill suggests code generation can solve is creating static types that map nicely to database tables. Now, database tables, as we all know, are pretty much dynamic types. You can add or remove columns to your hearts content with them, and at runtime! What fun! So Bill neatly solves his problem by generating new type declarations which are then compiled into his statically typed program. He probably also whacks a bunch of boiler plate code into his generated types. But what happenss when someone changes the database, as database admins are wont to do? Well, theoretically, you'd have to emit new generated code, and compile and re-link everything, right there on the spot, which will go very smoothly unless you have something in your actual handwritten code that depends on something in the database schema that has changed. Hmm, maybe making your static code depend on your dynamic database isnt such a good idea.

Hmm, oh, but why doesnt he just stick with dynamic types all the way through his code? Why not simply have a dynamic record type, whose columns are accessed by string names, just as in the database. Oh, but then, thats pretty much what Ruby does, and pretty much also what any Python database mapping layer does too.

If you worked at Microsoft, you'd be worrying about code completion and filling little dropdowns with helpful lists of words. Well, why not have your dropdown code go off to the database directly and query the schema to fill in the list

Maybe all this dynamic stuff is too slow for you, in which case you shouldn't be using a frikking relational database in the first place.

So far the only people to propose "code generation" that seems to work are the AOP crowd, who treat the concept with respect, and do it with a very light touch. They must have been paying attention they day they were taught that self-modifying code is bad form.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Code Generation: The Real Lesson of Rails Posted: Apr 2, 2006 10:14 PM
Reply to this message Reply
> One of the problems that Bill suggests code generation can
> solve is creating static types that map nicely to database
> tables. Now, database tables, as we all know, are pretty
> much dynamic types. You can add or remove columns to your
> hearts content with them, and at runtime! What fun! So
> Bill neatly solves his problem by generating new type
> declarations which are then compiled into his statically
> typed program. He probably also whacks a bunch of boiler
> plate code into his generated types. But what happenss
> when someone changes the database, as database admins are
> wont to do? Well, theoretically, you'd have to emit new
> generated code, and compile and re-link everything, right
> there on the spot, which will go very smoothly unless you
> have something in your actual handwritten code that
> depends on something in the database schema that has
> changed. Hmm, maybe making your static code depend on your
> dynamic database isnt such a good idea.
>
In all the projects I've ever worked on, the database schema was one aspect of the release. If you change the database in a way that requires the application to change, which we did from release to release, then you need to deploy a new version of the application. It doesn't matter how dynamic your metaprogramming is.

> Hmm, oh, but why doesnt he just stick with dynamic types
> all the way through his code? Why not simply have a
> dynamic record type, whose columns are accessed by string
> names, just as in the database. Oh, but then, thats pretty
> much what Ruby does, and pretty much also what any Python
> database mapping layer does too.

Well, because I'm using Java whose compiler does static type checking, and I want to take advantage of that in my generated code.

piglet

Posts: 63
Nickname: piglet
Registered: Dec, 2005

Re: Code Generation: The Real Lesson of Rails Posted: Apr 3, 2006 10:39 AM
Reply to this message Reply
Bill: "In all the projects I've ever worked on, the database schema was one aspect of the release. If you change the database in a way that requires the application to change, which we did from release to release, then you need to deploy a new version of the application. It doesn't matter how dynamic your metaprogramming is."

Completely agree. I don't know in which kind of project DB admins would "add or remove columns to your hearts content, and at runtime". In the projects I have seen, the admin people were more likely to freak out and start yelling whenever a minor db schema change was requested.

damien: "Maybe all this dynamic stuff is too slow for you, in which case you shouldn't be using a frikking relational database in the first place."

What are ya talking about? You think RDBs are too slow?

Gabe da Silveira

Posts: 1
Nickname: dasil003
Registered: Sep, 2006

Re: Code Generation: The Real Lesson of Rails Posted: Sep 16, 2006 3:01 PM
Reply to this message Reply
I'm surprised no one has made note of the fact the static code generation is not as flexible as dynamic code generation, because you have to define all possibilities. Rails for instance has dynamic finder methods such as:

Object.find_by_name_and_email

In fact you can find by any number of fields dynamically. The number of dynamic finders that are possible are all the possible permutations of the fields, so you can see it's completely impractical to generate these in advance.

Rails uses code generation to great effect, but it's really not the lesson. The overall lesson of Rails is that a simpler web framework goes a long long way. More specifically Rails is successful because of how it leverages the Ruby language. Rails knockoffs in other languages are like soda without the fizz. What you need is a practitioner (and advocate) of a language to have vision of how they'd like to develop web applications and create a new framework around that vision.

I think it's tough in Java, because the existing tools are great, so where's the incentive to invent something significantly different? Of course Ruby will also face this problem when it comes to making something different from Rails. There's also the issue of inertia. Will a lighter framework feel like it's moving backwards? Unless there's real dissatisfaction in the community, something simpler will not be able to catch on.

Andy Dent

Posts: 165
Nickname: andydent
Registered: Nov, 2005

Re: Code Generation: The Real Lesson of Rails Posted: Oct 16, 2006 11:32 PM
Reply to this message Reply
> Yes, we don't touch the generated code, because it isn't
> "source" code anymore.

one litmus test for this - is your generated code being checked into version control?

It shouldn't be (unless you have some really paranoid rules and also check in .obj files ;-) )

Xin Zhao

Posts: 1
Nickname: xin
Registered: Apr, 2007

Re: Code Generation: The Real Lesson of Rails Posted: Apr 2, 2007 3:50 AM
Reply to this message Reply
I think if anyone is to pick code generation over mapping then a good architecture is very important.

I have spent some time developing a .net generator focused very much on extensible architecture for C#.

CodeAuthor - Generator for C# & SQL 2005 http://www.codeauthor.org/

Paul Butcher

Posts: 1
Nickname: paulbutche
Registered: Apr, 2007

Re: Code Generation: The Real Lesson of Rails Posted: Apr 6, 2007 6:15 AM
Reply to this message Reply
I read this article a while ago, and it got me thinking. I've just blogged some thoughts (OK - it took me a while to get them together :-)

http://about.82ask.com/2007/04/05/ruby-on-rails/

I would be very interested in any thoughts/comments.

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: Code Generation: The Real Lesson of Rails Posted: Apr 6, 2007 12:21 PM
Reply to this message Reply
With regard to RoR. There is a sub-cult which looked at Scaffolds, about a year ago, and asked the obvious question: "why not generate the whole UI from the Catalog?" He/they continued for a while. Haven't kept an eye on it, though. OTOH, Migrations in v1.2.3 seems to be moving in that direction. OTOOH, DHH still (last time I looked) remains adamant that the database be merely a passive datastore, all logic in app code; 'database' meaning only a SQL parser in front of the file system.

The argument that an OO system is somehow "better" than a Relational one is due, in all cases I've witnessed, to ignorance about what a RDBMS actually stores. Date's "What Not How" is a good place to start.

Then there's this recent quote I have from Craig Mullins (a mainframe DB2 expert; MF DB2 tends toward COBOL/VSAM in pretty shorts, so his insight is unusual):
Not only am I not aware of any resource that would advocate logical modeling of data in an unnormalized fashion, I would be against anyone reading it if it existed. The logical data model should be normalized. Normalization is the process of identifying the one best place each fact belongs.

What many OO folk won't/can't/don't accept is that the Relational model is OO data, and that method is not needed since the model itself constrains each part of the database to be consistent. Mullins last sentence is the key to all understanding: if you do that, then all else can be mechanically generated from the catalog (including domains, triggers, SP, FK, PK). It is what folks have been wishing for, but avoiding: a fully declarative development paradigm. But then, we need very few coders and very many database designers. Coders' sphincters start getting tight.

Flat View: This topic has 42 replies on 3 pages [ « | 1  2  3 ]
Topic: Code Generation: The Real Lesson of Rails Previous Topic   Next Topic Topic: Back at OOPSLA

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use