The Artima Developer Community
Sponsored Link

Artima Developer Spotlight Forum
Are Ruby's Open Classes a Poor Fit for Large Projects?

51 replies on 4 pages. Most recent reply: Sep 1, 2006 3:33 PM by Joao Pedrosa

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 51 replies on 4 pages [ « | 1 2 3 4 | » ]
Merriodoc Brandybuck

Posts: 225
Nickname: brandybuck
Registered: Mar, 2003

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 25, 2006 6:56 PM
Reply to this message Reply
Advertisement
> > What's a large project? Seriously. How large does a
> > project need to be?
>
> A good question. And how long, may I add? Not ever easy to
> answer, especially before to jump in 1.0 adventure without
> any engagement about 2.0. Anyway, I see three aspects.
>
> 1. How many technologies should a company invest in to
> deliver the same kind of projects? Is it a good long-term
> strategy to adopt a solution for small/medium projects and
> another for big/mega projects? If you're hiring a junior
> programmer you must take into account a period of time for
> training, that indeed would become two. Still, the time a
> programmer spends to get started in two programming
> languages could be spent to become expert in one only.
>

As often as the flavor of the month changes and the advances made in programming languages, I'm personally not concerned abot becoming an expert in only one. I recently started hacking on some C++ again at my new job. It's been years since I've done so consistently and while I realize I've forgotten quite a bit of what I knew, I've learned some things along the way that, overall, have made my software design skills better and my C++ now, I believe, is much better than what it was because of my exposure to other languages along the way. The things I've forgotten that are important are easily enough found in some book or online reference. I believe many people feel the same way.

Expertise is necessary in some instances but overall, I believe, a good programmer can become reasonably competent in most languages in a fairly short time. And your decisions to your questions need to weigh a lot of factors. Is time to market more important than stability? How flexible does the application need to be? How many platforms must it run on? Where are the bottlenecks? What languages are my staff proficient in? How easily can I find people proficient in my chosen platform and how much does that matter? Do we need tight integration to the hardware or OS? Is the project being sold to paying customers or is it an internal tool for the support staff? Personally I believe a good long term strategy is using the best tool to get the job done. If this results in different languages and platforms for different projects I'm perfectly fine with that. Many projects I've worked on use different languages for different parts of the project. Sometimes you don't have to be at the project level in order to have to make that decision.

> 2. I didn't talk about dynamic languages generally. I made
> specific mention of Ruby. Do any/most dynamic languages
> support open classes?
>

If you mean by open classes the ability to change the class by adding methods or overriding methods at runtime, then I know you can do this in python. Here is a trivial example with myClass residing in a module called SimpleClass.py



class myClass:
def __init__(self):
self.A = 1
self.B = "A"
def GetNum(self):
return self.A
def GetLetter(self):
return self.B


>>> x = SimpleClass.myClass()
>>> x.A
1
>>> x.B
'A'
>>> x.GetLetter()
'A'
>>> x.C = '+'
>>> x.C
'+'
>>> x.ShowOperator = lambda : '+'
>>> x.ShowOperator()
'+'
>>> x.GetLetter = lambda : 'Q'
>>> x.GetLetter()
'Q'
>>>



> 3. Let's assume that with a set of self-imposted practices
> a good development team addresses these issues. What about
> using third-party modules?
>

As with any third party module in any language, wouldn't you have to assume it would work as documented? I can only imagine that third party module that decided to mess with built-ins would get dropped by the community in a heartbeat. There are instances where this sort of thing may make sense for an internal project but anything that one would expect to get widely adopted should be very well behaved. C++ allows you similar flexibility in some respects and it has the same problem. Any deviation from standard behavior had better be very well documenated.

Joao Pedrosa

Posts: 114
Nickname: dewd
Registered: Dec, 2005

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 25, 2006 10:33 PM
Reply to this message Reply
By running the "sloccount" tool on my main Ruby project, I get:

SLOC Directory SLOC-by-Language (Sorted)
9388 gtk_rules ruby=9384,sh=4
6896 db_rules ruby=6896
3040 db_admin ruby=3040
2853 web_rules ruby=2853
2140 ruby_editor ruby=2136,sh=4
1912 report_rules ruby=1912
1908 erp_rules ruby=1908
1623 project_rules ruby=1623
881 common_rules ruby=881
642 bookmark_rules ruby=642
560 wiki_rules ruby=560
446 blog_rules ruby=446
395 forum_rules ruby=395
138 bin ruby=138
13 conf ruby=13
0 images (none)

Totals grouped by language (dominant language first):
ruby: 32827 (99.98%)
sh: 8 (0.02%)

Add to it another 1000 of lines of code of Ruby with another extension (rbh), instead of the usual (rb). "rbh" is HTML with Ruby inside, ala PHP to some extent.

Notice the size of the projects, how they don't need to be huge to be somewhat functional. I can prove they work, because some of them are online, like:
* My blog: http://ralip.com.br/jp/blog/ (Read the first post, in which I talk about security by escaping HTML in the database layer)
* My bookmark: http://ralip.com.br/jp/bookmark/
* My wiki: http://ralip.com.br/jp/wiki/
* And some screenshots of desktop projects: http://ralip.com.br/jp/wiki/wiki?p=screenshots

Notice that all of it is powered by Ruby, all of the web site. And I use mostly my own frameworks. Everything is modular, and you can see by the separation of the several projects, but let me show you the inside of one of them:
Project Rules - Meant to be a little project management utility:

dewd@corvina:~/workspace/pilar/project_rules$ ls
doc lib test
dewd@corvina:~/workspace/pilar/project_rules$ ls -l
total 12
drwxr-xr-x 2 dewd dewd 4096 2006-07-24 00:08 doc
drwxr-xr-x 3 dewd dewd 4096 2006-05-10 20:35 lib
drwxr-xr-x 3 dewd dewd 4096 2006-07-24 00:08 test
dewd@corvina:~/workspace/pilar/project_rules$ ls -l lib/
total 4
drwxr-xr-x 7 dewd dewd 4096 2006-08-12 14:15 project_rules
dewd@corvina:~/workspace/pilar/project_rules$ ls -l lib/project_rules/
total 24
drwxr-xr-x 4 dewd dewd 4096 2006-07-24 00:08 base
drwxr-xr-x 4 dewd dewd 4096 2006-06-20 09:18 fgr
-rw-r--r-- 1 dewd dewd 377 2006-08-17 14:09 project_rules.rb
drwxr-xr-x 4 dewd dewd 4096 2006-06-04 23:25 rr
drwxr-xr-x 2 dewd dewd 4096 2006-08-24 01:34 wr
drwxr-xr-x 2 dewd dewd 4096 2006-08-12 14:15 wr_conf


"fgr" is desktop GUI. "base" is the model (MVC like). "rr" is meant to be reporting, though it has almost nothing yet. "wr" is Web. "wr_conf" is configuration for the Web.

And here's the test directory:
dewd@corvina:~/workspace/pilar/project_rules$ ls -l test/
total 32
drwxr-xr-x 2 dewd dewd 4096 2006-05-26 20:02 project_test
-rw-r--r-- 1 dewd dewd 2168 2006-08-17 14:09 sample_data.rb
-rw-r--r-- 1 dewd dewd 486 2006-08-17 14:09 test_all.rb
-rw-r--r-- 1 dewd dewd 1592 2006-08-17 14:09 test_milestone_comment.rb
-rw-r--r-- 1 dewd dewd 2134 2006-08-17 14:09 test_milestone.rb
-rw-r--r-- 1 dewd dewd 1842 2006-08-17 14:09 test_project.rb
-rw-r--r-- 1 dewd dewd 1682 2006-08-17 14:09 test_task_comment.rb
-rw-r--r-- 1 dewd dewd 1495 2006-08-17 14:09 test_task.rb

Mostly system testing that I do.

Well, to my standards, that's the hugest project I have worked on so far, and it's heaven all the time, though the good structure of everything does not come free, and I doubt many people structure their projects well enough, modularly enough. I'm sure you could create big Ruby projects of your own, in team or not.

Peter Booth

Posts: 62
Nickname: alohashirt
Registered: Aug, 2004

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 26, 2006 9:22 PM
Reply to this message Reply
I suspect that this thread says more about people as rationalizing rather than rational beings, the effects of fear on software, and the wood for the trees.

I cannot recall a sick project that would have been saved by strict typing, objects , polymorphism, pick your sacred cow.

Remember we are all doomed to live at different points on the technology adoption curve. So whilst we ponder if ruby can work for large projects there is also someone arguing that Java will never scale for non-trivial apps, an d another person who is already using Ruby for a large program. I am huge Java fan but I can't get too excited about the risks of dynamic types. I could spout the theory but in my heart I know that it's crap.

I could equally ask the question "Is Java's verbosity and ease of coding a problem for large projects?"

Ivan Lazarte

Posts: 91
Nickname: ilazarte
Registered: Mar, 2006

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 27, 2006 12:51 AM
Reply to this message Reply
i'm not "afraid" of large projects with dynamic classes, i just see it as impractical most likely eventually pointless.

everytime i see one of these ruby vs. java arguments, i think of people claiming "see, working with mud is SO much easier! it's moldable and hardens! why do you waste time with brick! haha, you fool".

i will always see it as working to build mud huts vs. knowing how to make buildings. i've enjoyed the mud too, trust me.

Joao Pedrosa

Posts: 114
Nickname: dewd
Registered: Dec, 2005

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 27, 2006 1:15 AM
Reply to this message Reply
Actually I have grown up my own gray hair as well.

Sometimes I get validation from other people as well, like from these slides someone posted on another thread:
http://research.microsoft.com/~toddpro/papers/disruptive.ppt

But most of all, the validation comes at the end of the day, when I fix one more problem that I won't have to fix again in the future. I have learned to appreciate programming even more, by fixing ordinary programming problems which won't need to be fixed again.

But that's my problem, indeed. Just because I want good maintenance, good development tools, etc, does not mean that other people want those kinds of things as well. Though there's some paradox when you hope the next tool or the next version of your tool will fix some of the problems for you.

As far as I am concerned, Windows Vista is the only mainstream tool which promises some productivity improvements in a mainstream way, though it generally does not mean good winds for the Java folks.

Anyway... Keep waiting, that I'll keep working.

Jules Jacobs

Posts: 119
Nickname: jules2
Registered: Mar, 2006

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 27, 2006 3:02 AM
Reply to this message Reply
> i'm not "afraid" of large projects with dynamic classes, i
> just see it as impractical most likely eventually
> pointless.

For most things, dynamic classes are indeed pointless, so you ignore them. But for some things, it's incredibly useful, so you use them. In Ruby you have a choice.

class Foo
  // getter
  def bar()
    return @bar // return instance variable "bar"
  end
 
  // setter, this also is a method.
  def bar=(new_value)
    @bar = new_value
  end
end
 
- or -
 
class Foo
  attr_accessor :bar
end
 
a = Foo.new
a.bar = 3
a.bar //=> 3


attr_accessor changes the class: it defines new methods. Yes, you can write your own attr_accessor in Ruby.

class Post < ActiveRecord::Base
  has_many :comments
end


Same story.

If you've never really used a dynamic language you can't imagine any advantages.

Michael Feathers

Posts: 448
Nickname: mfeathers
Registered: Jul, 2003

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 27, 2006 7:06 AM
Reply to this message Reply
> I had the same question but from a slightly different
> angle. These sorts of questions have come up before and
> where I think some of these more powerful constructs cause
> real problems is when 'large' is used to describe the team
> or organization that is working on the project.
>
> I think if the project is large only in the sense that it
> has a lot of files and/or LOC but the team that is working
> on the project is small, say 10 people or less (I wanted
> to say 6 initially, but that seems overly constraining for
> some reason) then I don't think you run into problems. If
> you have a few people and they are all on the same page as
> far as the tools being used, conventions for coding, when
> to use certain constructs and when not to, then I don't
> think you have a problem. After all, just because you can
> do something doesn't mean you should do something and a
> small group of sharp people should be able to figure out
> where that line is.

I don't think it's even just size; it's team cohesion. We shouldn't expect to get cohesive code without a cohesive team. A team needs to be able to settle its own conventions and adhere to them. If it can't, development will go off the skids regardless of any other choice you make, including language.

Jules Jacobs

Posts: 119
Nickname: jules2
Registered: Mar, 2006

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 27, 2006 9:35 AM
Reply to this message Reply
That's true. I'm currently creating an application (PHP) with someone who doesn't care about code quality. If it works, it's ok. He doesn't even like it if I refactor his code. The result is that we now have a big mess.

Andrew Norris

Posts: 8
Nickname: wanorris
Registered: Aug, 2006

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 27, 2006 11:55 AM
Reply to this message Reply
> It seems logical to me that all these things in static
> languages that slow individuals down may help groups work
> together more effectively. But I just don't know because I'm
> not willing to take a chance building a large project on a
> dynamic language unless I'm sure ahead of time.

Groovy.

No one will ever force you to do anything differently, at least up to the point where your employer (or client) decides to go in a different direction and throws you under the bus. No one forced Cobol programmers to do anything differently until they started to not be able to find jobs (other than the ones hired to maintain all the old Cobol code).

Java is more productive than Cobol. By this, what I mean is that if you hand two teams of equal baseline skill a set of requirements -- one with Java, one with Cobol -- and each team can figure out how to work together effectively, the team using Java is going to get more done. It's going to have a better ROI, etc.

My claim, for which I'm not going to try to provide direct evidence here, is that by the same standard, Ruby is more productive than Java. *If* you can find a way to get a team to work effectively together, you will get more done in Ruby.

Your code base will be smaller and more flexible, so that when management changes direction and forces you to gut a whole bunch of requirements that everyone told you were set in stone, it won't be like trying to navigate through quicksand.

Your team members, once they know the language and tools, will get more done in a day when they output the same number of lines of code.

By virtue of being more concise, well-written code will be easier to understand to newcomers, because there will be less constructs to understand.

Because compilation goes away, the development cycle (write, build, test, repeat) will be reduced and your developers will be less frustrated by theor environment.

So that's why you should care about languages like Ruby: they potentially increase productivity, ROI, and all the other things that make your team a valuable asset to your employer (or clients).

The $64,000 question is from my qualification above: "*If* you can find a way to get a team to work effectively together, you will get more done in Ruby." So can you get a team to work effectively in Ruby?

How about this, instead: as a compromise, try Python. It's nearly as powerful as Ruby (though Python advocates will presumably say it is as good or better), it looks a lot more like Java, and the language comes with a substantial prebuilt consensus on what well-coded Python looks like, so you don't have to go to the trouble of writing 100 pages of coding standards to keep rogue coders from writing code that looks like obfuscated Perl. You just teach people the language, then enforce best practices that someone else has already written.

It *is* dynamically typed, and you are going to have to write unit tests. Think of unit tests as part of the overhead of doing software engineering in a dynamic language. Even with the overhead of unit tests, there is a significant net productivity win, and you will catch other bugs in unit testing that you wouldn't have caught until later.

Now maybe you've read all this and you still think that it's not really important for you, personally, to start learning dynamic languages. After all, there are a lot more Java jobs in the world, and you likely have a massive base of pre-existing code in Java. Fair enough.

But remember: Cobol programmers had all the same reasons to not bother to move to newer languages. The technology industry thrives on evolving new technologies to improve on old ones. It you let yourself stagnate, you run the risk of becoming obsolete.

Kay Schluehr

Posts: 302
Nickname: schluehk
Registered: Jan, 2005

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 27, 2006 11:57 AM
Reply to this message Reply
> > Large projects are often a lot smaller in Ruby.
> >
> See? Dynamic languages don't scale to large projects! The
> code is too crisp and concise to allow the project to get
> very big.

Bill, since you seem to show real interest in this matters why don't you attend to a sprint? I know that Pythonistas sprint many times a year on quite some projects - in particular before conferences. Attendees of the PyPy project ( which is quite large and complex and who welcomes PyPy newbies on most of its sprints ) sprint in a six weeks frequency. I guess Rubys are sprinting often as well. So it's up to your own taste and topic preference. It would be interesting to read about your experiences as a "dynamic language" sceptics.

Andrew Norris

Posts: 8
Nickname: wanorris
Registered: Aug, 2006

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 27, 2006 12:27 PM
Reply to this message Reply
> i'm not "afraid" of large projects with dynamic classes, i
> just see it as impractical most likely eventually pointless.

> everytime i see one of these ruby vs. java arguments, i
> think of people claiming "see, working with mud is SO much
> easier! it's moldable and hardens! why do you waste time
> with brick! haha, you fool".

> i will always see it as working to build mud huts vs.
> knowing how to make buildings. i've enjoyed the mud too,
> trust me.

You know, I seem to remember C++ programmers complaining that garbage collection was similar fools' gold, with similarly clever analogies.

"Delegating memory management to a virtual machine that you can't manage directly is great for second-raters who don't want to do real work themselves, but it's not for anyone who wants to build a *real* application, because you can always manage your own system better than some *subsystem* can.

"It's like outsourcing key parts of your operation to a team of consultant that you have no direct control of. Sure, some people might be willing to do that. But *I'll* feel a lot more confident knowing we took the time to do the work ourselves, and we did it right."


BTW, I don't personally hate static languages at all, and there are situations where they are ideal. But I don't think they can be dismissed out of hand without careful study. If nothing else, you may learn new techniques from different programming idioms that will actually improve your code in your primary language.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 27, 2006 4:58 PM
Reply to this message Reply
> > > Large projects are often a lot smaller in Ruby.
> > >
> > See? Dynamic languages don't scale to large projects!
> The
> > code is too crisp and concise to allow the project to
> get
> > very big.
>
> Bill, since you seem to show real interest in this matters
> why don't you attend to a sprint? I know that Pythonistas
> sprint many times a year on quite some projects - in
> particular before conferences. Attendees of the PyPy
> project ( which is quite large and complex and who
> welcomes PyPy newbies on most of its sprints ) sprint in a
> six weeks frequency. I guess Rubys are sprinting often as
> well. So it's up to your own taste and topic preference.
> It would be interesting to read about your experiences as
> a "dynamic language" sceptics.
>
That's a good idea. I'll see if I can do that. I think there are some around here, because I'm not far from Google.

What I really care about is increasing producitivity, and dynamic languages definitely seem to do that in the small, though usually with some performance cost at runtime.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 27, 2006 5:13 PM
Reply to this message Reply
> My claim, for which I'm not going to try to provide direct
> evidence here, is that by the same standard, Ruby is more
> productive than Java. *If* you can find a way to get a
> team to work effectively together, you will get more done
> in Ruby.
>
I believe your claim for small projects. I'm not sure for large ones, though, because I don't have any experience doing that.

> Now maybe you've read all this and you still think that
> it's not really important for you, personally, to start
> learning dynamic languages. After all, there are a lot
> more Java jobs in the world, and you likely have a massive
> base of pre-existing code in Java. Fair enough.
>
I have used both Python and Ruby. Artima has a Python script that parses an Apache log file and adds stuff to a database every night, and one or two other small Python scripts running in production. I have also used Python a lot to help me do little jobs, little one-off scripts. And we have a Ruby script that does part of our build. What I haven't done is a big project in Python or Ruby, just little things.

But that said, it isn't just productivity. The productivity increase that I see in small projects I do believe has a performance cost. So I use such tools when a performance hit doesn't matter. Like the nightly batch job. If it takes 10 minutes instead of 5 minutes, it doesn't really matter.

There's also the problem of less tool support. Java just has much better tool support, and that matters. APIs too. Java has amazing APIs. So it isn't a single dimension decision based solely on productivity.

Peter Booth

Posts: 62
Nickname: alohashirt
Registered: Aug, 2004

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 27, 2006 5:20 PM
Reply to this message Reply
Please don't misunderstand me. I'm not suggesting anything about you or anyone I don't know.

Personally, I recall how I was initially dismissive of Unix as a VMS guy, then dismissive of C as a Fortran guy, and dismissive of OO as a C/Fortran guy. In each case my "rational" argumenents turned out to be well-intentioned rationalizations that I discarded when my experience contradicted them. I downloaded Ruby five years ago and put it aside saying "closures, WTF? how come there's no decent debugger?" I attribute all of this resistance to to fear of change, and I am someone who typically sits two notches behind the bleeding edge.

Regarding bricks vs mud, I can remember when the servlet API was released I heard Jim Waldo speak and vowed then and there to ditch my job and get a Java programming job. That was hard to do but I did it imbued with optimism "with this amazing language I can write clean, multithreaded networked code that runs on any machine w/o crazy pointer crashes with good tools. Programming will be so easy in a few year's time and we'll be able to tune code so much better." At that time I was working on a C++ system that did X that was 60kloc of confusing, acceptably efficient code. Skip seven years and I was working on a similar system written in 600kloc of less confusing, verbose, code that supported an identical workload on a system that was about 100x the CPU power as the prior system. What happened to the dream?

I am not dissing Java (which I love) or people who are not interested in Python/Ruby/Nice/Scala/pick your next gen language. I am saying that the question, as posed, is as much a red herring as "can interpeted Java ever be fast enough for a real business system?"

Hype, skepticism, fear of change, rationalizing by smart people, revolutionary change, have always been part of software development and probably always will be. Recognizing this and how it affects you is 100 times more important than whether Ruby's Open Classes impact Large Projcts.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: Are Ruby's Open Classes a Poor Fit for Large Projects? Posted: Aug 27, 2006 5:20 PM
Reply to this message Reply
> 1. How many technologies should a company invest in to
> deliver the same kind of projects? Is it a good long-term
> strategy to adopt a solution for small/medium projects and
> another for big/mega projects? If you're hiring a junior
> programmer you must take into account a period of time for
> training, that indeed would become two. Still, the time a
> programmer spends to get started in two programming
> languages could be spent to become expert in one only.
>
I think so, because programmers need to be able to grab whichever tool is the best one for the job at hand. I wrote about this here:

http://www.artima.com/commentary/langtool.html

I think an organization should really pick these languages and establish standards and such, build expertise in the chosen technologies.

Flat View: This topic has 51 replies on 4 pages [ « | 1  2  3  4 | » ]
Topic: The Death of the General-Purpose IDE and the Last Java Developer Previous Topic   Next Topic Topic: JOTM Transactions in Spring and Hibernate

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use