Re: Software Development Has Stalled
Posted: Feb 11, 2010 12:18 PM
> > I think we've discovered why Software Development has
> > stalled. I present exhibit A, this discussion thread.
> > it interesting? Sure. In a purely theoretical way that
> > little impact on how people use, or even write,
> > day to day.
> > The discussion has taken on a life of its own. There is
> > lot of talk about data representation (is a spreadsheet
> > really just a collection of cells?), code vs. data (I'm
> > surprised nobody has yet mentioned prolog and its ilk.
> > Maybe they have and I've just glanced over it?),
> > 'scriptability of systems' (I'm still not sure I can
> > explain that in plain English to the person sitting
> > from me in the office) and, of course, lisp.
> > That the discussion has gone on for 100+ posts and does
> > little to address Bruce's initial thought I think
> > his initial thought brilliantly. We developers, as a
> > group, generally latch onto the interesting part of a
> > bigger problem and dissect and analyze and deconstruct
> > reconstruct that particular piece and in many instances
> > this piece does little to advance the solution to the
> > bigger problem. Sure, we're making some progress, but
> > as much as we might otherwise make if we were to
> > occasionally take a step back and look at the bigger
> > picture.
> I don't think so.
> Bruce proposed that
> 'software development has stalled because 'Programming
> tools are no longer where the greatest potential lies',
> and my reply is
> 'software development can resume if X is done' (X refers
> to my above comments).
Bruce's thought is that development has stalled. That a lot of the big wins have been had and that we are not using the tools we already have well, never mind making BIG new advances in tools or implementation. Your 'X', from reviewing previous posts, involve decoupling, which I was taught in my introductory programming class in 1991 and references to the many things lisp has done right. So you are proposing that we move forward by going back to things that are between 19 and 50 years old. I find that particularly humorous because you were the first person to take issue with the text book definition of innovate that I found and posted :-)
And many have gone on tangents that have something to do with defining language constructs that might make things easier in some cases, but are clearly a loss in others. We already know a lot about typing and type systems and relational data stores and modeling and how the current implementations fall short of the theory proposed decades ago. And in defense of people's positions they point to material that has existed for many, many year. Move along, nothing NEW to see here.
> The question is: what do YOU propose for continuing the
> software progress.
There was no question. Bruce simply made a statement based on his opinion and observation.
But since you asked, I think a lot of the big wins we have left to gain from tools involve figuring out how to enable the majority of programmers to write robust systems that can execute as many tasks in parallel as possible. It is mind boggling to me how much computing power is utterly wasted on the average corporate network. From what I've seen, we are still scratching the surface here.
Just before Dr. Dobb's and the other CMP publications all had their print editions killed, there was a series of articles talking about programming the cell processors that power the Playstation. With the architecture you can make some tried and true algorithms really sing, but programming it was hard in a lot of cases. You need to totally rethink how you break up jobs and traverse structures with millions upon millions of nodes because of the architecture. http://www.drdobbs.com/cpp/197801624
Your average programmer doesn't have the skill to do this kind of work efficiently and the tools certainly aren't here to make this easy. Code blocks/closures are still gaining traction and only the very rudimentary elements of functional programming are starting to see the light of day in mainstream languages and get adopted by your average developer. The idea of map/reduce is deep black magic to some and if you threw them in front of scheme or erlang or haskell they would probably break down and cry before even giving it a chance and taking the time to figure out what they might be good for, why they are good and where the shortfalls are. Other than obscure syntax and lack of libraries to easily access databases. Those most people can figure out in about 10 minutes.
Other posts have led me to believe that some people have an idea that there is one magical language somewhere that would be good for all problems. I don't think that will ever be the case. Some of this can be hidden, and in fact in some cases already has been hidden, as processors get faster and memory gets more abundant and cheaper. That you can program a reasonably intricate system in python or ruby is a testament to how abundant computing resources are. 15 years ago the things people now write in these languages would have been a joke because in the average case they would have been so damn slow that they would be unusable.
I personally don't think such a language exists. I think there are one of two roads you can take.
One is to have groups of languages that are good for certain kinds of problems. And these languages will evolve over time to cherry pick features from other languages that can be useful. At this point I think my favorite example of this is D, which looks to be a solid systems language and has pulled elements from a variety of other languages to take care of a lot of mundane things that don't matter in most cases, but when they do matter you can pry the box open and tinker with them. Another is erlang. It is becoming the defacto language for people that want to write large stable parallel systems. I've run across a few good examples of online game engines and other things that people have put together in the past couple years. Very recent and things that a few years ago a single or group of 2 or 3 programmers couldn't even imagine putting together, never mind actually build and have something working.
The other road is to have a language so flexible and open that you can construct your own DSL to do your specific task very well. I think, much to many people's chagrin here, that C++ was the start of that trend. I can simply type C++ and I can already tell a few people are reflexively convulsing. Many people, even C++ experts, have made it pretty clear that the language is LARGE and nobody, not even Stroustrup himself, knows all the nooks and crannies and everything it can do. I think that tradition has been carried on by Ruby and I would point to Rails and Rake as two very fine examples of DSL's that do a bang up job of making programing in their domain much simpler than in the general case. But knowing Rails doesn't help you with Rake very much and to take advantage of all of the features of either you need to have a pretty darn good understanding of Ruby to begin with.
It's lunch time :-)