The Artima Developer Community
Sponsored Link

Weblogs Forum
Software Development Has Stalled

164 replies on 11 pages. Most recent reply: Mar 28, 2010 10:20 AM by Florin Jurcovici

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 164 replies on 11 pages [ « | 1 ... 5 6 7 8 9 10 11 | » ]
Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled Posted: Feb 11, 2010 3:34 AM
Reply to this message Reply
Advertisement
> Shared, disciplined data is the point of the relational
> database (and to a lesser extent, its predecessors); this
> is what Codd and Date, et al, have built. Coders
> generally want to be able to scribble willy-nilly in the
> data lump without concern for others. Again, doing so is
> the main cause of the Data Babel.

So, are you suggesting that the problem is that we need more disciplined data, instead of less disciplined data?

Actually, I think the problem is that data are more disciplined than need to be. The programmer should have been able to enter data and their description and let the computer discover the similarities, patterns and links between the data.

Rinie Kervel

Posts: 26
Nickname: rinie
Registered: Oct, 2005

Re: Software Development Has Stalled Posted: Feb 11, 2010 3:56 AM
Reply to this message Reply
> > I think we should discover the models and represent
> them
> > in the code, without changing the the view that is
> > presented in the GUI.
>
> I still don't get it. In what other way can we view a
> spreadsheet? can you give me an example?

Google gives:
http://www.j-paine.org/spreadsheet_structure_discovery.html

Our term "structure discovery" denotes the recovery of structure, such as the grouping of cells, that was intended by a spreadsheet.s author but is not explicit in the spreadsheet

What is missing in my opion from the 'A spreadsheet is a bunch of cells' modeling is this structure. If you discover and use this structure you can be much more efficient (code wise and memory usage)... Spreadsheet is a bunch of cells gives efficiency like 'A text is a bunch of lines, implemented as a linked list'

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled Posted: Feb 11, 2010 5:13 AM
Reply to this message Reply
> > > I think we should discover the models and represent
> > them
> > > in the code, without changing the the view that is
> > > presented in the GUI.
> >
> > I still don't get it. In what other way can we view a
> > spreadsheet? can you give me an example?
>
> Google gives:
> http://www.j-paine.org/spreadsheet_structure_discovery.html
>
>
> Our term "structure discovery" denotes the recovery of
> structure, such as the grouping of cells, that was
> intended by a spreadsheet.s author but is not explicit in
> the spreadsheet
>
> What is missing in my opion from the 'A spreadsheet is a
> bunch of cells' modeling is this structure. If you
> discover and use this structure you can be much more
> efficient (code wise and memory usage)... Spreadsheet is a
> bunch of cells gives efficiency like 'A text is a bunch of
> lines, implemented as a linked list'

So what you mean is that a spreadsheet may contain hidden structure since data in one cell may depend to data in another cell? and you want this structure to be available?

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Software Development Has Stalled Posted: Feb 11, 2010 6:05 AM
Reply to this message Reply
> So your question about sql row not being a collection of
> column objects is related directly to the need for a way
> to script programs and systems.

I don't see it that way, no. The question I asked is the question I asked. No more, no less.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled Posted: Feb 11, 2010 6:32 AM
Reply to this message Reply
> > So your question about sql row not being a collection
> of
> > column objects is related directly to the need for a
> way
> > to script programs and systems.
>
> I don't see it that way, no. The question I asked is the
> question I asked. No more, no less.

So, if your question "about an alternative to a spreadsheet" is not about the scriptability of systems as Kervel said, what is it about? how does it relate to the discussion?

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Software Development Has Stalled Posted: Feb 11, 2010 6:48 AM
Reply to this message Reply
> > > So your question about sql row not being a collection
> > of
> > > column objects is related directly to the need for a
> > way
> > > to script programs and systems.
> >
> > I don't see it that way, no. The question I asked is
> the
> > question I asked. No more, no less.
>
> So, if your question "about an alternative to a
> spreadsheet" is not about the scriptability of systems as
> Kervel said, what is it about? how does it relate to the
> discussion?

That's not the question I asked but it doesn't really matter now. If you are really curious about what I wrote you can always go back and look.

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: Software Development Has Stalled Posted: Feb 11, 2010 6:59 AM
Reply to this message Reply
> Actually, I think the problem is that data are more
> disciplined than need to be. The programmer should have
> been able to enter data and their description and let the
> computer discover the similarities, patterns and links
> between the data.

Then you're patient zero of the data mess. Please, read Codd and Date and Gray and the like. No offense, really; but transaction processing, as documented first by Gray and Reuter ( http://www.amazon.com/Transaction-Processing-Concepts-Techniques-Management/dp/1558601902/ref=sr_1_1?ie=UTF8&s=books&qid=1265900239&sr=1-1 ) and currently by Weikum and Vossen ( http://www.amazon.com/Transactional-Information-Systems-Algorithms-Concurrency/dp/1558605088/ref=sr_1_1?ie=UTF8&s=books&qid=1265900183&sr=1-1 ) is as non-trivial an exercise as exists in data systems. The notion that the problem can be solved by a few hackers over a weekend... No.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled Posted: Feb 11, 2010 7:01 AM
Reply to this message Reply
> > > > So your question about sql row not being a
> collection
> > > of
> > > > column objects is related directly to the need for
> a
> > > way
> > > > to script programs and systems.
> > >
> > > I don't see it that way, no. The question I asked is
> > the
> > > question I asked. No more, no less.
> >
> > So, if your question "about an alternative to a
> > spreadsheet" is not about the scriptability of systems
> as
> > Kervel said, what is it about? how does it relate to
> the
> > discussion?
>
> That's not the question I asked but it doesn't really
> matter now. If you are really curious about what I wrote
> you can always go back and look.

I am trying to understand how do you see the problem of software development having stalled, and what possible solutions would you see.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled Posted: Feb 11, 2010 7:03 AM
Reply to this message Reply
> > Actually, I think the problem is that data are more
> > disciplined than need to be. The programmer should have
> > been able to enter data and their description and let
> the
> > computer discover the similarities, patterns and links
> > between the data.
>
> Then you're patient zero of the data mess. Please, read
> Codd and Date and Gray and the like. No offense, really;
> but transaction processing, as documented first by Gray
> and Reuter (
> http://www.amazon.com/Transaction-Processing-Concepts-Techn
> iques-Management/dp/1558601902/ref=sr_1_1?ie=UTF8&s=books&q
> id=1265900239&sr=1-1 ) and currently by Weikum and Vossen
> (
> http://www.amazon.com/Transactional-Information-Systems-Alg
> orithms-Concurrency/dp/1558605088/ref=sr_1_1?ie=UTF8&s=book
> s&qid=1265900183&sr=1-1 ) is as non-trivial an exercise as
> exists in data systems. The notion that the problem can
> be solved by a few hackers over a weekend... No.

I don't see what transactions and hackers working over a weekend have to do with what I said. You can still have transactions in undisciplined data.

Merriodoc Brandybuck

Posts: 225
Nickname: brandybuck
Registered: Mar, 2003

Re: Software Development Has Stalled Posted: Feb 11, 2010 7:08 AM
Reply to this message Reply
I think we've discovered why Software Development has stalled. I present exhibit A, this discussion thread. Is it interesting? Sure. In a purely theoretical way that has little impact on how people use, or even write, software day to day.

The discussion has taken on a life of its own. There is a lot of talk about data representation (is a spreadsheet really just a collection of cells?), code vs. data (I'm surprised nobody has yet mentioned prolog and its ilk. Maybe they have and I've just glanced over it?), 'scriptability of systems' (I'm still not sure I can explain that in plain English to the person sitting across from me in the office) and, of course, lisp.

That the discussion has gone on for 100+ posts and does little to address Bruce's initial thought I think proves his initial thought brilliantly. We developers, as a group, generally latch onto the interesting part of a bigger problem and dissect and analyze and deconstruct and reconstruct that particular piece and in many instances this piece does little to advance the solution to the bigger problem. Sure, we're making some progress, but not as much as we might otherwise make if we were to occasionally take a step back and look at the bigger picture.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled Posted: Feb 11, 2010 8:22 AM
Reply to this message Reply
> I think we've discovered why Software Development has
> stalled. I present exhibit A, this discussion thread. Is
> it interesting? Sure. In a purely theoretical way that has
> little impact on how people use, or even write, software
> day to day.
>
> The discussion has taken on a life of its own. There is a
> lot of talk about data representation (is a spreadsheet
> really just a collection of cells?), code vs. data (I'm
> surprised nobody has yet mentioned prolog and its ilk.
> Maybe they have and I've just glanced over it?),
> 'scriptability of systems' (I'm still not sure I can
> explain that in plain English to the person sitting across
> from me in the office) and, of course, lisp.
>
> That the discussion has gone on for 100+ posts and does
> little to address Bruce's initial thought I think proves
> his initial thought brilliantly. We developers, as a
> group, generally latch onto the interesting part of a
> bigger problem and dissect and analyze and deconstruct and
> reconstruct that particular piece and in many instances
> this piece does little to advance the solution to the
> bigger problem. Sure, we're making some progress, but not
> as much as we might otherwise make if we were to
> occasionally take a step back and look at the bigger
> picture.

I don't think so.

Bruce proposed that

'software development has stalled because 'Programming tools are no longer where the greatest potential lies',

and my reply is

'software development can resume if X is done' (X refers to my above comments).

The question is: what do YOU propose for continuing the software progress.

Merriodoc Brandybuck

Posts: 225
Nickname: brandybuck
Registered: Mar, 2003

Re: Software Development Has Stalled Posted: Feb 11, 2010 9:18 AM
Reply to this message Reply
> > I think we've discovered why Software Development has
> > stalled. I present exhibit A, this discussion thread.
> Is
> > it interesting? Sure. In a purely theoretical way that
> has
> > little impact on how people use, or even write,
> software
> > day to day.
> >
> > The discussion has taken on a life of its own. There is
> a
> > lot of talk about data representation (is a spreadsheet
> > really just a collection of cells?), code vs. data (I'm
> > surprised nobody has yet mentioned prolog and its ilk.
> > Maybe they have and I've just glanced over it?),
> > 'scriptability of systems' (I'm still not sure I can
> > explain that in plain English to the person sitting
> across
> > from me in the office) and, of course, lisp.
> >
> > That the discussion has gone on for 100+ posts and does
> > little to address Bruce's initial thought I think
> proves
> > his initial thought brilliantly. We developers, as a
> > group, generally latch onto the interesting part of a
> > bigger problem and dissect and analyze and deconstruct
> and
> > reconstruct that particular piece and in many instances
> > this piece does little to advance the solution to the
> > bigger problem. Sure, we're making some progress, but
> not
> > as much as we might otherwise make if we were to
> > occasionally take a step back and look at the bigger
> > picture.
>
> I don't think so.
>
> Bruce proposed that
>
> 'software development has stalled because 'Programming
> tools are no longer where the greatest potential lies',
>
> and my reply is
>
> 'software development can resume if X is done' (X refers
> to my above comments).
>

Bruce's thought is that development has stalled. That a lot of the big wins have been had and that we are not using the tools we already have well, never mind making BIG new advances in tools or implementation. Your 'X', from reviewing previous posts, involve decoupling, which I was taught in my introductory programming class in 1991 and references to the many things lisp has done right. So you are proposing that we move forward by going back to things that are between 19 and 50 years old. I find that particularly humorous because you were the first person to take issue with the text book definition of innovate that I found and posted :-)

And many have gone on tangents that have something to do with defining language constructs that might make things easier in some cases, but are clearly a loss in others. We already know a lot about typing and type systems and relational data stores and modeling and how the current implementations fall short of the theory proposed decades ago. And in defense of people's positions they point to material that has existed for many, many year. Move along, nothing NEW to see here.

> The question is: what do YOU propose for continuing the
> software progress.

There was no question. Bruce simply made a statement based on his opinion and observation.

But since you asked, I think a lot of the big wins we have left to gain from tools involve figuring out how to enable the majority of programmers to write robust systems that can execute as many tasks in parallel as possible. It is mind boggling to me how much computing power is utterly wasted on the average corporate network. From what I've seen, we are still scratching the surface here.

Just before Dr. Dobb's and the other CMP publications all had their print editions killed, there was a series of articles talking about programming the cell processors that power the Playstation. With the architecture you can make some tried and true algorithms really sing, but programming it was hard in a lot of cases. You need to totally rethink how you break up jobs and traverse structures with millions upon millions of nodes because of the architecture. http://www.drdobbs.com/cpp/197801624

Your average programmer doesn't have the skill to do this kind of work efficiently and the tools certainly aren't here to make this easy. Code blocks/closures are still gaining traction and only the very rudimentary elements of functional programming are starting to see the light of day in mainstream languages and get adopted by your average developer. The idea of map/reduce is deep black magic to some and if you threw them in front of scheme or erlang or haskell they would probably break down and cry before even giving it a chance and taking the time to figure out what they might be good for, why they are good and where the shortfalls are. Other than obscure syntax and lack of libraries to easily access databases. Those most people can figure out in about 10 minutes.

Other posts have led me to believe that some people have an idea that there is one magical language somewhere that would be good for all problems. I don't think that will ever be the case. Some of this can be hidden, and in fact in some cases already has been hidden, as processors get faster and memory gets more abundant and cheaper. That you can program a reasonably intricate system in python or ruby is a testament to how abundant computing resources are. 15 years ago the things people now write in these languages would have been a joke because in the average case they would have been so damn slow that they would be unusable.

I personally don't think such a language exists. I think there are one of two roads you can take.

One is to have groups of languages that are good for certain kinds of problems. And these languages will evolve over time to cherry pick features from other languages that can be useful. At this point I think my favorite example of this is D, which looks to be a solid systems language and has pulled elements from a variety of other languages to take care of a lot of mundane things that don't matter in most cases, but when they do matter you can pry the box open and tinker with them. Another is erlang. It is becoming the defacto language for people that want to write large stable parallel systems. I've run across a few good examples of online game engines and other things that people have put together in the past couple years. Very recent and things that a few years ago a single or group of 2 or 3 programmers couldn't even imagine putting together, never mind actually build and have something working.

The other road is to have a language so flexible and open that you can construct your own DSL to do your specific task very well. I think, much to many people's chagrin here, that C++ was the start of that trend. I can simply type C++ and I can already tell a few people are reflexively convulsing. Many people, even C++ experts, have made it pretty clear that the language is LARGE and nobody, not even Stroustrup himself, knows all the nooks and crannies and everything it can do. I think that tradition has been carried on by Ruby and I would point to Rails and Rake as two very fine examples of DSL's that do a bang up job of making programing in their domain much simpler than in the general case. But knowing Rails doesn't help you with Rake very much and to take advantage of all of the features of either you need to have a pretty darn good understanding of Ruby to begin with.

It's lunch time :-)

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Software Development Has Stalled Posted: Feb 11, 2010 9:24 AM
Reply to this message Reply
> > That's not the question I asked but it doesn't really
> > matter now. If you are really curious about what I
> wrote
> > you can always go back and look.
>
> I am trying to understand how do you see the problem of
> software development having stalled, and what possible
> solutions would you see.

I don't think I said that I believe software development has stalled. If I did, I apologize for lying because I don't really think that.

I think software development is maturing albeit very slowly. What has stalled is constant churn of recycled ideas and proposed silver bullets.

I believe that the core skill of software programming is an intractable problem that cannot be solved by any technology. The shift from trying to find silver bullets to building tools that help people solve those problems is the reason the development tool bubble is popping.

Frankly, I think this industry would be far better off without people who continue to push the magical thinking that a tool or language or framework will somehow solve all the problems of software development.

Rinie Kervel

Posts: 26
Nickname: rinie
Registered: Oct, 2005

Re: Software Development Has Stalled Posted: Feb 11, 2010 9:31 AM
Reply to this message Reply
> Just before Dr. Dobb's and the other CMP publications all
> had their print editions killed, there was a series of
> articles talking about programming the cell processors
> that power the Playstation. With the architecture you can
> make some tried and true algorithms really sing, but
> programming it was hard in a lot of cases. You need to
> totally rethink how you break up jobs and traverse
> structures with millions upon millions of nodes because of
> the architecture. http://www.drdobbs.com/cpp/197801624
The rethinking has already been done and is called map reduce. Trouble is, that you should not solve that in a single program, but in a scalable way: IO-protocol HTTP or protocol buffers, grid file system and say CouchDB or Hadoop...

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Software Development Has Stalled Posted: Feb 11, 2010 9:34 AM
Reply to this message Reply
> Other posts have led me to believe that some people have
> an idea that there is one magical language somewhere that
> would be good for all problems. I don't think that will
> ever be the case.

I absolutely agree. This idea needs to die. It's no more rational thinking that there should be one kind of knife or one kind of hammer or one kind of automobile. If people would stop trying to find this one tool, they could focus finding the right tool-set that meets their needs and making sure the tools in it work together.

Flat View: This topic has 164 replies on 11 pages [ « | 5  6  7  8  9  10  11 | » ]
Topic: Heron Tackles the WideFinder Challenge Previous Topic   Next Topic Topic: Setting Multiple Inheritance Straight

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use