Weblogs Forum - Software Development Has Stalled

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Weblogs Forum
Software Development Has Stalled

164 replies on 11 pages. Most recent reply: Mar 28, 2010 10:20 AM by Florin Jurcovici

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 164 replies on 11 pages [ « | 1 ... 7 8 9 10 11 | » ]

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Software Development Has Stalled

Posted: Feb 11, 2010 6:59 PM

> Frankly, I think this industry would be far better off
> without people who continue to push the magical thinking
> that a tool or language or framework will somehow solve
> all the problems of software development.

Let's not forget magical development processes.

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Software Development Has Stalled

Posted: Feb 11, 2010 7:28 PM

>refocus on problem solving and what value our solutions provide

This is what I tried getting at earlier in the discussion. Software development hasn't stalled as far as tools and development processes go, we have plenty of those (and a few people have even figured out good combinations of them). What's stalled is new frontiers for computing. Over the past 15 years, there's been a rush to take existing business processes and map them directly to a computer process: if someone has been filling out a requisition form, now that form will be on a computer screen. Okay great, done. What can be done next to make an end user say, "Wow!" (and a CEO want to fund it) ?

(Personally, I've thought it would be fun to take lessons from machine learning and operations research to write software aimed at replacing CEO's, but I can't seem to get funded for it. ;-)

Jan Samohyl

Posts: 14
Nickname: asgard
Registered: Feb, 2008

Re: Software Development Has Stalled

Posted: Feb 12, 2010 12:33 AM

> But since you asked, I think a lot of the big wins we have
> left to gain from tools involve figuring out how to enable
> the majority of programmers to write robust systems that
> can execute as many tasks in parallel as possible. It is
> mind boggling to me how much computing power is utterly
> wasted on the average corporate network. From what I've
> seen, we are still scratching the surface here.

I disagree. While this is certainly an interesting problem, it is probably not much problem in practice.

Have you noticed that the software is getting slower, more memory hungry and bloated over the years? So when our predecessors wrote software with poor algorithms and no parallelism, and still have it more efficient, how did they managed that? It certainly was not lack of our ability to handle parallelism. In figuring out this problem, that sir, that is where the real efficiency of software is.

Rinie Kervel

Posts: 26
Nickname: rinie
Registered: Oct, 2005

Re: Software Development Has Stalled

Posted: Feb 12, 2010 3:08 AM

> > Yup. I think you need multiple levels of abstractions
> and
> > accept irregularities, existing programs and protocols.
> > I think a reason for stalling is that we try to use
> > objects for everything: persistent data and I/O, and
> don't
> > reuse abstractions that work but are not purely OOP.
> > So no Corba, Soap but REST and JSON, HTTP and SqLite.
>
> Does anybody else find this contradictory? The more
> irregularities and existing programs and protocols you
> decide to work with, the more difficult it is to come up
> with clean abstractions. Eventually you may end up with so
> many abstractions that you have one unique abstraction for
> each problem, which is the same as having no abstractions
> at all.
It is contradictory. The point I tried to make is that we should accept Browsers, Javascript, HTTP and SQL are a fact of life. Sure they have limitations, but that won't be solved by abstracting them to OOP with ORMs, SOAP or GWT.
It is my personal opinion that a lot of the 'stalling' is the result of modeling everything as OOP. The discussion in this thread are mostly about single programs. If you partition a problem into multiple programs, some standardized interface is necessary: map reduce scaling with cheap hardware works for Google/Amazon/Ebay/Yahoo type of operations. By getting the OOP out of the I/O you can connect java programs with mongodb import etc...

So I would settle on the abstraction that works reasonable for WAN clouds and try that first on a local application (top down) instead of pushing a OOP protocol like corba, soap or ESB to multiple sites (bottom up).

> Raise your hand if you've ever worked on something that is
> so overly abstracted that the abstractions can't be
> applied to any other project you have worked on, are
> working on or will ever work on. My hand is high in the
> air. Anybody else?
Sure, that is how I view a lot of the AJAX, Java and .NET frameworks operate. YQL may be simple but not overly abstracted

Rinie Kervel

Posts: 26
Nickname: rinie
Registered: Oct, 2005

Re: Software Development Has Stalled

Posted: Feb 12, 2010 4:33 AM

It does not solve all, but gmail feels more responsive than outlook or itunes imho because it does not try to be GUI and datastore in one program.

> you describe to my display (tv, computer monitor, etc.) at
> a rate of greater than 30 frames per second.
Again Youtube does some scaling (not perfect I know) but trying to keep the realtime critical local (video stream) and the rest cloud/caching...
> The kinds of
> things the PlayStation does rely very heavily on the
> Playstation multiprocessing, real time graphics etc
True they are not covered in my approach.
But reading the data from the blu-ray could be 'clouded' with a local cache...
So i am just stating that some progress could be made if you solve things at different latency levels and different programming languages.

> MapReduce you so haughtily trot out (I mention it in the
> paragraph following the one you quote, too, it's not new
> to me) had Google file the patent in 2004 and that itself
> refers to a patent document filed in 1975. Yes, MapReduce
> is, in some form at least, older than Yoda.
Idea is not new, implementation and 'google petabytes scale is' often implementation is as important as idea.
>
> As if
> grid file systems and distributed databases are the
> solution to every problem. In some ways they seem to be
> solutions looking for a problem. I see them recommended a
> lot, but used in practice very seldom.>
> If it was so great, why
> isn't that the datastore used for something like amazon or
> yahoo or google?
...
> I don't know how somebody
> with as much data as amazon or yahoo stores their data and
> has it quickly accessible.
Hadoop is the open source version of googles approach.
Production usage at Yahoo and Facebook
http://wiki.apache.org/hadoop/PoweredBy

>
> And how is a distributed database and HTTP anything going
> to help with my on board guidance system and heads up
> display in my fighter jet? Or with the imaging on the
> medical equipment that will show me the insides of my
> heart or lungs in real time? Or any of probably hundreds
> of other problems that don't fit the model you propose?
Realtime is a problem but again partitioning the critical parts in playstation and remote data (your figther yet also gets a lot of remote data) makes the critical pieces smaller.
> ...But the concept is the same. I'm pointing
> my dumb terminal (web browser) at some piece of big iron
No big iron but Cluster of unreliable commodity hardware.
That's why I think it is innovation. Combined with there flexibility that uses the capacity of your terminal for web workers, WebGL etc...

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: Software Development Has Stalled

Posted: Feb 12, 2010 8:18 AM

> if someone has been filling out a requisition
> form, now that form will be on a computer screen. Okay
> great, done. What can be done next to make an end user
> say, "Wow!" (and a CEO want to fund it) ?

A couple of points flow (forgive the pun) from this:
- in commercial/business applications, unless and until there is something, semantically, to replace that requisition form, all we're doing is scribbling around the edges. Even Apple, for all its alleged innovation, is still implementing the PARC interface. How old is that? About 1980, depending on how you measure. Consider the names of our standard widgets: list, button, radio button, check box, tab, menu, and so on. All just pixelated analogs of paper, pencil, and mechanical toys. It's still just a paper form asking for some stuff. We've not really innovated, just adapted. The the light pen/screen, and mouse, that was innovation. But that requisition form had already been adapted to the character mode VT-100 (and some would argue in a superior implementation).

- replace forms, with what? We need truly new hardware that does more than copy-cat mechanical devices on a screen. Multi-touch and motion sensing screens are marginal changes. Or do we? It just could be that the requisition form really is the best vehicle for asking for some stuff. Limiting ourselves to existing hardware, how can we make it easier? My contention has always been that GUI's are maximally useful (or even minimally useful, when I'm in a bad mood) only when you never touch the keyboard: all input is picked with the mouse/whatever. In order to do that, we need to present all input in limited options. In order to do that, we need to disaggregate data structures. You see where this is headed? BCNF databases; narrow "records", easily displayed (list, buttons, boxes) and touched. But that flies in the face of flat-file zealots from xml-land. With increasing density of screens, it becomes easier to do, if the coders would only be compelled to let go of datastore specification.

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Software Development Has Stalled

Posted: Feb 12, 2010 12:12 PM

> A couple of points flow (forgive the pun) from this:
> - in commercial/business applications, unless and until
> there is something, semantically, to replace that
> requisition form, all we're doing is scribbling around the
> edges. Even Apple, for all its alleged innovation, is
> still implementing the PARC interface. How old is that?
> About 1980, depending on how you measure. Consider the
> e names of our standard widgets: list, button, radio
> button, check box, tab, menu, and so on. All just
> pixelated analogs of paper, pencil, and mechanical toys.
> It's still just a paper form asking for some stuff.
> . We've not really innovated, just adapted. The the
> light pen/screen, and mouse, that was innovation. But
> that requisition form had already been adapted to the
> character mode VT-100 (and some would argue in a superior
> implementation).

Agreed. Essentially, the invariant in this scenario is the data - set of data elements to be specific - while the variants are the persistence and transport mechanisms. To continue my dream of being a broken record, company leaders already have digital persistence and transport. What can we now entice them with?

> - replace forms, with what?

This question is critical to ask.

> It just could be that the
> requisition form really is the best vehicle for asking for
> some stuff.

If by requisition form you mean the data elements collected that are necessary for decision-making on a potential purchase order, then absolutely yes.

> Limiting ourselves to existing hardware, how
> can we make it easier?

Bingo! The question is, what can we make easier? Electronic transport and persistence of the "form" are already implemented.

> My contention has always been that
> GUI's are maximally useful (or even minimally useful, when
> I'm in a bad mood) only when you never touch the keyboard:
> all input is picked with the mouse/whatever.

I'm of the same mind.

> In order to
> o do that, we need to present all input in limited
> options. In order to do that, we need to disaggregate
> data structures. You see where this is headed? BCNF
> databases; narrow "records", easily displayed (list,
> buttons, boxes) and touched. But that flies in the face
> of flat-file zealots from xml-land. With increasing
> density of screens, it becomes easier to do, if the coders
> would only be compelled to let go of datastore
> specification.

I'm extremely comfortable thinking in sets. With XML, I'm still experimenting to see where its potential is.

mandeep bhatia

Posts: 1
Nickname: mandeep
Registered: Feb, 2010

Re: Software Development Has Stalled

Posted: Feb 12, 2010 2:17 PM

Dear Mr. Bruce,
I think that there could be lot more innovation in programming languages if people could maintain the fun factor or improve that. I started with BBC Micro Basic, it was great fun and it let me do lots of stuff without delving into huge piles of manuals and books. It was intuitive and fun, it satisfied its objectives of using most of the language and defined routines to make best use you can. More fun features could be added to that, Instead people make languages more and more complex with huge number of featuers and lot of libraries implemented, which one needs to refer to to get stuff done. The simplistic approach of doing things is dying, and software being developed now deters many from trying. I do not undermine the importance of concepts in c, c++, parallel programming, GUI,Event driven programming.. but i do not agree with the way a software developer has to go today to get things implemented.. lot of creativity is thrashed right away, defeating the purpose of language.
I wish to make a simpler, better, fun system myself as I love programming and I do not want the fun to end. programming is art of solving problems, its not just putting stuff into code.
Regards,
Mandeep

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: Software Development Has Stalled

Posted: Feb 13, 2010 11:22 AM

> With XML, I'm still experimenting to see where its potential is.

I'll wait quietly here on the Group W Bench.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled

Posted: Feb 16, 2010 4:57 AM

> Yeah, but it is the differences that can kill you. If
> you've done them all, I'm sure you've worked with people
> that were somewhere between mediocre and good with
> enterprise apps that would cry if they had to follow the
> rigor of something that was shipping for the military.
> Likewise I've known people that worked on military grade
> stuff that couldn't handle the pace and flexibility needed
> to do enterprise stuff. Some folks just aren't wired that
> way.

Agreed, but these are not technical difficulties.

> Definitely interested. I'd be interested to see how your
> thoughts compare with what D has already done. While D
> certainly isn't as flexible as LISP, it was originally put
> together as a better C++ (it performs almost as well in
> most situations from the benchmarks I've seen and it keeps
> the one thing I love about C++ which is at best an
> afterthought in almost every other language and framework
> out there, and that is deterministic destruction) and it
> has some pretty solid primitives for parallel programming
> that take care of most of the issues with threads.

Yes, deterministic destruction is one of the things I like in C++.

This particular topic is a big discussion and perhaps a new thread should be opened in Artima for this.

>
> >My opinion is that we can have big new advances if we go
>
> >back to some concepts that were abandoned but seem to
> >worth revisiting.
>
> So you're saying innovation isn't all it's cracked up to
> be? Because this is much more in line with my
> view/definition of innovation that I originally put forth,
> which you beat up. Revisiting existing, abandoned concepts
> is about as far from 'new' as you can get.

I wouldn't revise my definition of innovation though; I'd stick with my idea of no innovations existing.

> At some point data needs context. badbread can be a
> string, a hex representation of an integer or a memory
> address, depending on the context and what level you are
> comfortable working at. I can understand why you want to
> do away with that sort of thing, but in some cases, it is
> important to know things like register addresses and
> memory locations. After all, somebody has to write all
> these interpreters and compilers and programming tools. In
> the average case you don't need, or even want, to know
> about these things, but when you need them, you need them.

I don't mean that current programming languages should be abandoned. They all have their use.

My thoughts are on a level above that; more specifically, about the inter-operation and co-operation of applications and platforms. The programming language plays a role in this, but definitely not the major one.

> Having a big pile of stuff without any structure or
> context is good in some cases and bad in others. If I am a
> herpatologist who specializes in studying constrictors and
> google for 'python', I'll find the results to be useless.
> Information, like sound, without structure and context is
> just noise.

I am not talking about getting rid of structure and context. I am talking about having such structure and context that makes the co-operation of applications and platforms possible.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled

Posted: Feb 16, 2010 4:58 AM

> > > > You can still have transactions in undisciplined
> > data.
> > >
> > > No, you can't. I'll leave it that.
> >
> > Yes, you can. The concept of transaction is about
> 'either
> > all succeeds, or nothing succeeds'.
>
> There is a tad more to it than that, in practice. What
> matters is the implementation, and therein lies the
> discipline: the transaction manager. Again, if you want
> to read the algebra involved, get W&V; if just the
> concepts then G&R.

Can you please give me an example where a transaction is not possible if the data are undisciplined?

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled

Posted: Feb 16, 2010 5:05 AM

> What I am saying is that we need to leave behind the idea
> that if we can just get everyone to use <i>X</i> or follow
> the <i>Y</i> paradigm, all our issues would evaporate.
>
> This would be a big move forward on two fronts. 1. On the
> tool user side, we would refocus on problem solving and
> what value our solutions provide instead of what
> technologies we are using. 2. Tool builders would have
> to accept that while what they create will have a lot of
> value, it won't, by itself, be enough. In other words,
> the ability to be composable with other solutions will be
> a major requirement.
>
> I think we are already seeing this shift and there are
> already many people on the maker and user side (all makers
> are also users) who have internalized this concept. What
> remains is to bring everyone else along.

But for this to happen we need a unified representation of code as data.

This problem of composability of software extends to other domains; it's not only about the problem of development toolsets.

I think that the lack of composability is one of the fundamental reasons why software development has stalled in terms of usability and features.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled

Posted: Feb 16, 2010 5:15 AM

> Agreed. Essentially, the invariant in this scenario is the
> data - set of data elements to be specific - while the
> variants are the persistence and transport mechanisms.

So, again, the problem is about data.

> If by requisition form you mean the data elements
> collected that are necessary for decision-making on a
> potential purchase order, then absolutely yes.

Again, the issue here is the data.

>
> > Limiting ourselves to existing hardware, how
> > can we make it easier?
>
> Bingo! The question is, what can we make easier?
> Electronic transport and persistence of the "form" are
> already implemented.

But not in a unified manner...not in a common language understood across platforms and applications.

There lies the actual problem.

> I'm extremely comfortable thinking in sets. With XML, I'm
> still experimenting to see where its potential is.

Or we can throw out sets or XML for a moment and think on a more fundamental and abstract level: the level of information.

Once we define what information is and how can it be uniformly represented across platforms and applications, the question 'sets or XML' goes away; we can then organize information as sets or trees.

The problem with the question 'sets or XML' is that sets or XML is considered the information, whereas they are not information, they are ways to organize information.

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: Software Development Has Stalled

Posted: Feb 16, 2010 11:30 AM

> Can you please give me an example where a transaction is
> not possible if the data are undisciplined?

You and I are married, and share a single bank account. With undisciplined (not under transaction control; this part is obvious) data, I can withdraw funds simultaneously as you do, thus sending the account into default. This is basic transaction management. Those who yap about "eventual consistency" and the like are full of baloney.

The discipline is imposed by the single, central transaction manager. The algebra of distributed transaction management is in W&V, while the description is in G&R. It is a non-trivial problem which has been around for a couple of decades. It is doable, but there becomes a God TM among (or in addition to) the distributed TMs.

The notion that disparate clients can update a datastore independently of a transaction manager, or what amounts to the same thing, act as each's own transaction manager, is where the database illiterate fail. Einstein did not ignore Newton.

Achilleas Margaritis

Posts: 674
Nickname: achilleas
Registered: Feb, 2005

Re: Software Development Has Stalled

Posted: Feb 23, 2010 3:06 AM

> > Can you please give me an example where a transaction
> is
> > not possible if the data are undisciplined?
>
> You and I are married, and share a single bank account.
> With undisciplined (not under transaction control; this
> s part is obvious) data, I can withdraw funds
> simultaneously as you do, thus sending the account into
> default. This is basic transaction management. Those who
> yap about "eventual consistency" and the like are full of
> baloney.
>
> The discipline is imposed by the single, central
> transaction manager. The algebra of distributed
> transaction management is in W&V, while the description is
> in G&R. It is a non-trivial problem which has been around
> for a couple of decades. It is doable, but there becomes
> a God TM among (or in addition to) the distributed TMs.
>
> The notion that disparate clients can update a datastore
> independently of a transaction manager, or what amounts to
> the same thing, act as each's own transaction manager, is
> where the database illiterate fail. Einstein did not
> ignore Newton.

We actually spoke about different things. When I say 'undisciplined data' I mean data not related to a particular schema, not data not going through a transaction manager.

It goes without saying that without transactions, you ...can't have transactions.

The real issue is that data can be used in a myriad of ways, but the strict and formal representation of data as tables and relations in an RDBMS is an obstacle to reusing the data in interesting ways.

Flat View: This topic has 164 replies on 11 pages [ « | 7 8 9 10 11 | » ]

Previous Topic

Next Topic