The Artima Developer Community
Sponsored Link

Weblogs Forum
Software Development Has Stalled

164 replies on 11 pages. Most recent reply: Mar 28, 2010 10:20 AM by Florin Jurcovici

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 164 replies on 11 pages [ « | 1 2 3 4 5 6 7 8 9 ... 11  | » ]
robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: Software Development Has Stalled Posted: Feb 3, 2010 10:17 AM
Reply to this message Reply
Advertisement
> The idea that there is
> a bottomless well of smart qualified developers willing to
> accept paltry wages just over the horizon is absurd.

They have no interest in "smart qualified developers". They seek to industrialize development as Henry Ford transformed automobile manufacture. One dictator, many serfs. If you're just maintaining some ancient application, whose guts only a few old guys understand, it might be lucrative.

In that company I referenced earlier, the local Mr. Big announced that starting immediately (this was about 2003), the only on-shore "resources" who would persist would be those who managed. All else would be done either off-shore/off-shore or off-shore/on-shore. In a cruel twist of fate (from Mr. Big's point of view), the coder types got slowly eliminated; however, when they tried the tactic with what are called Business Analysts (the software in question is used in various corners of the Financial Services industry, and is mostly COBOL/VSAM from before most people were born) it never worked out. The business process is so embedded in that olde code (which was initially defined by BAs decades ago and has grown like topsy since) that a Vulcan Mind Meld never happened.

A rational organization would take that as a sign that it might be time to replace that rotting, rusty bridge. But, no; the mental lock-in was viewed as too, too valuable. So arcane that nobody but a few heart attacks waiting to happen understand the business logic. But managers did get their quarterly bonuses for saving money on coders.

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Software Development Has Stalled Posted: Feb 3, 2010 11:26 AM
Reply to this message Reply
> They have no interest in "smart qualified developers".
> They seek to industrialize development as Henry Ford
> d transformed automobile manufacture. One dictator, many
> serfs. If you're just maintaining some ancient
> application, whose guts only a few old guys understand, it
> might be lucrative.

Well spoken Robert and IMO spot on.

Several questions come to mind. Is IT still a strategic advantage for organizations? If so, is it true for only certain sectors? Put another way, could most sectors simply buy vanilla vendor software, install it, and not even have developers around?

bug not

Posts: 16
Nickname: bugmenot2
Registered: May, 2005

Re: Software Development Has Stalled Posted: Feb 4, 2010 5:57 AM
Reply to this message Reply
> Personally, I think we've been caught in a rut for
> decades.
>

I agree.

I think the problem is that marketing took over. Once selling computer languages became a big business (c. 1983), languages were popularized not on their merits but with marketing skill.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Software Development Has Stalled Posted: Feb 4, 2010 7:04 AM
Reply to this message Reply
> > The idea that there is
> > a bottomless well of smart qualified developers willing
> to
> > accept paltry wages just over the horizon is absurd.
>
> They have no interest in "smart qualified developers".
> They seek to industrialize development as Henry Ford
> d transformed automobile manufacture. One dictator, many
> serfs. If you're just maintaining some ancient
> application, whose guts only a few old guys understand, it
> might be lucrative.

I totally agree that this is the mindset of a lot of people but I also believe that these people are extremely ignorant.

There are other people who have a different take in that they think they can get people who are as a talented or better in some other place for a huge discount. While that was/is true because of some quirks of history, the effect can only be temporary and the faster people rush to that other place, the sooner the imbalance will end.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Software Development Has Stalled Posted: Feb 4, 2010 7:34 AM
Reply to this message Reply
> Several questions come to mind. Is IT still a strategic
> advantage for organizations?

If you flip the question, the answer is obvious, I think:

Q. Can an organization in the 21st century have a viable strategy without IT?
A. Not in a competitive (data-driven) industry.

> If so, is it true for only
> certain sectors? Put another way, could most sectors
> simply buy vanilla vendor software, install it, and not
> even have developers around?

This has not been the case anywhere where I have worked and I've worked in several really different types of industries. A lot of companies think they can but I think they don't realize that what they lose is worth more than what they save. In these organizations you hear a lot about the 80/20 rule. Ironically enough the original idea behind this is the exact opposite. It proposes that 20% of your requirements produce 80% of the value. If you arbitrarily dismiss things that aren't supported by a vendor, you risk losing the bulk of the value of having a system. This is why these organizations are never happy with their software. There's also the loss of competitive advantage which is another discussion.

I think IT is slow to catch up with the reality that cost of custom software is dropping rapidly largely because of open-source and that the quality of custom software is increasing. On the other hand, off-the-shelf software is getting more expensive because of decreased competition and the quality (IMO) is dropping.

I don't know if every organization will need developers. But maybe that makes sense for medium to large organizations. A single talented developer can build systems that do the work of hundreds if not many thousands of traditional employees. When people talk about the high cost of application development, they must be using some sort of 'new' math.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Software Development Has Stalled Posted: Feb 4, 2010 8:06 AM
Reply to this message Reply
> I think IT is slow to catch up with the reality that cost
> of custom software is dropping rapidly largely because of
> open-source and that the quality of custom software is
> increasing. On the other hand, off-the-shelf software is
> getting more expensive because of decreased competition
> and the quality (IMO) is dropping.

I forgot to add that a lot of custom software is saddled by bad ideas about the 'right' way to do things and therefore don't show these gains.

Merriodoc Brandybuck

Posts: 225
Nickname: brandybuck
Registered: Mar, 2003

Re: Software Development Has Stalled Posted: Feb 4, 2010 8:33 AM
Reply to this message Reply
> I don't know if every organization will need developers.
> But maybe that makes sense for medium to large
> e organizations. A single talented developer can build
> systems that do the work of hundreds if not many thousands
> of traditional employees. When people talk about the high
> cost of application development, they must be using some
> sort of 'new' math.

The other thing I've seen that is impossible to quantify because you don't see it until it happens that these systems can allow people to do work that they otherwise would not have been able to do. I don't think there is a good way to account for that. So let's say your talented developer builds a system that can do the work of a couple hundred employees.

One obvious side effect of this is that those couple hundred employees simply don't have jobs anymore. The other side effect I've seen, which is more interesting in a lot of ways, is that those folks end up finding ways to do more or better work that adds a lot more value.

In my experience most of these systems that end up doing the work of many people don't just suddenly appear on a given date and immediately make people obsolete. They evolve over time. I think that is what accounts for the perception of the high cost of development in most cases. If you could say "I'm going to start on Feb 10, 2010, finish on June 22, 2011 and have a system that will do the work of 325 employees" it would be very, very simple to figure out if that system should be built based on cost. You take the salaries and benefits of the people that will be replaced, the cost of the effort to build the system and an estimate of maintenance and figure out what side of the fence the value of the system lives on. That just doesn't seem to happen in real life, though.

What happens is that you say it will be done on June 22, 2011 and it isn't really done until Mid November. So the people are still working. And the system doesn't work 100% correctly so you can't just fire everybody since you need to produce things correctly for your clients. And it's unknown how long it will take to work all the bugs out of the system. Over time, as issues are worked out, people move onto other things as the system does the more rote parts of all these people's jobs. Maybe some people are entirely replaceable, but not all of them as initially projected, so instead of getting rid of 325 employees maybe only 100 or so end up having to look for new jobs. I don't think there's a way to account for the value of what the existing employees are doing very easily, or to account for the time savings, so what you have is a system that was delivered many months late, that probably sent you quite a bit over budget and it only 'saved' you about 1/3 of what you originally projected. That's pretty much a failure by every measurable objective that you had at the outset. From this viewpoint you are telling people that a system that cost much more than you said and saved them much less than projected is a success?

I don't think that scenario is very uncommon.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Software Development Has Stalled Posted: Feb 4, 2010 8:46 AM
Reply to this message Reply
> One obvious side effect of this is that those couple
> hundred employees simply don't have jobs anymore. The
> other side effect I've seen, which is more interesting in
> a lot of ways, is that those folks end up finding ways to
> do more or better work that adds a lot more value.

In a growing company, good software allows you to do more and more without adding people. I prefer to focus on that aspect for obvious reasons.

Bad software, on the other hand, often requires adding more (expensive) people while not adding enough productivity to make up for it. In other words, you need to eliminate the need for a bunch of temps to pay for people to do all the manual massaging necessary to make your SaaS package work at the closest possible approximation to what you actually need.

A lot of the large COTS packages fall into this category. You it costs a ton to get something that's not exactly what you need. I call this buy-and-build. Pay millions for the software, pay millions more for 'experts' to customize it. It still doesn't do quite what you need but now you need to add some full-time resource to administrate and maintain it.

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Software Development Has Stalled Posted: Feb 4, 2010 8:49 AM
Reply to this message Reply
> Q. Can an organization in the 21st century have a viable
> strategy without IT?
> A. Not in a competitive (data-driven) industry.

Okay, now put yourself into the mind of a CEO for a medium to large business.

You funded strategic software projects during the rush to digitize business through the 90's to early 2K's. Every department has its own application and the applications (roughly) work together. Your business is now as digitized as its competitors. Now what? What's the remaining competitive advantage? What are you (Mr/Ms CIO) going to convince me to fund?

I am not trying to claim there is nothing left to fund. However, the rush to get all departments online in every business through the 90's and early 2K's happened already so the funding is adjusted down to meet what is necessary for competitive advantage. Which is what? (rhetorical: it has to be answered by each business according to its market)

The rest of your response I agree with and my experiences match your own. I'm not sure businesses have caught up with how irrational it is to use vendor packages - available to your competition - for competitive advantage applications.

Merriodoc Brandybuck

Posts: 225
Nickname: brandybuck
Registered: Mar, 2003

Re: Software Development Has Stalled Posted: Feb 4, 2010 9:11 AM
Reply to this message Reply
> > One obvious side effect of this is that those couple
> > hundred employees simply don't have jobs anymore. The
> > other side effect I've seen, which is more interesting
> in
> > a lot of ways, is that those folks end up finding ways
> to
> > do more or better work that adds a lot more value.
>
> In a growing company, good software allows you to do more
> and more without adding people. I prefer to focus on that
> aspect for obvious reasons.
>

The only problem with that is a LOT of work out there is in putting together systems for already large companies. It's fine to focus where you want, but the reality of the situation is that somebody somewhere has to do the work for the behemoth companies out there.

> Bad software, on the other hand, often requires adding
> more (expensive) people while not adding enough
> productivity to make up for it. In other words, you need
> to eliminate the need for a bunch of temps to pay for
> people to do all the manual massaging necessary to make
> your SaaS package work at the closest possible
> approximation to what you actually need.
>

I totally agree. But this is also tricky because a lot of times people don't know what they want or what will really save them money. Agile is supposed to solve this problem by having people on site and in constant communication about projects to address these problems, but that never happens in practice because people have actual work to do, rather than babysit the developer. So you are trading off manual messaging for the SaaS package for manual messages for a human being.

> A lot of the large COTS packages fall into this category.
> You it costs a ton to get something that's not exactly
> y what you need. I call this buy-and-build. Pay millions
> for the software, pay millions more for 'experts' to
> customize it. It still doesn't do quite what you need but
> now you need to add some full-time resource to
> administrate and maintain it.

My Dad used to work for Pitney Bowes. He saw this time and time and time again. I'm only too familiar with the stories.

In theory I completely agree with you. In practice it's a very very messy problem, especially as organizations get larger. And focusing on small organizations as you suggest only delays the problem. What happens when your small organization is swallowed up by a conglomerate that sees the insane profit and margins your small, extremely efficient company is making? Well, they have to integrate that small company into the whole. In the process they usually corrupt or completely destroy the things that made the company worth acquiring in the first place. True, at this point, it is not YOUR problem (lucky bastard.... :-) but it is somebody's problem and the advice of staying small and focused on writing good software to get more work done before you add people isn't helpful.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Software Development Has Stalled Posted: Feb 4, 2010 10:48 AM
Reply to this message Reply
> Okay, now put yourself into the mind of a CEO for a medium
> to large business.
>
> You funded strategic software projects during the rush to
> digitize business through the 90's to early 2K's. Every
> department has its own application and the applications
> (roughly) work together. Your business is now as digitized
> as its competitors. Now what? What's the remaining
> competitive advantage? What are you (Mr/Ms CIO) going to
> convince me to fund?

Well, I've never worked at a place where all the systems work together. I guess it depends on what you mean by rough. Actually, I've never worked at a place where the systems work very well at all.

The reality is that if your business isn't changing pretty much all the time, you won't be around. Changing business means changing software (in the most general sense.) There's no shortage of work to be done. The problem is that management seems to think that once they've got the software, it's done. I find that strange given that the whole point of software is that it's easy to change. I guess it stems from the fact that most contemporary software projects fail to deliver much value one day one, much less years into use.

Frankly, it seems like software is getting worse in a lot of ways. A lot of companies continue to use old green screen apps because they work. I've seen many attempts to replace these older systems completely fail to launch. It's not that they don't fund the work, it's that the money goes into a black-hole.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Software Development Has Stalled Posted: Feb 4, 2010 11:01 AM
Reply to this message Reply
> The only problem with that is a LOT of work out there is
> in putting together systems for already large companies.
> It's fine to focus where you want, but the reality of the
> situation is that somebody somewhere has to do the work
> for the behemoth companies out there.

I have to put on my "it's going to hurt but it's good for you" capitalist cheerleader hat: This is the normal cycle of economies. Yes, people will be let go but the extra efficiencies will allow for growth and new jobs. It's hard to see that in the US and Western Europe because the huge wealth imbalances in the world combined with a more global economy means most of those jobs are created elsewhere.

> I totally agree. But this is also tricky because a lot of
> times people don't know what they want or what will really
> save them money. Agile is supposed to solve this problem
> by having people on site and in constant communication
> about projects to address these problems, but that never
> happens in practice because people have actual work to do,
> rather than babysit the developer. So you are trading off
> manual messaging for the SaaS package for manual messages
> for a human being

You've hit the nail on the head. In theory, bringing in a consultant for a little while to set you up on a SaaS or big COTS package is cheaper. However, in reality, it's like saving 10% on a car with a cracked engine block. Yeah, you saved some dollars but you didn't get much value for what you spent. You'd have been better off buying nothing.

If you instead have dedicated technical advocates (that actually understand software) to determine when to buy and when to build and when to do a hybrid and can execute each kind of task, you are much better off.

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Software Development Has Stalled Posted: Feb 4, 2010 11:48 AM
Reply to this message Reply
> Well, I've never worked at a place where all the systems
> work together. I guess it depends on what you mean by
> rough. Actually, I've never worked at a place where the
> systems work very well at all.

Rough is that nightly batch job that dumps the results of a query into a text file to be picked up by a homegrown shell script and processed into another database. Every now and then the batch job gets messed up by an over long db backup and then you spend all day trying to fix the data damage done. That kind of rough. It sucks and is poorly designed but it works > 90% of the time so nobody bothers redesigning.


> The reality is that if your business isn't changing pretty
> much all the time, you won't be around.

Certainly a business needs to change around what it offers to its market. (Or how it offers to its market.) It gets back to what is your strategic advantage. For that, funding will be given. For everything else, you observed, "The problem is that management seems to think that once they've got the software, it's done". Once non-strategic portions of a business are automated, even if it's done in a crappy way, the business heads move on.

Software development will stop stagnation when we figure out how to provide new value to businesses and get funded to do it.

> Frankly, it seems like software is getting worse in a lot
> of ways. A lot of companies continue to use old green
> screen apps because they work. I've seen many attempts to
> replace these older systems completely fail to launch.
> It's not that they don't fund the work, it's that the
> e money goes into a black-hole.

I haven't seen green screen apps in places I work for at least 10 years.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Software Development Has Stalled Posted: Feb 4, 2010 12:45 PM
Reply to this message Reply
> Software development will stop stagnation when we figure
> out how to provide new value to businesses and get funded
> to do it.

It's not that we don't know how to provide value. It's that we don't know how to explain it in a way that makes sense to them. In other words, we are unable to quantify the value. For example in your case of how systems work together 'roughly', there's likely enough potential for savings to justify the work to fix it. The problem is that if you don't explain that in terms of time-value adjusted money, management won't know that. They will just see it as switching out one thing for another that does the same stuff.

It's only when you hit something really big like an inability to meet the goals of a business objective that management gets the value of good software. But by that point, IT has lost it's credibility.

> > Frankly, it seems like software is getting worse in a
> lot
> > of ways. A lot of companies continue to use old green
> > screen apps because they work. I've seen many attempts
> to
> > replace these older systems completely fail to launch.
> > It's not that they don't fund the work, it's that the
> > e money goes into a black-hole.
>
> I haven't seen green screen apps in places I work for at
> least 10 years.

They are all over. If you go to the Home Depot and order cabinet doors, they enter the orders in a terminal emulator. Likely your health insurance carrier uses at least one terminal based system. A lot of times you don't see the green screen because there are screen scraping guis that hide them. There probably a surprisingly large number of web sites are built by screen-scraping terminal-based systems. Airports are full of them in both the airlines and air traffic control. The thing is that a lot of these systems work a lot better than the gilded turds that are meant to replace them. They often lack some desirable modern features but they work.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Software Development Has Stalled Posted: Feb 4, 2010 12:53 PM
Reply to this message Reply
> Rough is that nightly batch job that dumps the results of
> a query into a text file to be picked up by a homegrown
> shell script and processed into another database. Every
> now and then the batch job gets messed up by an over long
> db backup and then you spend all day trying to fix the
> data damage done. That kind of rough. It sucks and is
> poorly designed but it works > 90% of the time so nobody
> bothers redesigning.

The other thing about these kinds of systems is that they often are losing data on a regular basis but no one notices. These kinds of issues are hard to quantify because the costs are unknown. People just work around the problems.

I once was helping someone with an issue in a production program and we finally determined that the program had never executed successfully. The contractors who wrote it were long gone and the users had continued to do everything on paper. No one in IT appeared to have any idea. As far as management was concerned, the money was well spent.

It's not that we can't provide value. It's that most management has no idea what value IT provides could provide or fails to provide. And that's the fault of the IT industry.

Flat View: This topic has 164 replies on 11 pages [ « | 1  2  3  4  5  6  7  8  9 | » ]
Topic: Heron Tackles the WideFinder Challenge Previous Topic   Next Topic Topic: Setting Multiple Inheritance Straight

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use