Weblogs Forum - Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Weblogs Forum
Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

50 replies on 4 pages. Most recent reply: May 16, 2008 1:38 PM by Thomas Cagley

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 50 replies on 4 pages [ « | 1 2 3 4 | » ]

Mike Petry

Posts: 34
Nickname: mikepetry
Registered: Apr, 2005

Re: Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Oct 31, 2007 10:08 AM

Here is a cynical thought:
Take all of your weakest programmers and put them in charge of metrics. All of a sudden your quality goes up.

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Oct 31, 2007 10:14 AM

> There probably is truth in metrics and metrics should be a
> powerful tool, but in reality there are many factors
> influencing the metrics we gather.
>
> When we want to evaluate a process using statistics we
> hope that measurable events are independent.

It's funny. I remember my first lecture in macroeconomics. The professor started the lecture by saying, "Two kinds of people use statistics - liars and politicians." After everyone had a laugh, he went on to say pretty much what you said.

Isaac Gouy

Posts: 527
Nickname: igouy
Registered: Jul, 2003

Re: Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Oct 31, 2007 10:24 AM

You write as though killing software projects is a bad thing!

Fail-fast makes it safe to innovate.

"Project Termination Doesn’t Equal Project Failure"
http://www.cs.unc.edu/~welch/class/comp145/media/docs/Boehm_Term_NE_Fail.pdf

Merriodoc Brandybuck

Posts: 225
Nickname: brandybuck
Registered: Mar, 2003

Re: Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Oct 31, 2007 2:51 PM

I think the issue with software metrics is that software is a lot more than the sum of its parts. Judging software by any one metric or a handful of metrics would be like judging a book by word count or a movie by the length of the film. There is so much more to it than any one metric or handful of metrics could measure that I believe the best thought out metrics should be nothing more than very rough guidelines.

David Halonen

Posts: 5
Nickname: dhalonen1
Registered: Oct, 2007

Re: Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Nov 1, 2007 6:01 AM

In his latest book, "Winning", Jack Welch states that business should be run like a small family store on the corner. That much process, focus and formality.

In many large, large companies, that sort of intimacy is replaced by bureaucracy. The upper crust is pretty much clueless as to the actual goings-on. So what is a poor schmuck to do?

I would offer that metrics are a useful tool to defend and grow a software group in such a domain.

I've worked in small companies and large. The problems & solutions are not identical. For example, can NASA deliver software for moon launch w/o metrics? What's the biggest team Joel has managed? 2 different universes.

Cameron Purdy

Posts: 186
Nickname: cpurdy
Registered: Dec, 2004

Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Nov 1, 2007 7:42 AM

New Study Finds that Projects Don't Kill Moronic Managers, Blunt Traumas Kill Moronic Managers.

Full Story at 11.

Alberto Savoia

Posts: 95
Nickname: agitator
Registered: Aug, 2004

Re: Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Nov 1, 2007 10:33 AM

> > Guns don't kill people, people
> > kill people
>
> And frankly that sums it up, IMHO. Avoiding a potentially
> useful tool because it can be abused/misused is not a sane
> strategy.

Perhaps we can require a 14-days waiting period and a permit to use some metrics :-).

Alberto

Mike Petry

Posts: 34
Nickname: mikepetry
Registered: Apr, 2005

Re: Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Nov 1, 2007 11:27 AM

Metrics can be hard for software folks because we are so binary. Either the finished product is cool or it sucks.

I personally believe in using proven methodologies and insist on delivering well designed, cleanly written, maintainable software. However, I have seen cases in which the most ugly software practices have delivered products that users loved. Conversely I have seen disciplined teams deliver high quality steaming piles of poo (from the users perspective).

What bothers me about metrics is that they can make things more complicated than simpler.

Do is suck or is it cool?

Vincent O'Sullivan

Posts: 724
Nickname: vincent
Registered: Nov, 2002

Re: Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Nov 2, 2007 6:15 AM

The purpose of a software metric is to make a complex thing simple so that it can be evaluated quickly. It's a compression process that reduces a piece of code to a single integer. In the same way that if a picture was reduced to a single pixel, all detail is lost in the process.

The average picture, reduced to a single pixel, would produce a pixel that is grey with a brightness of about 18%. If you take a picture and process it in that way and your result is slightly green with a brightness of 25% then what does that tell you about the quality of the original picture? Nothing.

If I run a metric against my day's work and the result is "17" and a co-worker's day's work can be summed up as "12", what does that tell anyone about the work that we've done today? How good is our code? How much of it is there? How complex was the task being coded? How old is the code? Etc., etc. All that information is not only lost but it was never embodied in the code in the first place.

The fact is that the metric - being a mechanical derivative of the code - is of no real value except to the individual developer and then only as a blunt pointer to potentially problematic code.

Manager are rarely interested in implementation details, so there little or no reason for them to be interested in a single numbers derivated from those details. Fortunately, my experience is exactly that.

Vince.

Ed Kirwan

Posts: 3
Nickname: kirwan
Registered: Nov, 2007

Re: Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Nov 2, 2007 8:25 AM

Metrics will never be 100% agreed-upon while disagreement is possible.

In other words, the only metrics that cannot be argued are those that are mathematically, provably true; software and proof, however, seldom go hand-in-hand.

You often have to jump through hoops just to prove the most basic of commonly-held beliefs:
http://www.edmundkirwan.com/encap/intro.html

Amen to Halonen's, "The lack of metrics and the ability to equate them to our business value is our own fault."

.ed

Alberto Savoia

Posts: 95
Nickname: agitator
Registered: Aug, 2004

Re: Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Nov 2, 2007 3:08 PM

> The purpose of a software metric is to make a complex
> thing simple so that it can be evaluated quickly. It's a
> compression process that reduces a piece of code to a
> single integer.
> ...
> The fact is that the metric - being a mechanical
> derivative of the code - is of no real value except to the
> individual developer and then only as a blunt pointer to
> potentially problematic code.

Hi Vincent,

I believe that with this line of thinking we are throwing away the proverbial baby with the bathwater.

Of course metrics simplify and reduce, that's what they are meant to do. It's a strength, not a weakness. Metrics can contain much more information that you give them credit for. By your line of reasoning, we should do away with most metrics and indices:

The Dow Jones Industrial Index, unemployment rate, MPG, cholesterol levels, ppm of X in Y (parts per million of X in Y - e.g. ppm of lead in kid toys), etc.

I believe that most people understand that a single number cannot tell the whole story. But a single number can be an extremely useful indicator and sometimes it's a necessity. Other than averaging stocks into some kind of index, or up/down ratio, or something else, for example, how are you going to communicate at a high-level what went on in the stock market?

Most things have properties and qualities that are of potential interest to somone. In grapes, let's say, dieters may care about calories, environmentalists may care about ppm of pesticide residue, vinemakers may care about % of sugar, and so on.

Similarly, if one believes that some code patterns are potentially risky (e.g. very complex code without any tests) and others are benefical, to me it makes a lot of sense to identify, measure, and study those those patterns.

I continue to be very surprised by some people's aversion to any and all software metrics. I love working with software. I think it's one of the best jobs in the world and I know that it involves a lot of creativity and unique problem solving. But I am not so enamored of it, or think it's such a special thing, that there are no aspects of it that can or should be measured and compared. On the contrary, I believe that our complete lack of any agreed-upon measurements and/or standards has hurt the software industry and will continue to do so.

Alberto

Peter Doubleday

Posts: 2
Nickname: aardvark
Registered: Nov, 2007

Re: Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Nov 2, 2007 5:49 PM

I look forward to a post that actually suggests some value, either empirical or philosophical, inherent in the use of software metrics. This one isn't it.

No, software metrics are not evil. Many things are evil. Software metrics are more on the lines of "pathetic." They are definitely useless -- although software planning metrics on a historical and statistical basis, a la Spolsky, may well be valuable.

We can only judge by results, because the IT industry presently has no real concept of blind tests. I would refer your readers to www.hacknot.info for an in-depth discussion on this issue. One of the points that "Ed" makes on that site, and I think it's important to the way we approach these nebulous measurements, is that we all (and I've become convinced that this applies to me, too) need to apply critical thinking in a logical way. Unfortunately, when you say:

"Better yet, the programmers would have great evidence to have the moronic manager fired."

... this is actually meaningless. First of all, you're proposing the lemma that there is, indeed, some sort of meaningful software metric, or collection thereof. (This being a lemma, it's not possible without further proof to say whether you are wrong in proposing it or the software manager is wrong in misusing it.)

Secondly, you define this as "great evidence." Even if you went forward from the lemma to the proof (QED), you'd still need quantifiable evidence based against, presumably, some sort of algorithmic expression of that proof. Which would probably fall within the domain of statistics. And would probably be questionable on the basis of methodology, even given a universal acceptance of the proof.

Thirdly, you assume that evidence of 100% failure to do one's job, as defined in the tedious legal crap that we all sign in order to do our job, is sufficient cause to be fired.

It's not.

There's no such thing as a silver bullet.

But if there was, I wouldn't reserve it for vampires -- because I <i>know</i> they don't exist. And I wouldn't reserve it for bosses, because I <i>know</i> my only control over them is to leave.

I'd reserve it for people who argue that software metrics have any conceivable value at all, without significant comparative work.

Morgan Conrad

Posts: 307
Nickname: miata71
Registered: Mar, 2006

Re: Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Nov 3, 2007 10:40 PM

Whoa Peter,

While I'm with you that today metrics may not mean much, IMO they are worth collecting as data, to determine if any are significant. We need to correlate various metrics with "did this project go well". Maybe, maybe, there is some metric or combination thereof that, for a given type of project, indicates that you are on the right track. Maybe there isn't.

But without collecting the metrics we have no chance.

Alberto Savoia

Posts: 95
Nickname: agitator
Registered: Aug, 2004

Re: Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Nov 4, 2007 7:50 AM

Peter,

Frankly, I don't see the need for such a vitriolic, and rather insulting, post. I noticed it's your first post on Artima, but still ...

We are trying to have a conversation here. If you read the original post more carefully, you'll see that it's trying solicit feedback and experiences from both sides of the metric camp so that we can learn from other people's experiences. We want to gain a better understanding of other people's position so we can revisit and adjust ours - not sling mud at each other.

> No, software metrics are not evil. Many things are evil.
> Software metrics are more on the lines of "pathetic."
> " They are definitely useless

Those are very strong words. I can see someone saying things like: "I have tried several metrics and found them not useful", or "software metrics don't really apply in my situation"; but I can't see how anyone can categorically label all metrics as 'pathetic' and 'definitively useless'.

Unless, that is, you consider yourself above all other people in the industry in terms of intelligence, experience and insight, and consider everyone who uses metrics as 'pathetic' and deluded. I, and many others, have found software metrics - when used judiciously and in the right context - to be incredibly useful; if not downright necessary in some cases.

> I'd reserve it [a silver bullet] for people who argue
> that software metrics have any conceivable value at
> all, without significant comparative work.

Thanks for the warning - I'll stay out of firing range :-).

I don't know what you would consider 'significant comparable' work; but I would argue that there has been a lot of significant and valuable research on software metrics at pretty much every major academic and industrial institution involved in software that you can name. You could probably spend the rest of your days catching up with all the work in this area.

If you took the time to check the body of research and experimentation in software metrics, you'll see that most researchers and practitioners approach the subject with a great deal of skepticism and respect. They know that metrics must be used with care. They know that, in the wrong hands, they can be OVER-used, ABused, and MISused. But they also know that properly used, software metrics can provide uniquely valuable insights.

IMO, to label the work of these people as 'pathetic' and 'definitely useless' shows not only a lack of respect, but a profound lack of understanding.

Vincent O'Sullivan

Posts: 724
Nickname: vincent
Registered: Nov, 2002

Re: Software Metrics Don't Kill Projects, Moronic Managers Kill Projects

Posted: Nov 5, 2007 3:26 AM

My argument is not that metrics are good or bad (they're just data, after all) but that they are gross simplification of the thing that they are measuring.

The question is always; does the metric provide useful information? I had to replace the fuel tank on my car recently. Knowing its volume was useful information but knowing its colour wasn't. (On the other hand; if I rode a motorbike then maybe the colour would be important, too.)

In the case of metrics that are compiled from the code; I think that a manager who has no interest in the code implementation details (because it's not something that they need to know) equally should have no interest in a metric generated from that code - whether it be line count, crappiness, complexity or whatever.

I said in my original statement that they are a blunt tool of interest only to the actual developer. I think that the 'crap' metric illustrates that. The 'crap' metric is a meta-metric in that it takes two metrics (each of which measured independent properties of the code) and combined then to produce a third figure. In doing so knowledge, of what the actual problem in the code was that was being flagged, was lost.

I have applied it to some code I wrote recently and it highlighted a number of methods that potentially warranted further attention. What it did not indicate is why those methods were 'crap'. To do that I needed to look at the two figures that were combined to created that metric. Until then I did not know if the code was too complex or too untested (or both). Fortunately, the tool you provided (for the eclipse IDE) makes that information readily available but the fact is that the original metric is just a flag that carries no information than to say "look here".

I therefore stand by my assertion that code metrics, that are mechanically generated from the code, are only of interest to the code developer and do not provide any useful information to a project manager.

Flat View: This topic has 50 replies on 4 pages [ « | 1 2 3 4 | » ]

Previous Topic

Next Topic