> I think the biggest problem with software metrics is that > we don't have any. ... > > It takes years of work to develop valid measurement > systems. We are impatient. In our impatience, we too often > fund people (some of them charlatans) who push unvalidated > tools instead of investing in longer term research that > might provide much more useful answers in the future.
Cem,
I believe that software metrics can, and often are, abuse, misused, overused, confused, etc. And that A LOT more work, experiments, and research is needed to improve the current state of affairs. Not to mention educating people in the proper ways to use (or not use) metrics.
But if I interpret your post (and position) correctly, the only conclusion I can draw is that - as of today - we should not be using ANY metric. Zero, nada, nyet. Is that the case? Should we burn any and all metrics tool? Remove all code coverage from our IDEs?
Are you suggesting a full moratorium on all metrics until we have invested a few years in "longer term research that might provide much more useful answers in the future."?
And, given that the overwhelming majority of metrics in existence today (all of which are invalid in your opinion) already come from researchers and academics, how can we make sure that THIS TIME we fund the right researchers and academics?
When you look at the body of work in software metrics, you see a bunch of charlatans and incompetent theorists, and consultants who are trying to swindle a bunch of clueless managers who, in turn, are going to abuse their poor programmers with those metrics. I see it a little differently...
There may be some bad apples (as in any field). But for the most part, I see a bunch of people, many of them very smart, who are motivated by a deep desire to understand and improve the way we design, write and test software. This is a very difficult task, made considerably more arduous by the constantly changing environment (i.e. every few years there are new programming models, languages, styles, etc.) Most of these people are smart enough to realize, and make it clear, that the metrics they are proposing and experimenting with are nowhere near perfect and that no single metric (or even a set of metrics) can tell the whole story. But that does not stop them from experimenting and using those metrics to learn more about them and, frankly, how else are you going to learn more and improve something if you don't experiment with it.
>When we put defective tools in the hands of executives >and managers, it's like putting a loaded gun in the >hands of a three-year old and later saying, "guns >don't kill people, people kill people." By all means, >blame the victim.
I find this attitude toward executives and managers surprisingly insulting, patronizing, and a gross over generalization. There are, for certain, some Dilbertesque managers and executives who will misunderstand and misuse metrics (that's the group that my YouTube video on "Metrics-Based Software Management" pokes fun at). But, based on my experience, most of them have enough sense to see metrics for what they are: a tool that, properly used will give them and their team some valuable (if not complete, perfect, or infallible) insight.
I have a hard time believing that you would hold such extreme positions; but I re-read your post several times and the only conclusion I can draw is that in your view:
i) As of today, there are ZERO metrics that meet your standard/definition for construct validity.
ii) Putting invalid metrics in the hands of managers and executives is like putting guns in the hands of three-year olds (who will then aim them at innocent developers).
iii) Therefore, we should not use ANY software metrics AT ALL until a group of enlightened researchers (which will probably exclude all the charlatans and incompetent nincompoops responsible for the current crop of metrics) has had sufficient time to perform experiments in a protected environment and might come up with some metrics safe for general use sometime in the future.
Is that right?
Alberto
P.S. Cem, while we might hold different opinions on how to improve/fix the state of software metrics, I believe we share several common goals. I have enormous respect for you , your work, and your passion for software quality and testing (which we share.) Not to mention the fact that I really like you on a personal level :-). I hope that this post is interpreted in the spirit in which it was written (i.e. a true desire to confirm my understanding of your position, not poke fun at it) and that we can continue this discussion in a constructive way that will help us (and the readers) gain a better understanding of different positions.
Flat View: This topic has 50 replies
on 51 pages
[
«
|
4445464748495051
|
»
]
Previous Topic
