> There is a theme that I am reading in many posts here that > seems something like "metrics are imperfect and flawed > therefore they are useless." I suspect that this idea says > much more about the speaker than metrics per se. Many > people appear to be drawn to software development because > of comfort with black and white, clear specific realities. > The irony is that software development embraces a number > of chaotic non-predictable processes and is much less > black and white than people think. > > Yet - how do you compare systems without metrics. A recent > conversation: > > Me: SO can give me an estimate of how large system X is?" > DEV: It's large > ME: How big is large? > DEV: Um I don't know exactly > ME: order of magnitude? Are we talking about thousands, > tens of thousands, even 100s thousands of classes? > > I don't understand how someone can work on a system > without knowing its size, and where the size is increasing > or decreasing.
In the interest of being a pedantic shmuck, if my system has no classes, is it then small and never growing?
Your token DEV's response should have been to ask "large how?". Lines of code? Cyclomatic complexity? Class count? Do you want just the classes we wrote or all classes that might be called by the program because it runs on the .NET framework? If the system consists of components in a variety of languages, how do you count the parts that may not have any classes?
What does 'large' in and of itself tell you anyway? I currently work on some large systems by any measure that I don't worry too much about the actual size because the pieces are well thought out and pretty well put together and updates and changes are pretty easy. I've worked on small systems (again, by just about any measure) that have made me want to cry because the were fragile and horrible to update.
I think most people's issue with metrics is that they attempt to take something that is, as you say, chaotic, and reduce to something very black and white. I don't get how you can take something that has been worked on for months or years, run it through some sort of crank and get a number and have it have any real meaning. I think most people imagine the following exchange when it comes to the use of metrics:
Manager: What is the WizzleWub count of the DungBomb project?
Manager: We were hoping for at least 19. I need to see you in my office...
I can only speak for myself, but my issue with metrics isn't whether they are flawed, but when they are used by people that don't really know what they represent to make some absolute evaluation of a system. That leads to trouble in most cases. And then instead of making the system better (by fixing problems, adding features, etc.) you are more worried about bringing the WizzleWub count up.
The ultimate measure of any software system is does it do what it was intended to do. Unfortunately there isn't any single number or metric that will tell you that.