The Artima Developer Community
Sponsored Link

Weblogs Forum
How Useful Are Code Metrics?

32 replies on 3 pages. Most recent reply: May 1, 2006 12:10 PM by Chris Chedgey

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 32 replies on 3 pages [ « | 1 2 3 | » ]
Max Lybbert

Posts: 314
Nickname: mlybbert
Registered: Apr, 2005

Re: How Useful Are Code Metrics? Posted: Apr 27, 2006 2:48 PM
Reply to this message Reply
Advertisement
/* My goal is to have the classes in this package depend on nothing else in com.artima.* ... But because the package contains only concrete classes, its abstractness is 0. As a result, its distance is high, 0.78, which according to Bob Martin's paper, should be considered a warning sign of bad coupling.
*/

I'm going to have to try this myself. You're probably already aware of this, but for those following at home, the most important part of what I quoted is:

/* My goal is to have the classes in this package depend on nothing else in com.artima.*
*/

Martin's metric *can* be a warning sign, but in your case, it's exactly what you want because you're looking at a stable implementation.

Unless you want to refactor com.artima.common into base classes, have everybody call into that, and break off the implementation somewhere else, bringing in object factories to make sure you get the right implementation ...

But you've got the experience to determine if the added complexity in the code base would be worth it. Perhaps if there's a chance you'll want to have more than one factory (i.e., the implementation in com.artima.common is only one way it could have been done, *and* there's a chance you'll want one of the other ways some day), then you can consider it.

Or you can make a reasoned judgement in this case. But either way, I'm going to have to try this myself. Having a computer flag potential refactoring possibilites can make life a little easier for me. And if I feed that output into a Neural Net, ... (diabolical laugh), I might go IPO!

Bill Venners

Posts: 2248
Nickname: bv
Registered: Jan, 2002

Re: How Useful Are Code Metrics? Posted: Apr 27, 2006 2:58 PM
Reply to this message Reply
Max Lybbert wrote:

> /* My goal is to have the classes in this package depend
> on nothing else in com.artima.*
> */
>
> Martin's metric *can* be a warning sign, but in your case,
> it's exactly what you want because you're looking at a
> stable implementation.
>
> Unless you want to refactor com.artima.common into base
> classes, have everybody call into that, and break off the
> implementation somewhere else, bringing in object
> factories to make sure you get the right implementation
> ...
>
I really don't like creating interfaces and factories unless I have a requirement today for multiple implementations of the interface. Such a move just increases the surface area in the name of future flexibility. I'd rather wait for the future. I was reviewing a design that Frank did at one point, which I think had a UserManager interface and one UserManagerImpl implementation. We had been talking about maybe having a Jini service someday implement UserManager, which is the main reason he made it an interface. But I felt it was a case of premature abstraction, and so we got rid of it. We just have a concrete UserManager class now.

I suspect part of my problem with the metrics is that they value the abstraction (abstract classes, interfaces) over the simplicity (reduced surface area) of the design.

> But you've got the experience to determine if the added
> complexity in the code base would be worth it. Perhaps if
> there's a chance you'll want to have more than one factory
> (i.e., the implementation in com.artima.common is only one
> way it could have been done, *and* there's a chance you'll
> want one of the other ways some day), then you can
> consider it.
>
In my opinion, the added complexity is not worth it, and that's why I don't have much confidence in the numbers. If I don't think this distance to main sequence of .78 is bad, then why should I have confidence that a .23 is good?

Chris Chedgey

Posts: 14
Nickname: chgrs
Registered: Apr, 2006

Re: How Useful Are Code Metrics? Posted: Apr 27, 2006 3:30 PM
Reply to this message Reply
> One pattern that has emerged is that removing complexity
> of one kind sometimes moves it elsewhere.

This is so true. A classic example is that 'cyclic dependencies only matter if they're between packages' (and I'd subscribe to the view that class-level tangles of a reasonable size are ok if not unavoidable).

Problem is, if you only control package-level cyclic dependencies but don't measure complexity at the inter-class level, you just push the problem down and end up with big fat packages with tangled class-level dependencies.

Similarly, I've seen code-bases where there are orderly, acyclic dependencies at the lower levels, but a tangled mess at the top level - flatten it out and all you've got is a big tangled mess.

This is why you need to think of structure as a continuum from the code-level to the top and don't let any point of composition become too complex.

Roland Pibinger

Posts: 93
Nickname: rp123
Registered: Jan, 2006

Re: How Useful Are Code Metrics? Posted: Apr 27, 2006 3:44 PM
Reply to this message Reply
> Problem is, if you only control package-level cyclic
> dependencies but don't measure complexity at the
> inter-class level, you just push the problem down and end
> up with big fat packages with tangled class-level
> dependencies.

But the motto still is 'strong cohesion - loose coupling'!

Slava Imeshev

Posts: 114
Nickname: imeshev
Registered: Sep, 2004

Re: How Useful Are Code Metrics? Posted: Apr 27, 2006 4:02 PM
Reply to this message Reply
> > It's definitely a good idea to run such tools as a
> > part of an integration build (or a daily build if the
> > codebase is large). I've also seen settings when any of these
> > tool returning non-zero bug count would fail the build, which
> > also makes sense.
> >
> Yes. I really want to make use of all the static analysis
> that's available out there. So much is available for free,
> and then there are a number of commercial tools as well.
> I'm not sure if a non-zero bug count should stop the
> build, though, because there are usually false reports, or
> reports you want to ignore.

This issue can be addressed easily. See inline:

> Findbugs for example finds a
> bunch of bugs in the code generated by JavaCC, which is
> how we implement our code generators. Those bugs really
> are there, but they aren't mine and they aren't critical
> to my project. I can turn reporting of those bugs off,
> i.e., tell Findbugs to ignore them, but then I could do
> that for anything, so what's the point of stopping the
> build?

The normal practice is to exclude paths to generated code when calling Findbugs and PMD.

PMD and Findbugs out of the box are too strict. Fortunately both of the tools provide an ability to limit notification to cases that make sense for your environment.

The typical approach to introducing making build fail on new coding errors is this:

1. Start running the static analysis tools as a part of the automated build. Don't fail the build if there are bug reports yet.
2. Adjust detection to exclude reporting for what you don't consider bugs.
3. Fix all the bug reports.
4. Turn on failing build if there are bug reports.

Drop me a line I'll send you PMD and Findbugs configurations that are already tweaked to report only issues that generally make sense.

Regards,

Slava Imeshev
http://www.viewtier.com

Andrew GJ Fung

Posts: 4
Nickname: relgar
Registered: Oct, 2003

Re: How Useful Are Code Metrics? Posted: Apr 27, 2006 4:06 PM
Reply to this message Reply
I think of metrics like compiler warnings: important to double check. Sometimes the automated analysis is spot on, partial, and sometimes totally wrong.

I favour simple metrics, like cyclomatic complexity and coupling measures. It's easy to understand what is being measured. It's also easy to understand why excessive branching and a high number of dependencies is detrimental to software quality and testability.

Chris Chedgey

Posts: 14
Nickname: chgrs
Registered: Apr, 2006

Re: How Useful Are Code Metrics? Posted: Apr 27, 2006 4:13 PM
Reply to this message Reply
> But the motto still is 'strong cohesion - loose coupling'!

Amen, and that happens at the code level. And once you've done that, you need to organize your code into a hierarchical design and architecture. And you can do that to give a relatively simple structure or a relatively complex structure and simple is better.

Max Lybbert

Posts: 314
Nickname: mlybbert
Registered: Apr, 2005

Re: How Useful Are Code Metrics? Posted: Apr 27, 2006 6:30 PM
Reply to this message Reply
/* In my opinion, the added complexity is not worth it, and that's why I don't have much confidence in the numbers. If I don't think this distance to main sequence of .78 is bad, then why should I have confidence that a .23 is good?
*/

The only reason for confidence would be if you agree with the underlying premises: (1) that the goals of abstractability and stability worthwhile, (2) that the metric actually tells you if you're arriving at these goals, and (3) that the inputs (abstractability and stability) should be weighted equally.

It would also help if the metric were tested against large (probably open source) code bases, and the numbers reported.

I haven't used metrics in general myself, nor have I used this metric in particular. It seems (instictively) that metrics can be as valuable or as worthless as job performance reviews. That is, if you mindlessly write to the metric, you shouldn't expect it to help at all, but if you use the metric to start a discussion, it could be very helpful.

***

Somebody else compared these metrics to compiler warnings. If I could pass compiler warnings into a Neural Network ... (diabolical laughter). And, yes, I know that wouldn't really work, but it should be enough to close two rounds of funding ;-).

Jay Sachs

Posts: 30
Nickname: jaysachs
Registered: Jul, 2005

Re: How Useful Are Code Metrics? Posted: Apr 28, 2006 8:56 AM
Reply to this message Reply
> Basically, just a box for each package or package group I
> care about, and lines with arrows showing the
> dependencies. I was able to draw a graph from the output
> of JDepend, but unable to do the customization of it.

I've had reasonably good success customizing the supplied XSL which transforms JDepend XML to .DOT, and rendering in GraphViz. However, I haven't found a good tool for dynamically manipulating the resulting graphs.

Jay Sachs

Posts: 30
Nickname: jaysachs
Registered: Jul, 2005

Re: How Useful Are Code Metrics? Posted: Apr 28, 2006 9:00 AM
Reply to this message Reply
> Problem is, if you only control package-level cyclic
> dependencies but don't measure complexity at the
> inter-class level, you just push the problem down and end
> up with big fat packages with tangled class-level
> dependencies.

Actually, my experience is the opposite. In order to avoid cyclic package dependencies, I've seen developers resort to spreading classes very thinly among packages. It ends up making it very difficult to describe the role of each package. I agree, though, that it's still a case of sweeping the complexity under some other rug.

Bill Venners

Posts: 2248
Nickname: bv
Registered: Jan, 2002

Re: How Useful Are Code Metrics? Posted: Apr 28, 2006 9:39 AM
Reply to this message Reply
> I've had reasonably good success customizing the supplied
> XSL which transforms JDepend XML to .DOT, and rendering in
> GraphViz. However, I haven't found a good tool for
> dynamically manipulating the resulting graphs.
>
In what way did you customize the XSL? I got a really nice graph from GraphViz and the .DOT file, but it just had too much information. For example, I would rather see java.* and javax.* as one box.

Jay Sachs

Posts: 30
Nickname: jaysachs
Registered: Jul, 2005

Re: How Useful Are Code Metrics? Posted: Apr 28, 2006 10:50 AM
Reply to this message Reply
> In what way did you customize the XSL? I got a really nice
> graph from GraphViz and the .DOT file, but it just had too
> much information. For example, I would rather see java.*
> and javax.* as one box.

To be precise, I customized first the packages I'm interested in. My concerns aren't with external dependencies, so I filter those all out of the JDepend analysis. Not that external dependencies aren't interesting, or a possible source of issues, but I tend to focus on a projects internal structure and relationships. Perhaps that's myopic.

Anyway, it sounds like you'd like to define equivalence classes among the packages (or nodes if it's done at the GV level). That seems possible but unpleasant to do with an XSL transform. A cursory glance at Graphviz tools doesn't show anything obvious to use here.

Bill Venners

Posts: 2248
Nickname: bv
Registered: Jan, 2002

Re: How Useful Are Code Metrics? Posted: Apr 28, 2006 2:56 PM
Reply to this message Reply
> Anyway, it sounds like you'd like to define equivalence
> classes among the packages (or nodes if it's done at the
> GV level). That seems possible but unpleasant to do with
> an XSL transform. A cursory glance at Graphviz tools
> doesn't show anything obvious to use here.
>
Now that I think more about it, I would rather just take java.* and javax.* off the diagram entirely, probably, as well as some other packages like log4j and such. But I did kind of want to combine everything under com.artima.jivecoupled.* under one package. We have a rule that certain packages can't talk to jivecoupled, because that's the legacy stuff that calls into Jive, which we're replacing. I don't care so much about the structure of jivecoupled itself, but would like to see one jivecoupled box on the diagram to make sure nothing depends on it that shouldn't.

I am also planning on setting up some kind of script or tool that will enforce these package layerings by looking at the code, so it will be caught that way. But I somehow also would like to look at a dependency graph generated from the code.

Max Lybbert

Posts: 314
Nickname: mlybbert
Registered: Apr, 2005

Again, talking out of my *ss Posted: Apr 28, 2006 3:36 PM
Reply to this message Reply
/* In what way did you customize the XSL?
*/

Sounds like a job for SAX to me. But I'm sure you thought of that. Is there a reason to avoid SAX, or are you just trying to not roll your own? (which is a good thing to avoid).

Erik Engbrecht

Posts: 210
Nickname: eengbrec
Registered: Apr, 2006

Compiler Warnings and Nueral Nets Posted: Apr 28, 2006 5:43 PM
Reply to this message Reply
>Somebody else compared these metrics to compiler warnings. If I could pass compiler warnings into a Neural Network ... (diabolical laughter). And, yes, I know that wouldn't really work, but it should be enough to close two rounds of funding ;-).

Sure it would. The Nueral Net could learn to do the same thing every programmer does with compiler warnings...

...ignore them....

Flat View: This topic has 32 replies on 3 pages [ « | 1  2  3 | » ]
Topic: How Useful Are Code Metrics? Previous Topic   Next Topic Topic: Is Test-First Development an Impediment to Creative Flow?


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us