The Artima Developer Community
Sponsored Link

Weblogs Forum
The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-in

38 replies on 3 pages. Most recent reply: Jan 29, 2008 11:12 AM by Alberto Savoia

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 38 replies on 3 pages [ « | 1 2 3 | » ]
Antonio R. Rodríguez S.

Posts: 6
Nickname: rodant
Registered: Jan, 2005

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-i Posted: Oct 5, 2007 4:30 AM
Reply to this message Reply
Advertisement
Thanks for the contribution. I tried it out on my Eclipse 3.2.2., but it seems not to work correctly. I can only see the test coverage statistics but no CRAP or complexity numbers, any idea?

Best regards,
Antonio Rodríguez Santiesteban.

Carfield Yim

Posts: 32
Nickname: carfield
Registered: Sep, 2002

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-in Posted: Oct 5, 2007 5:41 AM
Reply to this message Reply
>
> Do the tests run with the JUnit test runner in Eclipse?
> If so, then it sounds like you have found a bug in
> crap4j.
>
Yes, it run with the Junit test runner in Eclipse, how can I provide more information for you? I've try to go to the forumn of crap4j via the link at update site, but that is not available.

Antonio R. Rodríguez S.

Posts: 6
Nickname: rodant
Registered: Jan, 2005

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-i Posted: Oct 5, 2007 6:57 AM
Reply to this message Reply
Well, I tried it with another project and it worked fine. I still don't know, why it doesn't work for the other project. The tests are run, but the report page doesn't pop up:-(.

Regards,
Antonio.

Steven Devijver

Posts: 2
Nickname: devijvers
Registered: Oct, 2007

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-i Posted: Oct 5, 2007 7:59 AM
Reply to this message Reply
I have this method which has a CRAP score of 30 and a CRAP load of 5:

public boolean matches(Method method) {
return
method.getName().equals(methodName) &&
Arrays.equals(method.getParameterTypes(), params) &&
(
!declaringClass ||
method.getDeclaringClass().equals(clazz)
);
}

Could crap4j give me any clues on what kind of tests I should add?

Alberto Savoia

Posts: 95
Nickname: agitator
Registered: Aug, 2004

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-i Posted: Oct 5, 2007 9:43 AM
Reply to this message Reply
> Downloads fine at home.
>

Hi Vincent,

Thank you for trying the download from home. Those necessary but pesky firewalls/proxy have proven to be a headache for Eclipse plug-ins for a few people. To address that, we'll make a zip version available so people can point the Eclipse updater to a local directory and bypass all that.

> It produces some curious results though. On some
> occasions it even seems to imply my code could be
> improved...

Must clearly be a bug :-).

Alberto

Alberto Savoia

Posts: 95
Nickname: agitator
Registered: Aug, 2004

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-i Posted: Oct 5, 2007 10:21 AM
Reply to this message Reply
> I have this method which has a CRAP score of 30 and a CRAP
> load of 5:
>
> public boolean matches(Method method) {
> return
> method.getName().equals(methodName) &&
> Arrays.equals(method.getParameterTypes(), params) &&
> (
> !declaringClass ||
> method.getDeclaringClass().equals(clazz)
> );
> }
>
> Could crap4j give me any clues on what kind of tests I
> should add?

Hi Steven

First of all, thank you for trying crap4j and reporting back.

> Could crap4j give me any clues on what kind of tests I
> should add?

The method you mention has the form:

return A && B && (C || D)

Where:
A = method.getName().equals(methodName)
B = Arrays.equals(method.getParameterTypes(), params)
C = !declaringClass
D = method.getDeclaringClass().equals(clazz)

In theory, to make sure you have covered all possible combinations of true/false for the 4 components of your boolean-valued expression (i.e. A, B, C, and D). That would mean writing 16 (i.e. 2^4) test cases. In practice, however, you can probably get away with fewer combinations and still keep crap4j happy and protect yourself quite well.

You have probably noticed that crap4j shows code coverage information (green/red bars on the side of the source code view). You can use those to guide you. In the case of conditional expressions like the one you have, if you hover the mouse pointer over the coverage bars you should see detailed information on which of the conditional clauses it has not covered.

You should definitely have test cases like the following:

A,B,C,D: true, false, true, false

(i.e. same method name but different parameter types and the declaring class is not equal to clazz but !declaringClass is true). This one should return false.

A,B,C,D: true, true, true, false

(i.e. same method name and same parameter types. The declaring class is not equal to clazz but !declaringClass is true). This one should return true.

And so on.

You might also consider doing some refactoring to simplify the code and the testing (and lower the CRAP score). For example:

method.getName().equals(methodName) &&
Arrays.equals(method.getParameterTypes()

Could be extracted as a method called "sameMethodSignature(..., ...)".

I hope this helps. Thanks again for experimenting with crap4j.

Steven Devijver

Posts: 2
Nickname: devijvers
Registered: Oct, 2007

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-i Posted: Oct 5, 2007 10:34 AM
Reply to this message Reply
> > I have this method which has a CRAP score of 30 and a
> CRAP
> > load of 5:
> >
> > public boolean matches(Method method) {
> > return
> > method.getName().equals(methodName) &&
> > Arrays.equals(method.getParameterTypes(), params) &&
> > (
> > !declaringClass ||
> > method.getDeclaringClass().equals(clazz)
> > );
> > }
> >
> > Could crap4j give me any clues on what kind of tests I
> > should add?
>
> Hi Steven
>
> First of all, thank you for trying crap4j and reporting
> back.
>
> > Could crap4j give me any clues on what kind of tests I
> > should add?
>
> The method you mention has the form:
>
> return A && B && (C || D)
>
> Where:
> A = method.getName().equals(methodName)
> B = Arrays.equals(method.getParameterTypes(), params)
> C = !declaringClass
> D = method.getDeclaringClass().equals(clazz)
>
> In theory, to make sure you have covered all possible
> combinations of true/false for the 4 components of your
> boolean-valued expression (i.e. A, B, C, and D). That
> would mean writing 16 (i.e. 2^4) test cases. In practice,
> however, you can probably get away with fewer combinations
> and still keep crap4j happy and protect yourself quite
> well.
>
> You have probably noticed that crap4j shows code coverage
> information (green/red bars on the side of the source code
> view). You can use those to guide you. In the case of
> conditional expressions like the one you have, if you
> hover the mouse pointer over the coverage bars you should
> see detailed information on which of the conditional
> clauses it has not covered.
>
> You should definitely have test cases like the following:
>
> A,B,C,D: true, false, true, false
>
> (i.e. same method name but different parameter types and
> the declaring class is not equal to clazz but
> !declaringClass is true). This one should return false.
>
> A,B,C,D: true, true, true, false
>
> (i.e. same method name and same parameter types. The
> declaring class is not equal to clazz but !declaringClass
> is true). This one should return true.
>
> And so on.
>
> You might also consider doing some refactoring to simplify
> the code and the testing (and lower the CRAP score). For
> example:
>
> method.getName().equals(methodName) &&
> Arrays.equals(method.getParameterTypes()
>
> Could be extracted as a method called
> "sameMethodSignature(..., ...)".
>
> I hope this helps. Thanks again for experimenting with
> crap4j.

Hey Alberto,

Thanks for your reply!

I've been thinking about the metier of unit testing for years now and I tend to oscillate between "everything has to be tested" and "most things have to be tested". It depends on my mood and I guess on the type of code and projects I'm working on.

In this case the match() method is part of a utility class that is called regularly, according to crap4j this method is currently being called 54 times when I run all my tests.

I actually had to add the "declaring class" part to the boolean expression because in the case of some static methods I didn't get the overwritten methods I expected but the ones declared in parent classes.

So my (rethoric) question is: do I actually explicitly have to test this method and the utility method that calls it? It's obviously been tested by my other tests and I would quickly find bugs and already have in the past.

Thanks for any thoughts on this. I like crap4j.

Steven

Alberto Savoia

Posts: 95
Nickname: agitator
Registered: Aug, 2004

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-i Posted: Oct 5, 2007 11:15 AM
Reply to this message Reply
> So my (rethoric) question is: do I actually explicitly
> have to test this method and the utility method that calls
> it? It's obviously been tested by my other tests and I
> would quickly find bugs and already have in the past.
>

Hello again Steve,

I believe I can reuse one of my Testivus epigrams to answer your question about adequate testing.

http://www.artima.com/weblogs/viewpost.jsp?thread=204677

BTW, based on what I can gather from our brief interaction, you'd be the second programmer ... the smart one who already tests and has experience.

BTW, the full text of "The Way of Testivus" with similar bits of "wisdom" can be downloaded from:

http://www.agitar.com/downloads/TheWayOfTestivus.pdf

Another answer I like, but can't take credit for, is to "test everything that could possibly break". However, this approach might be too conservative when it comes to providing tests for code that will be inherited and have to be maintained by someone else. In those cases, I believe it pays to err on the side of having more tests.

....

One thing I'd like to clarify is the following. Based on the crap4j coverage info, you say that the method is called 50+ times, but in order to get a CRAP score of 30 there must be some paths that are still not exercised. In other words method invocation count only tells part of the story. If all invocations only exercise a same couple of path every time, you need to have a richer and more varied set of tests.

> Thanks for any thoughts on this. I like crap4j.

Thank you. It's been fun to work on it and - hopefully - we can get a lot of people using it so we can improve it over time.

Alberto

Robert Evans

Posts: 11
Nickname: bobevans
Registered: Jun, 2003

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-i Posted: Oct 5, 2007 4:12 PM
Reply to this message Reply
Hi Antonio, Carfield,

Can you describe the src structure of your project?
In particular, what's the name of the project, do the srcs and classdirs reside under the project, or are they imported into the workspace from another dir, etc..

We have a couple of issues that we fixed around workspace management, but being an alpha, not a lot of testing has been done yet.

Thanks,
Bob

Carfield Yim

Posts: 32
Nickname: carfield
Registered: Sep, 2002

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-i Posted: Oct 6, 2007 3:48 AM
Reply to this message Reply
> Hi Antonio, Carfield,
>
> Can you describe the src structure of your project?

I put all source under folder

/WEB-INF/src
/WEB-INF/src_test

class at /WEB-INF/classes
libraries at /WEB-INF/lib

> In particular, what's the name of the project, do the srcs
> and classdirs reside under the project, or are they
> imported into the workspace from another dir, etc..
>

Name of the project is "web_site", reside under the project.

> We have a couple of issues that we fixed around workspace
> management, but being an alpha, not a lot of testing has
> been done yet.
>

Raoul Duke

Posts: 2
Nickname: raoulduke1
Registered: Oct, 2007

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-i Posted: Oct 8, 2007 4:38 AM
Reply to this message Reply
Hello.

When running crap4j on a project I seem to get duplicates of a method:

public boolean equals(java.lang.Object)
se.mine.ext.mock.account . AccountKey 6 64,71 % 7,58 0

public boolean equals(java.lang.Object)
se.mine.ext.mock.account . AccountKey 6 0,00 % 42,00 6

The first one registers all unit tests and improving the coverage increases the coverage number (as it should). However, the second method is untouchable. I cannot get crap4j to register any coverage. Also, why do I get two instances of the method showing?

Viktor Nordling

Posts: 2
Nickname: viktorn
Registered: Oct, 2007

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-i Posted: Oct 8, 2007 5:17 AM
Reply to this message Reply
Seems like great stuff!

Unfortunately, I get a message saying "The chosen operation is not available" when i click "Run Crap4J".

Any ideas?

Cheers,
Viktor Nordling

Robert Evans

Posts: 11
Nickname: bobevans
Registered: Jun, 2003

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-i Posted: Oct 8, 2007 3:18 PM
Reply to this message Reply
Hi Viktor,

Eclipse has this funny behavior about open projects.

Here's a workaround: if you click inside the project in the package explorer and then on the top level project in the tree, it will work. I have a fix for this that should be in the next update (This week).

Thanks for reporting it.

Bob Evans

Robert Evans

Posts: 11
Nickname: bobevans
Registered: Jun, 2003

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-i Posted: Oct 8, 2007 3:20 PM
Reply to this message Reply
Are there two instances of AccountKey in the classes directories by any chance?

Raoul Duke

Posts: 2
Nickname: raoulduke1
Registered: Oct, 2007

Re: The Code C.R.A.P. Metric Hits the Fan - Introducing the crap4j Plug-i Posted: Oct 9, 2007 9:44 AM
Reply to this message Reply
Since it is the same package I doubt it. That would mean two files with the same name in the same directory. Doesn't seem plausible.

I have also noticed that when running crap4j on bigger projects, it seems to fail. It runs all the JUnit tests but then nothin happens. No error message, nothing in the eclipse .log, nothing pops up and no report =(

I can't really give anymore info on that bug since I get no information regarding why it fails. This has happened on more then one project. One project was really small (4 classes) but contained some annotations for J5EE (session bean with webservice), perhaps that was the issue... I dont know. The other project was larger but with no J5EE dependencies.

Flat View: This topic has 38 replies on 3 pages [ « | 1  2  3 | » ]
Topic: The Code C.R.A.P. Metric Hits the Fan -   Introducing the crap4j Plug-in Previous Topic   Next Topic Topic: ScalaTest 0.9.1 Released


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2014 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us