This post originated from an RSS feed registered with Java Buzz
by Danno Ferrin.
Original Post: Automated GUI Testing
Feed Title: And They Shall Know Me By My Speling Errors
Feed URL: http://shemnon.com/speling/archives/cat_GUI.rdf
Feed Description: Danno Ferrin's Many Reasons he was a Comp Sci Major and not an English Major
One
of the buggest deals that people in my office seem to make about XP is the
emphasis on Testing. (We could care less about Pair Programming, Iteration
driven stuff, and the way they do story management, none of us including
me really get disagree that what we have in those areas seems to work).
But from all of the reading most of the supprot I have seen is foducsing
on either object components or the lowest of the gui portions if even that.
For the longest time I have been trying to find a good GUI test too I can
integrate with the build, I think I found a hidden gem. But let's look at
what's out there and why I didn't use it...
Abbot
- Serious GUI bug in 1.4, that pane on the right you are supposed to put
stuff like classpath and class name in? Doesn't work, won't remember the
values or persist it to the model. Hence it is unusable for record.
Pounder
- I forgot exactly why, but it behaved poorly for me out of the box. I seem
to recall it used java.awt.Robot to do alot of the work, which would have
been good or bad depending on what you are trying to test. But the use on
Linux and the fact component sized change frequently would have been bad
news for it. It also takes over the computer to take over the test and does
it in real time, good or bad based on how you look at it. The other problem
is that it records every stinking possible awt event that exists! Too much
clean up on the test when you are done.
JFCUnit
- The lack of a record option is a killer here. If I'm not going to design
the grid bag container by hand I sure am not going to write the test cases
by hand. Especialy since some of the GUI is dynamicaly generated and the
structure of those components beneath that pane are difficult to derive by
hand. Yea, there is aFAQ item that involved wriring your own but it took
less effort to keep on looking
Jemmy
- (a) Lack of JUnit Integration, and (b) Lack of a record option, see my
entry on JFCUnit. (a) might not be such a big deal if (b) wasn't there and
(c) I had infinite time.
Any commercial test suite -
I don't know if anyone noticed out there but there is a recession going on.
The cost for a autometed GUI test suite to use on the scale we want to (every
developer that checks in code) it would cost us at leas one head count.
I don't want to see any more of my colleagues leave, we've already gotten
rid of the stinkers (and some of the good ones). That's why I'm being cheap
about it.
Well, that seems to be the extent
of the GUI test tools out there, and they've all got there blockers that
encourage me to look elsewhere. Or is that all of them? In doing a google
search on "Java GUI Test Tools" I stumbeled upon the mention of a java-gui-testing
group on Yahoo Groups, where I learned about the hidden gem I previously
mentioned:
Marathon
- not perfect but Good Enough. Record option, junit playback, was able to
get it working with my detateched test harness. Dosen't record random key
strokes like I would like it to, but it referes to the JComponents principally
by the setName(...) value you gave to it. It uses Jython to record the test
and play back the test script. The API for the types of tests it has is
a little limited, but the whole set of Java APIs are exposed so you can do
whatever you need to inside of the jython script, so to add some custom valdiation
logic you don't have to write a whole new stinking java class. And it integrates
into my JUnit setup, with the tests showing up in the junitreport.
Good Enough, and much better than what I had before. What I had
before was my memory and intuition of what I thought would break. But that
tends to be unreliable under time pressure. Doing the same tests over and
over again, and when it's automated you don't have to worry about the Intern
or Script Monkey (which is not the same as a QA person) finding a better
job!
Update : Glen
posted about this sunday, and I learned that after I wrote this. So I didn't
get to "out" marathon in the java.blogs community. But I felt my selection
reasons and some more exposure would be good for marathon.