I used to be rich client guy, but my current client is building web apps and so all of a sudden I'm a web app guy. Now I wish I’d paid more attention in the past. For my penance I promise to document whatever I learn on the forthcoming Agile Tools Functional Test Wiki (i.e. the wiki that we discussed last year at the AA-FTT meeting before A2008).

My client is building an ASP .NET application and we're trying to decide how to test it. The issues I have to deal with:

• The two QA people have no coding experience so any tool that requires significant coding will require more development support.
• The application will be run in IE on the IIS webserver. No other combinations need be tested.
• The application is really a document creation and management system. Once the documents have been created and approved they're submitted to an outside system.

My discoveries so far (via this list and the opensourcetesting.com) and some follow-up questions
Tools:

• Selenium seems to be written about the most although some users complain about slowness and less than intuitive api.
• Watin (now supports firefox) api seems to be well liked - but assertions have to be written in manually i.e. in code.
• Apodora - never heard of this tool before. From Open Source Testing: "Apodora is a framework/tool for automating functional testing of web applications. It provides the user with programmatic control of the web browser allowing them to interact directly with the browser's user interface. It uses a database backend in order to remember how to find your html elements. This also makes your scripts easier to maintain."
• FitNesse .NET - not entire convinced that the application is well suited to FitNesse style tests. But the team will make the decision. I really think I need to read chunks of Gojko's first book before passing any real judegement.
• Canoo WebTests - xml may be hard for customers and non programmers to write. Am I missing the point?
• Concordian – seems like a clever idea that will create a lot of work for developers.

Frameworks/IDEs

• Robot Framework - “Robot Framework is a generic keyword-driven test automation framework for acceptance level testing and acceptance test-driven development (ATDD). It has an easy-to-use tabular syntax for creating test cases and its testing capabilities can be extended by test libraries implemented either with Python or Java. Users can also create new keywords from existing ones using the same simple syntax that is used for creating test cases.”
• CubicTest – an Eclipse Based IDE for Selenium, promises “…It makes web tests faster and easier to write, and provides abstractions to make tests more robust and reusable.”
• StoryTestIQ - “STIQ is a mashup of Selenium and FitNesse. It is "wiki-ized" Selenium with widgets and features that make it easier to write and organize Selenium tests.”

Questions

Selenium

• Are the problems with Selenium slowness, need to insert delays after entering text and less than intitutive api real? (http://adamesterline.com/2007/04/23/watin-watir-and-selenium-reviewed/, http://www.testdrivendeveloper.com/2008/07/22/ComparingWatiNAndSeleniumForUITesting.aspx, http://hammett.castleproject.org/?p=120, ).
• Do you spend more time debugging the tests than finding real problems?
• Is Jay on the money is his positioning of Selenium: http://blog.jayfields.com/2008/07/immaturity-of-in-browser-testing.html “Selenium is best used by developers or testers when testing the most valuable (to the business) happy paths of a Javascript heavy web application that must function in several browsers”

Watin

• Does it support dialogs now?
• Does it suffer the same slowness problems as Selenium?
• Is it as hard to use with IE as Selenium can be?
• Is it compatible with Robot Framework?

If you’re using Selenium are you using any wrapper framework/IDE?

If you enjoyed this post, subscribe now to get free updates.