Mock objects provide a solution for stubbing out system dependencies during unit testing. In a recent article, Alex Ruiz weighs the downsides as well as the benefits of testing with mock objects.
Among the more frustrating aspects of unit testing are those tests that rely on many external dependencies, such as database connections, collaborator objects, or even the file system. Because a lot of context is assumed by such code, writing tests that isolate the behavior of individual methods is time-consuming and fragile.
Mock objects can be used to stub out such dependencies. Yet, relying on mock objects is not without danger, as Alex Ruiz points out in his article, BEA's Dev2Dev, Mock Objects: Shortcomings and Use Cases:
Testing code in isolation is difficult, especially when dealing with dependencies that are not easy or quick to set up in our tests. The more difficult it is to write and maintain unit tests, the greater the chances that developers will feel discouraged and stop writing tests...
Mock objects, or mocks, simulate difficult-to-use and expensive-to-use collaborators and provide a way to... set up complex dependencies in the testing environment..., verify that expected actions were executed by the code under test [and], simulate environment conditions that are difficult to generate...
A key point of Ruiz's article is that dependencies in a code base are often the source the bugs we would like to discover with unit testing. Stubbing out such dependencies can defeat the very purpose of testing:
Mocks can effectively speed up creation and execution of unit tests, but they cannot give us the confidence that the system, as a whole, works correctly. Mock testing may hide bugs or defects in the collaborators that are being mocked.
In addition, Ruiz notes that mock objects impose some design requirements on a system, and also require intimate knowledge of the system's implementation details:
Mock testing is glass box testing, which requires intimate knowledge of class internals. This is a side effect of the interaction-based nature of mocks. Justifiable changes in the implementation of a method may break tests using mocks, even if the result of executing such a method is still the same...
Expectations on class-based mocks may depend on protected or package-protected methods, which represent implementation details of a class. Such implementation details can (and will) change at any time, changing the interaction between the code under test and mocks, increasing the chances of breaking existing tests...
To what extent do you rely on mock objects in your unit tests? Do you agree with Ruiz's point that mock objects impose a lot of design constraints on a code base?
this troubles me a lot also. i strive against the scope of the mock objects. however, the fact that "the dependencies in a code base are often the source the bugs" is true in many cases, i use high-level test to get confidence.
In J2EE applications I have always seen at least three dependencies, server, database, and third party api. I do not mind to have server or database dependencies. I wish all third party api are test friendly and/or comes with a set of mock objects.
One of the best uses of mocks is for testing how the code handles error conditions that are hard to produce at will when testing against the real resource. For example, losing the network connection in the middle of a transaction.
In the end, unit testing in general can only test that you have implemented it according to your own assumptions. It cannot test that your assumptions are correct. Believing that unit testing is doing more than that is a road to disaster.
> I some how agree with the Author, and have many times > remind my developer of delivering good Return of > Investment (ROI) unit testing code rather then blindly try > to boost unit testing coverage.. I posted my view at my > blog : > http://coolboykl.blogspot.com/2007/05/100-unit-test-code-co > verage-using-mock.html > > Share me your view on this..
I agree that the most important thing is full functional and regression testing.
I think one of the biggest misunderstandings about unit testing is that it's about proving that the application works. It's really just a tool to find implementation mistakes on a small scale early. This improves productivity because full end-to-end testing is expensive and not a good place to find a lot of one-off bugs and other minor implementation issues. Really good unit testing means that your full functional testing is mainly testing only the interaction of the units with each other and with external actors (database, web, users, etc.)
I'm sorry, but these are old arguments. Alex should look at our OOPLSA 2004 paper "Mock Roles, Not Objects", rather than just our original from XP 2000.
No-one who knows what they're doing suggests writing only unit tests, with or without mocks.
The glass box problem is what people stumble over when they're new to Mocks. We find that by concentrating on collaborations between objects we spend less time wondering what's inside any particular one.
We recommend against class-based mocking, except in emergencies, because it leaves design decisions implicit. And so on.