The Artima Developer Community
Sponsored Link

Java Community News
Alex Ruiz on the Dangers of Mock Objects

6 replies on 1 page. Most recent reply: Dec 11, 2007 4:25 AM by Steve Freeman

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 6 replies on 1 page
Frank Sommers

Posts: 2642
Nickname: fsommers
Registered: Jan, 2002

Alex Ruiz on the Dangers of Mock Objects Posted: Aug 28, 2007 12:58 PM
Reply to this message Reply
Summary
Mock objects provide a solution for stubbing out system dependencies during unit testing. In a recent article, Alex Ruiz weighs the downsides as well as the benefits of testing with mock objects.
Advertisement

Among the more frustrating aspects of unit testing are those tests that rely on many external dependencies, such as database connections, collaborator objects, or even the file system. Because a lot of context is assumed by such code, writing tests that isolate the behavior of individual methods is time-consuming and fragile.

Mock objects can be used to stub out such dependencies. Yet, relying on mock objects is not without danger, as Alex Ruiz points out in his article, BEA's Dev2Dev, Mock Objects: Shortcomings and Use Cases:

Testing code in isolation is difficult, especially when dealing with dependencies that are not easy or quick to set up in our tests. The more difficult it is to write and maintain unit tests, the greater the chances that developers will feel discouraged and stop writing tests...

Mock objects, or mocks, simulate difficult-to-use and expensive-to-use collaborators and provide a way to... set up complex dependencies in the testing environment..., verify that expected actions were executed by the code under test [and], simulate environment conditions that are difficult to generate...

A key point of Ruiz's article is that dependencies in a code base are often the source the bugs we would like to discover with unit testing. Stubbing out such dependencies can defeat the very purpose of testing:

Mocks can effectively speed up creation and execution of unit tests, but they cannot give us the confidence that the system, as a whole, works correctly. Mock testing may hide bugs or defects in the collaborators that are being mocked.

In addition, Ruiz notes that mock objects impose some design requirements on a system, and also require intimate knowledge of the system's implementation details:

Mock testing is glass box testing, which requires intimate knowledge of class internals. This is a side effect of the interaction-based nature of mocks. Justifiable changes in the implementation of a method may break tests using mocks, even if the result of executing such a method is still the same...

Expectations on class-based mocks may depend on protected or package-protected methods, which represent implementation details of a class. Such implementation details can (and will) change at any time, changing the interaction between the code under test and mocks, increasing the chances of breaking existing tests...

To what extent do you rely on mock objects in your unit tests? Do you agree with Ruiz's point that mock objects impose a lot of design constraints on a code base?


isaac han

Posts: 1
Nickname: isaachan
Registered: Aug, 2007

Re: Alex Ruiz on the Dangers of Mock Objects Posted: Aug 28, 2007 8:23 PM
Reply to this message Reply
this troubles me a lot also. i strive against the scope of the mock objects. however, the fact that "the dependencies in a code base are often the source the bugs" is true in many cases, i use high-level test to get confidence.

Kunal Shah

Posts: 1
Nickname: shahkun3
Registered: Aug, 2007

Re: Alex Ruiz on the Dangers of Mock Objects Posted: Aug 28, 2007 9:35 PM
Reply to this message Reply
In J2EE applications I have always seen at least three dependencies, server, database, and third party api. I do not mind to have server or database dependencies. I wish all third party api are test friendly and/or comes with a set of mock objects.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Alex Ruiz on the Dangers of Mock Objects Posted: Aug 29, 2007 1:15 PM
Reply to this message Reply
One of the best uses of mocks is for testing how the code handles error conditions that are hard to produce at will when testing against the real resource. For example, losing the network connection in the middle of a transaction.

In the end, unit testing in general can only test that you have implemented it according to your own assumptions. It cannot test that your assumptions are correct. Believing that unit testing is doing more than that is a road to disaster.

Chen Shiang Khoo

Posts: 5
Nickname: coolboykl
Registered: Feb, 2007

Re: Alex Ruiz on the Dangers of Mock Objects Posted: Aug 29, 2007 9:26 PM
Reply to this message Reply
I some how agree with the Author, and have many times remind my developer of delivering good Return of Investment (ROI) unit testing code rather then blindly try to boost unit testing coverage.. I posted my view at my blog :
http://coolboykl.blogspot.com/2007/05/100-unit-test-code-coverage-using-mock.html

Share me your view on this..

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Alex Ruiz on the Dangers of Mock Objects Posted: Aug 30, 2007 9:53 AM
Reply to this message Reply
> I some how agree with the Author, and have many times
> remind my developer of delivering good Return of
> Investment (ROI) unit testing code rather then blindly try
> to boost unit testing coverage.. I posted my view at my
> blog :
> http://coolboykl.blogspot.com/2007/05/100-unit-test-code-co
> verage-using-mock.html
>
> Share me your view on this..

I agree that the most important thing is full functional and regression testing.

I think one of the biggest misunderstandings about unit testing is that it's about proving that the application works. It's really just a tool to find implementation mistakes on a small scale early. This improves productivity because full end-to-end testing is expensive and not a good place to find a lot of one-off bugs and other minor implementation issues. Really good unit testing means that your full functional testing is mainly testing only the interaction of the units with each other and with external actors (database, web, users, etc.)

Steve Freeman

Posts: 3
Nickname: smgfreeman
Registered: May, 2006

Re: Alex Ruiz on the Dangers of Mock Objects Posted: Dec 11, 2007 4:25 AM
Reply to this message Reply
I'm sorry, but these are old arguments. Alex should look at our OOPLSA 2004 paper "Mock Roles, Not Objects", rather than just our original from XP 2000.

No-one who knows what they're doing suggests writing only unit tests, with or without mocks.

The glass box problem is what people stumble over when they're new to Mocks. We find that by concentrating on collaborations between objects we spend less time wondering what's inside any particular one.

We recommend against class-based mocking, except in emergencies, because it leaves design decisions implicit. And so on.

We've written up quite a lot of this material at

http://www.mockobjects.com/labels/listening%20to%20the%20tests.html

S.

Flat View: This topic has 6 replies on 1 page
Topic: What is Consistent Hashing and Why You Should Care Previous Topic   Next Topic Topic: Sun's Gregg Sporar on the New NetBeans 6.0 Code Editor


Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2017 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use - Advertise with Us