Dev2Dev > Articles

---

Mock Objects: Shortcomings and Use Cases

by Alex Ruiz
08/22/2007

Abstract

Writing unit tests is not easy. Most of the time, we need to test code that uses complex collaborators such as databases, application servers, or software modules that have not been written yet. We also may need to deal with conditions that are difficult to generate in a test environment. Setting up these dependencies may require a considerable amount of time and energy, counteracting the benefits of having automated tests. This article looks at Mock Objects, a testing technique from the XP community that offers a way to test our code in isolation by simulating those external dependencies. As with any other tool, we need to be careful and avoid overusing them.

Mock Objects Overview

In recent years, developers have revived the benefits of writing their own tests, agreeing that finding and fixing defects in software is expensive. As a result, Unit Testing has become an essential part of the software development process, as a way to find coding mistakes and to help identify system requirements. The main goal of unit testing is to test in isolation a single unit of work, which is usually a class. Testing code in isolation is difficult, especially when dealing with dependencies that are not easy or quick to set up in our tests. The more difficult it is to write and maintain unit tests, the greater the chances that developers will feel discouraged and stop writing tests.

Tim Mackinnon, Steve Freeman, and Philip Craig introduced the idea of mock objects in their paper "Endo-Testing: Unit Testing with Mock Objects," which they presented at XP2000. Mock objects, or mocks, simulate difficult-to-use and expensive-to-use collaborators and provide a way to:

• set up complex dependencies in the testing environment (for example, simulate a database connection instead of connecting to a real database)
• verify that expected actions were executed by the code under test (for example, verify that a JDBC connection was closed after use—that is, the method close in java.sql.Connection was called at a specific moment)
• simulate environment conditions that are difficult to generate (for example, simulate a SQLException being thrown by the JDBC driver)

Although useful, mocks are not a panacea and their overuse can bring more harm than good to our projects.

Shortcomings of Mocks

Let's look at a few items that the mock programmer needs to be aware of.

Mocks may hide integration problems

This is especially true if we test code using mocks exclusively, without writing integration tests.

Consider the example in Figure 1.

Figure 1. Storing information of a new employee in a database

The EmployeeBO class offers business services related to Employees and uses EmployeeDAO to persist data in a relational database using JDBC. Testing EmployeeBO would imply setting up a database and populating it with test data.

Proponents of mock objects suggest that we can save a significant amount of time and effort by mocking EmployeeDAO, avoiding the overhead of setting up and using a real database in our tests. Mocks can effectively speed up creation and execution of unit tests, but they cannot give us the confidence that the system, as a whole, works correctly. Mock testing may hide bugs or defects in the collaborators that are being mocked. To find those defects, our test suites need to include integration tests. In our example, the system being tested uses a database to store employee information. Mock testing is limited to verify that the interaction between EmployeeBO and EmployeeDAO is correct—that is, that EmployeeBO calls only the expected methods from EmployeeDAO at the expected time. Only integration tests can help us discover problems, such as bugs in the JDBC driver or in the database itself, that should not be present when the application goes to production.

Mocks add clutter and duplication to test code

The following code listing uses EasyMock to test that EmployeeBO uses EmployeeDAO to store information of new employees and to update information of existing employees.

  @Before public void setUp() {
    mockEmployeeDAO = createMock(EmployeeDAO.class);
    employeeBO = new EmployeeBO(mockEmployeeDAO);
    employee = new Employee("Alex", "CA", "US");
  }

  @Test public void shouldAddNewEmployee() {
    mockEmployeeDAO.insert(employee);
    replay(mockEmployeeDAO);
    employeeBO.addNewEmployee(employee);
    verify(mockEmployeeDAO);    
  }

  @Test public void shouldUpdateEmployee() {
    mockEmployeeDAO.update(employee);
    replay(mockEmployeeDAO);
    employeeBO.updateEmployee(employee);
    verify(mockEmployeeDAO);    
  }

The test method shouldAddNewEmployee verifies correct interaction between the object under test (employeeBO) and the mock (mockEmployeeDAO). It expects employeeBO to call the method insert in mockEmployeeDAO, passing the same instance of Employee that it receives. Although simple, the method shouldAddNewEmployee has the following code that does not communicate its purpose, adding clutter to our test:

• A call to replay to notify EasyMock that all the expectations have been set
• A call to verify to notify EasyMock that the expectations should be verified

Usage of mocks usually follows this pattern:

1. Setup of the mock(s) and expectations
2. Execution of the code to test
3. Verification that the expectations were met

Such a pattern introduces duplication in test code, which is evident in the test methods shouldAddNewEmployee() and shouldUpdateEmployee(). The following template class, EasyMockTemplate, can help reduce both code clutter and duplication:

/**
 * Understands a template for usage of EasyMock mocks.
 * @author Alex Ruiz
 */
public abstract class EasyMockTemplate {

  /** Mock objects managed by this template */
  private final List<Object> mocks = new ArrayList<Object>();
   	
  /**
   * Constructor.
   * @param mocks the mock objects this template will manage.
   * @throws IllegalArgumentException if the list of mock objects is null or empty.
   * @throws IllegalArgumentException if the list of mock objects contains a null value.
   */ 
  public EasyMockTemplate(Object... mocks) {
    if (mocks == null) throw new IllegalArgumentException("The list of mock objects should not be null");
    if (mocks.length == 0) throw new IllegalArgumentException("The list of mock objects should not be empty");
    for (Object mock : mocks) {
      if (mock == null) throw new IllegalArgumentException("The list of mocks should not include null values");
      this.mocks.add(mock);
    }
  }
   
  /**
   * Encapsulates the common pattern followed when using EasyMock.
   * <ol>
   * <li>Set up expectations on the mock objects</li>
   * <li>Set the state of the mock controls to "replay"</li>
   * <li>Execute the code to test</li>
   * <li>Verify that the expectations were met</li>
   * </ol>
   * Steps 2 and 4 are considered invariant behavior while steps 1 and 3 should be implemented by subclasses of this template.
   */
  public final void run() {
    setUp();
    expectations();
    for (Object mock : mocks) replay(mock);
    codeToTest();
    for (Object mock : mocks) verify(mock);
  }

  /** Sets the expectations on the mock objects. */
  protected abstract void expectations();

  /** Executes the code that is under test. */
  protected abstract void codeToTest();

  /** Sets up the test fixture if necessary. */
  protected void setUp() {}
}

Now, let's refactor our test to use EasyMockTemplate:

  @Test public void shouldAddNewEmployee() {
    EasyMockTemplate t = new EasyMockTemplate(mockEmployeeDao) {
      @Override protected void expectations() {
        mockEmployeeDAO.insert(employee);
      }

      @Override protected void codeToTest() {
        employeeBO.addNewEmployee(employee);
      }
    };
    t.run();    
  }

  @Test public void shouldUpdateEmployee() {
    EasyMockTemplate t = new EasyMockTemplate(mockEmployeeDao) {
      @Override protected void expectations() {
        mockEmployeeDAO.update(employee);
      }

      @Override protected void codeToTest() {
        employeeBO.updateEmployee(employee);
      }
    };
    t.run();    
  }

EasyMockTemplate brings the following benefits:

1. Since the methods expectations and codeToTest are abstract, EasyMockTemplate forces developers to specify both expectations and the code to test, minimizing programming errors.
2. There is a clear separation of expectations and the code to test, which makes tests easier to understand and maintain.
3. We have eliminated code duplication since we do not have to call replay and verify anymore.

You can download the latest version of EasyMockTemplate here.

Tests using mocks may be fragile

Mock testing is glass box testing, which requires intimate knowledge of class internals. This is a side effect of the interaction-based nature of mocks. Justifiable changes in the implementation of a method may break tests using mocks, even if the result of executing such a method is still the same.

In our example, EmployeeBO interacts with EmployeeDAO to store employee information in a database using plain JDBC. Let's say we change the way we store information in the database—from JDBC to JPA, for example—by replacing EmployeeDAO with EmployeeJPA, storing exactly the same information in exactly the same database tables. We would expect our existing tests to pass, since the outcome (storing data in the database) has not changed. Unfortunately, our tests using mocks will fail simply because the interaction between EmployeeBO and EmployeeDAO no longer exists: EmployeeBO now uses EmployeeJPA to store data in the database, as shown in Figure 2.

Figure 2. Changing database persistence strategy breaks existing tests using mocks

Conversely, black box testing (for example, functional testing) is less likely to break if the internals of the system change while the outcome stays the same, because the testing is based solely on the knowledge of system requirements.

In our example, integration tests that verify the correctness of the data being stored in the database (instead of verifying how data was stored) will not fail after replacing EmployeeDAO with EmployeeJPA.

Pages: 1, 2

Next Page »