Evolving Test Suite

Owned by Former user (Deleted)
Last updated: Sept 07, 2010Version comment
2 min read

(TODO: MERGE THIS PAGE WITH ImproveTheTestSuite)

These are ideas for evolving and improving the existing test suite.

There are several problems with the existing test suite including:

  • most tests are integration tests rather than unit tests
  • the tests are not all independent of one another. In some cases tests depend on the database being left in a particular state by a previous test in order to run successfully. These dependencies are not documented and are often not obvious.
  • Test class naming is inconsistent. Some tests are named TestThisClass and others are ThisClassTest.
  • "Test Suites" such as ApplicationTestSuite are used to run tests. If a test is not added to a test suite it will not be run. It would be better to be able to run all tests automatically without depending on test suites.

To address the issue of test dependencies, we could try to systematically identify which tests either:

  • leave the database in a state other than a clean initial state. For these tests we could force a cleanup.
  • depend on the database being in a state other than a clean initial state. For these tests, we could modify the test to depend on a clean initial state.

Here is a proposed process for achieving this:

  • Rename MifosTestCase to MifosIntegrationTest and MifosMockStrutsTestCase to MifosMockStrutsIntegrationTest making clear that tests derived from these base classes initialize the database.
  • For each test derived from MifosIntegrationTest or MifosMockStrutsIntegrationTest, insure that if it overrides setUp(), the first call is super.setUp() and if it overrides tearDown, that the last call in teardown is super.tearDown(). Rename the test to be <SomeClass>IntegrationTest to make clear what kind of test it is and use a consistent naming scheme. This paves the way for <SomeClass>Test classes to be introduced which will be true unit tests.
  • Add a call to MifosIntegrationTest.setUp() and MifosMockStrutsIntegrationTest.setUp() which reinitializes the database to a clean initial state
  • Modify any tests which fail so that they pass starting from a clean initial state.
  • Add a call to MifosIntegrationTest.tearDown() and MifosMockStrutsIntegrationTest.tearDown() which compares the state of the database to a clean initial state and fails if they differ.
  • In any tests that fail, add cleanup code to restore the database to a clean initial state.
  • Undo the changes to MifosIntegrationTest.setUp() and MifosIntegrationTest.tearDown().
  • The tests should still pass and now they should be able to be run in any order.
  • Eliminate the xxxTestSuite classes and change the build to run tests based on the pattern *Test.