We use Jira (mifosforge.jira.com) for tracking Mifos test cases.

Benefits

• easy access with commonly accepted UI
• ability to combine test cases together for a test cycle (e.g. end of release test pass)
• organize test cases by component
• flag test cases as manual, automated, or to-be-automated
• report on test results pass/fail/fail-defer

Issue Types

We have three issue types for the MIFOSTEST project.

• "Test Case"  Test case type includes summary, component, description, assignee,and information on test case is automation.
• "Test SubTask"  For each test case, subtasks can be created to track the pass/fail result of each test case for each release.
• "Test Cycle" Test cycles can optionally be used to group a set of manual tests cases by using the Jira "Link" feature.

Test Case states

• Work flow for test cases is simply "Open", "Manual", and "Automated"
  • Open status is for newly created test cases.
  • Manual status is for test cases that are tested manually, including tests that can eventually automated.
  • Automated status is for test cases that are tested by an automated acceptance test or other test automation as noted in the test case.

Test Subtask states

• Work flow for test subtasks include pass, fail, and fail-defer.
  • Pass status is for test that passes acceptance criteria of test case.
  • Fail status is for test that fails acceptance criteria of test case.
  • Fail Defer status is for test that fails a minor part of the acceptance criteria of test case, or has a bug reported that isn't vital to the release (e.g. minor cosmetic issue with an error message)

Notes on original research

Before creating this test case project, we looked at a combination of the test case management implementations described by both Atlassian and Moodle:

Atlassian: http://confluence.atlassian.com/display/CONFEVAL/Customise+JIRA+For+Test+Case+Management

Moodle: http://docs.moodle.org/en/Jira_as_a_Test_Case_Management_Software