This page describes the steps for creating new acceptance tests for Mifos. To get started, the development environment needs to be set as described in Developer Setup.

Location of acceptance tests

Acceptance tests are in source control in acceptanceTests/src/test/java/org/mifos/test/acceptance

Acceptance tests organization

Acceptance test classes are organized in packages that relate to feature areas of the product e.g. loan, center, or user. The package name is also used to categorize the tests. The supporting page object and other helper methods are stored in /acceptance/framework. Helper methods for functional areas should be stored in a corresponding package under /acceptance/framework:

acceptance
   + group
      + GroupTest.java
   + framework
      + group
         + GroupSearchPage.java, etc.

The clearest way to view this organization is view the project in the Eclipse Navigator.

Data sets and expected data results are stored in acceptanceTests/src/test/resources/dataSets

Identify new acceptance test cases

We are creating automated acceptance tests that validate each build is able to execute the basic functionality of the application. These tests can be described as Build Validation Tests (BVT). The BVT test case tasks are being listed in the issue tracker to track their implementation. See Build Validation Tests for the list of acceptance tests.

If you are volunteering, please contact the Mifos team through the developer list server and coordinate which acceptance tests you would like to write.

Using id tags for selenium

Many of the UI elements in the JSP pages for Mifos lack id tags. These tags allow Selenium to be more robust and execute with different languages. For clarity, Mifos tags should use the following naming convention: <pagename>.<uiElement>.<elementName>.

• The page name will be the name of the jsp file being modifed.
• Each page should have a unique page id, specified by a hidden form field. If the page is called "FooBar.jsp", the page id should be specified like this: <span id="page.id" title="FooBar"/>. See CreateMultipleLoanAccounts.jsp for an example.
• The uiElement name refers to the type of HTML element. Use the following names for UI Elements:

• The elementName should describe the element e.g. cancel, submit, search, firstName. Longer element names should use a camel case convention.

To add id tags, do the following:

1. Open Eclipse and locate the proper jsp file. As an example, login.jsp (the login page)
2. search for ui elements - labels, text fields, buttons,...
3. Add the id tag. For basic html, add as follows:

<a id="homeheader.link.home" href="custSearchAction.do?method=getHomePage" class="tabfontwhite"><mifos:mifoslabel name="framework.home" bundle="FrameworkUIResources"></mifos:mifoslabel></a>

Many UI elements in Mifos use the Struts html-el tag library. To add an ID tag in to an html-el element, use styleId as follows:

<html-el:text styleId="login.input.username" property="userName" />

If using mifos tag library, use styleId also. This may not work in all cases due to incomplete implementations. In that case, enclose the text in span tags. Here is an example of span used with a fictional mifos custom tag on the login page:

<span id="login.label.password"><mifos:somecustomtag name="login.password" bundle="LoginUIResources"/></span>

Try to use styleId attributes instead of span tags wherever possible.

1. save and build mifos.

View UI elements using Firebug

To view the UI elements including newly created id tags, navigate Mifos with the Firefox plugin "Firebug" installed and enabled. Click on the "inspect" button on the Firebug tab and hover over controls in the Mifos UI to see the html for each UI element.

Experiment using Selenium IDE

To determine how your test can be best written, use Selenium IDE to record and playback an acceptance test. Confirm the selenium script is using any new id tags added to the product.

Page Objects

A page object encapsulates all the operations or "services" offered by a page in one place, so that if the implementation of the page changes, only the page object needs to be changed and not the tests that use that page. (The Don't Repeat Yourself principle.) This page object concept is documented in more detail here:

http://code.google.com/p/webdriver/wiki/PageObjects

Page objects in Mifos can be found here (and this is where new page objects should be added):

acceptanceTests/src/test/java/org/mifos/test/acceptance/framework

Page objects should always have at least one verify method, verifyPage(). This should call this.verifyPage(pageName), where pageName is the name of the jsp file (without .jsp). This method will use Selenium to verify that the jsp page's unique hidden form field matches what is expected. See CreateLoanAccountsSearchPage.java for an example.

Test Data

There are two test data forms currently - acceptance_test_dump.sql and dbUn datasets. DbUnit is deprecated. As part of MIFOS-4590  , there is currently an on-going effort to refactor the current suite of acceptance tests to minimize dependency on the DB unit based XML datasets.

Test data are stored in the following locations:

db/src/test/resources/sql/acceptance_test_dump.sql
acceptanceTests/src/test/resources/dataSets (DbUnit)

Procedure for maintaining acceptance_test_dump.sql 

If you want to make changes to the dump by using the application
Import all data from the dump into the database.
Use the application to setup/change (only) the data you want.
Export the data using following command.
mysqldump -u <your_user> -p<your_password> <your_database> --complete-insert --extended-insert --skip-add-locks --skip-comments --skip-quote-names > ~/acceptance_test_dump.sql
When commiting changes commit acceptance_test_dump.sql file as well.

If you want to make changes to the dump manually
Edit acceptance_test_dump.sql
When commiting changes push acceptance_test_dump.sql file as well.

DbUnit dependency

A new test group, no_db_unit, has been introduced to categorize tests that do not need DB unit dependency. Currently, the Maven POM for acceptance tests, ensures that all tests in the no_db_unit group are executed prior to executing the rest of the acceptance tests. This ensures that the no_db_unit group of tests do not inadvertently depend on the XML data sets used by the older tests.

In summary, if you are adding an acceptance test to the current suite, ensure that you look at an existing test in the no_db_unit category and see how the test data is setup & managed. Try & avoid loading your seed data from an XML dataset whenever possible.

DbUnit approach - deprecated and should not be used for new tests

There are two different kinds of data sets, validation and start data sets. Validation data sets are used to make sure that at the end of a test, the database looks correct. As of Jan 2011, we are working to eliminate the use of validation data sets.  Start data sets are used to set up a suitable environment for each test case and they're usually loaded at the very beginning of the test.

Every data set consists of two files:

• a txt file that describes what the data set consists of and/or its purpose.
• an xml file which is the DBUnit representation of the database (a dump).

Rather than generate all necessary test data for every acceptance test, we will create a basic set of test data that can be loaded via DBUnit. The details of the dataset acceptance_small_001 are described in Test Data Set. acceptance_default_003 is the default or empty Mifos database.

Generating and importing test data set is documented at the DbUnitDataImportExport page.

Upgrading data sets generated from an earlier database version to the current database version is documented at the DataSetUpgradeUtil page . Read this thread for more information about the data set upgrade process.

Writing Acceptance Tests

Here is a brief outline of the steps to follow in order to create an acceptance test:

• load an existing data set (or generate a new one as described above)
• record acceptance test operations using Selenium IDE
• if you will validate against a final database state, then dump the final database state (at the end of the operation) and create a new data set using DbUnitDataImportExport as described above
• write the acceptance test java code (such as CollectionSheetEntryTest.defaultAdminUserEntersSingleLoanPayment) which: _ loads an initial database state _ includes transitions to the page objects which map to the pages traversed during the test and any required data entry
• use the acceptance test code (with initially nonexistent methods) to generate (via Eclipse) the empty classes and methods that need to be implemented
• using existing page objects as examples and the Selenium IDE generated code, fill in the page object code needed to make the test functional
• test name should be a combination of words which describes the main purpose of the test
• test should verify that data entered on the create page are displayed properly on the preview page
• while writing the test remember to use appropriate line spacing to make the code easy to read
• it's good practice to add comments in the code which explain what is verified at each step of the test
• add comment to the acceptance test with the link to the related test case

Executable Specification Style (added Jan 2011)

To make acceptance tests more clear in their intent, we will write tests to with a executable specification style that uses internal test drivers or helpers to separate the test into two layers - a test implementation layer and a lower helper or application driver layer.  The acceptance test is written using internal domain specific language (test helpers) e.g. "createClient" as developed by the test authors.  These tests will be written using a Given, When, Then structure.  "Given" is the set up of the test case, "When" is the activity between the user and Mifos, and "Then" is the expected result.  This style and the concept of layered acceptance tests is described in more detail in Continuous Delivery, Chapter 8.   The goal of including this style is to simplify test writing and maintenance.  While the tests will be similar to behavior driven tests, we will still write tests in a Java to take advantage of its full development environment.  Some of these helpers could use Service Facade or API's of Mifos to build entities to support more complex use cases. 

Here is an example of an existing test from GroupTest.java:

@Test(sequential = true, groups = {"group","acceptance","ui"})
 @SuppressWarnings("PMD.SignatureDeclareThrowsException") // one of the dependent methods throws Exception
 public void createGroupInPartialApplicationStateTest() throws Exception {
 initRemote.dataLoadAndCacheRefresh(dbUnitUtilities, "acceptance_small_001_dbunit.xml", dataSource, selenium);
 CreateGroupEntryPage groupEntryPage = loginAndNavigateToNewGroupPage();
 CreateGroupSubmitParameters formParameters = getGenericGroupFormParameters();
 CreateGroupConfirmationPage confirmationPage = groupEntryPage.submitNewGroupForPartialApplication(formParameters);
 confirmationPage.verifyPage();
 GroupViewDetailsPage groupDetailsPage = confirmationPage.navigateToGroupDetailsPage();
 groupDetailsPage.verifyPage();
 groupDetailsPage.verifyStatus("Partial Application*");
 }

Might be written something like:

\*
\* Given Center exists,
\* And there is a user with rights to create a group,
\* When I login in as user "mifos"
\* And I create a Group with generic parameters
\* And I view Group Details
\* Then Group status is "Partial Application"
*/

@Test(sequential = true, groups = {"group","acceptance","ui"})
// http://mifosforge.jira.com/browse/MIFOSTEST-300
@SuppressWarnings("PMD.SignatureDeclareThrowsException") // one of the dependent methods throws Exception
public void createGroupInPartialApplicationStateTest() throws Exception {
//Given
initRemote.dataLoadAndCacheRefresh(dbUnitUtilities, "acceptance_standard_dbunit.xml", dataSource, selenium);
Center center = createCenter(getGenericCenterFormParameters());
//When
login("mifos");
Group group = createGroup(center, getGenericGroupFormParameters());
GroupDetails groupDetails = viewGroupDetails(group);
//Then
groupDetails.verifyStatus("Partial Application*");
}

Validation of results

• Test can validate success against html elements. For example, Changing status of an account and checking for the new status being displayed to the user.  Doing validation against the UI is faster and takes less maintenance than validating tests with dbunit.  Some existing test do validate with dbunit, so the following information is left here for reference on those tests:
• a test can include dbunit validation based on changes between the initial database state and the final database state as captured above. When validating against the database, here are a few guidelines:
  • Get a data set for only the tables relevant to the test case verification. This reduces the chance of spurious results and also reduces the complexity of the test case.
  • For our framework, any table you are inspecting needs to be included in the map "columnsToIgnoreWhenVerifyingTables" found in the DbUnitUtilities class. When adding a table to this map, identify the columns for the table that should be ignored. Columns to ignore typically include generated ID counters and dates.
  • Before comparing the results of the expected and actual data sets, the results may need sorted to ensure a consistent order. For some transaction tables, the data may be stored in the database by multiple threads, which results in different ordering per test run. To compare results for these types of tables, try sorting on other more stable information such as amount/fee.
  • To debug the expected and actual data sets, use the printTable method found in the class DbUnitUtilities.

Executing Acceptance Tests

See Running Acceptance Tests for details on how to run acceptance tests.

Test Categories

TestNG has the capability to categorize tests into groups. For our acceptance test framework, the tests we're writing now are all part of the `acceptance` group. Each feature area also has a associated group - e.g. `client`, `loan`, etc. Finally, the build validation tests or "smoke" tests all have the test group "smoke".  Smoke tests are the tests run by default on a developer's build, whereas all acceptance tests are run on the continuous integration server.  The smoke group should be used sparingly. 

Smoke tests or other categories can be grouped so all tests within a single test class are run. However, many times you might only want a subset of tests to be run during the smoke tests, as each test extends the time of each developer's build. To make a single test method run during the smoke group, you need to add the testng groups parameter before your test method. e.g. @Test(groups = "smoke"). Also, when flagging a single test method, the BeforeMethod and AfterMethod in your test class must always be run by adding (alwaysRun = true). See ClientTest.java for an example.

Existing Helpers

Troubleshooting

• Skype installs a Firefox extension that can cause Selenium RC to crash. Workaround is to remove the Skype extension from your Firefox or Chrome instance.
• If your test runs correctly when ran alone but fails with the error message "ERROR Server Exception: sessionId should not be null; has this session been started yet?", make sure that you have a @ContextConfiguration line before your class definition (see ClientLoanRepaymentPeriodTest).
• If your test doesn't run together with the other suites, make sure that the class name matches one of: Test, Test or *TestCase.