...

Page objects should always have at least one verify method, verifyPage(). This should call this.verifyPage(pageName), where pageName is the name of the jsp file (without .jsp). This method will use Selenium to verify that the jsp page's unique hidden form field matches what is expected. See CreateLoanAccountsSearchPage.java for an example.

Test Data

Test data is There are two test data forms currently - acceptance_test_dump.sql and dbUn datasets. DbUnit is deprecated. As part of MIFOS-4590 , there is currently an on-going effort to refactor the current suite of acceptance tests to minimize dependency on the DB unit based XML datasets.

Test data are stored in the following locationlocations:

db/src/test/resources/sql/acceptance_test_dump.sql
acceptanceTests/src/test/resources/dataSets 

There are two different kinds of data sets, validation and start data sets. Validation data sets are used to make sure that at the end of a test, the database looks correct. As of Jan 2011, we are working to eliminate the use of validation data sets.  Start data sets are used to set up a suitable environment for each test case and they're usually loaded at the very beginning of the test.

Every data set consists of two files:

• a txt file that describes what the data set consists of and/or its purpose.
• an xml file which is the DBUnit representation of the database (a dump).

Rather than generate all necessary test data for every acceptance test, we will create a basic set of test data that can be loaded via DBUnit. The details of the dataset acceptance_small_001 are described in Test Data Set. acceptance_default_003 is the default or empty Mifos database.

Generating and importing test data set is documented at the DbUnitDataImportExport page.

Upgrading data sets generated from an earlier database version to the current database version is documented at the DataSetUpgradeUtil page . Read this thread for more information about the data set upgrade process.

Writing Acceptance Tests

Here is a brief outline of the steps to follow in order to create an acceptance test:

• load an existing data set (or generate a new one as described above)
• record acceptance test operations using Selenium IDE
• if you will validate against a final database state, then dump the final database state (at the end of the operation) and create a new data set using DbUnitDataImportExport as described above
• write the acceptance test java code (such as CollectionSheetEntryTest.defaultAdminUserEntersSingleLoanPayment) which: _ loads an initial database state _ includes transitions to the page objects which map to the pages traversed during the test and any required data entry
• use the acceptance test code (with initially nonexistent methods) to generate (via Eclipse) the empty classes and methods that need to be implemented
• using existing page objects as examples and the Selenium IDE generated code, fill in the page object code needed to make the test functional
• test name should be a combination of words which describes the main purpose of the test
• test should verify that data entered on the create page are displayed properly on the preview page
• while writing the test remember to use appropriate line spacing to make the code easy to read
• it's good practice to add comments in the code which explain what is verified at each step of the test
• add comment to the acceptance test with the link to the related test case

Executable Specification Style (added Jan 2011)

To make acceptance tests more clear in their intent, we will write tests to with a executable specification style that uses internal test drivers or helpers to separate the test into two layers - a test implementation layer and a lower helper or application driver layer.  The acceptance test is written using internal domain specific language (test helpers) e.g. "createClient" as developed by the test authors.  These tests will be written using a Given, When, Then structure.  "Given" is the set up of the test case, "When" is the activity between the user and Mifos, and "Then" is the expected result.  This style and the concept of layered acceptance tests is described in more detail in Continuous Delivery, Chapter 8.   The goal of including this style is to simplify test writing and maintenance.  While the tests will be similar to behavior driven tests, we will still write tests in a Java to take advantage of its full development environment.  Some of these helpers could use Service Facade or API's of Mifos to build entities to support more complex use cases. 

Here is an example of an existing test from GroupTest.java:

@Test(sequential = true, groups = {"group","acceptance","ui"})
 @SuppressWarnings("PMD.SignatureDeclareThrowsException") // one of the dependent methods throws Exception
 public void createGroupInPartialApplicationStateTest() throws Exception {
 initRemote.dataLoadAndCacheRefresh(dbUnitUtilities, "acceptance_small_001_dbunit.xml", dataSource, selenium);
 CreateGroupEntryPage groupEntryPage = loginAndNavigateToNewGroupPage();
 CreateGroupSubmitParameters formParameters = getGenericGroupFormParameters();
 CreateGroupConfirmationPage confirmationPage = groupEntryPage.submitNewGroupForPartialApplication(formParameters);
 confirmationPage.verifyPage();
 GroupViewDetailsPage groupDetailsPage = confirmationPage.navigateToGroupDetailsPage();
 groupDetailsPage.verifyPage();
 groupDetailsPage.verifyStatus("Partial Application*");
 }

Might be written something like:

\*
\* Given Center exists,
\* And there is a user with rights to create a group,
\* When I login in as user "mifos"
\* And I create a Group with generic parameters
\* And I view Group Details
\* Then Group status is "Partial Application"
*/

@Test(sequential = true, groups = {"group","acceptance","ui"})
// http://mifosforge.jira.com/browse/MIFOSTEST-300
@SuppressWarnings("PMD.SignatureDeclareThrowsException") // one of the dependent methods throws Exception
public void createGroupInPartialApplicationStateTest() throws Exception {
//Given
initRemote.dataLoadAndCacheRefresh(dbUnitUtilities, "acceptance_standard_dbunit.xml", dataSource, selenium);
Center center = createCenter(getGenericCenterFormParameters());
//When
login("mifos");
Group group = createGroup(center, getGenericGroupFormParameters());
GroupDetails groupDetails = viewGroupDetails(group);
//Then
groupDetails.verifyStatus("Partial Application*");
}

Validation of results

• Test can validate success against html elements. For example, Changing status of an account and checking for the new status being displayed to the user.  Doing validation against the UI is faster and takes less maintenance than validating tests with dbunit.  Some existing test do validate with dbunit, so the following information is left here for reference on those tests:
• a test can include dbunit validation based on changes between the initial database state and the final database state as captured above. When validating against the database, here are a few guidelines:
  • Get a data set for only the tables relevant to the test case verification. This reduces the chance of spurious results and also reduces the complexity of the test case.
  • For our framework, any table you are inspecting needs to be included in the map "columnsToIgnoreWhenVerifyingTables" found in the DbUnitUtilities class. When adding a table to this map, identify the columns for the table that should be ignored. Columns to ignore typically include generated ID counters and dates.
  • Before comparing the results of the expected and actual data sets, the results may need sorted to ensure a consistent order. For some transaction tables, the data may be stored in the database by multiple threads, which results in different ordering per test run. To compare results for these types of tables, try sorting on other more stable information such as amount/fee.
  • To debug the expected and actual data sets, use the printTable method found in the class DbUnitUtilities.

Executing Acceptance Tests

See Running Acceptance Tests for details on how to run acceptance tests.

Test Categories

TestNG has the capability to categorize tests into groups. For our acceptance test framework, the tests we're writing now are all part of the `acceptance` group. Each feature area also has a associated group - e.g. `client`, `loan`, etc. Finally, the build validation tests or "smoke" tests all have the test group "smoke".  Smoke tests are the tests run by default on a developer's build, whereas all acceptance tests are run on the continuous integration server.  The smoke group should be used sparingly. 

Smoke tests or other categories can be grouped so all tests within a single test class are run. However, many times you might only want a subset of tests to be run during the smoke tests, as each test extends the time of each developer's build. To make a single test method run during the smoke group, you need to add the testng groups parameter before your test method. e.g. @Test(groups = "smoke"). Also, when flagging a single test method, the BeforeMethod and AfterMethod in your test class must always be run by adding (alwaysRun = true). See ClientTest.java for an example.

Existing Helpers

...

Helper class

...

Description

...

CollectionSheetEntryTestHelper

...

Contains methods to:

• navigate to CSE page
• fill the form
• submit

...

BatchJobHelper

...

Contains methods to run all/some batch jobs.

...

OfficeHelper
(there are currently two such classes)

Contains methods to:

...

(DbUnit)

Procedure for maintaining acceptance_test_dump.sql 

If you want to make changes to the dump by using the application
Import all data from the dump into the database.
Use the application to setup/change (only) the data you want.
Export the data using following command.
mysqldump -u <your_user> -p<your_password> <your_database> --complete-insert --extended-insert --skip-add-locks --skip-comments --skip-quote-names > ~/acceptance_test_dump.sql
When commiting changes commit acceptance_test_dump.sql file as well.

If you want to make changes to the dump manually
Edit acceptance_test_dump.sql
When commiting changes push acceptance_test_dump.sql file as well.

DbUnit dependency

A new test group, no_db_unit, has been introduced to categorize tests that do not need DB unit dependency. Currently, the Maven POM for acceptance tests, ensures that all tests in the no_db_unit group are executed prior to executing the rest of the acceptance tests. This ensures that the no_db_unit group of tests do not inadvertently depend on the XML data sets used by the older tests.

In summary, if you are adding an acceptance test to the current suite, ensure that you look at an existing test in the no_db_unit category and see how the test data is setup & managed. Try & avoid loading your seed data from an XML dataset whenever possible.

DbUnit approach - deprecated and should not be used for new tests

There are two different kinds of data sets, validation and start data sets. Validation data sets are used to make sure that at the end of a test, the database looks correct. As of Jan 2011, we are working to eliminate the use of validation data sets.  Start data sets are used to set up a suitable environment for each test case and they're usually loaded at the very beginning of the test.

Every data set consists of two files:

• a txt file that describes what the data set consists of and/or its purpose.
• an xml file which is the DBUnit representation of the database (a dump).

Rather than generate all necessary test data for every acceptance test, we will create a basic set of test data that can be loaded via DBUnit. The details of the dataset acceptance_small_001 are described in Test Data Set. acceptance_default_003 is the default or empty Mifos database.

Generating and importing test data set is documented at the DbUnitDataImportExport page.

Upgrading data sets generated from an earlier database version to the current database version is documented at the DataSetUpgradeUtil page . Read this thread for more information about the data set upgrade process.

Writing Acceptance Tests

Here is a brief outline of the steps to follow in order to create an acceptance test:

• load an existing data set (or generate a new one as described above)
• record acceptance test operations using Selenium IDE
• if you will validate against a final database state, then dump the final database state (at the end of the operation) and create a new data set using DbUnitDataImportExport as described above
• write the acceptance test java code (such as CollectionSheetEntryTest.defaultAdminUserEntersSingleLoanPayment) which: _ loads an initial database state _ includes transitions to the page objects which map to the pages traversed during the test and any required data entry
• use the acceptance test code (with initially nonexistent methods) to generate (via Eclipse) the empty classes and methods that need to be implemented
• using existing page objects as examples and the Selenium IDE generated code, fill in the page object code needed to make the test functional
• test name should be a combination of words which describes the main purpose of the test
• test should verify that data entered on the create page are displayed properly on the preview page
• while writing the test remember to use appropriate line spacing to make the code easy to read
• it's good practice to add comments in the code which explain what is verified at each step of the test
• add comment to the acceptance test with the link to the related test case

Executable Specification Style (added Jan 2011)

To make acceptance tests more clear in their intent, we will write tests to with a executable specification style that uses internal test drivers or helpers to separate the test into two layers - a test implementation layer and a lower helper or application driver layer.  The acceptance test is written using internal domain specific language (test helpers) e.g. "createClient" as developed by the test authors.  These tests will be written using a Given, When, Then structure.  "Given" is the set up of the test case, "When" is the activity between the user and Mifos, and "Then" is the expected result.  This style and the concept of layered acceptance tests is described in more detail in Continuous Delivery, Chapter 8.   The goal of including this style is to simplify test writing and maintenance.  While the tests will be similar to behavior driven tests, we will still write tests in a Java to take advantage of its full development environment.  Some of these helpers could use Service Facade or API's of Mifos to build entities to support more complex use cases. 

Here is an example of an existing test from GroupTest.java:

@Test(sequential = true, groups = {"group","acceptance","ui"})
 @SuppressWarnings("PMD.SignatureDeclareThrowsException") // one of the dependent methods throws Exception
 public void createGroupInPartialApplicationStateTest() throws Exception {
 initRemote.dataLoadAndCacheRefresh(dbUnitUtilities, "acceptance_small_001_dbunit.xml", dataSource, selenium);
 CreateGroupEntryPage groupEntryPage = loginAndNavigateToNewGroupPage();
 CreateGroupSubmitParameters formParameters = getGenericGroupFormParameters();
 CreateGroupConfirmationPage confirmationPage = groupEntryPage.submitNewGroupForPartialApplication(formParameters);
 confirmationPage.verifyPage();
 GroupViewDetailsPage groupDetailsPage = confirmationPage.navigateToGroupDetailsPage();
 groupDetailsPage.verifyPage();
 groupDetailsPage.verifyStatus("Partial Application*");
 }

Might be written something like:

\*
\* Given Center exists,
\* And there is a user with rights to create a group,
\* When I login in as user "mifos"
\* And I create a Group with generic parameters
\* And I view Group Details
\* Then Group status is "Partial Application"
*/

@Test(sequential = true, groups = {"group","acceptance","ui"})
// http://mifosforge.jira.com/browse/MIFOSTEST-300
@SuppressWarnings("PMD.SignatureDeclareThrowsException") // one of the dependent methods throws Exception
public void createGroupInPartialApplicationStateTest() throws Exception {
//Given
initRemote.dataLoadAndCacheRefresh(dbUnitUtilities, "acceptance_standard_dbunit.xml", dataSource, selenium);
Center center = createCenter(getGenericCenterFormParameters());
//When
login("mifos");
Group group = createGroup(center, getGenericGroupFormParameters());
GroupDetails groupDetails = viewGroupDetails(group);
//Then
groupDetails.verifyStatus("Partial Application*");
}

Validation of results

• Test can validate success against html elements. For example, Changing status of an account and checking for the new status being displayed to the user.  Doing validation against the UI is faster and takes less maintenance than validating tests with dbunit.  Some existing test do validate with dbunit, so the following information is left here for reference on those tests:
• a test can include dbunit validation based on changes between the initial database state and the final database state as captured above. When validating against the database, here are a few guidelines:
  • Get a data set for only the tables relevant to the test case verification. This reduces the chance of spurious results and also reduces the complexity of the test case.
  • For our framework, any table you are inspecting needs to be included in the map "columnsToIgnoreWhenVerifyingTables" found in the DbUnitUtilities class. When adding a table to this map, identify the columns for the table that should be ignored. Columns to ignore typically include generated ID counters and dates.
  • Before comparing the results of the expected and actual data sets, the results may need sorted to ensure a consistent order. For some transaction tables, the data may be stored in the database by multiple threads, which results in different ordering per test run. To compare results for these types of tables, try sorting on other more stable information such as amount/fee.
  • To debug the expected and actual data sets, use the printTable method found in the class DbUnitUtilities.

Executing Acceptance Tests

See Running Acceptance Tests for details on how to run acceptance tests.

Test Categories

TestNG has the capability to categorize tests into groups. For our acceptance test framework, the tests we're writing now are all part of the `acceptance` group. Each feature area also has a associated group - e.g. `client`, `loan`, etc. Finally, the build validation tests or "smoke" tests all have the test group "smoke".  Smoke tests are the tests run by default on a developer's build, whereas all acceptance tests are run on the continuous integration server.  The smoke group should be used sparingly. 

Smoke tests or other categories can be grouped so all tests within a single test class are run. However, many times you might only want a subset of tests to be run during the smoke tests, as each test extends the time of each developer's build. To make a single test method run during the smoke group, you need to add the testng groups parameter before your test method. e.g. @Test(groups = "smoke"). Also, when flagging a single test method, the BeforeMethod and AfterMethod in your test class must always be run by adding (alwaysRun = true). See ClientTest.java for an example.

Existing Helpers

...

• Skype installs a Firefox extension that can cause Selenium RC to crash. Workaround is to remove the Skype extension from your Firefox or Chrome instance.
• If your test runs correctly when ran alone but fails with the error message "ERROR Server Exception: sessionId should not be null; has this session been started yet?", make sure that you have a @ContextConfiguration line before your class definition (see ClientLoanRepaymentPeriodTest).
• If your test doesn't run together with the other suites, make sure that the class name matches one of: Test, Test or *TestCase.

DB unit dependency

As part of MIFOS-4590       , there is currently an on-going effort to refactor the current suite of acceptance tests to minimize dependency on the DB unit based XML datasets. As a result, a new test group, no_db_unit, has been introduced to categorize tests that do not need DB unit dependency. Currently, the Maven POM for acceptance tests, ensures that all tests in the no_db_unit group are executed prior to executing the rest of the acceptance tests. This ensures that the no_db_unit group of tests do not inadvertently depend on the XML data sets used by the older tests.

In summary, if you are adding an acceptance test to the current suite, ensure that you look at an existing test in the no_db_unit category and see how the test data is setup & managed. Try & avoid loading your seed data from an XML dataset whenever possible.

Procedure for maintaining acceptance_test_dump.sql (DBUnit approach is deprecated and should not be used for new tests)

If you want to make changes to the dump by using the application
Import all data from the dump into the database.
Use the application to setup/change (only) the data you want.
Export the data using following command.
mysqldump -u <your_user> -p<your_password> <your_database> --complete-insert --extended-insert --skip-add-locks --skip-comments --skip-quote-names > ~/acceptance_test_dump.sql
When commiting changes commit acceptance_test_dump.sql file as well.

...

• Chrome instance.
• If your test runs correctly when ran alone but fails with the error message "ERROR Server Exception: sessionId should not be null; has this session been started yet?", make sure that you have a @ContextConfiguration line before your class definition (see ClientLoanRepaymentPeriodTest).
• If your test doesn't run together with the other suites, make sure that the class name matches one of: Test, Test or *TestCase.