How to upgrade datasets - DatasetUpgradeUtil

Background

A Dataset is a dump of the entire contents of the Mifos database, xml-formatted by DBUnit. It is used either to initialize the database state before running an acceptance test, or to validate its expected state after the test. When the database undergoes an upgrade (either a schema change, a change in initial data, or some other transformation of existing data) the current DBUnit datasets may no longer be valid and may either cause tests to fail or abort in error or give erroneous results. The procedure outlined below updates the datasets to reflect current schema and default data.

DataSetUpgradeUtil

DataSetUpgradeUtil is a utility that upgrades DbUnit XML data set files from a previous data version to the current database version of Mifos. It can update an individual data set file or an entire directory of data set files at once.

The utility is located in application/src/main/java/org.mifos.framework.util.DataSetUpgradeUtil and works like this:

Loads the dataset into a fresh database using the previous data schema.
Uses the Mifos application's initialization procedures to upgrade the schema and apply any data changes needed to bring the data up to the current version.
Uses DBUnit to dump the data, replacing the original dataset.

Preparing for DatasetUpgrades

The easiest way to run this program is from within Eclipse, by right clicking the file DataSetUpgradeUtil.java and choosing "Run as Java Program". By clicking "Run configurations..." you get to set up arguments for the program.

All the arguments are listed when "h" or "-help" is supplied as a (command line) argument to the file.

By default the database "mifos_gazelle_acceptance" is used to load and upgrade the data. Its default user/password is "mifos/mifos". You can override this configuration in your local.properties file.

application/src/main/java/org/mifos/config/resources/acceptanceDatabase.properties

these properties prefix as "acceptance.database.*"

Example: whole directory upgrade

This example will upgrade all the *_dbunit.xml.zip files in the dataSets directory.

Set program arguments to:

-a <path-to-project-root-directory>/acceptanceTests/src/test/resources/dataSets -u root -p mysql

Example: single file upgrade usage

Let's say we want to upgrade a single data set filed named "acceptance_small_008_dbunit.xml.zip" that is at database version 213 to the current database version, 215.

Assuming that defaults are ok, set program arguments to:

-f <path-to-project-root-directory>/acceptanceTests/src/test/resources/dataSets/acceptance_small_008_dbunit.xml.zip -u root -p mysql

Step-by-step instructions how to upgrade dbunit data sets after adding database upgrade

Make sure your database upgrade is properly added to the current changelog file located under db/src/main/resources/changesets

Create up-to-date schema dump of the actual mifos database (without your upgrade)

Download the newest Mifos zip file
Unpack it and cd to the new directory

Recreate the empty Mifos database:

$ mysql -u mifos -pmifos -e "drop database mifos"
$ mysql -u mifos -pmifos -e "create database mifos"
$ mysql -u mifos -pmifos mifos < db/sql/base-schema.sql
$ mysql -u mifos -pmifos mifos < db/sql/base-data.sql
$ mysql -u mifos -pmifos mifos < db/sql/init_mifos_password.sql

Create mifos-db.properties

$ cp db/mifos-db-template.properties db/mifos-db.properties

Edit mifos-db.properties to look like:

username=mifos
password=mifos
defaultSchemaName=mifos
databaseHost=localhost
databasePort=3306

Run expand upgrades:
```
$ ./db/bin/expand_db.sh
```
Run contract upgrades:
```
$ ./db/bin/contract_db.sh
```

Make the dump:

$ mysqldump -l --no-data -u mifos -pmifos mifos > new_schema.sql

Run DataSetUpgradeUtil to update dbunit data sets

DataSetUpgradeUtil -a PATH_TO_MIFOS_REPO/acceptanceTests/src/test/resources/dataSets -u mifos -p mifos -d mifos -s PATH_TO_NEW_SCHEMA_DIR/new_schema.sql

Commit/Push updated dbunit data sets to the repository