Mifos 2.2 Potentially Large DB Changes and PreUpgrade Option

Moved to http://mifosforge.jira.com/wiki/display/MIFOS/Mifos+2.2+Upgrade+Instructions

Additional Mifos 2.2 Upgrade Notes

Mifos 2.2 contains a substantial amount of database related upgrades.  If you are a small or medium user, this shouldn't amount to much upgrade time and you can ignore this note as the normal mifos upgrade takes care of everything.  If you are a large/very large user it may matter.  See below to get an idea of how long the upgrade will be for you.

Database Changes

Remove unused currency indexes, add performance indexes and some timestamps.

Benefits

Increase general speed of creating schedules and making payments.
Increase speed of date based processing of schedules and account transactions e.g apply holiday batch job, any reporting on today's transactions
For schedule and customer attendance data, allow the possibility of a speedy incremental ETL job for data warehousing

Also, as a side-effect, there was a 12 to 15% decrease in InnoDB database space usage.

Indication of How Long the Mifos 2.2 upgrade is likely to take

1. Run the following query against your production database to get your 'Data Size' (substituting your mifos database name if it is not 'mifos'):

select sum((t.data_length) / 1048576) as data_size
from information_schema.`tables` t
where table_schema = 'mifos'

2. Looking at the table below, find where you sit against the 3 MFI datasets that we tested against
e.g if your data_size output is 1,000Mb expect the upgrade to take roughly 12-18 mins... if 10,000Mb then approx. 3 hours

Server hardware and software will differ and that may affect time taken. The tests below were run on a Windows Vista Laptop 4Gb mem 2.53Ghz Core 2 Intel and EC2 Cloud ubuntu instances. There didn't seem to be much in it between the windows laptop and the EC2 cloud instance performance.

Data Size

Pre Database Size

Post Database Size

Pre/Post DB Space Diff %

Time Taken

312Mb

691Mb

589Mb

-14.76

4 mins

1,945Mb

4,108Mb

3,519Mb

-14.34

35 mins

51,103Mb

99,825Mb

87,300MB

-12.5

1,046 mins (17.5 hrs)

Processing the Database Changes Pre-Upgrade

You should run this separate database upgrade when no-one else is using the system (or mifos will appear really slow to them)

If you decide the upgrade time is too long for you (or you want to pay forward some of the benefits highlighted above), you can run the database change part of the upgrade separately prior to upgrading to mifos 2.2 as the changes work fine on mifos 2.1.x.  Here are the steps.

1. Download the mifos 2.2 zip file and unzip it in a directory of your choosing.

2. Go to the unzipped directory and then go into its db directory.
    Copy the file mifos-db-template.properties to mifos-db.properties
    Edit mifos-db.properties with your mysql database informaton.  Mine is:

username=mifos
password=mifos
defaultSchemaName=mifos
databaseHost=localhost
databasePort=3306

3a. Windows
in a command prompt shell (cmd.exe), go to the db directory mentioned in 2. above and enter the following command:

call bin\expand_db_relG_special.bat

OR
3b. Ubuntu

In the db directory mentioned in 2. above enter the following command to allow you permission to run;
chmod 777 bin/*.sh

Then enter the follow command to run the database changes (if you see 'Liquibase Home is not set.' you can ignore it):

bash bin/expand_db_relG_special.sh

4.  Just leave the upgrade running.  If it is still running coming up to the time the system is need again then

    i) stop mysql which will cause the current upgrade step to fall over (no data damage though).  
    ii) start mysql again
    iii) kick off this upgrade again in your next available down-time slot.  It will continue from where it left off (and so on till it finishes)

In the event that any of these changes causes problems with mifos (they shouldn't) it is possible to undo the changes.  The 'rollback' sql can be found in the rollback section for each change in file changesets/changelog-PreRelease_G.xml

5. When you come to upgrading to mifos 2.2, the mifos upgrade process will recognise that the database changes have been done.