These notes are from a 26 March, 2007 conversation with Arun Kanabar at MFTech (the Mifos specialist working on the Grameen Koota migration).

Overview

Grameen Koota is in Bangalore, development is being done in Gujarat.

3 software developers have been working on the Grameen Koota migration plus a number of project coordinators based locally at each branch.

All Grameen Koota branches are now having Mifos iteration #15 installed.

The migration is being done branch by branch for about 40 branches. Each branch uses both their existing system, called Portfolio Manager (PM), and Mifos in parallel for some length of time. Each week, data is re-migrated into Mifos on the day when no meetings are held. This is done to capture any changes made in PM during the week, since staffing isn't available to enter changes into both systems. This means the migration procedure must be relatively fast and repeatable.

The MFTech migration tool being used for each branch was ready in October.

Requirements for migration were finalized in October, but on a branch by branch basis, how interest is calculated has changed and meeting schedules have changed since then. This makes the specification of the migration tool a moving target. Custom migration scripts need to be maintained on a branch by branch basis (since each branch may have a different set of changes).

6-7 Grameen Koota branches have currently been migrated and are using both Mifos and PM. In another 1-2 weeks some of these branches might start going live with Mifos and stop using the PM system.

It is estimated that it takes 5-6 months for a Mifos specialist to come up to speed on Mifos. For an MFI, it takes 3-4 months before they can start a migration since they need to understand their own system first.

For a Mifos specialist to generate a requirements document for a migration is an expensive operation because of the time required for a Mifos specialist to first understand the MFI's existing system. It is more likely to work out if there is someone internal to the MFI that already knows their system that can generate the requirements document that a Mifos specialist could then work with to do the migration.

Infrastructure Challenges

Staffing for managing PM alone was tight, and after Mifos was introduced the staffing was further reduced. Initially the staff was experienced and could quickly (within minutes) identify issues with discrepancies between Mifos and PM. Now errors are not identified for up to 2 weeks.

Migration is complicated by different branches being so spread out that people speak different languages from one branch to another.

Project coordinators who know the local language are used by MFTech to work on site at the branches being migrated. These people often come and go, which increases the challenge of the migration.

Technical Challenges

Lookup Value Overwriting

They started migration with Mifos iteration #7 and continued through iteration #15. One of the difficulties was with lookup values. An example was given of defining a new loan purpose to match a new loan purpose created in the PM system. This new loan purpose would be inserted directly into the database using SQL insert statements. Problems came when new lookup values were added in the next Mifos iteration. The new lookup values would have hard coded ids (rather than querying for the max id) and so the new ids introduced in a Mifos iteration would overwrite ids that had been created in the live Mifos instance. So there is an issue of how to handle on site customizations such that they can integrate with evolution of the Mifos built in values.

Mifos Schedule Population

The calendar for loan and customer schedules is populated by a batch job that Mifos runs on the last day of the year to generate the calendar for the following year. When migrating data through Jan. 2007, they had cases when the batch program had not run so the calendars for loan and customer schedules were not populated for 2007. They have taken care of loan schedules, but still need to handle customer schedules. They are adding code to their migration tool to insure that schedules for 2007 are created. If for some reason the Mifos batch job for generating schedules does not run on Dec. 31, then there is nothing to tell the Mifos administrator that something is wrong and no way for the administrator to run the batch job through Mifos. There should be some kind of alert or way for an administrator to know that the loan and customer schedules have not been generated for the next year. Some of the problems faced during the Grameen Koota migration were related to deficiencies in how Mifos is implemented. For example it seems like there should be a better way to handle schedules than the annual generation scheme. Improving on how loan schedules are handled in Mifos could have saved time and effort during migration.

One impact of the schedules not being correctly generated (ie. the batch job not running on Dec. 31) is that collection sheet generation fails. It would be useful if Mifos handled issues like these more gracefully since during the migration process, they should not have to be concerned with issues like this related to Mifos internals. These were the most difficult kinds of issues they had to deal with.

Data Mapping

In general the mapping of data at Grameen Koota from PM to Mifos was not too difficult because Mifos was designed using Grameen Koota as the base client, making sure that the Mifos functionality would fit the needs of Grameen Koota. Another client that Mifos was not initially targeted for might be more difficult.

There is a "setup" method in the java code for the MFTech migration tool which is run on start up and does the cleaning of the data. For example, every center has groups with 5 members in each group and they want to merge the groups such that there are 10 members in each group. The setup module handles the merging of the groups.

Some loan repayment data in the PM system was bad data. There were loans that showed a late payment, but Grameen Koota said there were no late payments. In this case, the loan repayment schedules had to be regenerated to ignore the data that said there was a late loan and create loan repayment data in Mifos that showed that everything was repaid on time.

An example of a problem they faced was that a branch had more than 99 centers. Rather than having center "100", they had center "A0". This was a problem since Mifos only accepts numeric ids.

Reports

Reports customers are very particular, the reports from Mifos need to look exactly like the reports they are used to.

Tools

There were lots of records involved in regenerating the loan repayment schedule. Initially the Kettle ETL tool was used to do this, but they had trouble with Kettle crashing while doing this so they started coding the operation directly in java.

Most projects done by MFTech involve XML to some extent, so they are very comfortable working with XML.

One issue to be aware of with XML is that the MFI will not be familiar with XML. So you can't ask the MFI to provide XML data, but you can ask them to give Excel data for example and they will write visual basic code or whatever is required to get Excel data. So the conversion from Excel (or whatever format is provided) to XML would need to be done by the Mifos specialist.

The target is to get the customer to take ownership of the data he is providing.

If you divide the migration process into two parts, source to intermediate and intermediate to final (data in Mifos), most of the problems come in the first part of the process, the source to intermediate step. They would get the most value out of tools to aid in the source to intermediate step.

Although it would be most useful to have tools that would help with the first part of the migration process maybe it is not possible to create generic tools to help with that part. The problems may be too specific to each migration. As more migrations are done, comparisons can be made from one to the next to see if there are common problems that a tool could help solve.