Data Migration - Historic data - no updates to be made
Current Status
- With the goal of adopting one standard, multi-use, configurable, easy to use tool for migrating historical data, the community is working to extend the SunGard Data Migration Toolkit
Introduction
Data Migration is typically about taking data from a source system and getting it into a destination system. This can be achieved manually, or through automation or some mix of both.
Data migration typically involves:
- analysis of source data
- extraction of source data
- transformation of source data into another form
- validation and repair of transformed data
- loading of data in the desitnation system (or new application)
- verifcation of loaded data and use of new application
Migrating to Mifos
In relation to mifos, data migration is used to get a customer off their existing MIS approach and onto mifos so they may use the mifos application.
There is an existing deployment methodology which involves a data migration stage of which three possible approaches are outlined:
- No Data Migration
- Opening Balances Only
- All Historical Data
No Data Migration
- Approach is applicable for new MFIs
- MFIs is small client base who may want to upon rollout open all new accounts in Mifos and run in parallel with the old system until all clients are managed in Mifos. (Low Impact)
Opening Balances Only
- Only migrate client data and opening balances for all loan and savings products. (Medium Impact)
Warning
Employing this approaches means dropping all historical data and as a result the Datawarehouse and historical based reports are not useful for these accounts out of the box.
As mifos doesn't really support opening balance for loan/savings accounts, and there might be issues around 'adjustments of last transactions' as mifos at present can make assumptions about the first transaction being a disbursement (with respect to loans)
Model Opening Balance for loan and savings accounts
If opening balance approach is useful and used often, we should build this capability into mifos domain model and services and as such when migrating data into mifos, avoid possible problems with other behaviour in the system.
All Historical Data
- Migrate all data into Mifos (High Impact)
Warning
As mifos (through an API or in model) doesn't really support the creation of loans, savings, client etc with historical data; Employing scripts that go straight to the database schema have being the only way of achieving something like this.
Model creation of clients, loans, savings etc with respect to historical data
We should build this capability into mifos domain model and services and as such when migrating data into mifos, avoid possible problems with other behaviour in the system.
Analysis of approaches used in past
It would be useful to track what approaches have being used in the past and why and any problems encountered while analysing source data/application (see Existing Data Migration Tips/Problems for some examples of issues). Ryan Whitney gave a useful overview of the data migration process that was employed for a customer recently and what approach they went with and showed what they used to implement this also.
MFI | Size/Tier | Organisational Details | Approach Desired | Approache Used | Notes |
|---|---|---|---|---|---|
Customer X | Tier 3 | ~100 office/branches | unknown | Opening Balances |
From keeping track of past and current migration details and we can better understand how to add value to mifos to make migrations a little easier and consistent hopefully.
Some Opinions
In Existing Data Migration Tips/Problems some issues were noted that caused problems. Some of the issues were around conceptual differences between source and destination systems and as such source may not have information for putting into destination and vice versa, mifos doesn't support a concept that is allowed in source system. If this concept was common enough in microfinance then would it be fair to say that mifos should try to model/support this concept?
Other Sources/links
Existing Data Migration Strategy