The following consists of tips and guide guidance for migrating historical data into Mifos via Selenium scripts are that are based off of the approach outlined by Sam Birney in this mailing list post. He used this technique while migrating data at Al Majmoua in Lebanon. This approach was utilized by Natu Launchande while deploying Mifos at Associacao Progresso in Mozambique.
...
First of all, watch this two - minute intro at the top of this web-page: http://seleniumhq.org/
...
- Do your base mifos configuration and export a clean database.
- Export data from the legacy MIS into an excel spreadsheet. Save the spreadsheet as a tab-delimited unicode text file (in our case), but there are many different ways to kill this rat (thanks for the colorful expression).
- Use Selenium IDE to record the Selenese code for creating a new client, approving him/her, creating a loan, approving the loan, disbursing the loan.
- Export this Selenese into the programming language of your choice, whatever is easiest for you to work with. I chose python because it is very good at parsing text files and manipulating strings and lists and because i am reasonably practiced at it, also, because I am reasonably experienced with it.
- Edit the exported code to basically do a for loop that reads each line of your exported data file and then performs the actions you recorded in the UI, passing in variables from the fields you read from the file.
- Run the script, test, make changes, revert to your exported database, run again, etc., repeat as needed, voila, finito!
...
Scripts used for the Al Majmoua deployment can be downloaded here. (Note: these are very specific to one MFI (because of configuration, custom
fields, business rules, etc.) so I think it is better to record & write and write your own selenium scripts for other MFIs).
...
No major issues came up as problems, but there have been tons of little details...things like fields we don't want to be mandatory that currently are, not being able to disburse loans for a future date, meeting schedules, etc, but we found a solution for all of them,one by one.
...
It only took me a couple days to get the basic client data migration automated - a few more days for individual loans and a few more days for group loans (which were a little trickier, but by then I had a good feel for Selenium). In fact In fact the basic idea was working within a few hours of starting, all those days were just for the various details of this or that field or for Mifos quirks.
Unfortunately it is not a generally reusable general re-usable tool at this point because it depends a lot on the specific configuration. For example, for example we are using Group Lending with Individual Monitoring, Loan Schedule Independent of Meeting Schedule, no Centers, etc., which all affect the UI and therefore the scripts.
Did you do the open balances automatically automatically? If yes, what was your approach. ?
We exported the principal remaining on the loan and the interest remaining on the loan as two separate values. Then the script entered the remaining principal as the loan amount in mifos and used a formula to calculate the effective interest rate for the remaining loan. The formula, for flat interest for monthly loans, was:
(((remaining interest) / (remaining principal)) * 12 * 100) / (installments left)
...