Overview
Saving Collection Sheets is probably the heaviest non-batch Mifos transaction. A save transaction broadly breaks down into 2 areas.
- Assemble Object Model (AOM) - mostly by reading collection sheet data
- Save Assembled Object Model (SAOM) - by throwing lots of inserts, updates and sometimes deletes at the mysql database.
There has been quite a bit performance work recently on (AOM) above (including preloading the hibernate session cache). Not much on (SOAM).
There's some suggestions for areas of improvement below that I intend to have a look at... but, of course, anyone's welcome.
Possible Changes and Tests (using Sungard's load test facility)
(SOAM) Move GL (Financial Transaction) data and processing out of Mifos
When a financial transaction occurs in mifos a small amount of GL pertinent information about it could be passed onto some GL event queue and get picked up by some GL system that is subscribed to that event queue. The GL system could duplicate the current mifos way of GL (in a different database) or, more likely just convert the feed into a format to be passed on to a common GL package.
Testing on a single machine indicated that about a quarter of the total collection sheet save was to do with this area (could be more now). Of course, this would affect the 'teller' model part of mifos too.
Decoupling would also help scalability a lot as for big customers the GL processing and data could be on different server machine(s) to mifos.
For GK 2009, over half of the records in the database were GL related.
AFAICS from working with collection sheets, GL functionality in mifos is 'fairly' well isolated from the rest of mifos so might be straightforward to do. Would need to involve people who've been involved in GL implementations
(SOAM) Archiving Financial Transactions (if not moved out of mifos)
The financial transaction tables are big and hold the top 3 spots in terms of mifos row count (see table below). An easy way to see what performance benefit (or not) can be got from archiving financial transactions (for save collection sheets) would be to run a load test once (for baseline) then again with the following tables truncated (tables listed in order they should be truncated).
csv-table:::header: "Table Name", "Rows (2009)", "Rows (2008)", "%Growth", "Position (no. of rows)"
:widths: 15, 10, 10, 8, 10
"financial_trxn", "115,107,267", "64,911,013", "77.33", "1" "fee_trxn_detail", "9,682,363", "8,751,059", "10.64", "9" "loan_trxn_detail", "45,509,174", "26,435,860", "72.15", "3" "customer_trxn_detail", "7,018,839", "1,924,169", "264.77", "10" "savings_trxn_detail", "0", "0", "0", "n/a" "account_trxn", "52,528,019", "27,924,920", "88.10", "2"
(SOAM) Insert, Update, Delete Processing
Hibernate Batch options to allow multiple inserts / updates per request.
(SOAM) Removing Currency Indexes
Removing Database indexes and foreign key relationships from currencyId fields. There are a lot of these (over 20 in the collection sheet area alone). However, testing on a single machine indicated a very small improvement (0 - 3%). Might be a bit more using Sungard's load testing facility.