Mifos Configuration Design Considerations, Goals and Discusssion

Design Considerations

Current Implementation

In the current implementation, configuration data is stored in SQL format in latest-data.sql. Updates to configuration data (along with updates to the database schema) are made using update_to_xxx.sql SQL scripts.

Two points worth considering are: ---------------------------------

1. the value of storing configuration data in the database vs. configuration files
2. if data is stored in the database, the value of defining configuration data in SQL format vs. some other format

Beginning with point #2: storing configuration data as SQL vs. some other format

Some benefits of defining configuration data as SQL:

• no extra code needs to be written to load configuration data into the database. Default configuration data is stored as executable SQL which inserts it.
• updating configuration data reuses code for automating schema updates. This mechanism is well defined and reliable.

Some drawbacks of defining configuration data as SQL:

• configuration data (in its current form) is hard to read and understand
• configuration data is tied to the current database schema (changing the schema means changing the format of the configuration data even if the data remains the same)
• configuration data (in its current form) has many hard coded database identifiers which cause problems when data is customized and then subsequently updated (the Grameen Koota case)

One of the reasons that configuration data is currently hard to read and understand is that database structure is intermingled with the configuration data. If the configuration data were separated from the database structure, then it could be expressed in whatever way would be most clear to end users and developers. Artifacts like database identifiers could be removed and the problems with hard coded identifiers could be avoided.

The cost associated with moving configuration data to a format other than SQL is that then a mechanism for reading the file, storing the data into the database and updating the data must be written. Using a standard format like XML or properties files means that the cost of reading the file is low, but there would still be some cost associated with saving the config data into the database and creating an update mechanism.

Point #1: storing configuration data in a database vs. an external configuration file

Benefits of storing configuration data in a database

• Load balancing is simplified. Configuration data can all come from a central database avoiding potential problems with having different configuration files on different servers).
• Data backup. If configuration is all in the database server, then backing up the database captures all the configuration data. If a configuration file (external to the WAR) is used, then it must also be backed up in order to capture the configuration data for Mifos.

Benefits of storing configuration data in configuration files -------------------------------------------------------------

• values can be easily updated by editing a text file with no need for a UI or a mechanism to load the values into the database
• new configuration values can be easily added (this could also be accomplished in the database if configuration data were stored in a generic way)
• a well established mechanism exists to support overriding and updating (using Apache Commons Configuration)

If load balancing and database backup are deemed to be significant issues, then another option to consider is to store data in the database and use configuration files to load that data. A mechanism could be created for exporting a configuration file containing editable configuration data. A user could then modify the configuration and then import the file to update the configuration values in the database. Also, the default data contained in latest-data.sql could be moved to a configuration file (or files) that are read once at startup as latest-data.sql is currently (problem with that design is that latest-data.sql is not read on startup).

Design Goals

• Mifos should provide default configuration data that can be customized in such a way that it is easy and safe to preserve customizations while upgrading to a new version of Mifos
• Localization of text in Mifos should be standardized as much as possible to use a single format and mechanism
• Customization of text in Mifos should be standardized as much as possible to make it easy to add customization where needed
• Mifos code should be simplified where possible by eliminate unnecessary database tables and database access
• Configuration files should be easy to read and be self documenting

Types of Configuration in Mifos ------------------------------- The following types of configuration data have been identified in Mifos:

• configuration data that should be set once at install time (and would cause problems if changed post install)
• configuration data that can be added to but not deleted from
• configuration data that can safely be changed while Mifos is running

Using Configuration Data Files

Most configuration data is currently stored in the database and is defined in SQL script files (such as latest-data.sql). This has led to problems such as an MFI customizing the SQL data and then subsequent Mifos versions adding to or changing the SQL configuration data such that the MFI customization is overwritten.

Configuration data will most likely be divided between multiple files with different purposes. For example:

• a file for default configuration data which will be loaded into the database. This would include General Ledger Codes (GL Codes) and Customizable Attribute Lists (aka Lookup-Values).
• a file for install time configuration. These are values that should not be changed after Mifos is up and running (for example Fiscal Year values and Accounting Rules)
• a file for configuration data that can be changed for a running Mifos instance (but is not changed often enough to justify a GUI) (for example a Session Timeout value)

By using one configuration file with default values (contained in the WAR) and a separate configuration file with customized values (that override default values), there is an simple and safe path for updating and adding to the set of default configuration values without disturbing the customized values.

Using Database Based Configuration ---------------------------------- If configuration data remains in the database, then it could be reorganized in order to meet some of the design goals supported by external configuration files.

Specifically, separating default configuration from customized configuration would make an easy path to update or add default values without disturbing customized values. This could be accomplished with database configuration by separating default values from customized values. Customized values would override default values. In this case default values could be freely update from one version of Mifos to the next.

This could be implemented using an Apache Commons Configuration backed by a database table rather than a file. If named configurations were used there could be a "Default" configuration with containing all values and then a "Custom" configuration containing only config values that had been customized. Value lookups could check the custom value first and if not found then get the default value. If values were cached (possibly with a in-memory BaseConfiguration) then it could be populated using the same scheme and updated if values were edited.

Configuration data stored in a uniform way (eg. using a Commons Configuration) would then open the possibility of creating a generic UI that could support automatic layout of config data such that when a new config item was added it would automatically show up on the page.

To deal with issues such as which values can be changed or not after Mifos installation, something like a ConfigurationItem class could be created which would hold then lookup name of the configuration data item, whether it is editable and possibly other information like a properties name for a localized label that could be used in the UI.

Localization

Currently web page text is stored in properties files while server-side text is stored in the database. This project will move localized text to properties files while having customized text (editable through the UI) stored in the database.

Having localized text externalized to properties files as much as possible should simplify the localization process by having all text to be translated in files of the same format rather than having to manage tracking translation data in the database and work to move it in and out of the database.

There are a number of cases in Mifos where lists of values are stored in the database rather than a simple enumeration. For example WeekDaysEntity objects store the days of the week in the database whereas a simple enumeration could do the same job. The WeekDay enumeration has already been created to replace it, but support for localized text for the WeekDay enumeration still needs to be added.

Label localization and customization could be supported by creating an enum of labels that referenced a localized label for the label (to present in the UI) and a localized property name. A Commons Configuration object could then track label customizations using the same property name that was used for the localized (default) text. For example:

public enum CustomLabel {
    ADDRESS1,
    CITY,
    STATE
}

String lookupLabel(CustomLabel label) {
     // look for the customized version of the label first
     String label = labelConfiguration.getString(label.toString());
     if (label != null) return label;

     // if no custom version, then return the default localized version
     return getLabelFromResourceBundle(label.toString());
}

labels.properties (resource bundle)

CITY=city

labels_es.properties (resource bundle)

CITY=ciudad

Tools/Libraries

This project will most likely use the Apache Commons Configuration library to support reading and managing configuration files. It includes support for XML based configuration files (as well as various other configuration sources) and includes features we need such as configuration value overriding.

Looking forward, it also integrates with Spring configuration files. This integration would allow a simple presentation of basic configuration data (appropriate for an MFI) in a Commons Configuration file and more elaborate application configuration (appropriate for a Mifos Specialist) in a Spring configuration file while allowing Spring to access the Commons Configuration data.

It is also possible that we will want to begin making use of some Spring Framework functionality.