Mifos BI Deployer Spec
Mifos Business Intelligence Deployment
As a Mifos systems administrator I want an easy way to deploy Mifos Business Intelligence code so I can effectively manage my Mifos reporting solution.
As a Mifos BI developer I want a fast and reliable way to develop Mifos Business Intelligence code so I can quickly deliver new BI features and bugfixes.
The Mifos BI software doesn't currently have a "build" per-se (no code needs to be compiled), the reports and ETL jobs simply need to be installed on a pentaho server. But we need some amount of automation (scripts, installer, whatever) to make sure that the Mifos BI software can be reliably deployed and updated.
Terminology
- BI = Business Intelligence
- "Install" may be considered a synonym of "deploy" in this spec.
Mifos BI code
includes:
- Reports
- Tally Integration (reports that output XML for tally) - .jar file, xaction, etc.
- MixXRBL (outputs XML)
- Mifos
- PPI, loan officers, etc.
- Database schema, data
- Kettle Jobs
Goals
- allows automated, simple installation of Mifos BI code
- initializes the data warehouse tables (if they don't already exist)
- updates data warehouse tables, if necessary
- loads initial data necessary for Mifos BI code to function
- uses only FLOSS tools
- provides Mifos BI code "releases" with a version number
- installs reports into a pentaho server
- updates existing reports, if necessary
- installs Kettle jobs into a pentaho server
- updates existing Kettle jobs, if necessary
- user may specify pentaho installation location (default is /opt/pentaho)
- works within a continuous integration system
- runs on Windows, Mac OS X and GNU/Linux (at least Ubuntu)
- Ubuntu is the primary target
- assumes java, MySQL, pentaho, etc. are already installed
- pentaho shall be installed separately by sysadmin
- but hopefully to a standard location, like /opt/pentaho
- pentaho shall be installed separately by sysadmin
- useful in Mifos hosted/cloud deployments as well as community/external deployments
Non-Goals
- i18n/L10n. The deploy procedure will be English only.
- install prerequisites
- install java
- install MySQL
- install pentaho
- provide a GUI (although this may be useful)
Development
Install/Deploy steps for a development machine
Upgrade steps for a development machine
- ...
- ...
Production
Install/Deploy steps for a production server
- install prereqs
- machine instance, OS, java, mysql, pentaho
- deploy Mifos BI software
- copy ETL jobs, reports, into appropriate places on server
- create data warehouse database
- schedule ETL jobs to run at regular intervals
- configure
- set up JNDI db connections
- startup/shutdown scripts
Upgrade steps for a production server
- update configuration, if necessary
- copy in new versions of BI software
Validate Environment
- for dev: run automated test suite
- Can connect to Pentaho console
- Can execute report
- report is pulling the right MFI data
- we can view the right versions (q: how to we verify report version?)
Questions
Testing
How shall we test the installation? Would an idea be to install a vanilla pentaho environment, run the installation/deployment and then confirm everything runs?
It seems obvious but I think there are already a few 'add-ons' to be taken into account... e.g. (1) Tally export... Udai did a java class to use freemarker and (2) Graymatter did some config for report parameters. Would these add-ons be part of installation or follow on from a customised Pentaho installation.
Priority
We need to prioritize having this procedure for our hosted customers (cloud) and then the procedure for those in our community who wish to deploy these reports themselves. Is this possible? Is the cloud process basically a subset of larger deployer that we can knock out and break down stories/cards for first?
Configuration
In Mifos, we have a custom properties file where you say how many decimals you want to be displayed for example.
How will this be handled in Pentaho when we deploy? Does each customer choose certain configurations, or do we have a standard file we apply to everyone?
Or will there be some things we set as default, while others such as decimals can be configured by the MFI.
Eventually, we want to allow consistency across Pentaho and Mifos.
Ideas
Designed for the Mifos cloud reporting environment
- Ubuntu GNU/Linux 10.04
When porting the installer to other platforms, the install code would (ideally) be mostly reusable.
Potential execution targets
- Java 6 version 1.6.x (Java or Groovy source only, please)
- Bash shell
- Python 2.6.x
May make use of environment or configuration variables
- JAVA_HOME - path to root of Java runtime installation
- BI_HOME - path to root of clone of Mifos "bi" git repository
- path to pentaho installation
- mysql connection information
Ideas from Graymatter
- create a Pentaho plugin to help with installing Mifos BI software
- update installed Mifos Intelligence Suite by asking a central server if updates are available
- assumes build script, source control for plugin
- what about kettle/PDI? (ie: ETL jobs)
- script that copies stuff into the right folders, for example, reports, ETL jobs
- buildscript (aka, install/deploy script, like an Ant build.xml file)
- has anything special been done in graymatter reports which requires doing anything special during the install/deploy of BI software?
Issues
- initial spike: MIFOSBI-56