Batch jobs are a series of back-end jobs on a computer that are executed through predefined scripts.
Mifos is configured to run a set of batch jobs nightly at 12 am. The server must be kept on at night
for these processes to run.
Mifos currently has 9 batch processes:
The goal of this project was to replace legacy scheduling system based on a Java Timer class and a
custom code (scheduled_tasks table, dependencies between tasks and catch-up execution hardcoded
in the Mifos sources) with a well known frameworks designed for enterprise scheduling
(Quartz and Spring Batch). Moreover, as a part of this project a simple UI was developed to display
basic information about performed batch jobs and to allow some management activities directly
from the Mifos administrative area.
2. Original Mifos Scheduler
In the original Mifos implementation all scheduling was done by MifosScheduler class, which used
java Timer class to run tasks. This meant, that:
all tasks were run sequentially
there was no configurable support for tasks dependencies (the only dependency between
PortfolioAtRiskTask and LoanArrearsTask was hard-coded in TaskHelper class) except
manipulation of initial-time/delay-time in task.xml file
Mifos used its own configuration format for job scheduling (task.xml). Even if there was a
possibility to configure delay-time, there were several places where it was hard-coded to 1 day:
exception was thrown in MifosScheduler.registerTasks() method when delay time < 86400 (1
catch-up execution functionality from TaskHelper.executeTask() method performed task
execution for every missing day (even when we could set delay time > 1 day)
3. Quartz integration
We have decided to use Quartz as a reliable replacement for Timer used by the MifosScheduler.
However, we had to ensure that the old task.xml files are still supported and all scheduling
functionality works as before. This specifically includes:
It should be possible to define dependencies between tasks
All missed tasks executions should be run by scheduler before the next planned execution
Mifos Scheduler should reflect all changes made to task.xml file
It is not possible to achieve the above goals using Quartz only, because:
Quartz does not support dependencies between tasks
Quartz does not support automatic execution of missed triggers (except the last one)
In order to schedule the same trigger/job (with modified time of execution for example) the
previous instance has to be unscheduled/removed. All information about missed executions
is lost in such case.
Taking into consideration the above Quartz limitations we have decided to configure this
framework with the following settings:
This means that we treat Quartz as a simple Timer replacement. We do not use JDBC JobStore, but
this can be easily configured by changing appropriate properties. We have added the following
Quartz tables to Mifos:
Catch-up execution code has been moved to MifosBatchJob class. This means that all batch jobs
that should have Mifos catch-up mechanism should extend this class. Because we do not have any
constraints on the package names anymore (in the original Mifos Scheduler all batch job classes
should be placed in a specific hard-coded package), we can easily configure batch jobs that do not
use Mifos catch-up mechanism.
5. Customizing task.xml file
In order to customize task.xml file included in the Mifos war file, please copy it from your tomcat/webapps to MIFOS_CONF directory.
For the backward compatibility old task.xml file is also supported (but only Simple Triggers can be
created in this way).
Database migration scripts transfer all information about last successful tasks invocations from the
old scheduled_tasks table to the new Spring Batch tables, so no information is lost.
New Batch Jobs management page has been added to display information about configured tasks
(their triggers, last run statuses, etc.). Moreover, it is now possible to run specified tasks on-demand and to put the whole scheduler into standby mode.
Batch Jobs management page is available from the Admin area:
We added also two new roles to run specified tasks on-demand and to put the whole scheduler into
8. Plans for future
In order to fully leverage Quartz and Spring Batch features (like parallel executions and clustering)
all currently used task helpers (classes that contain batch jobs business logic) should be rewritten.
However, this is out of scope of this project and will be performed in a 6-12 months from now.