Quartz Batch Jobs

Owned by Jeff Brewster (Unlicensed)
Last updated: Dec 19, 2010 by Jakub Sławiński (Unlicensed)Version comment

1. Background

Batch jobs are a series of back-end jobs on a computer that are executed through predefined scripts.
Mifos is configured to run a set of batch jobs nightly at 12 am. The server must be kept on at night
for these processes to run.
Mifos currently has 9 batch processes:

  1. ProductStatus
  2. SavingsIntPostingTask
  3. GenerateMeetingsForCustomerAndSavingsTask
  4. LoanArrearsTask
  5. PortfolioAtRiskTask
  6. LoanArrearsAgingTask
  7. ApplyCustomerFeeChangesTask
  8. GenerateMeetingsForCustomerAndSavingsTask
  9. ApplyHolidayChangesTask

The goal of this project was to replace legacy scheduling system based on a Java Timer class and a
custom code (scheduled_tasks table, dependencies between tasks and catch-up execution hardcoded
in the Mifos sources) with a well known frameworks designed for enterprise scheduling
(Quartz and Spring Batch). Moreover, as a part of this project a simple UI was developed to display
basic information about performed batch jobs and to allow some management activities directly
from the Mifos administrative area.

2. Original Mifos Scheduler

In the original Mifos implementation all scheduling was done by MifosScheduler class, which used
java Timer class to run tasks. This meant, that:

  • all tasks were run sequentially
  • there was no configurable support for tasks dependencies (the only dependency between
    PortfolioAtRiskTask and LoanArrearsTask was hard-coded in TaskHelper class) except
    manipulation of initial-time/delay-time in task.xml file

Mifos used its own configuration format for job scheduling (task.xml). Even if there was a
possibility to configure delay-time, there were several places where it was hard-coded to 1 day:

  • exception was thrown in MifosScheduler.registerTasks() method when delay time < 86400 (1
    day)
  • catch-up execution functionality from TaskHelper.executeTask() method performed task
    execution for every missing day (even when we could set delay time > 1 day)

3. Quartz integration

We have decided to use Quartz as a reliable replacement for Timer used by the MifosScheduler.
However, we had to ensure that the old task.xml files are still supported and all scheduling
functionality works as before. This specifically includes:

  1. It should be possible to define dependencies between tasks
  2. All missed tasks executions should be run by scheduler before the next planned execution
  3. Mifos Scheduler should reflect all changes made to task.xml file

It is not possible to achieve the above goals using Quartz only, because:

  1. Quartz does not support dependencies between tasks
  2. Quartz does not support automatic execution of missed triggers (except the last one)
  3. In order to schedule the same trigger/job (with modified time of execution for example) the
    previous instance has to be unscheduled/removed. All information about missed executions
    is lost in such case.

Taking into consideration the above Quartz limitations we have decided to configure this
framework with the following settings:

<props>
<prop key="org.quartz.scheduler.instanceName">MifosQuartzScheduler</prop>
<prop key="org.quartz.scheduler.instanceId">AUTO</prop>
<prop key="org.quartz.scheduler.skipUpdateCheck">true</prop>
<prop key="org.quartz.threadPool.class">org.quartz.simpl.SimpleThreadPool</prop>
<prop key="org.quartz.threadPool.threadCount">1</prop>
<prop key="org.quartz.jobStore.class">org.quartz.simpl.RAMJobStore</prop>
<prop key="org.quartz.jobStore.misfireThreshold">60000</prop>
</props>

This means that we treat Quartz as a simple Timer replacement. We do not use JDBC JobStore, but
this can be easily configured by changing appropriate properties. We have added the following
Quartz tables to Mifos:

QRTZ_BLOB_TRIGGERS
QRTZ_CALENDARS
QRTZ_CRON_TRIGGERS
QRTZ_FIRED_TRIGGERS
QRTZ_JOB_DETAILS
QRTZ_JOB_LISTENERS
QRTZ_LOCKS
QRTZ_PAUSED_TRIGGER_GRPS
QRTZ_SCHEDULER_STATE
QRTZ_SIMPLE_TRIGGERS
QRTZ_TRIGGERS
QRTZ_TRIGGER_LISTENERS

so our database can be used by the JDBC JobStore without any additional effort.

4. Spring Batch integration

We have decided to use Spring Batch to manage our batch jobs. We were convinced by the
following Spring Batch features:

  1. It can be easily integrated with Quartz (and other schedulers).
  2. It is possible to define dependencies between steps in Spring Batch.

Spring Batch tasks can be fully configured with Spring. Example configuration of the LoanArrears
and PortfolioAtRisk tasks (where the second task depends on the first one):

<bean name="LoanArrearsAndPortfolioAtRiskTask" parent="jobDetailBase"
class="org.springframework.scheduling.quartz.JobDetailBean">
<property name="jobClass" value="org.mifos.framework.components.batchjobs.helpers.PortfolioAtRiskTask" />

<property name="name" value="LoanArrearsAndPortfolioAtRiskTaskJob" />

</bean>
<bean id="loanArrearsAndPortfolioAtRiskTaskTrigger"
class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="name" value="LoanArrearsAndPortfolioAtRiskTaskJob"/>
<property name="jobDetail" ref="LoanArrearsAndPortfolioAtRiskTask" />
<property name="cronExpression" value="0 0 0 * * ?" />
</bean>
<batch:job id="LoanArrearsAndPortfolioAtRiskTaskJob" parent="jobBase" job-repository="jobRepository">
<batch:step id="LoanArrearsAndPortfolioAtRiskTask-step-1" next="LoanArrearsAndPortfolioAtRiskTask-step-2">
<batch:tasklet ref="LoanArrearsTaskTasklet" />
</batch:step>
<batch:step id="LoanArrearsAndPortfolioAtRiskTask-step-2">
<batch:tasklet ref="PortfolioAtRiskTaskTasklet" />
</batch:step>
</batch:job>
<bean name="LoanArrearsTaskTasklet"
class="org.mifos.framework.components.batchjobs.helpers.LoanArrearsHelper" />
<bean name="PortfolioAtRiskTaskTasklet"
class="org.mifos.framework.components.batchjobs.helpers.PortfolioAtRiskHelper" />

This is much more complicated than the previous task.xml file, but now nothing is hard-coded in
our code and everything can be modified by changing one xml file.

Database table scheduled_tasks has been removed. Execution times, statuses, exit values, etc. are
now stored in the new Spring Batch tables. We have added the following Spring Batch tables to
Mifos:

BATCH_JOB_EXECUTION
BATCH_JOB_EXECUTION_CONTEXT
BATCH_JOB_EXECUTION_SEQ
BATCH_JOB_INSTANCE
BATCH_JOB_PARAMS
BATCH_JOB_SEQ
BATCH_STEP_EXECUTION
BATCH_STEP_EXECUTION_CONTEXT
BATCH_STEP_EXECUTION_SEQ

Catch-up execution code has been moved to MifosBatchJob class. This means that all batch jobs
that should have Mifos catch-up mechanism should extend this class. Because we do not have any
constraints on the package names anymore (in the original Mifos Scheduler all batch job classes
should be placed in a specific hard-coded package), we can easily configure batch jobs that do not
use Mifos catch-up mechanism.

5. Customizing task.xml file

In order to customize task.xml file included in the Mifos war file, please copy it from your tomcat/webapps to MIFOS_CONF directory.

Next, please change import resource tag from

<import resource="dataSourceContext.xml" />

to

<import resource="classpath:org/mifos/config/resources/dataSourceContext.xml" />

If you want to use SimpleTriggers instead CronTriggers, please add the following bean definition:

<bean id="dateFormat" class="java.text.SimpleDateFormat">
  <constructor-arg value="HH:mm:ss" />
</bean>

change your trigger classes from CronTriggerBean to SimpleTriggerBean, remove cronExpression property and add the following ones instead:

<property name="startTime">
  <bean factory-bean="dateFormat" factory-method="parse">
    <constructor-arg value="24:00:00" />
  </bean>
</property>
<property name="repeatInterval" value="86400000" />

6. Backward compatibility

For the backward compatibility old task.xml file is also supported (but only Simple Triggers can be
created in this way).

Database migration scripts transfer all information about last successful tasks invocations from the
old scheduled_tasks table to the new Spring Batch tables, so no information is lost.

7. UI

New Batch Jobs management page has been added to display information about configured tasks
(their triggers, last run statuses, etc.). Moreover, it is now possible to run specified tasks on-demand and to put the whole scheduler into standby mode.
Batch Jobs management page is available from the Admin area:

We added also two new roles to run specified tasks on-demand and to put the whole scheduler into
standby mode.

8. Plans for future

In order to fully leverage Quartz and Spring Batch features (like parallel executions and clustering)
all currently used task helpers (classes that contain batch jobs business logic) should be rewritten.
However, this is out of scope of this project and will be performed in a 6-12 months from now.