This page describes key architectural aspects of the reporting module of Mifos.

BIRT overview

BIRT is an Eclipse-based open source reporting system for web applications, especially those based on Java and J2EE. BIRT has two main components: a report designer based on Eclipse, and a runtime component that you can add to your app server. BIRT also offers a charting engine that lets you add charts to your own application. More information on BIRT can be found at its Eclipse web site.

BIRT capabilities

Basic capabilities

Briefly, with BIRT you can

• visually lay out a report
• connect content to one or more data sources
• define CSS-like styles and data formats
• perform typical reporting functions such a sorting, grouping, and aggregating data, as well as manipulate numeric and string data
• allow the user to dynamically filter data based on input parameters
• build a report that contains subreports
• display data in charts
• build a shared report framework using templates and libraries of pre-defined reports and data sources.
• localize text using standard I18N Java conventions

Filtering data, input parameters, and cascading

This feature is important to Mifos – it is the mechanism by which a user's access to data is controlled. See the section on access control.

BIRT can filter data at several points in the flow of data from data sources to screen or print: at the database, by specifying WHERE clauses to append to SQL; after database retrieval, at the report element level.

Such filtering is static, built into the report design. But BIRT also allows for dynamic filtering by the user, allowing for a degree of ad-hoc reporting, by adding input parameters to the design. The user is presented with a UI form in which to either enter values or select from drop-down boxes. These values are entered as query parameters to filter the data retrieved.

A sequence of input parameters can be defined as cascading – a value selected for one constrains the range of possible values for the next. This feature is used by Mifos in several reports (see the section on access control).

BIRT components

A simplified architecture diagram of BIRT and related components is displayed below:

Most components are self-explanatory - see documentation on line at <http://www.eclipse.org/birt> or in two published books referenced at the end of this document. BIRT supports several kinds of data sources for accessing reporting information. Out of the box it supports JDBC, XML, and text data sources.

BIRT can also create scripted data sources that can retrieve reporting data from an application: The report designer embeds Javascript code into the report that supplies the reporting data to the report-generating engine. In a typical scenario (the one of interest to Mifos) the Javascript uses the interface of an application to supply the data.

Reporting module - architectural overview

This section describe the major components of the reporting module, and how they collaborate to perform reporting functions.

Report Module Components – Responsibilities

BIRT web viewer

The components of the BIRT Web Viewer are folded into the Mifos build - specifically, into the "BIRTWebViewer" top-level folder of the build directory - and embedded in the Mifos WAR file. Thus the application server deploys BIRT into the same JVM as Mifos services.

The primary rationale for this approach is that reports that use scripted data sources need to access Mifos objects. Since Mifos currently does not implement remote access to its objects, BIRT must reside in the same JVM as Mifos services.

The BIRT web viewer is responsible for

• asking the user to select values of the report parameters
• querying data sources to retrieve both parameter and report data
• generating the report from report data as filtered by report parameters
• presenting the report in desired format (HTML, PDF)

Mifos services

For reports that access data through Mifos using scripted data sources, the Mifos application is responsible for

• determining the list of reports available to the user, restricted according to the rules of access control.
• presenting the categorized list to the user for selection,
• running the BIRT Web Viewer for the report that the user selected,
• providing to BIRT the lists of possible values of report parameters, restricted according to rules of access control,
• validating report parameters that the user selects, and
• supplying report data to BIRT in the format that BIRT expects.

TODO: Current implementation of access control displays all reports to the user, even those that he or she is not permitted to generate.

Mifos operational database

Mifos operational tables

• store report metadata, including _ Id _ Name _ Category _ Active status * Access-control information
• store information that is accessed by the report viewer either via Hibernate or directly by SQL embedded in the report.

Mifos datamarts

Mifos datamarts hold snapshots of operational data, typically organized as star-schema. Datamarts are populated periodically with either transactional history, or snapshot summary data. Datamarts are responsible for providing efficient retrieval of data for reports.

NOTE: Design and implementation of datamarts and associated processes and reports is in progress.

BIRT-Mifos interaction

BIRT Report Viewer, Mifos and the databases collaborate to perform reporting functions, fulfilling their responsibilities as described above.

Reports using scripted data sources

This diagram shows a typical scenario for reports that use scripted data sources:

Mifos reporting service layer – details

The BIRT report viewer makes all requests through one or more reporting service classes. For example, the collection sheet report mifos/BirtReportDesign/LayoutReport.rptdesign queries the class org.mifos.application.reports.business.service.CollectionSheetReportService to retrieve all report data and options for parameter drop-down lists:

• Report Parameter retrieval _ public List<SelectionItem> getBranchOffices(Integer userId) _ public List<SelectionItem> getActiveLoanOfficers(Integer userId,Integer branchIdIntValue) _ public List<SelectionItem> getActiveCentersForLoanOfficer(Integer loanOfficerIdIntValue, Integer branchIdIntValue) _ public List<SelectionItem> getActiveGroupsForLoanOfficer(Integer loanOfficerId, Integer branchId) _ public List<DateSelectionItem> getMeetingDates(Integer branchIdIntValue,Integer officerIdIntValue, Integer customerId, CustomerLevel customerLevel) _ public List<SelectionItem> getMeetingDatesForGroup(Integer branchId, Integer groupId, Integer officerId) * public List<SelectionItem> getMeetingDatesForCenter(Integer branchId, Integer groupId, Integer officerId)
• Collection sheet report retrieval * public List<CollectionSheetReportDTO}> getCollectionSheets(Integer branchId, Integer officerId, Integer customerId, Date meetingDate, CustomerLevel customerLevel)

Cascading parameters and the service layer.

All but the last method are used to populate drop-down boxes for the report's cascading parameters: select branches; select loan officers for branches; select centers or groups for loan officers; and finally select meeting dates for which to display collection sheets. The user is presented with either a group selection drop-down box or center selection drop-down box, depending on whether the MFI has organized groups into centers.

TODO: The user is presented with not only the appropriate list of values to choose from but also the options --All-- and --NA--. The interaction between the BIRT report and the service layer to (a) determine when such options should be presented and (b) sort out such selections as they cascade, is very complex and poorly documented. It would be very difficult for another designer to understand how this works in order to reuse or adapt the logic to other reporting scenarios.

TODO: The collection sheet report determines whether to present group or center options based on a parameter that is hard-coded into the report. Thus an MFI would have to open the report design and modify its contents during installation.

Reports using JDBC data sources

Reports that rely on datamarts follow the same interaction, but use JDBC data sources to access reporting data from datamarts:

TODO: Currently no datamarts have been implemented. Work is in progress for a future release.

Creating, installing and customizing new reports

Creating new reports

New reports are created using the BIRT Report Designer, resulting in a .rptdesign file. This xml file encapsulates all information needed for the BIRT Web Viewer to generate, populate, and present the report. A description of the design process is beyond the scope of this document. See references below for more information. Also see <http://mifos.org/developers/wiki/Reporting in Mifos> for a detailed description of the design process.

TODO: The page on mifos.org mentioned above is out of date. For example, the screen shot of the open method on the scripted data source shows this initialization:

reportsDataService.initialize(userId);

This manner of passing userId to Mifos is deprecated. Scripted data sources now pass the userId on each call to the Mifos reporting services.

Uploading new reports

Mifos maintains metadata about the report so a user can locate the desired report and request it of the BIRT web viewer. The process of making a new report available to Mifos users is termed "uploading" a report. See the first reference below for existing documentation. is that information on mifos.org sufficient for this review. Here, we focus on the changes that need to be made to Mifos in order to make a new report available to users at a deployment site. We distinguish between reports that use scripted data sources and those that use SQL data sources, when significantly different changes are required.

Report dependencies

When installing a new report at a deployment site, the characteristics of the report may require changes to either Mifos code, Mifos database schema, or both.

New report parameters may require:

• New or altered MifOS code to
  • create the list of allowable values
  • access new or modified tables
  • apply access-control rules to restrict the allowable values to what the user is permitted to access
• New or altered database table schema

New, or differently processed, report data may require

• New or altered MifOS code to retrieve data required by the report
• New or altered database table schema

*NOTE: If a new report includes changes in any of these, it may be impossible to deploy it to a site without deploying an updated version of the entire application.

Deeper analysis is required to determine how, or whether, any of these gaps can be worked around.*

Customizing reports

Using pre-built libraries and templates, a report's look and feel and, to some extent, its layout, can be modified to meet the needs of the MFI. For example, a custom logo can be inserted.

TODO:Currently only one library exists – a libary of data sets for the cascading input parameters.

Access Control

The information that a user is permitted to see depends on the user's role (loan officer, branch manager) and other attributes such as his or her location in the office hierarchy. Rules constraining access fall into two categories, depending on the type of information being accessed:

• Information about the availability of a report – For example, an MFI may choose to not include general financial reports in the list displayed to loan officers.
• Information in the report itself. Rules include, for example, _ A loan officer can see information on individual clients only that are assigned to him or her _ A branch manager can see loan information only for loan officers at his or her branch * A manager of a higher-level office has access to all information along the organizational hierarchy starting at his or her office.

Implementation of access control

Restricting the list of reports

TODO: The current system does not restrict access to the full list of available reports.

Restricting report content

The reporting module restricts content by restricting the list of input parameters that the user can select. When the Birt web viewer requests the list of options, it passes the user's id to the MifOS service. The service constructs and returns the list of options based on the user's role and location.

If the report uses scripted data sources to retrieve report data, it passes the user's selected input parameters to the Mif service, that returns only data for those values.

References

1. See Eclipse web site for more information on BIRT.
2. Two paperback references are useful: a. gt;`_ gives basic information for report designers, focusing on JDBC data sources. b. `Integrating and Extending BIRT describes several advanced features of BIRT, the most relevant to Mifos being scripted data sources.
3. See Reporting in Mifos for a detailed description of the design process.