Upgrade Hibernate

Owned by Former user (Deleted)
Last updated: Jan 06, 2011 by Michael VorburgerVersion comment

Table of Contents

Background

Since April 2007, Mifos has been using Hibernate 3.0 beta4.

The purpose of this page is to document the current findings and the proposed plan for the upgrade of Hibernate to a stable release, that is, version 3.2.

Active Members

The following people are actively involved in this endeavour:

  • Van Mittal-Henkle (Grameen)
  • Beth Mazur (Grameen)
  • Massimiliano Parlione (IBM)
  • Stephen Horgan (IBM)
  • Chico Charlesworth (IBM)

SVN Repository (Branch)

A new SVN branch has been created under https://mifos.dev.java.net/svn/mifos/branches/hibernateUpgrade.

This branch will accommodate any changes required to upgrade Mifos to Hibernate 3.2. In addition, the branch will regularly be applied with the latest changes from the trunk by using the svnmerge utility.

Current Findings

As mifos currently stands, various issues have been encountered and are shown below in more detail. Conclusions have also been made in respect to unit testing and connection pooling.

Current Issues

The following exceptions have been found when running the application and the unit tests.

org.hibernate.TransactionException

Example in org.mifos.application.productsmix.business.ProductMixBOTest.setUp():

org.hibernate.TransactionException: Transaction not successfully started
at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:100)
at org.mifos.framework.persistence.TestObjectPersistence.persist(TestObjectPersistence.java:104)
at org.mifos.framework.util.helpers.TestObjectFactory.createMeeting(TestObjectFactory.java:1199)
at org.mifos.application.accounts.savings.util.helpers.SavingsTestHelper.createSavingsOffering(SavingsTestHelper.java:137)
at org.mifos.application.accounts.savings.util.helpers.SavingsTestHelper.createSavingsOffering(SavingsTestHelper.java:127)
at org.mifos.application.productsmix.business.ProductMixBOTest.setUp(ProductMixBOTest.java:21)

net.sourceforge.mayfly.UnimplementedException

Example in org.mifos.framework.security.AddActivityTest.startFromStandardStore():

net.sourceforge.mayfly.UnimplementedException: This feature is not yet implemented in Mayfly
at net.sourceforge.mayfly.jdbc.JdbcMetaData.getDatabaseProductVersion(JdbcMetaData.java:95)
at org.hibernate.cfg.SettingsFactory.buildSettings(SettingsFactory.java:89)
at org.hibernate.cfg.Configuration.buildSettings(Configuration.java:2073)
at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1298)
at org.mifos.framework.util.helpers.DatabaseSetup.mayflySessionFactory(DatabaseSetup.java:107)
at org.mifos.framework.persistence.TestDatabase.openSession(TestDatabase.java:67)
at org.mifos.framework.security.AddActivityTest.upgradeAndCheck(AddActivityTest.java:70)
at org.mifos.framework.security.AddActivityTest.startFromStandardStore(AddActivityTest.java:37)

org.hibernate.PropertyAccessException

Example in org.mifos.application.customer.struts.action.TestEditCustomerStatusAction.testFailurePreviewWithAllValuesNullForClient():

org.hibernate.PropertyAccessException: IllegalArgumentException occurred calling getter of org.mifos.framework.components.fieldConfiguration.business.EntityMaster.id
at servletunit.struts.MockStrutsTestCase.getActionServlet(MockStrutsTestCase.java:344)
at servletunit.struts.MockStrutsTestCase.actionPerform(MockStrutsTestCase.java:394)
at org.mifos.application.customer.struts.action.TestEditCustomerStatusAction.testFailurePreviewWithAllValuesNullForClient(TestEditCustomerStatusAction.java:292)

java.lang.ClassCastException

  • TODO

java.lang.NullPointerException

  • TODO

Unit Tests

The current unit tests are designed/written poorly and take far too long to run. For these reasons this has become a real problem when attempting the upgrade of Mifos to Hibernate 3.2.

There are two principle shortcomings:

  • Session and transaction management in the unit test code is convoluted and unstable.
  • Initial setup and tear down of unit tests are unreliable (e.g. if the tear down process fails, other potentially valid tests will also fail).

The following Hibernate Util method call analysis snapshot gives a glance of how unwieldy the unit tests have become:
csv-table:: Hibernate Util Method Call Analysis:header: "Hibernate Util Method Calls", "Application Source Code", "Unit Test Code"
:widths: 20, 20, 20

"getSessionTL()", 57, 483 "closeSession()", 48, 1066 "commitTransaction()", 29, 546 "Other Method Calls", 75, 131 "Total Method Calls", 209, 2226

Connection Pooling

All attempts to setup C3PO connection pooling with the current Mifos application have been unsuccessful.

It would appear that there are similar issues when trying to use connection pooling as when trying to use the latest version of Hibernate. Indeed, once the code works with Hibernate 3.2 then C3PO connection pooling should also work.

Note, that by default Mifos currently runs with Hibernate's built-in connection pooling which is strongly NOT recommended in a production environment. See http://www.hibernate.org/hib_docs/v3/reference/en/html/session-configuration.html#configuration-hibernatejdbc, which states: "Hibernate's own connection pooling algorithm is however quite rudimentary. It is intended to help you get started and is not intended for use in a production system or even for performance testing. You should use a third party pool for best performance and stability. Just replace the hibernate.connection.pool_size property with connection pool specific settings. This will turn off Hibernate's internal pool. For example, you might like to use C3P0."

The internal pool can be verified when the logs are set to the level of INFO:

INFO, org.hibernate.connection.DriverManagerConnectionProvider, ?, ? , ?, Using Hibernate built-in connection pool (not for production use!), 04 Mar 2008 17:49:05
INFO, org.hibernate.connection.DriverManagerConnectionProvider, ?, ? , ?, Hibernate connection pool size: 20, 04 Mar 2008 17:49:05

Proposed Plan

Realistically there are two options which can be taken into consideration:

  • Patch Approach - more conservative path, where attempts will be made to fix ongoing issues.
  • Total Refactor Approach - total re-design and re-write, specifically targeted to session and transaction management.

Patch Approach

The patched Mifos web application appears to be now running and working successfully with Hibernate 3.2.

Please get the latest from the SVN Branch mentioned above to build, test and deploy the Mifos application.

The following issues have been found and numerous fixes and workarounds have been put into place to allow for the Hibernate upgrade to work.

org.hibernate.TransactionException: Transaction not successfully started

  • Fixes have been applied so that the Mifos web application will run successfully.

net.sourceforge.mayfly.UnimplementedException

A fix has been applied to the mayfly code, namely to the net.sourceforge.mayfly.jdbc.JDBCMetaData class.

org.hibernate.PropertyAccessException

Various named queries have been fixed.

For example, in one named query the where clause "entity = :entityId" has been replaced with "entity.id = :entityId".

java.lang.ClassCastException: java.lang.Long

The latest version of Hibernate returns java.lang.Long for count type queries instead of java.lang.Integer. The code has been fixed in many places in order to resolve this issue.

java.lang.ClassCastException: short expected

Currently some named query parameter types are lax, in that they are set to a data type that doesn't evaluate correctly against the column in the query. For example, in some circumstances the column is mapped to a short but the query parameter is set to a java.lang.String. The code has been fixed in many places in order to resolve this issue.

java.lang.NullPointerException

This exception can obviously be thrown due to any number of reasons, but watch out for these issues:

  • Date comparison fix in named queries - For example, in one named query the where clause "loan.disbursementDate = :meeting_date" has been replaced with "loan.disbursementDate = date(:meeting_date)".

ROM Estimate

The Mifos application is now working with Hibernate 3.2, including the unit tests.

Work to plan and execute a full regression and load test is still to be done, and the estimate for this is yet to be determined as it would require the involvement of the QA team. Other development teams that have dependencies with the Hibernate porting should also be involved.

Total Refactor Approach

Time has been spent evaluating the possibility of re-designing and re-writing the Mifos application to have session and transaction management implemented correctly and efficiently. Reflecting on the state of the current architecture, it has been a real challenge to make a working prototype to improve on the current version, and more investigation is required to have anything practical at this time.

Possible (But NOT Definite) Architecture and Technology Stack

The following architecture overview and technology stack diagrams give an idea of what could be a proposed architecture solution for Mifos, but is NOT a definite answer. It looks into what could be done in terms of session and transaction management, but other considerations (such as saas, multitenancy, portability, etc) must be taken into account before agreeing on what needs to be done to improve Mifos in the future (i.e. after the 1.1. release).

ROM Estimate

It is extremely difficult at the moment to roughly estimate how long a total refactor would take, but it would be at least 2 months work for two developers to just re-design and re-write the session and transaction management currently implemented with Mifos, excluding the task of re-writing the unit tests. This is only a rough estimate that doesn't take into account the overall requirements for Mifos.

Current Recommendations and Risks

After evaluating both approaches, the more likely of the two to succeed at this time is the more conservative patch approach. This is not to say that a total refactor is not the best option in the long run, and should seriously be considered after the 1.1 release.

Note that, regardless of which option is chosen, without the plan and execution of a full regression and load test, it is risky to assume that the Hibernate upgrade will be a success.

Also, by default Mifos currently runs with Hibernate's built-in connection pooling which is NOT recommended in a production environment. And for this reason, it is strongly recommended that C3PO connection pooling be put into place and tested fully when the Hibernate upgrade task has been finished.