%age fee with too many places right of decimal causes Mifos to choke and gives bogus error message when creating loan

Description

If you try to create a loan with a percentage fee that has too much precision
(for example 3.0001%), the error message is wrong and misleading: "Please
specify required information." (And no fields are highlighted as missing.)

Mifos should catch the error and give a helpful error message, such as "Mifos
cannot handle that fee, please use a fee with three or fewer digits right of the
decimal point."

To reproduce, create a loan fee that is a % of the loan amount and associate it
with a loan product. Then create a loan and add the fee with four digits to the
right of the decimal. Hit Preview then Submit.

(The Jitegemea folks actually tried this because the payments didn't correspond
with their Excel spreadsheet, so they tried to tinker with adding a fee.)

The error log says Mifos tried to truncate the value:
15:37:25,562 ERROR [STDERR] Caused by: com.mysql.jdbc.MysqlDataTruncation: Data
truncation: Data truncated for column 'FEE_AMNT' at row 1
15:37:25,562 ERROR [STDERR] at
com.mysql.jdbc.SQLError.convertShowWarningsToSQLWarnings(SQLError.java:709)
15:37:25,562 ERROR [STDERR] at
com.mysql.jdbc.MysqlIO.scanForAndThrowDataTruncation(MysqlIO.java:3461)
15:37:25,562 ERROR [STDERR] at
com.mysql.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement.java:119
8)
15:37:25,562 ERROR [STDERR] at
com.mysql.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStatement.java:6
77)
15:37:25,562 ERROR [STDERR] at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1357)
15:37:25,562 ERROR [STDERR] at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1274)
15:37:25,562 ERROR [STDERR] at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1259)
15:37:25,562 ERROR [STDERR] at
org.hibernate.persister.entity.BasicEntityPersister.insert(BasicEntityPersister.java:
1729)
15:37:25,562 ERROR [STDERR] ... 66 more
15:37:25,562 INFO [STDOUT] ERROR, org.mifos, MifosExceptionHandler,
logException , 72, No resource is associated with k
ey "exception.accounts.create"., 05 May 2008 15:37:25
15:37:25,562 INFO [STDOUT] ERROR, org.mifos, MifosExceptionHandler,
logException , 72, exception.accounts.create, 05 May 2008 15:37:25
org.mifos.application.accounts.exceptions.AccountException:
org.mifos.framework.exceptions.PersistenceException: org.hib
ernate.exception.GenericJDBCException: could not insert:
[org.mifos.application.accounts.business.AccountFeesEntity]
at org.mifos.application.accounts.loan.business.LoanBO.save(LoanBO.java:996)
at
org.mifos.application.accounts.loan.business.LoanBO.save(LoanBO.java:1003)
at
org.mifos.application.accounts.loan.struts.action.LoanAccountAction.create(LoanAccountAction.java:371)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:274)
at org.apache.struts.actions.DispatchAction.execute(DispatchAction.java:194)
at org.mifos.framework.struts.action.BaseAction.execute(BaseAction.java:65)
at
org.mifos.framework.struts.action.MifosRequestProcessor.processActionPerform(MifosRequestProcessor.java:232)
at
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:224)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1194)
at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.mifos.framework.security.util.LoginFilter.doFilter(LoginFilter.java:109)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.mifos.framework.persistence.DatabaseVersionFilter.doFilter(DatabaseVersionFilter.java:39)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
at
org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:179)
at
org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
at
org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:241)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:580)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.mifos.framework.exceptions.PersistenceException:
org.hibernate.exception.GenericJDBCException: could not
insert: [org.mifos.application.accounts.business.AccountFeesEntity]
at
org.mifos.framework.persistence.Persistence.createOrUpdate(Persistence.java:50)
at
org.mifos.framework.persistence.Persistence.createOrUpdate(Persistence.java:34)
at org.mifos.application.accounts.loan.business.LoanBO.save(LoanBO.java:992)
... 39 more
Caused by: org.hibernate.exception.GenericJDBCException: could not insert:
[org.mifos.application.accounts.business.AccountFeesEntity]
at
org.hibernate.exception.ErrorCodeConverter.handledNonSpecificException(ErrorCodeConverter.java:92)
at
org.hibernate.exception.ErrorCodeConverter.convert(ErrorCodeConverter.java:80)
at
org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43)
at
org.hibernate.persister.entity.BasicEntityPersister.insert(BasicEntityPersister.java:1747)
at
org.hibernate.persister.entity.BasicEntityPersister.insert(BasicEntityPersister.java:2149)
at
org.hibernate.action.EntityIdentityInsertAction.execute(EntityIdentityInsertAction.java:34)
at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:239)
at
org.hibernate.event.def.AbstractSaveEventListener.performSaveOrReplicate(AbstractSaveEventListener.java:238)
at
org.hibernate.event.def.AbstractSaveEventListener.performSave(AbstractSaveEventListener.java:158)
at
org.hibernate.event.def.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:104)
at
org.hibernate.event.def.DefaultSaveOrUpdateEventListener.saveWithGeneratedOrRequestedId(DefaultSaveOrUpdateEv
entListener.java:184)
at
org.hibernate.event.def.DefaultSaveOrUpdateEventListener.entityIsTransient(DefaultSaveOrUpdateEventListener.j
ava:173)
at
org.hibernate.event.def.DefaultSaveOrUpdateEventListener.performSaveOrUpdate(DefaultSaveOrUpdateEventListener
.java:96)
at
org.hibernate.event.def.DefaultSaveOrUpdateEventListener.onSaveOrUpdate(DefaultSaveOrUpdateEventListener.java
:69)
at org.hibernate.impl.SessionImpl.saveOrUpdate(SessionImpl.java:416)
at org.hibernate.engine.Cascades$5.cascade(Cascades.java:153)
at org.hibernate.engine.Cascades.cascade(Cascades.java:721)
at org.hibernate.engine.Cascades.cascadeCollection(Cascades.java:860)
at org.hibernate.engine.Cascades.cascade(Cascades.java:739)
at org.hibernate.engine.Cascades.cascade(Cascades.java:817)
at org.hibernate.engine.Cascades.cascade(Cascades.java:789)
at
org.hibernate.event.def.DefaultSaveOrUpdateEventListener.cascadeOnUpdate(DefaultSaveOrUpdateEventListener.jav
a:313)
at
org.hibernate.event.def.DefaultSaveOrUpdateEventListener.performUpdate(DefaultSaveOrUpdateEventListener.java:
296)
at
org.hibernate.event.def.DefaultSaveOrUpdateEventListener.entityIsDetached(DefaultSaveOrUpdateEventListener.ja
va:214)
at
org.hibernate.event.def.DefaultSaveOrUpdateEventListener.performSaveOrUpdate(DefaultSaveOrUpdateEventListener
.java:91)
at
org.hibernate.event.def.DefaultSaveOrUpdateEventListener.onSaveOrUpdate(DefaultSaveOrUpdateEventListener.java
:69)
at org.hibernate.impl.SessionImpl.saveOrUpdate(SessionImpl.java:416)
at org.hibernate.impl.SessionImpl.saveOrUpdate(SessionImpl.java:411)
at
org.mifos.framework.persistence.Persistence.createOrUpdate(Persistence.java:43)
... 41 more
Caused by: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Data truncated
for column 'FEE_AMNT' at row 1
at
com.mysql.jdbc.SQLError.convertShowWarningsToSQLWarnings(SQLError.java:709)
at com.mysql.jdbc.MysqlIO.scanForAndThrowDataTruncation(MysqlIO.java:3461)
at
com.mysql.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement.java:1198)
at
com.mysql.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStatement.java:677)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1357)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1274)
at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1259)
at
org.hibernate.persister.entity.BasicEntityPersister.insert(BasicEntityPersister.java:1729)
... 66 more
15:37:25,578 INFO [STDOUT] ERROR, org.hibernate.util.JDBCExceptionReporter, ?,
? , ?, Column 'ACCOUNT_FEE_ID' cannot be
null, 05 May 2008 15:37:25
15:37:25,578 INFO [STDOUT] ERROR, org.hibernate.util.JDBCExceptionReporter, ?,
? , ?, Column 'ACCOUNT_FEE_ID' cannot be
null, 05 May 2008 15:37:25

Environment

Platform: All, OS: All

Activity

Show:
paulbrowne
May 15, 2008, 2:48 PM

Thanks for the additional information.

I've follow the instructions, created a new weekly client and new percent of
loan fee. When I create a new loan , I am able to see the new fee type in the
'Apply Additional Fees ' section of the loan page
( attachment screenshot : 1860-additional-fees.JPG).

However , in build 12962 I'm seeing different behaviour:

  • I am able to override the default percentage fee (e.g. 2.00001 %). If I try to
    add any more precision , (e.g. 2.000011) javascript removes that last number.

  • Even though 2.00001 is more one decimal place more precise than the first
    example that caused the issue (1.0001), I am still able to save and then view
    this number in the Database.

A second attachment is the HTML / javascript from the page that 'protects' this
field (i.e. removes extra numbers) is attached as 1860-additional-fees-html.txt

Can you let me know if there is anything that I am missing when I am trying to
reproduce this bug?

bberning
May 16, 2008, 1:13 PM

Hi Paul, Thanks for looking at this. Interesting! I tried it too, and like
you, I cannot reproduce it using the current software (mine is rev 12826 from
April 26th). I can repro it consistently in v1.0. Maybe the work that Van is
doing to improve financial calculations fixed this.?

How should I mark the Issue in the Issue Tracker? Fixed? (When it is back up.)

Cheers,
Bart

aliyaw
May 16, 2008, 6:02 PM

Normally I would resolve this issue as 'worksforme', however in this case, since
it was reproducing in 1.0 and not reproducing in current code, lets mark it as
'fixed'. Testers should verify this behavior against rev 12962.

Jeff Brewster
May 19, 2008, 7:41 PM

change QA contact to Arpita

arpita_a
May 30, 2008, 12:17 PM

Hi,

tested by creating a client loan account and attaching a % fee of 1% (changing
it to 1.0001% while attaching).
Faced no problem at all while creating the account or disbursing it. The
repayment schedule also got correctly calculated given the default rounding
settings in the Property Files.

Operating System / Arch / Version Windows XP / x86 / 5.1
Java Vendor / Version Sun Microsystems Inc. / 1.5.0_10
Database Vendor / Version MySQL / 5.0.45-community-nt
Database Driver / Version MySQL-AB JDBC Driver / mysql-connector-java-5.1.5 (
Revision: ${svn.Revision} )
Application Server Apache Tomcat/6.0.16
Mifos Database Version 197
Subversion Revision Number 13047

Thanks,
Arpita

Fixed

Assignee

paulbrowne

Reporter

bberning

Labels

URL

None

Story Points

None

Team

None

Scheduled For

None

Epic

None

Notify

None

productboard URL

None

Man Day Estimate

None

Components

Fix versions

Affects versions

Priority

Critical
Configure