Home

JDBC driver to support the development and testing of database driven applications. In record mode, the driver records all data requested from the database layer in files. These files replace the database connection when switching to replay mode.

Project Admins:

• mbcon

[T-42]

Hi,

Connection.getWarnings() throws an exception:

java.lang.RuntimeException: Unexpected invocation exception: getWarnings : java.lang.NullPointerException : null
    at conpar.dbda.test.driver.record.BaseRecordHandler.invoke(BaseRecordHandler.java:74)

I encountered that on Informix an reproduced on HSQLDB. Please see attached test case (with Eclipse project setup).
As you can see when switching to plain HSQLDB without DbdaTestDriver (i.e. using the commented DriverManager.getConnection statement instead), the underlying JDBC driver's getWarnings() returns successfully.

Regards

[mbcon]

Hi T-42,

Thanks a lot for filing this defect. Using your testcase, I think I was able to reproduce the bug locally. I have provided a fix: version R3_1. Would be great if you could give it a try ...

Many thanks,

mbcon

[T-42]

Hi,

thank you very much for the bug fix. I can confirm that that issue is fixed also on Informix.

Unfortunately, I run into another issue which I could not isolate into a test:

java.lang.RuntimeException: Unexpected invocation exception: getDate : java.lang.NullPointerException : null
    at conpar.dbda.test.driver.record.BaseRecordHandler.invoke(BaseRecordHandler.java:75)
    at com.sun.proxy.$Proxy229.getDate(Unknown Source)
    at org.apache.commons.dbcp2.DelegatingResultSet.getDate(DelegatingResultSet.java:309)
    at org.apache.commons.dbcp2.DelegatingResultSet.getDate(DelegatingResultSet.java:309)
    at org.hibernate.type.descriptor.sql.DateTypeDescriptor$2.doExtract(DateTypeDescriptor.java:76)
    at org.hibernate.type.descriptor.sql.BasicExtractor.extract(BasicExtractor.java:47)
    at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:260)
    at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:256)
    at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:246)
    at org.hibernate.type.AbstractStandardBasicType.hydrate(AbstractStandardBasicType.java:332)
    at org.hibernate.persister.entity.AbstractEntityPersister.hydrate(AbstractEntityPersister.java:2844)
    at org.hibernate.loader.Loader.loadFromResultSet(Loader.java:1741)
    at org.hibernate.loader.Loader.instanceNotYetLoaded(Loader.java:1667)
    at org.hibernate.loader.Loader.getRow(Loader.java:1556)
    at org.hibernate.loader.Loader.getRowFromResultSet(Loader.java:740)
    at org.hibernate.loader.Loader.processResultSet(Loader.java:985)
    at org.hibernate.loader.Loader.doQuery(Loader.java:943)
    at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:349)
    at org.hibernate.loader.Loader.doList(Loader.java:2615)
    at org.hibernate.loader.Loader.doList(Loader.java:2598)
    at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2430)
    at org.hibernate.loader.Loader.list(Loader.java:2425)
    at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:502)
    at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:379)
    at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:216)
    at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1488)
    at org.hibernate.query.internal.AbstractProducedQuery.doList(AbstractProducedQuery.java:1445)
    at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1414)
...

caused by

java.lang.NullPointerException
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1500)
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
    at java.io.ObjectOutputStream.writeUnshared(ObjectOutputStream.java:415)
    at conpar.dbda.test.driver.backend.taurus.TaurusDataOut.writeObject(TaurusDataOut.java:100)
    at conpar.dbda.test.driver.record.RecordResultSet.afterMethodInvoke(RecordResultSet.java:80)
    at conpar.dbda.test.driver.record.BaseRecordHandler.invoke(BaseRecordHandler.java:69)
    at com.sun.proxy.$Proxy229.getDate(Unknown Source)
    at org.apache.commons.dbcp2.DelegatingResultSet.getDate(DelegatingResultSet.java:309)
    at org.apache.commons.dbcp2.DelegatingResultSet.getDate(DelegatingResultSet.java:309)
    at org.hibernate.type.descriptor.sql.DateTypeDescriptor$2.doExtract(DateTypeDescriptor.java:76)
    at org.hibernate.type.descriptor.sql.BasicExtractor.extract(BasicExtractor.java:47)
    at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:260)
    at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:256)
    at org.hibernate.type.AbstractStandardBasicType.nullSafeGet(AbstractStandardBasicType.java:246)
    at org.hibernate.type.AbstractStandardBasicType.hydrate(AbstractStandardBasicType.java:332)
    at org.hibernate.persister.entity.AbstractEntityPersister.hydrate(AbstractEntityPersister.java:2844)
    at org.hibernate.loader.Loader.loadFromResultSet(Loader.java:1741)
    at org.hibernate.loader.Loader.instanceNotYetLoaded(Loader.java:1667)
    at org.hibernate.loader.Loader.getRow(Loader.java:1556)
    at org.hibernate.loader.Loader.getRowFromResultSet(Loader.java:740)
    at org.hibernate.loader.Loader.processResultSet(Loader.java:985)
    at org.hibernate.loader.Loader.doQuery(Loader.java:943)
    at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:349)
    at org.hibernate.loader.Loader.doList(Loader.java:2615)
    at org.hibernate.loader.Loader.doList(Loader.java:2598)
    at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2430)
    at org.hibernate.loader.Loader.list(Loader.java:2425)
    at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:502)
    at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:379)
    at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:216)
    at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1488)
    at org.hibernate.query.internal.AbstractProducedQuery.doList(AbstractProducedQuery.java:1445)
    at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1414)
...

[mbcon]

Hi T-42, thanks for the feedback. Let's see what can be done regarding the issues you still have.

I see that you are using Hibernate with a connection pool, and the Taurus backend.

1. Since every connection needs to be proxied for the test driver to work, using a pool with a size > 1 is discouraged. Could you try to configure the pool to use only 1 connection (minimum size and maximum size)? Not sure if you can configure Hibernate to use no pool at all. Having done that, please run your tests again.
2. If the issue does not go away, try to use another backend instead of Taurus.

I do not have access to an Informix database, so maybe the Informix driver behaves different than the Oracle, MySQL, or Derby drivers I am using. Nevertheless I checked the code again for NPEs looking at the stacktrace you have sent. I see that by calling getDate() on the resultset, the driver tries to record the date object, which is not null and checked for being serializable. Using the ObjectOutputStream to write out the date object leads to an NPE. In the stacktrace, the code line numbers do not match with my JDK. Which JDK were you using?

Best regards,

mbcon

[T-42]

Hi mbcon,

with the connection pool size 1, our application hung. jstack showed threads in org.hibernate.query.Query.list with the same "parking to wait for <0x0000000642450680>", but no other line with that number (i.e. no owner of that monitor). After 25 minutes with no changes in jstack output, I killed the process. Anyway, this looks like a bug on our side, not in DbdaTestDriver.

"pool-2-thread-13" #366 prio=5 os_prio=0 tid=0x0000000029ec1800 nid=0x7ec waiting on condition [0x0000000044e9d000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x0000000642450680> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:583)
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:442)
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
        at org.apache.commons.dbcp2.PoolingDataSource.getConnection(PoolingDataSource.java:134)
        at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533)
        at org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy$LazyConnectionInvocationHandler.getTargetConnection(LazyConnectionDataSourceProxy.java:403)
        at org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy$LazyConnectionInvocationHandler.invoke(LazyConnectionDataSourceProxy.java:376)
        at com.sun.proxy.$Proxy208.prepareStatement(Unknown Source)
        at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:146)
        at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:172)
        at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareQueryStatement(StatementPreparerImpl.java:148)
        at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1940)
        at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1909)
        at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1887)
        at org.hibernate.loader.Loader.doQuery(Loader.java:932)
        at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:349)
        at org.hibernate.loader.Loader.doList(Loader.java:2615)
        at org.hibernate.loader.Loader.doList(Loader.java:2598)
        at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2430)
        at org.hibernate.loader.Loader.list(Loader.java:2425)
        at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:502)
        at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:379)
        at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:216)
        at org.hibernate.internal.StatelessSessionImpl.list(StatelessSessionImpl.java:461)
        at org.hibernate.query.internal.AbstractProducedQuery.doList(AbstractProducedQuery.java:1445)
        at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1414)
        at com.md.persistence.dao.BusinessTransactionDao.lambda$loadMultithreadedBM$9(BusinessTransactionDao.java:501)
        at com.md.persistence.dao.BusinessTransactionDao$$Lambda$146/919831713.call(Unknown Source)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - <0x00000007569e6260> (a java.util.concurrent.ThreadPoolExecutor$Worker)

I tried your suggestion and set backendHandler = conpar.dbda.test.driver.backend.hippo.HippoBackend (and changed pool size back to normal). That solved the issue for the record phase.

The replay phase failed, but that's not DbdaTestDriver's fault either. (Hibernate cannot determine the database dialect, and the corresponding property is ignored due to hard-coded values in a badly written library we use. Maybe I'll fix that later, but for now I am giving up.)

BTW, this was on JDK version 1.8.0_131.

Thanks for your support!

[mbcon]

Hi T-42,
no good news indeed. Of course I would have liked the fact that dbdatestdriver could have helped you whatever you are working on.

I will spend some more time with a) Hibernate/JPA and b) the Taurus backend.

If you want to invest some more time using the driver in the future, just keep in mind the basic idea and the restrictions that stem from that. Basically, the driver records all requests sent to the database serially, and therefore in replay mode, the database requests need to occur in exactly the same order.
This is usually the case with JUnit tests that run ordered and serially. It is of course not the case with a multi-user web application. And it might not be ok with more sophisticated environments that do things in parallel autonomously (requesting data from the database to fill a cache in the background, triggered by a timer ...).
I am using the driver mainly for regression testing a large base of reports developed in Java, that use data from large datawarehouses. But of course it is also good to test webservices and any type of application backends. Maybe this helps you to determin if dbdatestdriver is suitable for your environment as well.

Keep going,

mbcon

[T-42]

What I wanted to achieve was just to speed up some manual testing during which I did the same database accesses again and again (in a Spring Boot / Batch application). Since the database in question is accessed read-only, I don't need any cleanup to repeat the test.
For the moment, I think I'll get the task done faster if I just live with the performance of the real database. But I will probably come back to dbdatestdriver some day.

Thanks again. Regards,
T-42

[mbcon]

Hi T-42,

Thanks for sharing your situation and setup.
I have fixed a bug in the local Logger class while playing around with JPA/Hibernate. The bug was related to TRACE logging. Maybe you ran into that issue. JPA works fine, at least for the (limited) testcase.

Thanks again for providing me some input to improve the library.

CU,

mbcon