java - EclipseLink Distributed Cache causing Null Primary Keys, what? -

java - EclipseLink Distributed Cache causing Null Primary Keys, what? -

ok, going long, here's tl;dr: when distributed cache coordination enabled, 1 of apps starts degrade in integrity , throws unusual exception. 'fix' disable caching. sucks. please help find issue in eclipselink can patch it!

eclipselink 2.5.1

we have these entities:

@mappedsuperclass public abstract class identifiable implements serializable { private static final long serialversionuid = 1l; @id @column(name = "id", nullable = false, updatable = false) @generatedvalue(strategy = generationtype.identity) public integer id; @version @column(name = "meta__version") private integer version; }

@entity @table(name = "customers", indexes = { @index(columnlist = "phone_number"), @index(columnlist = "email_address") }) @cacheable public class client extends identifiable { private static final long serialversionuid = 1l; @column(name = "last_name") public string lastname; @column(name = "first_name") public string firstname; .... }

@entity @table(name = "notes", indexes = { @index(columnlist = "customer_id, is_cleared") }) @cacheable public class note extends identifiable { private static final long serialversionuid = 1l; @manytoone @joincolumn(name = "customer_id") public client customer; @manytoone @joincolumn(name = "agent_id") public agent agent; @size(min = 1, max = 32767) @column(name = "text", length = 32767) public string text; @column(name = "is_cleared") public boolean iscleared; }

<?xml version="1.0" encoding="utf-8"?> <persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xsi:schemalocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd"> <persistence-unit name="databunker"> <provider>org.eclipse.persistence.jpa.persistenceprovider</provider> <jta-data-source>jdbc/databunker</jta-data-source> <mapping-file>databunker.orm.xml</mapping-file> <class>com.mycompany.customer</class> <class>com.mycompany.note</class> <exclude-unlisted-classes>true</exclude-unlisted-classes> <shared-cache-mode>enable_selective</shared-cache-mode> <properties> <!-- un-comment these properties dump generated sql --> <!-- <property --> <!-- name="eclipselink.logging.level.sql" --> <!-- value="fine" /> --> <!-- <property --> <!-- name="eclipselink.logging.parameters" --> <!-- value="true" /> --> <property name="eclipselink.ddl-generation.output-mode" value="database" /> <property name="eclipselink.ddl-generation" value="create-or-extend-tables" /> <property name="eclipselink.weaving.changetracking" value="false" /> <property name="javax.persistence.query.timeout" value="15000" /> <property name="eclipselink.cache.size.default" value="10000" /> <property name="eclipselink.cache.coordination.protocol" value="jms" /> <property name="eclipselink.cache.coordination.jms.topic" value="openejb:resource/jms/eclipselinktopic" /> <property name="eclipselink.cache.coordination.jms.factory" value="openejb:resource/jms/eclipselinktopicconnectionfactory" /> <property name="eclipselink.cache.coordination.jms.reuse-topic-publisher" value="true" /> <property name="eclipselink.cache.coordination.remove-connection-on-error" value="true" /> </properties> </persistence-unit> </persistence>

we have 2 apps. app1 writing/updates on customers, app2 writing/updates on notes. both utilize exact same persistent-unit library jar. after running awhile, notes app starts failing stack trace particular client object. restarting server fixes problem until starts occurring client object.

[http-apr-8080-exec-1] error c.mycompany.notes.restapi.restexceptionmapper - toresponse() unexpected exception javax.transaction.rollbackexception: unable commit: transaction marked rollback @ org.apache.geronimo.transaction.manager.transactionimpl.commit(transactionimpl.java:272) ~[geronimo-transaction-3.1.1.jar:3.1.1] @ org.apache.geronimo.transaction.manager.transactionmanagerimpl.commit(transactionmanagerimpl.java:252) ~[geronimo-transaction-3.1.1.jar:3.1.1] @ org.apache.openejb.core.transaction.jtatransactionpolicy.completetransaction(jtatransactionpolicy.java:328) ~[openejb-core-4.6.0.1.jar:4.6.0.1] @ org.apache.openejb.core.transaction.txrequired.commit(txrequired.java:75) ~[openejb-core-4.6.0.1.jar:4.6.0.1] @ org.apache.openejb.core.transaction.ejbtransactionutil.afterinvoke(ejbtransactionutil.java:76) ~[openejb-core-4.6.0.1.jar:4.6.0.1] @ org.apache.openejb.core.stateless.statelesscontainer._invoke(statelesscontainer.java:231) ~[openejb-core-4.6.0.1.jar:4.6.0.1] @ org.apache.openejb.core.stateless.statelesscontainer.invoke(statelesscontainer.java:181) ~[openejb-core-4.6.0.1.jar:4.6.0.1] @ org.apache.openejb.core.ivm.ejbobjectproxyhandler.synchronizedbusinessmethod(ejbobjectproxyhandler.java:268) ~[openejb-core-4.6.0.1.jar:4.6.0.1] @ org.apache.openejb.core.ivm.ejbobjectproxyhandler.businessmethod(ejbobjectproxyhandler.java:263) ~[openejb-core-4.6.0.1.jar:4.6.0.1] @ org.apache.openejb.core.ivm.ejbobjectproxyhandler._invoke(ejbobjectproxyhandler.java:86) ~[openejb-core-4.6.0.1.jar:4.6.0.1] @ org.apache.openejb.core.ivm.baseejbproxyhandler.invoke(baseejbproxyhandler.java:303) ~[openejb-core-4.6.0.1.jar:4.6.0.1] @ com.sun.proxy.$proxy108.getnotes(unknown source) ~[na:na] @ com.mycompany.notes.restapi.notetakercontroller.getnotes(notetakercontroller.java:68) ~[notetakercontroller.class:na] @ com.mycompany.notes.restapi.notetakercontroller$$owbnormalscopeproxy0.getnotes(com/mycompany/notes/restapi/notetakercontroller.java) ~[na:na] @ sun.reflect.generatedmethodaccessor70.invoke(unknown source) ~[na:na] @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:43) ~[na:1.7.0_60] @ java.lang.reflect.method.invoke(method.java:606) ~[na:1.7.0_60] @ org.apache.cxf.service.invoker.abstractinvoker.performinvocation(abstractinvoker.java:180) ~[cxf-api-2.6.9.jar:2.6.9] @ org.apache.cxf.service.invoker.abstractinvoker.invoke(abstractinvoker.java:96) ~[cxf-api-2.6.9.jar:2.6.9] @ org.apache.cxf.jaxrs.jaxrsinvoker.invoke(jaxrsinvoker.java:165) [cxf-rt-frontend-jaxrs-2.6.9.jar:2.6.9] @ org.apache.cxf.jaxrs.jaxrsinvoker.invoke(jaxrsinvoker.java:89) [cxf-rt-frontend-jaxrs-2.6.9.jar:2.6.9] @ org.apache.openejb.server.cxf.rs.autojaxrsinvoker.invoke(autojaxrsinvoker.java:66) [openejb-cxf-rs-4.6.0.1.jar:4.6.0.1] @ org.apache.cxf.interceptor.serviceinvokerinterceptor$1.run(serviceinvokerinterceptor.java:57) [cxf-api-2.6.9.jar:2.6.9] @ org.apache.cxf.interceptor.serviceinvokerinterceptor.handlemessage(serviceinvokerinterceptor.java:93) [cxf-api-2.6.9.jar:2.6.9] @ org.apache.cxf.phase.phaseinterceptorchain.dointercept(phaseinterceptorchain.java:262) [cxf-api-2.6.9.jar:2.6.9] @ org.apache.cxf.transport.chaininitiationobserver.onmessage(chaininitiationobserver.java:121) [cxf-api-2.6.9.jar:2.6.9] @ org.apache.cxf.transport.http.abstracthttpdestination.invoke(abstracthttpdestination.java:237) [cxf-rt-transports-http-2.6.9.jar:2.6.9] @ org.apache.openejb.server.cxf.rs.cxfrshttplistener.onmessage(cxfrshttplistener.java:170) [openejb-cxf-rs-4.6.0.1.jar:4.6.0.1] @ org.apache.openejb.server.rest.rsservlet.service(rsservlet.java:53) [openejb-rest-4.6.0.1.jar:4.6.0.1] @ javax.servlet.http.httpservlet.service(httpservlet.java:727) [servlet-api.jar:na] @ org.apache.catalina.core.applicationfilterchain.internaldofilter(applicationfilterchain.java:303) [catalina.jar:7.0.53] @ org.apache.catalina.core.applicationfilterchain.dofilter(applicationfilterchain.java:208) [catalina.jar:7.0.53] @ org.apache.tomcat.websocket.server.wsfilter.dofilter(wsfilter.java:52) [tomcat7-websocket.jar:7.0.53] @ org.apache.catalina.core.applicationfilterchain.internaldofilter(applicationfilterchain.java:241) [catalina.jar:7.0.53] @ org.apache.catalina.core.applicationfilterchain.dofilter(applicationfilterchain.java:208) [catalina.jar:7.0.53] @ com.mycompany.notes.restapi.security.hmacsecurityfilter.dofilter(hmacsecurityfilter.java:79) [hmacsecurityfilter.class:na] @ org.apache.catalina.core.applicationfilterchain.internaldofilter(applicationfilterchain.java:241) [catalina.jar:7.0.53] @ org.apache.catalina.core.applicationfilterchain.dofilter(applicationfilterchain.java:208) [catalina.jar:7.0.53] @ com.mycompany.cdi.requestbufferingfilter.dofilter(requestbufferingfilter.java:50) [java-common-1.0.10.jar:na] @ org.apache.catalina.core.applicationfilterchain.internaldofilter(applicationfilterchain.java:241) [catalina.jar:7.0.53] @ org.apache.catalina.core.applicationfilterchain.dofilter(applicationfilterchain.java:208) [catalina.jar:7.0.53] @ org.apache.catalina.core.standardwrappervalve.invoke(standardwrappervalve.java:220) [catalina.jar:7.0.53] @ org.apache.catalina.core.standardcontextvalve.invoke(standardcontextvalve.java:122) [catalina.jar:7.0.53] @ org.apache.tomee.catalina.openejbvalve.invoke(openejbvalve.java:45) [tomee-catalina-1.6.0.1.jar:1.6.0.1] @ org.apache.catalina.authenticator.authenticatorbase.invoke(authenticatorbase.java:501) [catalina.jar:7.0.53] @ org.apache.catalina.core.standardhostvalve.invoke(standardhostvalve.java:170) [catalina.jar:7.0.53] @ org.apache.catalina.valves.errorreportvalve.invoke(errorreportvalve.java:98) [catalina.jar:7.0.53] @ org.apache.catalina.valves.accesslogvalve.invoke(accesslogvalve.java:950) [catalina.jar:7.0.53] @ org.apache.catalina.core.standardenginevalve.invoke(standardenginevalve.java:116) [catalina.jar:7.0.53] @ org.apache.catalina.connector.coyoteadapter.service(coyoteadapter.java:408) [catalina.jar:7.0.53] @ org.apache.coyote.http11.abstracthttp11processor.process(abstracthttp11processor.java:1040) [tomcat-coyote.jar:7.0.53] @ org.apache.coyote.abstractprotocol$abstractconnectionhandler.process(abstractprotocol.java:607) [tomcat-coyote.jar:7.0.53] @ org.apache.tomcat.util.net.aprendpoint$socketwithoptionsprocessor.run(aprendpoint.java:2378) [tomcat-coyote.jar:7.0.53] @ java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor.java:1145) [na:1.7.0_60] @ java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:615) [na:1.7.0_60] @ java.lang.thread.run(thread.java:745) [na:1.7.0_60] caused by: javax.persistence.persistenceexception: exception [eclipselink-7197] (eclipse persistence services - 2.5.2.v20131113-a7346c6): org.eclipse.persistence.exceptions.validationexception exception description: null or 0 primary key encountered in unit of work clone [customer[agent=agent[agentname=api user,id=1,version=1],address=address[streetaddress=somewhere road,city=kansas city,zipcode=66210,state=state[postalabbreviation=ks,id=38,version=1],id=298219,version=1],leadstatus=leadstatus[name=new,id=1,version=1],leadsource=leadsource[name=irms,id=9,version=1],project=project[campaignname=outbound59734,id=2,version=1],version=7]], primary key [null]. set descriptors idvalidation or "eclipselink.id-validation" property. @ org.eclipse.persistence.internal.jpa.entitymanagersetupimpl$1.handleexception(entitymanagersetupimpl.java:696) ~[eclipselink-2.5.2-m1.jar:2.5.2.v20131113-a7346c6] @ org.eclipse.persistence.transaction.abstractsynchronizationlistener.handleexception(abstractsynchronizationlistener.java:275) ~[eclipselink-2.5.2-m1.jar:2.5.2.v20131113-a7346c6] @ org.eclipse.persistence.transaction.abstractsynchronizationlistener.beforecompletion(abstractsynchronizationlistener.java:170) ~[eclipselink-2.5.2-m1.jar:2.5.2.v20131113-a7346c6] @ org.eclipse.persistence.transaction.jtasynchronizationlistener.beforecompletion(jtasynchronizationlistener.java:68) ~[eclipselink-2.5.2-m1.jar:2.5.2.v20131113-a7346c6] @ org.apache.geronimo.transaction.manager.transactionimpl.beforecompletion(transactionimpl.java:527) ~[geronimo-transaction-3.1.1.jar:3.1.1] @ org.apache.geronimo.transaction.manager.transactionimpl.beforecompletion(transactionimpl.java:511) ~[geronimo-transaction-3.1.1.jar:3.1.1] @ org.apache.geronimo.transaction.manager.transactionimpl.beforeprepare(transactionimpl.java:413) ~[geronimo-transaction-3.1.1.jar:3.1.1] @ org.apache.geronimo.transaction.manager.transactionimpl.commit(transactionimpl.java:262) ~[geronimo-transaction-3.1.1.jar:3.1.1] ... 55 mutual frames omitted caused by: org.eclipse.persistence.exceptions.validationexception: exception description: null or 0 primary key encountered in unit of work clone [customer[agent=agent[agentname=api user,id=1,version=1],address=address[streetaddress=somewhere road,city=kansas city,zipcode=66210,state=state[postalabbreviation=ks,id=38,version=1],id=298219,version=1],leadstatus=leadstatus[name=new,id=1,version=1],leadsource=leadsource[name=irms,id=9,version=1],project=project[campaignname=outbound59734,id=2,version=1],version=7]], primary key [null]. set descriptors idvalidation or "eclipselink.id-validation" property. @ org.eclipse.persistence.exceptions.validationexception.nullprimarykeyinunitofworkclone(validationexception.java:1469) ~[eclipselink-2.5.2-m1.jar:2.5.2.v20131113-a7346c6] @ org.eclipse.persistence.descriptors.changetracking.deferredchangedetectionpolicy.calculatechanges(deferredchangedetectionpolicy.java:109) ~[eclipselink-2.5.2-m1.jar:2.5.2.v20131113-a7346c6] @ org.eclipse.persistence.descriptors.changetracking.deferredchangedetectionpolicy.calculatechangesforexistingobject(deferredchangedetectionpolicy.java:56) ~[eclipselink-2.5.2-m1.jar:2.5.2.v20131113-a7346c6] @ org.eclipse.persistence.internal.sessions.unitofworkimpl.calculatechanges(unitofworkimpl.java:664) ~[eclipselink-2.5.2-m1.jar:2.5.2.v20131113-a7346c6] @ org.eclipse.persistence.internal.sessions.unitofworkimpl.committodatabasewithchangeset(unitofworkimpl.java:1516) ~[eclipselink-2.5.2-m1.jar:2.5.2.v20131113-a7346c6] @ org.eclipse.persistence.internal.sessions.unitofworkimpl.issuesqlbeforecompletion(unitofworkimpl.java:3168) ~[eclipselink-2.5.2-m1.jar:2.5.2.v20131113-a7346c6] @ org.eclipse.persistence.internal.sessions.repeatablewriteunitofwork.issuesqlbeforecompletion(repeatablewriteunitofwork.java:352) ~[eclipselink-2.5.2-m1.jar:2.5.2.v20131113-a7346c6] @ org.eclipse.persistence.transaction.abstractsynchronizationlistener.beforecompletion(abstractsynchronizationlistener.java:158) ~[eclipselink-2.5.2-m1.jar:2.5.2.v20131113-a7346c6] ... 60 mutual frames omitted

eclipselink distributed updates default. seek @cache(coordinationtype = cachecoordinationtype.invalidate_changed_objects) instead

java java-ee caching jpa eclipselink