Publisher is up but it's sort of down.
An instance of Publisher that I had a chance to touch recently went down with a puzzling exception and unknown cause. What better way to liven up the day of your friendly neighborhood portal than a vague and somewhat unexplained service error for a component of your deployment that has not given you any trouble since the day you installed it.
A few triage details:
- Publisher Explorer is loading and navigating the folders is possible.
- Modifying and viewing select Publisher Explorer items is possible.
- However, a notable number of items fail to load: both content items associated to a data entry template, and the corresponding data entry template appear in the explorer view but do not load when clicked on.
- The Publisher diagnostic runs to completion showing no signs of a problem
- Attempts to load the inline Publisher editor fail.
So the fix to this was not to found in restarting Publisher, or any variable combination of Portal and/or Portal related services - for two reasons, 1. That would have made for a wholly uninterested blog post - and 2. We tried, and it did not solve anything.
com.plumtree.content.data.error.DataNotFoundException (DataPortalUUIDNotFound: Portal object not found.) at com.plumtree.content.rdbms.types.Portal$3.execute(Portal.java:131) at com.plumtree.content.util.transaction.Operation.run(Operation.java:264) at com.plumtree.content.rdbms.impl.RdbmsOperation.run(RdbmsOperation.java:63) at com.plumtree.content.rdbms.types.Portal.selectPortalId(Portal.java:124) at com.plumtree.content.data.impl.PortalToolboxRep.getPortalId(PortalToolboxRep.java:64) at com.plumtree.content.mbeans.portal.PortalConnectorImpl.startService(PortalConnectorImpl.java:81) at com.plumtree.content.management.AbstractContentManagementBean.start(AbstractContentManagementBean.java:109)
What are our logs telling us? Tomcat has a pretty consistent exception and stack trace that is being bubbled up to the browser when it fails to load the page(s) in question:
BEA ALI Publisher startup failed: javax.management.MBeanException: nested exception is com.plumtree.content.data.error.DataNotFoundException (DataPortalUUIDNotFound: Portal object not found.)
A pretty good eplanation of what is going on here - "Publisher startup failed", so we can throw out our diagnostic screen, which gave us a green light on Publisher components, the logs say it clear Publisher is not starting up (entirely/properly). The focus is now on Publisher and its inner-workings and not a rogue network, firewall, external configuration mishap, etc.
Diving deeper into the Tomcat log we went off to track down where our exceptions turn up. Note that our browser side error mentions that 'startup failed' so our next step should be to power down Publisher, back-up and clear the logs, and then fire Publisher back up and see if anything pops up as 'Hey, you guys, this is definitely broken - over here, right here'. The error we found was almost identical to the stack trace and exception we saw on the browser side, with the addition that now we saw the error exactly where we thought it should be, in the deployment of the Publisher web application.
This time we focus on:
DataPortalUUIDNotFound: Portal object not found.
The UUID for portal components are visible by navigating in your administration view to a Portal object's "Properties" and the unique ID for that object will be listed part way down. For Publisher, the startup is failing this time around because a Portal object it is expecting is not there. The UUIDs of interest to us, that are related to Publisher will be located in:
Digging around for UUIDs we checked and verified that the ContentServerGatewayWebServiceUUID did match the Publisher Gateway Web Service in the Portal administration view, but what we found upon further inspection was the UUID specified by WorkListPortletWebServiceUUID did not match the corresponding Publisher Work List Web Service. This is our likely culprit, and is confirmed with a quick peek at the code for PortalConnectorImpl.java we saw that the code was failing at getting the Work List Portlet UUID.
What is strange here is that it seems as though the UUID value does not match out of the box.
But a quick swap in with the proper UUID value grabbed from the Portal administration side for the Publisher Work List Web Service and one last restart of Publisher and all was right with our day, once again.