Step 1: Install Maven

Here’s a link to the official maven2 installation guide. Here are the steps I used to install on my windows laptop:

• Download and install the Sun Java JDK 6 update 18. Here’s the link: http://java.sun.com/javase/downloads/widget/jdk6.jsp.
• Download the latest maven installation zip file from here http://maven.apache.org/download.html
• Extract the zip and move ‘apache-maven-2.2.1′ into c:/Program Files/Apache Software Foundation/apache-maven-2.2.1
• Create a windows environment variable named “M2_HOME” and pointed it to the maven installation directory: M2_HOME=c:\Program Files\Apache Software
  Foundation\apache-maven-2.2.1
• Create a windows environment variable named “M2_REPO” and point it to “C:\Documents And Settings\dave.m2\repository” (this will make more sense after you read the section below on repositories).
• Make sure that the “JAVA_HOME” environment variable is set. For example, the JAVA_HOME environment variable on my windows laptop is set to C:\Program Files\Java\jdk1.6.0_18
• Add the mvn command to the windows “PATH” environment variable like so: PATH=…;%M2_HOME%\bin
• Finally, Test to ensure maven was installed correctly. Open a Windows DOS Command Prompt, and type “mvn -version”. If you see the following you should be good to go:
  C:\Users\Dave>mvn -version
Apache Maven 2.2.1 (r801777; 2009-08-06 15:16:01-0400)
Java version: 1.6.0_18
Java home: c:\Program Files (x86)\Java\jdk1.6.0_18\jre
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 7" version: "6.1" arch: "x86" Family: "windows"

Maven is installed. Let’s take a break, here’s an airplane landing with one wing:

Step 1b: Get the code

Here’s a Zip containing the complete portlet code: function1-twitter-portlet. Feel free to refer to this as I describe the steps to create a maven portlet project below. This zip contains a fully functionaly maven project. Extract the zip and open the pom.xml file. Add your twitter credentials to the file, and then open a command prompt. Change directories to wherever you extracted the zip and then run the following command:

mvn install

If maven is installed correctly, and your twitter credentials are valid, this command should download required jars from maven, run tests to make sure it can connect to twitter, and build a deploy-able portlet war file under directory named ‘target’!

Step 2: Create Basic pom.xml

Now that Maven is installed, we need to create an empty Maven Project. A the bare minimum, a Maven Project is nothing more than a folder that contains a single file named ‘pom.xml’. In Maven terminology, “POM” stands for “Project Object Model”. Maven will look for a pom.xml file to discover everything it needs to know about our Twitter Portlet Java Project. Once you’ve used Maven for a few different projects, this will become second nature…I promise. Eventually you’ll have a template pom.xml that you can simply copy into a new directory each time you’d like to create a new maven project.

First, we’ll create a new directory named ‘twitter-portlet’ (or just follow along by browsing the code from the zip file). Next, create a new pom.xml file and copy in the following

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.function1</groupId>
  <artifactId>wci-twitter-portlet</artifactId>
  <packaging>war</packaging>
  <version>0.1</version>

  <name>WebCenter Interation Twitter Portlet</name>
</project>

So far, we haven’t done anything more than describe our Twitter Portlet Project to Maven. Here’s a short overview of each line:

• The “modelVersion” is required by all Maven2 Projects and always has the value 4.0.0.
• The “groupidId” tells Maven that this project belongs in the “com.function1″ group. The best practice is to use your company’s name for the groupid, but it can be whatever you choose. Multiple projects (multiple pom.xml files) can all share the same group id.
• The “artifactId” defines the name of our specific project: wci-twitter-portlet.
• “packaging” tells maven what type of project we’re building. 99% of the time, this is either going to be “war” or “jar”. In this case, of course, we want a war to deploy as a remote portlet.
• The “version” tag defines what version of the wci-twitter-portlet we’re working on. In this case, its the first version ever so I chose to call it version “0.1″.
• The “name” is just a human readable name. Maven will use this when generating documentation.

Step 3: Directory Structure

Maven is very particular about the directory structure of a project. If this is the first time you’ve used Maven, this might make you feel a bit trapped. When I first started with Maven I tried to fight it by using my own directory structure. Over time, I’ve come to see the light. Trust me, just go with it. I’ve realized that the directory structure used by Maven is a proven best practice and in the end it really makes things consistent and easy. Here’s what the basic directory structure looks like:

• Create a directory named src/main/java. This is where we will put custom java code.
• Next create a directory named src/test/java. This is where unit test code goes.
• Now create a directory named src/main/resources. Here’s were all your configuration files go. For example: DTD, .properties files, xml files (such as log4j.xml), etc.
• Finally, create a directory named src/main/webapps. Here’s where your jsp’s, and WEB-INF/web.xml file, and any static stuff like images, and javascript.

Ok, directory structure is done, Time for a break! I think the things that this guy builds are freaking cool:

Step 4: Generate an Eclipse Project

By this point, we’ve given maven enough information to be able to create an Eclipse project for us. switch to your command prompt and run the following command:

mvn eclipse:eclipse

You should see a message “BUILD SUCCESFUL”. Open Eclipse and choose “File -> Import -> Existing Projects into Workspace”. Choose the twitter-portlet directory and import the wci-twitter-portlet eclipse project (notice Maven used the artifactid for the name of the eclipse project).

Step 5: IDK and DEPENDENCIES

Our Twitter Portlet will need to use the IDK to get the current username. We’ll need to depend on the WCI IDK. So, the next step is to download the IDK (if you haven’t yet). Once it’s downloaded, extract the installation zip and then find the WEB-INF directory under idk/10.3.0/devkit. Copy the WEB-INF folder into the wci-twitter-portlet directory into src/main/webapp/.

Now, delete the lib directory! Yep, you heard me. Just like taking off a bandaid, rip that lib directory out of there. After reading this article, you’ll never have to copy the idk lib directory again (ok, except maybe when they upgrade to 11g). Now it’s time to tell Maven to manage these jars for us.

Maven Repositories

Which leads us to the concept of Maven “repositories”. Maven looks for jar dependencies inside repositories. The Maven Book does a much better job at describing the concept than I can, but basically, a repository is simply directory structure containing maven artifacts.

By default, Maven looks first for dependencies here: http://repo1.maven.org/maven2/

Thanks to convention over configuration, everything inside the repository is organized using the following structure:

groupId/artifactId/version/artifactId-version.ext

For example, log4j, version 1.2.9 can be found here:

http://repo1.maven.org/maven2/log4j/log4j/1.2.9/.

In this case, groupid and artifactid are both set to ‘log4j’. Here’s another example, ‘axis’ can be found here:

http://repo2.maven.org/maven2/org/apache/axis/axis/1.4/

There are several tools available to help search the maven repository for stuff. There’s a Maven-Eclipse plugin that allows you to search and automatically add dedpendencies to a maven project. There’s also a website that allows you to search the maven repository: http://mvnrepository.com/. For example, when you do a search for ‘log4j’ this site will show you all available versions. If you drill down into version 1.2.15, you’ll see the following:

<dependency>
    <groupId>log4j</groupId>
    <artifactId>log4j</artifactId>
    <version>1.2.15</version>
</dependency>

I know, I know, xml makes my eyes glaze over too. To be honest, I’m not crazy about the fact that maven relies 100% on xml. When you write java code, you gotta deal with xml, what can I say?

So, if we added this snippet of xml to or pom.xml, maven will know that our project uses log4j. In fact, maven will be so nice as to find and download the appropriate log4j jar and make it available on the classpath of our project.

Maven is smart – as smart as they come. Instead of downloading jars every time you do a build, it only downloads the jars once. And instead of downloading the same jars into every maven project, it downloads them into a directory under your operating system user’s home directory. For example, on windows, this is C:\Documents and Settings\Dave\.m2\repository. On mac, this is /Users/Dave/.m2/repository.

Ok, that was a lot. Here’s a quick summary: The Maven repository located at http://repo2.maven.org is a remote repository. When you tell maven that you need log4j by including the xml snippet above, it will first check your local maven repository (which is under your os user’s home directory). If it can’t find the jar, it will then check http://repo1.maven.org/maven2/.

Dealing with IDK Jars

So, the problem is that none of the idk jars are available in either of those places. To solve this, we’ll need to copy the idk jars into a local maven repository of some sort.

The first step in doing so, is to re-arrange the jars inside the idk’s lib directory to match the directory structure that maven expects. Like so:

Next, we need to tell Maven to look at our new local directory repository in addition to the default repo at http://repo2.maven.org/maven2. Here’s the snippet of xml inside pom.xml that does so:

<repositories>
  <repository>
    <id>thirdparty</id>

    <name>Thirdparty Jars</name>
    <url>file://${basedir}/thirdparty/m2/repository</url>
    <layout>default</layout>
  </repository>

</repositories>

Now that the idk dependencies are available, lets add them to our project. We’re going to add a snippet of xml for each jar. Here’s a few examples to give you an idea:

 <dependencies>
   <dependency>

     <groupId>com.oracle.idk</groupId>
     <artifactId>activation</artifactId>
     <version>10.3</version>
  </dependency>

  <dependency>
    <groupId>com.oracle.idk</groupId>
    <artifactId>axis</artifactId>
    <version>10.3</version>

  </dependency>
  <dependency>
    <groupId>com.oracle.idk</groupId>
    <artifactId>commons-discovery</artifactId>
    <version>10.3</version>

  </dependency>
... more depenedencies ...
</dependencies>

Phwew! We’re getting to the end…before we do, here’s a old spice commercial that cracks me up:

Step 6: Write the Code!

At this point, we have everything to need to create a portlet that uses the IDK. Please take a look inside the zip to see the code. But why stop there? Let’s add some twitter! It just so happens that there’s a library called twitter4j available via maven:

    <dependency>
      <groupId>org.twitter4j</groupId>
      <artifactId>twitter4j-core</artifactId>
      <version>2.1.0</version>

    </dependency>

Run mvn eclipse:eclipse again to update your eclipse project with the new twitter4j jar. (If Eclipse is open while you run this command, remember to do a “refresh” in Eclipse so that Eclipse see’s the new jar)

Now, we can add some code to our portlet that uses twitter4j to retrieve the latest status from each twitter friend. Here’s the relevant snippet. Feel free to peruse the full source int he zip file:

 public List<String> getFriendsMostRecentTweet() {
    List<String> tweets = new ArrayList<String>();
    String senderID = getTwitterLogin();
    String senderPassword = getTwitterPassword(); 

    Twitter twitter = new TwitterFactory().getInstance(senderID,
            senderPassword);
    PagableResponseList<User> statuses;
    try {
        statuses = twitter.getFriendsStatuses();
        for (User nextTwitterUser : statuses) {
            tweets.add(nextTwitterUser.getName() + ": "
                    + nextTwitterUser.getStatusText());
        }
    } catch (TwitterException e) {
        System.out.println("Unable to retrieve tweets");
        return null;
    }
    return tweets;
}

Finally: Running the command “mvn install” will create a war file named wci-twitter-portlet-0.1.war under the target directory. Deploy that war to tomcat (or weblogic or whatever you’re using as a remote java portal server). Create the remote server, webservice, and portlet objects inside the WCI Portlet and marvel at the twitter messages!

PT Tags give you the ability to use out-of-the-box coding logic within simple HTML pages. With a little patience and a slightly different way of thinking about logic flow, PT Tags can be an inexpensive answer to your UI problem.

Function1 has been “roomies” with PT Tags for quite some time, and we love our often confusing friend. To help educate our readers on PT Tags, here’s an example of solving a joined communities tag problem for portal navigation.

The <pt:ptdata.mycommunitiesdata/> tag is a useful array to get all the communities for your portal installation. However the tag includes both joined and mandatory communities for a user. Recently, a customer asked us to remove mandatory and mandatory with tab communities so the navigation could be seperated. How? we thought.

The solution is to use a different list of data from <pt:data.mandtabcommsdata />. This tag contains an array of mandatory with tab communities. Our thought process was to loop through the communities data and compare it to a continuous loop of mandatory tab data. Whenever a link was found that did not have a matching mandatory link, we printed it as a joined community.

Those mandatory tab communities might still be needed elsewhere. The following logic first checks if there are any mandatory communities, then loops through all communities and prints a nice link. You can include any css on these links simply by adding the class attribute to the <pt:core.html/> tag.

Finally, administrators and community managers might need to edit their communities just like the out-of-the-box navigation. The following logic will print any additional buttons such as “manage communities”.

DOWNLOAD THE FULL CODE HERE

We’ve noticed that we spend so much time with Aqualogic that we think PT Tags is a person. Since it doesn’t really have a face, we’ve decided it would probably look a lot like Puck from MTV’s Real World.

If you peruse maven articles out in the blog-o-sphere, this is the message you’ll hear:

Using Maven will make you a better developer. Using Maven will make your organization more productive. Maven will solve your integration issues. Maven encourages unit testing. Maven will enforce DRY best practices and Convention over Configuration. Maven will save the rainforest and will give you thick, shiny, lustrous hair. Marcia, Marcia, Marcia! … Maven, Maven, Maven!

You (or your software dev team) might have already written quite a bit of java web apps and have managed to get by pretty well without Maven. You may have even written some killer java portal apps which your organization can’t live without.  

If you’re like me, you’re also probably a bit tired of learning scripting languages (ant, perl, make, bash, dos, ruby, on and on, and on). You’re probably hesitant to make the investment in learning another. Well, I hear you. I feel your pain. So, in this article, I’d like to give you some idea about what types of tedious tasks that Maven can help you eliminate (and by doing so, possibly save significant development cost). I also want to try to convince you that Maven is worth the effort. Stay tuned for later articles where I will dive deeper into the actual implementation details of how to best use Maven to manage your java projects.

So, to prove it to you that Maven is well worth the investment, I’ll walk you through a recent experience. As a new employee at Function1, I am very excited about the product suite currently in development. In particular, I wanted to help add some functionality to one of our new products named Formbuilder.

Remember Studio Server? You know, the one with the Tetris Easter Egg? Well, Formbuilder is everything Studio Server was, plus an easier to use interface and updated with the latest and greatest technologies. Formbuilder really makes it a no-brainer to create and administer surveys and polls. Formbuilder can be run as a standalone webapp inside any of the standard java web containers. It can also be installed to run as a portlet with tight integration with Web Center Interaction.

Here are the steps it took me (as a brand new F1 employee, totally unfamiliar with Formbuilder source code) to get to the point where I could compile and run Formbuilder in a development environment without Maven. Afterwards, I’ll compare how each step might have been accomplished using Maven. And you can be the judge ;-)

Formbuilder is a standard java web application. On the back end, it uses servlets to manage requests. The servlets talk to a service layer powered by hibernate that manages persisting and retrieving data to database. On the front end, it uses GWT and Ext-JS to provide the views. Administrators can create surveys (using a very slick and easy-to-use user interface, imo):

And, of course, Formbuilder also provides a page where you can complete and submit surveys:

When I first got a hold of the source code, I was really impressed how well everything was organized. The folder structure followed the normal java development best practices and coding conventions. There were no surprises from a software development point of view.  Even so, getting to the point where I could actually write code, was pretty slow going. If you’ve ever inherited a non-trivial java application to maintain, I’m sure you can relate. And in my experience, this “ramp up” time is generally accepted by most development teams as something that you “just have to roll up your sleeves and get done”. Here are the steps that were necessary for me to get from having zero knowlege of source code to where I was able to compile and run the Formbuilder application:

Check Out Source Code

The first step was to get the source code. A few emails and I had access to subversion and I was able to check out the latest and greatest using the svn command line tool.

Create Eclipse Project

Eclipse stores info about what it thinks are java projects under a file called “.project” and a folder named “.settings”. Since these had been checked into subversion, I was able to import the code directly into eclipse as an eclipse java project. Unfortunately, this only kinda-sorta worked. Since the eclipse project had been build on another machine, the Eclipse files contained hard coded paths. There were red X’s everywhere and Eclipse was not happy.

Resolved Dependencies

One of the biggest hassles working with Java is managing the thirdparty jars that each web app depends on. In this case, it was easy to see that all required jars were inside the war/WEB-INF/lib directory. However, I also noticed that all these jars had been checked into source control. For small to medium sized projects, checking jars into source control works ok. But, in my experience, as the source code grows so does the jar dependencies and pretty soon the number of jars becomes unmanageable. Checkouts from svn can quickly become excruciatingly slow. In this case, it wasn’t a big problem (but something to keep in mind for future maven articles :). I manually added each jar to the eclipse project’s build path and that made Eclipse a little happier, making a lot of the existing red X’s go away.

Compiling the UI Code

Google Web Toolkit (GWT) is a technology that allows you to write java and have it compiled into javascript. Formbuilder uses GWT to create some really slick web user interfaces. So, the next step was to download and install the Eclipse GWT plugin. This plugin handles all the configuration necessary to be able to “compile” gwt java code. After setting up the plugin, eclipse was 100% happy, all the compile errors (Red X’s) in eclipse were satisfied!

Build Database

So now it’s time to fire this bad boy up and try it out?! Oh right, I probably need to install a database. Some hunting and I found the hibernate.cfg.xml which contained the database credentials. I noticed there were 3 database flavor’s configured inside of the xml file. Two of the configurations were commented out. I learned that Formbuilder is compatible with serveral database vendors including MySql, Oracle, and Microsoft SQL Server. I chose to use MySql since it lends itself well for development. A few google searches later and I was able to use the mysql command line tool to create a new mysql database (and new mysql user) to match the credentials found in the hibernate config file. I then used the hibernate hbm2ddl tool to automatically generate the datbase schema based on annotations inside the source code. Very cool. Life was good.

Package

Okie-Dokie. Things are compiling. The database is created and is online. The next step is to build a war and then test. I found a helpful shell script (named deploy.sh) that bundled up all the compiled class files created by eclipse along with the WEB-INF directory (including all the jars) into a war file. I copied the war file into tomcat/webapps and pointed my browser to browse to http://localhost:8080/formbuilder and voila? Oh, Dag nabbit!

Runtime Error #1

From the stack trace, it looked like the servlet was not able to connect to the database. After some more hunting I found that there were several application properties files. One property file was used to build a war to run on a local development environment. Another properties file was used to build a war to run on a production server. The production properties file was “active” so I switched back to the local properties file. Ok, run deploy.sh again. Then copy the war over to tomcat/webapps again. Browse to the formbuilder page, and …. DOH!

Runtime Error #2

I was seeing the same database connection error. After some troubleshooting, this time it turned out to be that some of the values inside the properties files were overwriting some of the
values inside hibernate.cfg.xml. I could see that this was intentional. This way, the deploy.sh script could easily switch back and forth between building war for development vs a war for production. Run deploy.sh again, then copy the war over (…again!).

Runtime Errors #3-15

I was now able to connect to the database. But the next errors were several runtime ClassNotFound errors. Even though I had copied all jars over from WEB-INF/libs, it turned out that I needed to find and download a few extra from various web sites. It turned out that I was using a different version of Tomcat than other developers. So I ran the bash script to create the war (…again!). Then, I manually copied and pasted the war into tomcat/webapps (…again!).

Run!

Finally! Formbuilder was running inside tomcat. I was able to create and save a new survey successfully. All in all, it took probably close to 20 hrs to get to the point where I could start being productive and writing code.

Maven

Now, I’d like to describe the steps it took me to get up and running after I “mavenized” the formbuilder project. You ready for this?

c:\> mvn install

That’s it! No lie. That single command (1) resolves and downloads required thirdparty jars (2) compiles java to class files (3) compiles gwt java to javascript (4) generates the database schema (5) runs junit tests (6) creates a war (7) and actually starts an embedded tomcat server which runs the Formbuilder web app.

I’m convinced. So What Next?

None of the steps described above will come to a surprise to anyone who’s ever written a Java web application. It can be frustrating (and costly) to manually repeat the same housekeeping tasks over and over. Maven solves this problem by forcing you to organize all your Java projects exactly the same. It gives you a common set of commands for building Java applications. Think how great it would be to be able to run “mvn install” against any piece of source code that can be found in your organization and have it handle all the tediousness for you?! Automation is a beautiful thing.

Stay tuned for Part II where I’ll cover more on exactly which common development problems that Maven solves and specifics of how to configure Maven to do so.

A GWT application code consists of the generated GWT static resource files (HTML, Javascript, and CSS) on the front end, and server-side service code on the middle and back end. For performance reasons, static resource files are often deployed to the image server. And as of version 1.6, Google Web Toolkit easily allows for seamlessly changing the location of the generated GWT static resource files from the default location in the war deployment.

If you do wish to pursue separating the generated static resources from the backing server code, there are a few steps to take. The primary reason for this change is to avoid potential cross-domain issues that will arise from separating your resources; your imageserver may be running on imageserver.domain.com:80, while your application service might be running on appserver.domain.com:7001.

For this example, we’ll assume your application is called formbuilder.

• Under the war folder that GWT generates, you will find a folder with your application name, formbuilder. Copy that folder to a target folder on the image server, such as /imageserver/RemoteGadgets/formbuilder/gwt/.
• On the GWT entry point JSP page, you’ll make two changes. The first is to create a Javascript variable that stores the gatewayed URL of the servlet. The second is to change the GWT entry point Javascript reference.

• After the changes, your JSP will look something like the screenshot below.

• Finally, in the Java class that implements the EntryPoint interface, you will reference the gatewayed service URL using GWT’s Javascript Native Interface (JNSI).

Bonus tip: As discussed earlier in this post, Google Web Toolkit provides tools to create rich, complex web applications entirely through Java code. GWT was built with cross-browser compatibility in mind, but not all generated HTML is treated equally. For example, adding a widget that generates a block level element into a container that generates an inline element may result in an “Error: Invalid source HTML for this operation.” error in Internet Explorer. When building in GWT, remember that while you are coding in Java, your output is HTML + CSS + Javascript. As with any web application, be sure to test your GWT app in a variety of browsers.

Since the Portal’s inception during Plumtree days, a set of APIs were provided to programmatically interact with the underlying Portal objects.  The API’s name might have changed through the product’s evolution – Enterprise Development Kit 5.x (EDK) during Plumtree days, Interaction Development Kit (IDK) 6.x during the portal’s short lived years with BEA and it seems that Oracle will retain the IDK name as it just prefixed it with “Oracle WebCenter” and just changed the version numbering to 10.x Yet, the concept remains unchanged: a set of interfaces for Portal objects – portlets, documents, projects, folders, communities, security groups, etc.)

With that said, we compiled a high level example of how to programmatically import content items from an external CMS to UCM/Publisher. In a subsequent example we will address how to export content items from Publisher/UCM to HTML and then load these to a target CMS while leveraging its intrinsic API set.

//initially we want to establish a remote session to the portal where soapConnectionURL is web service end point
IRemoteSession remoteSession = RemoteSessionFactory.getExplicitLoginContext(
new URL(“http://YourPortalserver/ptapi/services/QueryInterfaceAPI”), yourUserName, yourPassword);
IContentFactory contentFactory = remoteSession.getContentFactory();

// get the required publisher object managers from IContentFactory
folderManager = contentFactory.getFolderManager();
contentItemManager = contentFactory.getContentItemManager();
propertyManager = contentFactory.getPropertyManager();
presentationTemplateManager = contentFactory.getPresentationTemplateManager();
dataEntryEntryTemplateManager = contentFactory.getDataEntryTemplateManager();

// create a text property to store HTML content
ITextBlockProperty textBlockProperty = propertyManager.createTextBlockProperty(“htmlTextProperty”, “htmlText”);
IFolder folder = folderManager.getFolderByPath(targetFolderPath”);

// create a presentation template that would use the html property
String templateText = “htmlTextProperty=<pcs:value expr=\”htmlTextProperty\”>unset</pcs:value>\n”;
IPresentationTemplate presentationTemplate = presentationTemplateManager.createPresentationTemplate(folder, “Presentation Template”, templateText);
presentationTemplate.store();

// create a data entry template and attach the newly created presentation template
IDataEntryTemplate dataEntryTemplate = dataEntryTemplateManager.createDataEntryTemplate(folder, “Data Entry Template”);
dataEntryTemplate.addProperty(textBlockProperty);
dataEntryTemplate.attachPresentationTemplate(presentationTemplate);
dataEntryTemplate.store();

//create a content item and set the rich/block text property with the HTML from the source CMS
IContentItem contentItem = contentItemManager.createContentItem(folder, “contentItemNameGoesHere”, dataEntryTemplate);
textBlockProperty.setTextBlockProperty(“Hello <strong><font color=\”#000000\”>World</font></strong>”);
contentItemManager.checkInItem(contentItem, “Initial checkin.”);

// publish the content item
contentItemManager.publishContentItem(contentItem);

The above code sample is a draft for how you can upload an HTML string from a separete CMS into Publisher as a content item, some of these method calls should be refactored outside the main method since, more likely, you’ll be looping through the source CMS content repository. Feel free to ping us if you have any questions.

Thanks.
 

Making Thread dumps easier to view

Seeing as how most people’s idea of a good time isn’t sifting through pages upon pages of JVM stack traces, several good people have developed tools that make it easier to view and analyze complex thread dumps. Personally, I like and use the IBM Alphaworks Thread and Monitor Dump Analyzer. Pretty straight forward to use. Just save your thread dump off to a file and open it up in the Analyzer for a pretty view of all your threads that looks like this:

I’ve found this tool useful when I’m trying to look for patterns across many threads in a thread dump. Give it a try, you might like it.

A real world example of how all the rambling I’ve just gone through came in handy

Long ago, in a galaxy far, far away, I was working on a Plumtree consulting project with a large company. These folks had been having problems with their portal deployment. Periodically, the portal would start eating up 100% of the server CPU, and would never release it until the portal application server was bounced. During the time that the server CPU was pegged, users couldn’t access the portal…the customer was not happy. It was one of those big deals where VP-level people at the customer were yelling at VP/Executive level people at Plumtree on a daily basis. So here I am in this high stress situation. I arrive onsite with the customer and, for three days, nothing happens. The application doesn’t freak out, and all is fine. At this point, I’m starting to get cautiously optimistic that maybe we won’t see any problems during my remaining two days onsite, and somebody else will have to deal with the problem later :) But, alas, it wasn’t to be. About halfway through my second to last day with the customer everything went to hell. All of a sudden nobody could access the portal, so we start debugging. Sure enough, first thing we see is that the portal process is eating up 100% of the server CPU. These folks were running on a Unix platform, so I ask them to try:

kill -QUIT <portal pid>

After explaining for 20 minutes that, no, this command won’t kill the process, the server ops folks generated a thread dump and sent it over to review. After about 5 minutes of looking at the portal thread dump, it was pretty obvious that something was amiss with the portal code. The stack trace for EVERY SINGLE THREAD (All 100 or so of them) was exactly the same. They were all stuck running the same native method. Just to be sure this wasn’t some freak anomaly, we generated another thread dump 15 minutes later, and, yep, all the stack traces looked the same. So this tells us that there’s a problem with some native C code being used by the application…a good start. Unfortunately, since the problem was in native code, we couldn’t get a full stack trace from the JVM…it only goes so far as to let you know that it’s making a native call. So we dig one level deeper and trace the portal process. Running a trace/truss on a Unix process spits out all the system level calls that are being made. Wouldn’t you know it, when we looked at the trace output, 99% of the calls being made were:

poll(0)

Now, I wasn’t an expert at interpreting this level of data, but I knew enough that “poll” had something to do with sockets, and that there definitely shouldn’t be so many of those calls. Long story a little bit shorter, after a bunch of conference calls with Plumtree support and engineering, where we shared the data we’d gathered from the thread dumps and process traces, the engineering team found a very low level bug in the software and shipped a fix out to the customer…tragedy narrowly averted.

Conclusion

Did you really make it this far? If so, thanks for sticking with me, and I hope you found something of use in this long, winding post.  If you’re ever stuck trying to make sense of a thread dump, feel free to drop us a line or post a comment here…we’ll do our best to help.

Take care, and remember, when dealing with production troubleshooting, “Do…or do not, there is no try“. (This doesn’t have really have much to do with the post, but I’m kind of tired and pulling at straws for good ways to tie the Star Wars metaphor back in). Anyhow, as always, thanks for reading…see you next time.

Welcome back all, and thanks for joining me for the final installment of a three part series on decompiling Java code and analyzing stack traces. If you’re interested in the back story, you can read about decompiling Java here, and analyzing basic stack traces here.

When we last left off, our hero was frozen in a block of Carbonite and left to his doom. Err…sorry, wrong story, let me start again.

When we last left off, we walked through analyzing a simple stack trace to run down a bug in a standalone Java program. This is all fine and good, but what happens when you’re running a Java application server (like Tomcat or Weblogic) that has hundreds of concurrent threads running tens of different web apps? And you’re just trying to figure out why your particular web application is hung?

Well, if you’re lucky, the developer of your web application was a good boy/girl, and they’re logging stack traces for you in a log file somewhere. If this is the case, then you can open up the log and analyze the trace like we did the last go round.

Often times though, you’re not so lucky, and you have to dig a little deeper to figure out what’s going on. Say, for instance, your web application just starts responding very slowly. You see from access logs that responses are being served back to users, but they’re about 5 times slower than normal…WTF? Or, what if your application server starts pegging the box at 100% CPU…what to do? Or, out of nowhere, your app server starts throwing Java.lang.OutOfMemory exceptions…Gah!!! Sadly, these nebulous problems seem to happen in a production environment more often than most of us would like to admit. And when they do occur, it’s usually a high stress situation because there’s probably a production outage and nobody really knows why. So, how do we find a better fix to the problem than the traditional, “Let’s just bounce it and see what happens” response? Why, we use “The Force”, of course. Except in this case, “The Force”, is just a set of debugging tips that I’m getting ready to share with you as follows:

1. Take a deep breath and don’t freak out when a bunch of people start yelling.
2. Understand the severity of the situation. Figure out how much downtime you can tolerate for debugging before things really hit the fan.
3. Set expectations. Let people know when they can expect the issue to be fixed. If you know a bounce will temporarily fix the problem, then set a drop-dead time for debugging and schedule a bounce. Let folks know that if you don’t find a root-cause of the problem, that they should expect to see the issue again.
4. Gather as much information about the problem as you can. What, specifically, are users experiencing? Can you reproduce? What are the symptoms? What log messages do you have? What does the server environment look like? etc.
5. Try to relate this issue to something you’ve seen in the past. Does this look like a problem you saw last week or last month? What did you do to fix it then? Why is it popping up again now?
6. Eliminate possible external causes. Is this actually a network problem in disguise? Is the database acting up? Is some other process on the server eating up CPU? Is the server constantly swapping because it doesn’t have enough RAM to handle all its business?
7. Eliminate environment changes as possible causes. Has the server environment changed recently? Could this be causing your problem?
8. Look in earnest at your application server process. Are we bumping up against the Java max. heap size? Are we seeing lots of garbage collection? Is our thread pool exhausted? Is our JDBC pool maxed out?
9. Take a look inside of the application server JVM. Generate a thread dump to get a snapshot of how your application server is behaving. Analyze the stack traces in the thread dump, and look for points of interest, like…race conditions, stuck threads, deadlock, etc.
10. Open a ticket with the software vendor (If applicable)
11. Pray the issue fixes itself and doesn’t come back
12. Find a new job so you don’t have to deal with these problems anymore

Read on for more fascinating details about The Stack Trace…

At the risk of getting back on topic, the rest of this post will focus on point 9: Take a look inside of the application server JVM. Did you know that you can generate a snapshot of all the JVM’s threads? Doing so, in practice, creates a big stack trace that tells you what your application server is doing at a point in time. For instance, here’s part of a thread dump/stack trace from a Weblogic Server running on my laptop:

Looks a lot more complicated than that simple stack trace we looked at last time, huh?

Before we start trying to make sense out of all this information, let’s first step back and talk about how we generate a thread dump in the first place. Basically you have a few options. If you’re running on a Unix platform, you can send SIGQUIT to the JVM and it will push a thread dump to the standard error stream. You send SIGQUIT to the process as follows:

kill -QUIT <pid>

or

kill -3 <pid>

The above commands are identical, and don’t worry…even though we’re using the “kill” command, they won’t actually shutdown/kill your process.

What’s that you say? You’re running your app server on Windows and this Unix example is totally useless? Let me digress for a minute and say…if you’re going to be running on Windows, you might as well be running IIS/.NET, it’s just a better fit. But I assume it will take you a while to change your OS and/or development platform, so while we’re waiting, here’s how you can generate a thread dump for your app server on Windows.

1) Start your Application Server from a DOS prompt

2) Type <Ctrl><Break> in DOS window running your application to send a thread dump to the DOS terminal

Additionally, if you happen to be running Weblogic 9+, you can generate a stack trace through the Admin console by going to Servers-><Your Server>->Monitoring->Threads->Dump Thread Stats

Great, so now we have about 2500 lines of output…how do we do something useful with all this data? Let’s start by discussing what we’re looking at from a high-level. As I mentioned earlier, our thread dump is a snapshot of the state of the JVM at a point in time. Since we know our application server is a multi-threaded application, and that each thread has its own call stack, the thread dump is actually just “n” stack traces, where “n” is equal to the number of threads running inside of your application server. So let’s first look at each stack in isolation. For instance, you might have one thread that looks like:

The first line just gives us some information about the thread itself. We see that this thread looks like it’s an execution thread that services user requests. Furthermore, it’s the sixth member of a thread pool that’s used to handle all incoming requests.

Now, following the same principles discussed in my last post, let’s look at the trace from the bottom up:

Reading bottom to top, we see that the thread was spawned by the application server code, and not through a main method, and that it’s sitting in Object Wait state. We can interpret this to mean that the thread is sitting idle, just waiting to service requests. In short, not much of interest is going on here. So let’s look at another example like this:

By the name, we can assume that this thread is being used internally by the application server to monitor garbage collection. Once you start looking at a few application server thread dumps, you’ll notice that there are a bunch of threads like this one. That is, the app server spawns a bunch of threads to do internal housekeeping. Do yourself a favor, just ignore these stacks. For our purposes, they’re just noise. So either skim over them, or delete them altogether from your thread dump, they’re only taking up space and causing confusion.

OK, so we’ve looked over all the individual stack traces that are actually serving user requests and don’t have anything that’s particularly compelling…now what? Now we look for trends. Did you happen to notice that the stack traces for all your threads look the same? Maybe they’re all waiting for a JDBC call to be returned…this implies that perhaps we have a problem at the database level. Let’s go look at that database again. Are all the threads in your execution pool busy without any waiting to service requests? If so, maybe you’re over-loading your application server and need to add more threads to the execution pool. Still nothing interesting? All the threads look like they’re behaving normally? Well, let’s complicate things a bit more by taking another thread dump. By taking a second (or third or fourth) thread dump, we now have a chronology of how the JVM is behaving at distinct points in time. So compare and contrast what specific threads are doing across each of your thread dumps. If, for instance, you see that all the threads show the same stack trace in all of your thread dumps, you can logically conclude that the JVM/application server isn’t servicing requests the way it should be, and you have “stuck threads”. So what’s causing these stuck threads? Two likely candidates:

1. Something external to the JVM is acting up. Is some other process on the server eating up all the CPU and starving your application server process? Is the server out of RAM?
2. Your web application is dead-locked, or in a race condition. This is the result of poor programming in multi-threaded applications, and is often very difficult to run down. Basically, thread A is holding a resource that thread B is waiting for, and thread B is holding a resource that thread A is waiting for. The two threads will never move forward in their execution because they’re in a stalemate over who will give up their resources first. This is probably the worst explanation ever of dead-lock, so if you’re interested in more details, Google “Dining Philosphers“.

For the sake of argument, let’s assume that there’s nothing wrong with the server that’s running your process, so you think you might have a deadlock. How can you be sure? Conveniently, stack traces in JDK 1.5+ actually give you a little deadlock summary at the bottom like so:

If you see a deadlock message in the stack trace, you can be about 100% sure there’s something funky going on with the code. At this point, you need to either dive deep into the code and figure out where the threading logic is flawed, or send all the information you’ve collected over to someone who can.

I know I told you this was the final installment at the start of this post…but it got longer than expected.  So…join us next week for  some cool stack trace tools and a real world example.  Until then….may the force be with you.

The HTML rewrites cause the browser to make the HTTP request “behind the scenes”, and when a response comes back, the portal refreshes the content inside the portlet <div> tag.

But there are some things to know about this AJAX stuff, so here are a couple of refresher points about AJAX:

1) The response to an AJAX request is basically just a text string to a browser, and it’s up to your JavaScript to interpret it.  Often you do something like:

document.getElementById("responseDivTag").innerHTML = response.getResponse();

… to refresh content.  But note that this doesn’t tell the browser to “process” the response – specifically, JavaScript that comes back in the response won’t run, because all we’re doing is setting the HTML to a string that comes back from the server.  In order to run JavaScript in the response, you should look into the JavaScript “eval()” method, which will take a string returned from the server and run it as JavaScript.  Just make sure you don’t include the <script> tags in your response if you really are returning JavaScript and are parsing it as such.

2) The response does not have to actually be HTML!  It’s just a string to the browser, and you can do anything with it.  The most common use (which all of our products use) is to return JSON, or “JavaScript Object Notation”, which can then be treated as objects that your script can handle however you want.  Let’s say you just want to know if there was a success or failure: you could literally just return a “0″ or “1″ in your response and write something like:

if (response.getResponseText.equals("1"))


   alert("success!");

else

   alert("fail");

Obviously, this just barely scratches the surface on AJAX, and you can rest assured that you haven’t heard the last of it.  AJAX is the cornerstone of pretty much all future Oracle portal technologies, and if you’re a web developer who’s not all that familiar with it, trust me:  you will be soon.

Without further ado, I give you…Stack Wars II: The Stacktrace Strikes Back (I’m completely aware that I’m abusing the metaphor here, but isn’t that really what blogging is all about?).

Standard disclaimer: This post is intended for a technical audience with a focus on production support. Also, everything here is Java focused, but you can certainly apply some of the concepts in a .NET environment as well…you’ll just have to create your own screencaps to replace the examples I’ve included below.

So, what is a stack trace, and why should you care? Well, one question at a time please.

What is a stack trace?

Wikipedia says a Stack Trace is, “A report of the active stack frames instantiated by the execution of a program.” Now, I vaguely understand the Wikipedia definition, but I have also have a computer science degree from a second tier state university, so let me try to translate for those of you who were smart enough to get degrees in something besides CompSci: a Stacktrace is a snapshot of a program’s behavior at a point in time. In the Java world, a stack trace will tell you which method was being executed at the time the trace was generated, along with its complete call stack, and usually line numbers as well. Take a look at the following simple stack trace below as an example:

Why should you care?

Good question. I’d venture a guess and say that about 99.99% of the world doesn’t need to know nor care about stack traces. But here you are reading this post none-the-less, so here’s why they’re important:

1) Good programmers almost always print stack traces out in log files when an error in a program occurs. This gives us a useful tool to track down bugs. Whether you’re just reporting information to a support team somewhere, or getting a little sassy and trying to fix a problem yourself, the stack trace is like a map for finding treasure buried deep in code. Except that instead of finding actual treasure, you’re just finding a logic error. And instead of getting rich, you just get to complain about a problem, and maybe fix it.

2) You can tell the JVM to generate a stack trace for a running process. Doing so allows us to take a snapshot of the JVM at an arbitrary point in time, and see what all its threads are up to. This is useful when trying to figure out why a process (Tomcat for instance) is zombied (i.e. it’s running, but not responding to requests), or when you’re trying to fix deadlock issues, which are particularly difficult to run down.

Hit the jump for learning more about reading and interpreting stack traces!

Reading Stack Traces

Stack Traces range from very easy to very difficult to read, depending on what you’re looking at. For instance, a stack trace of a single-threaded, stand-alone Java program is almost trivial to interpret, whereas trying to find dead-locks in a Java application server that has 100+ concurrent threads can be pretty cumbersome. Let’s use our last example again to walk through interpreting a “simple” stack trace:

When reading traces, I recommend first taking a look at the top line of the stack trace to see what error is occurring. In this case, it looks like we’re getting a NullPointerException in the “doEvenMoreStuff” method on line 22 of Broken.java:

After getting a rough idea of the error being thrown, it’s worthwhile to read the stack from the bottom up; least specific to most specific. In our example, for instance, we see from the last line of the trace that the first call in the stack originated in the main method of Broken.java. Since we all know that “main” is the first thing invoked when running a program from the command-line, we can feel confident that stepping through Broken.java beginning on line 36 in the main method will help us fully understand what’s gone wrong in the code. So we look at the next line up in the trace, where we see a call from main to the method, “doStuff”. “doStuff” calls “doMoreStuff” on line 16 of Broken.java, which in turn calls the “doEvenMoreStuff” method. Finally we get back to the NullPointerException, which occurs on line 22:

At this point, you have a few options to continue your troubleshooting.

1) Assuming you’re trying to fix a problem with some commercially supported software, you can open a trouble ticket with the vendor. Provide symptoms that you’ve noticed and the stack trace you’ve captured. The support team can then pass off the trace to an engineer for a looksy, and maybe you end up with a patch somewhere down the road.

2) Google the hell out of the exception. The internet is a strange and beautiful place, and you’d be amazed how many times you can find answers out there. For instance, let’s say that instead of seeing, “NullPointerException” in the first line of your stack trace, you instead see something like, “AnalyticsConfigurationException”. You start Googling around for things like, “ALUI AnalyticsConfigurationException”, “AnalyticsConfigurationException”, etc. and see what you see. Maybe you find nothing, but maybe you find some blog post that tells you this error means that you have a typo in line 18 of some configuration file.

Just to repeat…Google the hell out of the exception. Low cost, high potential reward in doing so. You know those smart tech guys who sit in the corner and don’t really get enough sun…I guarantee you that one of their best tricks is knowing how to use the internet to find information.

3) You want to be a maverick like John McCain, and decide to decompile the code to fix the problem yourself. Well God bless you, and God bless the United States of America…take a look here for more details.

4) You happen to have the source code for this application available, and decide to match up the stack trace to the code. This is pretty easy to do when you have the original source, as the line numbers in your stack trace will match up to the code you have. So just read the line numbers in the stack from bottom up, and step through the code accordingly; like this:

That’s it for now. I hope this rambling has helped you get a handle on why stack traces are useful, and how to start making sense of them. Be sure to stay tuned for next week’s installment, “Return of the Stack Trace”, where we walk through interpreting some more involved traces…Same bat time, same bat channel. Until then, may the force be with you (Mixing metaphors is also totally acceptable in the wonderful world of blogging).

Basically, open up your portal in FireFox (this doesn’t seem to work in IE), and hit CTRL-SHIFT-3.  That is, hold down CTRL and SHIFT and hit 1, 2, or 3.  You’ll get a little JS window at the bottom of your portal page showing some JS debug information that you can use to see what’s going on under the covers and evaluate ALI portal JS objects in real time.

The “1, 2, or 3″ that I mentioned above refers to the log threshold you’d like to look at (use “0″ to turn it off); three is the highest.

Fun?  If you’re a curious techie who gets a thrill out of this sort of stuff.  Useful?  I’ve never used it (but then again, I only meet the first two of these criteria.  OK, fine, who am I kidding, I meet all three – but it’s still never been useful for me).

Find any practical uses for this hidden feature?  Hit me up in the comments.