Recently in Cool Tools Category

Hi, my name is Brian.  I like sunsets, long walks on the beach, puppies, and de-compiling Java code.  If you have similar interest, please read on.  On the other hand, if you're not a Java programmer, this post might not be for you.

So of all the cool tools out there, JAD is, without a doubt, my absolute favorite...and it's not even close.  One of the beautiful things about Java is that the byte-code interpreted by the JVM can be reverse-engineered into source code.  JAD, or JavA Decompiler, is a tool for doing just as the name implies: decompiling byte code into source.  But so what?  Why would this possibly be useful?  Well, here's a list of just a few things that I've done by de-compiling code:

• Found software bugs
• Saved about 5 weeks of hassle with BEA support by pointing them to the line of source code that is causing a bug
• Re-compiled bug fixes back into code as a band-aid solution to a problem (Note that this probably voids any/all support agreements that you've ever had with any one)
• Understood how/why a piece of software works like it does (I.e. "Why is Analytics deleting all a bunch of historic information when I run a specific job...there's no document on it")
• Found cool/un-documented software features (Again...no support, but playing around with stuff is fun)
• This one is my favorite: Long ago, for "Bring your kids to work day", I was on the hook for talking to a bunch of 12 year olds about programming.  So I de-compiled a Ms. Pacman applet, found the code that does collision detection, modified it to not work, and re-compiled the new code back in.  The net result was a Ms. Pacman who could run through ghosts, and a bunch of kids learning how "If" statements work.
• Also, this guy I know might have one time de-compiled a VI plugin for Eclipse to take out the license key check.  Don't worry though, I quickly told him how he was a bad boy, and he's been on the straight and narrow ever since.

OK, sorry for the rambling there, but just wanted to give you a flavor of the stuff you can do by de-compiling code.  More generally though, if you're a programmer at heart, it's always a good idea to look at source code, just to understand how things work.  Of course there are fringe benefits too:

• Fellow portal geeks marvel over how "smart" you are because you "de-compiled" the portal and found some problem.
• You'll get to complain about what idiots the people who wrote the code are
• Men will want to be you
• Women will want to be with you

So, let's look at a quick example, shall we?  JAD is a text-based utility, so get ready to fire-up your DOS prompts and/or Shell windows.  I'm writing this post on a PC, so my examples will be through DOS, but UNIX folks, just make the normal DOS->*NIX changes, and you'll be fine.

Let's assume that I've been trying to get a piece of proprietary software to run for a while now, and every time I start it up, I get a stacktrace like the following:   

Now, you could argue that I should know better than to purchase a piece of software named, "Broken", but I'd argue right back that you're getting in the way of my example. 

Want to see JAD in action? Join me after the link for all the details!

Granted, the vast majority of the Cool Tools we've listed here aren't exclusive to the AquaLogic line, and frankly, while I've used this one before to diagnose problems with the AD and LDAP Identity Services, I probably wouldn't have even included it until recently, when the new ALI Directory Service made its debut.

The tool is Softerra's LDAP Browser, and it comes in both free and paid versions.  Using it (and the tip for the proper user name to authenticate with), you can peruse the account information exposed by the new service.  Simply start it up, point it at the right port with the right credentials, and browse away like you would with Windows Explorer or Regedit:

The two interesting things I've noticed off the bat is that the base DN (Distinguished Name) for the user and group OUs (Organisational Units) is dc=bea,dc=com, and that different auth sources have different OUs under that (while native Plumtree accounts exist at the top level.

We've re-posted some of Ray Gao Dev2Dev entries more than once, but this is one I'd have linked to even if Dev2Dev wasn't going away because, well, it's a Cool Tool.

WS Investigator solves the age-old problem of easily identifying all of your Web Services, Portlets, Pages, and Communities deployed to your ALI portal.  Originally written for 6.0, I haven't tested it against 6.5 but have no doubt that it'd work (our product LockDown uses similar APIs and works across all 6.x versions), WS Investigator is a .NET tool (a slightly older Java version is also available) that works as advertised, providing quick views into some of the most important portal administrative objects you've got:

The project can be found (at least for the time being) here on Dev2Dev, or feel free to download the Java or .NET versions after the jump.

FWIW, I suggested we change "Cool Tools" to "Kewl Tewls" to capture the strong ALI tween demographic, but I got over-ruled.  I guess the kids these days are too busy playing with their Twitters and their Facebooks to care about our old friend ALI.  Maybe if BEA would just rename "Aqualogic User Interaction" to something a little more Web 2.0, like Twinte or Youzoom, then the kids would come running.  But instead of just holding my breath for the next month waiting for Oracle to re-brand Plumtree yet again, I thought I'd share a decidely un-hip, un-Web 2.0, but none-the-less really useful, tool with you: Log Parser. 

So what is this mysterious kewl tewl...a new micro-blogging social tag mash-up platform?  Nay, it's actually just a....log parser :(  But it's awesome, and  here's why...

Log Parser let's you use SQL syntax to quickly get meaningful information out of Gigabytes of log data.  The tool is designed to parse arbitrary log formats, meaning that you can run it against things like: IIS access and error logs, Apache access and error logs, Windows event logs, Tomcat logs, etc.  It's even designed to be extensible, so, if you're motivated, you can write custom parsers for just about anything.  If, like me, you've ever had the misfortune of trying to manually make sense out of 50,000 lines of web server access logs, you can probably appreciate the utility of a tool like Log Parser.

Let's take a minute to consider a concrete example.  A few weeks ago, I was working with an ALI client who had a total of 300 portal users, of which, maybe 150 used the portal at all.  Oddly, Analytics was reporting about 10,000 daily hits to their portal.  This seemed abnormally high, which lead me to believe either a) somebody is scanning/attacking the portal, or b) some monitoring tool is hitting the portal too often.  So we decided to take a look at the portal IIS access logs to get some more granular information, and used Log Parser to answer the following questions:

• What IPs are generating the most traffic to the portal? (If the bulk of the traffic is coming from one IP, we probably have an over-active monitoring tool)
• What are the most popular URLs in the portal? (Monitoring would likely only hit one URL, but somebody doing a scan would hit many)
• Are we seeing a bunch of 404s returned from the Web Server (A scanner would likely generate hits to a bunch of pages that don't exist)

 Here's how we did it:

1. Opened up a handy DOS prompt to the IIS log file directory at: c:\windows\system32\Logfiles\W3SVC1
2. Ran Log Parser from the command line as follows:

//Traffic by IP

logparser "SELECT c-ip, Count(*) as Hits FROM ex*.log GROUP BY c-ip"

c-ip            Hits
--------------- -------------------
192.168.0.1     8,920
192.168.0.8     54
192.168.1.93    37
...

 

//Popular URLs

logparser "SELECT cs-uri-stem, Count(*) as Hits FROM ex*.log GROUP BY cs-uri-stem"

cs-uri-stem           Hits
--------------------- -------------------
/portal/server.pt     9,312
/portal/server.pt?... 137
/portal/server.pt?... 83
...

 

//404s

logparser "SELECT count(*) as Errors FROM ex*.log WHERE sc-status=404"

Errors
------
32

 

So, in about 2 minutes, Log Parser told us that most of the client's portal traffic was coming from one IP address, and that it was all going to /portal/server.pt, not some unknown URL.  A quick nslookup of 192.168.0.1 confirmed that the traffic was coming from an over-active health monitor, all our questions were answered, and the world was at peace.  It would have probably taken an hour or more to gather the same information by hand.  So Log Parser saved me some headache, and our client some money...not bad for an old-fashioned command-line tool.  Of course, Log Parser didn't auto-blog this post for me...but maybe in the next release, when it gets a slick AJAX interface and re-named LogPress.

I've done a couple of "back-fills" from production environments to development environments recently, and let me tell you, they're a real pain.  Inevitably, everything ends up getting messed up when you do a database port from one environment to the next - Search, Automation, Knowledge Directory, Remote Servers, and so on.

One thing that I haven't had to worry much about due to Fabien Sanglier's recent release of search-and-replace utility called the PTURLReplace, described in detail in his article, "ALUI Administration Tool for Environment Refresh: String Replacing for URLS".

The tool works as advertised, and has been a great timesaver for me.  It's easy to use, with a simple command-line batch file to prompt for some simple information before going to work:

Check out Fabien's article for the download link and a more detailed description of how the tool works!

UPDATE 8/20/08: Changed links in this post to point to Fabien's new blog URL.

UPDATE 8/23/2008: Fabien has updated his blog post to point to his Google Code project for this tool.  You can download it here. 

I ran into a problem recently where my Automation Server wasn't starting up. Checking the logs, it showed that something was already running on port 7777. Sure, it's easy enough to change the automation server port (hint: it's in serverconfig.xml), but Automation Server used to run on this server, so I needed to find out what other application was using the port.

While the embedded app servers in most ALUI products use pretty unique numbers, apparently Plumtree's random number generator has gotten much better since the default automation server port was originally set at port 7777.

Occasionally port conflicts like this will bite you, and while a "netstat -a" will give you a bunch of information about ports and what they're doing, they don't tell you much about the applications holding them.

Enter NirSoft's CurrPorts. It's a great little freeware application that gives you much more information about the ports that are in use, how long they've been that way, and the processes and users holding them:

This certainly isn't a tool you'll use on a daily basis, but the next time you have a port conflict, you'll be glad it's in your arsenal.

Visibility into network traffic can be a huge help in diagnosing issues. We've already covered TcpTrace, which allows you to proxy traffic through it and view the traffic as it flows.

TcpTrace is great for most applications (such as monitoring traffic between ALUI portal tiers), but it can be a little limiting when you're trying the monitor traffic between the browser and the portal. The reason for this is that you point your browser at TcpTrace and have it make the request to the portal on your behalf. This causes the browser to "see" different URLs for different content: from the browser's perspective, portal pages are coming from http://localhost:9999/, but images and js still come from http://portal.function1.com/imageserver/. And the "host" header sent to the portal is "localhost" rather than the name of the actual server, which can result in JS errors or other unexpected server-side behavior.

A better way to monitor traffic from the browser to the portal is ProxyTrace. ProxyTrace is a virtual proxy server that routes traffic to the target site transparently, so the browser's URL will go to http://portal.function1.com/portal/server.pt, but you'll still be able to see the traffic because it's routed through the proxy.

Setting up ProxyTrace is simple: you just give it a port to listen on. You don't even need to set a destination IP and port, because that information comes from the browser. Once it's running, you configure your browser to use this port for your proxy server (in IE, it's Tools: Internet Options: Connections: LAN Settings):

Next time you make any HTTP requests, you'll see all the traffic coming from the browser:

Automatically checking links in the portal is hard. The problem is, the same page could have multiple URLs that resolve to the same item. Here, both of the following links open a community with an ID of 200:

• http://server/portal/server.pt?open=512&objID=200&PageID=0&cached=true&mode=2&userID=226
• http://server/portal/server.pt?in_hi_userid=226&spaceID=8&space=CommunityPage& parentid=&control=SetCommunity&PageID=0&CommunityID=200&parentname=

This makes automated crawling a pain, even for Google to some extent.

Today's Cool Tool won't really overcome this issue (in fact, nothing short of a "portal-aware" link scanner would, or "Friendly URLs" - hint, hint). But it's free, and it does a decent job generating useful reports in complex portal environments:

Download it here!

Do you look at a lot of log files during the course of a day? Are you sick of closing and reopening files in NotePad and EditPlus to see the changes reflected in those files? Do you wish there was an easy way to see things in log files that you care about? If so, then this "Cool Tool" is for you...

BareTail allows you to have multiple text files open at the same time and it auto refreshes the text files as new entries are added to it. You can even specify custom highlighting so that important information from the log files is immediately visible to you. In the screen shot below, I was using BareTail on an IIS log and wanted to highlight requests to two particluar files:




BareTail is a product by Bare Metal Software. Those of you with a UNIX background might recognize the back part of the product name; this product is like a "tail -f" on steroids (Sorry, MLB). Here is a list of features as listed by Bare Metal Software:

Real-time file viewing

• Optimized real-time viewing engine
• View files of any size (> 2GB)
• Scroll to any point in the whole file instantly
• View files over a network
• Configurable line wrapping
• Configurable TAB expansion
• Configurable font, including spacing and offset to maximise use of screen space

Follow tail mode

• View the end of a growing file in real time, no matter how fast the file grows
• Like "tail -f" on Unix systems, but with many more features

Tail multiple files

• Simultaneously monitor multiple files for changes using tabs
• Visual indication on each tab of file status and changes
• Tabs may be positioned on any side of the window and oriented horizontally or vertically

Configurable highlighting

• Lines containing particular strings can be highlighted to help you notice important text
• Highlight colours are fully customisable

Many file formats

• Windows / DOS text files (lines end in CR/LF pairs)
• Unix text files (lines end in LF)
• Microsoft IIS logfiles (and other files terminated with a string of nulls)

BareTail is a free product available from Bare Metal Software. They have a pay version if you want to get rid of the annoying splash screen in the beginning, but the free version has all the same features as the pay version. They also have a BareTail pro which has even more features, but I haven't used that version yet.

To download the product or learn more, go to www.baremetalsoft.com

Wow. This was going to be a post about the exceptional ALUI portal add-on called LiquidSkin, developed by BID Services. It's not a diagnostic tool, but still fits into the theme of the "Cool Tools" feature by making administrators' jobs easier by greatly simplifying the creation of rich portal navigation schemes.

For example, this interface was created with little more than a couple .XML files and some images/style sheets:

But, a new entry to the Bea blogosphere, Fabien Sanglier, has written two fantastic articles about LiquidSkin that pretty much say it all:

• Overview of BEA ALUI LiquidSkin: Part 1
• Overview of BEA ALUI LiquidSkin: Part 2

Welcome, Fabien!