Connecting to Glassbox using JMX

The Glassbox UI is a wonderful way for anyone to quickly identify major bottlenecks. It quickly lists out most of the potential problems and even common solutions if any call exceeds the SLA.

But if you want to delve into even more details, you would need to use the excellent set of MBeans provided by Glassbox. If you have a default JMX configuration for your Glassbox installation, the following is the connection string:

service:jmx:rmi:///jndi/rmi://localhost:7232/GlassboxTroubleshooter

To connect using JConsole, provide this without username and password.

Navigate to Tree->Glassbox->stats in the left pane and click on your application for operations monitored by Glassbox - you may have to perform some activity on your application for the monitors to log anything of course.

Additional Steps if encountering frequent “java.lang.OutOfMemoryError: PermGen space” exception.

Glassbox does take up some additional memory slowing the startup. Also, it tends to place additional load in the PermGen (Permanent Generation) memory space - this would result in the Server crashing out with "java.lang.OutOfMemoryError: PermGen space" exception (see http://blogs.sun.com/fkieviet/entry/classloader_leaks_the_dreaded_java for details).

The fix is a simple matter of increasing your max PermGen size (default is 64MB in Sun Servers). Use the following options when starting the JVM:
-XX:+CMSClassUnloadingEnabled
-XX:+CMSPermGenSweepingEnabled
-XX:MaxPermSize=128m

This actually works for any genuine PermGen error. But note that in most cases, PermGen running out of space is because of inefficient or incorrect cleaning up for Garbage collection - that is, some where in your code, you are unnecessarily keeping references even after use.

Steps to configure Glassbox for Sun App Server

Prerequisites
Ensure that the steps are being carried out by the same user who created / installed the Sun App Server to avoid read / access issues.

Requirements:
Sun App Server 8.1
glassbox.war (latest file is available from Glassbox website)
Java 1.4 or above

Configuration steps:
1) Create a new base directory for Glassbox (eg: D:\Sun\glassbox). This should contain the configuration details & library files. I have gathered the neccessary files into glassbox_config.zip to make it simple - simply unzip the contents into this directory. See below on how to do this the hard way...
Additional key properties like elapsed time threshold, minimum slow threshold & thread-monitor interval can be specified in glassbox.properties in this directory.

2) Go to Sun One Admin Console and select Configuration->JVM Settings. Select the Profiler tab.

Name the profiler, Glassbox, and enable profiler adding the following details :
Classpath: D:\Sun\glassbox\lib\aspectjweaver.jar${path.separator}D:\Sun\glassbox\lib\glassboxMonitor.jar
JVM Option1: -Dglassbox.install.dir=D:\Sun\glassbox
JVM Option2: -javaagent:D:\Sun\glassbox\lib\aspectjweaver.jar
In the above settings, use whatever location chosen as base directory for Glassbox in step 1, if different from D:\Sun\glassbox.

By doing it this way, you can easily decide when you need Glassbox by coming here and turning on or off the Enable Profiler option.

3) Deploy glassbox.war on the server. Also modify server.policy to have the following additional entry:
// Following grant block is added for glassbox
grant codeBase "file:${com.sun.aas.instanceRoot}/applications/j2ee-modules/glassbox/-" {
permission java.lang.RuntimePermission "setContextClassLoader";
permission java.lang.RuntimePermission "getClassLoader";
permission java.lang.RuntimePermission "createClassLoader";
permission java.security.AllPermission;
};
The Glassbox Web Client would be available at <server-app-url>/glassbox after restart.

4) Restart the Sun App server. If the agent was installed successfully, the following should be present in the server.log:
“Glassbox Agent has started successfully.”


Obtaining Configuration files from latest glassbox.war
Extract all the files from glassbox.war located in the install/glassbox14 directory and copy the files to the target configuration directory.
Move the 2 copied jars into a lib sub-directory.

The final structure should be as below:
\--------
glassbox.properties
readme.txt
runtime.properties
\lib-----
aspectjweaver.jar
glassboxMonitor.jar


Additional steps for using Glassbox with JVM 1.4
If the Sun App Server uses a Sun / IBM 1.4 JVM, the createJavaAdapter has to be updated and added to the JVM start options –Xbootclasspath along with the unmodified file, java14Adapter.jar.

Extract aspectj14Adapter.jar and createJavaAdapter.jar from the glassbox.war located in install/glassbox14
Run java -jar createJavaAdapter.jar using the Java version you will use to run the server.
Copy the aspectj14Adapter.jar and updated createJavaAdapter.jar to the Glassfish base configuration directory.
Add the following additional JVM options in the Profiler section of Sun App Server:
-Xbootclasspath/p:\java14Adapter.jar
-Xbootclasspath/a:\createJavaAdapter.jar
Restart the Sun App server.


Additional steps for allowing Sun App Servers to give remote RMI access in Unix
If there are errors when accessing JMX on a different server than the one on which Glassbox is installed, ensure that the target remote server has the given permissions by adding the following in the server.policy file:
grant {
permission java.net.SocketPermission "*:1024-65535", "connect,accept";
};