This document provides an overview of tools available in the Java Development Kit (JDK) that allow for powerful introspection and manipulation of the Java Virtual Machine (JVM) and running applications. It discusses the java.lang.instrument API for injecting Java agents, the Java Debugging Interface (JDI) for debugging, the JVM Tool Interface (JVMTI) for heap and frame introspection, and examples of using these tools to build interactive debuggers, inject code at runtime, and test concurrency. Code samples and links to further resources are also provided.

1. JDK Power Tools
an overview of fun and useful tools
the JDK already provides you with
Tobias Lindaaker
Software Developer @ Neo Technology
twitter:! @thobe / @neo4j / #neo4j
email:! tobias@neotechnology.com
web:! http://neo4j.org/
web:! http://thobe.org/

2. The Tools
๏java.lang.instrument
๏Java agents
๏Java Debugging Interface
๏JVMTI heap introspection
๏JVMTI frame introspection
2

3. java.lang.instrument
3

4. java.lang.instrument.Instrumentation
๏The “missing” sizeof() operation:
•long getObjectSize( Object )
๏Inspecting loaded classes:
•Class[] getAllLoadedClasses()
•Class[] getInitiatedClasses( ClassLoader )
๏Transforming classes:
•redefineClasses( ClassDefinition(Class, byte[])... )
•addTransformer( ClassFileTransformer )
‣byte[] transform( ClassLoader, name, Class, byte[] )
•retransformClasses(Class...)
•setNativeMethodPrefix( ClassFileTransformer, String ) 4

5. public static void premain(
String agentArgs, Instrumentation inst)
5
๏java ... -javaagent:<jarfile>[=options]
๏Instrumentation parameter is optional in the signature
๏META-INF/MANIFEST.MF:
•Premain-Class [required; qualified classname of premain class]
•Boot-Class-Path [optional; class path]
•Can-Redefine-Classes [optional; true/false - request capability]
•Can-Retransform-Classes [optional; true/false - request capability]
•Can-Set-Native-Method-Prefix [optional; true/false - request capability]

6. Java agents
6

7. public static void agentmain(
String agentArgs, Instrumentation inst)
7
๏Executed when the agent attaches to an already running JVM
๏Instrumentation parameter is optional in the signature
๏META-INF/MANIFEST.MF:
•Agentmain-Class
+the same optional parameters as for premain():
+Boot-Class-Path
+Can-Redefine-Classes
+Can-Retransform-Classes
+Can-Set-Native-Method-Prefix
๏Requires support for loading agents in the JVM
๏Allows to add code to the JVM after-the-fact

8. The Use Case: add code live
๏Provide statically accessible entry point into your app
๏From here extend the app with the code loaded by the agent
๏Used by the JDK, when attaching with jconsole through PID
8

9. com.sun.tools.attach
๏com.sun.tools.attach.VirtualMachine
•Represents a JVM process
•static methods providing entry points for:
‣attaching to a JVM by process id
‣listing all running JVM processes (like jps)
๏Example (from the docs, simplified from jconsole source code):
// attach to target VM
VirtualMachine vm = VirtualMachine.attach( "1337" );
// get system properties in target VM
Properties props = vm.getSystemProperties();
// construct path to management agent
String agent = props.getProperty( "java.home" )
+ File.separator + "lib"
+ File.separator + "management-agent.jar";
// load agent into target VM, spcifying parameters for the agent
vm.loadAgent( agent, "com.sun.management.jmxremote.port=5000" );
// done - detach
vm.detach(); 9

10. Tricks: Registry-less RMI
๏RMI Registry just provides serialized objects specifying:
•Implemented Remote interfaces
•How to reach the actual object (host+port+id)
๏You can serialize the object yourself and pass it as the string
parameter to an agent
๏Now your agent has a talkback channel to the process that
spawned it
10

11. code.sample();
11
http://github.com/thobe/java-agent

12. Java Debugging Interface
12
Building an interactive
debugger is HARD, but
that isn’t the only thing
you can do with the JDI.
A Scriptable debugger is
much easier to build.

13. SubprocessTestRunner
13
๏Runs a JUnit test case in a sub process
๏Host process connects to the sub process with JDI
๏Define breakpoints for the code under test
๏Useful for triggering certain ordering between threads
•Triggering race conditions
๏Used at Neo Technology for Concurrency Regression Tests
•Prefer refactoring to allow testing concurrency,
but that isn’t always the appropriate short term solution

14. Simple example
@RunWith(SubprocessTestRunner.class)
public class SampleTest {
@Debugger.Using(BreakPoints.class)
@Test public void shouldHandleRaceCondition() {
new Sample().entryPoint();
}
static class BreakPoints extends Debugger {
List<Integer> loopVariables = new ArrayList<Integer>();
{ enable( "handle_someMethod" ); }
@Handler(on=ENTRY,type=Sample.class,method="someMethod")
void handle_someMethod() {
enable( "handle_loopBody" );
}
@Handler(on=ENTRY,type=Sample.class,method="loopBody")
void handle_loopBody(){
loopVariables.add( frame( 1 ).getInt( "i" ) );
}
@Override protected void finish() {
assertEquals( asList( 1, 2, 3, 4 ), loopVariables );
}
}
} 14

15. Simple example
@RunWith(SubprocessTestRunner.class)
public class SampleTest {
@Debugger.Using(BreakPoints.class)
@Test public void shouldHandleRaceCondition() {
new Sample().entryPoint();
}
static class BreakPoints extends Debugger {
List<Integer> loopVariables = new ArrayList<Integer>();
{ enable( "handle_someMethod" ); }
@Handler(on=ENTRY,type=Sample.class,method="someMethod")
void handle_someMethod() {
enable( "handle_loopBody" );
}
@Handler(on=ENTRY,type=Sample.class,method="loopBody")
void handle_loopBody(){
loopVariables.add( frame( 1 ).getInt( "i" ) );
}
@Override protected void finish() {
assertEquals( asList( 1, 2, 3, 4 ), loopVariables );
}
}
} 14
public class Sample {
public void entryPoint() {
for ( int i = 0; i < 5; i++ ) {
loopBody();
someMethod();
}
}
private void someMethod() { /* do nothing */ }
private void loopBody() { /* do nothing */ }
}

16. Simple example
@RunWith(SubprocessTestRunner.class)
public class SampleTest {
@Debugger.Using(BreakPoints.class)
@Test public void shouldHandleRaceCondition() {
new Sample().entryPoint();
}
static class BreakPoints extends Debugger {
List<Integer> loopVariables = new ArrayList<Integer>();
{ enable( "handle_someMethod" ); }
@Handler(on=ENTRY,type=Sample.class,method="someMethod")
void handle_someMethod() {
enable( "handle_loopBody" );
}
@Handler(on=ENTRY,type=Sample.class,method="loopBody")
void handle_loopBody(){
loopVariables.add( frame( 1 ).getInt( "i" ) );
}
@Override protected void finish() {
assertEquals( asList( 1, 2, 3, 4 ), loopVariables );
}
}
} 14
{ enable( "handle_someMethod" ); }

17. Simple example
@RunWith(SubprocessTestRunner.class)
public class SampleTest {
@Debugger.Using(BreakPoints.class)
@Test public void shouldHandleRaceCondition() {
new Sample().entryPoint();
}
static class BreakPoints extends Debugger {
List<Integer> loopVariables = new ArrayList<Integer>();
{ enable( "handle_someMethod" ); }
@Handler(on=ENTRY,type=Sample.class,method="someMethod")
void handle_someMethod() {
enable( "handle_loopBody" );
}
@Handler(on=ENTRY,type=Sample.class,method="loopBody")
void handle_loopBody(){
loopVariables.add( frame( 1 ).getInt( "i" ) );
}
@Override protected void finish() {
assertEquals( asList( 1, 2, 3, 4 ), loopVariables );
}
}
} 14
@Handler(on=ENTRY,type=Sample.class,method="someMethod")
void handle_someMethod() {
enable( "handle_loopBody" );
}

18. Simple example
@RunWith(SubprocessTestRunner.class)
public class SampleTest {
@Debugger.Using(BreakPoints.class)
@Test public void shouldHandleRaceCondition() {
new Sample().entryPoint();
}
static class BreakPoints extends Debugger {
List<Integer> loopVariables = new ArrayList<Integer>();
{ enable( "handle_someMethod" ); }
@Handler(on=ENTRY,type=Sample.class,method="someMethod")
void handle_someMethod() {
enable( "handle_loopBody" );
}
@Handler(on=ENTRY,type=Sample.class,method="loopBody")
void handle_loopBody(){
loopVariables.add( frame( 1 ).getInt( "i" ) );
}
@Override protected void finish() {
assertEquals( asList( 1, 2, 3, 4 ), loopVariables );
}
}
} 14
@Handler(on=ENTRY,type=Sample.class,method="loopBody")
void handle_loopBody(){
loopVariables.add( frame( 1 ).getInt( "i" ) );
}

19. Simple example
@RunWith(SubprocessTestRunner.class)
public class SampleTest {
@Debugger.Using(BreakPoints.class)
@Test public void shouldHandleRaceCondition() {
new Sample().entryPoint();
}
static class BreakPoints extends Debugger {
List<Integer> loopVariables = new ArrayList<Integer>();
{ enable( "handle_someMethod" ); }
@Handler(on=ENTRY,type=Sample.class,method="someMethod")
void handle_someMethod() {
enable( "handle_loopBody" );
}
@Handler(on=ENTRY,type=Sample.class,method="loopBody")
void handle_loopBody(){
loopVariables.add( frame( 1 ).getInt( "i" ) );
}
@Override protected void finish() {
assertEquals( asList( 1, 2, 3, 4 ), loopVariables );
}
}
} 14
@Override protected void finish() {
assertEquals( asList( 1, 2, 3, 4 ), loopVariables );
}

20. Simple example
@RunWith(SubprocessTestRunner.class)
public class SampleTest {
@Debugger.Using(BreakPoints.class)
@Test public void shouldHandleRaceCondition() {
new Sample().entryPoint();
}
static class BreakPoints extends Debugger {
List<Integer> loopVariables = new ArrayList<Integer>();
{ enable( "handle_someMethod" ); }
@Handler(on=ENTRY,type=Sample.class,method="someMethod")
void handle_someMethod() {
enable( "handle_loopBody" );
}
@Handler(on=ENTRY,type=Sample.class,method="loopBody")
void handle_loopBody(){
loopVariables.add( frame( 1 ).getInt( "i" ) );
}
@Override protected void finish() {
assertEquals( asList( 1, 2, 3, 4 ), loopVariables );
}
}
} 14

21. code.samples();
15
https://github.com/thobe/subprocess-testing

22. JVMTI frame introspection
16

23. Frame introspection
๏Native interface (JVMTI)
๏Abilities:
•Get method and instruction offset for each element on stack
•Force early return from method
•Pop frame and re-run method
•Get local variable from call stack
•Get notification when frame is popped (method is exited)
(Forcing a frame to be popped does not trigger the event)
17

24. Dynamic access to local variables
public @Test void sampleOfAccessingLiveCallFrame() {
int anInt = 4; // random number - by dice
String aString = "some string value";
18

25. Dynamic access to local variables
public @Test void sampleOfAccessingLiveCallFrame() {
int anInt = 4; // random number - by dice
String aString = "some string value";
// get the tool
ToolingInterface tools =
ToolingInterface.getToolingInterface();
18

26. Dynamic access to local variables
public @Test void sampleOfAccessingLiveCallFrame() {
int anInt = 4; // random number - by dice
String aString = "some string value";
// get the tool
ToolingInterface tools =
ToolingInterface.getToolingInterface();
// get the frame
CallFrame myFrame = tools.getCallFrame( 0 );
18

27. Dynamic access to local variables
public @Test void sampleOfAccessingLiveCallFrame() {
int anInt = 4; // random number - by dice
String aString = "some string value";
// get the tool
ToolingInterface tools =
ToolingInterface.getToolingInterface();
// get the frame
CallFrame myFrame = tools.getCallFrame( 0 );
// get local
assertEquals( anInt, frame.getLocal( "anInt" ) );
18

28. Dynamic access to local variables
public @Test void sampleOfAccessingLiveCallFrame() {
int anInt = 4; // random number - by dice
String aString = "some string value";
// get the tool
ToolingInterface tools =
ToolingInterface.getToolingInterface();
// get the frame
CallFrame myFrame = tools.getCallFrame( 0 );
// get local
assertEquals( anInt, frame.getLocal( "anInt" ) );
// update local
anInt = 17; // most random number - proven!
assertEquals( anInt, frame.getLocal( "anInt" ) );
18

29. Dynamic access to local variables
public @Test void sampleOfAccessingLiveCallFrame() {
int anInt = 4; // random number - by dice
String aString = "some string value";
// get the tool
ToolingInterface tools =
ToolingInterface.getToolingInterface();
// get the frame
CallFrame myFrame = tools.getCallFrame( 0 );
// get local
assertEquals( anInt, frame.getLocal( "anInt" ) );
// update local
anInt = 17; // most random number - proven!
assertEquals( anInt, frame.getLocal( "anInt" ) );
// fancy get
assertSame( aString
((CallFrame)myFrame.getLocal( "myFrame" ))
.getLocal( "aString" ) );
18

30. Dynamic access to local variables
public @Test void sampleOfAccessingLiveCallFrame() {
int anInt = 4; // random number - by dice
String aString = "some string value";
// get the tool
ToolingInterface tools =
ToolingInterface.getToolingInterface();
// get the frame
CallFrame myFrame = tools.getCallFrame( 0 );
// get local
assertEquals( anInt, frame.getLocal( "anInt" ) );
// update local
anInt = 17; // most random number - proven!
assertEquals( anInt, frame.getLocal( "anInt" ) );
// fancy get
assertSame( aString
((CallFrame)myFrame.getLocal( "myFrame" ))
.getLocal( "aString" ) );
// convenient this
assertSame( this, frame.getThis() );
}
18

31. code.samples();
19
http://github.com/thobe/java-tooling

32. JVMTI heap introspection
20

33. Heap introspection
๏Native interface (JVMTI)
๏Abilities:
•Following references
•Tagging objects and classes
•Getting tagged objects
•Transitive reachability traversal
•Iterating through all reachable objects
•Iterating through all objects (reachable and not)
•Iterating through all instances of a class
•Getting object size
•Getting object monitor usage 21

34. code.sample();
22
http://github.com/thobe/java-tooling

35. Putting it all together:
Live introspection
23

36. code.demo();
24
http://github.com/thobe/introscript

37. Finding out more
25

38. Web resources
26
๏Java Debug Interface (JDI):
http://docs.oracle.com/javase/7/docs/jdk/api/jpda/jdi/
๏JVM Tooling Interface:
http://docs.oracle.com/javase/7/docs/platform/jvmti/jvmti.html
๏JNI - JVMTI is an “extension” of JNI:
http://docs.oracle.com/javase/7/docs/technotes/guides/jni/spec/jniTOC.html
๏Overview of JDK tools:
http://docs.oracle.com/javase/7/docs/technotes/tools/index.html
๏JVM Attach API:
http://docs.oracle.com/javase/7/docs/jdk/api/attach/spec/

39. Code samples
๏http://github.com/thobe/java-tooling, contains:
•Library for building tools in java
‣Soon: cross compiled native binaries for Mac OS X,Windows, Linux
•Code examples showing how to use these tools
๏https://github.com/thobe/subprocess-testing, contains:
•JDI-based JUnit TestRunner
๏http://github.com/thobe/java-agent, contains:
•Java agent code injection
๏http://github.com/thobe/introscript, contains:
•javax.script.ScriptEngine that attaches to other process
and introspects it with above tools
27

40. http://neotechnology.com