Bci for Beginners

  • 712 views
Uploaded on

The range of tools available for the Java platform is one of its many strengths, and a surprising number of these tools rely on bytecode instrumentation (BCI) and bytecode generation to do their work. …

The range of tools available for the Java platform is one of its many strengths, and a surprising number of these tools rely on bytecode instrumentation (BCI) and bytecode generation to do their work. A few of the many examples of where BCI is used are profiling tools, static analysis tools (FindBugs), dynamic languages (Nashorn, JRuby), and code coverage (EMMA). But BCI is a black art to most. This session aims to take away the mystery and show you what BCI is, how it is used, and how you can get started on writing your own BCI-based tooling.

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
712
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
19
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • As the title of the session suggests, the idea was to get together people who are interested in BCI, but don't yet know where they are going with it. I'm not a world expert! Can help you avoid some of the pitfalls (new lines in jar manifest files) Suggestions for tools to help you
  • Bytecode technically refers to the opcodes in a class, that is the method bodies. However, the whole class file can be modified BCI can be added/removed at will while an application is running
  • Not a hack, Been available since java 5
  • At load time can do almost anything: Restrictions: You can't go mad! If you removed a method from a class which implements an interface, the class would no longer verify, and would fail to load Some things can't be done whilst running maybe allow you to replace a newer version if for some reason it was difficult to get a new version/recompile
  • Examples: Monitoring, logging/tracing, profiling Servlets running in an application server (tomcat, websphere, glassfish) Running the server, no idea what the application does Can inject instrumentation at the top of this method to record URL, time taken, where the request has come from..
  • There are other mechanisms for doing some of the things mentioned previously Why do them using BCI?
  • My main focus is dynamic BCI, where the classes are modified either as the JVM loads the classes, or whilst an application is running. Agents can be written in c, or java. Java agents are easier to deal with JVMTI has a larger interface, allows your agent to do more than just BCI
  • Remember the new line at the end of the manifest!
  • A class file transformer gets a call back whenver a new class is loaded Retransform is how you trigger modifcation of an already running class
  • All transformations boil down to: Take an array of bytes, copy and modify the array, give it back to the JVM
  • Offline- all of the above would be done by reading in a class file into a byte array
  • Summary Java SDK provides a mechanism for transforming classes Gives you a a byte array Not very much other help Now, need some help to actually twiddle the bytes
  • The following slides focus on making it easy to add new logic to an existing class Other frameworks are available Javassist, bcel n.b A basic understanding of how a JVM works is still required
  • Easier to learn, but non-trivial java api Does provide helper methods to do make common tasks easier (e.g. local variable sorter) Now give brief overview, No details emphasis, this is just java, and there are only 3 classes and three methods
  • This is class one, create a new instance of a method adapter
  • In method adapter, override one method, create a MethodAdapter class
  • This gets you to the point of: My class is parsed, I have a method body to create/modify ASM provides methods to add bytecodes but Doesn't tell you what bytecodes to add! Knowledge of the JVM architecture is still required Next step, make creating new logic easier
  • This is just part of the output of javap Also includes the whole of the constant pool Every method in the class All the meta data (stack maps, line number tabel, variable table) Not much help to go rebuild a similar method in your agent But I do recommend it as a help to understanding how the JVM works
  • Show the profiler logic in java show the bci logic in java Show byte code plugin to turn that into ASM (show view, other, byte code, use switch to change between asm and non-asm) Paste ASM code into boiler plate method transformer
  • The following slides focus on the task of - adding new logic to an existing class n.b A basic understanding of how a JVM works is still required

Transcript

  • 1. © 2013 IBM Corporation Iain Lewis 30 Sept 2013 BCI for Beginners
  • 2. © 2013 IBM Corporation Important Disclaimers THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES. ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE. IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE. IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS 2
  • 3. © 2013 IBM Corporation About me  Iain Lewis  QA Engineer, IBM Java Technology Center, IBM Hursley, U.K.  13 years experience developing and deploying Java SDKs   Contact info – iain_lewis@uk.ibm.com  Visit the IBM booth #5112 and meet other IBM developers at JavaOne 2013 3
  • 4. © 2013 IBM Corporation Agenda  Who am I, and why am I here  What is BCI?  BCI and the JVM  Practical BCI
  • 5. © 2013 IBM Corporation Who am I?  Test IBM's Java SDK – I break Java  Focus on system testing in QA – Multi-threaded load testing – Third-party applications  System testing the Java SDK means writing Java  Can the VM handle any application or Java code?  Recently wrote some BCI based testing
  • 6. © 2013 IBM Corporation Why are you here?  Have heard of BCI  Would like to know more about – What BCI is – What BCI can do  Would like practical help on – How to get started – Where to get help – What tools are available – 
  • 7. © 2013 IBM Corporation What is BCI?  Bytecode Instrumentation is modifying a Java class without the source  Can be done offline, at load time, or at any point afterwards, and multiple times  Application and most SDK classes can be changed  Class File BCI Modified Class JVM Disk/Network Runtime
  • 8. © 2013 IBM Corporation Why BCI?  Support for BCI at runtime is built into the Java SDK  Java bytecodes are well documented and easy to work with  BCI is supported by many tools  BCI is widely used
  • 9. © 2013 IBM Corporation What can you do with BCI?  Offline/at load time – Replace the entire class with a different one! – Add/Remove fields and methods  At any time – Modify the logic of existing methods – Insert new logic into existing methods  Allows you to do – Profiling/monitoring – Code coverage – Code generation – Runtime logging 
  • 10. © 2013 IBM Corporation What can you do with BCI? (cont) public void service(ServletRequest req, ServletResponse res) throws ServletException, java.io.IOException { // insert new logic here // existing method logic }
  • 11. © 2013 IBM Corporation Why BCI?  Doesn't have to be compiled in to a class  Doesn't need access to the source  Can be turned on and off  Can be applied to an arbitrary application  When turned off, has no performance impact
  • 12. © 2013 IBM Corporation How do you do it? JVM interface  BCI is supported at the JVM level  Supported via 'agents' and the Instrumentation interface  java -javaagent:/path/to/agent.jar[=options] MyClass  Native agents also possible via JVMTI 
  • 13. © 2013 IBM Corporation How do you do it? JVM interface (cont.) import java.lang.instrument.Instrumentation; public class MyJavaAgent { public static void premain(String agentArgument, Instrumentation instrumentation) {} } Manifest-Version: 1.0 Premain-Class: MyJavaAgent Can-Retransform-Classes: true
  • 14. © 2013 IBM Corporation How do you do it? JVM interface (cont.)  Allows your agent to: – Register as a ClassFileTransformer – Transformers get a call back when a class is loaded, in order to transform it  Can also trigger retransformations via retransformClasses() •  Documentation of Instrumentation : – http://docs.oracle.com/javase/7/docs/api/index.html?java/lang/instrument/Instrumentation.htm void addTransformer(ClassFileTransformer transformer)
  • 15. © 2013 IBM Corporation How do you do it? JVM Interface (cont)  A class is an array of bytes  A transformer returns a modified copy of the byte array Class byte array from disk Transformer 1 Copied/Modified byte array Transformer 2 Final byte array public static byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) {}
  • 16. © 2013 IBM Corporation How do you do it? Modifying Bytes (the hard way)  Read the Java Virtual Machine Specification – http://docs.oracle.com/javase/specs/jvms/se7/html/  Parse the class byte array – http://docs.oracle.com/javase/7/docs/api/java/io/DataInputStream.html  Return a modified version
  • 17. © 2013 IBM Corporation How do you do it? Modifying Bytes (the hard way) DataInputStream is = new DataInputStream(new ByteArrayInputStream(classFileBuffer))  int magic = is.readInt(); System.out.println("Magic number is " + magic); // 0xCAFEBABE  int minor = is.readShort(); System.out.println("minor number is " + minor); // 0 for Java 7  int major = is.readShort(); System.out.println("major number is " + major); // 51 for Java 7
  • 18. © 2013 IBM Corporation How do you do it? Modifying Bytes (the easier way)  Use a framework – e.g. ASM http://asm.ow2.org  Use javac/javap  Use a bytecode plugin for your IDE – Bytecode outline plugin for Eclipse http://andrei.gmxhome.de/bytecode/index.html  Write as much of your code in Java as possible
  • 19. © 2013 IBM Corporation Using a Framework (ASM)  Provides an abstraction layer to hide the details of a class  Easier to learn (provides symbolic constants, helper APIs)  Widely used  Under active development   Is an additional dependency  Doesn't tell you what bytecodes to create!
  • 20. © 2013 IBM Corporation A transformer in 3 classes and 3 methods – class 1 /** Implementation of transform() in ClassFileTransformer */ public byte[]transform(...) throws IllegalClassFormatException { ClassWriter writer = new ClassWriter(ClassWriter.COMPUTE_FRAMES); ClassAdapter1 adapter = new ClassAdapter1(writer); ClassReader cr = new ClassReader(classfileBuffer); cr.accept(adapter, ClassReader.SKIP_FRAMES); return writer.toByteArray(); }
  • 21. © 2013 IBM Corporation A transformer in 3 classes and 3 methods – class 2 /** Override of visitMethod in ClassVisitor */ public MethodVisitor visitMethod(...) { MethodVisitor mv = cv.visitMethod(...); return new MethodAdapter1(mv); }
  • 22. © 2013 IBM Corporation A transformer in 3 classes and 3 methods – class 3 /** Override of visitCode in MethodVisitor */ public void visitCode() { mv.visitCode(); // Use ASM api calls to add profiler logic here }
  • 23. © 2013 IBM Corporation Writing bytecodes with javac/javap  Writing logic in bytecode is possible, but not simple  Get your computer to do it for you!  Write your logic in Java  javac turns Java code into bytecode, packaged up in a class file  javap -v turns a class file into a human-readable format  Why javap? (and why not) – Shipped with the Java SDK – Easy to use – Shows you the whole of the class, not just methods (helpful for learning) – Not easy to translate javap output into the Java code to produce it
  • 24. © 2013 IBM Corporation Javap on HelloWorld public static void main(String[] args) { System.out.println(“Hello World”); } javap -v HelloWorld.class
  • 25. © 2013 IBM Corporation Javap on HelloWorld (...) public static void main(java.lang.String[]); flags: ACC_PUBLIC, ACC_STATIC Code: stack=2, locals=1, args_size=1 0: getstatic #16 // Field java/lang/System.out:Ljava/io/PrintStream; 3: ldc #22 // String Hello World 5: invokevirtual #24 // Method java/io/PrintStream.println:(Ljava/lang/String;)V 8: return LineNumberTable: line 7: 0 line 9: 8 LocalVariableTable: Start Length Slot Name Signature 0 9 0 args [Ljava/lang/String; (...)
  • 26. © 2013 IBM Corporation Writing bytecodes with the Bytecode Outline Eclipse Plugin  Plugin for Eclipse  Turns Java source code into bytecodes  Also shows you the Java code using ASM API calls to create the equivalent bytecode  (demo)
  • 27. © 2013 IBM Corporation Demo  Very simple counting profiler  Prints out the number of times each method in your application was called  (demo)
  • 28. © 2013 IBM Corporation Recap  Use a framework – e.g. ASM http://asm.ow2.org  Get your computer to write your bytecode for you – Use javac and javap – Use the Bytecode Outline Plugin for Eclipse (or similar)  Write as much of your code in Java as possible   BCI is very powerful and used in many products  JVM knowledge is required but...  There are many tools available to make it easier
  • 29. © 2013 IBM Corporation29 http://ibm.co/JavaOne2013 IBM booth #5112 Wednesday 1pm - 3pm
  • 30. © 2013 IBM Corporation References  Eclipse Bytecode Outline plugin (allows you to use asmifier in Eclipse) – http://andrei.gmxhome.de/bytecode/index.html  ASM – http://asm.ow2.org/index.html  java.lang.instrument package documentation – http://docs.oracle.com/javase/7/docs/api/index.html?java/lang/instrument/package- summary.html
  • 31. © 2013 IBM Corporation References  Inside the Java Virtual Machine – Bill Venners – http://www.artima.com/insidejvm/ed2/jvm.html  Java VM Spec – http://docs.oracle.com/javase/specs/jvms/se7/html/index.html