Power tools in Java

1. 1 Power Tools in Java SDK Elek Márton DPC Consulting Kft

2. 2 • Modular static html site generator • (like Jekyll, github pages) • Do custom transformation on each text file • Example transformations: • ascii link break → <br/> • plain text → text with html header/footer <html>....<body>{...}</body></html> Example application

3. 3 Code example – types public class Resource { private Map<String, String> headers = new HashMap<>(); private String content; … } public interface Transformer { public void transform(Resource resource); }

4. 4 Code example – processing System.out.println("Transforming " + file); Resource r = new Resource(new String(Files.readAllBytes(file))); for (Transformer transformations : transformers) { transformations.transform(r); } return r;

5. 5 Modularity

6. 6 • Kezdeti megoldás: private List<Transformer> transformations = new ArrayList(); public Generator() { transformations.add(new HtmlFrameTransformator()); transformations.add(new Nl2BrTransfomer()); } for (Transformer transformations : transformers) { transformations.transform(r); } • Cél • Dinamikusan kiterjeszthető • lib/-be új JAR → automatikusan elinduljon Modularitás

7. 7 Step one: using descriptors • bin/ • site-generator.bat • site-generator.sh • lib/ • site-generator-1.0.jar • site-generator-extension-1.0.jar • config • transform-implementations.cfg: hu.dpc.javabar.powertools.impl.HtmlFrameTransformator hu.dpc.javabar.powertools.impl.Nl2BrTransfomer hu.dpc.javabar.powertools.impl.YoutubeTransformer

8. 8 Step two: descriptor on classpath • bin/ • site-generator.bat, site-generator.sh • lib/ • site-generator-1.0.jar • site-generator-1.0.jar:/transform-implementations.cfg hu.dpc.javabar.powertools.impl.HtmlFrameTransformator hu.dpc.javabar.powertools.impl.Nl2BrTransfomer • site-generator-extension-1.0.jar • site-generator-extension-1.0.jar:/transform-implementations.cfg hu.dpc.javabar.powertools.impl.YoutubeTransformer Enumeration<URL> resources = getClass().getClassLoader().getResources("/transform-implementations.cfg");

9. 9 Step three: Service Provider Interface • Standard, part of the JDK • Descriptor are on the classpath • Contains the class names of the implementations • Using nameconvention (META-INF/services/hu.dpc...Interface) • Simple API (ServiceLoader), only a few classes • Not included: • Dependency management • Versioning • Priority handling • Only answers the question: • Who implemented a specific interface

10. 10 SPI code example • META-INF/services/hu.dpc.javabar.powertools.api.Transformer • hu.dpc.javabar.powertools.impl.HtmlFrameTransformer • hu.dpc.javabar.powertools.impl.Nl2BrTransfomer ServiceLoader<Transformer> transformers = ServiceLoader.load(Transformer.class); for (Transformer transfomer : transformers) { ... }

11. 11 SPI – where is it used • (almost) everywhere • multiple location at JDK (eg. XML processor implementation) • Even the OSGi has adapter for SPI • Very effective even if it's very simple • Easy solution to avoid circular dependencies of the projects

12. 12 Automatic SPI generation

13. 13 How to generate SPI descriptors? • Service Provider Interface META-INF/services/... should be created and filled with implementation automatically

14. 14 Annotation processors • Part of the javac from Java 6 • Run at compile time • Class/descriptor/... files could be generated based on specific annotated classes • Could be activated with SPI • META-INF/services/ javax.annotation.processing.Processor • Multiple iteration (processing the generated files) • Parameters could be used

15. 15 Annotation processor public @interface Service { } @Service public class Nl2BrTransfomer implements Transformer { @Override public void transform(Resource resource) { resource.setContent( resource.getContent().replaceAll("nn", "<br/>") ); } }

16. 16 Annotation processor example @SupportedAnnotationTypes("hu.dpc.javabar.powertools.Service") @SupportedSourceVersion(SourceVersion.RELEASE_7) public class ServiceAnnotationProcessor extends AbstractProcessor { Map<String, Set<String>> services = new HashMap<String, Set<String>>(); @Override public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) { … } }

17. 17 Annotation processor example Map<String, Set<String>> services = new HashMap<String, Set<String>>(); @Override public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) { processingEnv.getMessager().printMessage(Diagnostic.Kind.NOTE, "@service generation"); if (roundEnv.processingOver()) { writeOutServices(roundEnv); return false; } //process annotations ... }

18. 18 Annotation processor example public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) { //write out the file ... Elements elements = processingEnv.getElementUtils(); for (Element element : roundEnv.getElementsAnnotatedWith(Service.class)) { TypeElement type = (TypeElement) element; TypeElement contract = (TypeElement) ((DeclaredType) type.getInterfaces().get(0)).asElement(); String cn = elements.getBinaryName(contract).toString(); Set<String> v = services.get(cn); if (v == null) { services.put(cn, v = new TreeSet<String>()); } v.add(cn); } return false; }

19. 19 Generating code for (Element element : roundEnv.getElementsAnnotatedWith(Service.class)) { TypeElement type = (TypeElement) element; JavaFileObject jfo = processingEnv.getFiler().createSourceFile( type.getQualifiedName() + "GeneratedInfo"); BufferedWriter bw = new BufferedWriter(jfo.openWriter()); bw.append("package "); bw.append(((PackageElement) type.getEnclosingElement()).getQualifiedName()); bw.append(";n"); bw.append("import hu.dpc.javabar.powertools.Service;n"); bw.newLine(); bw.newLine(); bw.append("@Service n"); bw.append(String.format("public class %s{", type.getSimpleName())); bw.newLine(); bw.append("public static final boolean PROCESSED = true;" ); bw.append("}"); bw.flush();

20. 20 Project to generate SPI • To generate SPI descriptors, try • http://metainf-services.kohsuke.org/ • https://code.google.com/p/spi/ <dependency> <groupId>org.kohsuke.metainf-services</groupId> <artifactId>metainf-services</artifactId> <version>1.1</version> <optional>true</optional> </dependency>

21. 21 Runtime Javadoc access

22. 22 Documentation of SPI implementation /** * Replace new double new lines with html line break. */ @Service("Replace new double new lines with html line break") public class Nl2BrTransfomer implements Transformer { @Override public void transform(Resource resource) { resource.setContent(resource.getContent().replaceAll("nn", "<br/>")); } } Available transformations: HtmlFrameTransformator : ... Nl2BrTransfomer : Replace new double new lines with html line break

23. 23 How to access javadoc at runtime? >>> def hello(name): ... """Print hello for the name""" ... print("Hello", name) ... >>> hello("Bela") Hello Bela >>> print(hello.__doc__) Print hello for the name user=> (defn hello "Print hello for the name" [name] (str "Hello " name)) #'user/hello user=> (hello "Bela") "Hello Bela" user=> (get (meta (var hello)) :doc) "Print hello for the name" • The documentation of the annotated service class should come from javadoc documents • Easy with other languages(Python, Clojure)

24. 24 Javadoc – Doclets • Formatting classes for custom javadoc generation • Doclet API could access all of the javadoc and class metadata information • Could be used to generate custom javadoc page but also to generate XML descriptors

25. 25 Doclet example public class GenerateDocletResource { public static boolean start(RootDoc rootDoc) { for (ClassDoc cd : rootDoc.specifiedClasses()) { Path p = Paths.get(root + "/.../" + cd.qualifiedTypeName() + ".javadoc"); Files.createDirectories(p.getParent()); Files.write(p, cd.commentText().getBytes()); } return true; } }

26. 26 Doclet loading public String getJavadoc(Class type) { String resourceName = "/" + type.getCanonicalName() + ".javadoc"; try (InputStream s = getClass().getResourceAsStream(resourceName)){ if (s == null) { return "Javadoc resource is missing for class " + type; } return new Scanner(s).useDelimiter("//Z").next(); } catch (IOException ex) { ... } }

27. 27 Using it from Gradle dependencies { doclet project(":javadoc-metadata-generator") } task generateDocs(type: Javadoc) { source = sourceSets.main.allJava options = new SimpleJavadocOption() options.docletpath = configurations.doclet.files.asType(List) options.doclet = "hu.dpc.javabar.powertools.doc.GenerateDocletResource" } compileJava.dependsOn generateDocs

28. 28 Using it from Maven <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-javadoc-plugin</artifactId> <version>2.9.1</version> <executions> <execution> <id>doc-resource-generator</id> <phase>prepare-package</phase> <goals>javadoc</goals> </execution> </executions> <configuration> <doclet>org.umlgraph.doclet.UmlGraphDoc</doclet>  <docletArtifact> <groupId>org.umlgraph</groupId> <artifactId>doclet</artifactId> <version>5.1</version> </docletArtifact> <additionalparam>-views</additionalparam> <useStandardDocletOptions>true</useStandardDocletOptions> </configuration> </plugin>

29. 29 Runtime metrics

30. 30 Java Management Extension (JMX) • Well known solution from the Standard Java API • Management • Monitoring • All of the JVM information could be accessed as well • Custom metrics could be added

31. 31 JMX JSR • JSR 3: Java Management Extension (JMX) • JSR 160: JMX Remote API • JSR 255: JMX 2.0 = JSR 3 + 160 + … (Inactive) • JSR 262: Web Services Connector for JMX (Inactive) • Jolokia: REST interface adapter

32. 32 JMX Architecture Source wikipedia

33. 33 JMX – Probe level • MBean – Managed Object: bármilyen resource • Attributes • Operations • Notifications • MBean types • Standard • Dynamic • MXBeans

34. 34 Simple JMX example public interface GeneratorMBean { public void stop(); public int getProcessedFileNo(); } MBeanServer server = ManagementFactory.getPlatformMBeanServer(); ObjectName name = new ObjectName("hu.dpc.javabar.powertools", "name", "Generator"); server.registerMBean(generator, name); //generator implements GeneratorMBean

36. 36 JMX platform Mbean server MBeanServer server = ManagementFactory.getPlatformMBeanServer();

37. 37 JMX custom Mbean server MBeanServer server = MbeanServerFactory.createMBeanServer(); Registry registry = LocateRegistry.createRegistry(1234); JMXServiceURL jmxServiceURL = new JMXServiceURL("rmi", "localhost", 1235, "/jndi/rmi://localhost:1234/jmx"); JMXConnectorServer connectorServer = JMXConnectorServerFactory.newJMXConnectorServer(jmxServiceURL, null, server); connectorServer.start(); System.out.println(jmxServiceURL); // service:jmx:rmi://localhost:1235/jndi/rmi://localhost:1234/jmx

38. 38 JMX client API JMXServiceURL url = new JMXServiceURL("service:jmx:rmi://localhost:1235/jndi/rmi://localhost:1234/jmx"); JMXConnector jmxc = JMXConnectorFactory.connect(url, null); MBeanServerConnection mbsc = jmxc.getMBeanServerConnection(); ObjectName name = new ObjectName("hu.dpc.javabar.powertools", "name", "Generator"); GeneratorMBean mbean = JMX.newMBeanProxy(mbsc, name, GeneratorMBean.class); System.out.println(mbean.getProcessedFileNo()); //12

40. 40 JMX custom MBeanServer MBeanServer server = MbeanServerFactory.createMBeanServer(); Registry registry = LocateRegistry.createRegistry(1234); JMXServiceURL jmxServiceURL = new JMXServiceURL("rmi", "localhost", 1235, "/jndi/rmi://localhost:1234/jmx"); JMXConnectorServer connectorServer = JMXConnectorServerFactory.newJMXConnectorServer(jmxServiceURL, null, server); connectorServer.start(); System.out.println(jmxServiceURL); // service:jmx:rmi://localhost:1235/jndi/rmi://localhost:1234/jmx

41. 41 JMX platform MBeanServer MBeanServer server = ManagementFactory.getPlatformMBeanServer();

42. 42 Agents

43. 43 Java Agents • Written in Java or C/C++ • Native: Java Virtual Machine Tool Interface (JVM TI) • -agentlib, -agentpath • Java: • -javaagent • Features • run custom command (in the application JVM-ben) • manipulating bytecode loading • Activating • Before application startup (-javaagent) • Attaching at runtime (Attach API)

44. 44 Java Agent interface public static void premain(String agentArgs, Instrumentation inst) { } • Intstrumentation • addTransformer(ClassFileTransformer transformer) • Registers the supplied transformer. • appendToBootstrapClassLoaderSearch(JarFile jarfile) • Specifies a JAR file with instrumentation classes to be defined by the bootstrap class loader. • appendToSystemClassLoaderSearch(JarFile jarfile) • Specifies a JAR file with instrumentation classes to be defined by the system class loader • ClassFileTransformer. • transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer)

45. 45 Activating Java Agent • Before application start • -javaagent:<jarpath>[=<options>] • META-INF/MANIFEST.MF: Premain-Class: … • public static void premain(...) • During application run • Attach API • META-INF/MANIFEST.MF: Agent-Class: … • public static void agentmain(...)

46. 46 Attach API //JVMs of the CURRENT user List<VirtualMachineDescriptor> vms = VirtualMachine.list(); for (VirtualMachineDescriptor virtualMachineDescriptor : vms) { VirtualMachine vm = VirtualMachine.attach(virtualMachineDescriptor); if (vm != null) { System.out.println(virtualMachineDescriptor.displayName()); System.out.println("Java version = " + vm.getSystemProperties(). getProperty("java.version")); } }

47. 47 JMX connection – with agents static final String CONNECTOR_ADDRESS = "com.sun.management.jmxremote.localConnectorAddress"; VirtualMachine vm = VirtualMachine.attach(id); String connectorAddress = vm.getAgentProperties().getProperty(CONNECTOR_ADDRESS); if (connectorAddress == null) { String agent = vm.getSystemProperties().getProperty("java.home") + File.separator + "lib" + File.separator + "management-agent.jar"; vm.loadAgent(agent); connectorAddress = vm.getAgentProperties().getProperty(CONNECTOR_ADDRESS); } JMXServiceURL url = new JMXServiceURL(connectorAddress); JMXConnector = JMXConnectorFactory.connect(url);

48. 48 Where is it used? • Bytecode manipulation • AspectJ • Spring • Persistence frameworks • BTrace • premain/agent: • jre/lib/management-agent.jar

49. 49 Summary • JDK Power tools • Service Provider Interface – for modularity • Annotation processors – compile time processing • Javadoc doclet – Javadoc (pre)processing • JMX – runtime metrics, methods • Agentek, Attach API – byte code manipulation, runtime modification

50. 50 Credits • Thanks for the image authors: • Container, Sandor Volenszki, Creative Commons • Blueprint, Will Scullin, Creative Commons • Robot, Avram Cheaney, Creative Commons • Control Panel, Paul, Creative Commons • PCB, Karl-Ludwig G. Poggemann, Creative Commons

51. 51 Elek Márton / DPC Consulting marton.elek@dpc.hu http://www.meetup.com/bpjavabar/