Dynamic Groovy Edges


Published on

Spring and Groovy to create dynamic integration edges and regression isolation.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Dynamic Groovy Edges

  1. 1. 1 Managing Change and Achieving Regression Isolation Dynamic Groovy Edges May 21st, 2012 Prepared for: Triangle Java User’s Group (TriJUG)
  2. 2. 2 Who Am I? Well…  Java Architect with ICF Ironworks  Adjunct Professor  Started with HTML and Lotus Notes in 1992 • In the interim there was C, C++, VB, Lotus Script, PERL, LabVIEW, etc.  Not so much an Early Adopter as much as a Fast Follower of Java Technologies • Learned Java 1.1 in 1997, J2EE in 1999  Alphabet Soup (MCSE, ICAAD, ICASA, SCJP, SCJD, PMP, CSM)  LinkedIn: http://www.linkedin.com/in/iamjimmyray  Blog: http://jimmyraywv.blogspot.com/ Avoiding Tech-sand
  3. 3. 3 Managing Change and Achieving Regression Isolation Dynamic Groovy Edges
  4. 4. 4 JEE Application Architecture  Arranged into layers  When there are N layers: • There are N-1 layer junctions • There are (N-1) X 2 layer edges In this diagram there are three layers, two layer junctions, and four layer edges. We control edges
  5. 5. 5 The Issue with Layers  Often written by someone else • You often have little control or influence over the implementation of the layer to which you must integrate • You control your side (edge) of the interface junction  Notorious change zones • Changes occur at different paces
  6. 6. 6 Keeping Up the Pace  Pace Layering Approach to Application Design • http://www.doozer.com/developer-exchange/entry/accelerating-innovation- by-adopting-a-pace-layered-application-strategy Complex systems can be decomposed into multiple layers, where the layers change at different rates. The “fast layers” learn, absorb shocks and get attention; the “slow layers” remember, constrain and have power. One of the implications of this model is that information architects can do what they have always done–slow, deep, rich work; while tagging can spin madly on the surface. If it is worth keeping, it will seep down into the lower layers. – Donna Maurer
  7. 7. 7 When Layers Change  In Java web applications, regression analysis and testing are usually tied to the boundaries of the application or deployment artifacts: JARs, WARs, EARS etc. • Upstream and downstream dependencies also come into play • Not to be confused with functional testing of changed components  If you redeploy the entire WAR or EAR, you could find yourself testing that entire application boundary. • As well as its integration junctions (critical patches not withstanding)
  8. 8. 8 Detecting Changes  How do we detect changes to web services as soon as possible? • XMLUnit/JUnit Tests • Easily written in Groovy • Hudson scheduled Ant tasks DIFFERENCE CODE:19:CHILD_NODELIST_LENGTH_ID:Comparing 2 nodes with different numbers of children Control(prod) Node: <xs:sequence xmlns:xs="http://www.w3.org/2001/XMLSchema"> <xs:element minOccurs="0" name="address" type="tns:address"/> <xs:element minOccurs="0" name="employeeId" type="xs:string"/> <xs:element minOccurs="0" name="hireDate" type="xs:dateTime"/> </xs:sequence> Target(dev) Node: <xs:sequence xmlns:xs="http://www.w3.org/2001/XMLSchema"> <xs:element minOccurs="0" name="address" type="tns:address"/> <xs:element minOccurs="0" name="department" type="tns:department"/> <xs:element minOccurs="0" name="employeeId" type="xs:string"/> <xs:element minOccurs="0" name="hireDate" type="xs:dateTime"/> </xs:sequence>
  9. 9. 9 Demo – Detecting Changes  Groovy Console  JUnit  XMLUnit  Apache 2.2  Tomcat 6.0.35
  10. 10. 10 CI Using Groovy XmlUnit Tests  Wrap Groovy script with Ant  Execute in Hudson build  Use Gradle with the Groovy Plugin • Run your Groovy test within Gradle build
  11. 11. 11 Managing Change  How do we better manage the change between the layers? • Understand that we cannot stop change • Reduce regression testing needs by shrinking regression boundaries • Prepare our teams for the changes with more dynamic reaction techniques • Start with Groovy beans • Or…Start with Java beans and move to Groovy as needed • Detect changes as early as possible, and before they impact downstream systems
  12. 12. 12 Groovy Edges  Build your edges with Groovy (or other dynamic JVM language) objects • Groovy can be written as a script and parsed at run- time and compiled into Java byte-code and then loaded by the Groovy class loader. • Groovy scripts can contain Groovy and Java syntax in the script file (*.groovy). • Groovy can use the same Java APIs that are used by traditional Java code. • Groovy adds features that make it easier to work with XML (primary language of choice for most integrations) • Markup Builder • XML Slurper
  13. 13. 13 Groovy Markup Builder  Allows developers to use “builder patterns” to construct XML def writer = new StringWriter(); def xml = new MarkupBuilder(writer); xml.'ser:writeEmployee'('xmlns:ser':"http://service.rjug.ironworks.com/"){ employee{ dateOfBirth(emsRequest.getEmployee().getDateOfBirth()) firstName(emsRequest.getEmployee().getFirstName()) lastName(emsRequest.getEmployee().getLastName()) middleName(emsRequest.getEmployee().getMiddleName()) employeeId(emsRequest.getEmployee().getEmployeeId()) hireDate(emsRequest.getEmployee().getHireDate()) address{ addressLine1(emsRequest.getEmployee().getAddress().getAddressLine1()) …
  14. 14. 14 A Word About Hard-Coding (or not)  When is hard-coding not really? • Our internal beliefs have shifted slightly.  How about when it is in Groovy? • Especially when that Groovy is treated as content and not part of a rigid deployment architecture? • Still have to work within the boundaries (constraints) of the JVM
  15. 15. 15 Groovy XML Slurper  Groovy construct for efficient partial XML processing. EmployeeManagerServiceResponse emsResponse = new EmployeeManagerServiceResponse(); GPathResult response; try{ response = new XmlSlurper().parseText(rawResponse.getWriter().toString()); emsResponse.getEmployee().setFirstName(response.'return'.firstName.text()); emsResponse.getEmployee().setLastName(response.'return'.lastName.text()); …
  16. 16. 16 Groovy XML Slurper – Another Example slurper.jobList.job.each { it.docs.doc.each { doc = new DocDto(); docs.add(doc); doc.setDocumentId(new BigInteger(it.documentId.toString())); //doc.setDcmntNbr(new BigInteger(it.dcmntNbr.toString())); doc.setOriginalFilename(it.originalFilename.toString()); doc.setPhasecode(it.phasecode.toString()); doc.setRcvdDt(it.rcvdDt.toString()); doc.setStatuscode(it.statuscode.toString()); doc.setStepcode(it.stepcode.toString()); … … <jobList> <job> <docs> <doc> <originalFilename>562132403073245.pdf</originalFilename> <documentId>42</documentId> <rcvdDt>2009-01-10 00:00:00.0</rcvdDt> <phasecode>A</phasecode> <stepcode>B</stepcode> <statuscode>C</statuscode> </doc> </docs>…
  17. 17. 17 A Word About Groovy Slurping XML Payloads  No longer tied to JAXB • This is uncomfortable for some traditional Java developers  Can use Groovy closures on collections (even nested collections)  Easy to enable/disable getter calls, especially if Groovy is stored outside of rigid deployment structure (CMS, Apache, etc.) • This is scary to some • Still requires Configuration Management rigor • It’s about risk tolerance • It could be that this approach is better suited for non-production environments within certain organizations
  18. 18. 18 Groovy XML Slurper Trade-offs  Use the Groovy XML Slurper when you do not have to parse the entire XML • Slurper uses paths expressions as apposed to constructing a DOM • Path expression might be executed multiple times • Could be slower in larger, more complex XML  There are also differences when editing XML • The XML Parser allows for edits inline • The XML Slurper would require another “slurp” after changes are applied.  The XML Slurper is well suited for simple web service clients • Allows direct access via getters to “return” object graph
  19. 19. 19 Groovy Edge Component – Web Service Client  For SOA interfaces implemented with web services, web service clients are primarily the target edge component for dynamic updating.  Using Groovy to write these web services clients makes it easier to update clients. • Features can be added or removed and quickly “redeployed” via content changes, and not necessarily Java deployments  Spring will reload the Groovy web service client when it changes  This provides a great solution for prototyping in an Agile environment.  For MVC containers, the controller is the edge component that is up for “Groovy-enizing”, “Groovy-ization”, etc.
  20. 20. 20 Spring Beans <lang:groovy id="GroovyInterfaceEdge" script-source="${groovy.url}" refresh-check-delay="1000" > <lang:property name="key" value="value"/> </lang:groovy>
  21. 21. 21 Groovy Location Property groovy.url=http://localhost/groovy/EmployeeManagerServiceImpl.gro ovy  Not only that, but… • Store your properties outside of the WAR/EAR, reachable via URL
  22. 22. 22 Loading Properties <!-- Read service related properties from the classpath --> <bean id="services.propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlacehold erConfigurer"> <property name="location"> <value>http://localhost/properties/service.properties</value> </property> </bean>
  23. 23. 23 Demo – Web Services Provider  Simple JAX-WS Bottom-up web service  Environment: • Apache2.2 • Tomcat 6.0.35 • Java 1.6 JRE • JAX-WS RI 2.2 • JAXB 2.2.3 • SoapUI 4.0.0 • Spring Data Layer – MongoDB Data Repository • MongoDB NoSQL Database (Upcoming TriJUG Event: 07/16/12 Deep Mistry Mongo DB)
  24. 24. 24 Demo – Web Services Consumer  Simple Spring WS Client  Environment: • Apache2.2 • Tomcat 6.0.35 • Java 1.6 JRE • Spring Framework 3.0.x • Spring WS • Groovy Web Services Framework
  25. 25. 25 This Is Not Bleeding Edge  Yet, it is still new to a lot of Spring users  January 2009: http://www.ibm.com/developerworks/java/library/j- groovierspring2/index.html  At the NVSS NFJS conference in Reston in April 2012, there were still sessions on Spring and Groovy integration with dynamic beans • NFJS Due in Raleigh, NC – August 24-26 • http://www.nofluffjuststuff.com/home/main
  26. 26. 26 Spring Bean Configuration Issues  Spring needs to be able to locate Groovy script content in order to load configurations. ERROR - ContextLoader.initWebApplicationContext(220) | Context initialization failed org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'employeeManagerService' defined in file [C:Toolsapache-tomcat-6.0.16w ebappsDynamicGroovyTutorialWEB-INFclassescontextservice_ws.xml]: Cannot resolve reference to bean 'employeeManagerServiceImpl' while setting bean property 'groovyImpl';
  27. 27. 27 Spring Bean Config With Content Proxy  Use a Content Proxy • The Content Proxy is deployed into a separate WAR and addressable via a URL (perhaps a RESTful web service) that would retrieve the dynamic Spring Bean configuration scripts • If the scripts are not found in script repository, the Content Proxy can load the scripts from a backup site or even from inside the application boundaries – Default Location. • The Content Proxy is a layer of indirection to better manage the link between your Spring application and the content management system that you may be using. • Content Proxies are also very helpful when scripts (or other content) are loaded from a CMS through an API that would not link directly to the Spring Bean configurator. • Provides a layer of indirection and intelligence for Cache Forward Architectures.
  28. 28. 28 Avoiding the Race Condition  Using a ServletContextListener allows you to specify how long the application will try to seek the ContentProxy (or other URLs) in other WARs that still may be spinning-up, before it lets the rest of the application load. • Reduces the “race-condition” introduced by URLs across multiple WARs
  29. 29. 29 ServletContextListener web.xml Configuration … <context-param> <description></description> <param-name>proxyUrl</param-name> <param-value>http://localhost/proxy.html</param-value> </context-param> <context-param> <description></description> <param-name>stopInterval</param-name> <param-value>60000</param-value> </context-param> <listener> <listener-class>com.ironworks.rjug.container.listener.ContentProxyListener</listener- class> </listener> …
  30. 30. 30 Inline Groovy – You can, but why would you? package com.ironworks.framework.scripting; import org.springframework.core.io.ResourceLoader; import org.springframework.scripting.ScriptSource; import org.springframework.scripting.support.*; public class URLScriptFactoryPostProcessor extends ScriptFactoryPostProcessor{ @Override protected ScriptSource convertToScriptSource(String beanName, String scriptSourceLocator, ResourceLoader resourceLoader) { if (scriptSourceLocator.startsWith(INLINE_SCRIPT_PREFIX)) { return new StaticScriptSource( scriptSourceLocator.substring(INLINE_SCRIPT_PREFIX.length()), beanName); } else if (scriptSourceLocator.startsWith("classpath")) { return new ResourceScriptSource(resourceLoader.getResource(scriptSourceLocator)); } else{ return new URLScriptSource(resourceLoader.getResource(scriptSourceLocator)); } } }
  31. 31. 31 Spring Bean Delegating Construct  Use a Spring Bean Delegate construct as an alternative to ServletContextListener and ContentProxy for handling potentially unreliable or dynamic URLs during application startup  Deploy JAXB version of services, but leave them open to replacement by Groovy, without needing another deployment • Still able to use Dynamic Groovy Edges, if JAXB version of service client requires updating <bean id="iw_service.getData" class="com.ironworks.vdc.framework.service.DelegatingService"> <property name="delegates"> <list> <ref bean="iw_service.getDataGroovy" /> <ref bean="iw_service.getDataJaxb" /> </list> </property> </bean>
  32. 32. 32 Spring Bean Delegate Construct Code public final class DelegatingService<REQUEST extends ServiceRequest, RESPONSE extends ServiceResponse> implements Service<REQUEST, RESPONSE> { private List<Service<REQUEST, RESPONSE>> delegates = new ArrayList<Service<REQUEST, RESPONSE>>(); @Override public RESPONSE execute(final REQUEST request) { for (Service<REQUEST, RESPONSE> service : delegates) { if (!(service instanceof AvailabilityStatus) || ((AvailabilityStatus) service).isAvailable()) { return service.execute(request); } } throw new UnsupportedOperationException( "No service implementation available to execute request"); } public void setDelegates(final List<Service<REQUEST, RESPONSE>> delegates) { this.delegates = delegates; } }
  33. 33. 33 AvailabilityStatus Interface public interface AvailabilityStatus { /** * Indicate whether this service instance is available. * * @return true if the service is available, false otherwise. */ boolean isAvailable(); }
  34. 34. 34 Spring Delegate Construct Code Explanation  DelegatingService is substituted into the class attribute of the bean definition  Possible “delegates” are added to the property list as objects  DelegatingService looks for viable services in the list of “delegates” • Could be services that implement the AvailabilityStatus interface, and are available (isAvailable()) • Beans that implement AvailabilityStatus, but are not available our by-passed – This allows Groovy substitutes to be enabled/disabled quickly • Could be services that do not implement AvailabilityStatus
  35. 35. 35 Some Tradeoffs for Groovy Approach  Groovy Web Clients • Pros • Easier to program with Getters and Setters approach to accessing and mutating XML objects for request and response • Markup Builder • XML Slurper • Cons • Lose auto-generation of object graphs via JAXB • Lose tighter integration to WSDL/XSD and makes it harder to track down changes. • It’s not for every change…more complex WSDLs, possibly with nested object architectures, would not be good candidates for Groovy re-write
  36. 36. 36 What About All Those Classes?  Almost every technical decision comes with a trade off…  When using dynamic Groovy with the approach we have outlined, obviously we are generating more “non-setup”, Groovy classes in the JVM. • Every time Spring detects a changed script, Groovy will parse it and load another class. • Depending on your JVM, GC settings and JVM memory settings, you could experience OOM issues with PermGen
  37. 37. 37 Does Your JVM Have a PermGen?  IBM: No PermGen, classes exist in the native C-Heap  Oracle/WebLogic JRockit: No PermGen, classes exist in the native C-Heap  Oracle/Sun Hotspot: Yes, PermGen in Java Heap • Prior to Java 5, the Hotspot JVM assumed that classes were forever • Then came CMS (Concurrent Mark Sweep) class unloading settings
  38. 38. 38 What Can We Do With PermGen?  Use GC settings for the CMS Collector • Java 1.6: -XX:+CMSClassUnloadingEnabled • Java 1.5: -XX:+CMSPermGenSweepingEnabled  Both require CMS GC enabled • This may not be suitable for all applications  Increase size (not really desirable) • Switch to 64 bit, if you have not already  Increase JVM bounce frequency (not really desirable) • Not really a good long term strategy
  39. 39. 39 Don’t Use PermGen.  Some of our projects use JRockit • While we do not see the same sensitivity to PermGen in our dynamic environment, occasionally we still see issues with class memory areas • Mostly in non-production environments where we tend to be more dynamic  Problem: JRockit does not exist beyond Java 7 • Oracle has announced that it is being merged with Hotspot in Java 7
  40. 40. 40 Use Java 7?  In Java 7, PermGen is supposed to go away over time. • This implies better class unloading and memory management • JRockit merges with Hotspot • When? • http://javaeesupportpatterns.blogspot.com/2011/10/java-7-features- permgen-removal.html • http://blogs.oracle.com/java/entry/java_7_questions_answers
  41. 41. 41 Groovy Spring MVC Controllers  Spring MVC controllers are another example of edge components that could benefit from more dynamic deployment and less rigid deployment structures • Remember: Just because it is marked as Groovy (*.groovy) does not preclude you from writing some Java <lang:groovy id="claimsController" script-source="${groovyClaimsController}" refresh-check- delay="1000" > <lang:property name="view" value="claim_summary"/> <lang:property name="detailView" value="claim_detail"/> <lang:property name="unsolicitedView" value="claim_unsolicited"/> <lang:property name="errorView" value="claim_error"/> <lang:property name="serviceDownView" value="claim_service_down"/> </lang:groovy>>
  42. 42. 42 Development Environment Configuration  Eclipse Helios SR2  Groovy-Eclipse plugin (The Codehaus) – 2.5.1.xx-2011027…  Groovy 1.8.6  Maven 2.0 (m2) Eclipse Integration – Sonatype, Inc • gmaven-plugin – 1.4.0  MongoDB 2.0.4
  43. 43. 43 Summary  We cannot control changes in layers beyond our influence or control  We can better prepare for change by using dynamic technologies in the edges of interface junctions. • Groovy blurs the once clear line between hard coding and properties configurations. • By using Spring to dynamically load Groovy scripts, stored outside of web applications, web applications can narrow regression boundaries by reducing the need to deploy EARs and WARs. • The Content Proxy provides a layer of indirection that smoothes the process for locating dynamic scripts an providing digestible data for the Spring Bean Configuration process. • Testing WSDL and XSD artifacts with XMLUnit provides early warning of web service changes and failures
  44. 44. 44 Special Thanks To:  Laith Al-Samir – ICF Ironworks • Spring WS application and Groovy WS Client  Mike Killeen – ICF Ironworks • Groovy/XMLUnit/JUnit WSDL/XSD tests  Vince Jorrand – ICF Ironworks • Spring Bean Delegates Construct