Dynamic Groovy Edges
Upcoming SlideShare
Loading in...5

Dynamic Groovy Edges



Spring and Groovy to create dynamic integration edges and regression isolation.

Spring and Groovy to create dynamic integration edges and regression isolation.



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Dynamic Groovy Edges Dynamic Groovy Edges Presentation Transcript

  • 1 Managing Change and Achieving Regression Isolation Dynamic Groovy Edges May 21st, 2012 Prepared for: Triangle Java User’s Group (TriJUG)
  • 2 Who Am I? Well…  Java Architect with ICF Ironworks  Adjunct Professor  Started with HTML and Lotus Notes in 1992 • In the interim there was C, C++, VB, Lotus Script, PERL, LabVIEW, etc.  Not so much an Early Adopter as much as a Fast Follower of Java Technologies • Learned Java 1.1 in 1997, J2EE in 1999  Alphabet Soup (MCSE, ICAAD, ICASA, SCJP, SCJD, PMP, CSM)  LinkedIn: http://www.linkedin.com/in/iamjimmyray  Blog: http://jimmyraywv.blogspot.com/ Avoiding Tech-sand
  • 3 Managing Change and Achieving Regression Isolation Dynamic Groovy Edges
  • 4 JEE Application Architecture  Arranged into layers  When there are N layers: • There are N-1 layer junctions • There are (N-1) X 2 layer edges In this diagram there are three layers, two layer junctions, and four layer edges. We control edges
  • 5 The Issue with Layers  Often written by someone else • You often have little control or influence over the implementation of the layer to which you must integrate • You control your side (edge) of the interface junction  Notorious change zones • Changes occur at different paces
  • 6 Keeping Up the Pace  Pace Layering Approach to Application Design • http://www.doozer.com/developer-exchange/entry/accelerating-innovation- by-adopting-a-pace-layered-application-strategy Complex systems can be decomposed into multiple layers, where the layers change at different rates. The “fast layers” learn, absorb shocks and get attention; the “slow layers” remember, constrain and have power. One of the implications of this model is that information architects can do what they have always done–slow, deep, rich work; while tagging can spin madly on the surface. If it is worth keeping, it will seep down into the lower layers. – Donna Maurer
  • 7 When Layers Change  In Java web applications, regression analysis and testing are usually tied to the boundaries of the application or deployment artifacts: JARs, WARs, EARS etc. • Upstream and downstream dependencies also come into play • Not to be confused with functional testing of changed components  If you redeploy the entire WAR or EAR, you could find yourself testing that entire application boundary. • As well as its integration junctions (critical patches not withstanding)
  • 8 Detecting Changes  How do we detect changes to web services as soon as possible? • XMLUnit/JUnit Tests • Easily written in Groovy • Hudson scheduled Ant tasks DIFFERENCE CODE:19:CHILD_NODELIST_LENGTH_ID:Comparing 2 nodes with different numbers of children Control(prod) Node: <xs:sequence xmlns:xs="http://www.w3.org/2001/XMLSchema"> <xs:element minOccurs="0" name="address" type="tns:address"/> <xs:element minOccurs="0" name="employeeId" type="xs:string"/> <xs:element minOccurs="0" name="hireDate" type="xs:dateTime"/> </xs:sequence> Target(dev) Node: <xs:sequence xmlns:xs="http://www.w3.org/2001/XMLSchema"> <xs:element minOccurs="0" name="address" type="tns:address"/> <xs:element minOccurs="0" name="department" type="tns:department"/> <xs:element minOccurs="0" name="employeeId" type="xs:string"/> <xs:element minOccurs="0" name="hireDate" type="xs:dateTime"/> </xs:sequence>
  • 9 Demo – Detecting Changes  Groovy Console  JUnit  XMLUnit  Apache 2.2  Tomcat 6.0.35
  • 10 CI Using Groovy XmlUnit Tests  Wrap Groovy script with Ant  Execute in Hudson build  Use Gradle with the Groovy Plugin • Run your Groovy test within Gradle build
  • 11 Managing Change  How do we better manage the change between the layers? • Understand that we cannot stop change • Reduce regression testing needs by shrinking regression boundaries • Prepare our teams for the changes with more dynamic reaction techniques • Start with Groovy beans • Or…Start with Java beans and move to Groovy as needed • Detect changes as early as possible, and before they impact downstream systems
  • 12 Groovy Edges  Build your edges with Groovy (or other dynamic JVM language) objects • Groovy can be written as a script and parsed at run- time and compiled into Java byte-code and then loaded by the Groovy class loader. • Groovy scripts can contain Groovy and Java syntax in the script file (*.groovy). • Groovy can use the same Java APIs that are used by traditional Java code. • Groovy adds features that make it easier to work with XML (primary language of choice for most integrations) • Markup Builder • XML Slurper
  • 13 Groovy Markup Builder  Allows developers to use “builder patterns” to construct XML def writer = new StringWriter(); def xml = new MarkupBuilder(writer); xml.'ser:writeEmployee'('xmlns:ser':"http://service.rjug.ironworks.com/"){ employee{ dateOfBirth(emsRequest.getEmployee().getDateOfBirth()) firstName(emsRequest.getEmployee().getFirstName()) lastName(emsRequest.getEmployee().getLastName()) middleName(emsRequest.getEmployee().getMiddleName()) employeeId(emsRequest.getEmployee().getEmployeeId()) hireDate(emsRequest.getEmployee().getHireDate()) address{ addressLine1(emsRequest.getEmployee().getAddress().getAddressLine1()) …
  • 14 A Word About Hard-Coding (or not)  When is hard-coding not really? • Our internal beliefs have shifted slightly.  How about when it is in Groovy? • Especially when that Groovy is treated as content and not part of a rigid deployment architecture? • Still have to work within the boundaries (constraints) of the JVM
  • 15 Groovy XML Slurper  Groovy construct for efficient partial XML processing. EmployeeManagerServiceResponse emsResponse = new EmployeeManagerServiceResponse(); GPathResult response; try{ response = new XmlSlurper().parseText(rawResponse.getWriter().toString()); emsResponse.getEmployee().setFirstName(response.'return'.firstName.text()); emsResponse.getEmployee().setLastName(response.'return'.lastName.text()); …
  • 16 Groovy XML Slurper – Another Example slurper.jobList.job.each { it.docs.doc.each { doc = new DocDto(); docs.add(doc); doc.setDocumentId(new BigInteger(it.documentId.toString())); //doc.setDcmntNbr(new BigInteger(it.dcmntNbr.toString())); doc.setOriginalFilename(it.originalFilename.toString()); doc.setPhasecode(it.phasecode.toString()); doc.setRcvdDt(it.rcvdDt.toString()); doc.setStatuscode(it.statuscode.toString()); doc.setStepcode(it.stepcode.toString()); … … <jobList> <job> <docs> <doc> <originalFilename>562132403073245.pdf</originalFilename> <documentId>42</documentId> <rcvdDt>2009-01-10 00:00:00.0</rcvdDt> <phasecode>A</phasecode> <stepcode>B</stepcode> <statuscode>C</statuscode> </doc> </docs>…
  • 17 A Word About Groovy Slurping XML Payloads  No longer tied to JAXB • This is uncomfortable for some traditional Java developers  Can use Groovy closures on collections (even nested collections)  Easy to enable/disable getter calls, especially if Groovy is stored outside of rigid deployment structure (CMS, Apache, etc.) • This is scary to some • Still requires Configuration Management rigor • It’s about risk tolerance • It could be that this approach is better suited for non-production environments within certain organizations
  • 18 Groovy XML Slurper Trade-offs  Use the Groovy XML Slurper when you do not have to parse the entire XML • Slurper uses paths expressions as apposed to constructing a DOM • Path expression might be executed multiple times • Could be slower in larger, more complex XML  There are also differences when editing XML • The XML Parser allows for edits inline • The XML Slurper would require another “slurp” after changes are applied.  The XML Slurper is well suited for simple web service clients • Allows direct access via getters to “return” object graph
  • 19 Groovy Edge Component – Web Service Client  For SOA interfaces implemented with web services, web service clients are primarily the target edge component for dynamic updating.  Using Groovy to write these web services clients makes it easier to update clients. • Features can be added or removed and quickly “redeployed” via content changes, and not necessarily Java deployments  Spring will reload the Groovy web service client when it changes  This provides a great solution for prototyping in an Agile environment.  For MVC containers, the controller is the edge component that is up for “Groovy-enizing”, “Groovy-ization”, etc.
  • 20 Spring Beans <lang:groovy id="GroovyInterfaceEdge" script-source="${groovy.url}" refresh-check-delay="1000" > <lang:property name="key" value="value"/> </lang:groovy>
  • 21 Groovy Location Property groovy.url=http://localhost/groovy/EmployeeManagerServiceImpl.gro ovy  Not only that, but… • Store your properties outside of the WAR/EAR, reachable via URL
  • 22 Loading Properties <!-- Read service related properties from the classpath --> <bean id="services.propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlacehold erConfigurer"> <property name="location"> <value>http://localhost/properties/service.properties</value> </property> </bean>
  • 23 Demo – Web Services Provider  Simple JAX-WS Bottom-up web service  Environment: • Apache2.2 • Tomcat 6.0.35 • Java 1.6 JRE • JAX-WS RI 2.2 • JAXB 2.2.3 • SoapUI 4.0.0 • Spring Data Layer – MongoDB Data Repository • MongoDB NoSQL Database (Upcoming TriJUG Event: 07/16/12 Deep Mistry Mongo DB)
  • 24 Demo – Web Services Consumer  Simple Spring WS Client  Environment: • Apache2.2 • Tomcat 6.0.35 • Java 1.6 JRE • Spring Framework 3.0.x • Spring WS • Groovy Web Services Framework
  • 25 This Is Not Bleeding Edge  Yet, it is still new to a lot of Spring users  January 2009: http://www.ibm.com/developerworks/java/library/j- groovierspring2/index.html  At the NVSS NFJS conference in Reston in April 2012, there were still sessions on Spring and Groovy integration with dynamic beans • NFJS Due in Raleigh, NC – August 24-26 • http://www.nofluffjuststuff.com/home/main
  • 26 Spring Bean Configuration Issues  Spring needs to be able to locate Groovy script content in order to load configurations. ERROR - ContextLoader.initWebApplicationContext(220) | Context initialization failed org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'employeeManagerService' defined in file [C:Toolsapache-tomcat-6.0.16w ebappsDynamicGroovyTutorialWEB-INFclassescontextservice_ws.xml]: Cannot resolve reference to bean 'employeeManagerServiceImpl' while setting bean property 'groovyImpl';
  • 27 Spring Bean Config With Content Proxy  Use a Content Proxy • The Content Proxy is deployed into a separate WAR and addressable via a URL (perhaps a RESTful web service) that would retrieve the dynamic Spring Bean configuration scripts • If the scripts are not found in script repository, the Content Proxy can load the scripts from a backup site or even from inside the application boundaries – Default Location. • The Content Proxy is a layer of indirection to better manage the link between your Spring application and the content management system that you may be using. • Content Proxies are also very helpful when scripts (or other content) are loaded from a CMS through an API that would not link directly to the Spring Bean configurator. • Provides a layer of indirection and intelligence for Cache Forward Architectures.
  • 28 Avoiding the Race Condition  Using a ServletContextListener allows you to specify how long the application will try to seek the ContentProxy (or other URLs) in other WARs that still may be spinning-up, before it lets the rest of the application load. • Reduces the “race-condition” introduced by URLs across multiple WARs
  • 29 ServletContextListener web.xml Configuration … <context-param> <description></description> <param-name>proxyUrl</param-name> <param-value>http://localhost/proxy.html</param-value> </context-param> <context-param> <description></description> <param-name>stopInterval</param-name> <param-value>60000</param-value> </context-param> <listener> <listener-class>com.ironworks.rjug.container.listener.ContentProxyListener</listener- class> </listener> …
  • 30 Inline Groovy – You can, but why would you? package com.ironworks.framework.scripting; import org.springframework.core.io.ResourceLoader; import org.springframework.scripting.ScriptSource; import org.springframework.scripting.support.*; public class URLScriptFactoryPostProcessor extends ScriptFactoryPostProcessor{ @Override protected ScriptSource convertToScriptSource(String beanName, String scriptSourceLocator, ResourceLoader resourceLoader) { if (scriptSourceLocator.startsWith(INLINE_SCRIPT_PREFIX)) { return new StaticScriptSource( scriptSourceLocator.substring(INLINE_SCRIPT_PREFIX.length()), beanName); } else if (scriptSourceLocator.startsWith("classpath")) { return new ResourceScriptSource(resourceLoader.getResource(scriptSourceLocator)); } else{ return new URLScriptSource(resourceLoader.getResource(scriptSourceLocator)); } } }
  • 31 Spring Bean Delegating Construct  Use a Spring Bean Delegate construct as an alternative to ServletContextListener and ContentProxy for handling potentially unreliable or dynamic URLs during application startup  Deploy JAXB version of services, but leave them open to replacement by Groovy, without needing another deployment • Still able to use Dynamic Groovy Edges, if JAXB version of service client requires updating <bean id="iw_service.getData" class="com.ironworks.vdc.framework.service.DelegatingService"> <property name="delegates"> <list> <ref bean="iw_service.getDataGroovy" /> <ref bean="iw_service.getDataJaxb" /> </list> </property> </bean>
  • 32 Spring Bean Delegate Construct Code public final class DelegatingService<REQUEST extends ServiceRequest, RESPONSE extends ServiceResponse> implements Service<REQUEST, RESPONSE> { private List<Service<REQUEST, RESPONSE>> delegates = new ArrayList<Service<REQUEST, RESPONSE>>(); @Override public RESPONSE execute(final REQUEST request) { for (Service<REQUEST, RESPONSE> service : delegates) { if (!(service instanceof AvailabilityStatus) || ((AvailabilityStatus) service).isAvailable()) { return service.execute(request); } } throw new UnsupportedOperationException( "No service implementation available to execute request"); } public void setDelegates(final List<Service<REQUEST, RESPONSE>> delegates) { this.delegates = delegates; } }
  • 33 AvailabilityStatus Interface public interface AvailabilityStatus { /** * Indicate whether this service instance is available. * * @return true if the service is available, false otherwise. */ boolean isAvailable(); }
  • 34 Spring Delegate Construct Code Explanation  DelegatingService is substituted into the class attribute of the bean definition  Possible “delegates” are added to the property list as objects  DelegatingService looks for viable services in the list of “delegates” • Could be services that implement the AvailabilityStatus interface, and are available (isAvailable()) • Beans that implement AvailabilityStatus, but are not available our by-passed – This allows Groovy substitutes to be enabled/disabled quickly • Could be services that do not implement AvailabilityStatus
  • 35 Some Tradeoffs for Groovy Approach  Groovy Web Clients • Pros • Easier to program with Getters and Setters approach to accessing and mutating XML objects for request and response • Markup Builder • XML Slurper • Cons • Lose auto-generation of object graphs via JAXB • Lose tighter integration to WSDL/XSD and makes it harder to track down changes. • It’s not for every change…more complex WSDLs, possibly with nested object architectures, would not be good candidates for Groovy re-write
  • 36 What About All Those Classes?  Almost every technical decision comes with a trade off…  When using dynamic Groovy with the approach we have outlined, obviously we are generating more “non-setup”, Groovy classes in the JVM. • Every time Spring detects a changed script, Groovy will parse it and load another class. • Depending on your JVM, GC settings and JVM memory settings, you could experience OOM issues with PermGen
  • 37 Does Your JVM Have a PermGen?  IBM: No PermGen, classes exist in the native C-Heap  Oracle/WebLogic JRockit: No PermGen, classes exist in the native C-Heap  Oracle/Sun Hotspot: Yes, PermGen in Java Heap • Prior to Java 5, the Hotspot JVM assumed that classes were forever • Then came CMS (Concurrent Mark Sweep) class unloading settings
  • 38 What Can We Do With PermGen?  Use GC settings for the CMS Collector • Java 1.6: -XX:+CMSClassUnloadingEnabled • Java 1.5: -XX:+CMSPermGenSweepingEnabled  Both require CMS GC enabled • This may not be suitable for all applications  Increase size (not really desirable) • Switch to 64 bit, if you have not already  Increase JVM bounce frequency (not really desirable) • Not really a good long term strategy
  • 39 Don’t Use PermGen.  Some of our projects use JRockit • While we do not see the same sensitivity to PermGen in our dynamic environment, occasionally we still see issues with class memory areas • Mostly in non-production environments where we tend to be more dynamic  Problem: JRockit does not exist beyond Java 7 • Oracle has announced that it is being merged with Hotspot in Java 7
  • 40 Use Java 7?  In Java 7, PermGen is supposed to go away over time. • This implies better class unloading and memory management • JRockit merges with Hotspot • When? • http://javaeesupportpatterns.blogspot.com/2011/10/java-7-features- permgen-removal.html • http://blogs.oracle.com/java/entry/java_7_questions_answers
  • 41 Groovy Spring MVC Controllers  Spring MVC controllers are another example of edge components that could benefit from more dynamic deployment and less rigid deployment structures • Remember: Just because it is marked as Groovy (*.groovy) does not preclude you from writing some Java <lang:groovy id="claimsController" script-source="${groovyClaimsController}" refresh-check- delay="1000" > <lang:property name="view" value="claim_summary"/> <lang:property name="detailView" value="claim_detail"/> <lang:property name="unsolicitedView" value="claim_unsolicited"/> <lang:property name="errorView" value="claim_error"/> <lang:property name="serviceDownView" value="claim_service_down"/> </lang:groovy>>
  • 42 Development Environment Configuration  Eclipse Helios SR2  Groovy-Eclipse plugin (The Codehaus) – 2.5.1.xx-2011027…  Groovy 1.8.6  Maven 2.0 (m2) Eclipse Integration – Sonatype, Inc • gmaven-plugin – 1.4.0  MongoDB 2.0.4
  • 43 Summary  We cannot control changes in layers beyond our influence or control  We can better prepare for change by using dynamic technologies in the edges of interface junctions. • Groovy blurs the once clear line between hard coding and properties configurations. • By using Spring to dynamically load Groovy scripts, stored outside of web applications, web applications can narrow regression boundaries by reducing the need to deploy EARs and WARs. • The Content Proxy provides a layer of indirection that smoothes the process for locating dynamic scripts an providing digestible data for the Spring Bean Configuration process. • Testing WSDL and XSD artifacts with XMLUnit provides early warning of web service changes and failures
  • 44 Special Thanks To:  Laith Al-Samir – ICF Ironworks • Spring WS application and Groovy WS Client  Mike Killeen – ICF Ironworks • Groovy/XMLUnit/JUnit WSDL/XSD tests  Vince Jorrand – ICF Ironworks • Spring Bean Delegates Construct