The document discusses Building an Integration Platform with Grails and Gradle. It proposes a plugin pattern for integration platforms, including elements like plugin identification, lifecycle management, dependencies, services, configuration, and metadata. It then describes how Moab Web Services (MWS) at Adaptive Computing implements an integration platform based on this pattern using Grails and Gradle. Key aspects of MWS's implementation are plugin identification via class names and annotations, lifecycle management via states and a control service, and dynamic configuration of plugins.

1. Building an Integration Platform
with Grails and Gradle
Brian Saville 18 Oct 2012
Adaptive Computing
1 © 2012 ADAPTIVE COMPUTING, INC.

2. Me
Brian Saville
bsaville@adaptivecomputing.com
Twitter: @bksaville
Plugins:
▪ External Config Reload
▪ Spring Security OAuth2 Provider
▪ New Doc
▪ Artefact Messaging
▪ Contributor on Others
2 © 2012 ADAPTIVE COMPUTING, INC.

3. Adaptive Computing Highlights
▪ Leaders in cloud and HPC software
▪ 10+ years developing Moab® technology
▪ Leadership team w/ $1.5B+ in exits
▪ 15 issued patents, 34 pending
▪ Backed by top-tier investors:
▪ Global Partnerships: HP, IBM, SGI, Cray, Microsoft, etc.
▪ Over 200 customers such as:
3 © 2012 ADAPTIVE COMPUTING, INC.

4. Moab Web Services
▪ What is it?
▪ Grails 2.0.3
▪ RESTful Web Services
▪ Business Logic
▪ Plugin Framework (Beta)
▪ What does it do?
▪ Wraps C Core Product
▪ Integrates “old world” non-standardized API with
“new world” APIs
4 © 2012 ADAPTIVE COMPUTING, INC.

5. Atypical Grails Deployment
▪ MWS Deployed on Client Servers
▪ WAR file
▪ Tomcat 6
▪ Oracle/Sun JDK 6
▪ MongoDB 2.0+
▪ Customization on Client Sites
5 © 2012 ADAPTIVE COMPUTING, INC.

6. Integration Platform
▪ Reasons
▪ Extensibility
▪ In-House Developers
▪ External Developers
▪ Dynamic Updates to Application Behavior
▪ Why Create Our Own?
▪ Consists of Plugin Framework (Grails)
and SDK (Gradle)
6 © 2012 ADAPTIVE COMPUTING, INC.

7. 7 © 2012 ADAPTIVE COMPUTING, INC.

8. Why Do We Care?
▪ Integration Platform Developers
1) Provides a standard pattern to follow
2) Identifies required and optional framework
components
3) Treads ground for new integration platforms
4) Identifies hurdles and potential problems
▪ Grails/Gradle Users
1) New advanced application of Grails and Gradle
2) Grails plugin framework can be evaluated with
the pattern
3) Gradle plugin framework can be evaluated with
the pattern
8 © 2012 ADAPTIVE COMPUTING, INC.

9. Integration Platform
Pattern
9 © 2012 ADAPTIVE COMPUTING, INC.

10. The Pattern
▪ Plugin Identification
▪ Lifecycle and State Management
▪ Dependency Management
▪ Extend Application
▪ Services
▪ Configuration Management
▪ Lifecycle Hooks
▪ Metadata
▪ Development (SDK)
10 © 2012 ADAPTIVE COMPUTING, INC.

11. Plugin Identification
▪ By Convention
▪ Class names
▪ Packaging (JAR, Java or Groovy file)
▪ By Interface(s)
▪ IPlugin
▪ Activator (OSGI)
▪ By Configuration
▪ MANIFEST.MF
11 © 2012 ADAPTIVE COMPUTING, INC.

12. Lifecycle and State Management
▪ Concept of State
▪ Stopped and started
▪ More states optional
▪ Control Over State
▪ How to stop, how to start?
▪ How to add plugin? (Removal not required)
12 © 2012 ADAPTIVE COMPUTING, INC.

13. Dependency Management
▪ Internal Dependencies
▪ Build process
▪ Consider IDE
▪ External Dependencies
▪ Fat JARs
▪ Dependency resolution when adding
13 © 2012 ADAPTIVE COMPUTING, INC.

14. Extend Application
▪ Different in Each Case
▪ Custom Services
▪ Web services
▪ Internal services
▪ Provide Resources
▪ Report or Persist Data
▪ …
14 © 2012 ADAPTIVE COMPUTING, INC.

15. Services
▪ Stateless Services
▪ Control services
▪ Logging
▪ Dependent Services
▪ Configuration
▪ Data persistence
15 © 2012 ADAPTIVE COMPUTING, INC.

16. Configuration Management
▪ Access Application Configuration
▪ (Optional) Access Private
Configuration
▪ Preferences
16 © 2012 ADAPTIVE COMPUTING, INC.

17. Lifecycle Hooks
▪ State Transition Hooks
▪ Starting and stopping
▪ Before, after, and/or during transition?
▪ Other Hooks
▪ Setup and configuration
▪ How Are They Defined?
▪ Convention
▪ Interfaces (Configuration)
17 © 2012 ADAPTIVE COMPUTING, INC.

18. Development (SDK)
▪ Dependency Management
▪ Build with internal dependencies
▪ Include external dependencies
▪ Testing
▪ Unit
▪ Integration
▪ Deployment
▪ (Optional) RAD Tool
18 © 2012 ADAPTIVE COMPUTING, INC.

19. Metadata
▪ By Convention
▪ By Configuration
▪ Types of Metadata
▪ Project/Plugin information
▪ Author information
▪ (Optional) Configuration constraints
▪ (Optional) Dependency information
19 © 2012 ADAPTIVE COMPUTING, INC.

20. Non-Essential Pattern Elements
▪ Security
▪ Deployment System/Central
Repository
▪ Inversion of Control (IoC) Container
▪ Data Persistence
▪ Individual to Plugin
▪ Application Aware
▪ Common Clients and Parsers
▪ XML/JSON
20 © 2012 ADAPTIVE COMPUTING, INC.

21. Non-Essential Pattern Elements Cont.
▪ Events – Built-In and Custom
▪ Decomposition of Logic
▪ Messaging (i18n)
▪ Startup/Shutdown Concerns
▪ Scheduled Jobs
▪ Inter-Project Dependencies
▪ Environment Handling
21 © 2012 ADAPTIVE COMPUTING, INC.

22. Grails Plugins
▪ What is Missing?
▪ Standardized configuration management
▪ Platform core is addressing this
▪ Framework is Mature
22 © 2012 ADAPTIVE COMPUTING, INC.

23. MWS Integration
Platform
23 © 2012 ADAPTIVE COMPUTING, INC.

24. Introduction
▪ Plugin Type – Plugin Definition Class
▪ One per Class
▪ Stored as Groovy or JAR files
▪ Internally is a Grails artefact
▪ Plugin Instance
▪ Multiple instances for a single plugin type
possible
▪ Internally is DB entry and Spring Bean
▪ Dynamic (Running Application)
24 © 2012 ADAPTIVE COMPUTING, INC.

25. Plugin Identification
▪ “Ends With” Class Names
▪ Plugin Types
▪ Translators
▪ Autowired Classes
▪ Spring Annotated Classes
▪ @Component
▪ @Autowired
▪ @Resource
▪ @Scope
25 © 2012 ADAPTIVE COMPUTING, INC.

26. Lifecycle and State Management
▪ States
▪ Started, Stopped, Paused
▪ Plugin Control Service
▪ Start, stop, pause, resume
▪ Adding Plugins
▪ User Interface
▪ REST API
26 © 2012 ADAPTIVE COMPUTING, INC.

27. Dependency Management
▪ SDK for Build Dependencies
▪ Grails JARs
▪ Spring JARs
▪ Support Fat JARs
▪ Dependencies Resolved Before Adding
27 © 2012 ADAPTIVE COMPUTING, INC.

28. Extend Application
▪ Reporting – Application Aware Data
Persistence
▪ Expose Custom Web Services
▪ Methods by convention
▪ Almost full REST API
▪ Unsecured annotation
▪ Routed by URL mappings
▪ Built-In Events by Convention
28 © 2012 ADAPTIVE COMPUTING, INC.

29. Services
▪ Logging (Similar to Grails)
▪ Data Persistence Services
▪ Control Service
▪ Individual Datastore Service
▪ SSL Service - SSLSocketFactory
▪ not-yet-commons-ssl with PEM files
▪ In memory keystore
▪ Internal REST Service
▪ Request and response with optional attributes
▪ Returns object with response, easy access
properties, parsed data
29 © 2012 ADAPTIVE COMPUTING, INC.

30. Configuration Management
▪ Application Configuration
▪ Grails application bean
▪ Plugin Private Configuration
▪ Metaclass injected “config” property
30 © 2012 ADAPTIVE COMPUTING, INC.

31. Lifecycle Hooks
▪ Implemented as Methods on Plugin
Types
▪ beforeStart, afterStart, beforeStop, afterStop
▪ configure
▪ If Not Implemented, Inject Empty
Methods
▪ Called from Control Service
31 © 2012 ADAPTIVE COMPUTING, INC.

32. Metadata
▪ Convention
▪ Plugin project compiled class included in JAR
▪ Ends with “Project”
▪ Configuration
▪ Properties on plugin project class
▪ =~ Grails Plugin Class
32 © 2012 ADAPTIVE COMPUTING, INC.

33. Metadata – Dynamic Configuration
Constraints
▪ Configuration is Dynamic Map
▪ Utilize Grails Constraints and
Conventions
▪ Redo min/max to remove value checking
▪ Add custom plugin constrained property (CP)
▪ Add custom plugin CP builder
▪ Add Custom Constraints
▪ Type (Class) – Infer if not specified
▪ Default value
▪ Required (nullable)
▪ Scriptable URL (exec, file, http)
▪ Customized validator with additional methods
33 © 2012 ADAPTIVE COMPUTING, INC.

34. i18n Messaging
▪ Message Property Files
▪ Included in JAR
▪ Loaded with individual message source per
project
▪ Message Method Injected on Plugins
▪ Used to Resolve Constraint Errors
▪ Not quite multi-lingual
34 © 2012 ADAPTIVE COMPUTING, INC.

35. Development (SDK)
▪ Commons JAR
▪ Interfaces, classes, constants, annotations
▪ Testing JAR – TestFor
▪ Gradle Project
▪ Dependencies
▪ Testing and Commons
▪ Creates fat JARs (lib and resolved deps)
▪ Additional build tasks
▪ Handles i18n property files
▪ Cobertura code coverage reports
▪ Unit testing with Spock
35 © 2012 ADAPTIVE COMPUTING, INC.

36. Development (SDK) - Commands
▪ Gradle
▪ create-project -Pproject-name=MyProject
▪ create-plugin -Pplugin-name=sample.MyPlugin
▪ test
▪ upload-test -Pmws.url=http://localhost/mws
▪ (Eventually) Python CLI Wrapper
▪ mplugin create-project MyProject
▪ mplugin create-plugin sample.MyPlugin
▪ mplugin test
▪ mplugin upload --url=http://localhost:8080/mws
36 © 2012 ADAPTIVE COMPUTING, INC.

37. Non-Essential Pattern Elements Status
▪ Security - Yes
▪ Central Repository – No
▪ IoC Container – Spring
▪ Data Persistence – Individual
Datastore and Application Aware
▪ Common Clients and Parsers – Client
Decided and Groovy
▪ Events – Built-In Only
▪ Decomposition – IoC with Conventions
and Annotations
37 © 2012 ADAPTIVE COMPUTING, INC.

38. Non-Essential Pattern Elements Status
Cont.
▪ Startup/Shutdown Concerns – Initial
Plugins
▪ Scheduled Jobs – Quartz
▪ Inter-Project Dependencies – No
▪ Environment Handling – Application
Configuration Parameter
38 © 2012 ADAPTIVE COMPUTING, INC.

39. Comparison to Grails Plugins
Concept MWS Plugins Grails Plugins
RAD Tool Partial X
Configuration X No Standard
Integration Tests X
Multiple X
Instances
Custom REST Partial X
Metadata Partial X
Languages Groovy Groovy & Java
Dynamic X
Lifecycle X X
i18n Partial X
39 © 2012 ADAPTIVE COMPUTING, INC.

40. Demo
40 © 2012 ADAPTIVE COMPUTING, INC.

41. Problems
Encountered
41 © 2012 ADAPTIVE COMPUTING, INC.

42. JVM ClassLoaders
▪ Separating ClassLoaders
▪ Locking JARs
▪ Fat JARs
▪ Unresolved: Reloading Typed
References
42 © 2012 ADAPTIVE COMPUTING, INC.

43. Language Support
▪ Groovy – Works Great
▪ Java – Interfaces and Metaclass Issues
43 © 2012 ADAPTIVE COMPUTING, INC.

44. Questions?
44 © 2012 ADAPTIVE COMPUTING, INC.