Download as PDF, PPTX
Uploaded byAndrew Coleman
IBM InterConnect 2015 - IIB Effective Application Development
The document discusses considerations for building effective connectivity solutions with IBM Integration Bus. It recommends (1) designing solutions that make use of built-in IIB features, (2) designing for performance and scalability from the start, and (3) designing solutions with administration and monitoring in mind. It also discusses techniques like using shared libraries and subflows, modeling message formats, and patterns to simplify development and improve reusability. Testing is emphasized as a critical part of the development process.
More Related Content
Similar to IBM InterConnect 2015 - IIB Effective Application Development
![[WSO2Con EU 2017] Building Next Generation Banking Middleware at ING: The Rol...](https://cdn.slidesharecdn.com/ss_thumbnails/ingmodelbank-final-171107154840-thumbnail.jpg?width=640&height=640&fit=bounds)
IBM InterConnect 2015 - IIB Effective Application Development
- 1. © 2015 IBMCorporation IBM Integration Bus Effective Application Development Andrew Coleman IBM Integration Bus Development Co-Chair, W3C XML Query Working Group
- 2. What should youconsider when building connectivity solutions? ▪ What technology is provided to help you? ▪ What design techniques should you know about? ▪ What should you try to avoid? Main points ▪ Design with IIB features ▪ Design for performance and scalability ▪ Design for administration ▪ Design with interfaces ▪ Design consistently ▪ Test Effective Application Development
- 3. 1. Design withIIB facilities
- 4. Tools of theTrade!
- 5. Create Your ConnectivitySolution Built-in nodes encapsulate transports, technologies and applications • Our intent is always to make the common tasks easy, and the rest possible! • Use the built-in nodes to reduce the amount of custom code required • This makes best use of the built-in facilities like activity trace and resource statistics
- 6. Integration Bus caninteract with a huge range of environments Transports and protocols • MQ / SOAP / HTTP / TCPIP / JMS Data repositories • File / Database / IMS Applications • SAP / Siebel / PeopleSoft / JDEdwards • CICS / IMS • .Net APIs • SCA • CORBA All input nodes can be configured to work with any message format. Getting data in and out of IIB
- 7. Transformation Options WMB andIntegration Bus have several transformation options: ▪ Mapping ▪ XSLT ▪ ESQL ▪ Java ▪ .NET Reflects the importance of transformation in connectivity solutions ▪ User-defined nodes supported for Java and C/C++ Every transformation option has strengths and weaknesses! ▪ Performance and scalability ▪ Backend integration ▪ Skill sets and learning curve ▪ Developer usability ▪ Portability and maintenance Use a transformation technology appropriate to the problem at hand!
- 8. Subflows are simplymessage flows that are invoked from another flow • Input and output nodes in the subflow become terminals in the main flow • Use subflows to break up large problems into smaller more manageable chunks Subflows are directly deployable to the runtime • Shared subflows deployed just once per execution group (or application) • No need to redeploy message flows after changes to shared routines are made • Redeployment of a subflow is automatically picked up by any consumers Subflows
- 9. Message Modelling Models areneeded for parsing, validation and transformation • Models avoid the need to write custom code to parse messages! Graphical mapper requires models to display the message structure • ESQL editor provides in line validation of code that navigates message trees
- 10. XML documents andmessages are modelled using XML schemas • XSDs and WSDLs are deployed directly to the runtime – no import required! Non XML data (both text and binary) is defined using DFDL schemas • Data Format Description Language standardised through the Open Grid Forum • DFDL schemas replace message set projects and the MRM message domain • Input nodes support DFDL alongside XMLNSC, JSON, MRM etc New model based tooling to design and test DFDL schema models • Test parsing, test data generation and parse-by-example all built into editor! • As with XSDs, DFDL schemas are also deployed directly to the runtime Message Modelling
- 12. 2. Design forPerformance and Scalability
- 13. Design for Performance Thereare several areas that incur a processing cost • Parsing and serialization of messages as they enter and exit the flow • Business logic such as transformations executed during message processing • Transformation cost includes general complexity, tree navigation and tree copying • External resources such as interactions with databases and queue managers Message flow design is critical to achieve high performance and scalability • For example using global units of work to achieve transactionality across a flow Performance is a large (and important) topic in its own right • There is a wealth of information and guidance available online!
- 14. Design for Scalability Largemessages can consume large amounts of memory/CPU There is an industry trend toward larger data sets • Already apparent in IIB deployments A flow with a large memory overhead can • slow down other flows by causing swapping • in the worst case, bring down the execution group Recommendations • Consider whether a message 10 times the current size would be able to be handled without a problem • Where appropriate, use large message handling techniques in your flows ( see Large Messaging sample )
- 15. Design for Scalability Loopsin a message flow are exactly like a recursive function call • So each iteration of the loop consumes some stack space Avoid loops unless you are certain that the number of iterations will not become large • Stack overflows will bring down the execution group along with all running flows! • There is usually an alternative design that avoids looping
- 16. 3. Design forAdministration
- 17. Applications, Libraries andServices Applications package end-to-end connectivity solutions • The concept of an application is shared between the toolkit and runtime • Applications are deployed and managed as a single unit of isolation Libraries package resources for reuse (flows, scripts, models) Resources in an application are not visible to anything else • Use applications to manage your solutions inside an execution group A Service is an Application with a well defined interface (WSDL)
- 18. Shared Libraries New sharedcontainer for reusable artefacts • Subflows, message models (XSD, DFDL) • ESQL, maps • NOT flows! Saves memory • Multiple applications can reference a single copy of a shared lib Separately deployable from the application Shared libraries have no running state • Cannot be started or stopped • No runtime threads assigned v8/9 static libraries continue to work as before 18 V10
- 19. Static Libraries versesShared Libraries Static Libraries 19 Shared Libraries (v10) App1 Lib1 Lib2 App2 Lib1 Lib2 App3 Lib1 Lib2 App1 App2ShLib1 ShLib2 App3
- 20. Configurable Services Use configurableservices to externalize deployment time properties • Allows deployment information to be configured by a system administrator • For example, credentials for the database nodes and FTP server and port • Ideal for moving solutions through staging environments (test to production)
- 21. Monitoring Monitoring tools availablevia Web UI and Integration Explorer • Statistics monitor resource usage across execution groups • Range of options to update and visualise the resource statistics • Activity Logging to focus on resource usage
- 22. Monitoring Monitoring also providedon individual message flows • Quick and easy way to identify hot spots in your solutions!
- 23. 4. Design withInterfaces
- 24. Database Services DBMS representssystem of record for key business entities • Customers, accounts, partners, products all stored in databases • Integration Bus tools discover and represent these key data • Integration services extends access to end-user applications New integration tools discover key database assets 1. Connect to DBMS (e.g. Oracle, DB2 ) 2. Discover source artefacts (tables, views, etc.) 3. Map CRUD operations to service interface 4. Save in canonical WSDL document – Custom bindings for SQL access 5. Re-use database WSDL in multiple scenarios Many uses for database service definition • WSDL contains both logical and physical database information • Drag and drop WSDL to automatically create SQL access methods • Create new integration service to exploit customized database access – End-user application consumes as regular (e.g.) web service Products PartnersCustomers Accounts
- 25. MQ Services Service definitionsallow you to make best use of available resources • Facilitates sharing of service information between users and systems • Allows users to understand interfaces (e.g. CustomerAddress.Update operation) • Provides a connector with which to exchange technical configuration (e.g. hostname) • Provides attachment points for associated policies (e.g. authorization) New framework enables discovery, cataloguing and re-use of services • Discovery connectors translate the service provider description to a common model • Interrogate IT systems for definition of technical assets – objects, functions and interaction points • User selects and refines definition of technical assets • Service definitions created and associated with technical assets • Discovered service definitions stored in embedded registry • Use catalogued services to configure integration solutions Initial implementation discovers and catalogs MQ service definitions 1. Discover queues from referenced queue manager endpoint 2. IB develops MQ service definition and stores in registry 3. Use service definitions to configure MQ nodes New and existing nodes will be updated over time • Completely aligned with runtime connector framework – Simple protocol points appropriate for style of interaction – Allows for simple development of custom connectors
- 26. REST APIs New containertype for implementing REST APIs Generates framework based on Swagger definition 26 V10
- 27.
- 28. Patterns for SimplifiedDevelopment Creates top-down, parameterized connectivity solutions Reduces common problems in flow development Establishes best practices for integration Reduces time-to-value for solution development Patterns are easily extended with regular IIB functionality
- 29. Pattern Generation Pattern generationcreates the production ready projects • Generated projects reflect the configuration choices of the pattern user • Configuration is saved so that the pattern can be re-generated if required
- 30. Built-In Patterns Integration Busprovides a core set of built-in patterns These implement a variety of common scenarios • Web service front end to a MQ based application • Processing data stored in a file and routing to one or more queues • Adding a proxy in front of a web service provider • Processing data from an SAP system and routing to MQ • Shredding messages and routing to one or more queues Patterns are selected based on client feedback and field experience This core set of patterns continues to grow with each release
- 31. Pattern Authoring Pattern becomeseven more useful when you can create your own! • Every organization has their own repeating connectivity patterns! • Pattern authoring is the name we give to this technology in WMB and Integration Bus We recommend you start with a working solution • One or more projects Pattern authoring is a design activity • It may be long lived • It is often not sequential Using patterns is a top-down activity driven by a requirement, but: • Authoring a working solution is (typically) a bottom-up activity • So pattern authoring bridges these two different approaches Patterns have their own development cycle • Pattern Authoring editor supports this design activity
- 32. Create Your WorkingSolution No change at all - design your solutions as you do today • Pattern authoring does not change the tools you use to create solutions • The key to a good pattern is to create a good working solution!
- 33. Design Your Pattern Straightforwardto create patterns using the Pattern Authoring editor! • Design the user interface which is presented to your pattern users • Easy to add branding - style sheets, images and other files as required
- 34.
- 35. The Integration APIis our Java application development API • Packaged along with the administration API in ConfigManagerProxy.jar • Provides a complete Java API to manipulate message flows (and subflows) The Integration API can be used in standalone applications or in patterns • Wide variety of applications are now possible such as business data mashups • Easy to implement mass creation of message flows during migration projects Integration API
- 36. Pattern Communities Pattern authoringcreates patterns whose value multiplies as they are shared and used by a community of developers Public and private communities are equally interesting! A community needs to offer more than just a repository of assets! • Space for content such as patterns and subflow nodes • Forums to host discussions and threaded conversations • Categorisation is important as the community grows! (taxonomies) • Essential administrative functions such as user management • News articles and broadcasts keep the site fresh There are literally dozens of content management systems available On the commercial side we have tried Lotus Connections and Rational Asset Manager Open source projects such as Drupal and Joomla are compelling!
- 37. Creating a CommunitySpace
- 38.
- 39. Flow Exerciser 40 Click tostart recording
- 40.
- 41. Flow Exerciser 42 Now recording– the flow is greyed out (read-only)
- 42.
- 43.
- 44.
- 45. Flow Exerciser 46 Path throughthe flow is highlighted Click to see message content Can re-send the recorded messages (logical tree assembly)
- 46. Summary • Use thefeatures of WMB & IIB to their full extent! • Message modelling, transformation options, applications and libraries • Capabilities link in with activity trace, record edit replay, and more • Libraries and subflows make for simpler designs • Deployable subflows and libraries extend the benefits to the runtime • Consider production and performance early on in your design • In particular, applications and configurable services • Your system administrator will thank you for this! • Patterns address time-to-value for your solutions • It takes just a few minutes to create a pattern! • Why not create a patterns community in your organisation? • Developer Edition • Free edition of IB for use in evaluation, dev, test and production usage • Fully functional: all nodes available and no time limitations • Throughput rate limited to 1TPS per integration flow 52
- 47. Thank You Your Feedbackis Important! Access the InterConnect 2015 Conference CONNECT Attendee Portal to complete your session surveys from your smartphone, laptop or conference kiosk.