Anatomy of an Enterprise Integration Architecture
•Download as PPTX, PDF•
3 likes•1,033 views
Presentation delivered at INTEGRATE 2018 conference in London on 6th June
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Anatomy of an Enterprise Integration Architecture
Similar to Anatomy of an Enterprise Integration Architecture (20)
More from Daniel Toomey
More from Daniel Toomey (20)
Recently uploaded
Recently uploaded (20)
Anatomy of an Enterprise Integration Architecture
- 1. Dan Toomey Integration Architect and Microsoft AzureMVP Anatomy of an Enterprise Integration Architecture
- 2. Who am I? • Principal Consultant, Mexia • Microsoft Azure MVP • MCSE, MCT, MCPD, MCTS BizTalk & Azure • Pluralsight Author • www.mindovermessaging.com • @daniel2me
- 3. Who was I? • Principal Trombonist, Singapore Symphony Orchestra • USAF Band of Liberty • M.Mus, B.Mus - The Juilliard School • Founding member: – Paragon Ragtime Orchestra – Palisades Brass Quintet Photo by Brian Merritt
- 9. It’s about layers… … and the speed of change
- 10. How are we doing it? What are we doing it with? Where can we do it better?
- 12. Gartner’s Pace Layered Application Strategy
- 13. Gartner’s Pace Layers • New apps, ad hoc, new business reqs • Consumer-grade technologies Systems of Innovation • Unique processes / capabilities • Best of breed, SaaS, sometimes modules of a suite Systems of Differentiation • Core transaction processing • Critical master data Systems of Record
- 14. InnovationDifferentiationRecord Business Process Data / Information Change Control / Governance Business Engagement Planning Horizon
- 15. How are we integrating?
- 16. Systems of Innovation Systems of Differentiation Systems of Record E X T E R N A L S Y S T E M S
- 17. Systems of Innovation Systems of Differentiation Systems of Record Change Control API Design Testing Regime Rate of Change
- 18. What are we integrating with?
- 19. Systems of Innovation Systems of Differentiation Systems of Record Rate of Change
- 20. Technology Characteristics Technology Use When Considerations Product APIs • Product has granular APIs with a modern interface • APIs align with business needs • Vendor support is available + Tight integration with System of Record – Difficult or expensive to change or customize – May not suit business data model Web Service / REST APIs (IIS) • Exposing REST or SOAP interfaces • Implementing custom validation/security • Mapping to a canonical model + Inexpensive to host + Easy to consume – Development effort required API Management • Exposing APIs in the cloud • Implementing policy based security & access control • Leveraging caching/auditing/analytics/etc. + Customizable façade + Developer portal – Requires VNet Integration – No on-prem option – Expensive option if not using additional features Service Fabric • Aligning to a microservices architecture • Catering for multiple programming languages • Automated redundancy, load balancing, and no-downtime deployments are required + Can host anywhere + Supports containers – Development effort required BizTalk Server • OOTB adapters are available/suitable • Robust platform is required + BAM tracking available + Single platform integration – Expensive – Specialised dev skills required
- 21. Systems of Innovation Systems of Differentiation Systems of Record Rate of Change
- 22. Technology Characteristics Technology Use When Considerations Logic Apps • Business logic can be cloud hosted • Connecting with SaaS systems or other Azure services + Rapid development + 200+ built-in connectors! – No VNet support (until ISE becomes available) – No on-prem option (yet) Azure Functions • Need to run discrete pieces of stateless arbitrary code on demand • Integrating with other Azure services • Visual Studio development is preferred • Automated unit testing is a must + Good CI/CD support + VNet supported + Can run on-prem – Not as many connectors as Logic Apps Web/Mobile Apps • Cloud hosting is desired • Supporting multiple devices • Need a flexible programming model • Need exposure to external clients • Desire Blue/Green deployment slots + Good CI/CD support + Numerous deployment options + Azure Relay / VNet Integration supported – Not ideal in themselves for long running processes – Consider security layer for hybrid apps Service Fabric • Aligning to a microservices architecture • Catering for multiple programming languages • Automated redundancy, load balancing, and no-downtime deployments are required + Can host anywhere + Supports containers – Development effort required – Infrastructure investments (on-prem only) BizTalk Server • OOTB adapters are available/suitable • Robust platform is required • Wanting a reliable/durable workflow capability • Leveraging Business Rules Engine and/or BAM + Single platform for integration + Can be hosted on-prem or in Azure (IaaS) – Expensive – Specialised dev skills required
- 23. Systems of Innovation Systems of Differentiation Systems of Record Rate of Change
- 24. Technology Characteristics Technology Use When Considerations Microsoft Flow • Automating simple processes and tasks • Empowering business users to create their own integrations • Existing connectors are fit for purpose + Rapid development + Can be migrated easily to Logic Apps * Requires Office365 Power Apps • Developing in-house apps for devices • Leveraging built-in connectors + Easy integration with Flow / SharePoint / Dynamics 365 / Teams / etc. + Multi-platform * Requires Office365 Power BI • Need to quickly build custom charts and visuals • Integrate with multiple data sources * Relies on data sources Cognitive Services • Seeking advanced insights and analytic capabilities + Multiple services/APIs available (Vision, Knowledge, Language, Speech, Search) * Programming skills required Machine Learning • Seeking insights through predictive analytics * Data science skills required Bots • Seeking more human interaction with customers • Automating routine information retrieval or routing to appropriate support personnel * Programming skills required * Bots need to be trained well to function as expected
- 25. Systems of Innovation Systems of Differentiation Systems of Record Rate of Change
- 26. Technology Characteristics Technology Use When Considerations Event Grid • Building event-driven applications • Managing notifications • High scalability and throughput required • Handling events within Azure (or anywhere) + Resilient (retries up to 24 hours) + Push-push model + Easy to integrate * Small message size – No dead-lettering Event Hubs • Ingesting big data / streaming data • Replay / archiving is desired * Needs at least one downstream processor Relays • Need hybrid connectivity without firewall changes + Can use Hybrid Connection or WCF Relay On-Prem Data Gateway • Connecting Logic Apps to on-premises systems • Bridging SaaS applications to LOB systems + Good alternative to a VNet if using Logic Apps – Only supported by enterprise connectors Service Bus Queues • Decoupling sender / receiver processes • Each message should only be processed once • Data can flow through the cloud + Extremely resilient and full-featured – No on-prem option Service Bus Topics • Decoupling systems / processes via pub / sub • Supporting multiple subscribers to a message • Data can flow through the cloud + Extremely resilient and full-featured – No on-prem option BizTalk Server • Requiring robust pub/sub messaging • Leveraging BAM for tracking • Using OOTB adapters + Single platform for integration – Expensive – Specialised dev skills required
- 27. Where can we integrate better?
- 28. Consider how the application will be used
- 29. Make sure your System of Record layer is solid http://slideplayer.com/slide/3890847/
- 30. Limit customisation within the System of Record
- 31. Consider using canonical data models https://api-map.com/
- 32. Loosely couple inter-layer communications
- 33. Allow room for innovation
Editor's Notes
- These two pics set the theme for this presentation… can you guess what it is?
- (NEED TO RUN ANIMATION TO REVEAL WHAT’S INSIDE THE GRAPH) Let’s look at the different application landscapes we see in Enterprise Architectures today. Monolith – one app does it all More common – best of breed apps, need integration Less common – “Micro-apps” – lots of integration!!
- Not all apps are created equal (with regards to the pace of change) Picture vs Movie Concept of layers
- Layers for rate of change – how is integration applied?
- Enterprise agility. How we use MS Tech to move faster, accommodate different layers which move at different speeds
- Let’s get a better look at those layers
- Introduced in 2010 Application strategy framework Help address disconnect btwn Business & IT leaders TRANSFORMATION DOES NOT OCCUR WHILE WAITING IN LINE (Kent Weare)
- Record: Core Common processes – not unique within the industry Data: Core records – very high quality/audit, reporting Example Systems: Core ERP/SCM/CRM suites or legacy systems Differentiation: Unique / differentiating processes, rigorous/detailed, medium pace Data: Analytics & forecasting – often combining SOR data and other data Systems: Best of breed, SaaS, sometimes modules of a suite Innovation: Emerging processes, experiments/POCs, often manual/basic, “lite” Data: Increasingly external, advanced analytics/models, scenario planning Systems: Experimentation “sandbox”, Portals, content mgmt. & collaboration, “lite” app dev
- Now that we understand the application layers, what does this mean for integration?
- Entity/Task Services: Aligned to core SORs Abstraction to the main data entities Agnostic of the business processes that are composed of these ; reusable Business Services: Define the business logic of an organisation Perform data aggregation (split-aggregate), routing, filtering and often choreograph the invocation across multiple services which orchestrate the creates, updates, and deletes and aggregate query results into a common format across multiple systems. Innovation Services: Composed of services from both other layers and also external sources Functions is usually peripheral to core business processes
- How do the MS integration stack offerings serve the needs of each layer?
- What do we use to build the API layer over our Systems of Record?
- Why choose one over the other? Cloud hosted vs. on-prem Custom code vs. OOTB adapters How does it fit your architecture? Investment in an overall integration platform?
- THIS IS THE LAYER WHERE MOST OF THE INTEGRATION HAPPENS BizTalk – orchestration, BRE, BAM Logic Apps – workflow and connectivity to the cloud Web/Mobile Apps – customer engagement with hybrid integration Azure Functions – stateless code, OR durable functions for persistent workflow Service Fabric – microservices solution, cloud or on-prem
- Cloud hosted vs. on-prem Where are you comfortable developing? (VS vs Browser) What are you dev skills? (BizTalk vs C#) What are you uptime requirements? How does it fit your architecture? Investment in an overall integration platform?
- Flow – for connecting things, automating tasks Power Apps – for rapidly creating apps for devices (internal use) Cog Svcs / Bots – new ways to engage with the customer and harness information Insights – Power BI, Data Lake Analytics, ML
- What is your innovation goal? Connecting things Automating workflows Creating an in house app for devices Gaining insight through data analytics Engaging with your customers in a more natural way
- Events vs. Messages Service Bus – advanced messaging capabilities Hybrid connectivity BizTalk!
- Why choose one over the other? Eventing vs Messaging Streaming big data Criticality of the data being moved Establishing hybrid connectivity Investment in existing platform/architecture
- Let’s talk about some best practices
- EXAMPLES: Flow vs. Logic App, Event Grid vs. Service Bus (mission critical) Security (use APIM) Change (BizTalk vs. Logic App?) Data (PCI or PII?)
- Everything depends on a stable foundation
- Don’t buy a system with no extensibility Clear cut integration points ANECDOTE: Signature at BOQ
- ANECDOTE: Salesfore integration with Signature Success story from Tom: Canonical models All data input systems fed data in their own bespoke format and messaging/transport semantics. All had channel adapters that converted the bespoke messages to XML representation of the business’s canonical schema. The canonical schema was an extension to the industry standard model. No effort was given to trim this model to the current scope, but if new data had to be supported, it would be added according to the standard model first (if possible) and if not then a further custom extension would be created. All consumers consumed the business’s canonical model. Consumers did not perform schema validation. They simply expected the data points they needed to be where the model said they would be, and would raise errors accordingly. This allowed the model to be extended without having to maintain downstream systems.
- Because rate of change Own your space, reduce dependencies Oil for the friction between the layers
- “You have the ability to improve your business!” (Integrators can influence executives by showing them how MS can rapidly provide solutions, make things happen)