When to use What? A look at choosing Integration Technology
Upcoming SlideShare
Loading in...5
×
 

When to use What? A look at choosing Integration Technology

on

  • 907 views

There’s never been more options -- or more confusion -- about which technologies to use when connecting systems? When is BizTalk Server the right choice? Should I still use WCF? Is the Windows Azure ...

There’s never been more options -- or more confusion -- about which technologies to use when connecting systems? When is BizTalk Server the right choice? Should I still use WCF? Is the Windows Azure Service Bus ready for production scenarios? In this session, we’ll first do an overview of each core technology in the integration portfolio. Then we’ll review a simple decision framework for down-selecting your choices. Finally, we’ll have an interactive discussion (and demonstration) of real-life scenarios and which technology is the right fit.

Statistics

Views

Total Views
907
Views on SlideShare
510
Embed Views
397

Actions

Likes
0
Downloads
20
Comments
0

2 Embeds 397

http://www.biztalk360.com 394
https://twitter.com 3

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Who am i
  • Where are we / problem
  • Previous book (cover) <br /> Technology count: WCF, WF, AppFabric, BizTalk, ESB, SSIS, SSSB, Sync Framework, Azure SB Relay, StreamInsight <br /> Technology count TODAY: WCF, WF, AppFabric, BizTalk, ESB, SSIS, SSSB, Sync Framework, Azure SB relay, Azure SB Brokered, Azure VMs, Azure Notification Hubs, BizTalk Services, StreamInsight <br /> LET’S TALK THROUGH SOME TRENDS THAT EMERGED SINCE THIS BOOK
  • Source: http://www.flickr.com/photos/davenielsen/5718247676/sizes/l/ <br /> <br /> Cloud computing <br /> Change to the traditional computing paradigm <br /> Host matters less; elastic, on-demand <br /> Accepted part of an organization’s app portfolio <br /> New types of applications and scenarios <br /> Widely distributed <br /> Mix of hosting providers <br /> Different integration scenarios <br /> Cloud to cloud, ground to cloud, cloud to ground
  • Source: http://www.flickr.com/photos/mattandbeccasteudelwedding/9067696311/sizes/l/ <br /> <br /> Micro Services <br /> Something championed by ThoughtWorks a while back, and now something Netflix (and others) do a lot of <br /> Mini applications/services <br /> No heavy, complex services <br /> Very specific services that are easy to consume and weave together <br /> Think of cloud providers who have distinct services for provisioning services, autoscale, etc
  • Source: http://www.flickr.com/photos/24752578@N02/5589897831/sizes/l/ <br /> <br /> Lightweight Integration <br /> It’s not just about a powerful, centralized integration server <br /> That’s of course always been the case, but new attention paid to simplified messaging <br /> Something between a basic queue and a full blown ESB <br /> New tools, new way to connect lightweight services together
  • Source: http://www.flickr.com/photos/anekoho/9638930676/sizes/h/ <br /> <br /> NoSQL endpoints <br /> New types of endpoints, often outside your direct control <br /> Relational databases are less important now than ever <br /> Still a place for them, but major companies are finding they don’t need them <br /> We’ve replaced our SQL backend with one <br /> To the integrator, this adds new scenarios
  • Source: http://www.flickr.com/photos/peternijenhuis/6959653638/sizes/h/ <br /> <br /> Automation <br /> DevOps and the cultural movement to unprecedented synergy between Dev and Ops teams <br /> Automated builds, continuous integration, continuous deployment <br /> Possible to get rid of snowflake servers, and move to immutable servers
  • Source: http://www.flickr.com/photos/suckamc/5907306188/sizes/l/ <br /> <br /> We want to be at a place where we have the simplest toolset possible, but one that meets our needs <br /> This continued expansion of available integration services is logical, when considering these trends <br /> There are more and more tools for specific scenarios (micro-integration services), and fewer monolithic tools that serve EVERY purpose
  • How do we get there? <br /> Walk through a decision framework <br /> Talk through the available technologies <br /> Apply these principles <br />
  • What’s a decision framework? <br /> Way to analyze a situation and identify the right fit <br /> There cannot be a massive decision matrix that drops the correct solution in your lap <br /> Instead, there are multiple dimensions to consider, and those may be unique to each organization and situation <br /> Steps <br /> First, let’s discuss the sources that describe our situation <br /> Then, we’ll talk about the 4 dimensions to discuss
  • Input into a decision … <br /> Functional requirements <br /> What is the application supposed to do? <br /> Business requirements
  • Input into a decision … <br /> Non functional requirements <br /> Describe what the system itself needs to do to support the functional requirements <br /> security, availability, maintainability, compliance, exception handling, etc <br /> You can argue if these are actually functional requirements, and in some cases they are
  • Input into a decision … <br /> Derived requirements <br /> Ones that aren’t explicit stated, but can be derived from what’s needed <br /> Based on a functional requirement to provide a certain type of data, I might derive a requirement to persist data in a warehouse for long range analytics
  • Input into a decision … <br /> Organization direction/strategy <br /> This is the least transferrable! <br /> What are the company goals, investments? <br /> These are often fixed, and non-negotiable <br /> <br /> <br /> NOW, LET’S TALK ABOUT THE DECISION CRITERIA ITSELF
  • Solution design – design patterns required <br /> Data volumes, data set size, guaranteed delivery, data sources/destinations, latency, sync/async, data processing (workflow, quality operations), SSO, failure handling <br /> MODERN UPDATES … (reliability of endpoint, location of endpoint, constraints imposed by source/destination, circuit breakers) <br />
  • Solution development – what it takes to build the solution <br /> Developer availability, learning curve, developer setup, IDE and tool integration, support system, source control integration, automated builds <br /> MODERN UPDATES … (works in cloud, web-based IDEs, continuous integration/delivery support) <br /> <br />
  • Solution operations – what it takes to maintain the solution <br /> Event logging, tuning ability, handle bursts and failures, backup/restore routines, support tools, versioning capabilities, scriptability <br /> MODERN UPDATES … (immutable runtimes, rapid releases/rollbacks, works in distributed fashion) <br /> <br />
  • Organizational considerations – enterprise standards and org direction <br /> Long term solution, vendor support, uses existing software investment, limited budget impact, build vs buy compatibility, risk tolerance, speed to market <br /> MODERN UPDATES … (OSS, commodity skill sets, easy/no upgrade) <br /> <br />
  • Product: BizTalk Server <br /> Tag Line: Comprehensive application integration <br /> Most Recent Release: 4/13 <br /> Capabilities: Durable messaging, orchestration, technology adapters, trading partner management, BAM, BRE, etc <br /> Core Use Cases: Async messaging-based integration between applications; stateful, long running workflow processes that coordinate interactions between systems <br /> Buy-Hold-Sell: Buy
  • Product: ESB Toolkit <br /> Tag Line: “Dynamic messaging” <br /> Most Recent Release: 4/13 with BizTalk 2013 <br /> Capabilities: Dynamic routing, services for mapping and pipelines <br /> Core Use Cases: Diverse message paths; message flow modeling, supremely loose coupling, centralized exception management <br /> Buy-Hold-Sell: Soft Buy <br />
  • Product: Windows Server AppFabric <br /> Tag Line: Advanced service hosting <br /> Most Recent Release: 3/12 <br /> Capabilities: Caching, advanced WCF and WF hosting; troubleshooting; persistent workflows <br /> Core Use Cases: WCF service management, hosting long running WF services <br /> Buy-Hold-Sell: Hold <br />
  • Product: Service Bus for Windows <br /> Tag Line: On-premises durable messaging <br /> Most Recent Release: 10/13 <br /> Capabilities: Running messaging aspects of Azure Service Bus on-premises; topics and queues; manage with new Windows Azure Pack on Windows <br /> Core Use Cases: Start work on premises and move to cloud; basic pub-sub messaging; <br /> Buy-Hold-Sell: Hold <br />
  • Product: Workflow Manager <br /> Tag Line: On-premises workflow host <br /> Most Recent Release: 3/13 (CU) <br /> Capabilities: multi-tenancy, tracking and monitoring, instance management, versioning support <br /> Core Use Cases: Host and manage WF apps; <br /> Buy-Hold-Sell: Hold <br /> <br /> http://msdn.microsoft.com/library/jj193528 <br />
  • Product: WCF <br /> Tag Line: Sophisticated service framework <br /> Most Recent Release: <br /> Capabilities: Contract first service design for REST and SOAP services; configuration-centric service model that is extreme (overly?) extensible <br /> Core Use Cases: Building SOAP based web services; Sync or async <br /> Buy-Hold-Sell: Soft Buy <br />
  • Product: ASP.NET Web API <br /> Tag Line: Modern web services <br /> Most Recent Release: 1/14 <br /> Capabilities: REST/HTTP service design; extensible model; Odata support <br /> Core Use Cases: web service design <br /> Buy-Hold-Sell: Buy <br />
  • Product: Windows Workflow <br /> Tag Line: Lightweight, durable processes <br /> Most Recent Release: <br /> Capabilities: declarative models that describe a process; built-in and custom activities; durable workflow; service endpoints <br /> Core Use Cases: long running workflows; exposing business processes as services <br /> Buy-Hold-Sell: Soft Buy <br />
  • Product: SSIS (also Master Data Services?) <br /> Tag Line: Bulk data transfer <br /> Most Recent Release: SQL 2014 (no big changes) <br /> Capabilities: Data integration and data transformation; lots of data sources (files, FTP, HTTP, SMTP, RDBMS); bulk movement <br /> Core Use Cases: <br /> Buy-Hold-Sell: Buy <br />
  • Product: StreamInsight <br /> Tag Line: Event stream processing <br /> Most Recent Release: 6/12 <br /> Capabilities: high volume pattern matching against event streams; adapter model; Iobservable, Ienumerable support; multiple hosting options; LINQ queries <br /> Core Use Cases: situational awareness from data streams <br /> Buy-Hold-Sell: Soft Hold <br />
  • Product: Windows Azure Virtual Machines – BizTalk Image <br /> Tag Line: Full BizTalk in the Cloud <br /> Most Recent Release: June 2013 <br /> Capabilities: Run full BizTalk Server in the cloud; no HA with SQL clustering <br /> Core Use Cases: Dev/test scenarios <br /> Buy-Hold-Sell: Hold <br />
  • Product: Windows Azure Service Bus Relays <br /> Tag Line: Services that span boundaries <br /> Most Recent Release: <br /> Capabilities: Traverse networks and securely expose sync and async WCF services <br /> Core Use Cases: partner integration, mobile device integration, remote worker tools, cloud to cloud integration <br /> Buy-Hold-Sell: Strong Buy <br />
  • Product: Service Bus Brokered Messaging <br /> Tag Line: Durable messaging in the cloud <br /> Most Recent Release: <br /> Capabilities: Durable messaging through queues (single reader), and topics (broadcast with subscriptions) <br /> Core Use Cases: partner integration, mobile integration, cloud to cloud integration <br /> Buy-Hold-Sell: Buy <br />
  • Product: Service Bus Notification Hubs <br /> Tag Line: Scalable message distribution <br /> Most Recent Release: <br /> Capabilities: Broadcast messages to mobile devices and Windows 8; <br /> Core Use Cases: <br /> Buy-Hold-Sell: Buy <br />
  • Product: Windows Azure BizTalk Services <br /> Tag Line: Lightweight cloud message broker <br /> Most Recent Release: <br /> Capabilities: Link applications via cloud-hosted broker; source/destination adapters; strong data mapper <br /> Core Use Cases: connecting partners, cloud apps <br /> Buy-Hold-Sell: Strong Hold <br />
  • MANY other products that work well in a Windows and cloud world <br /> Brokers like Apache Kafka <br /> Queue technologies like RabbitMQ, or Amazon SQS <br /> Cloud-based integration tools like SnapLogic and Informatica Cloud <br /> SaaS-integrated services for message publication and workflow <br /> E.g. Dynamics CRM Online or Salesforce.com
  • Options … <br /> Use messaging and use BizTalk to send messages to vendor endpoints <br /> PRO: Good tracking, chance for enrichment before sending out; guaranteed delivery <br /> CON: hard to quickly change endpoints; puts responsibility on internal team to manage/scale; <br /> Use messaging and use BizTalk to send to Topics, local DB, and notification Hubs <br /> Use messaging and send loans directly to SB topics (and internal DB) from WF <br /> PRO <br /> CON <br /> Send loans to SB topic and one listener is internal DB <br /> PRO: Simple, offload responsibility for delivery <br /> CON: No coordination; chance of race condition; no notification for vendors of new messages <br /> Send loans to SB topic, one listener is internal DB, and also use Notification Hubs for alerting <br /> Provide REST-ful query API (exposed via SB) on loan DB and use notification hubs to send messages to providers to query and provide their best offer <br /> PRO: Simple; no replication; easy to onboard new vendors; offload availability to 3rd party; durable; <br /> CON: Not event based; need to chain together way to notify vendors; no orchestration or tracking; limited visibility into vendor processing <br />
  • Let’s consider the option that leverages the Service Bus for all this <br /> Send a message to Notification Hubs so very easy to broadcast info and push alerts <br /> Use Topics for scalable, durable messaging <br /> Subscribe ourselves to update internal systems

When to use What? A look at choosing Integration Technology When to use What? A look at choosing Integration Technology Presentation Transcript

  • brought to you by When to Use What? A Look at Choosing Integration Technology RICHARD SEROTER @rseroter
  • Richard Seroter Director of Product Mgmt at CenturyLink Cloud Microsoft MVP for Integration Technical Trainer at Pluralsight Cloud Editor for InfoQ.com 3-time Book Author
  • WHERE ARE WE?
  • Integration Technologies Covered: 10 Integration Technologies TODAY: 14+
  • TRENDS
  • CLOUD COMPUTING
  • MICRO SERVICES
  • LIGHTWEIGHT INTEGRATION
  • NOSQL ENDPOINTS
  • AUTOMATION
  • DESIRED END STATE?
  • MAINTAINABLE SIMPLICITY
  • GETTING THERE
  • DECISION FRAMEWORK
  • FUNCTIONAL REQUIREMENTS NON FUNCTIONAL REQUIREMENTS DERIVED REQUIREMENTS ORGANIZATIONAL STRATEGY
  • FUNCTIONAL REQUIREMENTS NON FUNCTIONAL REQUIREMENTS DERIVED REQUIREMENTS ORGANIZATIONAL STRATEGY
  • FUNCTIONAL REQUIREMENTS NON FUNCTIONAL REQUIREMENTS DERIVED REQUIREMENTS ORGANIZATIONAL STRATEGY
  • FUNCTIONAL REQUIREMENTS NON FUNCTIONAL REQUIREMENTS DERIVED REQUIREMENTS ORGANIZATIONAL STRATEGY
  • Data volumes Guaranteed delivery Data sources / destinations Latency Authentication / authorization Failure handling Endpoint location, reliability, constraints Circuit breakers Design DevOperations Org Strategy
  • Design Dev Operations Org Strategy Developer availability Learning curve Developer setup IDE and tool integration Support system Automated build support Deploy to cloud Web-based dev tools Continuous integration / delivery
  • Design Dev Operations Org Strategy Event logging Performance tuning knobs Burst and failure handling Backup / restore tools Scriptability Immutable servers Rapid release / rollback Distributed deployments
  • Design DevOperations Org Strategy Long term solution fit Vendor support available Uses existing (software) investments Budget impact Build vs. buy philosophy Risk tolerance Speed to market OSS components Commodity skill sets Simple upgrades
  • PRODUCT EVALUATION
  • BIZTALK SERVER “Comprehensive application integration” BUYHOLDSELL
  • ESBTOOLKIT “Dynamic BizTalk messaging” BUYHOLDSELL
  • WINDOWS SERVER APPFABRIC “Advanced service hosting” BUYHOLDSELL
  • SERVICE BUS FOR WINDOWS “On-premises durable messaging” BUYHOLDSELL
  • “On-premises, multi- tenant Workflow host” BUYHOLDSELL WORKFLOW MANAGER
  • WCF “Sophisticated service framework” BUYHOLDSELL
  • ASP.NET WEB API “Modern web services” BUYHOLDSELL
  • WINDOWS WORKFLOW “Lightweight long- running processes” BUYHOLDSELL
  • SSIS “Bulk transfer between data sources” BUYHOLDSELL
  • STREAM INSIGHT “Complex event stream processing” BUYHOLDSELL
  • WINDOWS AZURE VM “Full BizTalk Server in the cloud” BUYHOLDSELL
  • SERVICE BUS RELAY “Services that span network boundaries” BUYHOLDSELL
  • SERVICE BUS BROKERED MESSAGING “Durable messaging in the cloud” BUYHOLDSELL
  • “Scalable mobile message distribution” BUYHOLDSELL SERVICE BUS NOTIFICATION HUBS
  • BIZTALK SERVICES “Lightweight cloud message broker” BUYHOLDSELL
  • APACHE KAFKA AMAZON SQS RABBIT MQ MULE SNAPLOGIC INFORMATICA
  • EXAMPLE SCENARIO
  • QUICKQUALIFY “Get multiple loan quotes within minutes!” BUSINESS NEED COMPANY OVERVIEW • Accept loan parameters from web customers • Store loan request in on-premises system • Reliably provide loan request to volatile list of external providers • Track responses and show the customer choices • High uptime, low latency integration • Strong development team • Over-extended Ops team • Decentralized org, local autonomy • Microsoft-friendly organization
  • QUICKQUALIFY Customer website Loan DB ESB Services ETL CRM System Loan VendorLoan Vendor Loan Vendor CLOUD
  • DEMONSTRATION
  • CONCLUSION