brought to you by
When to Use What?
A Look at Choosing Integration Technology
RICHARD SEROTER
@rseroter
Richard Seroter
Director of Product Mgmt at CenturyLink Cloud
Microsoft MVP for Integration
Technical Trainer at Pluralsig...
WHERE ARE WE?
Integration Technologies Covered:
10
Integration Technologies TODAY:
14+
TRENDS
CLOUD COMPUTING
MICRO SERVICES
LIGHTWEIGHT
INTEGRATION
NOSQL ENDPOINTS
AUTOMATION
DESIRED END STATE?
MAINTAINABLE
SIMPLICITY
GETTING THERE
DECISION FRAMEWORK
FUNCTIONAL REQUIREMENTS
NON FUNCTIONAL REQUIREMENTS
DERIVED REQUIREMENTS
ORGANIZATIONAL STRATEGY
FUNCTIONAL REQUIREMENTS
NON FUNCTIONAL REQUIREMENTS
DERIVED REQUIREMENTS
ORGANIZATIONAL STRATEGY
FUNCTIONAL REQUIREMENTS
NON FUNCTIONAL REQUIREMENTS
DERIVED REQUIREMENTS
ORGANIZATIONAL STRATEGY
FUNCTIONAL REQUIREMENTS
NON FUNCTIONAL REQUIREMENTS
DERIVED REQUIREMENTS
ORGANIZATIONAL STRATEGY
Data volumes
Guaranteed delivery
Data sources / destinations
Latency
Authentication / authorization
Failure handling
Endpo...
Design
Dev
Operations
Org
Strategy
Developer availability
Learning curve
Developer setup
IDE and tool integration
Support ...
Design
Dev
Operations
Org
Strategy
Event logging
Performance tuning knobs
Burst and failure handling
Backup / restore tool...
Design
DevOperations
Org
Strategy
Long term solution fit
Vendor support available
Uses existing (software) investments
Bud...
PRODUCT EVALUATION
BIZTALK
SERVER
“Comprehensive
application integration”
BUYHOLDSELL
ESBTOOLKIT
“Dynamic BizTalk
messaging”
BUYHOLDSELL
WINDOWS
SERVER
APPFABRIC
“Advanced service
hosting”
BUYHOLDSELL
SERVICE
BUS FOR
WINDOWS
“On-premises durable
messaging”
BUYHOLDSELL
“On-premises, multi-
tenant Workflow host”
BUYHOLDSELL
WORKFLOW
MANAGER
WCF
“Sophisticated service
framework”
BUYHOLDSELL
ASP.NET
WEB API
“Modern web services”
BUYHOLDSELL
WINDOWS
WORKFLOW
“Lightweight long-
running processes”
BUYHOLDSELL
SSIS
“Bulk transfer between
data sources”
BUYHOLDSELL
STREAM
INSIGHT
“Complex event stream
processing”
BUYHOLDSELL
WINDOWS
AZURE VM
“Full BizTalk Server in
the cloud”
BUYHOLDSELL
SERVICE BUS
RELAY
“Services that span
network boundaries”
BUYHOLDSELL
SERVICE BUS
BROKERED
MESSAGING
“Durable messaging in
the cloud”
BUYHOLDSELL
“Scalable mobile
message distribution”
BUYHOLDSELL
SERVICE BUS
NOTIFICATION HUBS
BIZTALK
SERVICES
“Lightweight cloud
message broker”
BUYHOLDSELL
APACHE KAFKA
AMAZON SQS
RABBIT MQ
MULE
SNAPLOGIC
INFORMATICA
EXAMPLE SCENARIO
QUICKQUALIFY
“Get multiple loan quotes within minutes!”
BUSINESS NEED COMPANY OVERVIEW
• Accept loan parameters from web c...
QUICKQUALIFY
Customer
website
Loan DB
ESB Services ETL
CRM System
Loan VendorLoan Vendor
Loan Vendor
CLOUD
DEMONSTRATION
CONCLUSION
Upcoming SlideShare
Loading in...5
×

When to use What? A look at choosing Integration Technology

1,306

Published on

There’s never been more options -- or more confusion -- about which technologies to use when connecting systems? When is BizTalk Server the right choice? Should I still use WCF? Is the Windows Azure Service Bus ready for production scenarios? In this session, we’ll first do an overview of each core technology in the integration portfolio. Then we’ll review a simple decision framework for down-selecting your choices. Finally, we’ll have an interactive discussion (and demonstration) of real-life scenarios and which technology is the right fit.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,306
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
30
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Who am i
  • Where are we / problem
  • Previous book (cover)
    Technology count: WCF, WF, AppFabric, BizTalk, ESB, SSIS, SSSB, Sync Framework, Azure SB Relay, StreamInsight
    Technology count TODAY: WCF, WF, AppFabric, BizTalk, ESB, SSIS, SSSB, Sync Framework, Azure SB relay, Azure SB Brokered, Azure VMs, Azure Notification Hubs, BizTalk Services, StreamInsight
    LET’S TALK THROUGH SOME TRENDS THAT EMERGED SINCE THIS BOOK
  • Source: http://www.flickr.com/photos/davenielsen/5718247676/sizes/l/

    Cloud computing
    Change to the traditional computing paradigm
    Host matters less; elastic, on-demand
    Accepted part of an organization’s app portfolio
    New types of applications and scenarios
    Widely distributed
    Mix of hosting providers
    Different integration scenarios
    Cloud to cloud, ground to cloud, cloud to ground
  • Source: http://www.flickr.com/photos/mattandbeccasteudelwedding/9067696311/sizes/l/

    Micro Services
    Something championed by ThoughtWorks a while back, and now something Netflix (and others) do a lot of
    Mini applications/services
    No heavy, complex services
    Very specific services that are easy to consume and weave together
    Think of cloud providers who have distinct services for provisioning services, autoscale, etc
  • Source: http://www.flickr.com/photos/24752578@N02/5589897831/sizes/l/

    Lightweight Integration
    It’s not just about a powerful, centralized integration server
    That’s of course always been the case, but new attention paid to simplified messaging
    Something between a basic queue and a full blown ESB
    New tools, new way to connect lightweight services together
  • Source: http://www.flickr.com/photos/anekoho/9638930676/sizes/h/

    NoSQL endpoints
    New types of endpoints, often outside your direct control
    Relational databases are less important now than ever
    Still a place for them, but major companies are finding they don’t need them
    We’ve replaced our SQL backend with one
    To the integrator, this adds new scenarios
  • Source: http://www.flickr.com/photos/peternijenhuis/6959653638/sizes/h/

    Automation
    DevOps and the cultural movement to unprecedented synergy between Dev and Ops teams
    Automated builds, continuous integration, continuous deployment
    Possible to get rid of snowflake servers, and move to immutable servers
  • Source: http://www.flickr.com/photos/suckamc/5907306188/sizes/l/

    We want to be at a place where we have the simplest toolset possible, but one that meets our needs
    This continued expansion of available integration services is logical, when considering these trends
    There are more and more tools for specific scenarios (micro-integration services), and fewer monolithic tools that serve EVERY purpose
  • How do we get there?
    Walk through a decision framework
    Talk through the available technologies
    Apply these principles
  • What’s a decision framework?
    Way to analyze a situation and identify the right fit
    There cannot be a massive decision matrix that drops the correct solution in your lap
    Instead, there are multiple dimensions to consider, and those may be unique to each organization and situation
    Steps
    First, let’s discuss the sources that describe our situation
    Then, we’ll talk about the 4 dimensions to discuss
  • Input into a decision …
    Functional requirements
    What is the application supposed to do?
    Business requirements
  • Input into a decision …
    Non functional requirements
    Describe what the system itself needs to do to support the functional requirements
    security, availability, maintainability, compliance, exception handling, etc
    You can argue if these are actually functional requirements, and in some cases they are
  • Input into a decision …
    Derived requirements
    Ones that aren’t explicit stated, but can be derived from what’s needed
    Based on a functional requirement to provide a certain type of data, I might derive a requirement to persist data in a warehouse for long range analytics
  • Input into a decision …
    Organization direction/strategy
    This is the least transferrable!
    What are the company goals, investments?
    These are often fixed, and non-negotiable


    NOW, LET’S TALK ABOUT THE DECISION CRITERIA ITSELF
  • Solution design – design patterns required
    Data volumes, data set size, guaranteed delivery, data sources/destinations, latency, sync/async, data processing (workflow, quality operations), SSO, failure handling
    MODERN UPDATES … (reliability of endpoint, location of endpoint, constraints imposed by source/destination, circuit breakers)
  • Solution development – what it takes to build the solution
    Developer availability, learning curve, developer setup, IDE and tool integration, support system, source control integration, automated builds
    MODERN UPDATES … (works in cloud, web-based IDEs, continuous integration/delivery support)

  • Solution operations – what it takes to maintain the solution
    Event logging, tuning ability, handle bursts and failures, backup/restore routines, support tools, versioning capabilities, scriptability
    MODERN UPDATES … (immutable runtimes, rapid releases/rollbacks, works in distributed fashion)

  • Organizational considerations – enterprise standards and org direction
    Long term solution, vendor support, uses existing software investment, limited budget impact, build vs buy compatibility, risk tolerance, speed to market
    MODERN UPDATES … (OSS, commodity skill sets, easy/no upgrade)

  • Product: BizTalk Server
    Tag Line: Comprehensive application integration
    Most Recent Release: 4/13
    Capabilities: Durable messaging, orchestration, technology adapters, trading partner management, BAM, BRE, etc
    Core Use Cases: Async messaging-based integration between applications; stateful, long running workflow processes that coordinate interactions between systems
    Buy-Hold-Sell: Buy
  • Product: ESB Toolkit
    Tag Line: “Dynamic messaging”
    Most Recent Release: 4/13 with BizTalk 2013
    Capabilities: Dynamic routing, services for mapping and pipelines
    Core Use Cases: Diverse message paths; message flow modeling, supremely loose coupling, centralized exception management
    Buy-Hold-Sell: Soft Buy
  • Product: Windows Server AppFabric
    Tag Line: Advanced service hosting
    Most Recent Release: 3/12
    Capabilities: Caching, advanced WCF and WF hosting; troubleshooting; persistent workflows
    Core Use Cases: WCF service management, hosting long running WF services
    Buy-Hold-Sell: Hold
  • Product: Service Bus for Windows
    Tag Line: On-premises durable messaging
    Most Recent Release: 10/13
    Capabilities: Running messaging aspects of Azure Service Bus on-premises; topics and queues; manage with new Windows Azure Pack on Windows
    Core Use Cases: Start work on premises and move to cloud; basic pub-sub messaging;
    Buy-Hold-Sell: Hold
  • Product: Workflow Manager
    Tag Line: On-premises workflow host
    Most Recent Release: 3/13 (CU)
    Capabilities: multi-tenancy, tracking and monitoring, instance management, versioning support
    Core Use Cases: Host and manage WF apps;
    Buy-Hold-Sell: Hold

    http://msdn.microsoft.com/library/jj193528
  • Product: WCF
    Tag Line: Sophisticated service framework
    Most Recent Release:
    Capabilities: Contract first service design for REST and SOAP services; configuration-centric service model that is extreme (overly?) extensible
    Core Use Cases: Building SOAP based web services; Sync or async
    Buy-Hold-Sell: Soft Buy
  • Product: ASP.NET Web API
    Tag Line: Modern web services
    Most Recent Release: 1/14
    Capabilities: REST/HTTP service design; extensible model; Odata support
    Core Use Cases: web service design
    Buy-Hold-Sell: Buy
  • Product: Windows Workflow
    Tag Line: Lightweight, durable processes
    Most Recent Release:
    Capabilities: declarative models that describe a process; built-in and custom activities; durable workflow; service endpoints
    Core Use Cases: long running workflows; exposing business processes as services
    Buy-Hold-Sell: Soft Buy
  • Product: SSIS (also Master Data Services?)
    Tag Line: Bulk data transfer
    Most Recent Release: SQL 2014 (no big changes)
    Capabilities: Data integration and data transformation; lots of data sources (files, FTP, HTTP, SMTP, RDBMS); bulk movement
    Core Use Cases:
    Buy-Hold-Sell: Buy
  • Product: StreamInsight
    Tag Line: Event stream processing
    Most Recent Release: 6/12
    Capabilities: high volume pattern matching against event streams; adapter model; Iobservable, Ienumerable support; multiple hosting options; LINQ queries
    Core Use Cases: situational awareness from data streams
    Buy-Hold-Sell: Soft Hold
  • Product: Windows Azure Virtual Machines – BizTalk Image
    Tag Line: Full BizTalk in the Cloud
    Most Recent Release: June 2013
    Capabilities: Run full BizTalk Server in the cloud; no HA with SQL clustering
    Core Use Cases: Dev/test scenarios
    Buy-Hold-Sell: Hold
  • Product: Windows Azure Service Bus Relays
    Tag Line: Services that span boundaries
    Most Recent Release:
    Capabilities: Traverse networks and securely expose sync and async WCF services
    Core Use Cases: partner integration, mobile device integration, remote worker tools, cloud to cloud integration
    Buy-Hold-Sell: Strong Buy
  • Product: Service Bus Brokered Messaging
    Tag Line: Durable messaging in the cloud
    Most Recent Release:
    Capabilities: Durable messaging through queues (single reader), and topics (broadcast with subscriptions)
    Core Use Cases: partner integration, mobile integration, cloud to cloud integration
    Buy-Hold-Sell: Buy
  • Product: Service Bus Notification Hubs
    Tag Line: Scalable message distribution
    Most Recent Release:
    Capabilities: Broadcast messages to mobile devices and Windows 8;
    Core Use Cases:
    Buy-Hold-Sell: Buy
  • Product: Windows Azure BizTalk Services
    Tag Line: Lightweight cloud message broker
    Most Recent Release:
    Capabilities: Link applications via cloud-hosted broker; source/destination adapters; strong data mapper
    Core Use Cases: connecting partners, cloud apps
    Buy-Hold-Sell: Strong Hold
  • MANY other products that work well in a Windows and cloud world
    Brokers like Apache Kafka
    Queue technologies like RabbitMQ, or Amazon SQS
    Cloud-based integration tools like SnapLogic and Informatica Cloud
    SaaS-integrated services for message publication and workflow
    E.g. Dynamics CRM Online or Salesforce.com
  • Options …
    Use messaging and use BizTalk to send messages to vendor endpoints
    PRO: Good tracking, chance for enrichment before sending out; guaranteed delivery
    CON: hard to quickly change endpoints; puts responsibility on internal team to manage/scale;
    Use messaging and use BizTalk to send to Topics, local DB, and notification Hubs
    Use messaging and send loans directly to SB topics (and internal DB) from WF
    PRO
    CON
    Send loans to SB topic and one listener is internal DB
    PRO: Simple, offload responsibility for delivery
    CON: No coordination; chance of race condition; no notification for vendors of new messages
    Send loans to SB topic, one listener is internal DB, and also use Notification Hubs for alerting
    Provide REST-ful query API (exposed via SB) on loan DB and use notification hubs to send messages to providers to query and provide their best offer
    PRO: Simple; no replication; easy to onboard new vendors; offload availability to 3rd party; durable;
    CON: Not event based; need to chain together way to notify vendors; no orchestration or tracking; limited visibility into vendor processing
  • Let’s consider the option that leverages the Service Bus for all this
    Send a message to Notification Hubs so very easy to broadcast info and push alerts
    Use Topics for scalable, durable messaging
    Subscribe ourselves to update internal systems
  • When to use What? A look at choosing Integration Technology

    1. 1. brought to you by When to Use What? A Look at Choosing Integration Technology RICHARD SEROTER @rseroter
    2. 2. Richard Seroter Director of Product Mgmt at CenturyLink Cloud Microsoft MVP for Integration Technical Trainer at Pluralsight Cloud Editor for InfoQ.com 3-time Book Author
    3. 3. WHERE ARE WE?
    4. 4. Integration Technologies Covered: 10 Integration Technologies TODAY: 14+
    5. 5. TRENDS
    6. 6. CLOUD COMPUTING
    7. 7. MICRO SERVICES
    8. 8. LIGHTWEIGHT INTEGRATION
    9. 9. NOSQL ENDPOINTS
    10. 10. AUTOMATION
    11. 11. DESIRED END STATE?
    12. 12. MAINTAINABLE SIMPLICITY
    13. 13. GETTING THERE
    14. 14. DECISION FRAMEWORK
    15. 15. FUNCTIONAL REQUIREMENTS NON FUNCTIONAL REQUIREMENTS DERIVED REQUIREMENTS ORGANIZATIONAL STRATEGY
    16. 16. FUNCTIONAL REQUIREMENTS NON FUNCTIONAL REQUIREMENTS DERIVED REQUIREMENTS ORGANIZATIONAL STRATEGY
    17. 17. FUNCTIONAL REQUIREMENTS NON FUNCTIONAL REQUIREMENTS DERIVED REQUIREMENTS ORGANIZATIONAL STRATEGY
    18. 18. FUNCTIONAL REQUIREMENTS NON FUNCTIONAL REQUIREMENTS DERIVED REQUIREMENTS ORGANIZATIONAL STRATEGY
    19. 19. Data volumes Guaranteed delivery Data sources / destinations Latency Authentication / authorization Failure handling Endpoint location, reliability, constraints Circuit breakers Design DevOperations Org Strategy
    20. 20. Design Dev Operations Org Strategy Developer availability Learning curve Developer setup IDE and tool integration Support system Automated build support Deploy to cloud Web-based dev tools Continuous integration / delivery
    21. 21. Design Dev Operations Org Strategy Event logging Performance tuning knobs Burst and failure handling Backup / restore tools Scriptability Immutable servers Rapid release / rollback Distributed deployments
    22. 22. Design DevOperations Org Strategy Long term solution fit Vendor support available Uses existing (software) investments Budget impact Build vs. buy philosophy Risk tolerance Speed to market OSS components Commodity skill sets Simple upgrades
    23. 23. PRODUCT EVALUATION
    24. 24. BIZTALK SERVER “Comprehensive application integration” BUYHOLDSELL
    25. 25. ESBTOOLKIT “Dynamic BizTalk messaging” BUYHOLDSELL
    26. 26. WINDOWS SERVER APPFABRIC “Advanced service hosting” BUYHOLDSELL
    27. 27. SERVICE BUS FOR WINDOWS “On-premises durable messaging” BUYHOLDSELL
    28. 28. “On-premises, multi- tenant Workflow host” BUYHOLDSELL WORKFLOW MANAGER
    29. 29. WCF “Sophisticated service framework” BUYHOLDSELL
    30. 30. ASP.NET WEB API “Modern web services” BUYHOLDSELL
    31. 31. WINDOWS WORKFLOW “Lightweight long- running processes” BUYHOLDSELL
    32. 32. SSIS “Bulk transfer between data sources” BUYHOLDSELL
    33. 33. STREAM INSIGHT “Complex event stream processing” BUYHOLDSELL
    34. 34. WINDOWS AZURE VM “Full BizTalk Server in the cloud” BUYHOLDSELL
    35. 35. SERVICE BUS RELAY “Services that span network boundaries” BUYHOLDSELL
    36. 36. SERVICE BUS BROKERED MESSAGING “Durable messaging in the cloud” BUYHOLDSELL
    37. 37. “Scalable mobile message distribution” BUYHOLDSELL SERVICE BUS NOTIFICATION HUBS
    38. 38. BIZTALK SERVICES “Lightweight cloud message broker” BUYHOLDSELL
    39. 39. APACHE KAFKA AMAZON SQS RABBIT MQ MULE SNAPLOGIC INFORMATICA
    40. 40. EXAMPLE SCENARIO
    41. 41. QUICKQUALIFY “Get multiple loan quotes within minutes!” BUSINESS NEED COMPANY OVERVIEW • Accept loan parameters from web customers • Store loan request in on-premises system • Reliably provide loan request to volatile list of external providers • Track responses and show the customer choices • High uptime, low latency integration • Strong development team • Over-extended Ops team • Decentralized org, local autonomy • Microsoft-friendly organization
    42. 42. QUICKQUALIFY Customer website Loan DB ESB Services ETL CRM System Loan VendorLoan Vendor Loan Vendor CLOUD
    43. 43. DEMONSTRATION
    44. 44. CONCLUSION
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×