SlideShare a Scribd company logo
1 of 72
Many Application Types…
Many Application Models…
Web Hosting                               High Performance Computing
          Massive scale infrastructure             Parallel & distributed
                                                     processing
          Burst & overflow capacity
                                                    Massive modeling & simulation
          Temporary, ad-hoc sites
                                                    Advanced analytics
Application Hosting
          Hybrid applications
                                          Information Sharing
                                                    Reference data
          Composite applications
                                                    Common data repositories
          Automated agents / jobs
                                                    Knowledge discovery & mgmt
Media Hosting & Processing
          CGI rendering
                                          Collaborative Processes
                                                    Multi-enterprise integration
          Content transcoding
                                                    B2B & e-commerce
          Media streaming
                                                    Supply chain management
Distributed Storage
                                                    Health & life sciences
          External backup and storage
                                                    Domain-specific services
Computing
 -Oriented
Application
Computing-Oriented Applications

 Calculation intensive
 Focus on
   Algorithms
   Speed
   Parallelism
Data-
 Oriented
Application
Data-Oriented Applications

 Data access intensive
 Focus on
     Storage Capacity
     Storage Realibility
     Access Speed
     Query API
Traditional CRUD application


 Let’s take a step back. Why do we build applications like we do today?

          It started with a stack of paper…




…that needed to be keyed                                  …and along came
    into the machine                                       the CRUD app!
Traditional scale-up
architecture
   Common characteristics
       synchronous processes
       sequential units of work
       tight coupling
       stateful
       pessimistic concurrency
       clustering for HA
       vertical scaling
Statefulness
Syncronous
   Calls
Tightly
  coupled
architectures
The ability to increase or reduce the
number of resources without             Scalability
affecting the end user experience.
Traditional scale-up architecture

  To scale, get bigger
   servers
       expensive
       has scaling limits
       inefficient use of
        resources




web               app server   data store
When failure is happening…
When problems occur…
   bigger failure impact




web         app server                web   app server




web         app server   data store   web                data store
Complex solution…
DB-
oriented
Paura??
Stale Data

 In computer processing, if a
  processor changes the value of
  an operand and then, at a
  subsequent time, fetches the
  operand and obtains the old
  rather than the new value of
  the operand, then it is said to
  have seen stale data.
Why is CQRS needed?

   To understand this better, let’s look at a basic multi-user system.




                    Retrieve data
                                            Retrieve data

User is looking at stale
          data                                               Modify data




            Stale data is inherent in a multi-user system.
     The machine is now the source of truth…not a piece of paper.
Why is CQRS needed?

 All of this to provide scalability & a
  consistent view of the data.
                But did we succeed?
Why is CQRS needed?

                   Back to our CRUD app…



                                                             ?
                          ?



   ?          ?
                                                             ?
                          ?




Where is the consistency? We have stale data all over the place!
Why is new model needed?

 Since stale data always exists, is all of this
  complexity really needed to scale?
   No, we need a different approach.
 One that offers extreme scalability
 Inherently handle multiple users
 Can grow to handle complex problems
  without growing development costs
Use more pieces, not bigger pieces

     LEGO 7778 Midi-scale Millennium
     Falcon
     • 9.3 x 6.7 x 3.2 inches (L/W/H)
     • 356 pieces
                                        LEGO 10179 Ultimate Collector's Millennium Falcon
                                        • 33 x 22 x 8.3 inches (L/W/H)
                                        • 5,195 pieces
Scale out
Fundamental concepts
   Horizontal scaling
   Small pieces, loosely coupled
   Distributed computing best practices
       asynchronous processes (event-driven design)
       parallelization
       idempotent operations (handle duplicity)
       optimistic concurrency
       shared nothing architecture
       fault-tolerance by redundancy
        and replication
       etc.
Cloud-Scale Architecture
Design                                   Data & Content
     Horizontal scaling                      De-normalization
     Service-oriented composition            Logical partitioning
     Eventual consistency                    Distributed in-memory cache
     Fault tolerant (expect failures)        Diverse data storage options
Security                                       (persistent & transient, relational &
                                               unstructured, text & binary, read &
     Claims-based authentication &
                                               write, etc.)
      access control
                                         Processes
     Federated identity
                                              Loosely coupled components
     Data encryption & key mgmt.
                                              Parallel & distributed processing
Management
                                              Asynchronous distributed
     Policy-driven automation
                                               communication
     Aware of application lifecycles
                                              Idempotent (handle duplicity)
     Handle dynamic data schema and
                                              Isolation (separation of concerns)
      configuration changes
Scale-out architecture
   Common
    characteristics
       small logical
        units of work
       loosely-coupled
        processes
       stateless          web   app server   data store
       event-driven
        design
       optimistic         web   app server   data store
        concurrency
       partitioned data
       redundancy
        fault-tolerance
       re-try-based
        recoverability
Scale-out architecture
 To scale, add
  more servers          web   app server   data store

     not bigger
      servers           web   app server   data store

 When problems
  occur                 web   app server   data store


     smaller failure
      impact
                        web   app server   data store


     higher            web   app server   data store
      perceived
      availability      web   app server   data store
     simpler
      recovery
Scale-out architecture + distributed
computing                               parallel tasks

   Scalable
    performance at                web    app server      data store
    extreme scale
       asynchronous              web    app server      data store
        processes
       parallelization           web    app server      data store
       smaller
        footprint                 web    app server      data store
        optimized
        perceived response time
    
        resource usage            web    app server      data store
       reduced async tasks
        response time
                                  web    app server      data store
       improved
        throughput
How does CQRS work?

                                          Task-based UI
   Why rethink the User Interface?
» Grids don’t capture the user’s intent
CQRS
As a concept
 A set of principles
 A way of thinking about software
   architecture.
As a pattern
 Is a way of designing and developing
   scalable and robust enterprise
   solutions where reads are
   independent from writes.
What is not
 The CQRS pattern says nothing about
   how this should be implemented
Some new
architectures
  [CQRS?!?]
Common components of
the CQRS pattern
 Task-based UI
     ViewModels
   Commands
   Domain Objects
   Events
   Persistent View Model
Task-Driven User Interface

 Scrum-based analysis
 Collect user-stories
   Scenario
 Each user-story is not
  an entity
 Every user story is a
  task
How does CQRS work?
Rethinking the User Interface
   Adjust UI design to capture intent
       what did the user really mean?
       intent becomes a command

   Why is intent important?
       Last name changed because of
        misspelling
       Last name changed because of
        marriage
       Last name changed because of
        divorce
   User interface can affect your architecture
View Models
   ViewModel
       Only Data
       Flat, only strings
   Why DomainModel is not
    good?
       Views should not know how
        to traverse the DM
       Views usually need less
        properties
       Using ORMs you might start a
        SQL query by mistake
   How to do it?
       Copy the properties needed
        from DM to VM
       Possibly flatten data
How does CQRS work?
   Validation
       increase likelihood of command
        succeeding
       validate client-side
       optimize validation using persistent view
        model
   What about user feedback?
       Polling: wait until read model is updated
       Use asynchronous messaging such as
        email
           “Your request is being processed. You will
            receive an email when it is completed”
       Just fake it!
           Scope the change to the current user.
            Update a local in-memory model
How do Commands work?

   Commands encapsulate the user’s intent
    but do not contain business logic, only
    enough data for the command
   What makes a good command?
       A command is an action – starts with a
        verb
       The kind you can reply with: “Thank you.
        Your confirmation email will arrive shortly”.
        Inherently asynchronous.
   Commands can be considered messages
       Messaging provides an asynchronous
        delivery mechanism for the commands. As
        a message, it can be routed, queued, and
        transformed all independent of the sender
        & receiver
Domain Model
 The domain model is
  utilized for processing
  commands; it is
  unnecessary for
  queries.
 Unlike entity objects
  you may be used to,
  aggregate roots in
  CQRS only have
  methods (no
  getters/setters)
Events
   Events describe changes
    in the system state
   An Event Bus can be
    utilized to dispatch
    events to subscribers
   Events primary purpose
    update the read model
   Events can also provider
    integration with external
    systems
   CQRS can also be used in
    conjunction with Event
    Sourcing.
Persistent View Model
   Reads are usually the most
    common activity – many
    times 80-90%. Why not
    optimize them?
   Read model is based on
    how the user wants to see
    the data.
   Read model can be
    denormalized RDBMS,
    document store, etc.
   Reads from the view model
    don’t need to be loaded
    into the domain model,
    they can be bond directly
    to the UI.
Persistent View Model

                  Data Duplicated, No Relationships, Data Pre-Calculated
Customer Service Rep view                              Supervisor view
          List of customers                            List of customers
ID        Name    Phone                   ID    Name     Phone Lifetime value




          Rep_Customers_Table                     Supervisor_Customers_Table
     ID          Name           Phone      ID     Name        Phone        Lifetime Value
When should not use
CQRS?
 CQRS can be overkill for simple
  applications.
 Don’t use it in a non-
  collaborative domain or where
  you can horizontally add more
  database servers to support
  more users/requests/data at
  the same time you’re adding
  web servers – there is no real
  scalability problem – Udi Dahan
When should I use CQRS?

Guidelines for using CQRS:
 Large, multi-user systems CQRS is
   designed to address concurrency issues.
 Scalability matters With CQRS you can
   achieve great read and write performance.
   The system intrinsically supports scaling
   out. By separating read & write
   operations, each can be optimized.
 Difficult business logic CQRS forces you
   to not mix domain logic and
   infrastructural operations.
 Large or Distributed teams you can split
   development tasks between different
   teams with defined interfaces.
Compute Services

 Low Availability Computing
  Nodes
 High Availability Computing
  Node
Service
  Level
Agreement
Low Available Compute
Node
   Many virtual servers of public clouds are
    offered at a low availability. Sometimes,
    availability is additionally expressed in an
    uncommon manner. For example, Amazon
    guarantees an availability of EC2 instances of
    99.95% during a service year of 365 days [8].
       99.95% means about 4,4h/yr
   However, this does not mean that a single
    instance has 99.95% availability during this
    time period, as could be expected.
   Instead, unavailability is defined as the state
    when all running instances cannot be reached
    longer than five minutes and no replacement
    instances can be provisioned.
Elastic Infrastructure
   Resources shall be assigned to and
    revoked from applications dynamically
    depending on the current load.
   The infrastructure must support dynamic
    provisioning and deprovisioning of
    resources
   This functionality must be offered
    through an API to be used by atomized
    management tools and the applications
    that are hosted by the environment.
   An elastic infrastructure supports the
    dynamic allocation of (virtual) resources
    that constitute a common resource pool.
Storage Consistency

 Strict Consistency
 Eventual Consistency
Strict Consistency
   A storage offering usually consists of
    multiple replicas to ensure fault
    tolerance. It is of major importance that
    the consistency of the data contained in
    these replicas is pertained at all times
    while the performance is of secondary
    importance.
   The highest level of consistency is
    granted if all replicas are updated if the
    data contained by them is altered.
    However, this would mean that the
    availability of the overall storage
    solution is decreased drastically. It has to
    be ensured that it is available even if not
    all replicas are available, but still the
    correct version of the data is read.
Eventual Consistency
   Eventually consistent data storage allows
    reducing data consistency to increase
    availability and performance, since the impact
    of network partitioning is reduced and fewer
    replicas have to be accessed during read and
    write operation.
   While strictly consistent databases ensure that
    always at least one of the current version is
    read, eventually consistent databases allow
    that obsolete versions may also be read.
   This increases the availability of the storage
    offering since only one replica has to be
    available to successfully execute a read
    operation.
ACID vs. BASE
CAP (Consistency, Availability, Partition) Theorem


 At most two of these properties for any shared-data system

                  Consistency + Availability
      C       A   • High data integrity
          P       • Single site, cluster database, LDAP, xFS file system,
                    etc.
                  • 2-phase commit, data replication, etc.
                  Consistency + Partition
      C       A   • Distributed database, distributed locking, etc.
          P       • Pessimistic locking, minority partition unavailable, etc.


                  Availability + Partition
      C       A   • High scalability
                  • Distributed cache, DNS, etc.
          P
                  • Optimistic locking, expiration/leases, etc.
Hybrid architectures

• Scale-out (horizontal)                          Scale-up (vertical)
    – BASE: Basically Available, Soft state,          ACID: Atomicity, Consistency,
      Eventually consistent                            Isolation, Durability
                                                      availability first; best effort
    – focus on “commit”
                                                      aggressive (optimistic)
    – conservative (pessimistic)
                                                      transactional
    – shared nothing                                  favor accuracy/consistency
    – favor extreme size                              e.g., BI & analytics, financial
    – e.g., user requests, data collection &           processing, etc.
      processing, etc.


       Most distributed systems employ both approaches
Storage Services

   Relations Data Storage
   Blob Data Storage
   Block Data Storage
   NoSQL Storage
Relational Data Store

 An application uses a central database for
  storing data elements and performs
  complex queries on them
Blob Storage
   A distributed application needs to manage large data
    elements, such as virtual server images or videos, which are
    too large for traditional databases.
   In a distributed application data elements must be made
    available to all application components and to distributed
    users. Access to the data needs to be performed in a
    standardized fashion and access control has to be established.
   Organize the data elements in a folder hierarchy similar to a
    traditional file system. Give each data element a unique
    identifier that can be used to access it over a network. Also,
    establish access control mechanisms.
Block Storage
   Resources in clouds are often unreliable (low
    available compute nodes).
   Therefore, the data that they access locally
    shall in fact be stored in a high available central
    data store. This way, if a server fails the data is
    not lost, but a new server can be started to use
    the secured data.
   Offer data elements in a central storage that
    can be accessed by distributed servers and
    integrate them as local drives.
NoSQL Storage
 Need to handle very large amounts of data
  and also need to be adjusted to new user
  demands flexibly.
 Database solution is required that focuses on
  scaling out rather than on optimizing the use
  of a single resource and that can adjust
  flexibly to changes of the data structure.
 Use a schema-free storage solution, with
  limited query capabilities to enable extreme
  scale-out through easy data replication.
Communication Services

   Message Oriented Middleware
   Reliable Messaging
   Exactly Once Delivery
   At least Once Delivery
Message-oriented middleware
   Different applications usually use different languages,
    data formats, and technology platforms. When one
    application (component) needs to exchange
    information with another one, the format of the target
    application has to be respected.
   Sending messages directly to the target application
    results in a tight coupling of sender and receiver since
    format changes directly affect both implementations.
   Connect applications through an intermediary, the
    message oriented middleware, that hides the
    complexity of addressing and availability of
    communication partners as well as supports
    transformation of different message formats.
Reliable Messaging
 The message transfer from one
  communication partner to the other is
  performed under transactional context.
 Especially, this transaction subsumes the
  operation performed to store the messages in
  persistent storage.
 Thus, if an error occurs during message
  receiving, sending, or processing the
  transaction can be compensated transferring
  the overall system back to a correct and
  consistent state.
At-least once
   The receiver of messages sends special acknowledge
    messages to the sender.
   If the sender does not receive such an
    acknowledgement message in a given time frame it
    retransmits the message.
   Thus, messages, which are lost due to communication
    errors, are still received eventually.
   However, duplicate messages can occur, for example, if
    an acknowledgement message is lost.
   To reduce the communication overhead,
    acknowledgement messages can be sent either after
    each individual message or after an agreed upon
    number of messages.
Exaclty-once delivery
 Whenever a message is created it is
  associated with a unique identifier.
 This is used by a filtering component on the
  message path to delete duplicates.
 It does so by storing the identifiers of
  messages it has already seen.
 The identifiers of messages passing through
  this filtering component are then compared
  to the identifiers that have been recorded to
  identify and delete duplicates.

More Related Content

What's hot

Building Scalable Databases on AWS - AWS Summit 2012 - NYC
Building Scalable Databases on AWS - AWS Summit 2012 - NYCBuilding Scalable Databases on AWS - AWS Summit 2012 - NYC
Building Scalable Databases on AWS - AWS Summit 2012 - NYC
Amazon Web Services
 
Dell - Storage 12sept2012
Dell - Storage 12sept2012Dell - Storage 12sept2012
Dell - Storage 12sept2012
Agora Group
 
Accel Partners New Data Workshop 7-14-10
Accel Partners New Data Workshop 7-14-10Accel Partners New Data Workshop 7-14-10
Accel Partners New Data Workshop 7-14-10
keirdo1
 
Wed 1130 aasman_jans_color
Wed 1130 aasman_jans_colorWed 1130 aasman_jans_color
Wed 1130 aasman_jans_color
DATAVERSITY
 
Accelerating micro strategy for real time bi
Accelerating micro strategy for real time biAccelerating micro strategy for real time bi
Accelerating micro strategy for real time bi
Kognitio
 
Ibm big data ibm marriage of hadoop and data warehousing
Ibm big dataibm marriage of hadoop and data warehousingIbm big dataibm marriage of hadoop and data warehousing
Ibm big data ibm marriage of hadoop and data warehousing
DataWorks Summit
 

What's hot (20)

Building Scalable Databases on AWS - AWS Summit 2012 - NYC
Building Scalable Databases on AWS - AWS Summit 2012 - NYCBuilding Scalable Databases on AWS - AWS Summit 2012 - NYC
Building Scalable Databases on AWS - AWS Summit 2012 - NYC
 
Business and Online Services - Ben Kepes
Business and Online Services - Ben KepesBusiness and Online Services - Ben Kepes
Business and Online Services - Ben Kepes
 
RavenDB overview
RavenDB overviewRavenDB overview
RavenDB overview
 
Dell - Storage 12sept2012
Dell - Storage 12sept2012Dell - Storage 12sept2012
Dell - Storage 12sept2012
 
Infochimps #1 Big Data Platform for the Cloud
Infochimps #1 Big Data Platform for the CloudInfochimps #1 Big Data Platform for the Cloud
Infochimps #1 Big Data Platform for the Cloud
 
Azure SQL Database Managed Instance
Azure SQL Database Managed InstanceAzure SQL Database Managed Instance
Azure SQL Database Managed Instance
 
Aras Connected Cloud for PLM
Aras Connected Cloud for PLMAras Connected Cloud for PLM
Aras Connected Cloud for PLM
 
Azure SQL DB Managed Instances Built to easily modernize application data layer
Azure SQL DB Managed Instances Built to easily modernize application data layerAzure SQL DB Managed Instances Built to easily modernize application data layer
Azure SQL DB Managed Instances Built to easily modernize application data layer
 
Enterprise Content Management and Microsoft Office SharePoint Server 2007 - U...
Enterprise Content Management and Microsoft Office SharePoint Server 2007 - U...Enterprise Content Management and Microsoft Office SharePoint Server 2007 - U...
Enterprise Content Management and Microsoft Office SharePoint Server 2007 - U...
 
Using hadoop to expand data warehousing
Using hadoop to expand data warehousingUsing hadoop to expand data warehousing
Using hadoop to expand data warehousing
 
Accel Partners New Data Workshop 7-14-10
Accel Partners New Data Workshop 7-14-10Accel Partners New Data Workshop 7-14-10
Accel Partners New Data Workshop 7-14-10
 
Engineering Interoperable and Reliable Systems
Engineering Interoperable and Reliable SystemsEngineering Interoperable and Reliable Systems
Engineering Interoperable and Reliable Systems
 
Wed 1130 aasman_jans_color
Wed 1130 aasman_jans_colorWed 1130 aasman_jans_color
Wed 1130 aasman_jans_color
 
Accelerating micro strategy for real time bi
Accelerating micro strategy for real time biAccelerating micro strategy for real time bi
Accelerating micro strategy for real time bi
 
What are the Business Benefits of Microsoft Azure
What are the Business Benefits of Microsoft AzureWhat are the Business Benefits of Microsoft Azure
What are the Business Benefits of Microsoft Azure
 
4 Ways To Save Big Money in Your Data Center and Private Cloud
4 Ways To Save Big Money in Your Data Center and Private Cloud4 Ways To Save Big Money in Your Data Center and Private Cloud
4 Ways To Save Big Money in Your Data Center and Private Cloud
 
Sybase IQ Big Data
Sybase IQ Big DataSybase IQ Big Data
Sybase IQ Big Data
 
Ibm big data ibm marriage of hadoop and data warehousing
Ibm big dataibm marriage of hadoop and data warehousingIbm big dataibm marriage of hadoop and data warehousing
Ibm big data ibm marriage of hadoop and data warehousing
 
The Executive View on Big Data Platform Hosting - Evaluating Hosting Services...
The Executive View on Big Data Platform Hosting - Evaluating Hosting Services...The Executive View on Big Data Platform Hosting - Evaluating Hosting Services...
The Executive View on Big Data Platform Hosting - Evaluating Hosting Services...
 
Seattle Scalability - GigaSpaces / Cassandra
Seattle Scalability - GigaSpaces / CassandraSeattle Scalability - GigaSpaces / Cassandra
Seattle Scalability - GigaSpaces / Cassandra
 

Viewers also liked

Viewers also liked (8)

Azure DocumentDb
Azure DocumentDbAzure DocumentDb
Azure DocumentDb
 
Azure DocumentDB 101
Azure DocumentDB 101Azure DocumentDB 101
Azure DocumentDB 101
 
Azure DocumentDB
Azure DocumentDBAzure DocumentDB
Azure DocumentDB
 
[PASS Summit 2016] Azure DocumentDB: A Deep Dive into Advanced Features
[PASS Summit 2016] Azure DocumentDB: A Deep Dive into Advanced Features[PASS Summit 2016] Azure DocumentDB: A Deep Dive into Advanced Features
[PASS Summit 2016] Azure DocumentDB: A Deep Dive into Advanced Features
 
Introduction to Azure DocumentDB
Introduction to Azure DocumentDBIntroduction to Azure DocumentDB
Introduction to Azure DocumentDB
 
Introducing DocumentDB
Introducing DocumentDB Introducing DocumentDB
Introducing DocumentDB
 
DocumentDB - Azure Fridays São Paulo
DocumentDB - Azure Fridays São PauloDocumentDB - Azure Fridays São Paulo
DocumentDB - Azure Fridays São Paulo
 
Webinar - Introduction to Azure DocumentDB
Webinar - Introduction to Azure DocumentDBWebinar - Introduction to Azure DocumentDB
Webinar - Introduction to Azure DocumentDB
 

Similar to Application architecture for cloud

13h00 p duff-building-applications-with-aws-final
13h00   p duff-building-applications-with-aws-final13h00   p duff-building-applications-with-aws-final
13h00 p duff-building-applications-with-aws-final
Luiz Gustavo Santos
 
Building Web Applications on AWS - AWS Summit 2012 - NYC
Building Web Applications on AWS - AWS Summit 2012 - NYCBuilding Web Applications on AWS - AWS Summit 2012 - NYC
Building Web Applications on AWS - AWS Summit 2012 - NYC
Amazon Web Services
 
Choosing Your Windows Azure Platform Strategy
Choosing Your Windows Azure Platform StrategyChoosing Your Windows Azure Platform Strategy
Choosing Your Windows Azure Platform Strategy
drmarcustillett
 
AWS Summit 2011: Architecting in the cloud
AWS Summit 2011: Architecting in the cloudAWS Summit 2011: Architecting in the cloud
AWS Summit 2011: Architecting in the cloud
Amazon Web Services
 

Similar to Application architecture for cloud (20)

Azure and cloud design patterns
Azure and cloud design patternsAzure and cloud design patterns
Azure and cloud design patterns
 
13h00 p duff-building-applications-with-aws-final
13h00   p duff-building-applications-with-aws-final13h00   p duff-building-applications-with-aws-final
13h00 p duff-building-applications-with-aws-final
 
Building Applications with AWS
Building Applications with AWSBuilding Applications with AWS
Building Applications with AWS
 
Building Web Applications on AWS - AWS Summit 2012 - NYC
Building Web Applications on AWS - AWS Summit 2012 - NYCBuilding Web Applications on AWS - AWS Summit 2012 - NYC
Building Web Applications on AWS - AWS Summit 2012 - NYC
 
High Performance Databases
High Performance DatabasesHigh Performance Databases
High Performance Databases
 
Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, K...
Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, K...Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, K...
Cloud architecture and deployment: The Kognitio checklist, Nigel Sanctuary, K...
 
Choosing Your Windows Azure Platform Strategy
Choosing Your Windows Azure Platform StrategyChoosing Your Windows Azure Platform Strategy
Choosing Your Windows Azure Platform Strategy
 
Analytics in the Cloud
Analytics in the CloudAnalytics in the Cloud
Analytics in the Cloud
 
AWS Summit 2011: Architecting in the cloud
AWS Summit 2011: Architecting in the cloudAWS Summit 2011: Architecting in the cloud
AWS Summit 2011: Architecting in the cloud
 
Java Batch for Cost Optimized Efficiency
Java Batch for Cost Optimized EfficiencyJava Batch for Cost Optimized Efficiency
Java Batch for Cost Optimized Efficiency
 
OreDev 2008: Software + Services
OreDev 2008: Software + ServicesOreDev 2008: Software + Services
OreDev 2008: Software + Services
 
Optimizacija namestitev sap_ki_delujejo_na_infrastrukturi_net_app
Optimizacija namestitev sap_ki_delujejo_na_infrastrukturi_net_appOptimizacija namestitev sap_ki_delujejo_na_infrastrukturi_net_app
Optimizacija namestitev sap_ki_delujejo_na_infrastrukturi_net_app
 
AWS RDS Presentation - DOAG Conference
AWS RDS Presentation - DOAG Conference AWS RDS Presentation - DOAG Conference
AWS RDS Presentation - DOAG Conference
 
Microsoft Cloud Computing for Ecommerce
Microsoft Cloud Computing for EcommerceMicrosoft Cloud Computing for Ecommerce
Microsoft Cloud Computing for Ecommerce
 
Internet Scale Architecture
Internet Scale ArchitectureInternet Scale Architecture
Internet Scale Architecture
 
Innovating for Growth
Innovating for Growth Innovating for Growth
Innovating for Growth
 
Amazon Web Services
Amazon Web ServicesAmazon Web Services
Amazon Web Services
 
Tapping the cloud for real time data analytics
 Tapping the cloud for real time data analytics Tapping the cloud for real time data analytics
Tapping the cloud for real time data analytics
 
Vision - The Agile Data Center
Vision - The Agile Data CenterVision - The Agile Data Center
Vision - The Agile Data Center
 
benefits of SQL Server 2008 R2 Enterprise Edition
benefits of SQL Server 2008 R2 Enterprise Editionbenefits of SQL Server 2008 R2 Enterprise Edition
benefits of SQL Server 2008 R2 Enterprise Edition
 

More from Marco Parenzan

More from Marco Parenzan (20)

Azure IoT Central per lo SCADA engineer
Azure IoT Central per lo SCADA engineerAzure IoT Central per lo SCADA engineer
Azure IoT Central per lo SCADA engineer
 
Azure Hybrid @ Home
Azure Hybrid @ HomeAzure Hybrid @ Home
Azure Hybrid @ Home
 
Static abstract members nelle interfacce di C# 11 e dintorni di .NET 7.pptx
Static abstract members nelle interfacce di C# 11 e dintorni di .NET 7.pptxStatic abstract members nelle interfacce di C# 11 e dintorni di .NET 7.pptx
Static abstract members nelle interfacce di C# 11 e dintorni di .NET 7.pptx
 
Azure Synapse Analytics for your IoT Solutions
Azure Synapse Analytics for your IoT SolutionsAzure Synapse Analytics for your IoT Solutions
Azure Synapse Analytics for your IoT Solutions
 
Power BI Streaming Data Flow e Azure IoT Central
Power BI Streaming Data Flow e Azure IoT Central Power BI Streaming Data Flow e Azure IoT Central
Power BI Streaming Data Flow e Azure IoT Central
 
Power BI Streaming Data Flow e Azure IoT Central
Power BI Streaming Data Flow e Azure IoT CentralPower BI Streaming Data Flow e Azure IoT Central
Power BI Streaming Data Flow e Azure IoT Central
 
Power BI Streaming Data Flow e Azure IoT Central
Power BI Streaming Data Flow e Azure IoT CentralPower BI Streaming Data Flow e Azure IoT Central
Power BI Streaming Data Flow e Azure IoT Central
 
Developing Actors in Azure with .net
Developing Actors in Azure with .netDeveloping Actors in Azure with .net
Developing Actors in Azure with .net
 
Math with .NET for you and Azure
Math with .NET for you and AzureMath with .NET for you and Azure
Math with .NET for you and Azure
 
Power BI data flow and Azure IoT Central
Power BI data flow and Azure IoT CentralPower BI data flow and Azure IoT Central
Power BI data flow and Azure IoT Central
 
.net for fun: write a Christmas videogame
.net for fun: write a Christmas videogame.net for fun: write a Christmas videogame
.net for fun: write a Christmas videogame
 
Building IoT infrastructure on edge with .net, Raspberry PI and ESP32 to conn...
Building IoT infrastructure on edge with .net, Raspberry PI and ESP32 to conn...Building IoT infrastructure on edge with .net, Raspberry PI and ESP32 to conn...
Building IoT infrastructure on edge with .net, Raspberry PI and ESP32 to conn...
 
Anomaly Detection with Azure and .NET
Anomaly Detection with Azure and .NETAnomaly Detection with Azure and .NET
Anomaly Detection with Azure and .NET
 
Deploy Microsoft Azure Data Solutions
Deploy Microsoft Azure Data SolutionsDeploy Microsoft Azure Data Solutions
Deploy Microsoft Azure Data Solutions
 
Deep Dive Time Series Anomaly Detection in Azure with dotnet
Deep Dive Time Series Anomaly Detection in Azure with dotnetDeep Dive Time Series Anomaly Detection in Azure with dotnet
Deep Dive Time Series Anomaly Detection in Azure with dotnet
 
Azure IoT Central
Azure IoT CentralAzure IoT Central
Azure IoT Central
 
Anomaly Detection with Azure and .net
Anomaly Detection with Azure and .netAnomaly Detection with Azure and .net
Anomaly Detection with Azure and .net
 
Code Generation for Azure with .net
Code Generation for Azure with .netCode Generation for Azure with .net
Code Generation for Azure with .net
 
Running Kafka and Spark on Raspberry PI with Azure and some .net magic
Running Kafka and Spark on Raspberry PI with Azure and some .net magicRunning Kafka and Spark on Raspberry PI with Azure and some .net magic
Running Kafka and Spark on Raspberry PI with Azure and some .net magic
 
Time Series Anomaly Detection with Azure and .NETT
Time Series Anomaly Detection with Azure and .NETTTime Series Anomaly Detection with Azure and .NETT
Time Series Anomaly Detection with Azure and .NETT
 

Recently uploaded

“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
Muhammad Subhan
 
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial IntelligenceRevolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
Precisely
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
FIDO Alliance
 

Recently uploaded (20)

Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
Google I/O Extended 2024 Warsaw
Google I/O Extended 2024 WarsawGoogle I/O Extended 2024 Warsaw
Google I/O Extended 2024 Warsaw
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
 
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial IntelligenceRevolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptx
 
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...Hyatt driving innovation and exceptional customer experiences with FIDO passw...
Hyatt driving innovation and exceptional customer experiences with FIDO passw...
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
 
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfFrisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptxCyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
Cyber Insurance - RalphGilot - Embry-Riddle Aeronautical University.pptx
 

Application architecture for cloud

  • 1.
  • 3. Many Application Models… Web Hosting High Performance Computing  Massive scale infrastructure  Parallel & distributed processing  Burst & overflow capacity  Massive modeling & simulation  Temporary, ad-hoc sites  Advanced analytics Application Hosting  Hybrid applications Information Sharing  Reference data  Composite applications  Common data repositories  Automated agents / jobs  Knowledge discovery & mgmt Media Hosting & Processing  CGI rendering Collaborative Processes  Multi-enterprise integration  Content transcoding  B2B & e-commerce  Media streaming  Supply chain management Distributed Storage  Health & life sciences  External backup and storage  Domain-specific services
  • 5. Computing-Oriented Applications  Calculation intensive  Focus on  Algorithms  Speed  Parallelism
  • 7. Data-Oriented Applications  Data access intensive  Focus on  Storage Capacity  Storage Realibility  Access Speed  Query API
  • 8.
  • 9. Traditional CRUD application Let’s take a step back. Why do we build applications like we do today? It started with a stack of paper… …that needed to be keyed …and along came into the machine the CRUD app!
  • 10. Traditional scale-up architecture  Common characteristics  synchronous processes  sequential units of work  tight coupling  stateful  pessimistic concurrency  clustering for HA  vertical scaling
  • 12. Syncronous Calls
  • 14. The ability to increase or reduce the number of resources without Scalability affecting the end user experience.
  • 15. Traditional scale-up architecture  To scale, get bigger servers  expensive  has scaling limits  inefficient use of resources web app server data store
  • 16. When failure is happening…
  • 17. When problems occur…  bigger failure impact web app server web app server web app server data store web data store
  • 21. Stale Data  In computer processing, if a processor changes the value of an operand and then, at a subsequent time, fetches the operand and obtains the old rather than the new value of the operand, then it is said to have seen stale data.
  • 22. Why is CQRS needed? To understand this better, let’s look at a basic multi-user system. Retrieve data Retrieve data User is looking at stale data Modify data Stale data is inherent in a multi-user system. The machine is now the source of truth…not a piece of paper.
  • 23. Why is CQRS needed?  All of this to provide scalability & a consistent view of the data. But did we succeed?
  • 24. Why is CQRS needed? Back to our CRUD app… ? ? ? ? ? ? Where is the consistency? We have stale data all over the place!
  • 25. Why is new model needed?  Since stale data always exists, is all of this complexity really needed to scale?  No, we need a different approach.  One that offers extreme scalability  Inherently handle multiple users  Can grow to handle complex problems without growing development costs
  • 26.
  • 27. Use more pieces, not bigger pieces LEGO 7778 Midi-scale Millennium Falcon • 9.3 x 6.7 x 3.2 inches (L/W/H) • 356 pieces LEGO 10179 Ultimate Collector's Millennium Falcon • 33 x 22 x 8.3 inches (L/W/H) • 5,195 pieces
  • 28.
  • 30. Fundamental concepts  Horizontal scaling  Small pieces, loosely coupled  Distributed computing best practices  asynchronous processes (event-driven design)  parallelization  idempotent operations (handle duplicity)  optimistic concurrency  shared nothing architecture  fault-tolerance by redundancy and replication  etc.
  • 31. Cloud-Scale Architecture Design Data & Content  Horizontal scaling  De-normalization  Service-oriented composition  Logical partitioning  Eventual consistency  Distributed in-memory cache  Fault tolerant (expect failures)  Diverse data storage options Security (persistent & transient, relational & unstructured, text & binary, read &  Claims-based authentication & write, etc.) access control Processes  Federated identity  Loosely coupled components  Data encryption & key mgmt.  Parallel & distributed processing Management  Asynchronous distributed  Policy-driven automation communication  Aware of application lifecycles  Idempotent (handle duplicity)  Handle dynamic data schema and  Isolation (separation of concerns) configuration changes
  • 32. Scale-out architecture  Common characteristics  small logical units of work  loosely-coupled processes  stateless web app server data store  event-driven design  optimistic web app server data store concurrency  partitioned data  redundancy fault-tolerance  re-try-based recoverability
  • 33. Scale-out architecture  To scale, add more servers web app server data store  not bigger servers web app server data store  When problems occur web app server data store  smaller failure impact web app server data store  higher web app server data store perceived availability web app server data store  simpler recovery
  • 34. Scale-out architecture + distributed computing parallel tasks  Scalable performance at web app server data store extreme scale  asynchronous web app server data store processes  parallelization web app server data store  smaller footprint web app server data store optimized perceived response time  resource usage web app server data store  reduced async tasks response time web app server data store  improved throughput
  • 35.
  • 36. How does CQRS work? Task-based UI Why rethink the User Interface? » Grids don’t capture the user’s intent
  • 37. CQRS As a concept  A set of principles  A way of thinking about software architecture. As a pattern  Is a way of designing and developing scalable and robust enterprise solutions where reads are independent from writes. What is not  The CQRS pattern says nothing about how this should be implemented
  • 39. Common components of the CQRS pattern  Task-based UI  ViewModels  Commands  Domain Objects  Events  Persistent View Model
  • 40. Task-Driven User Interface  Scrum-based analysis  Collect user-stories  Scenario  Each user-story is not an entity  Every user story is a task
  • 41. How does CQRS work? Rethinking the User Interface  Adjust UI design to capture intent  what did the user really mean?  intent becomes a command  Why is intent important?  Last name changed because of misspelling  Last name changed because of marriage  Last name changed because of divorce  User interface can affect your architecture
  • 42. View Models  ViewModel  Only Data  Flat, only strings  Why DomainModel is not good?  Views should not know how to traverse the DM  Views usually need less properties  Using ORMs you might start a SQL query by mistake  How to do it?  Copy the properties needed from DM to VM  Possibly flatten data
  • 43. How does CQRS work?  Validation  increase likelihood of command succeeding  validate client-side  optimize validation using persistent view model  What about user feedback?  Polling: wait until read model is updated  Use asynchronous messaging such as email  “Your request is being processed. You will receive an email when it is completed”  Just fake it!  Scope the change to the current user. Update a local in-memory model
  • 44. How do Commands work?  Commands encapsulate the user’s intent but do not contain business logic, only enough data for the command  What makes a good command?  A command is an action – starts with a verb  The kind you can reply with: “Thank you. Your confirmation email will arrive shortly”. Inherently asynchronous.  Commands can be considered messages  Messaging provides an asynchronous delivery mechanism for the commands. As a message, it can be routed, queued, and transformed all independent of the sender & receiver
  • 45. Domain Model  The domain model is utilized for processing commands; it is unnecessary for queries.  Unlike entity objects you may be used to, aggregate roots in CQRS only have methods (no getters/setters)
  • 46. Events  Events describe changes in the system state  An Event Bus can be utilized to dispatch events to subscribers  Events primary purpose update the read model  Events can also provider integration with external systems  CQRS can also be used in conjunction with Event Sourcing.
  • 47. Persistent View Model  Reads are usually the most common activity – many times 80-90%. Why not optimize them?  Read model is based on how the user wants to see the data.  Read model can be denormalized RDBMS, document store, etc.  Reads from the view model don’t need to be loaded into the domain model, they can be bond directly to the UI.
  • 48. Persistent View Model Data Duplicated, No Relationships, Data Pre-Calculated Customer Service Rep view Supervisor view List of customers List of customers ID Name Phone ID Name Phone Lifetime value Rep_Customers_Table Supervisor_Customers_Table ID Name Phone ID Name Phone Lifetime Value
  • 49.
  • 50. When should not use CQRS?  CQRS can be overkill for simple applications.  Don’t use it in a non- collaborative domain or where you can horizontally add more database servers to support more users/requests/data at the same time you’re adding web servers – there is no real scalability problem – Udi Dahan
  • 51. When should I use CQRS? Guidelines for using CQRS:  Large, multi-user systems CQRS is designed to address concurrency issues.  Scalability matters With CQRS you can achieve great read and write performance. The system intrinsically supports scaling out. By separating read & write operations, each can be optimized.  Difficult business logic CQRS forces you to not mix domain logic and infrastructural operations.  Large or Distributed teams you can split development tasks between different teams with defined interfaces.
  • 52.
  • 53. Compute Services  Low Availability Computing Nodes  High Availability Computing Node
  • 55. Low Available Compute Node  Many virtual servers of public clouds are offered at a low availability. Sometimes, availability is additionally expressed in an uncommon manner. For example, Amazon guarantees an availability of EC2 instances of 99.95% during a service year of 365 days [8].  99.95% means about 4,4h/yr  However, this does not mean that a single instance has 99.95% availability during this time period, as could be expected.  Instead, unavailability is defined as the state when all running instances cannot be reached longer than five minutes and no replacement instances can be provisioned.
  • 56. Elastic Infrastructure  Resources shall be assigned to and revoked from applications dynamically depending on the current load.  The infrastructure must support dynamic provisioning and deprovisioning of resources  This functionality must be offered through an API to be used by atomized management tools and the applications that are hosted by the environment.  An elastic infrastructure supports the dynamic allocation of (virtual) resources that constitute a common resource pool.
  • 57. Storage Consistency  Strict Consistency  Eventual Consistency
  • 58. Strict Consistency  A storage offering usually consists of multiple replicas to ensure fault tolerance. It is of major importance that the consistency of the data contained in these replicas is pertained at all times while the performance is of secondary importance.  The highest level of consistency is granted if all replicas are updated if the data contained by them is altered. However, this would mean that the availability of the overall storage solution is decreased drastically. It has to be ensured that it is available even if not all replicas are available, but still the correct version of the data is read.
  • 59. Eventual Consistency  Eventually consistent data storage allows reducing data consistency to increase availability and performance, since the impact of network partitioning is reduced and fewer replicas have to be accessed during read and write operation.  While strictly consistent databases ensure that always at least one of the current version is read, eventually consistent databases allow that obsolete versions may also be read.  This increases the availability of the storage offering since only one replica has to be available to successfully execute a read operation.
  • 61. CAP (Consistency, Availability, Partition) Theorem At most two of these properties for any shared-data system Consistency + Availability C A • High data integrity P • Single site, cluster database, LDAP, xFS file system, etc. • 2-phase commit, data replication, etc. Consistency + Partition C A • Distributed database, distributed locking, etc. P • Pessimistic locking, minority partition unavailable, etc. Availability + Partition C A • High scalability • Distributed cache, DNS, etc. P • Optimistic locking, expiration/leases, etc.
  • 62. Hybrid architectures • Scale-out (horizontal)  Scale-up (vertical) – BASE: Basically Available, Soft state,  ACID: Atomicity, Consistency, Eventually consistent Isolation, Durability  availability first; best effort – focus on “commit”  aggressive (optimistic) – conservative (pessimistic)  transactional – shared nothing  favor accuracy/consistency – favor extreme size  e.g., BI & analytics, financial – e.g., user requests, data collection & processing, etc. processing, etc. Most distributed systems employ both approaches
  • 63. Storage Services  Relations Data Storage  Blob Data Storage  Block Data Storage  NoSQL Storage
  • 64. Relational Data Store  An application uses a central database for storing data elements and performs complex queries on them
  • 65. Blob Storage  A distributed application needs to manage large data elements, such as virtual server images or videos, which are too large for traditional databases.  In a distributed application data elements must be made available to all application components and to distributed users. Access to the data needs to be performed in a standardized fashion and access control has to be established.  Organize the data elements in a folder hierarchy similar to a traditional file system. Give each data element a unique identifier that can be used to access it over a network. Also, establish access control mechanisms.
  • 66. Block Storage  Resources in clouds are often unreliable (low available compute nodes).  Therefore, the data that they access locally shall in fact be stored in a high available central data store. This way, if a server fails the data is not lost, but a new server can be started to use the secured data.  Offer data elements in a central storage that can be accessed by distributed servers and integrate them as local drives.
  • 67. NoSQL Storage  Need to handle very large amounts of data and also need to be adjusted to new user demands flexibly.  Database solution is required that focuses on scaling out rather than on optimizing the use of a single resource and that can adjust flexibly to changes of the data structure.  Use a schema-free storage solution, with limited query capabilities to enable extreme scale-out through easy data replication.
  • 68. Communication Services  Message Oriented Middleware  Reliable Messaging  Exactly Once Delivery  At least Once Delivery
  • 69. Message-oriented middleware  Different applications usually use different languages, data formats, and technology platforms. When one application (component) needs to exchange information with another one, the format of the target application has to be respected.  Sending messages directly to the target application results in a tight coupling of sender and receiver since format changes directly affect both implementations.  Connect applications through an intermediary, the message oriented middleware, that hides the complexity of addressing and availability of communication partners as well as supports transformation of different message formats.
  • 70. Reliable Messaging  The message transfer from one communication partner to the other is performed under transactional context.  Especially, this transaction subsumes the operation performed to store the messages in persistent storage.  Thus, if an error occurs during message receiving, sending, or processing the transaction can be compensated transferring the overall system back to a correct and consistent state.
  • 71. At-least once  The receiver of messages sends special acknowledge messages to the sender.  If the sender does not receive such an acknowledgement message in a given time frame it retransmits the message.  Thus, messages, which are lost due to communication errors, are still received eventually.  However, duplicate messages can occur, for example, if an acknowledgement message is lost.  To reduce the communication overhead, acknowledgement messages can be sent either after each individual message or after an agreed upon number of messages.
  • 72. Exaclty-once delivery  Whenever a message is created it is associated with a unique identifier.  This is used by a filtering component on the message path to delete duplicates.  It does so by storing the identifiers of messages it has already seen.  The identifiers of messages passing through this filtering component are then compared to the identifiers that have been recorded to identify and delete duplicates.

Editor's Notes

  1. Picture source: http://en.wikipedia.org/wiki/Amdahl%27s_law
  2. To build for big scale – use more of the same pieces, not bigger pieces; though a different approach may be needed