SlideShare a Scribd company logo
1 of 28
Download to read offline
Scaling Mature Systems
© OPENJAW TECHNOLOGIES 20122
Teaching an
old dog new
tricks
© OPENJAW TECHNOLOGIES 20123
Change in environment
• Different usage or purpose
• Increase in products
• Increase in traffic
© OPENJAW TECHNOLOGIES 20124
Morten Jørgensen
Chief Architect
OpenJaw Technologies
Focus:
• Support product functionality
• Operational efficiency
• Scalability and fault tolerance
© OPENJAW TECHNOLOGIES 20125
System Load
ResourceConsumption
Scalability vs Performance
© OPENJAW TECHNOLOGIES 20126
• Distributed transactions
• Resource contention
• State/session management
• Deployment process
Common Scalability Blockers
Distributed Transactions
© OPENJAW TECHNOLOGIES 20128
• ACID principle
• Pessimistic
• Slow
• BASE principle
• Optimistic
• Fast
ACID vs. BASE
© OPENJAW TECHNOLOGIES 20129
• Come at a cost:
• Latency
• Performance
• Scalability
• Availability
• Software “failure” may make business sense!
Distributed Transactions
© OPENJAW TECHNOLOGIES 201210
Distributed Transactions
© OPENJAW TECHNOLOGIES 201211
• Reconfigure transaction management
• Implement failure detection
• Report or log failures
• Implement failure handling processes
• Convince business owner
Distributed Transactions
Resource Contention
© OPENJAW TECHNOLOGIES 201213
• Contention around internal and external systems
• Consumer experience limited by availability,
performance and scalability of these systems
• Break dependency by using asynchronous
processes
• Caching is the easiest form of asynchronous
decoupling
Resource Contention
© OPENJAW TECHNOLOGIES 201214
• Implement using SQL or no-SQL database
• Choose platform based on your needs
• Simple approach
• Build cache key
• Cache lookup
• Cache storage
Cache Implementation
© OPENJAW TECHNOLOGIES 201215
• Trade-off between data quality and performance
• Measure cache hit-rate
• Measure data quality
• Tune cache according to both!
Cache Tuning
State & Session Management
© OPENJAW TECHNOLOGIES 201217
• Make your application stateless
• Remove session data from the application
• Increase in user sessions will not impact
single node
• Add new nodes when traffic increases
Stateless Applications
© OPENJAW TECHNOLOGIES 201218
• Two options:
• Move to new platform
• Change existing platform
• J2EE conformance allowed us to use Tomcat
• Tomcat allowed session manager plugin
Stateless Applications
© OPENJAW TECHNOLOGIES 201219
Cassandra
KeySpace
Column Family
Key
Column
Schema
Table
Row
Column
SuperColumn
Column
Key
Session Manager
Session ID
Session Attribute Name
Session Object Value
© OPENJAW TECHNOLOGIES 201220
Cassandra Session Manager
Cassandra
“Ring”
StatelessLoadBalancer
Thrift
Application
Tomcat PLUGIN
Application
Tomcat PLUGIN
Application
Tomcat PLUGIN
© OPENJAW TECHNOLOGIES 201221
• Session object serialisation
• Session object synchronisation
• Network bandwidth
Cassandra Session Manager
Deployment Process
© OPENJAW TECHNOLOGIES 201223
• Your application is decoupled
• Your application is stateless
• You can add nodes when demand increases
• How quickly can you bring a new node up?
• Is your application an elastic resource?
Deployment Process
© OPENJAW TECHNOLOGIES 201224
• Remove everything but code!
• Extract session state
• Extract configuration
• Create a VM image as a template
• Instantiate VMs from the image on demand
Deployment Process
© OPENJAW TECHNOLOGIES 201225
Deployment Process
VM VM VMVM
Stateless Load Balancer
Client
Admin
Console
Results
© OPENJAW TECHNOLOGIES 201227
• Product built over 10 years
• Large solution with high complexity
• Made Internet scalable with small amount of
engineering resources
• Ability to scale is now a key selling point
Our Results
Thank you!

More Related Content

What's hot

Transform Your Cloud Validation Strategy from Cloudy to Clear
Transform Your Cloud Validation Strategy from Cloudy to ClearTransform Your Cloud Validation Strategy from Cloudy to Clear
Transform Your Cloud Validation Strategy from Cloudy to ClearTechWell
 
Amplexor Drupal for the Enterprise seminar - Acquia: dream it, Drupal it
Amplexor Drupal for the Enterprise seminar - Acquia: dream it, Drupal itAmplexor Drupal for the Enterprise seminar - Acquia: dream it, Drupal it
Amplexor Drupal for the Enterprise seminar - Acquia: dream it, Drupal itAmplexor
 
Webinar: Build Better Software: Removing the Constraints Slowing Dev/Test Tea...
Webinar: Build Better Software: Removing the Constraints Slowing Dev/Test Tea...Webinar: Build Better Software: Removing the Constraints Slowing Dev/Test Tea...
Webinar: Build Better Software: Removing the Constraints Slowing Dev/Test Tea...Skytap Cloud
 
Top 10 DevOps Areas Need To Focus
Top 10 DevOps Areas Need To FocusTop 10 DevOps Areas Need To Focus
Top 10 DevOps Areas Need To Focusdevopsjourney
 
Testing Applications—For the Cloud and in the Cloud
Testing Applications—For the Cloud and in the CloudTesting Applications—For the Cloud and in the Cloud
Testing Applications—For the Cloud and in the CloudTechWell
 
Agile Development and DevOps in the Oracle Cloud
Agile Development and DevOps in the Oracle CloudAgile Development and DevOps in the Oracle Cloud
Agile Development and DevOps in the Oracle Cloudjeckels
 
Rundeck Office Hours: Best Practices for Access Control Policies
Rundeck Office Hours:  Best Practices for Access Control PoliciesRundeck Office Hours:  Best Practices for Access Control Policies
Rundeck Office Hours: Best Practices for Access Control PoliciesTraciMyers6
 
2014 July Webinar Modern DR Workshop
2014 July Webinar Modern DR Workshop2014 July Webinar Modern DR Workshop
2014 July Webinar Modern DR WorkshopBluelock
 
Moving from application automation to true DevOps by including the database
Moving from application automation to true DevOps by including the databaseMoving from application automation to true DevOps by including the database
Moving from application automation to true DevOps by including the databaseRed Gate Software
 
Multi-tenancy: A Core commercetools Differentiator
Multi-tenancy: A Core commercetools DifferentiatorMulti-tenancy: A Core commercetools Differentiator
Multi-tenancy: A Core commercetools DifferentiatorKelly Goetsch
 
Keynote: Architecting for Continuous Delivery (Pivotal Cloud Platform Roadshow)
Keynote: Architecting for Continuous Delivery (Pivotal Cloud Platform Roadshow)Keynote: Architecting for Continuous Delivery (Pivotal Cloud Platform Roadshow)
Keynote: Architecting for Continuous Delivery (Pivotal Cloud Platform Roadshow)VMware Tanzu
 
What is Zenoss as a Service?
What is Zenoss as a Service? What is Zenoss as a Service?
What is Zenoss as a Service? Zenoss
 
Service Mesh: Two Big Words But Do You Need It?
Service Mesh: Two Big Words But Do You Need It?Service Mesh: Two Big Words But Do You Need It?
Service Mesh: Two Big Words But Do You Need It?DevOps.com
 
Driving Enterprise Architecture Redesign: Cloud-Native Platforms, APIs, and D...
Driving Enterprise Architecture Redesign: Cloud-Native Platforms, APIs, and D...Driving Enterprise Architecture Redesign: Cloud-Native Platforms, APIs, and D...
Driving Enterprise Architecture Redesign: Cloud-Native Platforms, APIs, and D...Chris Haddad
 
Oracle BPM Suite Development: Getting Started
Oracle BPM Suite Development: Getting StartedOracle BPM Suite Development: Getting Started
Oracle BPM Suite Development: Getting StartedRevelation Technologies
 
AMIS 25: DevOps Best Practice for Oracle SOA and BPM
AMIS 25: DevOps Best Practice for Oracle SOA and BPMAMIS 25: DevOps Best Practice for Oracle SOA and BPM
AMIS 25: DevOps Best Practice for Oracle SOA and BPMMatt Wright
 
Cloud Testing: The Future of software Testing
Cloud Testing: The Future of software TestingCloud Testing: The Future of software Testing
Cloud Testing: The Future of software TestingBugRaptors
 
Delivering New Visibility and Analytics for IT Operations
Delivering New Visibility and Analytics for IT OperationsDelivering New Visibility and Analytics for IT Operations
Delivering New Visibility and Analytics for IT OperationsGabrielle Knowles
 
EDA for QAs
EDA for QAsEDA for QAs
EDA for QAsvodQA
 

What's hot (20)

Planning open stack-poc
Planning open stack-pocPlanning open stack-poc
Planning open stack-poc
 
Transform Your Cloud Validation Strategy from Cloudy to Clear
Transform Your Cloud Validation Strategy from Cloudy to ClearTransform Your Cloud Validation Strategy from Cloudy to Clear
Transform Your Cloud Validation Strategy from Cloudy to Clear
 
Amplexor Drupal for the Enterprise seminar - Acquia: dream it, Drupal it
Amplexor Drupal for the Enterprise seminar - Acquia: dream it, Drupal itAmplexor Drupal for the Enterprise seminar - Acquia: dream it, Drupal it
Amplexor Drupal for the Enterprise seminar - Acquia: dream it, Drupal it
 
Webinar: Build Better Software: Removing the Constraints Slowing Dev/Test Tea...
Webinar: Build Better Software: Removing the Constraints Slowing Dev/Test Tea...Webinar: Build Better Software: Removing the Constraints Slowing Dev/Test Tea...
Webinar: Build Better Software: Removing the Constraints Slowing Dev/Test Tea...
 
Top 10 DevOps Areas Need To Focus
Top 10 DevOps Areas Need To FocusTop 10 DevOps Areas Need To Focus
Top 10 DevOps Areas Need To Focus
 
Testing Applications—For the Cloud and in the Cloud
Testing Applications—For the Cloud and in the CloudTesting Applications—For the Cloud and in the Cloud
Testing Applications—For the Cloud and in the Cloud
 
Agile Development and DevOps in the Oracle Cloud
Agile Development and DevOps in the Oracle CloudAgile Development and DevOps in the Oracle Cloud
Agile Development and DevOps in the Oracle Cloud
 
Rundeck Office Hours: Best Practices for Access Control Policies
Rundeck Office Hours:  Best Practices for Access Control PoliciesRundeck Office Hours:  Best Practices for Access Control Policies
Rundeck Office Hours: Best Practices for Access Control Policies
 
2014 July Webinar Modern DR Workshop
2014 July Webinar Modern DR Workshop2014 July Webinar Modern DR Workshop
2014 July Webinar Modern DR Workshop
 
Moving from application automation to true DevOps by including the database
Moving from application automation to true DevOps by including the databaseMoving from application automation to true DevOps by including the database
Moving from application automation to true DevOps by including the database
 
Multi-tenancy: A Core commercetools Differentiator
Multi-tenancy: A Core commercetools DifferentiatorMulti-tenancy: A Core commercetools Differentiator
Multi-tenancy: A Core commercetools Differentiator
 
Keynote: Architecting for Continuous Delivery (Pivotal Cloud Platform Roadshow)
Keynote: Architecting for Continuous Delivery (Pivotal Cloud Platform Roadshow)Keynote: Architecting for Continuous Delivery (Pivotal Cloud Platform Roadshow)
Keynote: Architecting for Continuous Delivery (Pivotal Cloud Platform Roadshow)
 
What is Zenoss as a Service?
What is Zenoss as a Service? What is Zenoss as a Service?
What is Zenoss as a Service?
 
Service Mesh: Two Big Words But Do You Need It?
Service Mesh: Two Big Words But Do You Need It?Service Mesh: Two Big Words But Do You Need It?
Service Mesh: Two Big Words But Do You Need It?
 
Driving Enterprise Architecture Redesign: Cloud-Native Platforms, APIs, and D...
Driving Enterprise Architecture Redesign: Cloud-Native Platforms, APIs, and D...Driving Enterprise Architecture Redesign: Cloud-Native Platforms, APIs, and D...
Driving Enterprise Architecture Redesign: Cloud-Native Platforms, APIs, and D...
 
Oracle BPM Suite Development: Getting Started
Oracle BPM Suite Development: Getting StartedOracle BPM Suite Development: Getting Started
Oracle BPM Suite Development: Getting Started
 
AMIS 25: DevOps Best Practice for Oracle SOA and BPM
AMIS 25: DevOps Best Practice for Oracle SOA and BPMAMIS 25: DevOps Best Practice for Oracle SOA and BPM
AMIS 25: DevOps Best Practice for Oracle SOA and BPM
 
Cloud Testing: The Future of software Testing
Cloud Testing: The Future of software TestingCloud Testing: The Future of software Testing
Cloud Testing: The Future of software Testing
 
Delivering New Visibility and Analytics for IT Operations
Delivering New Visibility and Analytics for IT OperationsDelivering New Visibility and Analytics for IT Operations
Delivering New Visibility and Analytics for IT Operations
 
EDA for QAs
EDA for QAsEDA for QAs
EDA for QAs
 

Similar to Scaling mature systems

Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...Microsoft Décideurs IT
 
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...Microsoft Décideurs IT
 
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...Microsoft Technet France
 
Enterprise DevOps in the Age of Docker & Microservices
Enterprise DevOps in the Age of Docker & MicroservicesEnterprise DevOps in the Age of Docker & Microservices
Enterprise DevOps in the Age of Docker & MicroservicesXebiaLabs
 
Enterprise Monitoring 2018: Converged Application & Infrastructure Monitoring...
Enterprise Monitoring 2018: Converged Application & Infrastructure Monitoring...Enterprise Monitoring 2018: Converged Application & Infrastructure Monitoring...
Enterprise Monitoring 2018: Converged Application & Infrastructure Monitoring...eG Innovations
 
Build and Manage Hadoop & Oracle NoSQL DB Solutions- Impetus Webinar
Build and Manage Hadoop & Oracle NoSQL DB Solutions- Impetus WebinarBuild and Manage Hadoop & Oracle NoSQL DB Solutions- Impetus Webinar
Build and Manage Hadoop & Oracle NoSQL DB Solutions- Impetus WebinarImpetus Technologies
 
Hybrid Cloud Transformation Fast Track.pptx
Hybrid Cloud Transformation Fast Track.pptxHybrid Cloud Transformation Fast Track.pptx
Hybrid Cloud Transformation Fast Track.pptxzhunli4
 
Il paradigma DevOps e Continuous Delivery Automation
Il paradigma DevOps e Continuous Delivery Automation Il paradigma DevOps e Continuous Delivery Automation
Il paradigma DevOps e Continuous Delivery Automation HP Enterprise Italia
 
Securing the Cloud Native stack
Securing the Cloud Native stackSecuring the Cloud Native stack
Securing the Cloud Native stackHector Tapia
 
Building Cloud-Native Applications with a Container-Native SQL Database in th...
Building Cloud-Native Applications with a Container-Native SQL Database in th...Building Cloud-Native Applications with a Container-Native SQL Database in th...
Building Cloud-Native Applications with a Container-Native SQL Database in th...NuoDB
 
Securing the Cloud Native Stack
Securing the Cloud Native StackSecuring the Cloud Native Stack
Securing the Cloud Native StackApcera
 
Modernize Traditional Applications with Docker EE
Modernize Traditional Applications with Docker EEModernize Traditional Applications with Docker EE
Modernize Traditional Applications with Docker EERiedwaan Bassadien
 
1 architecture & design
1   architecture & design1   architecture & design
1 architecture & designMark Swarbrick
 
Get Loose! Microservices and Loosely Coupled Architectures
Get Loose! Microservices and Loosely Coupled Architectures Get Loose! Microservices and Loosely Coupled Architectures
Get Loose! Microservices and Loosely Coupled Architectures DevOps.com
 
Get Loose! Microservices and Loosely Coupled Architectures
Get Loose! Microservices and Loosely Coupled ArchitecturesGet Loose! Microservices and Loosely Coupled Architectures
Get Loose! Microservices and Loosely Coupled ArchitecturesDeborah Schalm
 
Migrate Oracle WebLogic Applications onto a Containerized Cloud Data Center
Migrate Oracle WebLogic Applications onto a Containerized Cloud Data CenterMigrate Oracle WebLogic Applications onto a Containerized Cloud Data Center
Migrate Oracle WebLogic Applications onto a Containerized Cloud Data CenterJingnan Zhou
 

Similar to Scaling mature systems (20)

Developer want change Ops want control - devops
Developer want change Ops want control - devopsDeveloper want change Ops want control - devops
Developer want change Ops want control - devops
 
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
 
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
 
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
Windows Server 2003 EOS : l'opportunité de repenser votre IT et mettre en pla...
 
Enterprise DevOps in the Age of Docker & Microservices
Enterprise DevOps in the Age of Docker & MicroservicesEnterprise DevOps in the Age of Docker & Microservices
Enterprise DevOps in the Age of Docker & Microservices
 
Enterprise Monitoring 2018: Converged Application & Infrastructure Monitoring...
Enterprise Monitoring 2018: Converged Application & Infrastructure Monitoring...Enterprise Monitoring 2018: Converged Application & Infrastructure Monitoring...
Enterprise Monitoring 2018: Converged Application & Infrastructure Monitoring...
 
Build and Manage Hadoop & Oracle NoSQL DB Solutions- Impetus Webinar
Build and Manage Hadoop & Oracle NoSQL DB Solutions- Impetus WebinarBuild and Manage Hadoop & Oracle NoSQL DB Solutions- Impetus Webinar
Build and Manage Hadoop & Oracle NoSQL DB Solutions- Impetus Webinar
 
Hybrid Cloud Transformation Fast Track.pptx
Hybrid Cloud Transformation Fast Track.pptxHybrid Cloud Transformation Fast Track.pptx
Hybrid Cloud Transformation Fast Track.pptx
 
Il paradigma DevOps e Continuous Delivery Automation
Il paradigma DevOps e Continuous Delivery Automation Il paradigma DevOps e Continuous Delivery Automation
Il paradigma DevOps e Continuous Delivery Automation
 
DevOps Case Studies
DevOps Case StudiesDevOps Case Studies
DevOps Case Studies
 
Forecast 2014: Cloud-Aware Applications
Forecast 2014: Cloud-Aware Applications Forecast 2014: Cloud-Aware Applications
Forecast 2014: Cloud-Aware Applications
 
Securing the Cloud Native stack
Securing the Cloud Native stackSecuring the Cloud Native stack
Securing the Cloud Native stack
 
Building Cloud-Native Applications with a Container-Native SQL Database in th...
Building Cloud-Native Applications with a Container-Native SQL Database in th...Building Cloud-Native Applications with a Container-Native SQL Database in th...
Building Cloud-Native Applications with a Container-Native SQL Database in th...
 
Securing the Cloud Native Stack
Securing the Cloud Native StackSecuring the Cloud Native Stack
Securing the Cloud Native Stack
 
Modernize Traditional Applications with Docker EE
Modernize Traditional Applications with Docker EEModernize Traditional Applications with Docker EE
Modernize Traditional Applications with Docker EE
 
1 architecture & design
1   architecture & design1   architecture & design
1 architecture & design
 
Get Loose! Microservices and Loosely Coupled Architectures
Get Loose! Microservices and Loosely Coupled Architectures Get Loose! Microservices and Loosely Coupled Architectures
Get Loose! Microservices and Loosely Coupled Architectures
 
Get Loose! Microservices and Loosely Coupled Architectures
Get Loose! Microservices and Loosely Coupled ArchitecturesGet Loose! Microservices and Loosely Coupled Architectures
Get Loose! Microservices and Loosely Coupled Architectures
 
Apex day 1.0 fastest route to cloud sept 2015_julian lane
Apex day 1.0 fastest route to cloud sept 2015_julian laneApex day 1.0 fastest route to cloud sept 2015_julian lane
Apex day 1.0 fastest route to cloud sept 2015_julian lane
 
Migrate Oracle WebLogic Applications onto a Containerized Cloud Data Center
Migrate Oracle WebLogic Applications onto a Containerized Cloud Data CenterMigrate Oracle WebLogic Applications onto a Containerized Cloud Data Center
Migrate Oracle WebLogic Applications onto a Containerized Cloud Data Center
 

Recently uploaded

Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 

Recently uploaded (20)

Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 

Scaling mature systems

  • 2. © OPENJAW TECHNOLOGIES 20122 Teaching an old dog new tricks
  • 3. © OPENJAW TECHNOLOGIES 20123 Change in environment • Different usage or purpose • Increase in products • Increase in traffic
  • 4. © OPENJAW TECHNOLOGIES 20124 Morten Jørgensen Chief Architect OpenJaw Technologies Focus: • Support product functionality • Operational efficiency • Scalability and fault tolerance
  • 5. © OPENJAW TECHNOLOGIES 20125 System Load ResourceConsumption Scalability vs Performance
  • 6. © OPENJAW TECHNOLOGIES 20126 • Distributed transactions • Resource contention • State/session management • Deployment process Common Scalability Blockers
  • 8. © OPENJAW TECHNOLOGIES 20128 • ACID principle • Pessimistic • Slow • BASE principle • Optimistic • Fast ACID vs. BASE
  • 9. © OPENJAW TECHNOLOGIES 20129 • Come at a cost: • Latency • Performance • Scalability • Availability • Software “failure” may make business sense! Distributed Transactions
  • 10. © OPENJAW TECHNOLOGIES 201210 Distributed Transactions
  • 11. © OPENJAW TECHNOLOGIES 201211 • Reconfigure transaction management • Implement failure detection • Report or log failures • Implement failure handling processes • Convince business owner Distributed Transactions
  • 13. © OPENJAW TECHNOLOGIES 201213 • Contention around internal and external systems • Consumer experience limited by availability, performance and scalability of these systems • Break dependency by using asynchronous processes • Caching is the easiest form of asynchronous decoupling Resource Contention
  • 14. © OPENJAW TECHNOLOGIES 201214 • Implement using SQL or no-SQL database • Choose platform based on your needs • Simple approach • Build cache key • Cache lookup • Cache storage Cache Implementation
  • 15. © OPENJAW TECHNOLOGIES 201215 • Trade-off between data quality and performance • Measure cache hit-rate • Measure data quality • Tune cache according to both! Cache Tuning
  • 16. State & Session Management
  • 17. © OPENJAW TECHNOLOGIES 201217 • Make your application stateless • Remove session data from the application • Increase in user sessions will not impact single node • Add new nodes when traffic increases Stateless Applications
  • 18. © OPENJAW TECHNOLOGIES 201218 • Two options: • Move to new platform • Change existing platform • J2EE conformance allowed us to use Tomcat • Tomcat allowed session manager plugin Stateless Applications
  • 19. © OPENJAW TECHNOLOGIES 201219 Cassandra KeySpace Column Family Key Column Schema Table Row Column SuperColumn Column Key Session Manager Session ID Session Attribute Name Session Object Value
  • 20. © OPENJAW TECHNOLOGIES 201220 Cassandra Session Manager Cassandra “Ring” StatelessLoadBalancer Thrift Application Tomcat PLUGIN Application Tomcat PLUGIN Application Tomcat PLUGIN
  • 21. © OPENJAW TECHNOLOGIES 201221 • Session object serialisation • Session object synchronisation • Network bandwidth Cassandra Session Manager
  • 23. © OPENJAW TECHNOLOGIES 201223 • Your application is decoupled • Your application is stateless • You can add nodes when demand increases • How quickly can you bring a new node up? • Is your application an elastic resource? Deployment Process
  • 24. © OPENJAW TECHNOLOGIES 201224 • Remove everything but code! • Extract session state • Extract configuration • Create a VM image as a template • Instantiate VMs from the image on demand Deployment Process
  • 25. © OPENJAW TECHNOLOGIES 201225 Deployment Process VM VM VMVM Stateless Load Balancer Client Admin Console
  • 27. © OPENJAW TECHNOLOGIES 201227 • Product built over 10 years • Large solution with high complexity • Made Internet scalable with small amount of engineering resources • Ability to scale is now a key selling point Our Results

Editor's Notes

  1. I am going to talk to you about software architecture and scalability.Most talks that I have attended, blogs or books that I have read on this subject make the same assumption:- You are developing a new system.- You don’t need to migrate existing functionality.- You have unlimited engineering resources.- This is typical for large corporates or academia.As software architects we have to deal with large mature systems, and it is obviously very expensive to rewrite these from scratch.The same holds for refactoring or migrating functionality to a new platform.As an architect, you may be given a lot of stick due to scalability issues with your system.So why don’t our systems scale?
  2. The most common reason is that the environment that your software runs in has changed.This can be due to many factors. The most obvious are: - The number of users or traffic has increased. - The number of products you are offering has increased.But also, you may have a case where the purpose of a software component has changed over time.Your marketing or sales department may have decided that your internal test tool is something they can sell.This has actually happened in every single job that I have had since college - the worst case being the first tool that I ever wrote (phone call scanner to be used as evidence in court cases).All these environment changes put new demands on your software platform.Today I will talk about how we can make our existing systems scalable.
  3. Focus for today will be on scalability.I will go through my interpretation of scalability, so that we share a common understanding of the term.
  4. Performance is a single point on this graph.Scalability is the shape of the graph.In this case we see a system that scales linearly, the resource requirements increase linearly with the load on the system.This doesn’t necessarily mean that the performance in terms of response times, etc. remains constant.This graph shows a system that isn’t scalable.Performance is better than our first system (green), but at a certain point the resource requirements skyrocket.This is the ideal scenario, where a system scales better than linear.
  5. Over the next 30 minutes I will take you through these common scalability blockers.There are quick wins in all of these areas, and I hope that what I present to you will help you make your system scale.
  6. Unless you want to argue about old vs. new Pink Floyd, ACID (Syd Barret) is bad and BASE (Roger Waters) is good.ACID principle:Data consistency is the highest priorityAvailability is less importantPessimistic approachComplex and large overheadBASE principle:Consistency is less importantAvailability is the highest priorityOptimistic approachSimple and fastApproach:Assume that data will be eventually consistent, ORBuild processes to handle consistency exceptions
  7. Latency – Overhead of coordinationPerformance – Sequential sequencing of operationsScalability – large resource requirements for single operationAvailability – operation fails if a single system failsAllowing your system to “fail” according to the traditional ACID methodology may actually make business sense. Do you really need your transactions to run as a single atomic operation?
  8. OpenJaw’s Internet Booking Engine manages 2-phase transactions across multiple external systems.Most of these external systems are outside of our control, and include travel suppliers like - Expedia, HotelBeds, GTA - Amadeus, Sabre, SITA - Avis/Hertz - Payment gateways - Confirmation e-mailCheckout of a single shopping cart requires interaction with all of these systems, but from the consumer’s perspective this is a single booking with a single payment.We have the capability of running these as a 2-phase transaction, where payment and booking is done as a single all-or-nothing process. This makes all bookings slower than what they have to be, due to sequential processing.In most cases all systems return with a positive response – maybe with the exception of payment where there may be a relatively high failure rate due to the consumer’s credit limit. So why would we want a process that is consistently slow in order to cater for the 2-3% of requests where there are errors?A better approach is to charge payment first, and then run all bookings in parallel. Then we build processes around failed bookings, which could include alerting a call center agent to ring back the customer and propose an alternative product.
  9. This is a summary of the changes we made to our transaction process.
  10. You will have contention around internal and external resources.
  11. It is very easy to build and integrate a cache.There is a range of potential platforms, and you must choose the one that suits your needs.In-memory databases like Redis offers extreme performance, but have limitations like size and failover.Distributed caches like Cassandra offer high scalability and failover, but do not perform as well as in-memory databases.SQL databases offer security in case the cached data is sensitive.Implementing basic caching done in three steps: - Build the cache key - Cache lookup - Cache storage
  12. A cache is a simple way of buying performance by sacrificing data freshness or quality.In order to effectively cache you must measure both how well your cache performs (hit/miss rate) but also how good the data quality is. The latter is very often missed or ignored.You may also need to measure these for various product types and variations. In our case we measure per product type, and route (for flights) or destination (for hotels).This allows us to properly tune the cache – and achieving the ideal balance between performance and data-quality.
  13. This is the theory!How do you do this in practice?In our case we had a stateless middleware platform that scaled linearly.The stateful front-end used standard J2EE session management, and became a barrier for scalability.J2EE sessions were accessed by our MVC platform, Java beans and JSPs.Changing how we managed sessions would mean a huge refactoring task, which would cost us months of engineering effort,
  14. There are two ways of attacking this problem: - You can renew or replace your platform, which means you will have to refactor your application - You can change how your existing platform manages sessions.Our front-end application conformed to the J2EE spec, which allowed us to move to Tomcat. Tomcat is open-source and allows custom session managers to be implemented.On any platform where session/state is accessed through a single interface, you have the possibility of changing how session/state is persisted and shared.
  15. Our custom Tomcat session manager uses Cassandra to store session data. This is a common approach in the PHP/LAMP world.Cassandra is the master for session data, and session data sits in the Tomcat session manager only for the duration of a single request.The session manager contains an in-memory cache (write-back or write-through) that exists for the lifetime of a single request.Allows use of a stateless (non-sticky) load balancer.Available as open-source: Google “tomcat Cassandra”
  16. These are considerations for any distributed session/state management!Session object serialization must perform well: - Ensure that you only serialize what you need. - Optimize serialization, for instance by serializing XML using Xerces/XalanSession objects cannot be synchronized using Java locking. You may have to rewrite some session objects to be inherently thread-safe.You may also have to build code to handle inconsistencies between concurrent Ajax requests.One common pitfall when creating scalable systems is assuming that the network is always available. With this session manager, network bandwidth may become an issue! Consider using 3 network segments: - One in front-of your application - One for the middle-tier or back-end - One for Tomcat-CassandraWe implemented tunable compression in the Tomcat session manager: Trade-off between CPU and network bandwidth.
  17. Making this happen isn’t that difficult!
  18. Your application should be just that – an application.All data must be extracted to external stores: - Most applications store product data in back-end systems/databases. - You should store your session state in external storage. - You should also store all your configuration in external storage.Use a single URI or environment variable to point to your configuration store. Your hypervisor will allow you to set this variable per application node.The configuration store then points your application to the appropriate back-end and session store.This allows a single VM image to be used across all your environments; live, pre-live, UAT, test, etc.