SlideShare a Scribd company logo
Natural Laws of Software Performance The changing face of performance optimization
Who Am I? Kendall Miller One of the Founders of Gibraltar Software Small Independent Software Vendor Founded in 2008 Developers of VistaDB and Gibraltar Engineers, not Sales People Enterprise Systems Architect & Developer since 1995 BSE in Computer Engineering, University of Illinois Urbana-Champaign (UIUC) Twitter: @KendallMiller
Traditional Performance Optimization Run suspect use cases and find hotspots Very Linear Finds unexpected framework performance issues Final Polishing Step
Algorithms and Asymptotics Asymptotic (or ‘Big Oh’) Notation Describes the growth rate of functions Answers the question… Does execution time of A grow faster or slower than B? The rules of asymptotic notation say A term of n^3 will tend to dominate a term of n^2 Therefore We can discount coefficients and lower order terms And so f(n) = 6n^2 + 2n + 3 + n^3 Can be expressed as O(n) = n^3
You Can’t Optimize out of Trouble
So Where Are We?
Moore’s Law “The number of components in integrated circuits doubles every year” Components = Transistors
Processor Iron Triangle Clock Speed Power Speed of Light Size Complexity Manufacturing Process
A Core Explosion
Before you Leap into Optimizing… Algorithms are your first step Cores are a constant multiplier, algorithms provide exponential effect Everything we talk about today is ignored in O(n) Parallel processing on cores can get you a quick boost trading cost for modest boost Other tricks can get you more (and get more out of parallel)
Fork / Join Parallel Processing Split a problem into a number of independent problems Process each partition independently (potentially in parallel) Merge the results back together to get the final outcome (if necessary)
Fork / Join Examples Multicore Processors Server Farm Web Server Original Http servers literally forked a process for each request
Fork / Join in .NET System.Threading.ThreadPool Parallel.ForEach PLINQ Parallel.Invoke
Try it Now - PLINQ
Fork / Join Usage Tasks that can be broken into “large enough” chunks that are independent of each other Little shared state required to process Tasks with a low Join cost
Pipelines Partition a task based on stages of processing instead of data for processing Each stage of the pipeline processes independently (and typically concurrently) Stages are typically connected by queues Producer (prev stage) & Consumer (next stage)
Pipeline Examples Order Entry & Order Processing Classic Microprocessor Design Break the instruction processing into stages and process one stage per clock cycle GPU Design Combines Fork/Join with Pipeline
Pipeline Examples in .NET Not the ASP.NET processing Pipeline No parallelism/multithreading/queueing Stream Processing Map Reduce BlockingCollection<T> Gibraltar Agent
Pipeline Usage Significant shared state between data elements prevents decoupling them Linear processing requirements within parts of the workflow
Speculative Processing Isn’t there something you could be doing? Do the work now when you can, throw the results away if they aren’t useful
Speculative Processing Examples Microprocessor Branch Prediction Search Indexing
Speculative Processing Usage Shift work from a future, performance critical operation to an earlier one. Either always valid (never has to be rolled back) or easy to roll back
Latency – The Silent Killer The time for the first bit to get from here to there Typical LAN: 0.4ms
It’s the Law Speed of Light: 3x10^8 M/S About 0.240 seconds to Geosynchronous orbit and back About 1 foot per nanosecond 3GHz : 1/3rd ns period = 4 inches
5500 KM 18 ms TCP Socket Establish: 54ms London NewYork
Caching Save results of earlier work nearby where they are handy to use again later Cheat:  Don’t make the call Cheat More: Apply in front of anything that’s time consuming
Why Caching? Apps ask a lot of repeating questions. Stateless applications even more so Answers don’t change often Authoritative information is expensive Loading the world is impractical
Caching in Hardware Processor L1 Cache (typically same core) Processor L2 (shared by cores) Processor L3 (between proc & main RAM) Disk Controllers Disk Drives …
.NETCaching Examples ASP.NET Output Cache System.Web.Cache (ASP.NET only) AppFabric Cache
Go Asynchronous Delegate the latency to something that will notify you when it’s complete Do other useful stuff while waiting. Otherwise you’re just being efficient, not faster Maximize throughput by scheduling more work than can be done if there weren’t stalls
.NET Async Examples Standard Async IO Pattern .NET 4 Task<T> Combine with Queuing to maximize throughput even without parallelization
Visual Studio Async CTP async methods will compile to run asynchronously await forces method to stall execution until the async call completes before proceeding
Batching Get your money’s worth out of every latency hit Tradeoff storage for duration
General Batching Examples Shipping – Many packages on one truck Train travel TCP Sockets
Batching in Code SQL Connection Pooling HTTP Keep Alive DataSet / Entity Collections CSS Sprites
Optimistic Messaging Assume it’s all going to work out and just keep sending Be ready to step back & go another way when it doesn’t work out
Side Points Stateful interaction general increases the cost of latency Minimize Copying It takes blocking time to copy data, introducing latency Your Mileage May Vary Latency on a LAN can be dramatically affected by hardware and configuration
Critical Lessons Learned Algorithms, Algorithms, Algorithms Plan for Latency & Failure Explicitly Design for Parallelism

More Related Content

What's hot

Design principles of scalable, distributed systems
Design principles of scalable, distributed systemsDesign principles of scalable, distributed systems
Design principles of scalable, distributed systems
Tinniam V Ganesh (TV)
 
How to Build High Performance : WordPress
How to Build High Performance : WordPressHow to Build High Performance : WordPress
How to Build High Performance : WordPress
Dylan Burris
 
Webinar Slides: Geo-Distributed MySQL Clustering Done Right!
Webinar Slides: Geo-Distributed MySQL Clustering Done Right!Webinar Slides: Geo-Distributed MySQL Clustering Done Right!
Webinar Slides: Geo-Distributed MySQL Clustering Done Right!
Continuent
 
Blockchain for the DBA and Data Professional
Blockchain for the DBA and Data ProfessionalBlockchain for the DBA and Data Professional
Blockchain for the DBA and Data Professional
Karen Lopez
 
Architecting for the cloud cloud providers
Architecting for the cloud cloud providersArchitecting for the cloud cloud providers
Architecting for the cloud cloud providers
Len Bass
 
Presentation on Large Scale Data Management
Presentation on Large Scale Data ManagementPresentation on Large Scale Data Management
Presentation on Large Scale Data Management
Chris Bunch
 
10 Tricks to Ensure Your Oracle Coherence Cluster is Not a "Black Box" in Pro...
10 Tricks to Ensure Your Oracle Coherence Cluster is Not a "Black Box" in Pro...10 Tricks to Ensure Your Oracle Coherence Cluster is Not a "Black Box" in Pro...
10 Tricks to Ensure Your Oracle Coherence Cluster is Not a "Black Box" in Pro...
SL Corporation
 
Blockchain for the DBA and Data Professional
Blockchain for the DBA and Data ProfessionalBlockchain for the DBA and Data Professional
Blockchain for the DBA and Data Professional
Karen Lopez
 
Stream Collections - Scala Days
Stream Collections - Scala DaysStream Collections - Scala Days
Stream Collections - Scala Days
Greg Silin
 
Architecting for the cloud elasticity security
Architecting for the cloud elasticity securityArchitecting for the cloud elasticity security
Architecting for the cloud elasticity security
Len Bass
 
Kafka Summit SF 2017 - Running Kafka for Maximum Pain
Kafka Summit SF 2017 - Running Kafka for Maximum PainKafka Summit SF 2017 - Running Kafka for Maximum Pain
Kafka Summit SF 2017 - Running Kafka for Maximum Pain
confluent
 
Mtc learnings from isv & enterprise interaction
Mtc learnings from isv & enterprise  interactionMtc learnings from isv & enterprise  interaction
Mtc learnings from isv & enterprise interaction
Govind Kanshi
 
Mtc learnings from isv & enterprise (dated - Dec -2014)
Mtc learnings from isv & enterprise (dated - Dec -2014)Mtc learnings from isv & enterprise (dated - Dec -2014)
Mtc learnings from isv & enterprise (dated - Dec -2014)
Govind Kanshi
 
Production debugging web applications
Production debugging web applicationsProduction debugging web applications
Production debugging web applications
Ido Flatow
 
Dashboard project.
Dashboard project.Dashboard project.
Dashboard project.
ratankadam
 
Distributed systems and consistency
Distributed systems and consistencyDistributed systems and consistency
Distributed systems and consistency
seldo
 
Designing large scale distributed systems
Designing large scale distributed systemsDesigning large scale distributed systems
Designing large scale distributed systems
Ashwani Priyedarshi
 
Optimal Azure Database Development by Karel Coenye
 Optimal Azure Database Development by Karel Coenye Optimal Azure Database Development by Karel Coenye
Optimal Azure Database Development by Karel Coenye
ITProceed
 
Agile experiments in Machine Learning with F#
Agile experiments in Machine Learning with F#Agile experiments in Machine Learning with F#
Agile experiments in Machine Learning with F#
J On The Beach
 
Error in hadoop
Error in hadoopError in hadoop
Error in hadoop
Len Bass
 

What's hot (20)

Design principles of scalable, distributed systems
Design principles of scalable, distributed systemsDesign principles of scalable, distributed systems
Design principles of scalable, distributed systems
 
How to Build High Performance : WordPress
How to Build High Performance : WordPressHow to Build High Performance : WordPress
How to Build High Performance : WordPress
 
Webinar Slides: Geo-Distributed MySQL Clustering Done Right!
Webinar Slides: Geo-Distributed MySQL Clustering Done Right!Webinar Slides: Geo-Distributed MySQL Clustering Done Right!
Webinar Slides: Geo-Distributed MySQL Clustering Done Right!
 
Blockchain for the DBA and Data Professional
Blockchain for the DBA and Data ProfessionalBlockchain for the DBA and Data Professional
Blockchain for the DBA and Data Professional
 
Architecting for the cloud cloud providers
Architecting for the cloud cloud providersArchitecting for the cloud cloud providers
Architecting for the cloud cloud providers
 
Presentation on Large Scale Data Management
Presentation on Large Scale Data ManagementPresentation on Large Scale Data Management
Presentation on Large Scale Data Management
 
10 Tricks to Ensure Your Oracle Coherence Cluster is Not a "Black Box" in Pro...
10 Tricks to Ensure Your Oracle Coherence Cluster is Not a "Black Box" in Pro...10 Tricks to Ensure Your Oracle Coherence Cluster is Not a "Black Box" in Pro...
10 Tricks to Ensure Your Oracle Coherence Cluster is Not a "Black Box" in Pro...
 
Blockchain for the DBA and Data Professional
Blockchain for the DBA and Data ProfessionalBlockchain for the DBA and Data Professional
Blockchain for the DBA and Data Professional
 
Stream Collections - Scala Days
Stream Collections - Scala DaysStream Collections - Scala Days
Stream Collections - Scala Days
 
Architecting for the cloud elasticity security
Architecting for the cloud elasticity securityArchitecting for the cloud elasticity security
Architecting for the cloud elasticity security
 
Kafka Summit SF 2017 - Running Kafka for Maximum Pain
Kafka Summit SF 2017 - Running Kafka for Maximum PainKafka Summit SF 2017 - Running Kafka for Maximum Pain
Kafka Summit SF 2017 - Running Kafka for Maximum Pain
 
Mtc learnings from isv & enterprise interaction
Mtc learnings from isv & enterprise  interactionMtc learnings from isv & enterprise  interaction
Mtc learnings from isv & enterprise interaction
 
Mtc learnings from isv & enterprise (dated - Dec -2014)
Mtc learnings from isv & enterprise (dated - Dec -2014)Mtc learnings from isv & enterprise (dated - Dec -2014)
Mtc learnings from isv & enterprise (dated - Dec -2014)
 
Production debugging web applications
Production debugging web applicationsProduction debugging web applications
Production debugging web applications
 
Dashboard project.
Dashboard project.Dashboard project.
Dashboard project.
 
Distributed systems and consistency
Distributed systems and consistencyDistributed systems and consistency
Distributed systems and consistency
 
Designing large scale distributed systems
Designing large scale distributed systemsDesigning large scale distributed systems
Designing large scale distributed systems
 
Optimal Azure Database Development by Karel Coenye
 Optimal Azure Database Development by Karel Coenye Optimal Azure Database Development by Karel Coenye
Optimal Azure Database Development by Karel Coenye
 
Agile experiments in Machine Learning with F#
Agile experiments in Machine Learning with F#Agile experiments in Machine Learning with F#
Agile experiments in Machine Learning with F#
 
Error in hadoop
Error in hadoopError in hadoop
Error in hadoop
 

Viewers also liked

ASFWS 2013 - Advances in secure (ASP).NET development – break the hackers’ sp...
ASFWS 2013 - Advances in secure (ASP).NET development – break the hackers’ sp...ASFWS 2013 - Advances in secure (ASP).NET development – break the hackers’ sp...
ASFWS 2013 - Advances in secure (ASP).NET development – break the hackers’ sp...
Cyber Security Alliance
 
Chapter 09
Chapter 09Chapter 09
Chapter 09
Terry Yoast
 
Top 5 .NET Challenges, Performance Monitoring Tips & Tricks
Top 5 .NET Challenges, Performance Monitoring Tips & TricksTop 5 .NET Challenges, Performance Monitoring Tips & Tricks
Top 5 .NET Challenges, Performance Monitoring Tips & Tricks
AppDynamics
 
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)
OWASP Ottawa
 
Alfresco Security Best Practices 2014
Alfresco Security Best Practices 2014Alfresco Security Best Practices 2014
Alfresco Security Best Practices 2014
Toni de la Fuente
 
Zabbix Performance Tuning
Zabbix Performance TuningZabbix Performance Tuning
Zabbix Performance Tuning
Ricardo Santos
 
Under the Hood: Monitoring Azure and .NET - AppSphere16
Under the Hood: Monitoring Azure and .NET - AppSphere16Under the Hood: Monitoring Azure and .NET - AppSphere16
Under the Hood: Monitoring Azure and .NET - AppSphere16
AppDynamics
 
Call quality
Call qualityCall quality
An Introduction to Performance Testing
An Introduction to Performance TestingAn Introduction to Performance Testing
An Introduction to Performance Testing
SWAAM Tech
 
Introduction to performance testing
Introduction to performance testingIntroduction to performance testing
Introduction to performance testing
Tharinda Liyanage
 
Operations and-maintenance-best-practices
Operations and-maintenance-best-practicesOperations and-maintenance-best-practices
Operations and-maintenance-best-practices
Nikhil Nangia
 
Audit Checklist for Information Systems
Audit Checklist for Information SystemsAudit Checklist for Information Systems
Audit Checklist for Information Systems
Ahmad Tariq Bhatti
 

Viewers also liked (12)

ASFWS 2013 - Advances in secure (ASP).NET development – break the hackers’ sp...
ASFWS 2013 - Advances in secure (ASP).NET development – break the hackers’ sp...ASFWS 2013 - Advances in secure (ASP).NET development – break the hackers’ sp...
ASFWS 2013 - Advances in secure (ASP).NET development – break the hackers’ sp...
 
Chapter 09
Chapter 09Chapter 09
Chapter 09
 
Top 5 .NET Challenges, Performance Monitoring Tips & Tricks
Top 5 .NET Challenges, Performance Monitoring Tips & TricksTop 5 .NET Challenges, Performance Monitoring Tips & Tricks
Top 5 .NET Challenges, Performance Monitoring Tips & Tricks
 
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)
Security Code Review for .NET - Sherif Koussa (OWASP Ottawa)
 
Alfresco Security Best Practices 2014
Alfresco Security Best Practices 2014Alfresco Security Best Practices 2014
Alfresco Security Best Practices 2014
 
Zabbix Performance Tuning
Zabbix Performance TuningZabbix Performance Tuning
Zabbix Performance Tuning
 
Under the Hood: Monitoring Azure and .NET - AppSphere16
Under the Hood: Monitoring Azure and .NET - AppSphere16Under the Hood: Monitoring Azure and .NET - AppSphere16
Under the Hood: Monitoring Azure and .NET - AppSphere16
 
Call quality
Call qualityCall quality
Call quality
 
An Introduction to Performance Testing
An Introduction to Performance TestingAn Introduction to Performance Testing
An Introduction to Performance Testing
 
Introduction to performance testing
Introduction to performance testingIntroduction to performance testing
Introduction to performance testing
 
Operations and-maintenance-best-practices
Operations and-maintenance-best-practicesOperations and-maintenance-best-practices
Operations and-maintenance-best-practices
 
Audit Checklist for Information Systems
Audit Checklist for Information SystemsAudit Checklist for Information Systems
Audit Checklist for Information Systems
 

Similar to Natural Laws of Software Performance

Super Sizing Youtube with Python
Super Sizing Youtube with PythonSuper Sizing Youtube with Python
Super Sizing Youtube with Python
didip
 
A Practical Event Driven Model
A Practical Event Driven ModelA Practical Event Driven Model
A Practical Event Driven Model
Xi Wu
 
ConFoo 2017: Introduction to performance optimization of .NET web apps
ConFoo 2017: Introduction to performance optimization of .NET web appsConFoo 2017: Introduction to performance optimization of .NET web apps
ConFoo 2017: Introduction to performance optimization of .NET web apps
Pierre-Luc Maheu
 
Scalable Apache for Beginners
Scalable Apache for BeginnersScalable Apache for Beginners
Scalable Apache for Beginners
webhostingguy
 
Advanced web application architecture - Talk
Advanced web application architecture - TalkAdvanced web application architecture - Talk
Advanced web application architecture - Talk
Matthias Noback
 
Ajax Performance
Ajax PerformanceAjax Performance
Ajax Performance
kaven yan
 
Performance Analysis of Idle Programs
Performance Analysis of Idle ProgramsPerformance Analysis of Idle Programs
Performance Analysis of Idle Programs
greenwop
 
High Performance Coding2.pptx
High Performance Coding2.pptxHigh Performance Coding2.pptx
High Performance Coding2.pptx
ShymmaaQadoom1
 
Become a Performance Diagnostics Hero
Become a Performance Diagnostics HeroBecome a Performance Diagnostics Hero
Become a Performance Diagnostics Hero
TechWell
 
Develop in ludicrous mode with azure serverless
Develop in ludicrous mode with azure serverlessDevelop in ludicrous mode with azure serverless
Develop in ludicrous mode with azure serverless
Lalit Kale
 
Overview Of Parallel Development - Ericnel
Overview Of Parallel Development -  EricnelOverview Of Parallel Development -  Ericnel
Overview Of Parallel Development - Ericnel
ukdpe
 
5.7 Parallel Processing - Reactive Programming.pdf.pptx
5.7 Parallel Processing - Reactive Programming.pdf.pptx5.7 Parallel Processing - Reactive Programming.pdf.pptx
5.7 Parallel Processing - Reactive Programming.pdf.pptx
MohamedBilal73
 
Multithreading 101
Multithreading 101Multithreading 101
Multithreading 101
Tim Penhey
 
Understanding the Single Thread Event Loop
Understanding the Single Thread Event LoopUnderstanding the Single Thread Event Loop
Understanding the Single Thread Event Loop
TorontoNodeJS
 
Making it fast: Zotonic & Performance
Making it fast: Zotonic & PerformanceMaking it fast: Zotonic & Performance
Making it fast: Zotonic & Performance
Arjan
 
Handling Data in Mega Scale Systems
Handling Data in Mega Scale SystemsHandling Data in Mega Scale Systems
Handling Data in Mega Scale Systems
Directi Group
 
Performance Oriented Design
Performance Oriented DesignPerformance Oriented Design
Performance Oriented Design
Rodrigo Campos
 
Big Data Day LA 2016/ Big Data Track - Portable Stream and Batch Processing w...
Big Data Day LA 2016/ Big Data Track - Portable Stream and Batch Processing w...Big Data Day LA 2016/ Big Data Track - Portable Stream and Batch Processing w...
Big Data Day LA 2016/ Big Data Track - Portable Stream and Batch Processing w...
Data Con LA
 
Scaling Streaming - Concepts, Research, Goals
Scaling Streaming - Concepts, Research, GoalsScaling Streaming - Concepts, Research, Goals
Scaling Streaming - Concepts, Research, Goals
kamaelian
 
Building a Database for the End of the World
Building a Database for the End of the WorldBuilding a Database for the End of the World
Building a Database for the End of the World
jhugg
 

Similar to Natural Laws of Software Performance (20)

Super Sizing Youtube with Python
Super Sizing Youtube with PythonSuper Sizing Youtube with Python
Super Sizing Youtube with Python
 
A Practical Event Driven Model
A Practical Event Driven ModelA Practical Event Driven Model
A Practical Event Driven Model
 
ConFoo 2017: Introduction to performance optimization of .NET web apps
ConFoo 2017: Introduction to performance optimization of .NET web appsConFoo 2017: Introduction to performance optimization of .NET web apps
ConFoo 2017: Introduction to performance optimization of .NET web apps
 
Scalable Apache for Beginners
Scalable Apache for BeginnersScalable Apache for Beginners
Scalable Apache for Beginners
 
Advanced web application architecture - Talk
Advanced web application architecture - TalkAdvanced web application architecture - Talk
Advanced web application architecture - Talk
 
Ajax Performance
Ajax PerformanceAjax Performance
Ajax Performance
 
Performance Analysis of Idle Programs
Performance Analysis of Idle ProgramsPerformance Analysis of Idle Programs
Performance Analysis of Idle Programs
 
High Performance Coding2.pptx
High Performance Coding2.pptxHigh Performance Coding2.pptx
High Performance Coding2.pptx
 
Become a Performance Diagnostics Hero
Become a Performance Diagnostics HeroBecome a Performance Diagnostics Hero
Become a Performance Diagnostics Hero
 
Develop in ludicrous mode with azure serverless
Develop in ludicrous mode with azure serverlessDevelop in ludicrous mode with azure serverless
Develop in ludicrous mode with azure serverless
 
Overview Of Parallel Development - Ericnel
Overview Of Parallel Development -  EricnelOverview Of Parallel Development -  Ericnel
Overview Of Parallel Development - Ericnel
 
5.7 Parallel Processing - Reactive Programming.pdf.pptx
5.7 Parallel Processing - Reactive Programming.pdf.pptx5.7 Parallel Processing - Reactive Programming.pdf.pptx
5.7 Parallel Processing - Reactive Programming.pdf.pptx
 
Multithreading 101
Multithreading 101Multithreading 101
Multithreading 101
 
Understanding the Single Thread Event Loop
Understanding the Single Thread Event LoopUnderstanding the Single Thread Event Loop
Understanding the Single Thread Event Loop
 
Making it fast: Zotonic & Performance
Making it fast: Zotonic & PerformanceMaking it fast: Zotonic & Performance
Making it fast: Zotonic & Performance
 
Handling Data in Mega Scale Systems
Handling Data in Mega Scale SystemsHandling Data in Mega Scale Systems
Handling Data in Mega Scale Systems
 
Performance Oriented Design
Performance Oriented DesignPerformance Oriented Design
Performance Oriented Design
 
Big Data Day LA 2016/ Big Data Track - Portable Stream and Batch Processing w...
Big Data Day LA 2016/ Big Data Track - Portable Stream and Batch Processing w...Big Data Day LA 2016/ Big Data Track - Portable Stream and Batch Processing w...
Big Data Day LA 2016/ Big Data Track - Portable Stream and Batch Processing w...
 
Scaling Streaming - Concepts, Research, Goals
Scaling Streaming - Concepts, Research, GoalsScaling Streaming - Concepts, Research, Goals
Scaling Streaming - Concepts, Research, Goals
 
Building a Database for the End of the World
Building a Database for the End of the WorldBuilding a Database for the End of the World
Building a Database for the End of the World
 

More from Gibraltar Software

Introducing Agile to the Enterprise
Introducing Agile to the EnterpriseIntroducing Agile to the Enterprise
Introducing Agile to the Enterprise
Gibraltar Software
 
The raspberry pi – building an internet radio
The raspberry pi – building an internet radioThe raspberry pi – building an internet radio
The raspberry pi – building an internet radio
Gibraltar Software
 
Connecting to the raspberry pi from your laptop
Connecting to the raspberry pi from your laptopConnecting to the raspberry pi from your laptop
Connecting to the raspberry pi from your laptop
Gibraltar Software
 
Setting uptheraspberrypiusingberryboot
Setting uptheraspberrypiusingberrybootSetting uptheraspberrypiusingberryboot
Setting uptheraspberrypiusingberryboot
Gibraltar Software
 
Software Company Survival Guide
Software Company Survival GuideSoftware Company Survival Guide
Software Company Survival Guide
Gibraltar Software
 
A Year in the Life of an ISV
A Year in the Life of an ISVA Year in the Life of an ISV
A Year in the Life of an ISV
Gibraltar Software
 

More from Gibraltar Software (6)

Introducing Agile to the Enterprise
Introducing Agile to the EnterpriseIntroducing Agile to the Enterprise
Introducing Agile to the Enterprise
 
The raspberry pi – building an internet radio
The raspberry pi – building an internet radioThe raspberry pi – building an internet radio
The raspberry pi – building an internet radio
 
Connecting to the raspberry pi from your laptop
Connecting to the raspberry pi from your laptopConnecting to the raspberry pi from your laptop
Connecting to the raspberry pi from your laptop
 
Setting uptheraspberrypiusingberryboot
Setting uptheraspberrypiusingberrybootSetting uptheraspberrypiusingberryboot
Setting uptheraspberrypiusingberryboot
 
Software Company Survival Guide
Software Company Survival GuideSoftware Company Survival Guide
Software Company Survival Guide
 
A Year in the Life of an ISV
A Year in the Life of an ISVA Year in the Life of an ISV
A Year in the Life of an ISV
 

Recently uploaded

Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdfSession 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
UiPathCommunity
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
LizaNolte
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
christinelarrosa
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
Fwdays
 
Introducing BoxLang : A new JVM language for productivity and modularity!
Introducing BoxLang : A new JVM language for productivity and modularity!Introducing BoxLang : A new JVM language for productivity and modularity!
Introducing BoxLang : A new JVM language for productivity and modularity!
Ortus Solutions, Corp
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Ukraine
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
christinelarrosa
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
Safe Software
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
Fwdays
 
ScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking ReplicationScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking Replication
ScyllaDB
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
BibashShahi
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
zjhamm304
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 
Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
christinelarrosa
 
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin..."$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
Fwdays
 

Recently uploaded (20)

Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdfSession 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
"Scaling RAG Applications to serve millions of users",  Kevin Goedecke"Scaling RAG Applications to serve millions of users",  Kevin Goedecke
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
 
Introducing BoxLang : A new JVM language for productivity and modularity!
Introducing BoxLang : A new JVM language for productivity and modularity!Introducing BoxLang : A new JVM language for productivity and modularity!
Introducing BoxLang : A new JVM language for productivity and modularity!
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
 
ScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking ReplicationScyllaDB Tablets: Rethinking Replication
ScyllaDB Tablets: Rethinking Replication
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 
Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
 
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin..."$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
 

Natural Laws of Software Performance

  • 1. Natural Laws of Software Performance The changing face of performance optimization
  • 2. Who Am I? Kendall Miller One of the Founders of Gibraltar Software Small Independent Software Vendor Founded in 2008 Developers of VistaDB and Gibraltar Engineers, not Sales People Enterprise Systems Architect & Developer since 1995 BSE in Computer Engineering, University of Illinois Urbana-Champaign (UIUC) Twitter: @KendallMiller
  • 3. Traditional Performance Optimization Run suspect use cases and find hotspots Very Linear Finds unexpected framework performance issues Final Polishing Step
  • 4.
  • 5. Algorithms and Asymptotics Asymptotic (or ‘Big Oh’) Notation Describes the growth rate of functions Answers the question… Does execution time of A grow faster or slower than B? The rules of asymptotic notation say A term of n^3 will tend to dominate a term of n^2 Therefore We can discount coefficients and lower order terms And so f(n) = 6n^2 + 2n + 3 + n^3 Can be expressed as O(n) = n^3
  • 6. You Can’t Optimize out of Trouble
  • 7.
  • 9.
  • 10.
  • 11.
  • 12. Moore’s Law “The number of components in integrated circuits doubles every year” Components = Transistors
  • 13. Processor Iron Triangle Clock Speed Power Speed of Light Size Complexity Manufacturing Process
  • 15.
  • 16. Before you Leap into Optimizing… Algorithms are your first step Cores are a constant multiplier, algorithms provide exponential effect Everything we talk about today is ignored in O(n) Parallel processing on cores can get you a quick boost trading cost for modest boost Other tricks can get you more (and get more out of parallel)
  • 17. Fork / Join Parallel Processing Split a problem into a number of independent problems Process each partition independently (potentially in parallel) Merge the results back together to get the final outcome (if necessary)
  • 18. Fork / Join Examples Multicore Processors Server Farm Web Server Original Http servers literally forked a process for each request
  • 19. Fork / Join in .NET System.Threading.ThreadPool Parallel.ForEach PLINQ Parallel.Invoke
  • 20. Try it Now - PLINQ
  • 21. Fork / Join Usage Tasks that can be broken into “large enough” chunks that are independent of each other Little shared state required to process Tasks with a low Join cost
  • 22. Pipelines Partition a task based on stages of processing instead of data for processing Each stage of the pipeline processes independently (and typically concurrently) Stages are typically connected by queues Producer (prev stage) & Consumer (next stage)
  • 23. Pipeline Examples Order Entry & Order Processing Classic Microprocessor Design Break the instruction processing into stages and process one stage per clock cycle GPU Design Combines Fork/Join with Pipeline
  • 24. Pipeline Examples in .NET Not the ASP.NET processing Pipeline No parallelism/multithreading/queueing Stream Processing Map Reduce BlockingCollection<T> Gibraltar Agent
  • 25. Pipeline Usage Significant shared state between data elements prevents decoupling them Linear processing requirements within parts of the workflow
  • 26. Speculative Processing Isn’t there something you could be doing? Do the work now when you can, throw the results away if they aren’t useful
  • 27. Speculative Processing Examples Microprocessor Branch Prediction Search Indexing
  • 28. Speculative Processing Usage Shift work from a future, performance critical operation to an earlier one. Either always valid (never has to be rolled back) or easy to roll back
  • 29. Latency – The Silent Killer The time for the first bit to get from here to there Typical LAN: 0.4ms
  • 30. It’s the Law Speed of Light: 3x10^8 M/S About 0.240 seconds to Geosynchronous orbit and back About 1 foot per nanosecond 3GHz : 1/3rd ns period = 4 inches
  • 31. 5500 KM 18 ms TCP Socket Establish: 54ms London NewYork
  • 32.
  • 33. Caching Save results of earlier work nearby where they are handy to use again later Cheat: Don’t make the call Cheat More: Apply in front of anything that’s time consuming
  • 34. Why Caching? Apps ask a lot of repeating questions. Stateless applications even more so Answers don’t change often Authoritative information is expensive Loading the world is impractical
  • 35. Caching in Hardware Processor L1 Cache (typically same core) Processor L2 (shared by cores) Processor L3 (between proc & main RAM) Disk Controllers Disk Drives …
  • 36. .NETCaching Examples ASP.NET Output Cache System.Web.Cache (ASP.NET only) AppFabric Cache
  • 37. Go Asynchronous Delegate the latency to something that will notify you when it’s complete Do other useful stuff while waiting. Otherwise you’re just being efficient, not faster Maximize throughput by scheduling more work than can be done if there weren’t stalls
  • 38. .NET Async Examples Standard Async IO Pattern .NET 4 Task<T> Combine with Queuing to maximize throughput even without parallelization
  • 39. Visual Studio Async CTP async methods will compile to run asynchronously await forces method to stall execution until the async call completes before proceeding
  • 40. Batching Get your money’s worth out of every latency hit Tradeoff storage for duration
  • 41. General Batching Examples Shipping – Many packages on one truck Train travel TCP Sockets
  • 42. Batching in Code SQL Connection Pooling HTTP Keep Alive DataSet / Entity Collections CSS Sprites
  • 43. Optimistic Messaging Assume it’s all going to work out and just keep sending Be ready to step back & go another way when it doesn’t work out
  • 44. Side Points Stateful interaction general increases the cost of latency Minimize Copying It takes blocking time to copy data, introducing latency Your Mileage May Vary Latency on a LAN can be dramatically affected by hardware and configuration
  • 45. Critical Lessons Learned Algorithms, Algorithms, Algorithms Plan for Latency & Failure Explicitly Design for Parallelism
  • 46. Additional Information: Websites www.GibraltarSoftware.com www.eSymmetrix.com Follow Up Kendall.Miller@eSymmetrix.com Twitter: kendallmiller

Editor's Notes

  1. This is where your software ultimately runs