Natural Laws of Software Performance

Natural Laws of Software Performance
The changing face of performance optimization

Who Am I?
Kendall Miller
One of the Founders of Gibraltar Software
Small Independent Software Vendor Founded in 2008
Developers of VistaDB and Gibraltar
Engineers, not Sales People
Enterprise Systems Architect & Developer since 1995
BSE in Computer Engineering, University of Illinois Urbana-Champaign (UIUC)
@KendallMiller

Traditional Performance Optimization
Run suspect use cases and find hotspots
Very Linear
Finds unexpected framework performance issues
Final Polishing Step

Algorithms and Asymptotics
Asymptotic (or ‘Big Oh’) Notation
Describes the growth rate of functions
Answers the question…
Does execution time of A grow faster or slower than B?
The rules of asymptotic notation say
A term of n^3 will tend to dominate a term of n^2
Therefore
We can discount coefficients and lower order terms
And so f(n) = 6n^2 + 2n + 3 + n^3
Can be expressed as O(n) = n^3

You Can’t Optimize out of Trouble

So Where Are We?

Moore’s Law
“The number of components in integrated circuits doubles every year”
Components = Transistors

Processor Iron Triangle
Clock Speed
Power
Speed of Light
Size
Complexity
Manufacturing Process

A Core Explosion

Before you Leap into Optimizing…
Algorithms are your first step
Cores are a constant multiplier, algorithms provide exponential effect
Everything we talk about today is ignored in O(n)
Parallel processing on cores can get you a quick boost trading cost for modest boost
Other tricks can get you more (and get more out of parallel)

Fork / Join Parallel Processing
Split a problem into a number of independent problems
Process each partition independently (potentially in parallel)
Merge the results back together to get the final outcome (if necessary)

Fork / Join Examples
Multicore Processors
Server Farm
Web Server
Original Http servers literally forked a process for each request

Fork / Join in .NET
System.Threading.ThreadPool
Parallel.ForEach
PLINQ
Parallel.Invoke

Try it Now - PLINQ

Fork / Join Usage
Tasks that can be broken into “large enough” chunks that are independent of each other
Little shared state required to process
Tasks with a low Join cost

Pipelines
Partition a task based on stages of processing instead of data for processing
Each stage of the pipeline processes independently (and typically concurrently)
Stages are typically connected by queues
Producer (prev stage) & Consumer (next stage)

Pipeline Examples
Order Entry & Order Processing
Classic Microprocessor Design
Break the instruction processing into stages and process one stage per clock cycle
GPU Design
Combines Fork/Join with Pipeline

Pipeline Examples in .NET
Not the ASP.NET processing Pipeline
No parallelism/multithreading/queueing
Stream Processing
Map Reduce
BlockingCollection<T>
Gibraltar Agent

Pipeline Usage
Significant shared state between data elements prevents decoupling them
Linear processing requirements within parts of the workflow

It’s the Law
Speed of Light: 3x10^8 M/S
About 0.240 seconds to Geosynchronous orbit and back
About 1 foot per nanosecond
3GHz : 1/3rd ns period = 4 inches

Latency – The Silent Killer
The time for the first bit to get from here to there
Typical LAN: 0.4ms

5500 KM
18 ms
TCP Socket Establish: 54ms
London
NewYork

Caching
Save results of earlier work nearby where they are handy to use again later
Cheat: Don’t make the call
Cheat More: Apply in front of anything that’s time consuming

Why Caching?
Apps ask a lot of repeating questions.
Stateless applications even more so
Answers don’t change often
Authoritative information is expensive
Loading the world is impractical

Caching in Hardware
Processor L1 Cache (typically same core)
Processor L2 (shared by cores)
Processor L3 (between proc & main RAM)
Disk Controllers
Disk Drives
…

.NETCaching Examples
ASP.NET Output Cache
System.Web.Cache (ASP.NET only)
AppFabric Cache

Go Asynchronous
Delegate the latency to something that will notify you when it’s complete
Do other useful stuff while waiting.
Otherwise you’re just being efficient, not faster
Maximize throughput by scheduling more work than can be done if there weren’t stalls

.NET Async Examples
Standard Async IO Pattern
.NET 4 Task<T>
Combine with Queuing to maximize throughput even without parallelization

Visual Studio Async CTP
async methods will compile to run asynchronously
await forces method to stall execution until the async call completes before proceeding

Batching
Get your money’s worth out of every latency hit
Tradeoff storage for duration

General Batching Examples
SQL Connection Pooling
HTTP Keep Alive

.NET Batching
DataSet / Entity Sets

Optimistic Messaging
Assume it’s all going to work out and just keep sending

Side Points
Stateful interaction general increases the cost of latency
Minimize Copying
It takes blocking time to copy data, introducing latency
Your Mileage May Vary
Latency on a LAN can be dramatically affected by hardware and configuration

Critical Lessons Learned
Algorithms, Algorithms, Algorithms
Plan for Latency & Failure
Explicitly Design for Parallelism

Additional Information:
Websites
www.GibraltarSoftware.com
RockSolid.GibraltarSoftware.com
Follow Up
Kendall.Miller@eSymmetrix.com
Twitter: kendallmiller

Natural Laws of Software Performance

by Gibraltar Software

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 949

Statistics