Miningfor Gold
How Browser,Network,andApplicationTraceMash-UpsCanBoost
ApplicationPerformance
Software Group
IntroducingDellSoftware
• QuestSoftware, Enstratius are the foundation
•Part of “World’s Largest Startup” with 110k people
•Sponsors at GlueCon and Velocity
•6 employees in Boulder (shameless hometown plug)
Software Group
“Mining” for performance“gold”is moredifficult
than it everwas.
Software Group
Our “units of deployment”may begetting a bit
simpler
But our systems aregetting much more
complex…
Software Group
Software Group
We used to shoot for stability in a controlled
environment
But wenow shoot for resiliencyin an ever
changing environment
Software Group
Software Group
How do we handle thiscomplexity?
Observe,Measure and Filter
Understand Intrinsic
Latencies
Understand the
Transaction Path
Understand User Behavior
Understand Resource
Contention
Improve Logical Complexity Tune, Balance, Scale
How are your tools andtechniques helping you dothis today?
Software Group
Applications: Definedby Infrastructure?
CDN 3rd party
Challenges of Scale, Co-location, Tenancy, Cost
Software Group
Applications Should be Definedby Activity
How doyoutroubleshootandassessproblemsin complex dynamicenvironments? How areactivities
identified andunderstood?
Software Group
System Complexity is Emergent/Entropic
WhataboutNoSQL?RoR?Ansible?AWS?(ormagic unicorns?)
Complexityis linkedwith marketsuccess, is driven bycustomers,contributors,competitors,investors,
partners,even time itself
Complexity doesn’t really fade,
but takes onnew forms
Software Group
CompleteDiscrete Transaction Traces
• Enduser data(navigation timing andfull content)
• Transactionpathandparametersandtiming
• Relatedbackendresources(databases,queues)
• Full detailofusersession in context
Web
server
App
server
DB
server
End users Internet Firewall
Traceand mergebeforeaggregation
Related Infrastructure
Software Group
WhyDiscrete Traces?
Software Group
Response TimeBreakdown
Software Group
Response TimeAnalysis (multi-dimensional)
Software Group
So, your next question shouldbe“IfI’m analyzing
all thisdata in a constantly changing
environment,how the heckdo Ikeepup?”
Software Group
Answer?:
Automation
Software Group
The only way to ensure
resilienceat scaleis to
makeeachagent (or cluster
of agents) as“self-resilient”
as possible.
Software Group
In an automated system, provisioning, scaling,
failure tolerance, and disaster recovery are
different names for the same workflows
• The trigger for the workflow is the
distinguishing characteristic
• All are executions of the same workflows
addressing insufficient or excess computing
resources
• All tend to focus on at the workload level
Software Group
Anything you can monitor can serve as a
trigger for a scaling workflow
• Raw resource utilization
• Basic application metrics
• Understanding the transaction path
Software Group
Aggregated and/or discrete
transaction data can help identify the
proper workload to
scale/recover/etc.*
*No way of predicting exactly which measure
is best for all time
Software Group
Example:
Uses aggregate order
rate for specific movies
by region to make scaling
decisions.
What scales determined
by APM data.
Software Group
Example:
Several financial services
firms combine market
trading volume with
transaction volume to
determine when to scale.
Software Group
How do we handle thiscomplexity?
Observe,Measure and Filter
Understand Intrinsic
Latencies
Understand the
Transaction Path
Understand User Behavior
Understand Resource
Contention
Improve Logical Complexity Tune, Balance, Scale
How are your tools andtechniques helping you dothis today?
Software Group
Questions and
Answers

Mining for gold 2.0

Editor's Notes

  • #8 You experience complex adaptive systems every day. In ecosystems…
  • #9 Give an example of an end to end resolution that follow this path: * Hi IO on one DB Shard due to a lot of users logging in and doing expensive activity
  • #25 Give an example of an end to end resolution that follow this path: * Hi IO on one DB Shard due to a lot of users logging in and doing expensive activity