Instrumenting your Application Stack in a
Dynamically Scaling Environment
Mike Fiedler, Datadog
November 15, 2013
The Past
• Server uptime
• Intertwined applications
• SPOFs
The Present
• SOA
• Scale or Fail
• Data-driven decisions
Modern application stack
Failure is inevitable
Design to tolerate failure
business/service
Services

API-based

Users

• application
• OS
• instances

• Amazon S3, S*S
• Amazon RDS,
Amazon Redshif...
The Desire
CPU comparison (hypervisor)
CPU comparison (Agent)
My Stack
• HAProxy
• Apache HTTP
• Bottle.py
– Custom python webapp

• MongoDB
• Amazon EC2

• Chef
• Datadog
• Users
Tracking database operations
Moving up the stack…
Even higher…
Using metadata for more flavor
Replace an instance, keep the metrics
Enter…
Aggregating numbers
Comparing numbers
My App
from bottle import route
import pymongo
import json

db = pymongo.Connection('mongodb://localhost').test

@route('/...
Add 2 lines and you win!
from bottle import route
import pymongo
import json
from statsd import statsd

db = pymongo.Conne...
Profit!
Correlation
Don’t be caught unaware
Question Time!
We are sincerely eager to hear
your feedback on this
presentation and on re:Invent.
Please fill out an evaluation form
whe...
Instrumenting Application Stack in a Dynamically Scaling Environment (DMG212) | AWS re:Invent 2013
Upcoming SlideShare
Loading in …5
×

Instrumenting Application Stack in a Dynamically Scaling Environment (DMG212) | AWS re:Invent 2013

627 views

Published on

(Presented by Datadog) Gaining visibility into an application stack’s performance is necessary to understand how the stack is running and to configure alerts effectively. Instrumenting each component in the stack to produce metrics provides this insight. In an environment that scales automatically, hosts are being automatically added, removed, and reassigned. Using an automated methodology for instrumentation in these environments can improve results and save you time. This session includes a live demo component to show auto-instrumentation of hosts, graphing, and alerting on metrics.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
627
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
18
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Instrumenting Application Stack in a Dynamically Scaling Environment (DMG212) | AWS re:Invent 2013

  1. 1. Instrumenting your Application Stack in a Dynamically Scaling Environment Mike Fiedler, Datadog November 15, 2013
  2. 2. The Past • Server uptime • Intertwined applications • SPOFs
  3. 3. The Present • SOA • Scale or Fail • Data-driven decisions
  4. 4. Modern application stack
  5. 5. Failure is inevitable
  6. 6. Design to tolerate failure
  7. 7. business/service Services API-based Users • application • OS • instances • Amazon S3, S*S • Amazon RDS, Amazon Redshift • etc. • RUM Things that produce metrics
  8. 8. The Desire
  9. 9. CPU comparison (hypervisor)
  10. 10. CPU comparison (Agent)
  11. 11. My Stack • HAProxy • Apache HTTP • Bottle.py – Custom python webapp • MongoDB • Amazon EC2 • Chef • Datadog • Users
  12. 12. Tracking database operations
  13. 13. Moving up the stack…
  14. 14. Even higher…
  15. 15. Using metadata for more flavor
  16. 16. Replace an instance, keep the metrics
  17. 17. Enter…
  18. 18. Aggregating numbers
  19. 19. Comparing numbers
  20. 20. My App from bottle import route import pymongo import json db = pymongo.Connection('mongodb://localhost').test @route('/insert/:name') def insert(name): doc = {'name': name} db.words.update(doc, {"$inc":{"count": 1}}, upsert=True) return json.dumps(doc, default=default)
  21. 21. Add 2 lines and you win! from bottle import route import pymongo import json from statsd import statsd db = pymongo.Connection('mongodb://localhost').test @route('/insert/:name') @statsd.increment('fullstack.insert') def insert(name): doc = {'name': name} db.words.update(doc, {"$inc":{"count": 1}}, upsert=True) return json.dumps(doc, default=default)
  22. 22. Profit!
  23. 23. Correlation
  24. 24. Don’t be caught unaware
  25. 25. Question Time!
  26. 26. We are sincerely eager to hear your feedback on this presentation and on re:Invent. Please fill out an evaluation form when you have a chance.

×