Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Queues, Pools,and CachesPresented by: Gwen shapira, Senior consultant
About Myself•   13 years with a pager•   Oracle ACE•   Oak table member•   Senior consultant for Pythian•   @gwenshap•   h...
Why PythianRecognized Leader:•   Global industry-leader in remote database administration services and consulting for Orac...
© 2009/2010 Pythian
WHY?5   © 2009/2010 Pythian
OLTP:            High Concurrency            Low Latency            Low Variance6                 © 2009/2010 Pythian
7   © 2009/2010 Pythian
Our mission:    Use modern application design to    control database concurrency to its    maximum throughput, lower laten...
Nobody expects modern    application design!9            © 2009/2010 Pythian
Our Chief weapons are:     Connection Pools     Queues     Caches     And fanatical monitoring     And ruthless capacity p...
Connection Pools1
The ProblemOpening a database connectionis high latency operation.OLTP systems cant afford this latencyfor every user requ...
The Solution               © 2009/2010 Pythian
Application Business Layer        Application Data Layer                                               JNDI        DataSou...
New Problems           CPU + Latch                     Slow Resp.           contention                         Time       ...
5000 connectionsin pool…                                           ?But only 12 cores?16                   © 2009/2010 Pyt...
17   © 2009/2010 Pythian
18   © 2009/2010 Pythian
And that’s not all…     How long does it take to     open 5000 connections to     the database?19                 © 2009/2...
New Solutions     1.Load test     2.Limit connection pool     3.????     4.Low latency!!!20                  © 2009/2010 P...
Objection!1. But I don’t use all connections at once2. Our connection pool grows and shrinks   automatically              ...
"Dynamic connection pools are a     loaded gun pointed at your     system. Feeling lucky, punk?"             Graham Wood, ...
Objection!Problem: We can’t load testSolution: Guesstimate1. How many cores are available?2. How much time is spent squatt...
Objection!Problem: We have 5000 application serversSolution: Data Layer1. Separate servers running data layer2. Fewer serv...
Queues2
The ProblemWe have more user requests thandatabase connections                  © 2009/2010 Pythian
What do we do?     1. “Your call is important to us…”     2. Show them static content     3. Distract them with funny cat ...
Solution – Message Queue                  Msg 1                 Msg 2                 Msg 3                        .      ...
Application Business Layer                                Message                                                         ...
New Problems     Stuff developers say about message queues:     1. It is impossible to reliably monitor queues     2. Queu...
31   © 2009/2010 Pythian
Do Monitor:      1. Service time      2. Arrival rate      3. Queue size      4. Utilization32                      © 2009...
Capacity Planning     Myth: With proper capacity planning, queues     are not necessary     Fact: Over-provisioning is not...
Complex Systems     1. Queues are simple     2. Enterprise message        queues are complex     3. Match solution to prob...
Caches3
The ProblemOLTP scales best when working set iscached in RAMRDBMS have limitedmemory scalability                © 2009/201...
The Solution - Memcached       App         App                         App     App     Memcached   Memcached              ...
How is it awesome?     1. Less DB access     2. Less disk access     3. Distributed     4. Simple KV store     5. “Free” m...
Amazon ElastiCache     Memcached cluster of any size in     Amazon cloud     Unfortunately only accessible from EC2     9 ...
Linear Scalability?40                 © 2009/2010 Pythian
More Numbers     1.   0.007ms latency on my desktop     2.   2ms latency on cloud     3.   60K gets a second     4.   All ...
Application Business                    Message                          Layer                             Queue     Memca...
New Problems     •   Does not apply automatically     •   How to use it effectively?     •   How to monitor it?     •   Ho...
Use Case - Select     function get_username(int userid) {         /* first try the cache */         name = memcached_fetch...
Use Case - Update     function update_username(int userid, string     username) {       /* first update database */       ...
Usage Advice     1.   Use the ASH     2.   More memory, fewer cores     3.   DB is for durable writes     4.   Warm-up the...
How Big?     Cluster: As big as you can     Node: Not too big to fail.47              © 2009/2010 Pythian
What will we gain by adding 1G     cache?      1. You can’t calculate      2. Log all cache hits         and misses, by ke...
1ms * 0.95 + 5ms * 0.05     = 1.2ms49          © 2009/2010 Pythian
Monitor     1. Number of items, gets, sets        and misses     2. Number of evictions and        eviction time.     3. L...
Reminder:     1. Use Connection Pools     2. Limit the number of connections     3. Use queues to handle the excessive    ...
Thank you and Q&A     To contact us…           sales@pythian.com           1-866-PYTHIAN     To follow us…           http:...
Upcoming SlideShare
Loading in …5
×

Queues, Pools, Caches

2,708 views

Published on

Transaction processing systems are generally considered easier to scale than data warehouses. Relational databases were designed for this type of workload, and there are no esoteric hardware requirements. Mostly, it is just matter of normalizing to the right degree and getting the indexes right. The major challenge in these systems is their extreme concurrency, which means that small temporary slowdowns can escalate to major issues very quickly.

In this presentation, Gwen Shapira will explain how application developers and DBAs can work together to built a scalable and stable OLTP system - using application queues, connection pools and strategic use of caches in different layers of the system.

Published in: Technology
  • Be the first to comment

Queues, Pools, Caches

  1. 1. Queues, Pools,and CachesPresented by: Gwen shapira, Senior consultant
  2. 2. About Myself• 13 years with a pager• Oracle ACE• Oak table member• Senior consultant for Pythian• @gwenshap• http://www.pythian.com/news/author/shapira/• shapira@pythian.com 2 © 2009/2010 Pythian
  3. 3. Why PythianRecognized Leader:• Global industry-leader in remote database administration services and consulting for Oracle, Oracle Applications, MySQL and SQL Server• Work with over 150 multinational companies such as Forbes.com, Fox Sports, Nordion and Western Union to help manage their complex IT deploymentsExpertise:• One of the world’s largest concentrations of dedicated, full-time DBA expertise. Employ 7 Oracle ACEs/ACE Directors• Hold 7 Specializations under Oracle Platinum Partner program, including Oracle Exadata, Oracle GoldenGate & Oracle RACGlobal Reach & Scalability:• 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response 3 © 2009/2010 Pythian
  4. 4. © 2009/2010 Pythian
  5. 5. WHY?5 © 2009/2010 Pythian
  6. 6. OLTP: High Concurrency Low Latency Low Variance6 © 2009/2010 Pythian
  7. 7. 7 © 2009/2010 Pythian
  8. 8. Our mission: Use modern application design to control database concurrency to its maximum throughput, lower latency and make the system more predictable.8 © 2009/2010 Pythian
  9. 9. Nobody expects modern application design!9 © 2009/2010 Pythian
  10. 10. Our Chief weapons are: Connection Pools Queues Caches And fanatical monitoring And ruthless capacity planning And nice red uniforms!10 © 2009/2010 Pythian
  11. 11. Connection Pools1
  12. 12. The ProblemOpening a database connectionis high latency operation.OLTP systems cant afford this latencyfor every user request © 2009/2010 Pythian
  13. 13. The Solution © 2009/2010 Pythian
  14. 14. Application Business Layer Application Data Layer JNDI DataSource Interface DataSource Connection JDBC Driver Pool14 © 2009/2010 Pythian
  15. 15. New Problems CPU + Latch Slow Resp. contention Time Run out of Add More! connections15 © 2009/2010 Pythian
  16. 16. 5000 connectionsin pool… ?But only 12 cores?16 © 2009/2010 Pythian
  17. 17. 17 © 2009/2010 Pythian
  18. 18. 18 © 2009/2010 Pythian
  19. 19. And that’s not all… How long does it take to open 5000 connections to the database?19 © 2009/2010 Pythian
  20. 20. New Solutions 1.Load test 2.Limit connection pool 3.???? 4.Low latency!!!20 © 2009/2010 Pythian
  21. 21. Objection!1. But I don’t use all connections at once2. Our connection pool grows and shrinks automatically © 2009/2010 Pythian
  22. 22. "Dynamic connection pools are a loaded gun pointed at your system. Feeling lucky, punk?" Graham Wood, Oracle22 © 2009/2010 Pythian
  23. 23. Objection!Problem: We can’t load testSolution: Guesstimate1. How many cores are available?2. How much time is spent squatting a connection without running database queries?3. How much workload is IO-bound? © 2009/2010 Pythian
  24. 24. Objection!Problem: We have 5000 application serversSolution: Data Layer1. Separate servers running data layer2. Fewer servers3. Load balance based on capacity © 2009/2010 Pythian
  25. 25. Queues2
  26. 26. The ProblemWe have more user requests thandatabase connections © 2009/2010 Pythian
  27. 27. What do we do? 1. “Your call is important to us…” 2. Show them static content 3. Distract them with funny cat photos 4. Prioritize them 5. Acknowledge them and handle the request later 6. Jump them to the head of line 7. Tell them how long the wait is27 © 2009/2010 Pythian
  28. 28. Solution – Message Queue Msg 1 Msg 2 Msg 3 . . . Msg N28 © 2009/2010 Pythian
  29. 29. Application Business Layer Message Queue Application Data Layer DataSource Interface JNDI DataSource Connection JDBC Driver Pool29 © 2009/2010 Pythian
  30. 30. New Problems Stuff developers say about message queues: 1. It is impossible to reliably monitor queues 2. Queues are not necessary if you do proper capacity planning 3. Message queues are unnecessarily complicated.30 © 2009/2010 Pythian
  31. 31. 31 © 2009/2010 Pythian
  32. 32. Do Monitor: 1. Service time 2. Arrival rate 3. Queue size 4. Utilization32 © 2009/2010 Pythian
  33. 33. Capacity Planning Myth: With proper capacity planning, queues are not necessary Fact: Over-provisioning is not proper capacity planning Fact: Queue theory is capacity planning tool. Fact: Introduction of a few well defined and well understood queues will help capacity planning.33 © 2009/2010 Pythian
  34. 34. Complex Systems 1. Queues are simple 2. Enterprise message queues are complex 3. Match solution to problem requirements34 © 2009/2010 Pythian
  35. 35. Caches3
  36. 36. The ProblemOLTP scales best when working set iscached in RAMRDBMS have limitedmemory scalability © 2009/2010 Pythian
  37. 37. The Solution - Memcached App App App App Memcached Memcached Memcached Memcached App App37 © 2009/2010 Pythian
  38. 38. How is it awesome? 1. Less DB access 2. Less disk access 3. Distributed 4. Simple KV store 5. “Free” memory 6. Latency and availability resilience38 © 2009/2010 Pythian
  39. 39. Amazon ElastiCache Memcached cluster of any size in Amazon cloud Unfortunately only accessible from EC2 9 cents per node per hour!39 © 2009/2010 Pythian
  40. 40. Linear Scalability?40 © 2009/2010 Pythian
  41. 41. More Numbers 1. 0.007ms latency on my desktop 2. 2ms latency on cloud 3. 60K gets a second 4. All from the smallest possible servers at 38 cents per hour.41 © 2009/2010 Pythian
  42. 42. Application Business Message Layer Queue Memcached Application Data Layer DataSource Interface JNDI DataSource Connection JDBC Pool Driver42 © 2009/2010 Pythian
  43. 43. New Problems • Does not apply automatically • How to use it effectively? • How to monitor it? • How big?43 © 2009/2010 Pythian
  44. 44. Use Case - Select function get_username(int userid) { /* first try the cache */ name = memcached_fetch("username:" + userid); if (!name) { /* not found : request database */ name = db_select("SELECT username FROM users WHERE userid = ?", userid); /* then store in cache until next get */ memcached_add("username:" + userid, username); } return data; }44 © 2009/2010 Pythian
  45. 45. Use Case - Update function update_username(int userid, string username) { /* first update database */ result = db_execute("Update users set username=? WHERE userid=?", userid,username); if (result) { /* database update successful: update cache */ memcached_set("username:" + userid, username); }45 © 2009/2010 Pythian
  46. 46. Usage Advice 1. Use the ASH 2. More memory, fewer cores 3. DB is for durable writes 4. Warm-up the cache 5. Store nulls 6. Updates are tricky 7. Backward compatible schema46 © 2009/2010 Pythian
  47. 47. How Big? Cluster: As big as you can Node: Not too big to fail.47 © 2009/2010 Pythian
  48. 48. What will we gain by adding 1G cache? 1. You can’t calculate 2. Log all cache hits and misses, by key 3. Or sample 4. Run cache simulator 5. Predict avg. latency48 © 2009/2010 Pythian
  49. 49. 1ms * 0.95 + 5ms * 0.05 = 1.2ms49 © 2009/2010 Pythian
  50. 50. Monitor 1. Number of items, gets, sets and misses 2. Number of evictions and eviction time. 3. Low hit rate and high eviction rate? 4. Swapping 5. Average response time 6. Number of connections50 © 2009/2010 Pythian
  51. 51. Reminder: 1. Use Connection Pools 2. Limit the number of connections 3. Use queues to handle the excessive load 4. Use caches to make everything faster51 © 2009/2010 Pythian
  52. 52. Thank you and Q&A To contact us… sales@pythian.com 1-866-PYTHIAN To follow us… http://www.pythian.com/news/ http://www.facebook.com/pages/The-Pythian-Group/ http://twitter.com/pythian http://www.linkedin.com/company/pythian52 © 2009/2010 Pythian

×