Your SlideShare is downloading. ×
0
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Scale like an ant, distribute the workload - DPC, Amsterdam,  2011
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Scale like an ant, distribute the workload - DPC, Amsterdam, 2011

3,094

Published on

Many services / applications now a day are ill equipped with handling a sudden rush of popularity, as is often the case on the internet now a days, to a point where the services either become …

Many services / applications now a day are ill equipped with handling a sudden rush of popularity, as is often the case on the internet now a days, to a point where the services either become unavailable or unbearably slow.
By taking a chapter from the ant colonies in the wild, where their strength lies in their numbers and the fact that everyone works together towards the same goal, we can apply the same principle to our service by using systems such as
- gearman
- memcache
- daemons
- message queues
- load balancers
and many more, you can achieve greater performance, more redundancy, higher availability and have the ability to scale your services up and down as required easily.
During this talk attendees will be lead through the world of distributed systems and scalability, and shown the how, where and what, of how to take the average application and splitting it into smaller more manageable pieces

Published in: Technology, Travel
0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,094
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
114
Comments
0
Likes
8
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Distribute the workload Helgi Þormar Þorbjörnsson Dutch PHP Conference, Amsterdam, 19th May 2011 Thursday, 19 May 2011
  • 2. Who am I? Thursday, 19 May 2011
  • 3. Helgi Thursday, 19 May 2011
  • 4. VP of Engineering at Orchestra.io Helgi Thursday, 19 May 2011
  • 5. VP of Engineering at Orchestra.io Developer at PEAR Helgi Thursday, 19 May 2011
  • 6. VP of Engineering at Orchestra.io Developer at PEAR From Iceland Helgi Thursday, 19 May 2011
  • 7. VP of Engineering at Orchestra.io Developer at PEAR From Iceland @h on Twitter Helgi Thursday, 19 May 2011
  • 8. Why Distribute? Thursday, 19 May 2011
  • 9. Why Distribute? Thursday, 19 May 2011
  • 10. Why Distribute? Budget Thursday, 19 May 2011
  • 11. Why Distribute? Budget Efficiency Thursday, 19 May 2011
  • 12. Why Distribute? Budget Efficiency Perception Thursday, 19 May 2011
  • 13. Budget Thursday, 19 May 2011
  • 14. Budget Spend wisely Thursday, 19 May 2011
  • 15. Budget Spend wisely Commodity servers Thursday, 19 May 2011
  • 16. Budget Spend wisely Commodity servers Low overhead, high yield Thursday, 19 May 2011
  • 17. Budget Spend wisely Commodity servers Low overhead, high yield Cloud Computing (EC2) Thursday, 19 May 2011
  • 18. Efficiency 10 small servers > 1 big Thursday, 19 May 2011
  • 19. Venue Security Thursday, 19 May 2011
  • 20. 1000 people can exit quicker through 10 small doors than 1 big Thursday, 19 May 2011
  • 21. 1000 people can exit quicker through 10 small doors than 1 big Thursday, 19 May 2011
  • 22. 1000 people can exit quicker through 10 small doors than 1 big Thursday, 19 May 2011
  • 23. 1000 people can exit quicker through 10 small doors than 1 big Thursday, 19 May 2011
  • 24. Thursday, 19 May 2011
  • 25. Thursday, 19 May 2011
  • 26. Thursday, 19 May 2011
  • 27. Thursday, 19 May 2011
  • 28. Perception Thursday, 19 May 2011
  • 29. Perception Defer intensive processes Thursday, 19 May 2011
  • 30. Perception Defer intensive processes Give instant feedback Thursday, 19 May 2011
  • 31. Perception Defer intensive processes Give instant feedback Users keep on browsing Thursday, 19 May 2011
  • 32. Perception Defer intensive processes Give instant feedback Users keep on browsing Thursday, 19 May 2011
  • 33. “It all depends on how we look at things, and not how they are in themselves.” - Carl G. Jung Thursday, 19 May 2011
  • 34. Thursday, 19 May 2011
  • 35. Chapter from Nature Thursday, 19 May 2011
  • 36. Ant Colonies Thursday, 19 May 2011
  • 37. Algorithms Thursday, 19 May 2011
  • 38. Algorithms Scheduling Thursday, 19 May 2011
  • 39. Algorithms Scheduling Vehicle Routing Thursday, 19 May 2011
  • 40. Algorithms Scheduling Vehicle Routing Assignment Thursday, 19 May 2011
  • 41. Algorithms Scheduling Vehicle Routing Assignment Sets Thursday, 19 May 2011
  • 42. Algorithms Scheduling Vehicle Routing Assignment Sets Other Thursday, 19 May 2011
  • 43. Algorithms Scheduling Vehicle Routing Assignment Sets Other Thursday, 19 May 2011
  • 44. How do ants fit? Thursday, 19 May 2011
  • 45. How do ants fit? Strength in numbers Thursday, 19 May 2011
  • 46. How do ants fit? Strength in numbers Work together Thursday, 19 May 2011
  • 47. How do ants fit? Strength in numbers Work together Size benefits them Thursday, 19 May 2011
  • 48. Teamwork When faced with a problem they will solve the problem as one. Thursday, 19 May 2011
  • 49. Thursday, 19 May 2011
  • 50. Thursday, 19 May 2011
  • 51. What if they were bigger? Thursday, 19 May 2011
  • 52. Types of Ants Thursday, 19 May 2011
  • 53. Types of Ants Military Thursday, 19 May 2011
  • 54. Types of Ants Military Maids Thursday, 19 May 2011
  • 55. Types of Ants Military Maids Tunnel diggers Thursday, 19 May 2011
  • 56. Types of Ants Military Maids Tunnel diggers Food gatherers Thursday, 19 May 2011
  • 57. How does this map to my application? Thursday, 19 May 2011
  • 58. Thursday, 19 May 2011
  • 59. Colony = Application Thursday, 19 May 2011
  • 60. Colony = Application Ants = Components Thursday, 19 May 2011
  • 61. Colony = Application Ants = Components Ants do many different types of work to keep their colony running Thursday, 19 May 2011
  • 62. Architect for Distribution Thursday, 19 May 2011
  • 63. Characteristics Thursday, 19 May 2011
  • 64. Characteristics Decoupling Thursday, 19 May 2011
  • 65. Characteristics Decoupling Elasticity Thursday, 19 May 2011
  • 66. Characteristics Decoupling Elasticity High Availability Thursday, 19 May 2011
  • 67. Characteristics Decoupling Elasticity High Availability Concurrency Thursday, 19 May 2011
  • 68. Decoupling Thursday, 19 May 2011
  • 69. Application DB API Cache FE Thursday, 19 May 2011
  • 70. Application DB API Cache FE Thursday, 19 May 2011
  • 71. ApplicationDB API Cache FE Thursday, 19 May 2011
  • 72. ApplicationDB API Cache FE Cache Thursday, 19 May 2011
  • 73. ApplicationDB API Cache FE Cache API Thursday, 19 May 2011
  • 74. ApplicationDB API Cache FE Cache API API Thursday, 19 May 2011
  • 75. Elasticity Thursday, 19 May 2011
  • 76. Cloud Computing Thursday, 19 May 2011
  • 77. Load Balancing Thursday, 19 May 2011
  • 78. HA Proxy Nginx My Favourite Thursday, 19 May 2011
  • 79. Monitoring Thursday, 19 May 2011
  • 80. When do I need more servers? Thursday, 19 May 2011
  • 81. Needs to be around from the start! Thursday, 19 May 2011
  • 82. Keep records Thursday, 19 May 2011
  • 83. Spot trends Thursday, 19 May 2011
  • 84. Different types Thursday, 19 May 2011
  • 85. Different types Hardware Performance Thursday, 19 May 2011
  • 86. Different types Hardware Performance Software Performance Thursday, 19 May 2011
  • 87. Different types Hardware Performance Software Performance Availability Thursday, 19 May 2011
  • 88. Different types Hardware Performance Software Performance Availability Resourcing Thursday, 19 May 2011
  • 89. Different types Hardware Performance Software Performance Availability Resourcing Thursday, 19 May 2011
  • 90. Applications Thursday, 19 May 2011
  • 91. Applications New Relic Thursday, 19 May 2011
  • 92. Applications New Relic CloudKick Thursday, 19 May 2011
  • 93. Applications New Relic CloudKick ScoutApp Thursday, 19 May 2011
  • 94. Applications New Relic CloudKick ScoutApp Nagios Thursday, 19 May 2011
  • 95. Applications New Relic CloudKick ScoutApp Nagios Cacti Thursday, 19 May 2011
  • 96. Applications New Relic CloudKick ScoutApp Nagios Cacti Circonus Thursday, 19 May 2011
  • 97. Automation Thursday, 19 May 2011
  • 98. Want to sleep easy at night? Thursday, 19 May 2011
  • 99. Want to sleep easy at night? Go out partying without worrying about getting a phone call? Thursday, 19 May 2011
  • 100. Plug into your monitoring Thursday, 19 May 2011
  • 101. Bringing together Monitoring and Elastic behaviour into one beautiful whole! Thursday, 19 May 2011
  • 102. Add some intelligence to add / remove servers as needed based on current information. Thursday, 19 May 2011
  • 103. This is why good monitoring is essential or this wouldn’t be possible Thursday, 19 May 2011
  • 104. Just make sure it doesn’t turn into... Thursday, 19 May 2011
  • 105. Skynet!! Thursday, 19 May 2011
  • 106. High Availability Thursday, 19 May 2011
  • 107. Get a highly available and resilient setup by following a few of those recommendations Thursday, 19 May 2011
  • 108. Remember, even Google has outages Thursday, 19 May 2011
  • 109. Benefits Thursday, 19 May 2011
  • 110. Benefits Easy management Thursday, 19 May 2011
  • 111. Benefits Easy management Ability to stop / start servers quickly Thursday, 19 May 2011
  • 112. Benefits Easy management Ability to stop / start servers quickly Responsibilities are separate Thursday, 19 May 2011
  • 113. Benefits Easy management Ability to stop / start servers quickly Responsibilities are separate Quickly move to a new cluster Thursday, 19 May 2011
  • 114. Benefits Easy management Ability to stop / start servers quickly Responsibilities are separate Quickly move to a new cluster Reduced risk Thursday, 19 May 2011
  • 115. What to avoid Thursday, 19 May 2011
  • 116. Local Sessions Thursday, 19 May 2011
  • 117. Store sessions in DB / Memcache Solution Thursday, 19 May 2011
  • 118. Local Memory Thursday, 19 May 2011
  • 119. Networked Memcache Solution Thursday, 19 May 2011
  • 120. Local Files Thursday, 19 May 2011
  • 121. Local Uploads Thursday, 19 May 2011
  • 122. Writing to /tmp Thursday, 19 May 2011
  • 123. Store on S3 or a networked FS Solution Thursday, 19 May 2011
  • 124. Serve up static files from CDNs Solution Thursday, 19 May 2011
  • 125. Servers can vanish at any given time Thursday, 19 May 2011
  • 126. Internal APIs Thursday, 19 May 2011
  • 127. Application S3GFS FS Internal Storage API Thursday, 19 May 2011
  • 128. Application MySQLMongo Cache Internal DB API Thursday, 19 May 2011
  • 129. SOA Thursday, 19 May 2011
  • 130. Service Oriented Architecture Thursday, 19 May 2011
  • 131. Sort of :-) Thursday, 19 May 2011
  • 132. Eventually Consistent Thursday, 19 May 2011
  • 133. CAP Therom Thursday, 19 May 2011
  • 134. Consistency Availability Partition Tolerance Thursday, 19 May 2011
  • 135. Consistency All nodes see the same data at the same time Thursday, 19 May 2011
  • 136. Availability Node failures do not prevent survivors from continuing to operate Thursday, 19 May 2011
  • 137. Partition Tolerance The system continues to operate despite arbitrary message loss Thursday, 19 May 2011
  • 138. Consistency Availability Partition Tolerance Thursday, 19 May 2011
  • 139. Queue Systems Thursday, 19 May 2011
  • 140. Good for Thursday, 19 May 2011
  • 141. Good for Image Processing Thursday, 19 May 2011
  • 142. Good for Image Processing Distributed Logs Thursday, 19 May 2011
  • 143. Good for Image Processing Distributed Logs Data Mining Thursday, 19 May 2011
  • 144. Good for Image Processing Distributed Logs Data Mining Mass Emails Thursday, 19 May 2011
  • 145. Good for Image Processing Distributed Logs Data Mining Mass Emails Intensive transformation Thursday, 19 May 2011
  • 146. Good for Image Processing Distributed Logs Data Mining Mass Emails Intensive transformation Search Thursday, 19 May 2011
  • 147. Common Tools Thursday, 19 May 2011
  • 148. Common Tools Gearman Thursday, 19 May 2011
  • 149. Common Tools Gearman Hadoop Thursday, 19 May 2011
  • 150. Common Tools Gearman Hadoop Zero MQ (0MQ) Thursday, 19 May 2011
  • 151. Common Tools Gearman Hadoop Zero MQ (0MQ) RabbitMQ Thursday, 19 May 2011
  • 152. Common Tools Gearman Hadoop Zero MQ (0MQ) RabbitMQ And many others! Thursday, 19 May 2011
  • 153. Gearman Thursday, 19 May 2011
  • 154. Your Client Code Gearman Client API (C, PHP, Perl, MySQL UDF, ...) Gearman Job Server gearmand Gearman Worker API (C, PHP, Perl, Python, ...) Your Worker Code Your App Gearman Thursday, 19 May 2011
  • 155. A Story! Thursday, 19 May 2011
  • 156. Financial Software Thursday, 19 May 2011
  • 157. 3000+ Clients Thursday, 19 May 2011
  • 158. Each one has 5 external data sources Thursday, 19 May 2011
  • 159. Each data source is a web service Thursday, 19 May 2011
  • 160. Ran every 6 hours every day Thursday, 19 May 2011
  • 161. Cron 1 2 3 4 5 Job 1 Gearman 1 4 3 2 5 Web Services Processing Thursday, 19 May 2011
  • 162. But! That wasn’t enough Thursday, 19 May 2011
  • 163. Job kicked off on login Thursday, 19 May 2011
  • 164. Another Story! Thursday, 19 May 2011
  • 165. CloudSplit Thursday, 19 May 2011
  • 166. Near Real Time Cloud Analytics Thursday, 19 May 2011
  • 167. Clients install logging agent locally Thursday, 19 May 2011
  • 168. syslogd Thursday, 19 May 2011
  • 169. Public API Thursday, 19 May 2011
  • 170. Multiple Persistent Gearman Servers Thursday, 19 May 2011
  • 171. Internal DB API Thursday, 19 May 2011
  • 172. Agent Thursday, 19 May 2011
  • 173. Agent syslogd Thursday, 19 May 2011
  • 174. Agent syslogd API Thursday, 19 May 2011
  • 175. Agent syslogd API Load Balanced Thursday, 19 May 2011
  • 176. Agent syslogd API Gearman Gearman Load Balanced Thursday, 19 May 2011
  • 177. Agent syslogd API Gearman Gearman Load Balanced Persistent Thursday, 19 May 2011
  • 178. Agent syslogd API Gearman Gearman Worker Worker Worker Load Balanced Persistent Thursday, 19 May 2011
  • 179. Agent syslogd API Gearman Gearman Worker Worker Worker Internal API Load Balanced Persistent Thursday, 19 May 2011
  • 180. Agent syslogd API Gearman Gearman Worker Worker Worker Internal API Load Balanced Load Balanced Persistent Thursday, 19 May 2011
  • 181. Agent syslogd API Gearman Gearman CouchDB Worker Worker Worker Internal API Load Balanced Load Balanced Persistent Thursday, 19 May 2011
  • 182. CouchDB Setup Thursday, 19 May 2011
  • 183. Write vs Read Thursday, 19 May 2011
  • 184. Writes Thursday, 19 May 2011
  • 185. Writes Multi Master setup Thursday, 19 May 2011
  • 186. Writes Multi Master setup Replicated Thursday, 19 May 2011
  • 187. Writes Multi Master setup Replicated Deals with writes only Thursday, 19 May 2011
  • 188. Writes Multi Master setup Replicated Deals with writes only Thursday, 19 May 2011
  • 189. Reads Thursday, 19 May 2011
  • 190. Reads Multi Master setup Thursday, 19 May 2011
  • 191. Reads Multi Master setup Replicated from write cluster Thursday, 19 May 2011
  • 192. Reads Multi Master setup Replicated from write cluster Slaves handle website requests Thursday, 19 May 2011
  • 193. Reads Multi Master setup Replicated from write cluster Slaves handle website requests Thursday, 19 May 2011
  • 194. Heavy Map/Reduce usage for data Thursday, 19 May 2011
  • 195. Supervisord Thursday, 19 May 2011
  • 196. phpadvent.org/2009/daemonize- your-php-by-sean-coates Thursday, 19 May 2011
  • 197. Map/Reduce Thursday, 19 May 2011
  • 198. Map Thursday, 19 May 2011
  • 199. Map Master gets a problem to solve Thursday, 19 May 2011
  • 200. Map Master gets a problem to solve Breaks into multiple sub-problems Thursday, 19 May 2011
  • 201. Map Master gets a problem to solve Breaks into multiple sub-problems Distributed to multiple workers Thursday, 19 May 2011
  • 202. Map Master gets a problem to solve Breaks into multiple sub-problems Distributed to multiple workers A worker can take the same steps Thursday, 19 May 2011
  • 203. Map Master gets a problem to solve Breaks into multiple sub-problems Distributed to multiple workers A worker can take the same steps Answer passed back to Master Thursday, 19 May 2011
  • 204. Reduce Thursday, 19 May 2011
  • 205. Reduce Takes in answers from the map workers Thursday, 19 May 2011
  • 206. Reduce Takes in answers from the map workers Combines together to get an answer Thursday, 19 May 2011
  • 207. Reduce Takes in answers from the map workers Combines together to get an answer There can be multiple reducers Thursday, 19 May 2011
  • 208. process petabytes of data in few hours on commodity server farm Thursday, 19 May 2011
  • 209. CouchDB Thursday, 19 May 2011
  • 210. CouchDB Thursday, 19 May 2011
  • 211. CouchDB Highly Concurrent Thursday, 19 May 2011
  • 212. CouchDB Highly Concurrent Schema free, document based Thursday, 19 May 2011
  • 213. CouchDB Highly Concurrent Schema free, document based RESTful API Thursday, 19 May 2011
  • 214. CouchDB Highly Concurrent Schema free, document based RESTful API Map/Reduce Views Thursday, 19 May 2011
  • 215. CouchDB Highly Concurrent Schema free, document based RESTful API Map/Reduce Views Easy Replication Thursday, 19 May 2011
  • 216. Hadoop Thursday, 19 May 2011
  • 217. Hadoop is a framework for running applications on large clusters of commodity hardware. Thursday, 19 May 2011
  • 218. Thursday, 19 May 2011
  • 219. The Hadoop framework transparently provides applications both reliability and data motion Thursday, 19 May 2011
  • 220. Thursday, 19 May 2011
  • 221. Uses Map/Reduce concept to farm out work Thursday, 19 May 2011
  • 222. Thursday, 19 May 2011
  • 223. Distributed FS to handled node failure automagically Thursday, 19 May 2011
  • 224. Thursday, 19 May 2011
  • 225. Join 2 datasets together of a significant size Thursday, 19 May 2011
  • 226. Thursday, 19 May 2011
  • 227. 500 GB worth of log files with a large location dataset Thursday, 19 May 2011
  • 228. ØMQ Thursday, 19 May 2011
  • 229. ØMQ Thursday, 19 May 2011
  • 230. ØMQ Async Message System Thursday, 19 May 2011
  • 231. ØMQ Async Message System Thin and lightweight Thursday, 19 May 2011
  • 232. ØMQ Async Message System Thin and lightweight High Performance Thursday, 19 May 2011
  • 233. ØMQ Async Message System Thin and lightweight High Performance Simple Thursday, 19 May 2011
  • 234. ØMQ Async Message System Thin and lightweight High Performance Simple Scalable Thursday, 19 May 2011
  • 235. Thursday, 19 May 2011
  • 236. One socket can load balance to multiple end points Thursday, 19 May 2011
  • 237. Thursday, 19 May 2011
  • 238. Multiple end points can be funnelled into a single socket Thursday, 19 May 2011
  • 239. Thursday, 19 May 2011
  • 240. Handle deployments to multiple servers Thursday, 19 May 2011
  • 241. Thursday, 19 May 2011
  • 242. Scale is an example of that Thursday, 19 May 2011
  • 243. Thursday, 19 May 2011
  • 244. Mongrel2 is a web server that uses it in a similar way as fastcgi Thursday, 19 May 2011
  • 245. Thursday, 19 May 2011
  • 246. Move around text (JSON) and Binary data for real time communication Thursday, 19 May 2011
  • 247. Thursday, 19 May 2011
  • 248. Could have replaced syslogd and the external API in my previous example Thursday, 19 May 2011
  • 249. Code time? :-) Thursday, 19 May 2011
  • 250. Questions? helgi@orchestra.io Twitter: @h Joind.in: http://joind.in/3212 Thursday, 19 May 2011

×