Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
The VP R&D Open Seminar       Big Data Workshop            mokplan@gmail.com  http://blogs.microsoft.co.il/blogs/vprnd   h...
Presentation Objectiveshttp://www.webperformancetoday.com/2010/06/15/everything-you-wanted-to-know-about-web-performance/ ...
Why Do I Care?From 0 to 100 (US mass adaptation)   Phone:         100 yrs   Radio:          40 yrs   TV:             30 yr...
The Prime Suspecthttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
Assumptions…http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
Network Effect Solutionshttp://www.akamai.com/stateoftheinternet/                                  http://blogs.microsoft....
General ArchitectureClient       Server             Database            Apps         http://blogs.microsoft.co.il/blogs/vp...
The VP R&D Open SeminarCLIENT SIDE     http://blogs.microsoft.co.il/blogs/vprnd      http://top-performance.blogspot.com
It’s a World Made of Pixels   http://blogs.microsoft.co.il/blogs/vprnd    http://top-performance.blogspot.com
The VP R&D Open SeminarSERVER SIDE     http://blogs.microsoft.co.il/blogs/vprnd      http://top-performance.blogspot.com
General StrategiesOnlineIn Memory DatabasesLog files        http://blogs.microsoft.co.il/blogs/vprnd         http://top-pe...
In Memory Databaseshttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
700 Inserts/Sec                                             In Memory Engine                                             3...
The VP R&D Open SeminarGeneral StrategiesDATA SIDE             http://blogs.microsoft.co.il/blogs/vprnd              http:...
Strategy A - Shardinghttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
Strategy B – MapReduce          http://blogs.microsoft.co.il/blogs/vprndhttp://blogs.microsoft.co.il/blogs/vprnd       htt...
Strategy C - NoSQL       insert       get             multiget       remove       truncate             <Key, Value>http://...
The VP R&D Open SeminarCOLUMN ORIENTEDDATABASES     http://blogs.microsoft.co.il/blogs/vprnd      http://top-performance.b...
Column Oriented Database Databases+ INSERT+ GROUP BY, SUM …+ Compression- Join- DELETE, UPDATE        http://blogs.microso...
Cloud Serviceshttp://www.theregister.co.uk/2012/11/28/amazon_aws_redshift_data_warehousing/                               ...
The VP R&D Open SeminarSHARDING IN DEPTH     http://blogs.microsoft.co.il/blogs/vprnd      http://top-performance.blogspot...
Sharding Againhttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
Vertical Shardinghttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
Horizontal ShardingStatic Hashing                                                    Mod 10 = 0  Complex growth           ...
Horizontal ShardingKey locations are defined in a directory  Simple growth  Directory is SPOF  The Directory Can  be Very ...
Horizontal Sharding  Static Hashing with Directory Mapping    Simple Growth    The Small Directory Can be Cached on    Eac...
Horizontal ShardingEach key is signed by the DB#on creation  Simple growth  The Key Store Can be Cached on  Each App Serve...
Big Queryhttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
The Bottom Line: Grow ∞Thank you!and Keep Performing! Moshe Kaplan          http://blogs.microsoft.co.il/blogs/vprnd      ...
Upcoming SlideShare
Loading in …5
×

Big Data Seminar: Analytics, Hadoop, Map Reduce, Mongo and other great stuff

964 views

Published on

If you want to design an Analytics solution, you should have attended this workshop. Contact me if you need further information. Moshe Kaplan

Published in: Technology
  • Be the first to comment

Big Data Seminar: Analytics, Hadoop, Map Reduce, Mongo and other great stuff

  1. 1. The VP R&D Open Seminar Big Data Workshop mokplan@gmail.com http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  2. 2. Presentation Objectiveshttp://www.webperformancetoday.com/2010/06/15/everything-you-wanted-to-know-about-web-performance/ http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  3. 3. Why Do I Care?From 0 to 100 (US mass adaptation) Phone: 100 yrs Radio: 40 yrs TV: 30 yrs Mobile: 20 yrs Internet: 10 yrs Facebook: 2 yrs http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  4. 4. The Prime Suspecthttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  5. 5. Assumptions…http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  6. 6. Network Effect Solutionshttp://www.akamai.com/stateoftheinternet/ http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  7. 7. General ArchitectureClient Server Database Apps http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  8. 8. The VP R&D Open SeminarCLIENT SIDE http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  9. 9. It’s a World Made of Pixels http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  10. 10. The VP R&D Open SeminarSERVER SIDE http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  11. 11. General StrategiesOnlineIn Memory DatabasesLog files http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  12. 12. In Memory Databaseshttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  13. 13. 700 Inserts/Sec In Memory Engine 3000 Inserts/SecAmazonAWS InnoDB EngineStandard 700 Inserts/SecLargeInstance http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  14. 14. The VP R&D Open SeminarGeneral StrategiesDATA SIDE http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  15. 15. Strategy A - Shardinghttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  16. 16. Strategy B – MapReduce http://blogs.microsoft.co.il/blogs/vprndhttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  17. 17. Strategy C - NoSQL insert get multiget remove truncate <Key, Value>http://wiki.apache.org/cassandra/API http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  18. 18. The VP R&D Open SeminarCOLUMN ORIENTEDDATABASES http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  19. 19. Column Oriented Database Databases+ INSERT+ GROUP BY, SUM …+ Compression- Join- DELETE, UPDATE http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  20. 20. Cloud Serviceshttp://www.theregister.co.uk/2012/11/28/amazon_aws_redshift_data_warehousing/ http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  21. 21. The VP R&D Open SeminarSHARDING IN DEPTH http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  22. 22. Sharding Againhttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  23. 23. Vertical Shardinghttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  24. 24. Horizontal ShardingStatic Hashing Mod 10 = 0 Complex growth Mod 10 = 1 Simple Mod 10 = 2 Mod 10 = 3 Mod 10 = 4 Mod 10 = 5 Mod 10 = 6 Mod 10 = 7 Mod 10 = 8 Mod 10 = 9 http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  25. 25. Horizontal ShardingKey locations are defined in a directory Simple growth Directory is SPOF The Directory Can be Very Large http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  26. 26. Horizontal Sharding Static Hashing with Directory Mapping Simple Growth The Small Directory Can be Cached on Each App ServerMod 1000 = 4 http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  27. 27. Horizontal ShardingEach key is signed by the DB#on creation Simple growth The Key Store Can be Cached on Each App Server http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  28. 28. Big Queryhttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  29. 29. The Bottom Line: Grow ∞Thank you!and Keep Performing! Moshe Kaplan http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com

×