Big Data Seminar: Analytics, Hadoop, Map Reduce, Mongo and other great stuff

735
-1

Published on

If you want to design an Analytics solution, you should have attended this workshop. Contact me if you need further information. Moshe Kaplan

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
735
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Big Data Seminar: Analytics, Hadoop, Map Reduce, Mongo and other great stuff

  1. 1. The VP R&D Open Seminar Big Data Workshop mokplan@gmail.com http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  2. 2. Presentation Objectiveshttp://www.webperformancetoday.com/2010/06/15/everything-you-wanted-to-know-about-web-performance/ http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  3. 3. Why Do I Care?From 0 to 100 (US mass adaptation) Phone: 100 yrs Radio: 40 yrs TV: 30 yrs Mobile: 20 yrs Internet: 10 yrs Facebook: 2 yrs http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  4. 4. The Prime Suspecthttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  5. 5. Assumptions…http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  6. 6. Network Effect Solutionshttp://www.akamai.com/stateoftheinternet/ http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  7. 7. General ArchitectureClient Server Database Apps http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  8. 8. The VP R&D Open SeminarCLIENT SIDE http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  9. 9. It’s a World Made of Pixels http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  10. 10. The VP R&D Open SeminarSERVER SIDE http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  11. 11. General StrategiesOnlineIn Memory DatabasesLog files http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  12. 12. In Memory Databaseshttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  13. 13. 700 Inserts/Sec In Memory Engine 3000 Inserts/SecAmazonAWS InnoDB EngineStandard 700 Inserts/SecLargeInstance http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  14. 14. The VP R&D Open SeminarGeneral StrategiesDATA SIDE http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  15. 15. Strategy A - Shardinghttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  16. 16. Strategy B – MapReduce http://blogs.microsoft.co.il/blogs/vprndhttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  17. 17. Strategy C - NoSQL insert get multiget remove truncate <Key, Value>http://wiki.apache.org/cassandra/API http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  18. 18. The VP R&D Open SeminarCOLUMN ORIENTEDDATABASES http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  19. 19. Column Oriented Database Databases+ INSERT+ GROUP BY, SUM …+ Compression- Join- DELETE, UPDATE http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  20. 20. Cloud Serviceshttp://www.theregister.co.uk/2012/11/28/amazon_aws_redshift_data_warehousing/ http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  21. 21. The VP R&D Open SeminarSHARDING IN DEPTH http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  22. 22. Sharding Againhttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  23. 23. Vertical Shardinghttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  24. 24. Horizontal ShardingStatic Hashing Mod 10 = 0 Complex growth Mod 10 = 1 Simple Mod 10 = 2 Mod 10 = 3 Mod 10 = 4 Mod 10 = 5 Mod 10 = 6 Mod 10 = 7 Mod 10 = 8 Mod 10 = 9 http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  25. 25. Horizontal ShardingKey locations are defined in a directory Simple growth Directory is SPOF The Directory Can be Very Large http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  26. 26. Horizontal Sharding Static Hashing with Directory Mapping Simple Growth The Small Directory Can be Cached on Each App ServerMod 1000 = 4 http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  27. 27. Horizontal ShardingEach key is signed by the DB#on creation Simple growth The Key Store Can be Cached on Each App Server http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  28. 28. Big Queryhttp://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
  29. 29. The Bottom Line: Grow ∞Thank you!and Keep Performing! Moshe Kaplan http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com

×