Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Big Data in Real Time

382 views

Published on

Real time analytics on big data applications, what is real time, what are the challenges to achieve real time complex calculation on big data and how they can be solved.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Big Data in Real Time

  1. 1. Big Data in REAL TIMERon Zavner
  2. 2. We’re Living in a Real Time World… Social User Tracking & Homeland Security Engagement eCommerce Financial Services Real Time Search2 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved
  3. 3. The Flavors of Big Data Analytics Counting Correlating Research3 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved
  4. 4. Twitter in Numbers (March 2011) It takes a week for users to send 1 billion tweets Source: http://blog.twitter.com/2011/03/numbers.html4 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved
  5. 5. Twitter in Numbers (March 2011) On average, 140 million tweets get sent every day Source: http://blog.twitter.com/2011/03/numbers.html5 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved
  6. 6. Twitter in Numbers (March 2011) The highest throughput to date is6,939 tweets/sec. Source: http://blog.twitter.com/2011/03/numbers.html6 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved
  7. 7. Twitter in Numbers (March 2011) 460,000 new accounts are created daily Source: http://blog.twitter.com/2011/03/numbers.html7 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved
  8. 8. Challenge – Word Count Tweets8 ? ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved Count Count Word:Count
  9. 9. Analyze the Problem  Thousands of tweets per second to process  Aggregate counters for each word  Latency – less than a second  System needs to linearly scale  System needs to be fault tolerant  Querying & Persisting Data  Managing the system9 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved
  10. 10. Tier Based Architecture?10 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved
  11. 11. Data Grid 11 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved
  12. 12. Putting it all together12 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved
  13. 13. The 3 Most Popular Words on Twitter? 1. Just 2. Found 3. Love - August 201213 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved
  14. 14. Q&A RonZ@gigaspaces.com14 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved

×