Real time analytics on big data applications, what is real time, what are the challenges to achieve real time complex calculation on big data and how they can be solved.

1. Big Data in REAL TIME
Ron Zavner

2. We’re Living in a Real Time World…
Social User Tracking & Homeland Security
Engagement
eCommerce Financial Services Real Time Search
2 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved

3. The Flavors of Big Data Analytics
Counting Correlating Research
3 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved

4. Twitter in Numbers (March 2011)
It takes a week for users to
send 1 billion tweets
Source: http://blog.twitter.com/2011/03/numbers.html
4 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved

5. Twitter in Numbers (March 2011)
On average,
140 million
tweets get sent every day
Source: http://blog.twitter.com/2011/03/numbers.html
5 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved

6. Twitter in Numbers (March 2011)
The highest
throughput to date is
6,939 tweets/sec.
Source: http://blog.twitter.com/2011/03/numbers.html
6 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved

7. Twitter in Numbers (March 2011)
460,000 new
accounts
are created daily
Source: http://blog.twitter.com/2011/03/numbers.html
7 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved

8. Challenge – Word Count
Tweets
8
?
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved
Count
Count
Word:Count

9. Analyze the Problem
 Thousands of tweets per second to process
 Aggregate counters for each word
 Latency – less than a second
 System needs to linearly scale
 System needs to be fault tolerant
 Querying & Persisting Data
 Managing the system
9 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved

10. Tier Based Architecture?
10 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved

11. Data Grid 
11 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved

12. Putting it all together
12 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved

13. The 3 Most Popular Words on Twitter?
1. Just
2. Found
3. Love
- August 2012
13 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved

14. Q&A
RonZ@gigaspaces.com
14 ® Copyright 2011 Gigaspaces Ltd. All Rights Reserved