Big data analytics involves analyzing large and complex datasets. There are different types and orders of analytics, including first order analytics of individual data points and second order analytics involving relationships between data points. Examples of second order analytics are basket analysis of related purchased items, collaborative filtering to make recommendations, and social network analysis to understand user connections. Popular platforms for big data include Hadoop for storage and MapReduce for distributed processing, while newer technologies like Spark are gaining popularity. Understanding users and their relationships is key to predicting their needs and behavior through analytics.

1. Big Data Analytics
Applications and Platforms
Sina Sohangir
sina@adpardaz.com

2. BigData Analytics Sina Sohangir
Applications

3. BigData Analytics Sina Sohangir
Big data is like teenage sex: everyone talks
about it, nobody really knows how to do
it, everyone thinks everyone else is doing
it, so everyone claims they are doing it.

4. BigData Analytics Sina Sohangir
Know your users and
predict their needs

5. BigData Analytics Sina Sohangir
• First order analytics
• Second order analytics
• Item to Item: Basket analysis
• User to Item: Collaborative filtering
• User to User: Social network analysis
• User to User
• Explicit social networks
• Implicit social networks
• Customer prediction: Churn Analysis, Credit Scoring, Customer
Preferences/Tastes

6. BigData Analytics Sina Sohangir
First Order Data Analytics
• First order analytics: analyze data points
1. Individually with predefined set of rules or
2. In aggregate e.g. Clustering,Averages, Max, Min
• Scale linearly with sample points: O(n)

7. BigData Analytics Sina Sohangir
First Order Data Analytics

8. BigData Analytics Sina Sohangir
First Order Data Analytics

9. BigData Analytics Sina Sohangir
First Order Data Analytics
Samsung HUAWEI Sony
Alps LGE Asus
‫ایرانسل‬ ‫اول‬ ‫همراه‬ ‫رایتل‬

10. BigData Analytics Sina Sohangir
First Order Data Analytics
Viber WhatsApp Line Instragram Facebook
Dropbox ‫ابزار‬ ‫جعبه‬ ‫آپارات‬ ‫جملک‬ ‫پاپیون‬ ‫سرآشپز‬
‫ابجد‬ ‫فال‬ ‫نه؟‬ ‫یا‬ ‫عاشقی‬ ‫!واقعا‬

11. BigData Analytics Sina Sohangir
Second Order Data Analytics
• Second Order Analytics:
1. Item to Item: Basket analysis
2. User to Item: Collaborative filtering
3. User to User: Social network analysis,
• Scale super linearly

12. BigData Analytics Sina Sohangir
Item to Item: Basket Analysis
• Exact method - Association Rule learning
1. Find frequent item sets
2. Find confidence
• Approximation methods e.g. Min Hashing

13. BigData Analytics Sina Sohangir
Item to Item: Basket Analysis
• Used for product recommendation:

14. BigData Analytics Sina Sohangir
User to Item: Collaborative Filtering
• Neighborhood based approaches
• Matrix factorization

15. BigData Analytics Sina Sohangir
User to User: Social Network Analysis - 1
• Explicit Social Networks
1. Facebook friendship graph
2. Twitter follow graph
3. Call graph on cellular network

16. BigData Analytics Sina Sohangir
User to User: Social Network Analysis - 2
• Implicit Social Networks
1. Living in the same area
2. Working in the same area
3. Visiting similar websites/using similar mobile apps
4. Purchasing items from the same places
5. Having financial transactions together

17. BigData Analytics Sina Sohangir
User to User: Social Network Analysis - 3
• Predicting user behavior is significantly more accurate
using social information:
1. Churn analysis
2. Credit scoring
3. Customer preferences/tastes

18. BigData Analytics Sina Sohangir
Example: Churn Analysis
• Probability of churn when k friends have already
churned in a call graph

19. BigData Analytics Sina Sohangir
Platforms

20. BigData Analytics Sina Sohangir
Components of a BigData Ecosystem
• Distributed file system
• Distributed database
• Distributed computation
• Cluster management

21. BigData Analytics Sina Sohangir
Distributed File System
• Google File System
• Hadoop File System HDFS

22. BigData Analytics Sina Sohangir
Distributed Database
• BigTable — Google
• Cassandra — Facebook
• Presto — Facebook
• HBase —
• Vertica — HP
• Voldermort — LinkedIn
• Riak

23. BigData Analytics Sina Sohangir
Distributed Computation
• MapReduce — Google
• Hadoop — Apache Foundation
• Dremel — Google
• Spark
• Spark Streaming
• Apache Storm

24. BigData Analytics Sina Sohangir
Hadoop is Old News

25. BigData Analytics Sina Sohangir
Cluster Management

26. BigData Analytics Sina Sohangir
So Whats Hot Right Now?
• Spark — DataBricks
• Mesos — Mesosphere

27. BigData Analytics Sina Sohangir
How To Be On the Edge of BigData?

28. BigData Analytics Sina Sohangir
ThankYou
Interested in mobile advertising or big data analytics?
Contact me at: sina@adpardaz.com