Super Bowl is the annual championship game of the National Football League (NFL), the highest level of professional American football in the world. Because of its high importance, the commercials broadcasted during the game are also specially created and are very much talked about among the people. These commercials are broadcasted at a very high cost, say $5m for 30 seconds. The objective of this research is to find out the impact of these commercials over people..

1. Shambhavi Khare, Andrew Vernon, Vikal Gupta, Venkatesh Sharma
SuperBowl 2017
Dr. Chong Oh

2. Commercial Example

3. The SuperBowl
Super Bowl 2017 Statistics
• Viewers: 111.3 Million
• 30 sec of air time: $5 Million
• Brands: 56
• Ads: 66
• Famous for entertaining and catchy ads
• 5-6 hours long

4. Introduction - Objectives
• Gather and analyze data from Twitter on Super Bowl
commercials
• Use analyzer to determine overall sentiment of commercial
• Extract brand/product/commercial data and statistics
• Quickly provide results in infographic, word clouds, and white
paper
• Create infrastructure using new technologies

5. Introduction - Infrastructure
Previous Process New Process
PHP Python
REST API Streaming API
MySQL MongoDB
Tableau
Virtual Machines

6. Project Processes
ETL Processing and Cleaning
1. Preparation
a. Prepare servers
b. Create Twitter apps
c. Write Python code
2. Processing
a. Stream data from Twitter to
VMs
b. Extract Transform Load data
on single server
3. Analyses:
a. Sentiment Analysis
b. Data Statistics
c. Word Cloud Analysis
d. Tableau Analysis

7. Process Flow
Real Time Streaming
ETL
Analysis

8. Preparation Work
• Develop python code for automation of the project processes
• Create twitter Apps
• Prepare servers

9. Creating twitter Apps

10. Sample
Twitter
App

11. • Python Code setup in servers
– keywords
– database name, collection names

12. • Server Preparation
– Python 3.5
– MongoDB 3.2
Prepare Servers

13. Commercial Keyword Process
• Monitor commercial for keywords
– Hashtags, brand names, etc.
• Enter keywords in python script
• Record keyword and associated server

16. Real Time Streaming
Source : https://dev.twitter.com/streaming/overview

17. ETL
• Raw Data
• Desired Data
• Tweet Samples
• MongoDB terminologies
• Operators used from pymongo
• Adding fields desired to processed data
• Sentiment Classifier
• Polarity

18. Raw Data

19. Desired Data
AdGeneric
AdSpecific
BrandSpecific
brand
polarity
sentiment
quarter

20. Tweet Samples

21. MongoDB Terminologies

22. Operators used from PyMongo

23. Note Operator : $push
Adding fields desired to processed data

24. Sentiment Classifier : TextBlob
● Noun phrase extraction
● Part-of-speech tagging
● Sentiment analysis
● Classification (Naive Bayes, Decision Tree)
● Language translation and detection powered by Google Translate
● Tokenization (splitting text into words and sentences)
● Word and phrase frequencies
● Parsing

25. Polarity
0
+1-1 -0.6 0.6

26. Analysis
• Data Statistics
• Sentiment Analysis
• Word Cloud Analysis
• Tableau Analysis

27. Data Statistics
Total Ads: 66
Total Brands: 56
Total tweets collected: 1,063,236
Tweets collected during the game : 714447
Tweets collected related to Ads : 259,279
4PM
10PM

28. Sentiment Analysis

29. Word Cloud Analysis

30. Word Cloud Analysis

31. Tableau Analysis
• Popularity-can be misleading
• Count of tweets per time-zone
• Sentiment v/s timezone
• Brand v/s sentiment
• Positive count of tweets per time-zone

32. Analysis-Popularity can be Misleading

33. Analysis-Count of Tweets per Time-zone

34. Analysis-Sentiment v/s Timezone

35. Analysis-Brand v/s Sentiment

36. Analysis-Positive count of tweets per time-zone

37. Analysis- Overall Summary

38. Deliverables
• Infographics
• Learnings from the Infrastructure
– MongoDB(Unstructured Data)
– Python
– Tableau
– Twitter API
– Text Mining
• White Paper
• Research Paper

39. Infographic
● Tweet info
● Brand info
● Negative Analysis
● Positive Analysis
● Timezone Analysis

40. Challenges Faced
• Real Time Streaming
– Capturing correct
keywords
– Time Lag
– ID: String & Integer
• ETL
– Geolocation: Null values
– Multiple Brand Names
• Sentiment Analysis
– Defining the range polarity
• Word Cloud Analysis
– False Positives: Stranger
Things
• Tableau Analysis
– Limited resource
configurations

41. THANK YOU!
Shambhavi Khare, Andrew Vernon, Vikal Gupta, Venkatesh Sharma
Dr. Chong Oh