The twitter consumer sentiment is the most common text classification tool that analyzes an incoming tweet and conveys if the underlying sentiment is positive negative or neutral from the public. Taking India as a case study, we are going to analyze the sentiment of people during each phase of COVID-19 Pandemic.

1. Australia & New Zealand

2. Detecting and Visualizing
COVID-19 Pandemic Public
Sentiment using AWS
Comprehend and QuickSight
Vivek Raja P S | 2020

3. A big salute to all COVID-19
Front-line workers

4. COVID-19, Lockdown & Psychology
Orchestration of AWS Services
Data Collection
Setting up the environment
Extracting Sentiment
Visualizing the results
Q&A
Agenda of the talk

5. COVID-19, Lockdown &
Psychology

6. COVID-19
Coronavirus disease (COVID-19) is an infectious disease caused by a newly
discovered coronavirus.
Most people infected with the COVID-19 virus will experience mild to moderate
respiratory illness and recover without requiring special treatment. Older
people, and those with underlying medical problems like cardiovascular
disease, diabetes, chronic respiratory disease, and cancer are more likely to
develop serious illness.
Source: WHO

7. Lockdown & Quarantine
Taking India as my case-study, we are going to see the measures taken by the
government
First COVID-19
Positive Recorded
Jan 31
Mar 25
Lockdown 1.0
Apr 15
May 04
May 18
Jun 01
Jul 01
Lockdown 2.0
Lockdown 3.0
Lockdown 4.0
Unlock 1.0
Unlock 2.0
Jan 01 2020 Present

8. Psychology during Lockdown

9. Impact of COVID-19

10. Orchestration of AWS Services

11. Solution Workflow

12. Data Collection

13. Tools
Data Source Data Collection Tool Data Preprocessing Tool
Twitter GetOldTweets3 Library Numpy, Pandas,
JSON library

14. Tweet Collection Criterias
1. Language : English
2. Cities
3. Date Ranges
4. Keywords

15. Tweet Collection Criteria - Cities
Six Major Cities
New Delhi
Mumbai
Kolkata
Hyderabad
Bengaluru
Chennai

16. Tweet Collection Criteria - Cities
100km Radius per each city

17. Tweet Collection Criteria- Date Ranges

18. Tweet Collection Criteria- Keywords
15
Top Trending Keywords/ Hashtags

19. Setting up the environment

20. Create AWS Account
Create a new AWS Account if
you don’t have
*Note: Keep an eye on incurring additional costs

21. Create IAM Role
Create an IAM Role with the following roles attached

22. S3 Bucket
Set up the two Storage tools
DynamoDB Table
Set “text” as partition key

23. Extracting Sentiment

24. Lambda Function Setup
The lambda function must be
triggered whenever there is a
new object created in S3 bucket.
The data is sent to AWS
Comprehend to extract the
sentiment.
The results are updated in
DynamoDB table

25. Lambda Function
Runtime: Python 3.8
Import necessary libraries and initialize
AWS Comprehend and DynamoDB

27. Lambda Function
Define the Lambda function where the
event is the S3 object creation trigger
The lambda function takes the S3 object
that was created as event param

29. Lambda Function
For every record of the S3 object, the sentiment is extracted from
AWS Comprehend and the result is updated in DynamoDB

31. Visualizing the Sentiment

32. Export the DynamoDB data
Configure your AWS CLI and then execute the following command to export the
Dynamo DB. Replace the table name as per your config.
$ aws dynamodb scan — table-name covidsentiment — select
ALL_ATTRIBUTES — page-size 500 — max-items 100000 — output json |
jq -r “.Items” | jq -r “(.[0] | keys_unsorted) as $keys | $keys, map([.[
$keys[] ].S])[] | @csv” > table.csv

33. Upload the data to AWS QuickSight
Navigate to AWS QuickSight. Under the Analysis tab, click New Analysis ->
New Dataset -> Upload a file. Click and upload the exported DynamoDB data
CSV file.

34. Visualize the data using AWS QuickSight

35. Visualize the data using AWS QuickSight

36. Future Enhancement

37. How this can be enhanced?
1. You can use AWS Firehose to stream real-time data from Twitter to S3.
2. You can build a data pipeline between DynamoDB / or any other DB to
Quicksight Data source.
3. Host the dashboard as a web app using Elastic Beanstalk

38. Resources

39. How this can be enhanced?
COVID-19 Twitter Dataset:
https://github.com/Vivek0712/covidtwitterdataset
Step by step tutorial blog: https://vivekraja98.medium.com/detecting-and-
visualizing-twitter-sentiment-during-covid-19-pandemic-using-aws-

40. Let’s flatten the curve

41. Open for Q&A
Vivek Raja P S
/vivek0712
/Vivek0712
@Vivekraja007