Measuring & Predicting the fine dust(PM2.5) data

1. AWS IoT, AWS Greengrass For Time Series Analysis
“Measuring & Predicting the fine
dust(PM2.5) data.”
@aniotmaker
정보통신설비 고급감리/Consulting/Firmware Engineer/Back-end Developer
• Available Programming: Node.JS, JavaScript, SQL, Node-RED, Python, C&C++
• Seoul Weather Twitter Leader (https://twitter.com/aniotmaker)
• AWS-IoT User Group Leader (https://www.facebook.com/groups/awsiot)
• Node-RED User Group Leader (https://www.facebook.com/pg/nodereduser)
Haesung Lee
2019. 5. 14.

2. Contents
• Introduction
• Stacks For Seoul Weather Twitter
• Seoul Weather Architecture
• IoT Sensors with JSON format
• AWS IoT Core
• AWS QuickSight
• AWS IoT Analytics on SageMaker
• Tableau Analytics
• Spark Zeppline Analytics
• Twitter-bot
• Line-bot
• AWS CloudWatch
• PM2.5 dust sensor with Prediction
• Twitter-bot for Seoul weather
• Conclusion
• References
• Q & A
@aniotmaker

3. Introduction
!3
@aniotmaker
Ref. https://www.sciencemag.org/news/2017/01/brain-pollution-evidence-builds-dirty-air-causes-alzheimer-s-dementia

4. AWS IoT
Core
AWS
Greengrass
Amazon
QuickSight
Open
Weather Map
APIs
AWS SageMaker
@aniotmakerStacks For Seoul Weather Twitter
https://twitter.com/aniotmaker
!4
Amazon
EMR
Amazon
Athena
Amazon
CloudWatch
Amazon
EC2 Amazon
S3
AWS Glue
Amazon
Athena
Amazon
Kinesis
Firehose
Amazon
Redshift*
Lambda
function

5. Seoul Weather Architecture
AWS IoT
Core
AWS
Greengrass
Door
Sensor
Amazon
CloudWatch
Amazon
QuickSight
AWS IoT Analytics
Dust
Sensor
Open
Weather Map
APIs
Amazon
EC2
Time Series Analysis,
Weather & Dust
Information
!5
+
Train Model
https://twitter.com/aniotmaker
AWS SageMaker
@aniotmaker
Amazon
S3
+
Amazon
Kinesis
Firehose
AWS Glue
ETL
(Data Lake)
Sensors Unit
IoT (ML, Lamda / MQTT, TLS) Unit
Big Data Processing Unit
BI Visualisations Unit
Social Networks Unit
Amazon
Redshift*
(DataWarehouse)
Amazon
EMR
(Hadoop/ Spark)
MQTT
Lambda
function

6. IoT Sensors with JSON format
!6
A Dust Sensor A Door Sensor Open Weather API
@aniotmaker

7. AWS IoT Core
!7
AWS IoT
Core
AWS
Greengrass
Dust
Sensor

8. Amazon QuickSight
!8
AWS IoT
Core
AWS
Greengrass
Amazon
QuickSight
Dust
Sensor

9. AWS IoT Analytics on SageMaker
!9
AWS IoT Analytics
+
Train Model
AWS SageMakerAWS IoT
Core
AWS
Greengrass
Dust
Sensor
Amazon
S3
+

10. Apache Zeppline Analytics
AWS IoT
Core
AWS
Greengrass
Dust
Sensor Amazon
S3
!10

11. Tableau Analytics
!11
AWS IoT
Core
AWS
Greengrass
Dust
Sensor Amazon
S3

12. Twitter-bot
• AWS-IoT + Twitter API
• How To Predict the time of outside exercising?
!12
@aniotmaker

13. Line-bot
!13
• AWS-IoT + Line API
@aniotmaker

14. AWS CloudWatch
!14
@aniotmaker

15. PM2.5 dust sensor with Prediction
!15
@aniotmaker

16. Twitter-bot for Seoul weather @aniotmaker
!16

17. Conclusion @aniotmaker
As you can see, my purpose is to implement time series analysis with AWS IoT.
As much as possible to avoid fine dust
and time to find the best outdoor exercise timing.
Twitter would give you the best outdoor walking and exercise timing for your health
Red-Coloured line
Is prediction data
!17

18. References
• Designing MQTT Topics for AWS IoT Core (White Paper): https://d1.awsstatic.com/whitepapers/Designing_MQTT_Topics_for_AWS_IoT_Core.pdf
• Using Node-RED Library To Wire Telemetry Data From IoT Devices To The Cloud: https://labs.eleks.com/2019/01/node-red-library-iot-cloud.html
• Using AWS IoT Analytics to Prepare Data for QuickSight Time-Series Visualizations: https://aws.amazon.com/blogs/iot/using-aws-iot-analytics-to-
prepare-data-for-quicksight-time-series-visualizations/
• AWS IoT Analytics 기반 시계열 데이터 QuickSight 시각화 방법: https://aws.amazon.com/ko/blogs/korea/using-aws-iot-analytics-to-prepare-data-for-
quicksight-time-series-visualizations/
• AWS Docs: https://docs.aws.amazon.com/index.html
• What Is AWS IoT Greengrass? https://docs.aws.amazon.com/greengrass/latest/developerguide/what-is-gg.html
• 서버리스 실시간 데이터 처리 애플리케이션 구축: https://aws.amazon.com/getting-started/projects/build-serverless-real-time-data-processing-app-
lambda-kinesis-s3-dynamodb-cognito-athena/?trk=gs_card
• AWS 기반 지속 가능한 데이터 분석 플랫폼 구축하기 - 소성운, 지그재그 :: AWS Summit Seoul 2019 : https://www.slideshare.net/awskorea/aws-aws-
summit-seoul-2019-141290115
• AWS 기반 지속 가능한 데이터 분석하기 GitHub: https://github.com/awskrug/datascience-group/tree/master/workshop-sustainable_data_analysis
• Spark Streaming: IoT with Amazon Kinesis and Visualizing with Qubole Notebooks: https://www.qubole.com/blog/spark-streaming-iot-amazon-
kinesis-visualizing-qubole-notebooks/
@aniotmaker
!18

19. Q & A
@aniotmaker
서울날씨 @aniotmaker
@aniotmaker
!19