Introduction to AWS Athena

1. Using AWS Athena to Analyze
Structured Data Stored in S3
Feb 6, 2018 Dave North, Director of DevOps

2. |© 2018 Signiant, Inc.
Agenda
Intro
Athena Overview
ELB logs
ELB Log Analysis
Resources
2

3. |© 2018 Signiant, Inc.
Intro
3

4. |© 2018 Signiant, Inc. 4
ABOUT DAVE
@DaveNorth
linkedin.com/in/dnorth98
dnorth@signiant.com

5. |© 2018 Signiant, Inc.
MOVE LARGE FILES FAST
Over Public &
Private IP NetworksIn Their Entirety Within & Between Enterprises
... And Into the Cloud
5

6. |© 2018 Signiant, Inc.
Athena Overview
6

7. |© 2018 Signiant, Inc.
Serverless SQL Queries
7
SELECT *
FROM my_athena_table
WHERE my_field = ‘foo’

8. |© 2018 Signiant, Inc.
ELB Logs
8

9. |© 2018 Signiant, Inc. 9
ENABLING ELB LOGS

10. |© 2018 Signiant, Inc. 10
SAMPLE ELB LOGS

11. |© 2018 Signiant, Inc.
ELB Log Analysis
11

12. |© 2018 Signiant, Inc.
Old School
12
SELECT elb_status_code, count(*)
FROM my_elb_log_table
GROUP BY elb_status_code
ORDER BY count(*) DESC
ELB LOG ANALYSIS
cat access.log | cut -d '"' -f3 | cut -d ' '
-f2 | sort | uniq -c | sort -rn
Athena School

13. |© 2018 Signiant, Inc.
Demos
13

14. |© 2018 Signiant, Inc.
Resources
14

15. |© 2018 Signiant, Inc.
Resources
Parquet Log: (https://goo.gl/ZK6une)
Papertrail logs with Athena (https://goo.gl/QbW5M5)
https://github.com/Signiant/athena_table_defs
https://github.com/Signiant/aws-ecs-lb-athena-table
https://medium.com/signiant-engineering
15

16. INTELLIGENT FILE MOVEMENT SOFTWARE