Cloudlytics - Analyze S3 & CloudFront Logs

4,979 views
4,467 views

Published on

Cloudlytics is a Log analysis tool that helps you track all your Amazon S3 and CloudFront logs.
1) Why Log Processing
2) Use Cases
3) Sample Reports
4) What's Next

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,979
On SlideShare
0
From Embeds
0
Number of Embeds
589
Actions
Shares
0
Downloads
8
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Cloudlytics - Analyze S3 & CloudFront Logs

  1. 1. 1
  2. 2. 2. e-Learning 3. Media 4. Other Content Distribution 1. Customizable Reports 2. Popular Downloads 2. SPAM Attack Alerts 3. SPAM Attacks 4. Streaming Pattern Coming Soon 3. With Cloudlytics 1. Geographical Distribution Reports 2. Traditional Way 1. ISVs Use Cases Why Log Processing 1. Hidden Information in Logs 3 .External Plugins
  3. 3. Information Hidden in S3 & CloudFront Logs Object Details Download Status Download /Streaming Time Number of Bytes Transferred Details about Edge Locations IP Address of the Requester Referrer Link Time Taken to Download Object(S3) Details about Play, Pause, Stop (Streaming Content on CloudFront) 3 -
  4. 4. Uncover the Hidden Information • Generating logs for Amazon S3 & CloudFront logs is optional S3 logs • Log files are stored in S3 buckets CloudFront logs • CloudFront Log files are compressed Logs Stored in S3 and stored in .gz format • A log file is generated every hour, but we have seen varied patterns with multiple files generated every hour • No ready solution from AWS to process these log files 4 Logs Analyzed by CLOUDLYTICS
  5. 5. Traditional Log Processing • Extract data from the source using an ETL tool • Transform data and load in a data-warehouse • Takes days to process a few GBs of log files using traditional hardware • Alternately use a Hadoop distribution to process logs • But maintaining a Hadoop cluster is a huge overhead 5 Image Courtesy: www.fao.org
  6. 6. Log Processing with Cloudlytics • Cloudlytics - Analyze your Amazon S3 & CloudFront Logs • Detailed analysis of your S3 & CloudFront access patterns • Dynamic Graphs to get a 360 degree perspective • Scalable & Reliable service built using Amazon EMR & RedShift • Pay as you go 6
  7. 7. Log Processing – Big Data Approach • Cloudlytics extracts log files stored in S3 buckets • Processes the log files to transform information • Stores the processed data in a data-warehouse • Graphical and tabular reports generated from data-warehouse 7
  8. 8. Cloudlytics Use Cases 8
  9. 9. Independent Software Vendors (ISVs) • ISVs distribute downloadable software to end users across the globe • ISVs need to ensure that downloads are fast, helps improve user experience • ISVs need to track each download for success and failure • Identify broken links on the website, helps improve user experience • Identify the most popular downloads, focus on popular products • Identify spam attacks, help reduce bandwidth costs 9
  10. 10. E-Learning Companies • E-Learning companies distribute educational content in multiple formats (ebooks, audio, video etc.) • Figure out the most popular content • Figure out end user engagement by querying number of events per request (Play, Pause, Stop) Image Courtesy: http://www.elifescience.in/images • Get a breakup of requests by Operating System and Devices, develop content creation for specific platforms 10
  11. 11. Media Organizations • Large number of media assets available online • Content does not go any changes during it’s life cycle • Some content is extremely popular while others do not get any views • Identify the most popular content, set caching mechanism • Figure out end user engagement by querying number of events per request (Play, Pause, Stop) • Identify the edge locations from which the content is downloaded the most, optimize billing using CloudFront pricing Class 11
  12. 12. Cloudlytics REPORTS Which Reports Resonate with Your Business Needs? 12
  13. 13. Geographic Reports 13
  14. 14. Browser & OS Statistics 14
  15. 15. Detail IP Monitoring 15
  16. 16. Timeline Charts 16
  17. 17. The TOP 10 17
  18. 18. Edge Location Traffic 18
  19. 19. Get Started in 3 Easy Steps Register for Free Configure your Log buckets Analyze & Generate Reports 19
  20. 20. Pricing • No upfront costs • Register for free and analyze upto 25MB logs/month FREE • Pay only for the amount of logs you subscribe 20
  21. 21. To sum it Up Identifying Popular Downloads & Streams Identify Edge Location Traffic and optimize AWS Billing Get Geographical Distribution of Downloads & Streams Identify Spam Attacks Get Streaming Pattern for Video Content Improve User Experience by Calculating & Optimizing Latencies 21
  22. 22. Our Global User Reach 80+ Users and Counting! 22
  23. 23. Coming Soon! Image Courtesy: http://www.trophies.com/coming-soon/ 23
  24. 24. Thank You Visit :Cloudytics.com Contact: support@cloudlytics.com Call: +1 212 960 8532 Follow Us On : Our Blog : http://blog.blazeclan.com/

×