Your SlideShare is downloading. ×
0
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports

2,360

Published on

This presentation talks about the Following - …

This presentation talks about the Following -

-Working of AWS S3 & CloudFront Logs with respect to
Content Storing and Distribution.
-The hidden potential of your Stored S3 & CloudFront Logs
& Unlocking them with Cloudlytics
-Some of our Reports using Cloudlytics

Check the video embedded after the slideshare for a Live recording of our webinar conducted around this topic.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,360
On Slideshare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
17
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Featuring Our Latest Reports
  • 2. AGENDA Introduction To Amazon S3 & CloudFront Log Processing Using AWS Vs Traditional ways Log Processing With Cloudlytics- Big Data Approach Cloudlytics Use cases Cloudlytics Reports & Live Demo 2
  • 3. What is Amazon Simple Storage Service or S3? • Amazon Simple Storage Service is storage for the internet • Stores 2 trillion+ objects • 1.1 million requests per second at peak • Each time a request is made to access a file on S3, an entry to log file is created • Average size of each log entry 550B Image Courtesy: http://threatpost.com/files/2013/03/ 3
  • 4. What is Amazon CloudFront? Amazon CloudFront is a web service for content delivery 1 CloudFront decreases latencies 2 for object downloads and streams 5 Average size of each log entry 650B 4 3 Supports static and dynamic content, including web pages 4 Each time a request is made to access a file on CloudFront, an entry to log file is created
  • 5. Information Hidden in S3 & CloudFront Logs Object Details Download Status Download /Streaming Time Number of Bytes Transferred Details about Edge Locations IP Address of the Requester Referrer Link Time Taken to Download Object(S3) Details about Play, Pause, Stop (Streaming Content on CloudFront) 5 -
  • 6. Uncover the Hidden Information • Generating logs for Amazon S3 & CloudFront logs is optional S3 logs • Log files are stored in S3 buckets CloudFront logs • CloudFront Log files are compressed Logs Stored in S3 and stored in .gz format • A log file is generated every hour, but we have seen varied patterns with multiple files generated every hour • No ready solution from AWS to process these log files 6 Logs Analyzed by CLOUDLYTICS
  • 7. Traditional Log Processing • Extract data from the source using an ETL tool • Transform data and load in a data-warehouse • Takes days to process a few GBs of log files using traditional hardware • Alternately use a Hadoop distribution to process logs • But maintaining a Hadoop cluster is a huge overhead 7 Image Courtesy: www.fao.org
  • 8. Log Processing with Cloudlytics • Cloudlytics - Analyze your Amazon S3 & CloudFront Logs • Detailed analysis of your S3 & CloudFront access patterns • Dynamic Graphs to get a 360 degree perspective • Scalable & Reliable service built using Amazon EMR & RedShift • Pay as you go 8
  • 9. Log Processing – Big Data Approach • Cloudlytics extracts log files stored in S3 buckets • Processes the log files to transform information • Stores the processed data in a data-warehouse • Graphical and tabular reports generated from data-warehouse 9
  • 10. Cloudlytics Use Cases 10
  • 11. Independent Software Vendors (ISVs) • ISVs distribute downloadable software to end users across the globe • ISVs need to ensure that downloads are fast, helps improve user experience • ISVs need to track each download for success and failure • Identify broken links on the website, helps improve user experience • Identify the most popular downloads, focus on popular products • Identify spam attacks, help reduce bandwidth costs 11
  • 12. E-Learning Companies • E-Learning companies distribute educational content in multiple formats (ebooks, audio, video etc.) • Figure out the most popular content • Figure out end user engagement by querying number of events per request (Play, Pause, Stop) Image Courtesy: http://www.elifescience.in/images • Get a breakup of requests by Operating System and Devices, develop content creation for specific platforms 12
  • 13. Media Organizations • Large number of media assets available online • Content does not go any changes during it’s life cycle • Some content is extremely popular while others do not get any views • Identify the most popular content, set caching mechanism • Figure out end user engagement by querying number of events per request (Play, Pause, Stop) • Identify the edge locations from which the content is downloaded the most, optimize billing using CloudFront pricing Class 13
  • 14. Cloudlytics REPORTS Which Reports Resonate with Your Business Needs? 14
  • 15. Geographic Reports 15
  • 16. Browser & OS Statistics 16
  • 17. Detail IP Monitoring 17
  • 18. Timeline Charts 18
  • 19. The TOP 10 19
  • 20. Edge Location Traffic 20
  • 21. DEMO Let’s Look at Cloudlytics in Action Image Courtesy: SourceKeyit.com 21
  • 22. How to get started? So Where do you Get Started ?? Image courtesy: http://blogs.position2 .com/best-of-the-week-august-24-2012 22
  • 23. Get Started in 3 Easy Steps Register for Free Configure your Log buckets Analyze & Generate Reports 23
  • 24. Pricing • No upfront costs • Register for free and analyze upto 25MB logs/month FREE • Pay only for the amount of logs you subscribe 24
  • 25. Advantage - Cloudlytics • Scalable & Reliable • Developed using Amazon Web Services tools like Amazon EMR & Amazon Redshift • Developed by BlazeClan Technologies, leading Consulting Partner with Amazon Web Services • Pay as you go service with no contracts and no lock-ins 25
  • 26. To sum it Up Identifying Popular Downloads & Streams Identify Edge Location Traffic and optimize AWS Billing Get Geographical Distribution of Downloads & Streams Identify Spam Attacks Get Streaming Pattern for Video Content Improve User Experience by Calculating & Optimizing Latencies 26
  • 27. Our Global User Reach 80+ Users and Counting! 27
  • 28. Coming Soon! Image Courtesy: http://www.trophies.com/coming-soon/ 28
  • 29. Upcoming Webinar Check out Our Upcoming Webinars @ blazeclan.com/webinars 29
  • 30. Thank you Visit :Cloudytics.com Contact: support@cloudlytics.com Follow Us On : Our Blog : http://blog.blazeclan.com/

×