The document provides an overview of Amazon S3 and CloudFront log processing using Cloudlytics, highlighting its advantages over traditional log processing methods. It discusses the features of S3 and CloudFront, the insights gained from log analysis, and various use cases for independent software vendors, e-learning companies, and media organizations. Cloudlytics offers scalable, reliable log processing services with a pay-as-you-go pricing model, focusing on improving user experience and optimizing AWS billing.

1. Featuring Our
Latest
Reports

2. AGENDA
Introduction To Amazon S3 & CloudFront
Log Processing Using AWS Vs Traditional
ways
Log Processing With Cloudlytics- Big Data
Approach
Cloudlytics Use cases
Cloudlytics Reports & Live Demo
2

3. What is Amazon Simple Storage Service or S3?
• Amazon Simple Storage Service is storage for the
internet
• Stores 2 trillion+ objects
• 1.1 million requests per second at peak
• Each time a request is
made to access a file on S3,
an entry to log file is created
• Average size of each
log entry 550B
Image Courtesy: http://threatpost.com/files/2013/03/
3

4. What is Amazon CloudFront?
Amazon CloudFront is a
web service for content
delivery 1
CloudFront
decreases
latencies 2
for object
downloads
and streams
5
Average size of each
log entry 650B
4
3
Supports static
and dynamic
content, including
web pages
4
Each time a request
is made to access a file
on CloudFront, an entry
to log file is created

5. Information Hidden in S3 & CloudFront Logs
Object Details
Download Status
Download /Streaming
Time
Number of
Bytes
Transferred
Details about Edge
Locations
IP Address of the
Requester
Referrer Link
Time Taken to
Download Object(S3)
Details about Play,
Pause, Stop (Streaming
Content on CloudFront)
5
-

6. Uncover the Hidden Information
• Generating logs for Amazon S3 &
CloudFront logs is optional
S3 logs
• Log files are stored in S3 buckets
CloudFront
logs
• CloudFront Log files are compressed
Logs Stored
in S3
and stored in .gz format
• A log file is generated every hour, but we
have seen varied patterns with multiple files
generated every hour
• No ready solution from AWS to process
these log files
6
Logs Analyzed by
CLOUDLYTICS

7. Traditional Log Processing
• Extract data from the source using an ETL tool
• Transform data and load in a data-warehouse
• Takes days to process a few GBs of
log files using traditional hardware
• Alternately use a Hadoop distribution
to process logs
• But maintaining a Hadoop cluster is a huge overhead
7
Image Courtesy: www.fao.org

8. Log Processing with Cloudlytics
• Cloudlytics - Analyze your Amazon S3 &
CloudFront Logs
• Detailed analysis of your S3 & CloudFront
access patterns
• Dynamic Graphs to get a 360 degree
perspective
• Scalable & Reliable service built using
Amazon EMR & RedShift
• Pay as you go
8

9. Log Processing – Big Data Approach
• Cloudlytics extracts log files stored
in S3 buckets
• Processes the log files to
transform information
• Stores the processed data in
a data-warehouse
• Graphical and tabular reports generated from data-warehouse
9

10. Cloudlytics Use Cases
10

11. Independent Software Vendors (ISVs)
• ISVs distribute downloadable software to
end users across the globe
• ISVs need to ensure that downloads are
fast, helps improve user experience
• ISVs need to track each download for
success and failure
• Identify broken links on the website, helps improve user
experience
• Identify the most popular downloads, focus on popular products
• Identify spam attacks, help reduce bandwidth costs
11

12. E-Learning Companies
• E-Learning companies distribute educational
content in multiple formats
(ebooks, audio, video etc.)
• Figure out the most popular content
• Figure out end user engagement by querying
number of events per request (Play, Pause, Stop)
Image Courtesy: http://www.elifescience.in/images
• Get a breakup of requests by Operating System and Devices,
develop content creation for specific platforms
12

13. Media Organizations
• Large number of media assets
available online
• Content does not go any changes
during it’s life cycle
• Some content is extremely popular
while others do not get any views
• Identify the most popular content,
set caching mechanism
• Figure out end user engagement by querying number of events per
request (Play, Pause, Stop)
• Identify the edge locations from which the content is downloaded
the most, optimize billing using CloudFront pricing Class
13

14. Cloudlytics REPORTS
Which Reports Resonate
with Your Business Needs?
14

15. Geographic Reports
15

16. Browser & OS Statistics
16

17. Detail IP Monitoring
17

18. Timeline Charts
18

19. The TOP 10
19

20. Edge Location Traffic
20

21. DEMO
Let’s Look at
Cloudlytics in
Action
Image Courtesy: SourceKeyit.com
21

22. How to get started?
So Where do you
Get Started ??
Image courtesy: http://blogs.position2
.com/best-of-the-week-august-24-2012
22

23. Get Started in 3 Easy Steps
Register for Free
Configure your Log
buckets
Analyze & Generate
Reports
23

24. Pricing
• No upfront costs
• Register for free and analyze upto 25MB logs/month FREE
• Pay only for the amount of logs you subscribe
24

25. Advantage - Cloudlytics
• Scalable & Reliable
• Developed using Amazon Web Services
tools like Amazon EMR & Amazon Redshift
• Developed by BlazeClan Technologies,
leading Consulting Partner with
Amazon Web Services
• Pay as you go service with no contracts and no lock-ins
25

26. To sum it Up
Identifying Popular
Downloads & Streams
Identify Edge Location
Traffic and optimize AWS Billing
Get Geographical Distribution
of Downloads & Streams
Identify Spam
Attacks
Get Streaming Pattern
for Video Content
Improve User Experience by
Calculating & Optimizing Latencies
26

27. Our Global User Reach
80+ Users and
Counting!
27

28. Coming Soon!
Image Courtesy: http://www.trophies.com/coming-soon/
28

29. Upcoming Webinar
Check out Our
Upcoming Webinars
@ blazeclan.com/webinars
29

30. Thank you
Visit :Cloudytics.com
Contact: support@cloudlytics.com
Follow Us On :
Our Blog :
http://blog.blazeclan.com/