Application Analytics helping DevOps with
data driven decision making
Amod Gupta, Sr. Principal Product Manager
Mike Anand, Director of Product Marketing
YESTERDAYTODAY
BANK
Software defined business
Software agility drives rapid business velocity
IT needs to respond in real-time
deliver great customer experience
ensure zero downtime this holiday season
reduce customer churn rates
understand changing usage patterns
stay ahead of performance problems
release new features faster
When business wants to… IT needs to…
“What do you
need to get
these done?”
“Understand
business and
software
performance
correlation in
REAL-TIME!”
PM, Marketing,
Sales, Customer Success
Ops, Dev
Business data Marketing data Operations data
Traditional approach to analytics
IT cannot respond in real-time
Revenue
New Customer Growth
Churn
User preferences
Conversions
Campaign performance
Brower/Mobile performance
App performance
Infrastructure performance
Drop off rates
Loyalty Tiers
Product mix
Revenue per user
Revenue per tier
SEO Performance
Geo locations
Mobile usage
Network performance
Browser performance
Database performance
Network carrier performance
How much revenue was lost today from Tier-1 users
not converting because they experienced poor app
performance?
Enter custom big-data analytics….
5. ANALYZE4. CORRELATE3. STORE DATA1. CHANGE CODE 2. COLLECT DATA
Slow and iterative Cannot keep up with today’s agile apps
Hard to maintain for
fast changing apps
High costs – time, people,
software, hardware
Backwards looking
0110101101011010110101101
0110101101011010110101101
0110101101011010110101101
0110101101
0 1
0 1
0
11
0
1 01 0
1
0
101
011
IT is unable to unlock the data potential
Need specialists to understand
and analyze data
Any new business report
request is a dev project
Business insights are never
current nor granular enough
BIZ ANALYST
MKTG OPS IT OPS
What if you knew in real-time…
Shopping & Merchandise Categories
Ticket Sales
Average Response Time
$232,390
Revenue
impact of
poor
performance
Shops/Merch
generating
highest revenue
Revenue
generated by
ticket sales
Revenue Impact of
Poor Performance
This is the timestamp when
performance issue started
5 MIN
Wait Time
Dining Pass Sales
Users on
each ride
Top Park Rides
What if you knew in real-time…
Merchandise Categories
Ticket Sales
Average Response Time
$232,390
Revenue
impact of
poor
performance
Merchandise
generating
highest revenue
Users on
each ride
Breakdown of
ticket sales
Revenue Impact of
Poor Performance
Top Park Rides
This is the timestamp when
performance issue started
24 MIN
Wait Time
Age of Customers
Three common use cases
Advanced
performance analytics
Usage
analytics
Business impact
analytics
Business impact analytics
Identify failed individual transactions and respond within minutes
Business and IT collaborate to
run win-back campaigns
Capture all slow and failed
transactions + revenue impact
List impacted platinum
customers
Marketing runs
win-back campaigns
10% off
Log
Platinum
customer
British
Airways API
Flight to
London
Total cost
$1,800
Kevin Gold
IOS
AT&T
Laura Lewis
iPad
T-Mobile
Poor
experience
Flight to
New York
Code level
issue
Silver
customer
Total cost
$712
Matt Jacobs
IOS
AT&T
Good
experience
Gold
customer Flight to
Miami
Total cost
$650
Database
issue
Tracy Kraft
iPhone 6
AT&T
Poor
experience
Silver
customer
Code level
issue
Flight to
Tokyo
Total cost
$1,100
Libby Halll
iPad
T-Mobile
Poor
experience
Platinum
customer
Code level
issue
Flight to
New York
Total cost
$850
Jude
Germangy
Platinum
customer
Book Flight
Flight to
London
Travel
Airways API
Total cost
$1,800
Ready for real-time insights
Automatically collected
Fully correlated
Platinum customer, Jude Germangy, failed while booking a flight to London for
$1,800 while using a browser due to issues with the Travel Airways API
Answer individual user questions in real-time
Log
Platinum
customer
British
Airways API
Flight to
London
Total cost
$1,800
Kevin Gold
IOS
AT&T
Laura Lewis
iPad
T-Mobile
Poor
experience
Flight to
New York
Code level
issue
Silver
customer
Total cost
$712
Matt Jacobs
IOS
AT&T
Good
experience
Gold
customer Flight to
Miami
Total cost
$650
Database
issue
Tracy Kraft
iPhone 6
AT&T
Poor
experience
Silver
customer
Code level
issue
Flight to
Tokyo
Total cost
$1,100
Libby Halll
iPad
T-Mobile
Poor
experience
Platinum
customer
Code level
issue
Flight to
New York
Total cost
$850
Answer aggregate questions in real-time
Poor
experience
Poor
experience
Poor
experiencePoor
experience
Poor
experience
Poor
experience
Total cost
$1,800
Total cost
$850
Total cost
$712
Total cost
$1,100
Total cost
$650
A total of $4,785 revenues were at risk due to
poor experience while booking flights
Ready for real-time insights
Automatically collected
Fully correlated
Female Male
New Existing
ProductHome Gallery Basket Checkout
Gender
Customer
Journey
Time of day
Personalized
Device
08:00
12:00
00:00
04:00
04:00-
08:00
12:00
16:00
16:00
20:00
Desktop Mobile
Yes No
Age 21-29<21 30-45 45+
Usage analytics
The what, who, where, and how of users
{Customer Name} User Experience {Customer Name} Cost to Serve
Diamond Customers Performance
Advanced performance analytics
Performance views for specific customers and partners
Diamond Customers Requests
DIVING DEEPER
Architecture
•  Automatic
•  Structured &
Unstructured Data
•  Infinitely Scalable
•  Real Time
•  Visual Query
Building
•  Ad hoc analysis
OPTION 1
SLIDE
Conceptual Visualization
Analytics under the covers
Data
Collection
Data
Processing
& Storage
Data
Visualization
OPTION 1
SLIDE
Under the hood
Data Collection
Data
Collection
Data
Processing
& Storage
Data
Visualization
Controller
Machines
Java Agent
A
P
M
Events Service
A
N
A
L
Y
T
I
C
S
Transaction, Log and User Data
EUM
Processor
Machine Agent
.NET Agent
Analytics Agent
Machine Agent
UI
OPTION 1
SLIDE
Data
Collection
Data
Processing
& Storage
Data
VisualizationPlatform Architecture
Event Service
SaaS: Designed For Petabytes of Daily Data Volume
On-Premises: Start Small & Scale Horizontally
OPTION 1
SLIDE
Data
Collection
Data
Processing
& Storage
Data
VisualizationData Visualization
SaaS numbers in last 24 hours (are we ok showing these? If we are,
I can pull in more data from OA)
WHAT’S NEW
4.2 Features
BTs
(4.1)
Logs
(4.1)
Browser
RUM (GA)
Mobile RUM
(GA)
Custom Events
(New)
API
(New)
Richer Data Set
AppDynamics Query Language
Advanced Search
Visualization &
Reporting
Deeper Insights
Auto-correlation between Business
Transactions & Logs
Grok patters to include
Key Value Pairs
Custom
Dashboard
Funnel
Conversion
Multiple
X and Y
Schedule
Reporting
What’s new in 4.2
Security
Role Based Access Control SSO
Analytics Deep Dive Session
Tomorrow’s session dives more in depth about our
architecutre and upcoming features
Copyright © 2015 AppDynamics. All rights reserved. 23
Thank You
4.2 new data sources
APM Transactions
Browser Real-User
Monitoring
Mobile Real-User
Monitoring
Application &
Infrastructure Logs
01101011010110101101011010110101100
01101011010110101101011010110101100
01101011010110101101011010110101100
AppDynamics
Analytics
Synthetic
Monitoring
Custom Events
Logs and Transactions
new
Dashboard Integration & Reporting
Copyright © 2015 AppDynamics. All rights reserved. 26
Widgets from APM, EUEM & Analytics
on same dashboard
Convert dashboards into daily, weekly
reports
Interactive widgets – Move in and out
of the dashboard
Custom time ranges for each widget
More 4.2 Goodness
Copyright © 2015 AppDynamics. All rights reserved. 27
Log Agent
•  Transaction and Log correlation
•  Support for more log formats
Security •  Role based access control in analytics
Agent •  Node.js agent supports transaction analytics
API •  API support for data ingestion and extraction
New and Improved Widgets
Ranges and buckets Funnel Widget
Copyright © 2015 AppDynamics. All rights reserved. 29
Application
Performance
Business
Performance
AppDynamics Application Analytics
Correlation in real time
Copyright © 2015 AppDynamics. All rights reserved. 30
ADQL
•  Free text query language support
•  Programmatically search through your data
Dashboard &
Reporting
•  Save Analytics widgets to Custom Dashboards
•  Schedule reports
Improved UX
•  New Widgets – Funnel, Live stream, Percentile etc
•  Improvements to existing widgets – Multiple fields on X & Y axes
API
•  API for search and extraction
•  API for custom events ingestion
What’s new in 4.2
Security •  Role based access control for Analytics
Log & Transaction
•  Correlation between logs and transaction using BT GUID
•  Enhancing log collection & parsing

AppSphere 15 - Application Analytics helping DevOps with Data Driven Decision Making

  • 1.
    Application Analytics helpingDevOps with data driven decision making Amod Gupta, Sr. Principal Product Manager Mike Anand, Director of Product Marketing
  • 2.
  • 3.
    Software agility drivesrapid business velocity IT needs to respond in real-time deliver great customer experience ensure zero downtime this holiday season reduce customer churn rates understand changing usage patterns stay ahead of performance problems release new features faster When business wants to… IT needs to… “What do you need to get these done?” “Understand business and software performance correlation in REAL-TIME!” PM, Marketing, Sales, Customer Success Ops, Dev
  • 4.
    Business data Marketingdata Operations data Traditional approach to analytics IT cannot respond in real-time Revenue New Customer Growth Churn User preferences Conversions Campaign performance Brower/Mobile performance App performance Infrastructure performance Drop off rates Loyalty Tiers Product mix Revenue per user Revenue per tier SEO Performance Geo locations Mobile usage Network performance Browser performance Database performance Network carrier performance How much revenue was lost today from Tier-1 users not converting because they experienced poor app performance?
  • 5.
    Enter custom big-dataanalytics…. 5. ANALYZE4. CORRELATE3. STORE DATA1. CHANGE CODE 2. COLLECT DATA Slow and iterative Cannot keep up with today’s agile apps Hard to maintain for fast changing apps High costs – time, people, software, hardware Backwards looking 0110101101011010110101101 0110101101011010110101101 0110101101011010110101101 0110101101 0 1 0 1 0 11 0 1 01 0 1 0 101 011
  • 6.
    IT is unableto unlock the data potential Need specialists to understand and analyze data Any new business report request is a dev project Business insights are never current nor granular enough BIZ ANALYST MKTG OPS IT OPS
  • 7.
    What if youknew in real-time… Shopping & Merchandise Categories Ticket Sales Average Response Time $232,390 Revenue impact of poor performance Shops/Merch generating highest revenue Revenue generated by ticket sales Revenue Impact of Poor Performance This is the timestamp when performance issue started 5 MIN Wait Time Dining Pass Sales Users on each ride Top Park Rides
  • 8.
    What if youknew in real-time… Merchandise Categories Ticket Sales Average Response Time $232,390 Revenue impact of poor performance Merchandise generating highest revenue Users on each ride Breakdown of ticket sales Revenue Impact of Poor Performance Top Park Rides This is the timestamp when performance issue started 24 MIN Wait Time Age of Customers
  • 9.
    Three common usecases Advanced performance analytics Usage analytics Business impact analytics
  • 10.
    Business impact analytics Identifyfailed individual transactions and respond within minutes Business and IT collaborate to run win-back campaigns Capture all slow and failed transactions + revenue impact List impacted platinum customers Marketing runs win-back campaigns 10% off
  • 11.
    Log Platinum customer British Airways API Flight to London Totalcost $1,800 Kevin Gold IOS AT&T Laura Lewis iPad T-Mobile Poor experience Flight to New York Code level issue Silver customer Total cost $712 Matt Jacobs IOS AT&T Good experience Gold customer Flight to Miami Total cost $650 Database issue Tracy Kraft iPhone 6 AT&T Poor experience Silver customer Code level issue Flight to Tokyo Total cost $1,100 Libby Halll iPad T-Mobile Poor experience Platinum customer Code level issue Flight to New York Total cost $850 Jude Germangy Platinum customer Book Flight Flight to London Travel Airways API Total cost $1,800 Ready for real-time insights Automatically collected Fully correlated Platinum customer, Jude Germangy, failed while booking a flight to London for $1,800 while using a browser due to issues with the Travel Airways API Answer individual user questions in real-time
  • 12.
    Log Platinum customer British Airways API Flight to London Totalcost $1,800 Kevin Gold IOS AT&T Laura Lewis iPad T-Mobile Poor experience Flight to New York Code level issue Silver customer Total cost $712 Matt Jacobs IOS AT&T Good experience Gold customer Flight to Miami Total cost $650 Database issue Tracy Kraft iPhone 6 AT&T Poor experience Silver customer Code level issue Flight to Tokyo Total cost $1,100 Libby Halll iPad T-Mobile Poor experience Platinum customer Code level issue Flight to New York Total cost $850 Answer aggregate questions in real-time Poor experience Poor experience Poor experiencePoor experience Poor experience Poor experience Total cost $1,800 Total cost $850 Total cost $712 Total cost $1,100 Total cost $650 A total of $4,785 revenues were at risk due to poor experience while booking flights Ready for real-time insights Automatically collected Fully correlated
  • 13.
    Female Male New Existing ProductHomeGallery Basket Checkout Gender Customer Journey Time of day Personalized Device 08:00 12:00 00:00 04:00 04:00- 08:00 12:00 16:00 16:00 20:00 Desktop Mobile Yes No Age 21-29<21 30-45 45+ Usage analytics The what, who, where, and how of users
  • 14.
    {Customer Name} UserExperience {Customer Name} Cost to Serve Diamond Customers Performance Advanced performance analytics Performance views for specific customers and partners Diamond Customers Requests
  • 15.
  • 16.
    •  Automatic •  Structured& Unstructured Data •  Infinitely Scalable •  Real Time •  Visual Query Building •  Ad hoc analysis OPTION 1 SLIDE Conceptual Visualization Analytics under the covers Data Collection Data Processing & Storage Data Visualization
  • 17.
    OPTION 1 SLIDE Under thehood Data Collection Data Collection Data Processing & Storage Data Visualization Controller Machines Java Agent A P M Events Service A N A L Y T I C S Transaction, Log and User Data EUM Processor Machine Agent .NET Agent Analytics Agent Machine Agent UI
  • 18.
    OPTION 1 SLIDE Data Collection Data Processing & Storage Data VisualizationPlatformArchitecture Event Service SaaS: Designed For Petabytes of Daily Data Volume On-Premises: Start Small & Scale Horizontally
  • 19.
  • 20.
    SaaS numbers inlast 24 hours (are we ok showing these? If we are, I can pull in more data from OA)
  • 21.
  • 22.
    BTs (4.1) Logs (4.1) Browser RUM (GA) Mobile RUM (GA) CustomEvents (New) API (New) Richer Data Set AppDynamics Query Language Advanced Search Visualization & Reporting Deeper Insights Auto-correlation between Business Transactions & Logs Grok patters to include Key Value Pairs Custom Dashboard Funnel Conversion Multiple X and Y Schedule Reporting What’s new in 4.2 Security Role Based Access Control SSO
  • 23.
    Analytics Deep DiveSession Tomorrow’s session dives more in depth about our architecutre and upcoming features Copyright © 2015 AppDynamics. All rights reserved. 23
  • 24.
  • 25.
    4.2 new datasources APM Transactions Browser Real-User Monitoring Mobile Real-User Monitoring Application & Infrastructure Logs 01101011010110101101011010110101100 01101011010110101101011010110101100 01101011010110101101011010110101100 AppDynamics Analytics Synthetic Monitoring Custom Events Logs and Transactions new
  • 26.
    Dashboard Integration &Reporting Copyright © 2015 AppDynamics. All rights reserved. 26 Widgets from APM, EUEM & Analytics on same dashboard Convert dashboards into daily, weekly reports Interactive widgets – Move in and out of the dashboard Custom time ranges for each widget
  • 27.
    More 4.2 Goodness Copyright© 2015 AppDynamics. All rights reserved. 27 Log Agent •  Transaction and Log correlation •  Support for more log formats Security •  Role based access control in analytics Agent •  Node.js agent supports transaction analytics API •  API support for data ingestion and extraction
  • 28.
    New and ImprovedWidgets Ranges and buckets Funnel Widget
  • 29.
    Copyright © 2015AppDynamics. All rights reserved. 29 Application Performance Business Performance AppDynamics Application Analytics Correlation in real time
  • 30.
    Copyright © 2015AppDynamics. All rights reserved. 30 ADQL •  Free text query language support •  Programmatically search through your data Dashboard & Reporting •  Save Analytics widgets to Custom Dashboards •  Schedule reports Improved UX •  New Widgets – Funnel, Live stream, Percentile etc •  Improvements to existing widgets – Multiple fields on X & Y axes API •  API for search and extraction •  API for custom events ingestion What’s new in 4.2 Security •  Role based access control for Analytics Log & Transaction •  Correlation between logs and transaction using BT GUID •  Enhancing log collection & parsing