Developer and Fusion Middleware 1 _ Christian Ledwidge _ Application Performance Monitoring  - Improve User Experience reduce user abandonment.pdf
Upcoming SlideShare
Loading in...5
×
 

Developer and Fusion Middleware 1 _ Christian Ledwidge _ Application Performance Monitoring - Improve User Experience reduce user abandonment.pdf

on

  • 553 views

 

Statistics

Views

Total Views
553
Views on SlideShare
553
Embed Views
0

Actions

Likes
0
Downloads
12
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Developer and Fusion Middleware 1 _ Christian Ledwidge _ Application Performance Monitoring  - Improve User Experience reduce user abandonment.pdf Developer and Fusion Middleware 1 _ Christian Ledwidge _ Application Performance Monitoring - Improve User Experience reduce user abandonment.pdf Presentation Transcript

    • Applicaon  Performance  Monitoring:  Improve  user   experience,  reduce  user  abandonment   Chris&an  Ledwidge   Director  Solu&ons  Architecture   Monitoring  Quest  So9ware   The most comprehensive Oracle applications & technology content under one roof
    • Going  to  tell  you  a  li<le  story  before   we  begin  •  Established  well  established  travel  company  wants  to  get  into  the  online   travel  booking  engine  business   –  They  have  an  established  brand  in  the  package  holiday  market  with   their  own  fleet  of  aircra9  to  service   –  Margins  are  under  the  gun  and  they  need  to  find  new  areas  for   revenue  genera&on  •  Its  seems  an  obvious  choice  they  know  the  travel  business  a9er  all  •  Have  a  good  name  that  should  help  drive  users  to  their  new  website  •  They  ramp  up  a  big  team  of  marke&ng,  IT,  and  business  owners  to  kick  this   project  off  •  Some  solid  ini&al  success  and  word  of  mouth  spreads  and  site  hit  counts   move  in  the  direc&on  they  had  hoped  for     The most comprehensive Oracle applications & technology content under one roof
    • Story  con&nued  •  BUT!  •  Booking  numbers  are  missing  forecasts  by  a  solid  margin  •  Calls  to  their  call  center  are  going  thru  the  roof   –  Business:  these  call  are  ea&ng  up  our  margin  arent  they?  •  Nobody  in  IT  seems  to  be  able  to  tell  the  business  why  they  are  not   conver&ng  the  solid  site  hit  rate  into  completed  transac&ons  •  Lots  of  finger  poin&ng  and  conspiracy  theories  are  geSng  thrown  around   –  It’s  the  network!   –  No  it’s  the  DB!   –  Its  that  new  Java  applica&on  that  THEY  wrote!   –  Why  did  we  ever  go  Virtual?  •  Only  thing  the  business  cares  about  it  why  are  they  not  conver&ng  all  of   these  poten&al  customers  into  paid  clients?   The most comprehensive Oracle applications & technology content under one roof
    • Part  One  User  Experience  Management  (UEM)  Indicators  •  What  are  they  and  what  do  they  do  •  Five  key  indicators  and  how  they  can  be  used  •  What  sources  can  be  used  to  obtain  indicators  •  Important  points  to  remember   The most comprehensive Oracle applications & technology content under one roof
    • UEM  Indicators:   Bridge  the  Gap  between  Business  and  IT   Useable? UEM Indicators Stable? Right Content? • User counts Right Features? • Changes in user behavior Fast Enough? Business • Geographical variations • Problem impact • Service level agreementsWeb User Web App Back-end Systems Available? Fast Enough? Short MTTR? Proactive Problem Solving? IT The most comprehensive Oracle applications & technology content under one roof
    • UEM  Indicator:    User  Counts   Twenty-five users have registered in the last 15 Use Case minutes, but normally at this time, I see 200 or more people registering.Business ITDid  our  promo&onal  campaign  stop   Are  there  any  outages  impac&ng  producing  results  or  is  the  link  broken?   registra&ons  that  we  need  to   communicate  to  the  business?  Is  the  reduc&on  in  registra&ons  isolated  to   How  many  users  are  being  impacted  and  a  specific  region  or  is  it  across  the  board?   from  which  region(s)?   The most comprehensive Oracle applications & technology content under one roof
    • UEM  Indicator:    Changes  in  User  Behavior   Typically 25% of users complete this transaction, but Use Case today we see fewer than 15%; why the changes in user behavior?Business ITDid  the  new  site  rollout  hide  the  submit   Are  problems  with  the  applica&on  causing  bu<on  to  where  some  users  can  find  it   problems  with  this  transac&on  to  where  anymore?   users  can’t  complete  it?  Where  along  the  path  are  we  losing  the   How  can  we  help  the  business  restore  users  and  when  did  it  change?   service  and  measure  the  business  impact   of  the  change?   The most comprehensive Oracle applications & technology content under one roof
    • UEM  Indicator:    Geographical  Varia&ons   We see unexpected geographical variations in user Use Case volumes from our EMEA region.Business ITHow  many  users  are  being  impacted  and   Is  our  European  proxy  server  having  for  how  long?   performance  issues?  Is  this  a  problem  we  are  causing  or  is  it  one   Is  there  a  specific  service  provider  that  we  of  our  applica&on  delivery  partners?   or  an  ISP  that  our  users  are  using   experiencing  issues?   The most comprehensive Oracle applications & technology content under one roof
    • UEM  Indicator:    Problem  Impact   A user just called in and reported a problem that our Use Case monitors didn’t pick up; what is the problem impact?Business ITWhen  did  the  problem  start  and  what   Is  there  a  specific  web  server  area  of  the  site  is  being  impacted?   throwing  these  excep&ons?  Is  the  error  being  displayed  to  end  users  or   Is  the  right  web  administrator  is  it  just  an  image  failing  that  they  don’t   involved    to  get  this  fixed  or  really  see?   diagnosed?   The most comprehensive Oracle applications & technology content under one roof
    • UEM  Indicator:    Service  Level  Agreements   Use Case SLA goals are not being met?Business IT Do  the  performance  agreements  that  are   Are  the  IT  systems  mee&ng  the  agreed-­‐ in  place  guarantee  a  sa&sfactory  user   upon  delivery  of  services  to  end  users?   experience?  When  the  SLA  viola&ons  started  occurring   Were  our  Web,  App,  or  Database  systems  were  there  devia&ons  in  transac&on   at  fault  or  was  it  an  external  interface?  conversions?   The most comprehensive Oracle applications & technology content under one roof
    • User  Experience  Management  Data  Sources     •  User Counts •  Changes in User Behavior •  Geographical Variations •  Problem Impact •  Service Level AgreementsData Sources •  Web Log Files User Experience •  Synthetic Transaction Robots Management •  Client-Side Tagging •  Web Traffic Analysis Appliances Indicators •  Content Capture Appliances •  User Surveys The most comprehensive Oracle applications & technology content under one roof
    • Web Log FilesData Source Usefulness Limitations •  Error tracking •  Difficult to identify •  User click stream analysis individual user sessions. •  Demographic analysis •  Data set is partial due to caching. •  Performance The most comprehensive Oracle applications & technology content under one roof
    • Synthetic Transaction RobotsData Source Usefulness Limitations •  Measures transaction •  Measures only a small performance and percentage of locations. availability via scripted •  Exercises a small transactions executed percentage of the from remote locations at technology stack in larger set intervals. environments. •  Sometimes referred to as a “clean room” measurement. •  Good for service level reporting. The most comprehensive Oracle applications & technology content under one roof
    • Client-Side TaggingData Source Usefulness Limitations •  Good for measuring true •  Inserting and managing the scripts adds client performance complexity and overhead •  Gathering browser-based to the application events needed to •  Data security concerns understand Web 2.0 user interactions The most comprehensive Oracle applications & technology content under one roof
    • Web Traffic Analysis AppliancesData Source Usefulness Limitations •  100% sampling of •  Asynchronous applications make it transaction performance, difficult to see cause and volume, and errors. effect. •  Analysis has many •  Analyzed data set is perspectives including by partial due to external caching. server, by geography, by ISP, and even by user. The most comprehensive Oracle applications & technology content under one roof
    • Content Capture AppliancesData Source Usefulness Limitations •  Stores the full user •  Storage capacity sessions, including the concerns data returned from the •  Privacy and security web server. concerns •  Session replay and other perspectives on the session that are attainable only when full payloads are accessible The most comprehensive Oracle applications & technology content under one roof
    • User SurveysData Source Usefulness Limitations •  True voice-of-the- •  Typically only small customer samples are gathered, •  Can validate metric- and small samples may based hypotheses represent extremes. •  Answer questions like: o  Why are customers happy? o  Why are customers ready to leave? The most comprehensive Oracle applications & technology content under one roof
    • Part  Two  •  Measuring  user  experience  in  web  1.0  and  2.0   environments  •  Top  5  performance  measurement  challenges  •  Applying  approaches  to  address  the  challenges  •  Key  elements  to  remember   The most comprehensive Oracle applications & technology content under one roof
    • User  Experience  Management   Performance  Measurements   Internet Web App Back-end Systems Request Response Client  Time   Network  Time   Processing  Time  •   Web  1.0  –  Simple  requests  and  responses  •   Web  2.0  –  Asynchronous  requests  and  responses    •   CDNs  and  content  hos&ng  –  third-­‐party  effects   The most comprehensive Oracle applications & technology content under one roof
    • Web  1.0  –  Simple  Page  Request  No  Third-­‐party   Connection 1 Connection 2 Connection 3 1.  Request  ini&ated   Client Server Client Server Client Server   GET main.html 2.  Server  gets  first  1 packet   2 GET style.css   3.  Server  sends  last   GET frame1.html packet     4.  Final  response   GET image1.gif received  by  client   GET news.asp 3 4 The most comprehensive Oracle applications & technology content under one roof
    • Web  2.0  –  Asynchronous  Requests   Connection 1 1.  Request  ini&ated  Client Server   GET item.js 2.  Server  gets  first  1 packet   2   3.  Server  sends  last   packet   3   4.  Final  response  4 received  by  client   The most comprehensive Oracle applications & technology content under one roof
    • CDNs  and  Remote  Content  –  Third-­‐Party  Effects   Connection 1 Connection 2 Connection 3 Connection 4Client Server Client Server Client Akamai Client AkamaiGET main.html GET style.css 1 GET frame1.html GET perf.js GET news.asp 2 3 4 The most comprehensive Oracle applications & technology content under one roof
    • User  Experience  Management   Measuring  Performance  to  Solve  IT  Challenges   “The user tries to explain her “We initially can’t tell if the problem, but it really doesn’t “We don’t know how manyissue is on the client, in the help me resolve it.” and which users are having network, or buried in the this experience.” backend infrastructure.” “We learn about most “We lack historical data toproblems only when a user proactively detect unwanted calls in” performance trends and their IT root cause.” Internet Web User Web App Back-end Systems The most comprehensive Oracle applications & technology content under one roof
    • Challenge  One   “We  learn  about  most  problems  when  users  call”   Management  Approach   Minimal  Coverage   BeCer  Coverage   Best  Coverage  •  Log  parser-­‐-­‐  alerts  on   •  Log  parser-­‐-­‐  alerts  on   •  Synthe&c  package-­‐-­‐to   errors  or  performance   errors  or  performance     add  transac&on   perspec&ve   •  Synthe&c  package-­‐-­‐to   •  Selec&ve  page   add  transac&on   instrumenta&on-­‐-­‐ perspec&ve   captures  third  party   content  provider  issues   •  Passive  sniffer-­‐-­‐low   overhead  capture  all   solu&on   The most comprehensive Oracle applications & technology content under one roof
    • Challenge  Two   “User  explanaon  is  not  useful  for  resoluon”   Management  Approach   Minimal  Coverage   BeCer  Coverage   Best  Coverage  •  Call  the  user(s)  back  and   •  Enlist  support  teams  to   •  Selec&ve  page   inquire  about  issue   recreate  issue   instrumenta&on-­‐-­‐ captures  desktop  events  •  Scrape  web  logs  for   • Use  online  desktop   •  Passive  sniffer-­‐-­‐captures   traces  of  informa&on   sharing  to  record   full  content  and   about  the  issue   customer  scenario   recreates  scenario   The most comprehensive Oracle applications & technology content under one roof
    • Challenge  Three   “Can’t  isolate  issue  to  client,  network,  or  backend”   Management  Approach   Minimal  Coverage   BeCer  Coverage   Best  Coverage  •  Look  for  anomalies  in   •  Look  for  anomalies  in   •  Selec&ve  page   web  logs-­‐-­‐dismiss   web  logs-­‐-­‐dismiss   instrumenta&on-­‐-­‐ problem  if  no  anomalies   problem  if  no  anomalies   captures  third-­‐party   content  performance   • Use  synthe&c  solu&on  to   •  Passive  sniffer-­‐-­‐ isolate  issue-­‐-­‐if  it  shows   differen&ates  client  from   up   network  &me  for   requests   The most comprehensive Oracle applications & technology content under one roof
    • Challenge  Four   “Don’t  know  how  many  users  experiencing  issue”   Management  Approach   Minimal  Coverage   BeCer  Coverage   Best  Coverage  •  Count  anomalies  in  web   •  Count  anomalies  in  web   •  Selec&ve  page   logs-­‐-­‐es&ma&ng  impact   logs-­‐-­‐es&ma&ng  impact   instrumenta&on  when   from  counts   from  counts   third-­‐party  content  being   served   •  Use  synthe&c  solu&on  to   •  Passive  sniffer-­‐-­‐assess   judge  impact  according   impact  of  all  first-­‐party   to  bad  loca&ons   requests   The most comprehensive Oracle applications & technology content under one roof
    • Challenge  Five   “No  historic  info  to  detect  performance  anomalies”   Management  Approach   Minimal  Coverage   BeCer  Coverage   Best  Coverage  •  Customer  sa&sfac&on   •  Web  log  analysis  tools   •  Selec&ve  page   surveys   with  long  term  reten&on   instrumenta&on  when   third-­‐party  content  being   served  •  Web  log  analysis  tools   •  Synthe&c  solu&on  with   •  Passive  sniffer-­‐-­‐for  first   with  long  term  reten&on   base  lining  capabili&es   party  base-­‐lining  and   anomaly  detec&on   The most comprehensive Oracle applications & technology content under one roof
    • Important  Points  to  Remember  •  User Experience Management encourages a common understanding between IT and the business•  Common indicators enable collaboration•  Sources of User Experience Management information are many—use those that best fit your needs The most comprehensive Oracle applications & technology content under one roof
    • Part  Three  •  Why  Manage  Transac&on  Conversions    •  Web  Analy&cs  vs.  Transac&on  Conversion   Analysis    •  Measuring  Transac&on  Conversions    •  Top  5  Challenges  to  Managing  Conversions    •  Points  to  Remember     The most comprehensive Oracle applications & technology content under one roof
    • Why  Manage  Transac&on  Conversions  •  Why  do  we  manage  transac&ons  the  way   we  do?  •  Example  Transac&on  Types   ü Shopping  Carts   ü Ad  campaigns   ü Customer  registra&on   ü Login   The most comprehensive Oracle applications & technology content under one roof
    • Why  Transac&on  Analysis—We  already  have  Web  AnalyCcs?  Web  Analycs   Transacon  Analysis  •  Primary  goal:    How  effec&ve  your   •  Primary  goal:    What  kind  of   marke&ng  is…   experience  your  users  have…  •  Answers  “How  effec&ve  is  my   •  Answers  “What  influenced  user   marke&ng?”   transac&on  conversions?”  •  Observe  how  users  use  your   •  Observe  how  user  transac&on   website:   conversions  are  affected  by:   –  What  transac&ons  they  use  and  how   –  Content   o9en   –  Applica&on  func&onality   –  Click  stream  analysis   –  Applica&on  useability   –  Applica&on  stability   Complementary solutions   The most comprehensive Oracle applications & technology content under one roof
    • User  Experience  Management  TransacCon  Conversion  Analysis   The most comprehensive Oracle applications & technology content under one roof
    • Measuring  Transac&on  Conversions   Session Page 1 Page 2 Page 3 Page 4 Hit Hit Async Hit Hit Async Hit Hit Async Hit Hit Async Transaction Start Step 2 Step 3 End Capturing  Measurements  •  Unique  Request  ID  (Off  the  wire)   •  JavaScript  Instrumenta&on   –  URL,  Page  Title,  Content,  Variable,  Loca&on,   –  Captures  page  or  request  level   etc..,   informa&on  per  session   –  Persists  current  transac&on  state  per  session  in   –  Reports  transac&on  progress  back  to   memory   central  monitoring  server   –  Result  sent  to  a  third  or  first  party   The most comprehensive Oracle applications & technology content under one roof
    • User  Experience  Management    Increasing  Conversion  Rates  through  Conversion  Analysis     “We know where people“I need to understand bad “We can’t identify how are abandoning theconversion trends immediately many and which users website but we don’tto reduce lost business.” are associated with understand why.” conversion issues.”“How much money have “When people abandon the we lost because of this site, they often call customer problem?” service – which drives up our Business cost of service.” The most comprehensive Oracle applications & technology content under one roof
    • “Revenue  loss  -­‐  a  Key  Indicator  example”  1.   Recognized  add-­‐to-­‐cart  page   My  Transacon  Record  2.   Page  analyzed   My  Transacon  Record   Transac&on  Name   Buy  Funnel   •   Parse  $$  amount   Transac&on  Name   Buy  Funnel  3.   Updates  Transac&on  Record   Username   Mike  Smith   Username   Mike  Smith   •   Add  to  current  Cart  Amount   Browser     IE  8  4.   Transac&on  Completes  Successfully   Browser     IE  8   Cart  Amount   $625  5.   Increment  Sale  Amount   Cart  Amount   ….   •   Set  Lost  Sale  Amount  to  0   Transac&on  State   Ac&ve   Transac&on  State   Complete  Success   Sale  Amount   N/A   Sale  Amount   $625   Lost  Sale  Amount   N/A   The most comprehensive Oracle applications 0   Lost  Sale  Amount   & technology content under one roof
    • “Understanding  Bad  Conversion  Trends”   My  Transacon  Record  Transac&on  Name   Buy  Funnel  Username   Mike  Smith  Browser     IE  8  Cart  Amount   $625  Transac&on  State   Ac&ve  Sale  Amount   N/A  Lost  Sale  Amount   N/A  •  Quan&ta&ve  measures  available  in  Analy&cs  and  UEM  •  Allows  us  to  form  a  hypothesis  about  conversion  trends    •  Qualita&ve  valida&on  is  the  next  step  to  validate    •  Session  replay  is  that  valida&on   The most comprehensive Oracle applications & technology content under one roof
    • “The  5  W’s  of  Transac&on  Abandonment”  •  Where  are  users  dropping?   My  Transacon  Record   My  Transacon  Record  •  Which  users  are  dropping?   Transac&on  Name   My  Transacon  unnel   Transac&on  Name   Buy  F Record   Username   My  Transacon  unnel   Buy  F Record   Mike  Smith  •  Why  are  users  dropping?   Transac&on  Name   Username   Username   My  Transacon  unnel   Buy  F Record   Mike  Smith   Browser     Transac&on  Name   8   My  Transacon  unnel   IE   Buy  F Record   Mike  Smith   Browser     Transac&on  Name   8  IE   Buy  Funnel  •  When  did  users  start  dropping?   Cart  Amount   Username   Cart  Amount   Username   Transac&on  State     Browser   $625   Browser     Transac&on  Name   8  IE   $625   Mike  Smith   Ac&ve   IE  8   Buy  Funnel   Mike  Smith   Cart  Amount   Username   $625   Mike  Smith  •  What  were  users  doing  when  they  dropped?   Transac&on  State     Browser   Sale  Amount   Amount   Cart   Transac&on  State     Browser   Sale  Amount   Amount   Cart   Ac&ve   IE  8   N/A   N/A   $625   Ac&ve   IE  8   $625   Lost  Sale  Amount   Transac&on  State   N/A   Ac&ve   Sale  Amount   Amount   Cart   N/A   $625   Lost  Sale  Amount   Transac&on  State   N/A   Ac&ve   Sale  Amount   N/A   Lost  Sale  Amount   Transac&on  State   N/A   Ac&ve   Sale  Amount   N/A   Lost  Sale  Amount   N/A   Sale  Amount   N/A   Lost  Sale  Amount   N/A   Lost  Sale  Amount   N/A   The most comprehensive Oracle applications & technology content under one roof
    • “People calling customer service drive up costs”   •  Relying  on  customers  to  call  in   when  problems  with  transac&ons  is   costly   –  Lost  revenue   –  Damage  to  brand—reduces  repeat   purchases   –  Call  center  personnel  are  expensive   •  Transac&on  Conversion  Analysis   provides  insight  to:   –  Spot  unwanted  conversion  trends  before   they  impact  the  business   –  Increases  efficiency  of  call-­‐center  personnel —look  at  a  user  session  to  recreate  the   problem   The most comprehensive Oracle applications & technology content under one roof
    • Points  to  Remember  ü Transaction analysis is insurance for achieving and maintaining desired business outcomesü Weigh the costs of not doing versus doing conversion analysisü Is analytics enough or do you need something more to make you analytics actionable? The most comprehensive Oracle applications & technology content under one roof
    • Thank  you  for  your  &me      Please  drop  by  the  Quest  booth  in  the  vendor  hall  if  you  would  like  to  have  a  look  at  our  approach  to  Applica&on  Performance  Management   The most comprehensive Oracle applications & technology content under one roof