,    D                    W
Z    d                




    Mike Gualtieri - Senior Analyst, Architecture  Design, Forrester Research




    Steve Tack – Compuware APM CTO
    Twitter: smtack
No Website Is An Island

Mike Gualtieri, Principal Analyst
August 30, 2011




2   © 2010 Forrester Research, Inc. Reproduction Prohibited
      2009
Blog
  Advertising                                     No Website is an
          Analytics
                      News Feed                       island.


                              Social Network




                                                                Cloud Objects
                                                Search Engine
                       APIs


Shopping Cart
                                     Video

                                             Rating  Reviews
Proof that no website is an island!




                                                              Number of hosts per
                                                              user transactions by
                                                                   geography
4   © 2010 Forrester Research, Inc. Reproduction Prohibited
Why are websites
                                                              designed to use 3rd-party
                                                                  components and
                                                                     services?




5   © 2010 Forrester Research, Inc. Reproduction Prohibited
3rd-party components can
improve your customers’
        experience.
7   © 2010 Forrester Research, Inc. Reproduction Prohibited
3rd-party components
can help you deliver
 functionality faster.
9   © 2010 Forrester Research, Inc. Reproduction Prohibited
However,   …




                … there is a dark side
               to 3rd-party components
                          too.
3rd-party components can threaten the seven
 qualities of great websites
                   Quality                                        What it means
                                              Users’ perceptions of the usefulness, usability, and
     1      Experience
                                              desirability of the website
                                              The readiness of an website to perform its functions
     2      Availability
                                              when needed
                                              The speed with which an website performs a function
     3      Performance
                                              to meet business and user expectations
                                              Handle increasing or decreasing volumes of
     4      Scalability
                                              transactions, services, and data
                                              The ease with which a website can be changed or
     5      Adaptability
                                              extended
                                              Supports the security properties of confidentiality,
     6      Security                          integrity, authentication, authorization, and
                                              nonrepudiation
                                              Minimize the cost to build, operate, and change an
     7      Economy
                                              website without comprising its business value
11   © 2010 Forrester Research, Inc. Reproduction Prohibited
3rd-party component
             applies to mobile as
                      well.




12   © 2010 Forrester Research, Inc. Reproduction Prohibited
Blog
                                            The benefits of using
  Advertising                               3rd-party components
                                             outweigh the risks.
          Analytics   News Feed


                       Social Network




                                                             Cloud Objects
                                             Search Engine
Shopping Cart
                                  Video

                                          Rating  Reviews
But, 3rd-party means you don’t control it

     Service level agreements (SLAs) let you point the finger, but your
     customers will still hold you responsible
     Fault tolerance means that your cash or impression engine can continue
     to run even if a 3rd-party components are down
     Your business can not be held hostage by a 3rd-party service




14   © 2010 Forrester Research, Inc. Reproduction Prohibited
You need to support 3rd-party
   components, but mitigate their
risk on your customer experience.
Advice to mitigate risks of 3rd-party
components:
1. Choose 3rd-party components wisely
2. Decide on a mitigation strategy
3. Test it under all conditions
4. Monitor the user experience at all
   times. (Compuware to present)
Advice 1: Choose 3rd-party components wisely

     Do you know all the 3rd-party components you are using now?
       – Ask your developers, eCommerce professionals, web content managers, and
         marketing department?

     Perform a bit of due diligence on the component provider
       – Is this an established firm like Yahoo Shopping Cart or Google Analytics?
       – Is it a startup that has not yet gained momentum?
       – Can you get usage, outage, and performance data for the component?
       – Do your competitors or other web sites use the component?
       – If you choose to be an early adopter to gain competitive advantage, just make
         sure you mitigate the risks

     Assign a confidence level of 1 to 5 to their service level agreements (SLA)
       – See chart on next slide




17   © 2010 Forrester Research, Inc. Reproduction Prohibited
Use your SLA confidence level to choose
 components and establish a mitigation strategy

                       Confidence                                        Description
                         Level
                                   1                      No SLA published
                                   2                      SLA is under requirements or has rocky
                                                          history of breaches
                                   3                      SLA meets requirements and has had some
                                                          breaches
                                   4                      SLA meets requirements as has had
                                                          insignificant history of breaches
                                   5                      SLA exceeds requirements and has
                                                          insignificant history of breaches




18   © 2010 Forrester Research, Inc. Reproduction Prohibited
Twitter publishes performance and availability
 statistics for their APIs




19   © 2010 Forrester Research, Inc. Reproduction Prohibited
Advice 2: Decide on a mitigation strategy

     For each 3rd-party component that is used in your website, decide on a
     mitigation strategy in the event that the component fails to meet your
     requirements.
       – See mitigation strategies on next slide

                   Failure                                           Description
                  Condition
           Functionality                            The component is no longer providing the
           Bugs                                     correct or full functionality due to a change in
                                                    functionality or bugs
           Availability                             The component is not responding to
                                                    requests
           Performance                              The component is taking to long to process
                                                    the request
           Scalability                              The component is not able to handle the
                                                    number of requests
20   © 2010 Forrester Research, Inc. Reproduction Prohibited
3rd-party component mitigation strategies

     None
       –     You have decided that the risk is low or the cost to mitigate the risk is too high to have a
             strategy
       –     E.g. You are willing to miss stats for a period of time if Google analytics goes down

     Code-around fault tolerance
       –     Your code automatically detects a component failure and seamlessly works around this
             until the component is back online.
       –     E.g. Your eCommerce site accepts orders but will confirm payment later

     Alternate website
       –     Have an alternative website that you can direct users to
       –     E.g. Perhaps this is a prior version or one of you’re A/B tests

     Swap-out
       –     If you encountered persistent failure conditions with a third-party component you will swap
             it out for another component. This will usually require some coding or reconfiguration or
             your website or content.


21   © 2010 Forrester Research, Inc. Reproduction Prohibited
Advice 3: Test it under all conditions

     Automate functional testing of the components
     Test in all browsers

     Load test to make sure it will be:
       – Available
       – Perform
       – Scale




22   © 2010 Forrester Research, Inc. Reproduction Prohibited
•Use 3rd-party components to improve
your customer experience and deliver
functionality, faster
•Use them wisely
Thank you


Mike Gualtieri
mgualtieri@forrester.com
Twitter: mgualtieri

www.forrester.com




  © 2009 Forrester Research, Inc. Reproduction Prohibited
“How to Mitigate the Performance Risk of Third-party Components”




           Steve Tack - Compuware APM CTO
d                                

    /                 K                     h



                                  t
                  ^
        ^
    /


                              a
        ^



            D^
                          s
                                      E
W

^                      W               '
           Z       Z                   ^
K                   D           
                       W           s
^               D           d       W
^                                                         W   '

                        t
d

W
t




Source: 2006  2009 based on Forrester survey data, 2012* data estimated
^                         W       '

                                          E   ,
                   E

                   D

           /

               

                   d

          ,




                   Z

           




Source: Compuware APM platform data
^                          W                     '

        • d                                   d       W                 /


                – D

                – 

                – /


                     E
        • 

                                                      E                   ^       D

                                                  Z                D           E

Source: Compuware APM platform data
Z   Z   ^
Z   Z                                ^

        • Y                                                  ^                    W   t




        • 


                                                                                           W   t
        •                                                                                  ^          /
                                                                                           Avg. # of Hosts
                                                                                           per page




                                  E     19   12   29   12       15   8   9   8   7




Source: Compuware APM platform data
K   D
K                                   D               


                             h^ E                                                            h^ d
                                          W       ^                            ,       W    W




                                                                                                                     W





                                                                                                                     W
^




                                                                                                                     ,
W




                       '         t            '           D           '      t                    '          D
                   W                  /               t           W                  /                t
                           h^ E           W                   /           h^ d                W                  /
                                                  h^ E                                                h^ d
Source: Compuware APM platform data
K   D   

/
• 

• 

•
W                      s


                                                      d                                 


                                                                                                       
                                                              W       W                             
                                                                                             
                                                                                            /^W
                                                                                         W
                s            W                                                         ^
                                            t           
D                   ^                ^       ^                           D
                                                                           /^W


^
                                         E                                           
                                                                                     
                                                                                     E
        t                        D                    t E
    ^                                            K                                          D
W       s

                                      ^   /       




                                                          
                                                             D




Source: Compuware APM platform data
W                      s


                                         d
                                             • t             d                                 
                                             • t
                                             • ,
                                         W                                                                       
                                             • D                     W       W                                
                                                                                                    
                                           • /                                                        /^W
                                                                                                W
                s            W                                                                ^
                                                  t            
D                   ^                ^             ^                            D
                                                                                  /^W
                                                             
^
                                             E                                              
                                                                                            
                                                                                            E
        t                        D                          t E
    ^                                                  K                                           D
^        D   d     W
                   Origin (Amazon EC2 East)
• s


• 

      ^
• d                   Edge Cached (CDN)
^       D   d   W
•   d
•   ^




        s
W

^                  W                   '
           Z       Z                   ^
K                   D           
                       W           s
^               D           d       W

How to Mitigate the Performance Risk of Third-party Web Components

  • 1.
    , D W Z d Mike Gualtieri - Senior Analyst, Architecture Design, Forrester Research Steve Tack – Compuware APM CTO Twitter: smtack
  • 2.
    No Website IsAn Island Mike Gualtieri, Principal Analyst August 30, 2011 2 © 2010 Forrester Research, Inc. Reproduction Prohibited 2009
  • 3.
    Blog Advertising No Website is an Analytics News Feed island. Social Network Cloud Objects Search Engine APIs Shopping Cart Video Rating Reviews
  • 4.
    Proof that nowebsite is an island! Number of hosts per user transactions by geography 4 © 2010 Forrester Research, Inc. Reproduction Prohibited
  • 5.
    Why are websites designed to use 3rd-party components and services? 5 © 2010 Forrester Research, Inc. Reproduction Prohibited
  • 6.
    3rd-party components can improveyour customers’ experience.
  • 7.
    7 © 2010 Forrester Research, Inc. Reproduction Prohibited
  • 8.
    3rd-party components can helpyou deliver functionality faster.
  • 9.
    9 © 2010 Forrester Research, Inc. Reproduction Prohibited
  • 10.
    However, … … there is a dark side to 3rd-party components too.
  • 11.
    3rd-party components canthreaten the seven qualities of great websites Quality What it means Users’ perceptions of the usefulness, usability, and 1 Experience desirability of the website The readiness of an website to perform its functions 2 Availability when needed The speed with which an website performs a function 3 Performance to meet business and user expectations Handle increasing or decreasing volumes of 4 Scalability transactions, services, and data The ease with which a website can be changed or 5 Adaptability extended Supports the security properties of confidentiality, 6 Security integrity, authentication, authorization, and nonrepudiation Minimize the cost to build, operate, and change an 7 Economy website without comprising its business value 11 © 2010 Forrester Research, Inc. Reproduction Prohibited
  • 12.
    3rd-party component applies to mobile as well. 12 © 2010 Forrester Research, Inc. Reproduction Prohibited
  • 13.
    Blog The benefits of using Advertising 3rd-party components outweigh the risks. Analytics News Feed Social Network Cloud Objects Search Engine Shopping Cart Video Rating Reviews
  • 14.
    But, 3rd-party meansyou don’t control it Service level agreements (SLAs) let you point the finger, but your customers will still hold you responsible Fault tolerance means that your cash or impression engine can continue to run even if a 3rd-party components are down Your business can not be held hostage by a 3rd-party service 14 © 2010 Forrester Research, Inc. Reproduction Prohibited
  • 15.
    You need tosupport 3rd-party components, but mitigate their risk on your customer experience.
  • 16.
    Advice to mitigaterisks of 3rd-party components: 1. Choose 3rd-party components wisely 2. Decide on a mitigation strategy 3. Test it under all conditions 4. Monitor the user experience at all times. (Compuware to present)
  • 17.
    Advice 1: Choose3rd-party components wisely Do you know all the 3rd-party components you are using now? – Ask your developers, eCommerce professionals, web content managers, and marketing department? Perform a bit of due diligence on the component provider – Is this an established firm like Yahoo Shopping Cart or Google Analytics? – Is it a startup that has not yet gained momentum? – Can you get usage, outage, and performance data for the component? – Do your competitors or other web sites use the component? – If you choose to be an early adopter to gain competitive advantage, just make sure you mitigate the risks Assign a confidence level of 1 to 5 to their service level agreements (SLA) – See chart on next slide 17 © 2010 Forrester Research, Inc. Reproduction Prohibited
  • 18.
    Use your SLAconfidence level to choose components and establish a mitigation strategy Confidence Description Level 1 No SLA published 2 SLA is under requirements or has rocky history of breaches 3 SLA meets requirements and has had some breaches 4 SLA meets requirements as has had insignificant history of breaches 5 SLA exceeds requirements and has insignificant history of breaches 18 © 2010 Forrester Research, Inc. Reproduction Prohibited
  • 19.
    Twitter publishes performanceand availability statistics for their APIs 19 © 2010 Forrester Research, Inc. Reproduction Prohibited
  • 20.
    Advice 2: Decideon a mitigation strategy For each 3rd-party component that is used in your website, decide on a mitigation strategy in the event that the component fails to meet your requirements. – See mitigation strategies on next slide Failure Description Condition Functionality The component is no longer providing the Bugs correct or full functionality due to a change in functionality or bugs Availability The component is not responding to requests Performance The component is taking to long to process the request Scalability The component is not able to handle the number of requests 20 © 2010 Forrester Research, Inc. Reproduction Prohibited
  • 21.
    3rd-party component mitigationstrategies None – You have decided that the risk is low or the cost to mitigate the risk is too high to have a strategy – E.g. You are willing to miss stats for a period of time if Google analytics goes down Code-around fault tolerance – Your code automatically detects a component failure and seamlessly works around this until the component is back online. – E.g. Your eCommerce site accepts orders but will confirm payment later Alternate website – Have an alternative website that you can direct users to – E.g. Perhaps this is a prior version or one of you’re A/B tests Swap-out – If you encountered persistent failure conditions with a third-party component you will swap it out for another component. This will usually require some coding or reconfiguration or your website or content. 21 © 2010 Forrester Research, Inc. Reproduction Prohibited
  • 22.
    Advice 3: Testit under all conditions Automate functional testing of the components Test in all browsers Load test to make sure it will be: – Available – Perform – Scale 22 © 2010 Forrester Research, Inc. Reproduction Prohibited
  • 23.
    •Use 3rd-party componentsto improve your customer experience and deliver functionality, faster •Use them wisely
  • 24.
    Thank you Mike Gualtieri mgualtieri@forrester.com Twitter:mgualtieri www.forrester.com © 2009 Forrester Research, Inc. Reproduction Prohibited
  • 25.
    “How to Mitigatethe Performance Risk of Third-party Components” Steve Tack - Compuware APM CTO
  • 26.
    d / K h t ^ ^ / a ^ D^ s E
  • 27.
    W ^ W ' Z Z ^ K D W s ^ D d W
  • 28.
    ^ W ' t d W t Source: 2006 2009 based on Forrester survey data, 2012* data estimated
  • 29.
    ^ W ' E , E D / d , Z Source: Compuware APM platform data
  • 30.
    ^ W ' • d d W / – D – – / E • E ^ D Z D E Source: Compuware APM platform data
  • 31.
    Z Z ^
  • 32.
    Z Z ^ • Y ^ W t • W t • ^ / Avg. # of Hosts per page E 19 12 29 12 15 8 9 8 7 Source: Compuware APM platform data
  • 33.
    K D
  • 34.
    K D h^ E h^ d W ^ , W W W W ^ , W ' t ' D ' t ' D W / t W / t h^ E W / h^ d W / h^ E h^ d Source: Compuware APM platform data
  • 35.
    K D / • • •
  • 36.
    W s d W W /^W W s W ^ t D ^ ^ ^ D /^W ^ E E t D t E ^ K D
  • 37.
    W s ^ / D Source: Compuware APM platform data
  • 38.
    W s d • t d • t • , W • D W W • / /^W W s W ^ t D ^ ^ ^ D /^W ^ E E t D t E ^ K D
  • 39.
    ^ D d W Origin (Amazon EC2 East) • s • ^ • d Edge Cached (CDN)
  • 40.
    ^ D d W • d • ^ s
  • 41.
    W ^ W ' Z Z ^ K D W s ^ D d W