Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Service Virtualization - Next Gen Testing Conference Singapore 2013


Published on

Most major enterprises have invested millions of dollars on soIware performance lab infrastructure, that develop recurring maintenance effort and costs, unstable environments, and conflicts over constrained resources. To combat this drain and deliver value, companies need to find ways to optimize. Virtualizing the behavior and performance characteristics of test lab dependencies, CA LISA Service Virtualization has helped many enterprises achieving highly available labs that allow earlier performance testing with greater flexibility, at a much lower cost. Techniques discussed in this session include:

Removing capacity constraints from performance testing lab
Optimizing performance management by decomposing SLAs
Shift-Left: conducting incremental, iterative performance testing

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Service Virtualization - Next Gen Testing Conference Singapore 2013

  1. 1. Service  Virtualiza.on:   Reality  is  Overrated.   a  CA  LISA  Service  Virtualiza.on  Best   Prac.ce:  Transform  Performance  and  Scale   Min  Fang +65  9636  8356   NGTC  Conference  (c)  
  2. 2. Agenda     •  •  •  •  Constraints:  The  Enemy    of  Agility   What  is  Service  Virtualiza.on  of  Service  Virtualiza.on   Best  Prac.ce:  Transform  Performance  &  Scale   Next  Genera.on  Conference  (c)  
  3. 3.  Applica.on  Time-­‐to-­‐Market   HOW  DO  WE  INCREASE  “TOUCH  TIME”  AND  REDUCE  “IDLE  TIME”?   DEVELOPMENT   REGRESSION   QA   INTEGRATION   CHALLENGES   §  Constraints  of  dependent  systems  needed   to  move  the  SDLC  forward   §  Lack  of  CollaboraFon  creates  scheduling   conflicts  and  buffers  from  development     through  opera.ons   §  Increasing  Complexity  of  today’s   heterogeneous  enterprise  IT  architectures     §  Need  Complete  visibility  into  what   happens  in  produc.on   PRODUCTION   UAT   PERFORMANCE  
  4. 4. Bridging  the  Gaps  in  DevOps   COLLABORATIVE     PORTFOLIO  PLANNING   OPERATIONS   MONITORING   RELEASE  AUTOMATION   DATA  MINING   SERVICE       VIRTUALIZATION   AGILE   DEVELOP     &  TEST   ENVIRONMENT     MANAGEMENT   CONTINUOUS  FEEDBACK   PLAN     BUILD   RUN   DevOps  brings  the  applica.on  porbolio  forward  to  meet  market  challenges     with  collabora.on  and  efficiency  from  design  to  deployment  
  5. 5. Constraints:  The  Enemy  of  Agility     NGTC  Conference  (c)  
  6. 6. 1.  Insufficient  environment/capacity   for  and  development   UAT  Tester   SOAP   Performance  Tester   AppServer   MainFrame   100%   u.liza.on   Developer   Copyright  ©  2010  CA.  All  rights  reserved.  
  7. 7. 2.  Uncontrollable  environment  growth   SOAP   UAT  Tester   AppServer1   SOAP   Dev  Team1   AppServer2   Keep  adding   MIPS  to  keep   pace  with  new   requirements   SOAP   Training   MainFrame   AppServer3   Copyright  ©  2010  CA.  All  rights  reserved.  
  8. 8. 3.  System  Dependencies  =   Development  &  constraint   Tester/   Developer   SYSTEM   UNDER  TEST               Frontend   MainFrame   1.  Upgrading   MQ  version   ESB   3.  ESB  Team  will  only   complete  development  in  2   weeks  .me   2.  Timesharing   Mainframe  
  9. 9. What  is  Service  Virtualiza.on   NGTC  Conference  (c)  
  10. 10. Service  Virtualiza.on:  How  does  it  work?   CAPTURE   PROCESS   MODEL   R  Structured Conversations R  Heuristics R  Sophisticated Behavior R  Observe R  Analytics R  Understand R  Algorithms R  Dynamic Properties (Dates, values, etc.) R  Protocol-Level R  State R  Scenario Support §  Recorded traffic R  Test Data Sanitation §  Design specs R  Compiled Model vs. Stubs §  Sample RR pairs R  Automatic Healing §  Transaction logs §  Manual Creation §  Byte-Code
  11. 11. Captures  and  Models  Realis.c  Virtual  Services   Legacy   SOAP   ERP          Source              ApplicaFon   Capture     Database   Create  Virtual  Services   JDBC   MQ   §  Automa.cally  capture  realis.c  behavior   §  Op.mize  models  as  desired   Mainframe  
  12. 12. LISA  Virtualizes  test  environments  to  eliminate   costly  constraints   Legacy   SOAP/HTTP          Source              ApplicaFon   IT  resources   now  available   on  your  terms,   .me-­‐to-­‐market   and  team   produc.vity,   allowing  24/7     MQ   JDBC   ERP   VS1   DB1   VS2’   Virtual  Test  Environments   Database   Mainframe   W  W  W  .  I  T  K  O  .  C  O  M  
  13. 13. CA  LISA  Virtual  Service  Environments     Customers  can  virtualize  almost  any  IT  asset     §  Mainframes  &  Mainframe  Components     CICS-­‐CICS   DB2   MQ   §  Plaborms   §  SOA  Protocols   XML   SOAP   REST   HTTP   §  Risk  Solu.on  Providers   §  Credit  Bureau       §  SMS  Gateway  Providers   §  Bill  Payment  Services   §  Card  Processing  Services   §  Teleconference  Service  Provider   §  Travel  System  (GDS)   JMS   MQ   UDDI   .NET   CORBA   JDBC   JAVA  
  14. 14. How  Can  Service  Virtualiza.on  Help?   NGTC  Conference  (c)  
  15. 15. 1.  Model  Applica.on  Under   Development     Source  App   Virtual  service  can  be  modeled   from  design  ar.facts.  Allowing   development  and  integra.on  test   to  start  earlier     Virtual  Service   Model   Design  arFfacts,   transacFon  logs,   WSDL  files,  flat  files,  and   others  
  16. 16. 2.  Using  SV  for  training  environment   (Use  Client  without  Backend)   MainFrame   AppServer   Virtualize   AppServer  
  17. 17. 3.  Virtual  Data  Management   -­‐  For  Development  &   Applica.on   Server   Virtual  Service  Environment     Abu  Bakar     IC:  1234   Acc:  M123     Alvin  Koh     IC:  4567     Acc:  456     Arun  Kailasam    IC:  54354    Acc:D4345         You  have  full  control  over  data  in  virtual  service,  you  can  add  your   data  directly  into  the  virtual  service  for  the  purpose  of:   a)  Scenario-­‐based   b)   Copyright  ©  2010  CA.  All  rights  reserved.  
  18. 18. 3.  Virtual  Data  Management   -­‐  Create  or  Modify  virtual  data  directly   3.   How  do  my  applica.on   handle  -­‐10k  as  account   balance   1.  If  AccountID=   =1313094687566198   2.  Return  user   details  of  JACKY   LEE  (   customer)  
  19. 19. 4.  Using  SV  for  Performance  Engineering   1.  Interface  load   2.  Load   without  u.lizing   mainframe  MIPS   Test   Test   Test   1000  VU   (Interface)   SOAP   Virtualize   AppServer   3.  At  the  same  .me,  understand   the  txn/sec  requirement  and  to   size  your  mainframe  more   accurately     100  tps   (Virtual   Mainframe)  
  20. 20. Aligning  to  CIO  Goals   Do  More  with  Less     Number  of  Test  Environments   Number  of  Training  Environments     Complexity  in  the  environments   4   #1  consumer  retail   bank  of  America   eliminated  $100M  in   hardware   requirements  when   using  LISA  over  5   years.    Over  10x  ROI,   6x  in  year  1   3.5   3   2.5   2   Normal   1.5   1   0.5   CIO  Goal  accomplished!    CONTROL  costs   0   Year  0   Year  1   Year  2   Year  3   Year  4   Year  5   Year  6   Year  7   Halt  growth  by   using  LISA   3Y  TCO:   Reduce  overall   future  footprint   by  50%  or  more   5Y  TCO:   Reduce  overall   future  footprint   by  70%  or  more  
  21. 21. Best  Prac.ce:  Transform  Performance  &  Scale   NGTC  Conference  (c)  
  22. 22. When  customers  “struggle”  with  an  app…   In  the  .me  it  takes  to  blink  an  eye—400  milliseconds  to  be   exact,  online  AND  mobile  users  will  have  begun  to  abandon   your  applica.on   Your  compeFtor   You   43%   Abandon  the   transac.on,  try  later   on  computer   (Harris  Mobile  Transac.ons  Survey  for  TeaLeaf,    March  2011)   16%   Become  more     likely  to  prefer     a  compe.tor   14%   Log  a  complaint  with   customer  service   12%   Go  directly  to   a  compe.tor  
  23. 23. Performance  will  make  or  break  an  applica.on!.   When  problems  occur:   Development  Phase   •  Revenue  is  lost.   Requirements   •  Produc.vity  decreases.   •  Customers  leave.   •  The  longer  it  takes  to  find  the   defect,  the  more  it  Costs  to  fix.   Cost  of  Fixing  RaFo   1   Design   3  -­‐  6x   Coding   10x   Development   15  -­‐  40x   Acceptance   30  -­‐  70x   Opera.on/Produc.on   40  -­‐  1000     80%  of  the  TCO  of  an  applica.on  is  directly  related  to  finding  and  fixing  problems.   (Source:  “The  Economic  Impacts  of  Inadequate  Infrastructure  for  So<ware  Tes>ng,  NIST)   The  result:  on  average,  a  performance  issue  costs  an  organiza.on  at  least   $45,000/hour.    (Source:  Mul>ple  industry  analysts  have  published  es>mates  ranging  from  EMA’s  $45,000/hr  cost  to  Yankee  Group’s  and  Meta’s  es>mate  of  over  $90,000/hr  or  more  depending  on   the  industry  and  nature  of  the  failing  applica>on.)  
  24. 24. Service  Virtualiza.on:  Where  would  I  use  this?   Service   Virtualiza.on  
  25. 25. Eliminate  Capacity  Constraints   Major  North  American  Bank  Performance  &  Test  CoE         VSE         VSE   Shared  Mainframe   Service         VSE   Or  similar  heavy-­‐weight   implementa.on  environment   Before   A{er   Business  Problem   •  •  •  •  •  13  performance,  dev,  test  teams  forced  to  .meshare   on  mainframe-­‐based  services   Cost  of  addi.onal  environments  for  L&P  lab  and  the   movement  to  Agile  development/   Difficult  to  perform  load  due  to  system   dependencies   Only  2  of  5  major  releases  were    performance  tested     Offshore  teams  unable  to  complete  off  hours   performance  due  to  availability  issues   Business  Impact   •  Virtualize  in  8  days  what  took  2  years  of  custom-­‐coded  stub   development  to  create.     •  Virtualize  new  systems  in  1  hour  with  business  analysts   versus  weeks  of  custom  coding   •  •  •  Virtualized  environments  enabled  all  5  major  releases   plus  5  minor  releases  to  include    L&P   Over  $30  million  in  hardware  cost  avoidance   Offshore  teams  performing  300%  more  L&P  due  to   system  availability  
  26. 26. Customer   Success  Story   Avoid  costly  transacFon  fees  when  tesFng   third-­‐party  services     Leading  Global  Airline     Challenge   §  Test  and  valida.on  of  reserva.on  booking  system  was  incurring  $1  to  2   million  in  monthly  fees  to  access  live  third-­‐party  services   SoluFon  using  LISA  Virtualize   “We  rely  heavily  on   LISA  to  automate  the   tes?ng,  configura?on   and  virtualiza?on   aspects  of  our  test   and  development   environment.”   -­‐  Lead  Architect   §  LISA  simulated  the  behavior  of  third-­‐party  reserva.on  systems  as  virtual   services  that  were  available  24/7  without  usage  fees   §  No  longer  needed  to  coordinate  test  schedules  or  mock  transac.on  data   sets  with  third-­‐party  service  providers   §  Provided  freedom  for  development  and  QA  teams  to  test  earlier  in  the   so{ware  lifecycle   Client  Results   §  Reduced  service  usage  fees  to  save  over  $12M  annually   §  Lower  tes?ng  costs  by  using  stable,  virtualized  data  sets   §  Higher  quality  by  encouraging  earlier  and  more  frequent  
  27. 27. Example:  Performance  Middle-­‐Tiers   Using  LISA’s  Virtual  Service  Environments  (VSEs)   USERS:     Agents  /   Adjusters   19.0s   Booking   business   service   LISA   Test     XML/MQ   Conven.onal  load  can   only  expose  overall   performance  .me   UI  Load  Test   Tools   2.5s   3.3s   LISA   Test     Integra.on     Bus  (ESB)       SOAP/HTTP(S     LISA     VSE     PNR     Service   Frequent   Flyer  System   XML/MQ   ROI   Reserva.on   Service     GDS     LISA     VSE     SOAP/JMS   2.5s   LISA  manages   Test  data   LISA   Test   Applica.on   LISA     VSE     10.7s   Isolate,  test  and  tune  components   by  virtualizing  dependent  services     Slow  responding   transac.on  or  component   pinpointed  with  LISA  
  28. 28. Customer  Success   Story   Enabling  High  Performance  with   Constrained  Resources  &  Time     Leading  P&C  Insurance  Fund     Challenge   §  Enterprise  wide  “go-­‐live”  of  system  across  50  states   §  Performance  lags  could  not  be  pinpointed  with  UI   §  Middle-­‐.er  and  3rd  party  credit  systems  too  to  test.   “When  you  can  do   isola?on  at  a   component  level   against  the   performance  budget,   you  can  iden?fy  the   weak  link  in  the  chain   and  address  it..”   -­‐  Sr.  Director,  IT   Architecture  &   Performance   SoluFon  using  LISA  Virtualize  &  Test   §  Simulated  Web,  IBM  MQ/ESB,  Cloud/3rd  party  Credit  service,  and  Oracle   apps  with  Virtual  Services  for  24/7  inexpensive   §  Drive  50K  user+  load  tests  directly  against  isolated  components     §  “Prove  It”  quick  ROI  project  with  iTKO  engineers  and  SI  partner     Client  Results   §  Went  from  test  overruns  to  unlimited  Virtualized  performance  test  access,   and  next  phase  4  months  early   §  Delivered  ROI  within  one  month  with  skilled  resources   §  Improved  test  bed  availability  by  64%  and    increased  number  of   tests  run  by  10x  
  29. 29. Best  Prac.ce:  Decompose  Service  Levels   Increasing  efficiency  by  building  up  to  meet  a  “Performance  Budget”   Without  component-­‐level  performance  tesFng:   Try adding more test Hardware!! " SLA
 2.1 seconds" Order" Lookup" Test: " 4.0" Seconds" Solution" Price" Timeline $! 8 mo. " Test" 3.2
 Seconds" $! Still not within SLA!! 9 mo. PROJECT DUE! Isolated  component-­‐level  performance  tesFng  miFgates  risk:   Order" 0.5s" 0.5s" Lookup" Price" 0.4s" 1.9s" 0.4s" 0.5s" 1.2s" Use VS to isolate & tune components separately from dependencies! 1.2s" Solution" " Test:" 2.1
 Seconds" Aggregate solution tuned for performance budgets!
  30. 30. Produc.on  vs.  Test:   in  produc.on   4G  SingTel  Singapore   online  banking   shared  infrastructure   datacenter  –  Kuala  Lumpur     web  service    transac.ons   3G  Celcom  Kaula  Lumpur   2.5G  Telkomsel  Jakarta   Mobile  TransacFons   mobile  banking   shared  infrastructure   datacenter  –  Singapore     web  service     Singapore    Kaula  Lumpur   Jakarta   online  banking   shared  infrastructure   datacenter  –  Singapore   Internet  TransacFons  
  31. 31. ProducFon  vs.  Test:   in  test   without  network  virtualizaFon   Simulated  web  services   Load     Generator   with  network  virtualizaFon   Simulated  TransacFons   online  banking   shared  infrastructure   datacenter  –  Singapore   RealisFc  Mobile/Web/ Internet  Network   CondiFons   4G  SingTel  Singapore   Singapore   3G  Celcom  Kaula  Lumpur    Kaula  Lumpur   Load     Generator   2.5G  Telkomsel  Jakarta   mobile  banking   shared  infrastructure   datacenter  –  Singapore   TransacFons  with  real       world  network  condiFons   Load     Generator   Jakarta   virtual  locaFons   online  banking   shared  infrastructure   datacenter  –  Singapore  
  32. 32. Xyz  Bank  iPad  Edi.on  Mobile  App  Summary  Analysis       Time KBs  Data 80     72.2   70   60   50   46   32.8   40   30   20   10   0   44.6   21.3   20.7   14.6   7.1   17.2   33.2   27   22.4   20.5   12.3   11.7   31.9   19.1   8.2   WiFi   3G   3G  Busy  
  33. 33. Performance  Op.miza.on  -­‐  once  bo}lenecks   iden.fied  than  solu.on  is  to  op.mize  the   applica.on.  
  34. 34. References   §  Service  Virtualiza.on  Site  Home  Page  :     h}p://     §  The  Book  -­‐  Service  Virtualiza.on:  Reality  is  Overrated.   h}p://­‐Virtualiza.on-­‐Reality-­‐Is-­‐Overrated-­‐ebook/dp/ B009HAKER8/ref=dp_kinw_strp_1     §  Ge€ng  Started  with  Service  Virtualiza.on   h}p://€ng-­‐started-­‐with-­‐service-­‐virtualiza.on-­‐sv-­‐ minute-­‐2012     §  LISA  Introduc.on  –  What  Does  LISA  Do  ?   h}p://       Source  :  CA  Press  Release  |  
  35. 35. Thank You! MIN  FANG   MIN.FANG@CA.COM   +65  9636  8356   NGTC  Conference  (c)