MediaGeniX	  Con,nuous	  integra,on	  §  Context	  §  Current	  system	  §  Tools	                                     ...
Our	  Product	  Our	  Customers	  Technical	  Product	  Informa,on	  MEDIAGENIX	  
PRODUCT	  COMPANY	     20	  YEARS	  OF	      WHATS’ON	  
Halloween	  Next	  Year	  
Halloween	  next	  year	               	               	               	  
CUSTOMERS	  
VRT          RTBF  MTV                               VT4              ProductPro7         (15.000	  classes)         VTM  ...
90	  Employees	  §  30	  SoZware	  Developers	  §  6	  Project	  Managers	  §  13	  Func,onal	  Analysts	  §  17	  Cus...
Applica,on	  Overview	  §  1.9	  Million	  lines	  of	  code	  §  15.000	  own	  classes	  §  200.000	  methods	  §  1...
§  35	  customers	  wan,ng	  different	  things	  §  They	  want	  it	  tomorrow	  §  30	  developers	  ‘spawning’	  cod...
Context	  Current	  tes,ng	  system	  Tools	  CONTINUOUS	  INTEGRATION	  
Con$nuous	  integra$on	  	  § 	  Many	  isolated	  changes	  § 	  Many	  integra,ons	  § 	  Each	  is	  immediately	  t...
Con$nuous	  integra$on	  	  	  Development	  teams	  integrate	  frequently	  	         § 	  Avg:	  	  70/day	  Tes,ng	  ...
What	  do	  we	  test?	  	  As	  much	  as	  possible	         § 	  Unit	  tests	         § 	  Consistency	  checks	    ...
Code	  consistency	  checks	                    	  	  Check	  if	  code	  guidelines	  have	  been	  followed	  	  Why	  ?...
Code	  consistency	  checks	                       	  	  	                                            	  	  	  	  	  build...
Code	  consistency	  checks	                       	  	  	  	     	  	  	  TopApplica?on	              	  	  	  Applica,on...
UI	  tests	  and	  Acceptance	  tests	  	  	  Why	  ?	  § 	  Unit	  test	          § 	  Test	  a	  method	          § 	...
UI	  Tes,ng	  –	  A	  Person	  Editor	  
test_editPerson	           	  |	  person|	           	  person	  :=	  Person	  withFirstName:	  Angelina	  	           	  ...
test_editPerson	           	  |	  person|	           	  person	  :=	  Person	  withFirstName:	  Angelina	  	           	  ...
test_editPerson	           	  |	  person|	           	  person	  :=	  Person	  withFirstName:	  Angelina	  	           	  ...
UI	  Tes,ng	  –	  A	  Person	  Editor	  
test_editPerson	           	  |	  person|	           	  person	  :=	  Person	  withFirstName:	  Angelina	  	           	  ...
UI	  Tes,ng	  –	  A	  Person	  Editor	  
test_editPerson	           	  |	  person|	           	  person	  :=	  Person	  withFirstName:	  Angelina	  	           	  ...
Refactoring	  tests	  	  	  Why	  ?	         	      	             	                         	  	  	  	  	  	  	  	  	  Bas...
Refactoring	  tests	             	  	  test_methodX_deprecated	  	  	  	  	  	  	  	  "use	  methodY"	  	  	  	  	  	  	  ...
Goals	  § 	  Get	  test	  results	  as	  fast	  as	  possible	  § 	  Alert	  ASAP	  when	  build	  is	  broken	  § 	  E...
Challenges	  § 	  Tests	  take	  over	  12	  hours	  to	  run	  § 	  No	  out-­‐of-­‐the-­‐box	  solu,on	  ⇒ We	  Built	...
Mul$ple	  tests	  on	  same	  DB	  § 	  Problem:	  You	  assume	  a	  ‘clean	  DB’	  when	  you	  start	  running	  a	  t...
                                                4 Test Coordinators   4 x16 Test SlavesCurrent	  Tes,ng	  System	         ...
Store           .                                                                                           Test Master	  ...
Test Master                                                                                                               ...
 Test	  Coordinator	                                                                                                      ...
 Test	  Slave	                                                                                                            ...
                                                                                                                          ...
Test Master                                                                     To	  Load	                                ...
                                                                                                                   Op,miza...
 Op,miza,ons	  § 	  Assign	  test	  classes	  in	  a	  smart	  order	  § 	  ‘Nuke’	  tests	                             ...
 Test	  Results	  § 	  What?	  Timing?	          § 	  Build	  failed	  	  	  <	  45minutes	          § 	  AZer	  tests	...
Con,nuous	  integra,on	  –	  VERA/      Seaside	  interface	  
Con,nuous	  integra,on	  –	  VERA/      Seaside	  interface	  
VERA/Seaside	  interface	  
VERA/Seaside	  interface	  
Integrated	  Tools	  § Open	  SUnit	  on	  failed	  tests	  § Find	  origin	  failing	  test	  
    	  Demo	  
Goals	  § Get	  test	  results	  as	  fast	  as	  possible	  § Alert	  ASAP	  when	  build	  is	  broken	  § Emulate	  ...
Conclusion	  Goals	  have	  been	  met!	      §  Build	  with	  report	  on	  broken	  builds	          within	  45	  min...
Conclusion	  Future	  areas	  of	  improvement	  …	     §  Test	  every	  version	         §  Be@er	  pin-­‐point	  wher...
 	         Ques,ons?	  
Q:	  Which	  customisa,ons	  do	  customers	  want?	  A:	  See	  next	  presenta,on	  	  Q:	   Why	   does	   the	   produ...
Continuous Integration: a practical approach
Continuous Integration: a practical approach
Continuous Integration: a practical approach
Continuous Integration: a practical approach
Continuous Integration: a practical approach
Continuous Integration: a practical approach
Continuous Integration: a practical approach
Continuous Integration: a practical approach
Continuous Integration: a practical approach
Continuous Integration: a practical approach
Continuous Integration: a practical approach
Upcoming SlideShare
Loading in …5
×

Continuous Integration: a practical approach

645 views

Published on

ESUG 2012, Ghent

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
645
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Continuous Integration: a practical approach

  1. 1. MediaGeniX  Con,nuous  integra,on  §  Context  §  Current  system  §  Tools     CONTINUOUS  CONTENTS   INTEGRATION     Elke  Ma@hijs  and  Maïkel  Vandorpe   elke.ma@hijs@mediagenix.tv   maikel.vandorpe@mediagenix.tv  
  2. 2. Our  Product  Our  Customers  Technical  Product  Informa,on  MEDIAGENIX  
  3. 3. PRODUCT  COMPANY   20  YEARS  OF   WHATS’ON  
  4. 4. Halloween  Next  Year  
  5. 5. Halloween  next  year        
  6. 6. CUSTOMERS  
  7. 7. VRT RTBF MTV VT4 ProductPro7 (15.000  classes) VTM NRK YLE TV2 MANY  CUSTOMIZATIONS  
  8. 8. 90  Employees  §  30  SoZware  Developers  §  6  Project  Managers  §  13  Func,onal  Analysts  §  17  Customer  Support  Analysts  §  Other  …  
  9. 9. Applica,on  Overview  §  1.9  Million  lines  of  code  §  15.000  own  classes  §  200.000  methods  §  1000  database  tables  
  10. 10. §  35  customers  wan,ng  different  things  §  They  want  it  tomorrow  §  30  developers  ‘spawning’  code  §  Short  ,me  to  market  =>  Con,nuous  Integra,on  
  11. 11. Context  Current  tes,ng  system  Tools  CONTINUOUS  INTEGRATION  
  12. 12. Con$nuous  integra$on    §   Many  isolated  changes  §   Many  integra,ons  §   Each  is  immediately  tested  §   Detect  and  fix  problems  soon  
  13. 13. Con$nuous  integra$on      Development  teams  integrate  frequently     §   Avg:    70/day  Tes,ng  system   §   Each  integra,on  is  automa,cally  built  and  tested  Efficiently  detect  and  fix  problems  Significant  reduc,on  of  integra,on  problems  Develop  cohesive  soZware  even  in  a  limited  ,me  frame    
  14. 14. What  do  we  test?    As  much  as  possible   §   Unit  tests   §   Consistency  checks   §   UI  tests  and  Acceptance  tests  in  general   §   Refactoring  tests    Sequen,al  running  of  tests  take  over  12  hours  
  15. 15. Code  consistency  checks      Check  if  code  guidelines  have  been  followed    Why  ?   §   Consistent  code  makes  life  easy   §   Reduce  number  of  bugs          
  16. 16. Code  consistency  checks                  buildMenuBar          TopApplica?on          Applica,on  A   |  menu  |   (menu  :=  Menu  new)        Applica,on  B    addItem:  self  fileMenuItem;    addItem:  self  editMenuItem;        Applica,on  C    addItem:  self  toolsMenuItem.     self  addMenuBarItemsTo:  menu.     menu    addItem:  self  viewMenuItem;    addItem:  self  windowsMenuItem.   ^menu  
  17. 17. Code  consistency  checks                TopApplica?on        Applica,on  A        Applica,on  B        Applica,on  C   test_topApplica,on_buildMenuBar   "Dont  implement  #buildMenuBar.   use  #addMenuBarItemsTo:"       self  assert:  (self  implementorsOf:  #buildMenuBar                          inSubclassesOf:  TopApplica,on)   isEmpty.  
  18. 18. UI  tests  and  Acceptance  tests      Why  ?  §   Unit  test   §   Test  a  method   §   Change  the  method,  change  the  test  §   Acceptance  test   §   Emulate  a  user  process   §   More  stable  tests   =>  Never  change  func,onality,  unless  user  asks  for  it    
  19. 19. UI  Tes,ng  –  A  Person  Editor  
  20. 20. test_editPerson    |  person|    person  :=  Person  withFirstName:  Angelina        andLastName:  Jolie.    self  deny:  (Person  existsWhere:        [:p|  (p  firstName  =  Bruce)        &  (p  lastName  =  Willis)]).    self  openPersonEditorOn:  person.    self  editField:  #lastName  value:  Willis.    self  editField:  #firstName  value:  Bruce.      self  toolBarPress:  #saveCommand.    self  assert:  (Person  existsWhere:        [:p  |  (p  firstName  =  Bruce)      &  (p  lastName  =  Willis)])  
  21. 21. test_editPerson    |  person|    person  :=  Person  withFirstName:  Angelina        andLastName:  Jolie.    self  deny:  (Person  existsWhere:        [:p|  (p  firstName  =  Bruce)        &  (p  lastName  =  Willis)]).    self  openPersonEditorOn:  person.    self  editField:  #lastName  value:  Willis.    self  editField:  #firstName  value:  Bruce.        self  toolBarPress:  #saveCommand.    self  assert:  (Person  existsWhere:        [:p  |  (p  firstName  =  Bruce)      &  (p  lastName  =  Willis)])  
  22. 22. test_editPerson    |  person|    person  :=  Person  withFirstName:  Angelina        andLastName:  Jolie.    self  deny:  (Person  existsWhere:        [:p|  (p  firstName  =  Bruce)        &  (p  lastName  =  Willis)]).    self  openPersonEditorOn:  person.    self  editField:  #lastName  value:  Willis.    self  editField:  #firstName  value:  Bruce.        self  toolBarPress:  #saveCommand.    self  assert:  (Person  existsWhere:        [:p  |  (p  firstName  =  Bruce)      &  (p  lastName  =  Willis)])  
  23. 23. UI  Tes,ng  –  A  Person  Editor  
  24. 24. test_editPerson    |  person|    person  :=  Person  withFirstName:  Angelina        andLastName:  Jolie.    self  deny:  (Person  existsWhere:        [:p|  (p  firstName  =  Bruce)        &  (p  lastName  =  Willis)]).    self  openPersonEditorOn:  person.    self  editField:  #lastName  value:  Willis.    self  editField:  #firstName  value:  Bruce.        self  toolBarPress:  #saveCommand.    self  assert:  (Person  existsWhere:        [:p  |  (p  firstName  =  Bruce)      &  (p  lastName  =  Willis)])  
  25. 25. UI  Tes,ng  –  A  Person  Editor  
  26. 26. test_editPerson    |  person|    person  :=  Person  withFirstName:  Angelina        andLastName:  Jolie.    self  deny:  (Person  existsWhere:        [:p|  (p  firstName  =  Bruce)        &  (p  lastName  =  Willis)]).    self  openPersonEditorOn:  person.    self  editField:  #lastName  value:  Willis.    self  editField:  #firstName  value:  Bruce.        self  toolBarPress:  #saveCommand.    self  assert:  (Person  existsWhere:        [:p  |  (p  firstName  =  Bruce)      &  (p  lastName  =  Willis)])  
  27. 27. Refactoring  tests      Why  ?                          Basic  Whats’On     Method  X   Y Call    X   Y   Press  Bu@on   C Ac?on  A                            Whats’On  for  a  customer     Method  X     Ac?on  B      =>  Important  during  upgrades  
  28. 28. Refactoring  tests      test_methodX_deprecated                "use  methodY"                  self  assertNoImplementorsOf:  ‘methodX‘.                  self  assertNoReferencesTo:  ‘methodX’.      
  29. 29. Goals  §   Get  test  results  as  fast  as  possible  §   Alert  ASAP  when  build  is  broken  §   Emulate  actual  users  §   SLA:  Priority  1  bug      =>  deliver  bugfix  within  1  day  
  30. 30. Challenges  §   Tests  take  over  12  hours  to  run  §   No  out-­‐of-­‐the-­‐box  solu,on  ⇒ We  Built  a  distributed  system  ourselves  §   Mul,ple  tests  on  the  same  DB  
  31. 31. Mul$ple  tests  on  same  DB  §   Problem:  You  assume  a  ‘clean  DB’  when  you  start  running  a  test  §   Solu$on:  Take  snapshot  before  test,  restore  snapshot  aZer  test  
  32. 32.   4 Test Coordinators 4 x16 Test SlavesCurrent  Tes,ng  System     Image Creator Test Master
  33. 33. Store . Test Master  Tes,ng  a  version    §     Step  by  step:   §   The  version  has  to  be  loaded   §   Create  a  database   §   The  version  has  to  be  tested   §   The  developer  gets  the  results  §   The  test  master  orchestrates  this  whole  process    
  34. 34. Test Master Image Creator To  Load   To  Test     Your   …   version   …  Image  Creator   Your   …   version   …   Your  version   Your   …   version   …   Your  version   §   TM  instructs  IC  to  load  your  version   Store .§   Get  customers  code  from  Store  §   AZer  loading   §   Puts  the  new  image  on  a  fileserver   §   IC  no,fies  TM  he’s  done  
  35. 35.  Test  Coordinator   §   TM  instructs  a  TC  to  start  tes,ng  a  version   Test Master§   A  TM  is  responsible  for  1  whole  version   To  Load   To  Test   …   Your  version   …   …   4 Test Coordinators
  36. 36.  Test  Slave   §   TC  orders  TSs  to  copy  image   Test Master      and  create  a  local  database  §   When  database  ready,  start  tes,ng!   §   Test  Class  per  Test  Class   §   Report  back  to  TC  when  tes,ng  of  1  Class  is  done          and  receive  a  new  class  un,l  finished   1 of the 4 Test Coordinators 16 Test Slaves
  37. 37.   Op,miza,ons   Test Master §   Parallel  vs  sequen,al     Image Creator §   IC  can  load  4  versions  simultaneously   §   4  TCs        4  Versions  are  tested  simultaneously   §   16  TSs        Tes,ng  1  version  is  much  faster   1 of the 4 Test Coordinators 16 Test Slaves
  38. 38. Test Master To  Load   To  Test   Version  z     Version  (x+1)     y x              for  customer  C            for  customer  A     B   …   Version  (x+1)     y    Op,miza,ons            for  customer  B   A Version  (x+1)     …            for  customer  A   …  §   Dialog  ‘Would  you  like  to  schedule  the  tests’  §   Only  test  latest  available  version  §   TM  queues  can  be  manipulated   §   Priori,ze   §   Unschedule    
  39. 39.   Op,miza,ons   §   Time-­‐outs   Image Creator §   IC  aZer  30minutes   §   TC  aZer  75minutes   §   TS  aZer  +/-­‐  10minutes  
  40. 40.  Op,miza,ons  §   Assign  test  classes  in  a  smart  order  §   ‘Nuke’  tests   16 Test Slaves
  41. 41.  Test  Results  §   What?  Timing?   §   Build  failed      <  45minutes   §   AZer  tests  have  successfully  run   §   X  total  tests,  Y  failures,  Z  errors,  T  not  run   §   With  wai,ng  in  a  queue  +/-­‐  4hours   §   Without  wai,ng  in  a  queue  <  1hour  §   Saved  on  Store!  
  42. 42. Con,nuous  integra,on  –  VERA/ Seaside  interface  
  43. 43. Con,nuous  integra,on  –  VERA/ Seaside  interface  
  44. 44. VERA/Seaside  interface  
  45. 45. VERA/Seaside  interface  
  46. 46. Integrated  Tools  § Open  SUnit  on  failed  tests  § Find  origin  failing  test  
  47. 47.    Demo  
  48. 48. Goals  § Get  test  results  as  fast  as  possible  § Alert  ASAP  when  build  is  broken  § Emulate  actual  users  § SLA:  Priority  1  bug      =>  deliver  bugfix  within  1  day  
  49. 49. Conclusion  Goals  have  been  met!   §  Build  with  report  on  broken  builds   within  45  minutes   §  Test  results  can  come  in  within  1  hour   §  Prio  1  bugs  can  be  fixed  within  a  day   §  ‘Reasonable’  ,me  for  results   §  4  hours  on  average   §  Bonus:  integrated  tools  on  SUnit  
  50. 50. Conclusion  Future  areas  of  improvement  …   §  Test  every  version   §  Be@er  pin-­‐point  where  an  error  started   §  Speed  up  through-­‐put  of  unpriori,sed   tests  
  51. 51.     Ques,ons?  
  52. 52. Q:  Which  customisa,ons  do  customers  want?  A:  See  next  presenta,on    Q:   Why   does   the   product   need   to   change   that  oZen?  A:   Technical   evolu,on,   target   different  markets,  new  regula,ons,  …  

×