Automation Test Framework  Continue Improvement               Welcome Presentation          Presented by Christina Geng
AGENDA❶ Case Introduction❷ Testing Strategy Evolution❸ Case ROI Analysis❹ Success factor❺ Revelation
CASE INTRODUCTIONTitle:   7 Times Evolution of Test Automation Framework under JSProduct for Efficiency ImprovementTarget:...
WHO AND WHAT?Data        Analysis                                  Visualization        •   reporting, online analytical p...
WHO AND WHAT?Data                               Analysis                                             Visualization        ...
AGENDA❶ Case Introduction❷ Testing Strategy Evolution❸ Case ROI Analysis❹ Success factor❺ Revelation
What’s more… 2012.12                          Developer environment2012.09 2012.10                          Full Automatio...
VERSION-1TC: test pageScheduler: ASTECTest tool: SilkTestTC Management: Excel                       Test                  ...
VERSION-11.     +     First automation framework setup                                         -                      writ...
What’s more… 2012.12                          Developer environment2012.09 2012.10                          Full Automatio...
VERSION-2Scheduler: ASTECTC Management: Excel/XML (test manager)TC: test pageTest tool: Selenium Test DriverOther tools• A...
VERSION-2+1.   Reduce test pages                                                    -                         test driver2...
What’s more…  2012.12                          Developer environment2012.09 2012.10                          Full Automati...
VERSION-3                                            Unified Test DriverAbstract Test Data                     And How?   ...
VERSION-3            backgroud                   color                  visible                legend                 posi...
VERSION-3            backgroud                   color                  visible                legend                 posi...
VERSION-3            backgroud                   color                  visible                legend                 posi...
VERSION-3            backgroud                   color                  visible                legend                 posi...
VERSION-3            testUtil.generateChart(viz/line; TestData.a1a1m1m1; [legend.visible,false])                          ...
VERSION-3            Eliminate Test Page
VERSION-3        selenium1        selenium2                    {StartBrowser}                    IE, Chrome, Firefox (sele...
VERSION-3        selenium1                    {StartBrowser}                    Safari (selenium1)        selenium2   Sele...
VERSION-3        selenium1        selenium2   {TakeSnapshot}                    Point p = this.getElementOffset(targetElem...
VERSION-3        selenium1        selenium2   {FileCompare}                    cmd.exe ImageComparer.exe –file filepath1 f...
What’s more…  2012.12                          Developer environment2012.09 2012.10                          Headless     ...
VERSION-4            Table                 Text  Tooltip            Selection             Animation            Button     ...
VERSION-4            Selection
VERSION-4            Gesture
VERSION-4var eventObj = {             type : originalEvent.type,             timeStamp : originalEvent.timeStamp,         ...
What’s more…   2012.12                           Developer environment2012.09 2012.10                           Headless  ...
VERSION-5            How to arrange test case?
VERSION-5                              TC (.xls/.xml)                     import     Test     Link            export      ...
VERSION-5 Continuous Integration
VERSION-5                                    .xml               TC mngmt                                     .xls         ...
VERSION-5                                    .xml               TC mngmt                                     .xls         ...
What’s more…   2012.12                           Developer environment2012.09 2012.10                           Headless  ...
Engineering
VERSION-6                              Paint               Layout                 DisplayPhantomJS (Headless Webkit)      ...
VERSION-6                    config                                           Browser?                              Casper...
And that’s   OK
What’s more…2012.12                           Developer environment2012.09 2012.10                           Headless     ...
VERSION-7                                  .java    TC Management                                            Test Case Poo...
VERSION-7                                  .java    TC Management                                            Test Case Poo...
AGENDA❶ Case Introduction❷ Testing Strategy Evolution❸ Case ROI Analysis❹ Success factor❺ Revelation
CASE ROI ANALYSIS Functional Test: BAT (nightly), FA                            1)   Test machine                         ...
CASE ROI ANALYSIS Functional Test: BAT (nightly), FA                            1)   Test machine                         ...
AGENDA❶ Case Introduction❷ Testing Strategy Evolution❸ Case ROI Analysis❹ Success factor❺ Revelation
SUCCESS FACTOR     ❶ ❷❸❹           Focus
SUCCESS FACTOR      ❶ ❷❸❹        Team work
SUCCESS FACTOR   ❶ ❷❸❹Continuous Improvement
SUCCESS FACTOR       ❶ ❷❸❹        Learn 360°
AGENDA❶ Case Introduction❷ Testing Strategy Evolution❸ Case ROI Analysis❹ Success factor❺ Revelation
OUR REVELATION不被传统的测试方法束缚;从产品用到的技术找到突破;持续改进的思想;不断尝试的理念
CONTRIBUTOR      Xiaolu, Ye   Christina, Geng    Helen, Jiang     Chen, Zhou    Peipei, Jiang      Coco, Dai    Grace, Zha...
Upcoming SlideShare
Loading in …5
×

最终版Top100summit 耿晓倩-automation test framework continue improvement

507 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
507
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

最终版Top100summit 耿晓倩-automation test framework continue improvement

  1. 1. Automation Test Framework Continue Improvement Welcome Presentation Presented by Christina Geng
  2. 2. AGENDA❶ Case Introduction❷ Testing Strategy Evolution❸ Case ROI Analysis❹ Success factor❺ Revelation
  3. 3. CASE INTRODUCTIONTitle: 7 Times Evolution of Test Automation Framework under JSProduct for Efficiency ImprovementTarget: ❶ Build a robust test automation framework ❷ Manage test cases efficiently ❸ Reduce test case running time and maintenance cost ❹ Shared framework under scrum development
  4. 4. WHO AND WHAT?Data Analysis Visualization • reporting, online analytical processing • analytics • data mining • process mining • event processing • business performance management • benchmarking • text mining • predictive analytics
  5. 5. WHO AND WHAT?Data Analysis Visualization • reporting, online analytical processing Programming language: Javascript/Java • analytics • data mining Provide service: • process mining provides various type of visualization objects that can be used in The Visualization Service is a software module which • event processing applications on different platforms (Web, Mobile and Desktop). different • business performance management • A basic • visual component like chart (Bar, Column Pie, Line, etc ), Geo Map, Visualizationor Grid. benchmarking Gauge, Table types: • text mining• An advanced visual component like (Tree map, Candlestick, Tagcloud) • predictive analytics • Above visual component with interactions and animations• Combination of multiple visual components, it could be composed by same type of visual components or different type of component.
  6. 6. AGENDA❶ Case Introduction❷ Testing Strategy Evolution❸ Case ROI Analysis❹ Success factor❺ Revelation
  7. 7. What’s more… 2012.12 Developer environment2012.09 2012.10 Full Automation Test Link2011.12 2012.03 2012.06 Jenkins Data Driven Selenium 2011.6 We have it!
  8. 8. VERSION-1TC: test pageScheduler: ASTECTest tool: SilkTestTC Management: Excel Test page SilkTest
  9. 9. VERSION-11. + First automation framework setup - writing test case2. Overview functional test case coverage3. Reduce testing overlap for shared component HT Test Test ML page page resource SilkTest maintenance the relation between TC and record
  10. 10. What’s more… 2012.12 Developer environment2012.09 2012.10 Full Automation Test Link2011.12 2012.03 2012.06 Jenkins Data Driven Selenium 2011.6 We have it!
  11. 11. VERSION-2Scheduler: ASTECTC Management: Excel/XML (test manager)TC: test pageTest tool: Selenium Test DriverOther tools• Auto generate test plan from Excel to xml DIV DIV• Test Manager Test DIV page DIV Selenium
  12. 12. VERSION-2+1. Reduce test pages - test driver2. Reduce resource running cost3. Test steps turns to be keyword driven in xml case unit4. Write test case in .xls, auto generate .xml DIV DIV DIV DIV Test Test DIV DIV page page DIV DIV Selenium test data redundancy test case sync
  13. 13. What’s more… 2012.12 Developer environment2012.09 2012.10 Full Automation Test Link2011.12 2012.03 2012.06 Jenkins Data Driven Selenium 2011.6 We have it!
  14. 14. VERSION-3 Unified Test DriverAbstract Test Data And How? Wrapper API Eliminate Test Page
  15. 15. VERSION-3 backgroud color visible legend position verticalOffset horizontalOffset visible labelFont labelColor formatString initChartsByProperties([[layout.padding,[50,50,50,50]],[legend.visible,false]]); layout padding tooltip enable customization text mainValue subValue title main sub Selectability mode deSelectale defaultSelectedItems xAxisMember yAxisMember bar barMarkerSize measureMember Wrapper API
  16. 16. VERSION-3 backgroud color visible legend position verticalOffset horizontalOffset visible labelFont labelColor formatString initChartsByProperties([[layout.padding,[50,50,50,50]],[legend.visible,false]]); layout padding tooltip enable updateChartsByProperties([[layout.padding,[50,50,50,50]],[legend.visible,false]]); customization text mainValue subValue title main sub Selectability mode deSelectale defaultSelectedItems xAxisMember yAxisMember bar barMarkerSize measureMember Wrapper API
  17. 17. VERSION-3 backgroud color visible legend position verticalOffset horizontalOffset testUtil.generateChart(viz/line; TestData.a1a1m1m1) visible labelFont labelColor formatString initChartsByProperties([[layout.padding,[50,50,50,50]],[legend.visible,false]]); layout padding tooltip enable updateChartsByProperties([[layout.padding,[50,50,50,50]],[legend.visible,false]]); customization text mainValue subValue title main sub Selectability mode deSelectale defaultSelectedItems xAxisMember yAxisMember bar barMarkerSize measureMember Wrapper API
  18. 18. VERSION-3 backgroud color visible legend position verticalOffset horizontalOffset visible labelFont labelColor formatString testUtil.generateChart(viz/line; TestData.a1a1m1m1; [legend.visible,false]) initChartsByProperties([[layout.padding,[50,50,50,50]],[legend.visible,false]]); layout padding tooltip enable customization text mainValue subValue title main sub Selectability mode deSelectale defaultSelectedItems xAxisMember yAxisMember bar barMarkerSize measureMember Wrapper API
  19. 19. VERSION-3 testUtil.generateChart(viz/line; TestData.a1a1m1m1; [legend.visible,false]) public mySQL private testUtil.generateChart(viz/line; TestData.a1a1m1m1; legendVisibleTrue) Abstract Test Data
  20. 20. VERSION-3 Eliminate Test Page
  21. 21. VERSION-3 selenium1 selenium2 {StartBrowser} IE, Chrome, Firefox (selenium2) RemoteWebDriver webDriver = null; webDriver = new FirefoxDriver(); ImageComparer Robot() Unified Test Driver
  22. 22. VERSION-3 selenium1 {StartBrowser} Safari (selenium1) selenium2 SeleniumServer seleniumServer = new SeleniumServer(); seleniumServer.boot(); selenium = new DefaultSelenium("localhost", 4444, "*safariproxy "+ safariLocation,urlRoot); ImageComparer CommandExecutor executor = new SeleneseCommandExecutor(selenium); DesiredCapabilities dc = new DesiredCapabilities(); Robot() webDriver = new RemoteWebDriver(executor, dc); Unified Test Driver
  23. 23. VERSION-3 selenium1 selenium2 {TakeSnapshot} Point p = this.getElementOffset(targetElement); Dimension d = this.getElementSize(targetElement); BufferedImage bi = new Robot().createScreenCapture(p.x, p.y, d.width, d.height); ImageComparer Robot() Unified Test Driver
  24. 24. VERSION-3 selenium1 selenium2 {FileCompare} cmd.exe ImageComparer.exe –file filepath1 filepath2 – threshhold 100 ImageComparer Robot() Unified Test Driver
  25. 25. What’s more… 2012.12 Developer environment2012.09 2012.10 Headless Test Link2011.12 2012.03 2012.06 Full Automation Data Driven Selenium 2011.6 We have it!
  26. 26. VERSION-4 Table Text Tooltip Selection Animation Button Scroll CSS Checkbox Hover Theme Link Gesture Gesture Upload Dropdown Download
  27. 27. VERSION-4 Selection
  28. 28. VERSION-4 Gesture
  29. 29. VERSION-4var eventObj = { type : originalEvent.type, timeStamp : originalEvent.timeStamp, pageX : originalEvent.pageX, pageY : originalEvent.pageY };eventRecord.push(eventObj);var handler = function(event){ var clickEvt = document.createEvent(MouseEvents); clickEvt.initMouseEvent(event.type, event.pageX, event.pageY…); d3.select(.main .plot)[0][0].dispatchEvent(clickEvt) };testAPI_lasso_heatmap = [ {"type":"mouseover","timeStamp":1348739963751,"pageX":126,"pageY":94}, {"type":"mouseover","timeStamp":1348739963751,"pageX":485,"pageY":94}, {"type":"mousemove","timeStamp":1348739963766,"pageX":485,"pageY":208}, {"type":"mousemove","timeStamp":1348739963773,"pageX":126,"pageY":208}]
  30. 30. What’s more… 2012.12 Developer environment2012.09 2012.10 Headless Test Link, Jenkins 2011.12 2012.03 2012.06 Full Automation Data Driven Selenium 2011.6 We have it!
  31. 31. VERSION-5 How to arrange test case?
  32. 32. VERSION-5 TC (.xls/.xml) import Test Link export Executable TP save Result log
  33. 33. VERSION-5 Continuous Integration
  34. 34. VERSION-5 .xml TC mngmt .xls Test Case Pool Manual Auto 3. Build info 5. result write back 4. get test plan 2.TP/Platform/build info Test Driver CI Server Test Executor 5. result log Selenium2 Java/C++ 1. Trigger 4. get assets libs resources Build Machine Web Server Code repository Where we are.
  35. 35. VERSION-5 .xml TC mngmt .xls Test Link Test Case Pool Manual Auto 3. Build info 5. result write back 4. get test plan 2.TP/Platform/build info Test Driver CI Server 5. result log Java Tool Test Executor Selenium2 Java/C++ Jenkins 1. Trigger 4. get assets libs resources Build Machine Web Server Code repository Perforce Where we are.
  36. 36. What’s more… 2012.12 Developer environment2012.09 2012.10 Headless Test Link, Jenkins 2011.12 2012.03 2012.06 Full Automation Data Driven Selenium 2011.6 We have it!
  37. 37. Engineering
  38. 38. VERSION-6 Paint Layout DisplayPhantomJS (Headless Webkit) http://phantomjs.org
  39. 39. VERSION-6 config Browser? CasperJS IE/Safari/FF/ChromeTP Read JS list Open browser PhantomJS: RunJS (test step) 1. open webkit 2. runJS …. 3. Take snapshot 4. close Take Snapshot Result validation not complete not complete Save result log Reporting Test driver inner logic
  40. 40. And that’s OK
  41. 41. What’s more…2012.12 Developer environment2012.09 2012.10 Headless Test Link, Jenkins 2011.12 2012.03 2012.06 Full Automation Data Driven Selenium 2011.6 We have it!
  42. 42. VERSION-7 .java TC Management Test Case Pool local CI Server Test Executor Test Driver Selenium2 Java/C++ per check-in triggerresult log sync up Casper/PhantomJS Build Machine get resources P4 server Test Executor
  43. 43. VERSION-7 .java TC Management Test Case Pool CI Server Eclipse Plug-in local Test Executor Test Driver Selenium2 Java/C++ per check-in triggerresult log Jenkins Build Machine sync up Casper/PhantomJS get resources P4 server Test Executor Perforce
  44. 44. AGENDA❶ Case Introduction❷ Testing Strategy Evolution❸ Case ROI Analysis❹ Success factor❺ Revelation
  45. 45. CASE ROI ANALYSIS Functional Test: BAT (nightly), FA 1) Test machine 2) Assets (util, BM…) 1. Set up environment 3) TP 4) Libs 2. Test case execution 1) Browser compatibility 4.5h 2) Run in headless 1) Email sending 3. Test result reporting 2) Sonar integrated with CI 1) BM update 4. Maintenance 2) TC management 1000+ cases
  46. 46. CASE ROI ANALYSIS Functional Test: BAT (nightly), FA 1) Test machine 2) Assets (util, BM…) 1. Set up environment 3) TP 4) Libs 2. Test case execution 1) Browser compatibility 47min 2) Run in headless 1) Email sending 3. Test result reporting 2) Sonar integrated with CI 1) BM update 4. Maintenance 2) TC management 1000+ cases
  47. 47. AGENDA❶ Case Introduction❷ Testing Strategy Evolution❸ Case ROI Analysis❹ Success factor❺ Revelation
  48. 48. SUCCESS FACTOR ❶ ❷❸❹ Focus
  49. 49. SUCCESS FACTOR ❶ ❷❸❹ Team work
  50. 50. SUCCESS FACTOR ❶ ❷❸❹Continuous Improvement
  51. 51. SUCCESS FACTOR ❶ ❷❸❹ Learn 360°
  52. 52. AGENDA❶ Case Introduction❷ Testing Strategy Evolution❸ Case ROI Analysis❹ Success factor❺ Revelation
  53. 53. OUR REVELATION不被传统的测试方法束缚;从产品用到的技术找到突破;持续改进的思想;不断尝试的理念
  54. 54. CONTRIBUTOR Xiaolu, Ye Christina, Geng Helen, Jiang Chen, Zhou Peipei, Jiang Coco, Dai Grace, Zhao Minghao, Lu Kent, Wang Izzie, Wang Phil, XuNOTHE END SUPPORTED FROM Daoyu, Wei Li, Zhao Jimmy, Yang Alex, Feng Alex, Su Jiwei, Li ... … Email: Christina.Geng@sap.com Sina Weibo: 耿晓儿de测试

×