Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

0

Share

Download to read offline

PAC 2019 virtual Philip Webb

Download to read offline

Mark59 – A Jmeter/Selenium/CI Open Source P&V Solution

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

PAC 2019 virtual Philip Webb

  1. 1. Mark59 : A Jmeter-Selenium-CI Open Source Solution By Philip Webb
  2. 2. Overview • The Team • Problems and progression through solutions - ‘traditional P&V’ to CI P&V to ‘Mark59’ • A technical overview of Mark59 - scripting
  3. 3. Design Influences • My background • Mainframe applications, odds and ends, Java development, Functional Testing, P&V • Our IT environment • Functional (QA) testers executing Regression Suites • Server-based technologies • Cloud migration
  4. 4. SourceControl • Some-project_dir/My_favourite_LR-script_20180909_v3
  5. 5. Jenkins • Running individual LoadRunner scripts • Executing LoadRunner scenarios and report generation
  6. 6. Jenkins Reports
  7. 7. DataHunter • Test data management • Mark59 development begins with the ‘DataHunter’ application • … Later also becomes the Mark59 ‘sample applicaton’
  8. 8. DataHunter Usage: HomeScreen
  9. 9. DataHunter Usage: AddPolicyScreen
  10. 10. DataHunter Usage: AddPolicyResults Screen
  11. 11. Baselines • Comparisons in Excel • Baseline to baseline variation • ‘Mark59’ : Creation of the ‘Trend Analysis’ graphical application
  12. 12. Trend Analysis – The Graph The graphic spins ….
  13. 13. Trend Analysis – The Graph : Hover Over and Data Table
  14. 14. Trend Analysis – SimpleFiltering
  15. 15. Trend Analysis – Advanced Filters
  16. 16. SLAs
  17. 17. Transactional SLAData Entry • Note the SLA Bulk Edit functions
  18. 18. ApplicationDashboard (SLA summaries)
  19. 19. UsingJmeter with Java Selenium ? • Triggered by a ‘problem project’ • .. but with a full java selenium regression suite available • Alignment with design philosophy • Skills growth and knowledge sharing • Justified by cost savings • Jmeter • Java friendly, well-known established open source tool • ‘Mark59’ : Selenium-Implementation for Jmeter built
  20. 20. Mark59 Selenium Implemenation • Eclipse based development • Maven builds • Central
  21. 21. Report Changes for JmeterImplementation in Mark59 • Reports Generation
  22. 22. Henrik and theNeoload Selenium Recipe
  23. 23. Phil’s Mark59 Selenium and JmeterRecipe Ingredients • 1 x Fully cooked Selenium DSL • 1 x Raw Apache Jmeter • Min 1 fresh Web Browser (Chrome preferred) • Min 1 good Selenium Driver (ChromeDriver preferred) • 1 x Fully prepared Eclipse (Springboot Plugin optional) • 2 x Jars of Mark59 • Min 1 DIY script sauce Method • Mix together Ecplise, DSL and Driver • Pour in the jars of Mark59 • Add your script sauce and use a Maven to place the results an a clean jar • Add all into the raw Jmeter • Switch on the Jmeter and Run • Note: kitchen is assumed to have a git • Advanced Chefs • Should consider combining all ingredients in a Jenkins
  24. 24. Mark59 Data Flow Overview (Jmeter) Diagram Excludes: LoadRunner Processing, Logging, Archiving, API Target System
  25. 25. Scripting
  26. 26. Script-Jmeter Parameter Interface (1 of 2)
  27. 27. Script-Jmeter Parameter Interface (2 of 2)
  28. 28. Scripting- Script Structure and Inline Execution • jm.startTransaction("DH-lifecycle-0200-addPolicy"); protected Map<String, String> additionalTestParameters() { public class DataHunterLifecyclePvtScript extends SeleniumAbstractJavaSamplerClient { protected void runSeleniumTest(JavaSamplerContext context, JmeterFunctionsForSeleniumScripts jm, WebDriver driver) { jm.endTransaction("DH-lifecycle-0200-addPolicy"); public static void main(String[] args) throws InterruptedException{ Log4jConfigurationHelper.init(Level.INFO) ; DataHunterLifecyclePvtScript thisTest = new DataHunterLifecyclePvtScript(); //1: single thisTest.runSeleniumTest(KeepBrowserOpen.ONFAILURE); //2: multi-thread thisTest.runMultiThreadedSeleniumTest(2, 2000); //3: multi-thread with parms Map<String, java.util.List<String>>threadParameters = new java.util.LinkedHashMap<String,java.util.List<String>>(); threadParameters.put("USER", java.util.Arrays.asList( "USER-MATTHEW", "USER-MARK", "USER-LUKE", "USER-JOHN")); threadParameters.put(SeleniumDriverFactory.HEADLESS_MODE, java.util.Arrays.asList( "true" , "false" , "true" , "true")); thisTest.runMultiThreadedSeleniumTest(4, 2000, threadParameters); } } } You normally only chose one of the 3 options ! Mark59 instantiates the Selenium WebDriver
  29. 29. Selenium DSL basics ( 1 of3 )
  30. 30. Selenium DSL basics (2 of3)
  31. 31. Selenium DSL basics (3 of3) Method called by script
  32. 32. Selenium – Beyondthe Basics
  33. 33. Scripting– Logging andLog Control
  34. 34. Scripting: JavaDocs are awork inprogress…..
  35. 35. Scripting: Documentation(@ Beta4 release) • The Sample project : https://github.com/iagcl/mark59/tree/master/dataHunterPVTest • The “QuickStart”notes : https://github.com/iagcl/mark59 • https://mark59.com status : “Coming soon” That’s a tumbleweed
  36. 36. Jenkins CI
  37. 37. Jenkins JobFlows Build and Deploy Jmeter Test Executing a Single Injector Jmeter Test Executing a Jmeter Distributed Test
  38. 38. AJenkins Check Job: Parameters
  39. 39. AJenkins Results Job (1 of2) (fudged) links to logging directories on injectors
  40. 40. AJenkins Results Job (2of2 )
  41. 41. Injector LoadCapacity Data (Virtual /non-AWSServers) • All injectors : • Inter(R) Xeon(R) E7-2870 @ 2.40GHz 16 CPU (Virtual) 24 GB Ram 200GB Disk Win ‘16 • Largest current single server Selenium-only Test • 13,000 txns/hr , 45 threads : CPU Util Average 30% • ‘Just to Big’ for single server Selenium-only Test (10% degradation in responses) • 27,000 txns/hr , 60 threads , • When executed on a single injector : CPU Util Average 50 – 60 % • Distributedacross 2 injectors : CPU Util Average 20 – 30% • Largest distributed (2 servers) Selenium-only Run • 44,000 txns/hr , 100 threads : CPU Util Average 30-45%
  42. 42. LoadCapacity: historical CPU variance distributedinjectors • Largest distributed (2 server) Selenium-only Run (44000 txns/hr , 100 threads) Injector CPU Utilization Averages (last 10 soak runs) • Guesstimate max practical capacity using similiar sized VMs : ~ 10 servers => ~ ~ 200,00 txns/hr Injector 1 Injector 2
  43. 43. Thank You • Follow-up • If ‘Mark59’ gets traction we’ll monitor on Stack Overflow • Please watch for updates and progress at https://github.com/iagcl/mark59 • and soon at https://mark59.com

Mark59 – A Jmeter/Selenium/CI Open Source P&V Solution

Views

Total views

171

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

1

Shares

0

Comments

0

Likes

0

×