Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

10 Useful Testing Tools for Open Source Projects @ TuxCon 2015


Published on

If you count the alternatives, there are 50 tools for software testing focused on open source projects - test planning and management, test execution, test reporting, front-end and backend testing, automated mobile testing, security scanners, issue tracking and others

Published in: Technology
    Are you sure you want to  Yes  No
    Your message goes here
    Are you sure you want to  Yes  No
    Your message goes here
    Are you sure you want to  Yes  No
    Your message goes here

10 Useful Testing Tools for Open Source Projects @ TuxCon 2015

  1. 1. 10 Useful Testing Tools for Open Source Projects Peter Sabev QA Manager Questers @BORIME4KA
  2. 2. АКО ИМАХ ESP8266 ИЛИ ПОНЕ...
  4. 4. 1 час по-късно…
  7. 7. OPEN SOURCE VS. PROPRIETARY • Всичко е публично (код, тестове, бъгове, data flow и т. н.) • Лъсваш пред цял свят (но и ти помагат да си оправиш нещата). • Хората, които тестват и оправят бъгове, го правят ЗА УДОВОЛСТВИЕ. • Рядко нещо е публично. Никой отвън не знае какво има отдолу, как работи и кои данни къде отиват. • Ако тестовете ти са кофти, продуктът ти също може да е кофти и компанията губи пари и имидж. • Хората, които тестват и оправят бъгове, го правят ЗА ПАРИ
  8. 8. OPEN SOURCE VS. PROPRIETARY • Тестовете рядко зависят от конкретен environment, setup или конфигурация. • Сензитивна информация (пароли, device id, hostnames, ports) се подава като параметър, защото всичко е публично. • Тестовете са настроени да се пускат в тестовата среда на дадената компания. • Сензитивната информация обикновено не е проблем, защото тестовете се пускат в изолирана среда.
  9. 9. OPEN SOURCE VS. PROPRIETARY • Всеки може да допринесе за продукта и тестовете • Нужен е добър механизъм да отсееш хубавите неща от боклука. • При добре развито community, нещата се тестват много бързо от много хора, с много гледни точки. • Определена група професионалисти (или не чак такива) допринасят за продукта и тестовете. • Оправянето на проблеми минава през по-дълъг процес на одобрения, обикновено фиксът се тества от един човек и една гледна точка.
  10. 10. OPEN SOURCE VS. PROPRIETARY • Хората тестват каквото си искат, колкото си искат => част от нещата може да са неизтествани… • Практически няма централизирани tools, стратегия за тестване и поради тази причина static code analysis, performance, security и др. се тестват по-рядко. Има ясна тестова стратегия и тестов план
  11. 11. КАКВО Е НЕОБХОДИМО ЗА КАЧЕСТВЕН OPEN SOURCE ПРОЕКТ? • По-добра стратегия, процеси и управление на тестовете! • Continuous Integration и Continuous Deployment са изключително важни; • Задължително Code Review; • Подходящи интегрирани testing tools.
  12. 12. Благодаря за вниманието! Въпроси? Peter Sabev QA Manager skype: psabev82
  14. 14. TESTING TOOLS Requirements and Test Management API Testing Cross- Browser Testing Mobile Testing Performance Testing Web Robots Other Checks Test Reporting Tools Security Testing & Scanning Bug Tracking Backend Front-end
  15. 15. TOOL NO. 1: TESTLINK A GPL open source web-based requirements and test management and test execution system that allows executing test cases and tracking test results dynamically
  16. 16. BASIC FEATURES • Manage test cases, builds, results, milestones and tester assignments • Many people can work on test planning, design and execution simultaneously • Traceability between requirements and tests • Excellent bug tracking system integration (Mantis, JIRA, Bugzilla, FogBugz, Redmine, and others) • Custom UI, fields and user roles
  17. 17. THE ALTERNATIVES • Fitnesse (, acceptance testing and team collaboration • Tarantula (, modern tool for agile software projects test management, has requirements, small but growing community • qaManager (, releases tracking, good reporting, keeps track of testing cycles
  18. 18. TOOL NO. 2: SoapUI is a free and open source cross-platform tool that allows you to easily and rapidly create and execute automated functional, regression, compliance, and load tests
  19. 19. BASIC FEATURES • Simple drag-drop test creation • Complete test coverage for SOAP, WSDL, REST, JMS, db and others • Service mocking for SOAP, WSDL and REST • HTTP recording and replaying • Good load testing integration with LoadUI • Plugins support • Transform functional test into basic security test
  20. 20. THE ALTERNATIVES • Advanced REST client (, Google Chrome plugin for creating and testing custom HTTP requests • Rest Assured (, Testing and validating Java REST services in Ruby and Groovy • Postman (, construct, send and save requests quickly and analyze the responses sent by the API
  21. 21. TOOL NO. 3: A suite of tools to automate web browsers across many platforms. Selenium
  22. 22. BASIC FEATURES • Consists of basic and advanced part • Selenium IDE for simple automated scripts (a Firefox add-on) • Selenium WebDriver for robust, browser-based regression automation suites and tests • Very big community, including adoption in many commercial products • Supports Java, C#, Ruby, Python, Javascript, hundreds of other plugins • Scalable and portable - same scripts can be run on multiple environments (different number of machines, OS & browsers)
  23. 23. THE ALTERNATIVES • Watir (, open source (BSD) family of Ruby libraries for automating web browsers • WatiN (, inspired by Watir but uses C#. • Sikuli (, screenshot based automation • Canoo WebTest (, open source tool using Java 5, Groovy 1.6 and HTMLUnit 2.4 • Webrat (, a Ruby gem
  24. 24. TOOL NO. 4: Appium aims to automate any mobile app from any language and any test framework, with full access to back-end APIs and DBs from test code
  25. 25. BASIC FEATURES • Free, open-source, big community • Supports Selenium (i.e. tests can be written in Python, JS with Perl, Node.js, Java, Ruby, PHP, C#, RobotFramework, Objective-C, Clojure) • Focus on both iOS and Android native, hybrid or mobile web apps • Can be tested on real devices, simulators or emulators • Aims full access to mobile backend (e.g. turning wi-fi on/off) and DB • No need to recompile the apps tested
  26. 26. THE ALTERNATIVES • Ruby knowledge needed • Reuse scripts for iOS and Android • For iOS, needs adding additional library • All code written in Java • Heavy focus on Android and less on iOS • Generally very stable which is important for mobile world
  27. 27. TOOL NO. 5: Cross browser testing on multiple browsers, platforms and resolutions
  28. 28. BASIC FEATURES • Record/replay tool for different browsers (incl. mobile) and resolutions • Execute on one baseline browser and compare to the others • Screenshot comparison to check what exactly has failed • Pretty and easy-to-use UI, one-click re-execution • Automatic scan and Selenium scripts import plus execution • Made in Bulgaria by experienced QA specialists • Free for open source projects
  29. 29. THE ALTERNATIVES • Cloud access to 1000’s of device/browser combinations • Developer tools included in the browsers (such as Firebug, YSlow) • Separate use / automate accounts • Combines Selenium with JS unit tests • Very good integration with Appium • Video recordings for all manual test executions
  30. 30. TOOL NO. 6: Apache JMeter™ application is open source Java application designed to load test functional behavior and measure performance.
  31. 31. BASIC FEATURES • 100% open source, Java • Wide variety of server/protocol types to test with: HTTP, HTTPS, SOAP, REST, FTP, DB via JDBC, LDAP, JMS, SMTP, POP3, IMAP, MongoDB, TCP, native commands and shell scripts • Multithreading scalable for heavy load on a server, group of servers or network • Core extensible with pluggable functions • Caching and offline analysis/replaying of test results
  32. 32. THE ALTERNATIVES • OpenSTA (, distributed testing architecture for HTTP(S) heavy load tests with performance measurements • Grinder (, load testing framework with flexible scripting in Jython and Clojure • Pylot (, has GUI and console modes • Multi-Mechanize (, Python • Gatling ( is an open-source load testing framework based on Scala, Akka and Netty
  33. 33. TOOL NO. 7: The Zed Attack Proxy (ZAP) is an easy to use integrated penetration testing tool for finding vulnerabilities in web applications. p/OWASP_Zed_Attack_Proxy_Pr oject
  34. 34. BASIC FEATURES • The Open Web Application Security Project (OWASP) is a big community including corporations, universities and individuals • Works as intercepting proxy, no changes in code needed • Easy to use for people who are new to pen testing • Automated scanning plus manual tools for vulnerabilities • Supports SSL, smart cards, web sockets and wide range of scripting languages
  35. 35. THE ALTERNATIVES • Nessus ( scanner), passive scanner for network traffic • Wireshark (, protocol analyzer • OpenVAS (, open vulnerability scanner • Wapiti (, web apps scanner • Fiddler (, web debugging proxy • GoLismero (, combined scanners
  36. 36. TOOL NO. 8: GTmetrix tells you a lot about your website performance and gives recommendations by combining PageSpeed and YSlow scores.
  37. 37. BASIC FEATURES • Very easy to use – just type an URL • PageSpeed and YSlow scores • Page Load Details (time, size, number of requests) • Various Analysis Options – test from different regions and browsers • Waterfall, Video and Report History • Recommendations how to improve the website
  38. 38. THE ALTERNATIVES • Jenu/Xenu ( – Link checker • Markup Validation Service ( - HTML validator, supports HTML, XHTML, SMIL, CSS, MobileOK & other • Power Mapper ( • Website Speed Test ( • Spell Checker (, limited to 5 uses per day • Nibbler (, UX & marketing oriented
  39. 39. TOOL NO. 9: Serenity BDD helps you write better, more effective automated acceptance tests, and use these acceptance tests to produce world-class test reports and living documentation.
  40. 40. BASIC FEATURES • Based on Behaviour Driven Development (BDD) • Screenshots for each step in the test • Very nice-looking test reports including: • Details on passed/failed execution • Error messages for test cases • Execution times • Functional test coverage • Integration with popular bug tracking systems like JIRA
  41. 41. THE ALTERNATIVES • Testopia ( US/docs/Mozilla/Bugzilla/Testopia), a generic tool for tracking test cases, test management and test reporting • Zephyr (, test management and reporting platform, integrates with JIRA but can be used standalone • Sonar (, an open platform to manage code quality, works great with Java/Maven/Jenkins
  42. 42. TOOL NO. 10: The flexible and scalable issue tracker for software teams.
  43. 43. BASIC FEATURES • The most popular test management platform, free for open-source • Mobile browser support, email subscriptions and notifications • Highly customizable – gadgets, dashboards, filters, reports, workflows • Custom issue types, fields, statuses and resolutions • Massive plugin ecosystem, remote APIs • Bulk issue modifications • Wikipedia plus social network style collaboration
  44. 44. THE ALTERNATIVES • Bugzilla (, the open-source alternative to JIRA • Mantis (, open source, PHP/MySQL-based • Trac (, Python based • Redmine ( – Ruby-based, GPL, cross-platform • Trello (, simple collaboration tool with boards and cards, very useful for smaller projects
  45. 45. SPECIAL THANKS TO Dimitar Topuzov Slavy Slavov Borislav Traykov
  46. 46. QA BEER? qabulgaria/
  47. 47. Благодаря за вниманието! Въпроси? Peter Sabev QA Manager skype: psabev82