Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Automate Thyself

603 views

Published on

In this advanced session, we will investigate all the ways that you can automate your testing processes with TestBox and many CI and automation tools. From Jenkins integration, Travis CI, Node runners, Grunt watchers and much more. This session will show you the value of continuous integration and how to apply it with modern tools and technologies.

Main Points

Why we want to automate
Continuous Integration
ANT/CommandBox Test Runner
Setup of a Jenkins CI server
Travis CI integration
Pipelines CI integration
Node TestBox Runners
Grunt Watchers and Browser Live Reloads

Published in: Software
  • Be the first to comment

  • Be the first to like this

Automate Thyself

  1. 1. AUTOMATE THYSELF Luis Majano @lmajano @ortussolutions
  2. 2. WHO AM I? • Luis Majano • Computer Engineer • El Salvador • CEO of Ortus Solutions • Creator of all things Box • Sandals & Beaches -> ESRI -> Ortus
  3. 3. Story of Why People don’t test TestingTools ExecutingTests Automation Continuous Integration NOT ABOUT: BDD Basics of Testing
  4. 4. Where to start? Test Harness What to test? What not to test?
  5. 5. WHY PEOPLE DON’T TEST COMFORT
  6. 6. WHY PEOPLE DON’T TEST New Methodology New Learned Behavior
  7. 7. BIGGEST LIES IN SOFTWARE Don’t worry, we will create the tests and refactor it later! Testing will double my estimates on time
  8. 8. HOW DO YOU KNOW YOUR FEATURE IS DONE? Manual testing is only a momentary satisfaction!
  9. 9. HOW IT WORKS?
  10. 10. Bugs cost money Cost To Fix Time detected Requirements Design Building Testing Post-Release Time Introduced Requirements 1x 3x 5-10x 10x 10-100x Design --- 1x 10x 15x 25-100x Building -- -- 1x 10x 10-25x ^ Kaner, Cem; James Bach, Bret Pettichord (2001). Lessons Learned in Software Testing: A Context-Driven Approach. Wiley. p. 4. ISBN 0-471-08112-4.
  11. 11. • Just do it! • You will get dirty • It can hurt (a little) • Learned behavior NO MORE EXCUSES It will accelerate your development!
  12. 12. Continuous Integration Continuous Delivery Continuous Improvement AUTOMATION
  13. 13. AGENDA • Proven event-driven ColdFusion Framework • Follows OO principles and best practices • Conventions Based • Non-intrusive framework you code in ColdFusion • More than MVC => ColdBox Platform • SoftwareToolkit • Remote Framework • NOT A MAGIC FRAMEWORK,YOU STILL NEEDTO CODE!!
  14. 14. cbVue application install coldbox-samples/cbVue
  15. 15. TestBox Exposed API Docs Core Test Browser Test Harness Test Runner Samples install testbox —saveDev install testbox@be —saveDev
  16. 16. Test Harness Automated test results! xUnit/BDDTest Bundles Harness bootstrap HTML Runner ANT Runner
  17. 17. Test Bundle CFCs • Inherits from testbox.system.BaseSpec

  18. 18. RUN TESTS • Execute bundles via the URL • http://localhost/tests/bundle.cfc • SOAP Runner • HTTP/REST Runner • ANT Runner • NodeJS Runner • CommandBox Runner • Custom Runner
  19. 19. HOW IT WORKS? TESTBOX REPORTERS •ANTJunit :Variant of JUnit for Jenkins •Codexwiki : Mediawiki syntax •Console : Server console •Doc : Semantic HTML •Dot :Awesome Dots •JSON •JUnit •Raw : CFML Struct •Simple :A basic HTML reporter •Text : Back to the 80’s,Text •MinText : Great for CommandBox Runners •XML  •Tap :A test anything protocol reporter •Min : Minimalistic Report
  20. 20. Developer Automation
  21. 21. HOW IT WORKS? • Execute bundles via the URL • http://localhost/tests/bundle.cfc?method=runRemote • SOAP Runner • HTTP/REST Runner • ANT Runner • NodeJS Runner • CommandBox Runner • Custom Runner Bundle CFC Runner
  22. 22. HTML Runner
  23. 23. CommandBox Runner Update box.json with runner location
  24. 24. testbox run
  25. 25. { "runner": "http://127.0.0.1:57538/tests/ runner.cfm", "directory": "/tests/specs", "recurse": true } NodeJS Test Runner npm install -g testbox-runner create .testbox-runnerrc 1 2 Run it : testbox-runner2
  26. 26. HOW IT WORKS? • Execute bundles via the URL • http://localhost/tests/bundle.cfc?method=runRemote • SOAP Runner • HTTP/REST Runner • ANT Runner • NodeJS Runner • CommandBox Runner • Custom Runner ANT TESTING
  27. 27. HOW IT WORKS? DEVELOPER AUTOMATION 1.CommandBox Watchers 2.Grunt Watchers 3.Grunt Watchers + testbox-runner 4.Grunt Watchers + CommandBox 5.ColdBox Elixir
  28. 28. HOW IT WORKS? Code -> Tests -> Commit -> Pull -> Run -> Results -> Refactor -> Repeat -> CONTINUOUS INTEGRATION
  29. 29. HOW IT WORKS? Code -> Tests -> Commit -> Pull -> Run -> Results -> Refactor -> Repeat -> CI SERVERS
  30. 30. • Luis Majano • Computer Engineer Jenkins + CommandBox wget http://mirrors.jenkins-ci.org/war-stable/latest/jenkins.war box server start warpath=jenkins.war 1 2
  31. 31. Jenkins Features •FREE - Open Source Java application •Not so easy to configure and use •CI = Jenkins Jobs •Thousands of plugins and extensions •Jobs can be scheduled, parallelized, executed over different machines •Jenkins Slaves •Powerful but convoluted
  32. 32. Travis CI - Sign Up https://travis-ci.org/ Sign Up 1 2
  33. 33. Travis Features •FREE for Open Source Projects •Runs distributedVM’s and Container Support •Triggers Build Script via git repository commits (.travis.yml) •Multiple language support •Many integrations and extensions •Many notification types •No ability to schedule/manual builds •Great for open source projects!
  34. 34. Accounts Activate For Repositories 1 2 Create .travis.yml3 Commit to repository4 Travis CI - Getting Started CI Script runs on each commit, pull request, branches
  35. 35. TRAVIS + ANTlanguage: java branches: only: - development sudo: required dist: trusty before_install: - sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622 - sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a /etc/apt/sources.list.d/commandbox.list install: - sudo apt-get update && sudo apt-get --assume-yes install commandbox - box install - box server start script: - ant -f tests/build.xml
  36. 36. TRAVIS + COMMANDBOXlanguage: java sudo: required dist: trusty before_install: - sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622 - sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a /etc/apt/sources.list.d/commandbox.list install: - sudo apt-get update && sudo apt-get --assume-yes install commandbox - cd tests - box install - box server start script: - box testbox run
  37. 37. TRAVIS + MULTIPLE ENGINES language: java sudo: required env: matrix: - ENGINE=lucee@4.5 - ENGINE=lucee@5 - ENGINE=adobe@2016 - ENGINE=adobe@11 - ENGINE=adobe@10 before_install: - sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622 - sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a /etc/apt/sources.list.d/commandbox.list install: - sudo apt-get update && sudo apt-get --assume-yes install commandbox - box install before_script: - box server start cfengine=$ENGINE port=8500 script: - box testbox run
  38. 38. Gitlab https://gitlab.com/users/sign_in Sign Up 1 2 or Self-Hosted3
  39. 39. Gitlab Features • All in one tool: CI, repository, docker registry, issues/milestones, etc. • Same asTravis in concept - CI Runner • Docker based • CI Runners can be decoupled to a docker swarm • Idea of CI pipelines • Pipelines composed of jobs executed in stages • Jobs can have dependencies, artifacts, services, etc • Schedule Pipelines (Cron) • Jobs can track environments • Great stats + charting
  40. 40. .GITLAB-CI.YML image: ortussolutions/commandbox:alpine stages: - build - test - deploy build_app: stage: build script: # Install dependencies - box install production=true run_tests: stage: test only: - master # when: manual, always, on_success, on_failure script: - box server start - box testbox run
  41. 41. Resources • TestBox : ortussolutions.com/products/testbox • CommandBox: ortussolutions.com/products/commandbox • Slack: boxteam.herokuapp.com • CFML Slack: #box-products • travis-ci.org • jenkins.io • gitlab.com • Twitter • @lmajano • @ortussolutions
  42. 42. Q & A

×