100% walkaway automation
100% Walkaway Automation
100% walkaway automation
100%
› 100% automated
› 100% walkaway
› 100% of the time
100% walkaway automation
Walkaway
› Runs unattended
› Doesn’t need maintained
› Doesn’t break
› Fails fast when it does
› You know exactly what went wrong
› It’s easy to fix
100% walkaway automation
Automation
› Fast
› you don’t wait for it
› Reliable
› it doesn’t break
› Accurate
› no false positives
100% walkaway automation
It ain’t gonna happen.
› 100% walkaway automation is an ideal
› How do you work towards a goal that’s not achievable?
› That’s not S.M.A.R.T.
› Don’t let the perfect be the enemy of the good
› Don’t use it as an excuse to procrastinate
100% walkaway automation
What can I do now?
› Evaluate where you are
› Identify weak points
› Pick low hanging fruit
› Make a plan
› Set S.M.A.R.T. goals
› Short , medium, long term
100% walkaway automation
Where are you now?
› Unit tests
› Integration tests
› System tests
› Builds
› Deployments
› Requirements coverage
100% walkaway automation
Test Certification
http://dw/Teams/Test_Tools/Test_Certification_Guide
100% walkaway automation
Go
› PKG pipeline
› ACC pipeline
› PRD pipeline
What do you need to make Go work for you?
100% walkaway automation
Step away from the GUI
› It’s brittle
› It’s too high level
› Isolate the system
› Use fakes
› Test features at the feature level
100% walkaway automation
Apps are like Ogres
› Big, green, and smelly
› No, layers
› Isolate concerns
› What is your testing perspective?
100% walkaway automation
UI tests are fragile
› The UI changes frequently
› Layers of abstraction
› Too much maintenance
› Feedback is not instant
100% walkaway automation
What should you
automate?
› Repetitive
› Slow
› Error-prone
Automate if it is faster than doing it manually
100% walkaway automation
Unit Tests
› Code coverage
› Are they really unit tests?
› Mock dependencies
› Design for testability
› Architecture
› Components
100% walkaway automation
Feature Tests
› Test the feature
› Isolate the system
› Use fakes
› Describe the feature
› Tests as documentation
100% walkaway automation
System Tests
› Stable environment
› Stable tests
› Real (or nearly real) data
› For testing the whole system
› Not end-to-end unit tests
100% walkaway automation
Manual Tests
› Presentation & Layout
› Usability
› Exploratory testing
100% walkaway automation
Regression Tests
› Repeatable
› Automated
› Every time you build
100% walkaway automation
Automate defects
› Write a test that fails for every defect
› Automate it
100% walkaway automation
Build Automation
› Works for me
› Anyone should be able to checkout and run
› It should build on GO agents
› It should be clear why it fails
› Know your dependencies
› tools , scripts, environment variables, data
100% walkaway automation
Deployment
› Can you do it without manual intervention?
› What would it take?
› Manual tests, automated decisions
100% walkaway automation
Requirements
› Executable tests
› Traceability
› Testable acceptance criteria
100% walkaway automation
Let’s put the “D”
back in TDD
› Test Driven Development
› Write the tests first
› Anyone should be able to test
› It doesn’t have to be 100% automated yet
100% walkaway automation
Let’s put the “D”
back in BDD
› Behavior Driven Design
› Product should own acceptance criteria
› Business can help write tests
› Tests should express a user’s needs
100% walkaway automation
User Stories
› Are about the user
› Tell a story
› Have a plot
› with conflict and resolution
› Not just “Gherkin” unit tests
100% walkaway automation
Test Harnesses
› Make developers write the code
› Define the requirement
› What are your inputs?
› How do you check expected results?
100% walkaway automation
Who you gonna call?
SETs
› Evaluate your test certification goals
› Write test harnesses & tools
› Embed on your team
› Short term
› Help make code testable
› Teach test automation practices

walkaway-automation

  • 1.
    100% walkaway automation 100%Walkaway Automation
  • 2.
    100% walkaway automation 100% ›100% automated › 100% walkaway › 100% of the time
  • 3.
    100% walkaway automation Walkaway ›Runs unattended › Doesn’t need maintained › Doesn’t break › Fails fast when it does › You know exactly what went wrong › It’s easy to fix
  • 4.
    100% walkaway automation Automation ›Fast › you don’t wait for it › Reliable › it doesn’t break › Accurate › no false positives
  • 5.
    100% walkaway automation Itain’t gonna happen. › 100% walkaway automation is an ideal › How do you work towards a goal that’s not achievable? › That’s not S.M.A.R.T. › Don’t let the perfect be the enemy of the good › Don’t use it as an excuse to procrastinate
  • 6.
    100% walkaway automation Whatcan I do now? › Evaluate where you are › Identify weak points › Pick low hanging fruit › Make a plan › Set S.M.A.R.T. goals › Short , medium, long term
  • 7.
    100% walkaway automation Whereare you now? › Unit tests › Integration tests › System tests › Builds › Deployments › Requirements coverage
  • 8.
    100% walkaway automation TestCertification http://dw/Teams/Test_Tools/Test_Certification_Guide
  • 9.
    100% walkaway automation Go ›PKG pipeline › ACC pipeline › PRD pipeline What do you need to make Go work for you?
  • 10.
    100% walkaway automation Stepaway from the GUI › It’s brittle › It’s too high level › Isolate the system › Use fakes › Test features at the feature level
  • 11.
    100% walkaway automation Appsare like Ogres › Big, green, and smelly › No, layers › Isolate concerns › What is your testing perspective?
  • 12.
    100% walkaway automation UItests are fragile › The UI changes frequently › Layers of abstraction › Too much maintenance › Feedback is not instant
  • 13.
    100% walkaway automation Whatshould you automate? › Repetitive › Slow › Error-prone Automate if it is faster than doing it manually
  • 14.
    100% walkaway automation UnitTests › Code coverage › Are they really unit tests? › Mock dependencies › Design for testability › Architecture › Components
  • 15.
    100% walkaway automation FeatureTests › Test the feature › Isolate the system › Use fakes › Describe the feature › Tests as documentation
  • 16.
    100% walkaway automation SystemTests › Stable environment › Stable tests › Real (or nearly real) data › For testing the whole system › Not end-to-end unit tests
  • 17.
    100% walkaway automation ManualTests › Presentation & Layout › Usability › Exploratory testing
  • 18.
    100% walkaway automation RegressionTests › Repeatable › Automated › Every time you build
  • 19.
    100% walkaway automation Automatedefects › Write a test that fails for every defect › Automate it
  • 20.
    100% walkaway automation BuildAutomation › Works for me › Anyone should be able to checkout and run › It should build on GO agents › It should be clear why it fails › Know your dependencies › tools , scripts, environment variables, data
  • 21.
    100% walkaway automation Deployment ›Can you do it without manual intervention? › What would it take? › Manual tests, automated decisions
  • 22.
    100% walkaway automation Requirements ›Executable tests › Traceability › Testable acceptance criteria
  • 23.
    100% walkaway automation Let’sput the “D” back in TDD › Test Driven Development › Write the tests first › Anyone should be able to test › It doesn’t have to be 100% automated yet
  • 24.
    100% walkaway automation Let’sput the “D” back in BDD › Behavior Driven Design › Product should own acceptance criteria › Business can help write tests › Tests should express a user’s needs
  • 25.
    100% walkaway automation UserStories › Are about the user › Tell a story › Have a plot › with conflict and resolution › Not just “Gherkin” unit tests
  • 26.
    100% walkaway automation TestHarnesses › Make developers write the code › Define the requirement › What are your inputs? › How do you check expected results?
  • 27.
    100% walkaway automation Whoyou gonna call? SETs › Evaluate your test certification goals › Write test harnesses & tools › Embed on your team › Short term › Help make code testable › Teach test automation practices

Editor's Notes

  • #2 What do I mean by 100% walkaway automation?
  • #3 100% Automation 100% walkaway 100% of the time
  • #4 Anyone can run it Anyone can maintain it Anyone can add to it
  • #6 100% walkaway automation is an ideal. How do you work towards a goal that’s not achievable? That’s not S.M.A.R.T. Don’t let the perfect be the enemy of the good. Don’t use it as an excuse to procrastinate.
  • #9 http://dw/Teams/Test_Tools/Test_Certification_Guide
  • #21 Not just code dependencies Tools Scripts Environment variables Data