Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Why do we care about the data used ...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Data Strategies
PAUL MERRILL
@dpaul...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Agenda
Define Terms
Identify Constr...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Define Terms
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Define Terms
Data Strategy - the co...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Define Terms
Signed Data - data whi...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Define Terms
Constraint - a charact...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Define Terms
Constraints factor dow...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Identify Constraints
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Examples of Constraints
Shared Envi...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Examples of Constraints
Data Source...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Example Data Strategies
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Strategies Have 2 Parts:
Creational...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Elementary Approach
Creation - Test...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Elementary Approach
SUT
Execution 1...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Elementary Approach
Pros:
Simple
No...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Refresh Data Source Approach
Creati...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Refresh Data Source Approach
SUT
Ex...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Refresh Data Source Approach
Pros:
...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Selfish Data Generation
Creation - ...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Selfish Data Generation
SUT
Executi...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Selfish Data Generation
Pros:
Uniqu...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Data Generation & Batch Cleanup
Cre...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Data Generation & Batch Cleanup
SUT...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Data Generation & Batch Cleanup
Pro...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Data Generation & Immediate Cleanup...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Data Generation & Immediate Cleanup...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Data Generation & Immediate Cleanup...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Creational Strategies
Elementary Re...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Cleanup Strategies
Elementary
(none...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
How do we know what data to
clean u...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Signed Data
Record Name: “test_2132...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Marked Data
An “isTestData” field o...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Internally Persisted Data
Keep a li...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Externally Persisted Data
Keep a re...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
In Review
Defined Terms
Identified ...
www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Thank you!
PAUL MERRILL
@dpaulmerri...
Upcoming SlideShare
Loading in …5
×

Data Strategies in Testing

906 views

Published on

You’ve written a large suite of test cases over the last few months. But every time someone changes the data model or wipes the database, you find yourself working extra hours to fix broken test cases! Test cases that were once consistent, reliable, and passing now look like a wasteland of red fail.

Success in automated testing is tightly bound to how you interact with data, also known as your “Data Strategy”.

In this presentation we’ll walk through several data strategies and their pros and cons. We'll talk about the constraints you experience in your environment and how to mix and match strategies appropriately.

Published in: Software
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Data Strategies in Testing

  1. 1. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Why do we care about the data used by the system under test?
  2. 2. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Data Strategies PAUL MERRILL @dpaulmerrill
  3. 3. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Agenda Define Terms Identify Constraints Walk Through Examples
  4. 4. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Define Terms
  5. 5. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Define Terms Data Strategy - the combination of code, procedure and infrastructure that affect how tests interact with data to stimulate the System Under Test (SUT).
  6. 6. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Define Terms Signed Data - data which has been modified so that it is easily recognizable. For example, an account with a user name like “TestUser” or an email address like “automated@testing.com”
  7. 7. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Define Terms Constraint - a characteristic that applies a force in a way that is limiting to an automated test or its environment.
  8. 8. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Define Terms Constraints factor down to: Policy Cost Test Expectations Infrastructure People
  9. 9. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Identify Constraints
  10. 10. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Examples of Constraints Shared Environment – Cost – Policy – Infrastructure Downtime – Policy – Cost – People Technical Skills – Cost – People
  11. 11. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Examples of Constraints Data Source Growth – Cost – Policy – Infrastructure Repeatable Tests – Policy – Test Expectations – People – Infrastructure Data Variance – Policy
  12. 12. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Example Data Strategies
  13. 13. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Strategies Have 2 Parts: Creational Strategy - how and when data is created. Cleanup Strategy - the method by which a data source is returned to a benign state.
  14. 14. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Elementary Approach Creation - Tests assume specific data in SUT - no creational strategy Clean up - No clean-up strategy, no management of data
  15. 15. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Elementary Approach SUT Execution 1 Execution 2 Execution 3 Data Source
  16. 16. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Elementary Approach Pros: Simple No implementation (quick) Cons: Tests are not repeatable No isolation of data Does not work on different systems
  17. 17. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Refresh Data Source Approach Creation - Tests assume specific data in SUT Cleanup - Data managed by refreshing the data source prior to test execution
  18. 18. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Refresh Data Source Approach SUT Execution 1 Data Source Refresh Execution 2 Refresh Execution 3 Refresh
  19. 19. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Refresh Data Source Approach Pros: Clean test bed (data- wise) per test Uniform, one-size-fits- all process No variance of data Less time/cost to implement Total control of data Cons: Timely Requires Downtime Blocks shared environments No variance of data Stale data over time Doesn’t work on different systems
  20. 20. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Selfish Data Generation Creation - Tests create the data they need in SUT Clean up - No management of data after generation May be random (e.g. via Faker) Create with batch process or in test Create via SUT or directly in Data Source Create In-test or pre-test
  21. 21. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Selfish Data Generation SUT Execution 1 Execution 2 Execution 3 Data Source Generate Data Generate Data Generate Data
  22. 22. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Selfish Data Generation Pros: Unique data per test Does not block shared environments No downtime No stale data Tests can run anywhere Tests become repeatable Cons: Unchecked growth of data source Requires understanding of data model Tests need to know about data
  23. 23. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Data Generation & Batch Cleanup Creation - Data generated for each test Cleanup - Data cleaned up after test run Clean up may happen in data source or via SUT
  24. 24. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Data Generation & Batch Cleanup SUT Execution 1 Execution 2 Execution 3 Data Source Generate Data Generate Data Generate Data Batch Clean up
  25. 25. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Data Generation & Batch Cleanup Pros: Unique data per test Does not block shared environments Minimal downtime No stale data Growth of data source kept in check Tests run anywhere Tests are repeatable Cons: Requires deep understanding of data model Must know what to clean Significant scaffolding must be created
  26. 26. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Data Generation & Immediate Cleanup Creation - tests generate their own data Clean up - for each test case immediately after execution Uses a TearDown method supplied by the framework
  27. 27. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Data Generation & Immediate Cleanup SUT Execution 1 Execution 2 Execution 3 Data Source Generate Data Generate Data Generate Data Clean up Clean up Clean up
  28. 28. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Data Generation & Immediate Cleanup Pros: Tests are repeatable Unique data per test Does not block shared environments Minimal downtime No stale data Growth of data source kept in check No data left over from other tests Tests run anywhere Cons: Requires deep understanding of data model Must know what to clean Significant scaffolding must be created
  29. 29. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Creational Strategies Elementary Refreshed Data Source Data Generation Shared Environment Low Downtime Repeatable Tests Data Variance Minimal Data Growth Significant Technical Skills
  30. 30. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Cleanup Strategies Elementary (none) Refresh Data Source Batch Immediate (Signed, Marked or Persisted) Shared Environment Low Downtime Repeatable Tests Strong Technical Skills
  31. 31. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved How do we know what data to clean up?
  32. 32. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Signed Data Record Name: “test_21323” Field Name: “Record Name” Format: “test_<random5digs>” Persist in: – a data source – code
  33. 33. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Marked Data An “isTestData” field or Data that is otherwise known to be test data because of its format or type, for example: – A “test user” created it – Linked to known test denotation
  34. 34. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Internally Persisted Data Keep a list in memory: – userIds: • 12312 • 4322 • 23423 – emailIds: • 12312 • 423423 • 234
  35. 35. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Externally Persisted Data Keep a record in a data source: – userIds: • 12312 • 4322 • 23423 – emailIds: • 12312 • 423423 • 234
  36. 36. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved In Review Defined Terms Identified Constraints Walked Through Examples
  37. 37. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved Thank you! PAUL MERRILL @dpaulmerrill

×