The document discusses the importance of continuous testing for eCommerce applications, detailing various changes and testing strategies for new features, workflows, and data management. It emphasizes the need for reusable test scripts, integration with external systems, and effective test data management to ensure site quality and reduce revenue loss. It also highlights methods for testing high volumes of data and the significance of cross-browser and cross-device testing.

1. Continuous Testing
of eCommerce Apps
eureQa and Sauce Labs Webinar
September 7, 2016

2. • Co-Founder and CEO eureQa, LLC
• 20+ years experience in building, testing and managing Web
application projects across multiple industries
• bnittoor@sayeureqa.com
• @sayeureqa
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

3. Changes in an eCommerce site
Construction
Phase
Operations
Phase
RateofChange
• New Features & Workflows
• UI/CSS
• Integration with external
systems
Involves Code Changes
• Catalog (Product SKU) Updates
• Inventory Status
• Pricing
• Promotions & Discounts
Involves Data & Rules Changes
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

4. Changes in an eCommerce site
• New Features & Workflows
• UI/CSS
• Integration with external
systems
Involves Code Changes
• Catalog (Product SKU) Updates
• Inventory Status
• Pricing
• Promotions & Discounts
Involves Data, Content & Rules
Changes
Both types of changes require different
Testing Strategies
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

5. TESTING CODE CHANGES
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

6. Testing Code Changes
New Features &
Workflows
• Build new tests as you go
• High level of reuse across Test Cases/Scenarios
Scope of Test Test Solutions must be able to
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

7. Planning for Script Reuse
Copyright2016eureQa,LLC
AllRightsReserved
Manual Test Case
Step 1
Step 2
Step 3
Step 8
Step 9
BDD Scenario
Statement 1
Statement 2
Statement 3
Statement 8
Statement 9
GIVEN
THEN
WHEN
AND
AND
September7,2016

8. SCRIPT LIBRARY
Planning for Script Reuse
Copyright2016eureQa,LLC
AllRightsReserved
Test Case
Test Script 1
Test Script 2
Test Script 5
Test Script 6
Test Script 7
Step 1
Step 2
Step 3
Step 8
Step 9
Test Scenario
Test Script b
Test Script a
Test Script c
Test Script x
Test Script y
Test Case
Step a
Step b
Step c
Step f
Step g
Test Case
Step a
Step b
Step c
Step f
Step g
Test Scenario
September7,2016

9. Testing Code Changes
New Features &
Workflows
• Build new tests as you go
• High level of reuse across Test Cases/Scenarios
• Handle CSS/UI changes without breaking
scripts
• Tests must be elastic to layout changes
• Cross Browser & Cross Device Testing
UI/CSS
Scope of Test Test Solutions must be able to
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

10. Demo 1: Testing Code Changes
• Test Case/Scenario
• Search for a Book
• Search for product by KEYWORD and select from result list
• Verify product ATTRIBUTES
• Add selected product to the cart
• Verify QUANTITY OF ITEMS in Cart
• Delete All items from Cart
• Handling UI/CSS Changes
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

11. Testing Code Changes
New Features &
Workflows
• Build new tests as you go
• High level of reuse across Test Cases/Scenarios
• Handle CSS/UI changes without breaking
scripts
• Tests must be elastic to layout changes
• Cross Browser & Cross Device Testing
UI/CSS
• Trigger automatic execution of tests
• Integrate tests with external systems to
• Pull/Push data
• Trigger actions
• Get Results of Tests
Integration with
external systems
Scope of Test Test Solutions must be able to
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

12. Demo 2: REST API for
Integration
• eureQa REST API Support
• Start a test
• Optionally include test data & Meta Data
• Check on Status of test execution
• Get Results Summary for one or more tests
• Download Assets: screenshots, videos, logs etc.
• Stop/Abort a test
• Jenkins Integration Demo
• Trigger tests after build & deployment
• Send build #
• Get results summary in JUnit format
• Display results on Jenkins dashboard
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

13. TESTING DATA CHANGES
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

14. Testing Data Changes
Catalog (Product
SKU) Changes
• Run Test for a large set/variety of SKUs
• Check product attributes shown on site against
Catalog data
• Multi-Language, Currency & Localization
Scope of Test Test Solutions must be able to
• Collect Test Data in a Data File
• Parameterize Test Scripts to read Test Data
• Run the Test Scenario for all records in the Data File
Demo3:TestingDataChanges
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

15. Testing Data Changes
Catalog (Product
SKU) Changes
• Run Test for a large set/variety of SKUs
• Check product attributes shown on site against
Catalog data
• Multi-Language, Currency & Localization
Scope of Test Test Solutions must be able to
Pricing
• Check pricing for specific SKUs
• Check prices for all SKUs in a very short period
of time
Demo4:Usingasub-setofTestData
• Define Filter Criteria
• ‘Manual’ or ‘Automated’ Filters
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

16. Testing Data Changes
Catalog (Product
SKU) Changes
• Run Test for a large set/variety of SKUs
• Check product attributes shown on site against
Catalog data
• Multi-Language, Currency & Localization
• Use real-time inventory (stock status) in test
data
• Get data feeds or look up databases
• Clean up test data in database after test
execution
Inventory
Updates
Scope of Test Test Solutions must be able to
Pricing • Check pricing for specific SKUs
• Check prices for all SKUs in a very short period
of time
Demo5:Get TestDatafromdatabase
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

17. OTHER COMMON TEST SCENARIOS
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

18. Promotions & Discounts
Create
Promotion/
Discount
• Create new rules/
configurations
• SKUs
• User Profile
• Usage Pattern
• Time
• Location
• Generate Promotion
Code
Trigger Use of
Promotion/
Discount
Verify App/Site
response
• Simulate User
Profile
• Simulate Usage
pattern
• Input promo
code
• Was the code
accepted
• Were the correct
discounts applied
• Don’t forget
negative tests
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

19. Promotions & Discounts
Create
Promotion/
Discount
• Create new rules/
configurations
• SKUs
• User Profile
• Usage Pattern
• Time
• Location
• Generate Promotion
Code
Trigger Use of
Promotion/
Discount
Verify App/Site
response
• Simulate User
Profile
• Simulate Usage
pattern
• Input promo
code
• Was the code
accepted
• Were the correct
discounts applied
• Don’t forget
negative tests
Promo Code
Run the three steps in a single sequence to test Workflow
Test for different promos & discounts with appropriate Test Data
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved
Send from UI, DB or REST API

20. B2B Specific Scenarios
• Partner/Customer specific Catalogs
• Different views of Catalog - Same product, different SKU# or name
• Limiting access to specific SKUs in catalog
• Contracts
• Special Pricing
• Discounts/ Rebates
• Approvals
• Multi-Party Transactions
• Order from manufacturer (Special pricing/ contracts etc.)
• Shipment from Distributor
• Reconcile transactions between multiple parties
Strong Test Data Management and high level of script are a MUST HAVE to
build and effective testing solution for B2B eCommerce
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

21. Testing High Volumes
• Run a simple test (price check) for large data sets (entire
product catalog)
• Read data(in files or db) for hundreds of thousands of SKUs
• Partition data into smaller sets
• Run tests in parallel (tens or hundreds) to reduce elapsed test
time
• Collate Results
• Eliminate common problems that lead to increased cart
abandonment
• Incorrect Prices
• Wrong Content (image etc.) on Product page
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

22. In Conclusion
• Testing eCommerce apps/sites does not stop once the site
goes live
• Continuous testing is critical to
• Identify errors in Content, Data and Rules
• Reduce revenue loss due to site quality issues
• Continuous testing for eCommerce must
• Test all user workflows
• Incorporate strong Test Data Sourcing & Management
• Support on-demand and parallel test execution
• Include cross-browser and cross-device testing
September7,2016
Copyright2016eureQa,LLC
AllRightsReserved

23. http://www.sayeureqa.com
For more information & to Sign up
for a Personal Tour
Copyright2016eureQa,LLC
AllRightsReserved
September7,2016

24. Copyright2016eureQa,LLC
AllRightsReserved
September7,2016