The document consists of slides from a January 2013 seminar by the Dynacron Group discussing cloud-based load testing and browser visual comparison tools. Key features of various tools, including CloudBees and Neustar, are highlighted along with the advantages of cloud-based testing over local testing. The document also includes a section on the limitations and challenges faced during testing.

1. 1/28/2013 1
QA LAB IN THE CLOUD
Dynacron Group
Slides from Jan 2013 Seminar

2. 1/28/2013 2
Summary
• These slides are from a January 2013 seminar
by Dynacron Group
• The slides were intended to be viewed as part
of a demonstration
• We’ll probably post video of it next time
• To stay in touch, sign up for our newsletter!
• http://www.dynacrongroup.com/newsletter

3. 1/28/2013 3
Cross Browser Visual Comparison
• Layout
• Image Loading
• JavaScript Errors
• Smell test

4. 1/28/2013 4
Browser Visual Diff Tools
• Browsera
• BrowserBite
• Browsershots
• Adobe BrowserLab
• Sauce Scout
• ...and many more

5. 1/28/2013 5
Key Features
• Local/Remote
• Browser Selection
• Public/Private
• Authentication
• Screenshot Comparison
• Side by side? Overlay? Highlighting? Manual?
• Link Crawling
• JavaScript Validation

6. 1/28/2013 6
DEMONSTRATION
Browsera and BrowserBite

7. 1/28/2013 7
Cloud Infrastructure
• AWS
• Cloud Foundry
• Heroku
• CloudBees
• … and many, many more

8. 1/28/2013 8
CloudBees
• Complete Java Infrastructure Solution
• Development Pipeline
• Source Control: svn or git
• Build Server: Jenkins, Sonar
• Binary Repository
• Production
• Application Servers: Jboss, Tomcat, etc
• Database: MySQL, Mongo, Couch, ProstgreSQL
• Monitoring

9. 1/28/2013 9
CloudBees - Integration
• Github
• Cloud Services (AWS, Cloud Front, etc)
• IDE, Maven plugins, or Native tools for
deployment

10. 1/28/2013 10
DEMONSTRATION
CloudBees

11. Neustar Load Testing

12. 1/28/2013 12
About Neustar:
• Among other services, Neustar offers cloud-based
load testing.

13. 1/28/2013 13
Why choose cloud-based load
testing?
• Identify bottlenecks/unusual performance issues
before going live.
• Local load testing adds unnecessary load to
network/cpu.
• Local load tests originate in one location.

14. 1/28/2013 14
What does Neustar have to
offer?
• Real browser tests
Browser: Firefox
Asynchronous calls included or blacklisted
Screen shots, video playback of failures
• Basic tests
Headless Http client
Ideal for testing web services
10x cheaper than real browser tests
Direct Http calls, faster performance

15. 1/28/2013 15
What does Neustar have to
offer?
• Schedule tests, ramp up users to peak load.
• Tests originate from multiple IPs/geographical locations.
• Integrated metrics/report publishing.
• Query and export test results.
• Local and UI script validation.

16. 1/28/2013 16
Anatomy of a test script:
Test scripts written in Javascript
Tests occur
in a
transaction
A transaction
is comprised
of steps

17. 1/28/2013 17
Demo

18. 1/28/2013 18
Gotchas
• File based test resources are unreliable.
• Limited script size (Too many instructions exception).
• Limited client-side SSL support.
• Heap size issues running basic tests.

19. 1/28/2013 19
Gotchas
• No failure pagination (Large failure list causes logging
page to become unresponsive).
• Firefox-only in browser tests (Chrome support on
2013 roadmap).
• Number of available agents dependent on origin
datacenter (eg: more in DC than Oregon).
• Too many pauses/waits can cause transaction to fail.

20. 1/28/2013 20
Questions?