Brief of some of the best practices being followed at Google Organization for testing any software

3. Many Developers : One Tester
Many : One
Many: One

4. Quality Process: Philosophy
Every Body Is Engineer
Quality is Responsibility started/owned by
Management, Developers, Testers

5. Quality Process

6. Quality Process – Various Types Of Test
• verify the behavior of a single unit
of code
• No External Dependency. If
needed they are mocked.
• Used/Executed Frequently to lead
quick discovery of bug
• Shorter build and execution
times for tests.
• No/Minimum Test Infrastructure
Executed by Developer
• Execution Time : less than 100 ms
• Kill test target after 1 minute
• Lead To Code Quality
• Executed by SWE
• Automated
• validate the interaction of one or more
application modules
• External Dependency like interaction with
database, local file system
• Less Frequently Executed
• Test Infrastructure
• Executed by SET
• Execution Time : less than 1 sec
• Kill test target after 5 mins
• Lead to Product Quality
• Executed By SET
• Usually Automated
• Verify the whole application(all application
subsystems from the UI down to backend
• data storage)
• Must use external resources like remote
data base, file system
• Execution Time : ASAP
• Kill test target in 1 hour
• Lead to Product Quality
• Executed by TE
• Automated Or Maual
0
10%
20%
70%
Types Of Tests
Small Tests Medium Tests Large Tests

7. Quality Process – Test Execution System
Google Execution System
Indicates the potential
resource needs of a
test
Test running in this system shouldn’t
have any persistent side effect. They
must leave their environment exactly in
the state when it started
Uses test sizes to
distinguish fast
jobs from slower
ones
Cancels execution and reports a
failure for any test that exceeds the
time or resource requirements
associated with its test size.
Each test must be independent from
other tests so that tests can be
executed in any order.

8. Quality Process – Continuous Integration System
 Based On Dependency Analyses : Build rules are defined to decide which tests to run based on the changes done

9. Quality Process – Test Certified
 Set up test coverage bundles.
 Set up a continuous build.
 Classify your tests as Small, Medium, and Large.
 Identify nondeterministic tests.
 Create a smoke test suite.
Automate running of smoke tests
before submitting new code.
 Smoke tests should take less than 30 minutes to run.
 No nondeterministic tests.
 All significant features are tested by integration tests
 Total test coverage should be at least 40%.
 Test coverage from small tests alone should be at least 25%.
 Add a test for each nontrivial bug fix.
 Actively use available analysis tools.
 Total test coverage should be at least 60%.
 Test coverage from small tests alone should be at least 40%.
 Incremental coverage by small tests >= 50%.
 New significant features are tested by integration tests.
 Require tests for all nontrivial changes.
 No releases with red tests.
 Require a smoke test suite to pass before a submit.
 Incremental coverage by all tests >= 50%.
 Incremental coverage by small tests >= 10%.
 At least one feature tested by an integration test.
Level1
Level5
Idea :
Hiring/Getting Developers
participating in Testing

10. Quality Process – Test Planning
“Test plans are the first testing artifact created and the first one to die of
neglect.”
 10 Minute Test Plan

11. Quality Process – Test Planning
If the test plan is actually important, all of these things would be happening
every day………………
 Test plan actually drive testing activity throughout the entire lifecycle of a
product?
 Testers continually consult it as they divvy up the features among them so as to
divide and conquer the app?
 Developers insist the test plan be updated as they add and modify functionality?
 Development managers keep a copy open on their desktop as they manage their
todo list?
 Test manager refer to the contents of a test plan in status and progress meetings?

12. Quality Process – Test Planning- ACC
Qualities and characteristics that promote the product and
distinguish it from the competition. Attributes are the reasons
people would choose to use the product over a competitor.)
Components are the building blocks that together constitute the
system in question.
They are the core components and chunks of code that make the
software
what it is.
Capabilities are the actions the system performs at the
command of the user. They are responses to input, answers to
queries, and activities accomplished on behalf of the user.
Attribute
Component
Capability
C
C
A

13. Quality Process : Test Planning -ACC

14. Quality Process : GTA(Google Test Analytics)

15. Quality Process : Heat Map

16. Quality Processes : Crowd Sourcing(Testing)
 Instead of one tester trying to figure out the way
1,000 users will work, you have 1,000 users acting
like testers
 Weakness of the crowd is the time it takes for
them to learn an application and put it through
its paces
 Real-world multiplatform testing
 Flexibity : People from various Time Zones

17. Quality Process : Free Testing WorkFlow
 Cost Of Testing : ~0 Dollar
 Cost next to nothing
 Instantaneous test results
 Require little or no human intervention
 Very flexible because one size does not fit all

18. Impact
Impact
Every
Engineer
Every Team
Every
Manager
Annual
Discussions/Promotions
Hiring
Impact#High Quality
Impact#On Time Release Every Process
: Understand Project Goals, Schedule and do what no body has done ever.

19. Organization Structure
SWE(Software TE(TestSET(Software
Test Lead Manager
Test Engineering Manager
Test Director
Senior Test Director
Test Lead

20. Organization Structure
SWE(Software Engineer) SET(Software Engineer In Test) (TE)Test Engineer
Feature Developer Test Developer User Developer
Owner : Code for application to be tested/delivered Owner : Code for test Infrastructure/Automation Owner : Test Results for these infrastructure
Application Code Quality Test Infrastructure Code Quality
Product Quality(Partial)
Product Quality(security, privacy, performance,
reliability, usability, compatibility,
Globalization)
Unit Tests
FDD, Architectural design
TDD(Partial)
TDD
Test Plan(Partial)
Test Plan
Participate in writing tests Writes and owns these tests Execute these tests
SET interviewers focus on how the candidate thinks about
the solution
TE Interview
A good SET candidate should not have to be told to test the
code she writes. Itshould be an automatic part of her