5. Image Injection Across the SDLC
Release Timeline: Months → Weeks
Discussion
How often do you test image
injection? How many devices, how
much time?
What percentage of your tests involve
image injection?
What are the use cases?
CI
Coverage
Integration Tests
End-to-end Tests
Non-functional Tests
Unit
Tests
Int.
Tests
Unit
Tests
Monitoring
E2E
Tests
Why now? Because businesses are aligning to meet demand.
Process Blocks
A simple view of the delivery train, there’s a forward motion
There is one cycle, owned by the dev cycle, that’s where we want to sell into
Testing across the whole cycle: unit by devs, integration and end-to-end tests
We omitted detail like additional types of tests (performance, regression, etc.) in the test stage
What is “in-cycle”
Splitting the process at the developer Continuous Integration (CI) system
Though you may hear “Jenkins”, you still need to clarify who owns it (dev or QA)
Tests usually have to be automated to go in-cycle
Personas
QA is when we refer to tester outside the cycle (this is who we talk to today)
DevTest is when we refer to tester inside the cycle
Development are the people who code the app; tests and code are different
The world is not black and white; some QA are DevTest, some Devs do testing
In most IT, we see a difference between Devs who code and DevTest who test
Often we see DevTest report to the same place that development team does, not QA
Challenges
Quality
Customers don’t know what they need to cover in the “test” phase; we educate them on coverage, etc.
The same problems exist when testing is pulled into the “build” phase (CI in-cycle), but we’re not educating here
Coverage is more severe in “test” because it’s the last line of defense, but…
Coverage in-cycle helps teams accelerate because it’s faster to fix a bug when coding/building than wait for out-of-cycle testing
Velocity
Timelines are shrinking: 3-6 months down to 1-3 weeks
Usually what gets the squeeze is lengthier out-of-cycle testing, and fixing a bug found much later cost 7-10 times
WHY SHOULD WE CARE:
If we don’t start talking to development, we run the risk of being deemed irrelevant
CI happens continuously; the more CI cycles happen that include automated testing on real devices, the more USAGE
The more coverage they work into the cycle, the more they can shrink the out-of-cycle test phase
Thought leaders (Salesforce, Facebook) literally have zero out-of-cycle testing; not everyone can, but striving to cover more in-cycle has ROI
Desired State / Specific Challenges
Accelerate Time-to-Market
They’ve built this whole thing to move faster and it needs to always work
This thing is constantly on, and an unstable lab breaks the whole thing; can’t maintain this environment ourselves
Once we find an issue, we have limited information to troubleshoot; gather information, diagnose, resolve
To resolve some bugs, developers need access to latest devices => coverage
Minimize Risk and Cost
We must move faster, but if have escaped defects, we’re the ones held to it
Coverage is the biggest driver here
(all the normal value props)
Discovery Questions
This is just a subset, but it gets you into a broader conversation about
what’s driving their demand for devices, and
Who’s the real Economic Buyer (QA or Dev or higher)
Makes it easy to bring additional people into the conversation (i.e. from just QA to dev)
Be sensitive to who you’re asking: “How much more…” is a question for dev/CI
There will be many more questions that map to sizing, etc. to come shortly after QBR