SlideShare a Scribd company logo
[Slide: Blue Screen Of Death, Fitbit error, BBC 500 error]
Let's start with a quick survey. How many of you have had a program crash on your laptop? Who's had
an app crash on their phone? Now put your hand up if you had an app crash and then you tried to make
it crash again. You may be QA people.
We go by a lot of names. Quality Assurance, Quality Engineer, Tester. At Asynchrony, we're called
Quality Advocates. Quality is everyone's job on the team, and we're the point person. We're
the conscience, we're the scout, we're the bridge between dev-speak and customer, we're the most evil
hacker your software will probably encounter this side of a public release, and we're the dumbest new
user seeing it for the first time. Good QA people are the best ally a developer has, we make them look
good and they rely on us to put their work through the wringer. We don't do it because we're mean, or
to prove they did it wrong, we do it to make the app better.
[Slide: QA walks into a bar]
For instance, one of my teams routinely puts code in their web app to prevent the user from pasting
large chunks of text in. Why? Because if you get the plain text public domain edition of Tolstoy's "War
and Peace", you've got yourself three megabytes of text, which is about five hundred thousand words.
The first time I dumped that into a field on a web app and hit submit, the web service call rejected it and
the app got stuck. So they fixed that and I tried it again, and this time the database rejected it because it
couldn’t fit that much text into the field it was aiming at. So they fixed it again. Then I put War and
Peace directly into the database and had the app try and pull it out, which was another set of bugs.
So what do Agile QAs bring to a project? Let me show you three different scenarios. The first project ran
with no QA at all for the first 12 weeks. Then they got QA, and the five developers spent two weeks on
nothing but fixing the bugs, and at the end of the project, we got this:
[Slide: Case Tracker story type graph]
This is a chart showing what kind of things we worked on, and nearly a third of our time was spent fixing
bugs. User stories are the features, tech debt is stuff you have to fix but it doesn't provide a new feature
to the user, technical stories are things you have to do that aren't related to a user story, like fixing
memory leaks or database queries, design is wiring in the new mock-up from the UX team.
We defined a defect/bug as anything that was in the main branch of the code. The main branch is
supposed to be something you can release at any point, you never put bad code into the main branch.
So this was not good. Really not good. And it’s not that this team was full of sloppy coders, these are
smart people who write good code. But developers aren’t good at testing their own stuff. No-one wants
to kick their own baby. That’s what we’re there for.
The next project is one that's had a QA for the whole lifetime of the project, and we’re still adding new
features
[Slide: TMS story type graph]
So this is better, the bug rate is down to about 10%, so the team can spend more time working on
features. Ten percent of the time, developers were fixing bugs instead of writing new features. This is a
piece of software that's been released to the client several times, it's been in the field for well over a
year. That's a much better graph, but we can do better than that.
[Slide: Combinator story type graph]
This is from a project where QA was on the team from day one, and I was writing automated user
acceptance tests in the first week, using C# and NUnit inside Visual Studio. This app was installed at the
client site, and in a year of use by a company of over 2000 people, it's not had a single bug reported.
We're pretty proud of this one. We repeated this process on the next two projects with similar stats and
it's a model that's spreading. What it requires is a QA that can write code .
A good QA who can read and write code is an invaluable person on a team. Imagine you're testing a car,
but the hood has been welded shut. You can drive the car, you can see the dials, you can mess with the
wheels, but if you could only lift that hood, there's a whole world of testing opportunities that you've
never seen before. As a QA, if you can code, you understand how the app is put together, what it
depends on, and you know where to shove a screwdriver into the works to cause an error. You know
where the weak spots are. You also understand it when your team starts talking about something and
they haven't taken into account problems like internationalisation. Or accessibility. Or security,
encryption, network load, performance, database access on a shared cluster, any one of a hundred
things.
So what’s the difference between a developer and a coding QA? Not a huge amount, but the biggest
difference is we’re much more rare. Asynchrony has three, and those three are training another six, and
five people are teaching themselves using Code School and books. That's fourteen people out of a group
of twenty four. I think QA has the best job in the house. Developers are nice people who want to build
things that work. QA gets to play, and burn things down, and make a mess, and we don’t even have to
fix the problems, just find them and fight for the ones that will cause the users the most pain.
Who saw the movie Tron? The original one and Tron Legacy both had the same line: “I fight for the
Users!” That’s what QA does. We work with the users in mind. Smart ones, not so smart ones, malicious
ones, ones who speak other languages, ones who are colour-blind, ones who can’t use a keyboard, ones
who can’t use a mouse, partially-sighted ones, chronically busy ones, we fight for all the users.
What do we do on an Agile team?
We start working on day one of the first iteration, even before a line of code has been written. That’s
when we start thinking about how we’re going to test the application, how the team will work together,
what the definition of a “bug” is for this project. We might do some investigation and learn a new tool
we’ll need to know, or pull together some requirements for what we’ll need from the developers. One
thing my team lead builds for me is a data loader to put several thousand records into the database so I
can test under more realistic production conditions.
And it’s where we start thinking about roles. Who plays D&D? You should recognize this:
[Slide: D&D alignment chart]
We cycle through these roles all the time. Up in the corner, the Lawful Good square represents all the
“happy path” tests. These are the tests that the application does what it’s supposed to under ideal
conditions. Developers spend most of their time thinking about the Lawful Good users.
Chaotic Neutral is a fun role to play. There is a tool for Android developers called the Monkey. Monkey
will click everything clickable in the user interface, and in the Android system, as fast as it can, over and
over until something breaks or you get to the end of the test. Monkey has no rhyme or reason, it’s
totally random.
For Lawful Evil, you’re staying within the bounds of the application, but playing everything as a worst
case scenario. They give you an address field? You paste a photo of a kitten into the zip code field.
You’re testing a mobile app? You step into a Faraday Cage which cuts off all network signal, or you call
the test phone from your cell phone.
Chaotic Evil is where you really get to play. Anything goes here. Go into the database and put in
deliberately bad data, pull the network cable out of the test machine, poke at the API with a sharp stick
and see what squeaks. Change your system language, run something to make the network really slow
and unreliable, intercept the network traffic and tweak it to introduce errors, run a piece of JavaScript
that hits the submit button seven times in a row as fast as possible and see what gets through to the
database.
There are no limits but time and your imagination. A good QA has this sense of diabolical creative glee at
putting an app through the wringer. When we’re done with it, it will be stronger and ready to face the
users.
[Slide: Little Bobby Tables]
This is what's called a SQL Injection attack. If you don't sanitize your inputs by treating them as a block of
text and stripping out stuff that would look to the database like a command to run, bad things can
happen. This is from a web cartoon called xkcd, which doesn't stand for anything, but has a lot of geek
humour. You can also inject JavaScript to mess with a web app, or PHP script.
What makes a good QA?
QA is the one job where you are paid to tell the truth. You never sugar-coat it, you never tell it like it
isn’t, and your team is depending on you to be honest with them. How many jobs can you say that
about? We would never do this:
[Slide: Dilbert performance test failure]
The most important thing that a good QA does is ask questions. Lots of questions, even when we know
the answer. Especially when you know the answer will be “I have no idea”, or “I didn’t think of that” or
“Oh crap!” Those are the good questions. Sometimes we play dumb to get our team to explain things to
us. When you’re breaking things down to tell someone, you get it straight in your own head, and
sometimes that process shakes loose something you forgot to include, or a flaw in your logic, or a
requirement that was never added but should have been there, like the ability to print something in a
particular format. We ask a lot of questions, and we listen to everything.
[Slide: Stuff QA says]
If we don’t ask, no-one else will. We fight for the users, and the client, and our teams. Our QA
apprentices have two jobs: Listen, and ask questions about everything you don’t understand. There are
no dumb questions, because the right dumb question will make your team think about things
differently. We just don’t know which question is the right one is until we’ve asked all of them.
QAs are detail-oriented people. We see the painting that’s crooked on the wall, the typo in the help
files, the colour that's one shade off the rest of the screen system. We are creative in finding every
possible way to stress a system, bust a database, and blow out a web request. It’s a good day when we
find a bug, or a glitch, or a missed acceptance criteria. Those things are like buried treasure, and the
more effort it takes to track down a bug, the bigger the treasure. QAs are fearless. We’ll switch the
language on our systems to Mandarin to see how the app responds to switching locale. We’ll delete the
database out from under a running application. We’ll put something in the app that blows up the web
service, never makes it to the database, and overloads the server error-reporting tool. That was a good
day.
The tools in our arsenal are many and varied. One of my favorites is War and Peace. Another is a tool
which can throw thousands of HTTP requests at a server to simulate a crowd of users hitting the app all
at once. There’s a tool that lets you spy on the network traffic going in and out of your machine. There
are tools that can make your network connection slow and unreliable. Firebug is a tool for the Firefox
browser that lets you inject JavaScript into a page, check on variables, and see the inner workings of a
site. And then there’s these:
[Slide: Foreign text]
From top to bottom, that’s Arabic at the top, which is a non-Roman alphabet that you read from right to
left, not left to right. There aren’t any characters in common with English. Next up is Cyrillic, which is
used in Russia. Some letters are the same, some are very different. And finally we have Greek, a third set
of unique characters. If you’re testing an application that will be used in a non-English speaking country,
you need to test in those languages. Right-to-left text looks odd when it’s on a left-to-right layout. The
German work for “Cancel” is “Stornieren”, and that’s not going to fit on a button designed for the word
Cancel.
A good QA is always learning. If you always test in a certain way, you’ll find a certain set of bugs. And
when those bugs are fixed, you won’t find others until you change up your process. One way to do that
is to call in another QA and hand them our app. No explanations, just hand it over and watch what they
do with it. Every QA is different. We all see different things. For example, can anyone see what’s wrong
with this app?
[Slide: LDI app]
We found a glitch in the layout that only affected Motorola devices running Android 4.1.2. Didn’t see
this on the Android emulator, didn’t see it on the HTC phones, or the Samsung ones.
[Slide: LDI app with red circles]
Right under the C of account, a piece of the header is shifted down by one pixel. Not all the way across,
it bumps back up further along. “Please move it three pixels to the left” is a valid thing for QA to ask for,
if those three pixels make the app look more polished, more professional.
Asynchrony QA apprentices
I mentioned QA apprentices earlier. In April of 2014 we hired someone with no software background at
all. He worked side by side with me every day for three months and worked hard. He was the sole QA on
that team for about 9 months before it merged with another team. Right now, I’d rate him as one of our
better QAs and he’s still learning. He’s also teaching two other QAs that have joined that team.
When we hire an apprentice, they are assigned to a senior QA as their mentor. They work side by side,
the senior explains their process, and the apprentice listens and asks questions. When we hire an
apprentice we’re looking for people with a strong desire to learn, someone with curiosity about how
things work and how to break them, someone that can think on their feet, someone that’s not afraid to
ask for help and ask for people to explain things until they get it. We look for people that want to be part
of a team. At Asynchrony, the team succeeds together, and fails together.
We discovered that having an apprentice around was good for the team, and good for the person
mentoring them. When you have to explain why you do everything you normally do by reflex, it makes
you pause, review how you work, and spot the gaps in your process. The job of an apprentice is to listen
and ask questions. The job of the mentor is to answer all the questions, and teach their apprentice how
to be a good QA.
Asynchrony is currently training our fourth QA apprentice. She’s on an iOS team with one of our coding
QAs learning how to test an iPad app that lives in an ecosystem of connected applications.
Wrap up
I hope I've given you some idea of the life of a QA in a software company. As a QA, you develop a
superpower you can't turn off. There are typos everywhere. Software may just break around you. And
this stuff is funny to us. We giggle when we see the mall display screen that's showing an error, because
someone didn't test it fully. We critique every app on our phones and try to break things just for fun. We
see broken software, it's everywhere. And somewhere behind the software you use every day there was
a QA.
[Slide: I see broken software]

More Related Content

Recently uploaded

Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptxMAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
janagijoythi
 
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and CitiesThe Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
Arpan Buwa
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
Matthias Neugebauer
 
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
bellared2
 
Camunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptxCamunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptx
ZachWylie3
 
Zaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdfZaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdf
AmandaCheung15
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
DianaGray10
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
SynapseIndia
 
It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...
Zilliz
 
Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024
siddu769252
 
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
FIDO Alliance
 
Improving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning ContentImproving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning Content
Enterprise Knowledge
 
Step-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From ScratchStep-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From Scratch
softsuave
 
Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
Bhajan Mehta
 
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
FIDO Alliance
 
Types of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technologyTypes of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technology
ldtexsolbl
 
Mastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for SuccessMastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for Success
David Wilson
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
sunilverma7884
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
Google Developer Group - Harare
 

Recently uploaded (20)

Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptxMAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
 
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and CitiesThe Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
 
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
 
Camunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptxCamunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptx
 
Zaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdfZaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdf
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
 
It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...It's your unstructured data: How to get your GenAI app to production (and spe...
It's your unstructured data: How to get your GenAI app to production (and spe...
 
Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024
 
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
 
Improving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning ContentImproving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning Content
 
Step-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From ScratchStep-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From Scratch
 
Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
 
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
UX Webinar Series: Essentials for Adopting Passkeys as the Foundation of your...
 
Types of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technologyTypes of Weaving loom machine & it's technology
Types of Weaving loom machine & it's technology
 
Mastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for SuccessMastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for Success
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
 

Featured

2024 Trend Updates: What Really Works In SEO & Content Marketing
2024 Trend Updates: What Really Works In SEO & Content Marketing2024 Trend Updates: What Really Works In SEO & Content Marketing
2024 Trend Updates: What Really Works In SEO & Content Marketing
Search Engine Journal
 
Storytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design ProcessStorytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design Process
Chiara Aliotta
 
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
OECD Directorate for Financial and Enterprise Affairs
 
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
SocialHRCamp
 
2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
Marius Sescu
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
Expeed Software
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
Pixeldarts
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
marketingartwork
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
Skeleton Technologies
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
Neil Kimberley
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
contently
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
Albert Qian
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
SpeakerHub
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
Tessa Mero
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
Rajiv Jayarajah, MAppComm, ACC
 

Featured (20)

2024 Trend Updates: What Really Works In SEO & Content Marketing
2024 Trend Updates: What Really Works In SEO & Content Marketing2024 Trend Updates: What Really Works In SEO & Content Marketing
2024 Trend Updates: What Really Works In SEO & Content Marketing
 
Storytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design ProcessStorytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design Process
 
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
 
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
 
2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 

LaunchCoderGirl transcript, 5/20/2015 talk on Agile QA

  • 1. [Slide: Blue Screen Of Death, Fitbit error, BBC 500 error] Let's start with a quick survey. How many of you have had a program crash on your laptop? Who's had an app crash on their phone? Now put your hand up if you had an app crash and then you tried to make it crash again. You may be QA people. We go by a lot of names. Quality Assurance, Quality Engineer, Tester. At Asynchrony, we're called Quality Advocates. Quality is everyone's job on the team, and we're the point person. We're the conscience, we're the scout, we're the bridge between dev-speak and customer, we're the most evil hacker your software will probably encounter this side of a public release, and we're the dumbest new user seeing it for the first time. Good QA people are the best ally a developer has, we make them look good and they rely on us to put their work through the wringer. We don't do it because we're mean, or to prove they did it wrong, we do it to make the app better. [Slide: QA walks into a bar] For instance, one of my teams routinely puts code in their web app to prevent the user from pasting large chunks of text in. Why? Because if you get the plain text public domain edition of Tolstoy's "War and Peace", you've got yourself three megabytes of text, which is about five hundred thousand words. The first time I dumped that into a field on a web app and hit submit, the web service call rejected it and the app got stuck. So they fixed that and I tried it again, and this time the database rejected it because it couldn’t fit that much text into the field it was aiming at. So they fixed it again. Then I put War and Peace directly into the database and had the app try and pull it out, which was another set of bugs. So what do Agile QAs bring to a project? Let me show you three different scenarios. The first project ran with no QA at all for the first 12 weeks. Then they got QA, and the five developers spent two weeks on nothing but fixing the bugs, and at the end of the project, we got this: [Slide: Case Tracker story type graph] This is a chart showing what kind of things we worked on, and nearly a third of our time was spent fixing bugs. User stories are the features, tech debt is stuff you have to fix but it doesn't provide a new feature to the user, technical stories are things you have to do that aren't related to a user story, like fixing memory leaks or database queries, design is wiring in the new mock-up from the UX team. We defined a defect/bug as anything that was in the main branch of the code. The main branch is supposed to be something you can release at any point, you never put bad code into the main branch. So this was not good. Really not good. And it’s not that this team was full of sloppy coders, these are smart people who write good code. But developers aren’t good at testing their own stuff. No-one wants to kick their own baby. That’s what we’re there for. The next project is one that's had a QA for the whole lifetime of the project, and we’re still adding new features [Slide: TMS story type graph]
  • 2. So this is better, the bug rate is down to about 10%, so the team can spend more time working on features. Ten percent of the time, developers were fixing bugs instead of writing new features. This is a piece of software that's been released to the client several times, it's been in the field for well over a year. That's a much better graph, but we can do better than that. [Slide: Combinator story type graph] This is from a project where QA was on the team from day one, and I was writing automated user acceptance tests in the first week, using C# and NUnit inside Visual Studio. This app was installed at the client site, and in a year of use by a company of over 2000 people, it's not had a single bug reported. We're pretty proud of this one. We repeated this process on the next two projects with similar stats and it's a model that's spreading. What it requires is a QA that can write code . A good QA who can read and write code is an invaluable person on a team. Imagine you're testing a car, but the hood has been welded shut. You can drive the car, you can see the dials, you can mess with the wheels, but if you could only lift that hood, there's a whole world of testing opportunities that you've never seen before. As a QA, if you can code, you understand how the app is put together, what it depends on, and you know where to shove a screwdriver into the works to cause an error. You know where the weak spots are. You also understand it when your team starts talking about something and they haven't taken into account problems like internationalisation. Or accessibility. Or security, encryption, network load, performance, database access on a shared cluster, any one of a hundred things. So what’s the difference between a developer and a coding QA? Not a huge amount, but the biggest difference is we’re much more rare. Asynchrony has three, and those three are training another six, and five people are teaching themselves using Code School and books. That's fourteen people out of a group of twenty four. I think QA has the best job in the house. Developers are nice people who want to build things that work. QA gets to play, and burn things down, and make a mess, and we don’t even have to fix the problems, just find them and fight for the ones that will cause the users the most pain. Who saw the movie Tron? The original one and Tron Legacy both had the same line: “I fight for the Users!” That’s what QA does. We work with the users in mind. Smart ones, not so smart ones, malicious ones, ones who speak other languages, ones who are colour-blind, ones who can’t use a keyboard, ones who can’t use a mouse, partially-sighted ones, chronically busy ones, we fight for all the users. What do we do on an Agile team? We start working on day one of the first iteration, even before a line of code has been written. That’s when we start thinking about how we’re going to test the application, how the team will work together, what the definition of a “bug” is for this project. We might do some investigation and learn a new tool we’ll need to know, or pull together some requirements for what we’ll need from the developers. One thing my team lead builds for me is a data loader to put several thousand records into the database so I can test under more realistic production conditions. And it’s where we start thinking about roles. Who plays D&D? You should recognize this:
  • 3. [Slide: D&D alignment chart] We cycle through these roles all the time. Up in the corner, the Lawful Good square represents all the “happy path” tests. These are the tests that the application does what it’s supposed to under ideal conditions. Developers spend most of their time thinking about the Lawful Good users. Chaotic Neutral is a fun role to play. There is a tool for Android developers called the Monkey. Monkey will click everything clickable in the user interface, and in the Android system, as fast as it can, over and over until something breaks or you get to the end of the test. Monkey has no rhyme or reason, it’s totally random. For Lawful Evil, you’re staying within the bounds of the application, but playing everything as a worst case scenario. They give you an address field? You paste a photo of a kitten into the zip code field. You’re testing a mobile app? You step into a Faraday Cage which cuts off all network signal, or you call the test phone from your cell phone. Chaotic Evil is where you really get to play. Anything goes here. Go into the database and put in deliberately bad data, pull the network cable out of the test machine, poke at the API with a sharp stick and see what squeaks. Change your system language, run something to make the network really slow and unreliable, intercept the network traffic and tweak it to introduce errors, run a piece of JavaScript that hits the submit button seven times in a row as fast as possible and see what gets through to the database. There are no limits but time and your imagination. A good QA has this sense of diabolical creative glee at putting an app through the wringer. When we’re done with it, it will be stronger and ready to face the users. [Slide: Little Bobby Tables] This is what's called a SQL Injection attack. If you don't sanitize your inputs by treating them as a block of text and stripping out stuff that would look to the database like a command to run, bad things can happen. This is from a web cartoon called xkcd, which doesn't stand for anything, but has a lot of geek humour. You can also inject JavaScript to mess with a web app, or PHP script. What makes a good QA? QA is the one job where you are paid to tell the truth. You never sugar-coat it, you never tell it like it isn’t, and your team is depending on you to be honest with them. How many jobs can you say that about? We would never do this: [Slide: Dilbert performance test failure] The most important thing that a good QA does is ask questions. Lots of questions, even when we know the answer. Especially when you know the answer will be “I have no idea”, or “I didn’t think of that” or “Oh crap!” Those are the good questions. Sometimes we play dumb to get our team to explain things to us. When you’re breaking things down to tell someone, you get it straight in your own head, and sometimes that process shakes loose something you forgot to include, or a flaw in your logic, or a
  • 4. requirement that was never added but should have been there, like the ability to print something in a particular format. We ask a lot of questions, and we listen to everything. [Slide: Stuff QA says] If we don’t ask, no-one else will. We fight for the users, and the client, and our teams. Our QA apprentices have two jobs: Listen, and ask questions about everything you don’t understand. There are no dumb questions, because the right dumb question will make your team think about things differently. We just don’t know which question is the right one is until we’ve asked all of them. QAs are detail-oriented people. We see the painting that’s crooked on the wall, the typo in the help files, the colour that's one shade off the rest of the screen system. We are creative in finding every possible way to stress a system, bust a database, and blow out a web request. It’s a good day when we find a bug, or a glitch, or a missed acceptance criteria. Those things are like buried treasure, and the more effort it takes to track down a bug, the bigger the treasure. QAs are fearless. We’ll switch the language on our systems to Mandarin to see how the app responds to switching locale. We’ll delete the database out from under a running application. We’ll put something in the app that blows up the web service, never makes it to the database, and overloads the server error-reporting tool. That was a good day. The tools in our arsenal are many and varied. One of my favorites is War and Peace. Another is a tool which can throw thousands of HTTP requests at a server to simulate a crowd of users hitting the app all at once. There’s a tool that lets you spy on the network traffic going in and out of your machine. There are tools that can make your network connection slow and unreliable. Firebug is a tool for the Firefox browser that lets you inject JavaScript into a page, check on variables, and see the inner workings of a site. And then there’s these: [Slide: Foreign text] From top to bottom, that’s Arabic at the top, which is a non-Roman alphabet that you read from right to left, not left to right. There aren’t any characters in common with English. Next up is Cyrillic, which is used in Russia. Some letters are the same, some are very different. And finally we have Greek, a third set of unique characters. If you’re testing an application that will be used in a non-English speaking country, you need to test in those languages. Right-to-left text looks odd when it’s on a left-to-right layout. The German work for “Cancel” is “Stornieren”, and that’s not going to fit on a button designed for the word Cancel. A good QA is always learning. If you always test in a certain way, you’ll find a certain set of bugs. And when those bugs are fixed, you won’t find others until you change up your process. One way to do that is to call in another QA and hand them our app. No explanations, just hand it over and watch what they do with it. Every QA is different. We all see different things. For example, can anyone see what’s wrong with this app? [Slide: LDI app]
  • 5. We found a glitch in the layout that only affected Motorola devices running Android 4.1.2. Didn’t see this on the Android emulator, didn’t see it on the HTC phones, or the Samsung ones. [Slide: LDI app with red circles] Right under the C of account, a piece of the header is shifted down by one pixel. Not all the way across, it bumps back up further along. “Please move it three pixels to the left” is a valid thing for QA to ask for, if those three pixels make the app look more polished, more professional. Asynchrony QA apprentices I mentioned QA apprentices earlier. In April of 2014 we hired someone with no software background at all. He worked side by side with me every day for three months and worked hard. He was the sole QA on that team for about 9 months before it merged with another team. Right now, I’d rate him as one of our better QAs and he’s still learning. He’s also teaching two other QAs that have joined that team. When we hire an apprentice, they are assigned to a senior QA as their mentor. They work side by side, the senior explains their process, and the apprentice listens and asks questions. When we hire an apprentice we’re looking for people with a strong desire to learn, someone with curiosity about how things work and how to break them, someone that can think on their feet, someone that’s not afraid to ask for help and ask for people to explain things until they get it. We look for people that want to be part of a team. At Asynchrony, the team succeeds together, and fails together. We discovered that having an apprentice around was good for the team, and good for the person mentoring them. When you have to explain why you do everything you normally do by reflex, it makes you pause, review how you work, and spot the gaps in your process. The job of an apprentice is to listen and ask questions. The job of the mentor is to answer all the questions, and teach their apprentice how to be a good QA. Asynchrony is currently training our fourth QA apprentice. She’s on an iOS team with one of our coding QAs learning how to test an iPad app that lives in an ecosystem of connected applications. Wrap up I hope I've given you some idea of the life of a QA in a software company. As a QA, you develop a superpower you can't turn off. There are typos everywhere. Software may just break around you. And this stuff is funny to us. We giggle when we see the mall display screen that's showing an error, because someone didn't test it fully. We critique every app on our phones and try to break things just for fun. We see broken software, it's everywhere. And somewhere behind the software you use every day there was a QA. [Slide: I see broken software]