Expensify, Paysafe and PBS explain how they approach digital quality and how Applause helps them ensure excellent digital experiences.

1. Delivering Excellent
Digital Quality to Your
Customers
Dan Flannery
Associate Director of Digital Content, PBS Kids
Tim Golen
Bernhard Niebsch
Director (Engineering), Expensify
Head of Product, Paysafe

2. BEFORE WE GET
STARTED…
The webinar will last approximately 45 minutes to
1 hour (including Q&A)
You can send your questions using the
“Questions” function on GoToWebinar, we will ask
them after the session.
We will send you a link to the recording as well as
this slide deck
If you tweet during the webinar, feel free to
mention @Applause, @expensify,
@PlugIntoPaysafe and @PBSKIDS

3. Presenters of This Session
3
Daniel Flannery
Associate Director of
Digital Content
Bernhard Niebsch
Head of
Product
Tim Golen
Director
(Engineering)
Tom Gleich
VP Digital Quality,
Crowd Solutions

4. Today’s Agenda
4
1
3
2
4
Defining Digital Quality
Paysafe’s Payment UX
Expensify’s Company Culture
PBS Kids’ Mission Impacts QA

5. ACE AWARDS

6. 6
Excellence in Digital
Workforce Experiences
Excellence in Payment
Experiences
Excellence in Kids
Content & Learning

7. WHAT DOES DIGITAL QUALITY
MEAN TO YOU?

8. What Does Digital Quality Mean To You?
8
How do you quantify
quality?
What is the company
culture around quality?
How do you ensure
quality alongside
speed?

9. Send your questions using the “Questions” function on
GoToWebinar

10. Company Culture
10
Excellent digital quality isn’t about what Expensify does –
it’s about who Expensify is
Company values:
• Hire the best talent at a high cost that rewards people on merit
• Embrace a flat organization
• Generalists with deep and wide knowledge and skills
• Pursue the highest quality and simplest solution
But how does this impact your strategy and
team-building for development and QA?

11. Shifting to new features
11
Expensify’s culture enables it to:
• Have an excellent product that is first and foremost an internal tool and a public
expression of our own internal best practices
• Pivot insanely fast to new ideas without needing to hire new people or increase
head counts
For example …
• Expensify’s latest features include bill pay and invoicing
• At Expensify, the full-stack engineers mean we can redistribute our team very
quickly

12. How Applause and Expensify Team Up
12
Test Cases in Real-
World Conditions
*From May throughAugust 2020
Applause’s regression testing across a device/OS matrix enables Expensify to maintain its daily release cadence with
confidence and protect users from unexpected issues that arise from moving quickly
2,281 560
Testers Leveraging
Own Devices
216
Real-World Device
Configurations
55
CountriesWhere
Testing is
Conducted

13. Send your questions using the “Questions” function on
GoToWebinar

14. Payment processing: How we create value.
Issuing Acquiring
CONSUMER MERCHANT
Transaction
Processing Challenge: Find people
with local payment
options to test across so
many payment options
and use cases, and
validate the quality of
the payment rails

15. Paysafe Needs Confidence in Payments
15
Can customers in new
markets successfully use
a payment method in
that market?
Can customers
successfully get
refunds/credit for their
purchased product?
Can customers add
and use a new
payment method?
Can customers
successfully transfer
money with my app?
Will settlements and
refunds work properly in
each market?
Does a delayed payment
method process correctly?

16. How Applause and Paysafe Team Up
16
Test Cases in Real-
World Conditions
Paysafe has a dedicated testing manager who helps QA stay organized. As Paysafe underwent a significant backend
migration,Applause retested and validated about 100 production transactions to ensure bugs didn’t reach the
customer
450 208
Real-World Device
Configurations
50
CountriesWhere
Testing is
Conducted
>300
High-Value Bugs
Sent to Paysafe

17. Send your questions using the “Questions” function on
GoToWebinar

18. PBS Kids creates mobile games based on nearly every PBS program
Games release on a weekly basis, all requiring QA
Playfulness AND curriculum both need to be considered with PBS Kids games
PBS Kids: Quality Games for Kids
18

19. Fine-tuning the QA Process
19
• More cycles
• 1 extra cycle = 4 more games per week.
• Improve the quality of reported bugs.
• Define what is out of scope and what is by design before QA
• Only reproducible bugs allowed. If it’s not reproducible, it’s not fixable.
• Find efficiencies
• Identify cross-platform issues in QA reports (web, iOS, Android)
• Rethink what ‘Out of Scope’ means. Is it valuable to a different team?
• Keep screen recordings of bugs short.Those seconds add up!We cap all videos at 30
seconds.

20. How Applause and PBS Team Up
20
Test Cases in Real-
World Conditions
PBS Kids currently leverages three slots for functional testing, ensuring multiple games get tested per week
372 582
Testers Leveraging
Own Devices
336
Real-World Device
Configurations
3,700
Bugs Deemed
HighValue by PBS
Kids
*From January through October 2020

21. Q&ARemember: you can send your questions using the
“questions” function on gotowebinar