1. COMPANY NAME www.nicecompany.com
Bestpractices for buildinga
testingculture
Presented by : Mari Ju and John Oppenheimer
Optimizely User Group - August 2015

2. Product Marketing Manager
MariJu
About US
mari.ju@autodesk.com
linkedin.com/in/mariju
@mari4ju
Marketing Operations Analyst
John oppenheimer
john.oppenheimer@autodesk.com
linkedin.com/in/oppenheimerjohn

3. about Autodesk
Mission: Help people imagine,
design, and create a better world
Est. 1982
12+ million users
autodesk
Mission: Unlock the creativity in
everyone
Est. 2011
170+ million users
Autodesk consumerand3dPrinting

4. Testing atAutodesk
Spring ‘14 Fall ‘14 Spring ‘15 Fall ‘15
Start
small
Build
trust
GENERATE
Excitement
Advance &
REPEAT

5. Training
• Onsite demo
• Team sessions
• Dedicated champions
Pick a starter property
• Simple first test as proof of
concept
Phase 1: StartSmall
vs.
Original Variation
Results: Variation showed 24% more
subscriptions compared to the Original

6. Address potential concerns:
• Test proposals to larger team
• Allow for plenty of QA
• Connect developers with Optimizely
support team
• Have developers launch test
• One test at a time, simple tests only
Phase 2: Buildtrust

7. Spread testing throughout the organization:
• Add more products/domains
• Get more team members involved
Improve testing program:
• Develop testing workflow:
Data à User Testing à Hypothesis à A/B Test
• Advanced tests (multi-page, redirects, cross-platform)
that leverage integrations
• Establish testing as essential when updating the site
Share results (both wins and losses):
• Distribute reports via email and internal blog
• Celebrate team wins (Optie finalist)
PhaSE 3: Generate excitement

8. Continuously advance our testing program:
• Standardized idea backlog and prioritization rubric
• Create an archive of previous tests that’s easy to
share and search
• Work closely with designers and developers to
launch more technical tests
• Start experimenting with personalization
• Launch mobile app A/B testing program
Evangelize benefits of testing:
• Internally – other products, departments, office
locations, and more
• Externally – with partners and user groups
Phase 4: advance
Original Variation

9. 1
2
3
4
Demonstrate potential
impact of testing with a
simple test
1. Start small
Increase complexity and
frequency of tests for greater
impact, and share results
3. Generate excitement
Bring all stakeholders into
the process to gain trust,
and eventually autonomy
2.Buildtrust
Continue to evaluate and
improve testing program
4. ADVANCE
Steps for building atesting culture

10. COMPANY NAME www.nicecompany.com
Questions?
Mari Ju – mari.ju@autodesk.com
John Oppenheimer – john.oppenheimer@autodesk.com