By harnessing insights from experimentation, people across your organization can contribute ideas and decisions that take the customer experience to new levels. To take advantage of this, forward-thinking organizations are getting everyone involved in experimentation. These slides will share how General Assembly is cultivating a culture of experimentation and the impact it’s making company-wide.

1. Speakers
Cultivating a Culture of
Experimentation
Bryan Berger, Product Design Lead, General Assembly
Taylor Gilbert, Partnerships & Alliances, Optimizely

2. Today’s Speakers
Bryan
Berger
Taylor
Gilbert

3. Housekeeping
• We’re recording!
• Slides and recording will be
emailed to you after the
webinar is complete
• There will be time for
questions at the end

4. Agenda
• ‘Culture’ of Experimentation
• General Assembly’s Approach
• How to Energize Your Team
• Some Things We Tested

5. ‘Culture’
What does that
mean?

8. Turns out… culture is a
big deal.

9. of Fortune 500 companies from
1955 are still in business today12%
¼ companies fell off the list in
the last year alone

10. The Organization of the Future…. Arriving Now

11. 1. Culture has never been more important,
experimentation is the vehicle.
2. Your executives are struggling and they need your
help. They are looking for someone to step up and
lead/orchestrate.

12. General Assembly (GA) is a global
educational company. Focusing on
the most relevant and in demand
skills across data, design, business
and technology.
General Assembly is empowering
a global community to pursue
work they love through
best-in-class instruction and
access to opportunities.

13. Experimentation
at
General Assembly

14. A Little About Me
● I manage our Product Design Team, whose focus is on designing an
ecosystem to support online and in-person learning experiences.
● I also work quite extensively with our Marketing and Website CRO team to
improve the positioning of our Products.
● One of my many goals is to foster a fun and impactful culture of
experimentation deeply rooted in the design process.

15. Why Experiment?
● Progress over perfection. Experimentation allows our team to ideate multiple
solutions to a problem and not paralyze the process in favor of “the one”.
● Expand potential. Anyone with a hypothesis can partake. Inclusion has become
an ally in bringing down silos among our product teams.
● Gain insights. To develop a better understanding of our data, and how we can
affect the levers of improvement.
● Build consensus. Save time and money by testing your assumptions.

17. Experimentation at General Assembly
● Started to hit the limits of our current solutions.
● Our business was scaling quickly and along with it, our growth targets.
● We needed to start questioning things that have been “established”.
● Our testing program has been running for about a year.
● Prior to that we had no repeatable testing process in place.

18. How to Energize
Your Team

19. Make Testing Tangible

20. Get Creative

21. Really Creative...

22. Make It Known

23. Highlight and Acknowledge Success

24. Our Approach

25. Getting Started: Finding a Process
Where we all start
Where some give up
The goal, a smoother repeatable process

26. Our Initial Process

27. Our Current Process

28. Finding Great Ideas
● All ideas must contain:
○ Problem definition
○ Hypothesis
○ Audience Type (Persona)
○ Page url(s) affected
● All ideas are added to a backlog list
○ Tickets are created or updated once ideas are vetted for impact, time to significance, and
resource allocation.
● Hold weekly A/B testing meeting to:
○ Discuss how a specific test would be implemented
○ Review cost / benefits for different possible solutions
○ Amount of traffic needed
○ Length of the experiment
○ Next steps post test

29. Prioritizing Test Ideas
Website Tests are prioritized by our Marketing Team & Director of SEO/CRO
● We define quarterly Objectives & Key Results (OKRs) and roll up each test result toward one of
those objectives
● Test key routes (purchase flow, wayfinding, and value prop improvement)
● As of today, all tests aim to improve conversion and engagement at key steps of our funnel

30. Stay Organized

31. Share Results
Clear communication, acknowledgement,
and collaboration.
● #CRO Slack channel for asking questions, and
posting results.
● Announces test results and shoutouts.
● Big wins are also sent to an email group.
● Slack is effective. Everyone is engaged in it,
from engineering to execs.

32. Program Goals
Our current benchmark is to have at least one live test on each of the most critical
pages in our conversion funnel (to keep momentum):
○ New user signup modal
○ Homepage
○ Navigation
○ Product pages
○ Wayfinding systems
○ Discovery & catalog pages

33. Some Things We
Tested

34. 25% Off Class & Workshop Offer
Modal Incentive Copy Test
Original Variation

35. Drive more urgency to the next course start date
Course Start Date
Original Variation

36. Copy and styling tests in Navigation menu
Copy and Style in Navigation
Original Variation

37. Show the customer potential ROI of our courses
No original variation
Adding Sections
Original Variation

38. Looking Forward
● As we scale our experimentation program we’ll likely bring in an agency to
help build, design, & deploy tests in Optimizely.
● With their support, we’ll be able to move more quickly on our backlog of 80+
test ideas.
● Continue testing to inform design direction for our new site.
● Penetration into our Product Apps and onboarding flows.

39. Key Takeaways
● Become the champion of your experimentation program
● Make your testing program known, give it a physical presence
● Start backlogging and organizing ideas
● Establish a process to vet, prioritize and execute those ideas
● Set up a staging environment to act as your laboratory
● Hold yourselves accountable for stagnation
● Beware of vanity metrics.
● Demonstrate how testing can be inclusive, impactful and FUN!

40. Questions?

41. Thank You!
Feel free to reach out to continue the conversation:
@bryanberger @ga