• We explore such basics as play and add-to-queue
button placement, size, and functionality. We tested the switch
last year of whether the home page should be DVD or streaming
• We even test things like page load time, and streaming encoding
quality, and streaming startup time, to know which of those kinds of
optimizations should get the most of available engineering attention.
• We are very proud of our empirical focus, because it makes us
humble - we realize that most of the time, we don't know up-front
what customers want. The feedback from testing quickly sets us
straight, and helps make sure that our efforts are really focused at
optimizing the things that make a difference in the customer
• Overall we executed about 500 a/b tests on our web pages in a 20
month period which increased donation conversions by 49% and
sign up conversions by 161%.
• By turning the long donation form into 4 smaller steps we increased
the conversion rate by more than 5%.
• In the following test we had two photos. The first was a medium shot
of the President at the dinner table, but it didn't have much context as
nothing was in view/focus. Previous tests showed that large photos
with focus on the President increased conversions. The second photo
had a wider frame that revealed the First Lady and two dinner guests.
We hoped that users would be more likely to convert if they could see
just how close they would be sitting to the President of the United
States during dinner.
By changing the photo on the splash page we lifted conversions
by more than 19%.
• Pricing is difficult to pin down – used to be
• Free plan
• 50,000 visitors per month will see your
test in action
• Can’t do “Multi-page funnel” experiments
• “How to best use the Optimizely free
• Using Optimizely’s free plan: https://marktisans.com/best-
• A/B Testing with Google Analytics:
• Overview of usability testing techniques:
ASSESS YOUR WEBSITE
UNC Chapel Hill
With free usability testing tools