The document discusses the shortcomings of the optimization process in research, highlighting issues such as lack of user data, unclear criteria for testing, and poor involvement of the UX team. It raises concerns about ineffective testing leading to wasted resources and the absence of a follow-up mechanism to learn from failures. Additionally, it underscores the need for improved transparency and a structured process to enhance collaboration and outcomes.

1. Research is sometimes
conducted, primarily includes
competitive benchmarking, and
contains little to no user data.
KPIs are set for aesthetic
choices to improve CVR and
prevent loss of CVR.
Optimization chooses
abandonment or
Frankenstein and repeating the
test.
Optimization tests return with
muddy data, wasted money,
and no CX or UX evaluation.
The UX Team makes
wireframes to spec with little
to no voice in what's designed
or what splits are tested.
Frankenstein & Repeat
Issues of Transparency:
- What is Optimization Team's criteria, documentation, or guidelines?
- Why and what is being tested?
- Who decides what is being tested, and what constitutes success?
- How does the UX Team best plug-in to this process?
"This is what Apple does."
"But didn't engagement
increase for search on
mobile?"
Optimization or Leadership
submits last-minute aesthetic
changes to color, alignment, and
layout for additional splits.
Abandonment
"Why are we testing left
from right?"
"Let's add another split
with a blue link."
"We killed the test... we
lost too much money."
"Let's combine the slide-out
menu with the new icons,
and the color changes and
see how that does."
Issues of Improvement:
- Is there a process for re-visiting tests?
- Why is there no CX or UX follow-up and review for Optimization tests?
- Why aren't failures rewarded?
- How can the UX Team improve this process?
Vicious Cycle
A sequence of reciprocal cause and
effect in which two or more elements
intensify and aggravate each other,
leading inexorably to a worsening of
the situation.