This document discusses the importance of properly designing experiments to obtain specific and useful information. It explains that careful experimental design can identify boundaries and desired results ranges while adhering to specifications. The document outlines some basic principles of experimental design put forth by Ronald Fisher, including comparison to a standard, randomization, replication, blocking, and making designs orthogonal. When experiments are well-designed, they can provide quantitative and qualitative data to inform decision making using small sample sizes.

1. Barra Gwynn Enterprises (714) 271-0252
4174 Hillside Avenue Norco, CA 92860
WWW.BarraGwynn.com Page 1 of 2
This document represents the intellectual property of Barra Gwynn Enterprises, and may only be used as specified in the cover letter
accompanying it.
July, 2012
Design of Experiments
How many times, have you ever heard, “I need LESS information to make this decision!”?
Among other things, Leaders must be demonstrable Decision Makers and Visionaries. To
excel at leadership both of these attributes require the ability to understand the
fundamentals and methods to perform experiments. This extraordinary tool can identify
the boundaries and desired result range for a visionary, as well as adherence to the details
of the specifications and requirements in tasks at the lowest level.
Whether the experiment is to determine; subjective or objective result; a multi-variant
(lots of activity, processes and materials) or a digital (yes/no) issue; a sensitivity or
threshold boundary, the careful design provides not only the desired test, but likely, the
simplest means to get the test done.
From small single objective experiments to stratifying subjective opinions; sampling, polling
and testing are the most likely avenues to get data to analyze and measure. In most cases,
experiments are of the validation variety, rarely are some inclusive as to be complete
verification, but a good experiment needs a basis of verification. (Validation - an opinion
of confirmation of conformance; Verification - establishing factual basis for that opinion.)
What does that mean – understanding how to design experiments to provide specific
information saves time and money while providing quantitative and qualitative data to the
decision process.
Ronald Fisher’s book, The Design of Experiments, published in 1935, is the basis of so many
inspirations and tools to test. He started with a taste test – would people know the
difference in their tea if the milk or tea is poured first. Seems simple, or even frivolous,
but designing this experiment demonstrated the basics of establishing an experiment to
yield measurable results. Which he used to identify the basic nature of the tests to meet
the need for the experiment:
 Comparison – requires an “outside” acceptable standard to compare the results to the
standard to determine accuracy (think large statistical experiments)
 Randomization – a necessary basis for assigning individual to test groups, to increase the
“rigorousness” (think Monte Carlo)

2. Barra Gwynn Enterprises (714) 271-0252
4174 Hillside Avenue Norco, CA 92860
WWW.BarraGwynn.com Page 2 of 2
This document represents the intellectual property of Barra Gwynn Enterprises, and may only be used as specified in the cover letter
accompanying it.
 Replication – an attempt to validate while testing to establish original phenomena (think
results that vary in nature to arrive at an average)
 Blocking – designing tests to capture “blocks” of results (usually one variable at a time)
 Orthogonal – design experiments in multiple directions/dimensions to determine the
boundaries of the resulting condition (compare and contrast)
This launched a number of additional tools that allow better designs to achieve specific or
more efficient means to gain the same confidence that the desired testing provides.
Factorial experiments were Harold Hotelling’s contribution to develop a direct and indirect
confirmation sampling method testing, that could test two samples at a time of multiple
items with less than individually testing. Furthering the notion of limiting total testing
while gaining a level of confidence in the results, C. R. Rao developed orthogonal arrays.
Genichi Taguchi took these theories and experimental design and created a number of
methods that gave increased sensitivity information about the experiment’s variables while
testing in arrays.
In other words, testing techniques are becoming increasingly capable of using small
samples to gain wide range and accurate results on which decisions may be made with
increasing confidence.
The continuing and competitive search for better test methods, techniques and processes,
also means that more information is available to those interested in understanding the
“black box” better. In several cases, it allows you to better define the test you are trying
to perform, so that you get the results that will mean the most to you?
SUMMARY –
Tests and experiments are used to increase our knowledge, using them in a knowledgeable
manner can yield better results in quality, specificity and gradation if designed properly.
Given that sample testing is much less expensive than full testing, getting the best results
with the least experiment possible, has maximized the results. Sometimes, testing makes
profits by indicating directions not to take? For instance, had a major soft drink company
focused its tests, it might never have ventured into its “NEW” product? Clearly the cost
of that mistake is still being paid for!
Mallery (Mo) Aiken
Barra Gwynn Enterprises, LLC