'Test & learn goes mainstream' (iq, fall 2013) pdf


Published on

An Inovo perspective on the rapidly emerging 'Test & Learn' approach to getting from concept to market success.

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

'Test & learn goes mainstream' (iq, fall 2013) pdf

  1. 1.                           ‘Test & Learn’ Goes Mainstream   By  Brian  Christian     M   uch  of  the  strategic  innovation  work  that  Inovo  has  done  over  the  years  has  been  at  the  front   end   of   innovation,   or   in   what   we   call   the   Discovery   phase.   Oftentimes,   having   helped   our   clients   to   discover   a   set   of   attractive   opportunities,   we   are   asked   to   stick   around   and   assist   with   post-­‐ Discovery  phase  work.  For  strategic  innovations,  the  post-­‐Discovery  phases  are  commonly  referred  to   as  Incubation  and   Commercialization.   Recently,  several  of  our  clients  have  inquired  about  an  approach   to   Incubation   and   Commercialization   called   ‘Test   &   Learn.’   In   this   Innovation   Quarterly   article,   we   would  like  to  offer  our  perspective  on  the  emerging  Test  &  Learn  movement.   Before  we  dive  in,  it  may  be  helpful  to  remind  our  readers  what  we  mean  by  strategic  innovation.   (For   a   more   detailed   understanding,   please   see   our   white   paper   on   strategic   innovation   at   www.theinovogroup.com.)   Others   refer   to   this   type   of   innovation   as   breakthrough,   disruptive   or   radical.   At   Inovo,   we   prefer   the   descriptor   ‘strategic’   because   it   connotes   the   degree   of   long-­‐term   importance   this   type   of   innovation   has   internally   for   the   innovating   organization.   Not   all   strategic   innovations   are   radical   or   disruptive   and   neither   do   they   need   to   be.   Others   (such   as   Govindarajan   and   Trimble  in  10  Rules  for  Strategic  Innovators)  have  also  chosen  the  label  ‘strategic  innovation,’  and  the   world   has   come   to   recognize   that   strategic   innovation   requires   a   very   different   set   of   principles,   methods  and  skills  than  does  the  more  incremental,  sustaining  type  of  innovation.  It  is  also  appropriate   to  view  a  strategic  innovation  as  the  large  company  version  of  a  startup.   Chief  among  the  proponents  of  Test  &  Learn  are  Steve  Blank  (‘Customer  Development’),  Eric  Ries   (‘The   Lean   Startup’),   and   Alexander   Osterwalder   (‘Business   Model   Generation’).   For   the   purposes   of   this   article,   let’s   call   their   collective   methodologies   the   Test   &   Learn   Approach,   or   TLA.   TLA   goes   something   like   this:   The   startup   entrepreneur   has   a   vision   of   a   specific   innovative   offering   and/or   business  model  in  mind,  as  well  as  an  approach  for  achieving  that  vision.  The  precise  manifestation  of   The Innovation Quarterly (Fall 2013) | 1  
  2. 2. the   actual   offering   and   its   related   business   model   emerges   through   an   iterative   process   of   testing   with   potential   customers   in   an   approximation   of   the   eventual   marketplace.   The   entrepreneur   learns   from   these  tests  and  adapts  (or  ‘pivots’)  as  necessary  to  optimize  the  ‘product/market  fit.’  In  essence,  each   test   is   aimed   at   one   or   more   assumptions   or   hypotheses,   and   the   startup   learning   journey   will   succeed   or  fail  according  to  how  well  one  tests  these  assumptions.  The  entrepreneur  does  not  declare  failure   until  the  vision  has  been  proven  wrong  –  or  the  clock  or  funding  runs  out.  This  is  contrasted  against  the   traditional   startup   approach   in   which   more   thorough   planning   and   analysis   are   conducted   with   the   expectation   that   the   first   version   tested   in   the   market   is   essentially   the   right   one.   In   the   highly   uncertain   world   of   startups,   this   traditional   approach   often   leads   to   disappointment,   missed   expectations  and  investor  fatigue.   To   provide   a   deeper   perspective   on   TLA,   let’s   conduct   a   brief   ‘Lean’   primer.   One   of   the   foundational   principles   underlying   Lean   Thinking   is   the   elimination   of   waste   (muda   in   Japanese).   The   original  application  of  Lean  Thinking  at  Toyota  was  in  manufacturing.  In  Lean  Manufacturing,  waste  is   reduced  with  a  variety  of  specific  methods  and  tools  such  as  value  stream  mapping,  kanban  inventory   control,  and  cellular  manufacturing.  The  TLA  proponents  point  to  Lean  Manufacturing  as  the  inspiration   for   their   thinking.   For   some   reason,   they   have   chosen   not   to   cite   Lean   Product   Development   (as   explored   by   Kennedy,   Liker,   et   al.),   which   is   a   more   recent   application   of   Lean   Thinking.   This   is   also   sometimes  referred  to  as  Lean  Design  or  Lean  Innovation.     In   Lean   Product   Development,   waste   is   reduced   with   methods   such   as   customer-­‐defined   value   and   set-­‐based  concurrent  engineering.  In  the  Lean  set-­‐based  approach,  multiple  design  options  (a  set)  are   tested   in   parallel,   and   options   are   closed   out   only   as   knowledge   is   gained   and   assumptions   are   confirmed   or   refuted.   Eric   Ries   calls   this   ‘validated   learning.’   In   Lean   Product   Development,   testing   occurs   according   to   the   iterative   Plan-­‐Do-­‐Check-­‐Act   or   Shewhart/Deming   cycle.   Ries   has   labeled   this   the  Build-­‐Measure-­‐Learn  cycle.  In  Lean  Product  Development,  a  critical  means  for  eliminating  waste  is   to  include  end  user  or  customer  testing  in  the  PDCA  cycle  (customer-­‐defined  value)  to  ensure  design  fit   to   purpose.   Steve   Blank   calls   this   Customer   Development.   As   you   can   see,   the   foundation   for   TLA   is   more  directly  found  in  the  principles  and  methodologies  of  Lean  Product  Development  than  in  those  of     Lean  Manufacturing.     While   waste   elimination   would   not   seem   to   be   an   essential   element   of   a   strategic   innovation   process,   in   fact   it   is   quite   critical.   Its  value  is  less  in  cost  reduction  in  the  normal  operational  sense  and   more   in   the   conservation   of   precious   money   and   people   resources   allocated   to   strategic   innovation   projects.   When   confronted   with   a   vast   white   space   of   opportunity,   an   organization   must   be   able   to   discover   and   exploit   its   hidden   opportunities   within   restrictive   money   and   people   constraints.   This   would  not  be  possible  without  a  Lean  set-­‐based  approach.  Also,  as  these  opportunities  are  often  new   to   the   world   and/or   new   to   the   company,   uncertainty   remains   high   right   through   to   commercialization   of  the  new  offering  or  business  model.  The  greater  the  degree  of  uncertainty,  the  more  important  is  a   The Innovation Quarterly (Fall 2013) | 2  
  3. 3. Lean   set-­‐based   approach.   This   is   in   contrast   to   highly   predictable   situations   typical   of   sustaining   innovations,  for  which  the  traditional  linear  stage-­‐gate  model  is  a  reasonable  approach.   The   proponents   of   TLA   have   smartly   recognized   the   relevance   of   Lean   Thinking   (as   addressed   by   Ohno,   Womack,   et   al.)   for   startups   and   entrepreneurs.   By   doing   so,   they   have   broadened   the   application   of   Lean   Thinking   and   made   it   understandable   to   an   audience   that   probably   has   not   been   much   exposed   to   Lean   concepts.   In   addition,   through   Osterwalder’s   contributions,   they   have   broadened  the  application  of  Lean  beyond  the  offering  to  encompass  the  entire  business  model.   In   addition,   the   TLA   proponents   have   asserted   that   the   mindset   and   methodologies,   originally   developed  for  startups,  are  translatable  to  the  world  of  large-­‐company  strategic  innovation.  As  Blank   stated   in   an   article   in   the   May   2013   edition   of   the   Harvard   Business   Review,   “…despite   the   methodology’s  name  [‘The  Lean  Startup’],  in  the  long  term  some  of  its  biggest  payoffs  may  be  gained   by  the  large  companies  that  embrace  it.”  We  agree  with  this  assertion  and  are  happy  to  see  that  large   companies   are   starting   to   adopt   and   adapt   TLA   principles   and   methods   for   Incubation   and   Commercialization.   However,   there   are   also   some   caveats   to   be   made,   in   our   opinion,   regarding   the   application   of   TLA   for  large-­‐company  strategic  innovation.       1. Business  model  diversity:  As  we  move  from  intangible  products  distributed  through   virtual   channels   to   tangible   products   distributed   through   physical   channels,   the   methods  and  tools  required  to  support  TLA  vary  greatly.  In  fact,  business  models  are   becoming   increasingly   complex   with   a   blend   of   intangible,   tangible,   virtual   and   physical.   The   strategic   innovator   must   be   able   to   select   the   right   methods   and   tools   for  each  situation  from  a  vast  array  of  options.     2. Large   company   complexity:   TLA   emerged   as   a   framework   for   startups,   either   VC-­‐ funded  or  bootstrapped.  Startups  are  relatively  simple  organizations.  In  translating   TLA  to  large-­‐company  strategic  innovation,  it  is  important  to  complement  TLA  with   other   methods   and   tools   that   help   to   manage   a   complex   process   in   a   complex   organization.     The   first   caveat   relates   to   the   vast   difference   across   business   model   types   and   how   this   affects   the   way   that   one   designs   and   conducts   a   TLA   program.   Consider   the   differences   between   the   Incubation   and   Commercialization   of   a   mobile   device   gaming   app   like   Fruit   Ninja   and   that   of   a   human   drug   like   Lipitor,  or  that  of  a  new  commercial  jetliner  like  the  Boeing  787  Dreamliner.  As  business  models  move   from  intangible  to  tangible  offerings  and  from  virtual  to  physical  channels,  the  types  of  methods  and   tools  to  be  used  to  support  a  TLA  program  differ  greatly.       The Innovation Quarterly (Fall 2013) | 3  
  4. 4.     In   addition,   how   these   methods   and   tools   are   assembled   into   an   overall   TLA   program   design   will   vary  greatly  across  these  diverse  business  model  types.  Here  are  some  of  the  factors  that  vary  across   business   model   type   and   which   have   an   important   impact   on   the   design   of   a   TLA   program   for   Incubation  and  Commercialization.     1. Degree   of   invention:   When   the   required   invention   level   is   high,   iterations   on   the   product  development  side  take  a  long  time.  However,  market-­‐side  assumption  testing   need   not   come   to   a   halt   during   these   periods.   Valuable   input   into   the   invention   process  can  be  gained  while  waiting  for  the  minimum  viable  product  (MVP)  or  a  more   advanced   prototype.   The   ecosystem   that   determines   adoption   is   much   broader   than   the   customer   or   end   user,   and   valuable   information   about   the   ecosystem   (e.g.,   regulatory   constraints)   can   be   gained   during   these   extended   invention   periods.   Finally,   the  expected  invention  time  can  often  be  short-­‐circuited  by  scanning  the  landscape  for   available  technologies  or  technology  development  partners.  Recently,  Inovo  helped  a   large   CPG   client   (which   typically   spends   years   developing   its   strategic   innovations)   to   identify  a  material  technology  that  significantly  advanced  the  client’s  development  of  a   very   promising   new   product.   (This   high-­‐potential   offering   is   launching   in   the   marketplace  this  month,  so  we  will  share  more  about  it  in  the  near  future.)   2. Cost   of   prototype   construction:   When   the   cost   of   prototype   construction   is   high,   as   with   complex   systems   like   electric   vehicles,   it   is   often   practical   to   gain   customer   and   ecosystem  feedback  in  advance  of  a  full  system  investment  to  reduce  investment  risk.   In   these   situations,   it   can   make   sense   to   develop   in   silico   models   of   adoption   to   correlate  design  variations  to  customer  and  ecosystem  needs  and  desires.  A  few  years   The Innovation Quarterly (Fall 2013) | 4  
  5. 5. ago,   Inovo   built   an   adoption   model   for   the   electric   vehicle   market   to   support   the   product   design   efforts   of   an   electric   vehicle   battery   manufacturer.   It   is   also   possible   to   test  individual  elements  of  a  complex  system  and  piece  together  the  market  responses   to   build   a   complete   picture.   This   has   obvious   risks   but   can   be   superior   to   the   alternative   of   taking   one   large   risk   with   a   complete   system   prototype.   This   is   not   so   much   a   minimum   viable   prototype   as   a   set   of   minimum   viable   subsystems.   You   can   be   sure   that   for   its   Dreamliner,   Boeing   was   market-­‐testing   seat   design   with   passengers,   cockpit   design   with   pilots,   galley   design   with   flight   attendants,   and   cargo   bay   design   with   baggage   handlers   all   while   developing   and   testing   the   aircraft   itself   for   flight   worthiness.   3. Cost   of   in-­‐market   testing:   Sometimes   in-­‐market   testing   is   difficult,   costly   or   even   impossible.   This   is   true   in   many   business-­‐to-­‐business   situations   as   well   as   in   the   life   sciences.  In  these  situations,  it  is  often  practical  and  sometimes  necessary  to  alleviate   investment   risk   by   getting   end   user   and   broader   ecosystem   input   in   lieu   of   a   minimum   viable  product.  Several  years  ago,  Inovo  helped  a  Fortune  500  client  test  assumptions   related  to  the  use  of  a  proprietary  plastic  composite  to  replace  wooden  railroad  ties.   By   engaging   with   experts   across   the   railroad   industry   ecosystem,   we   were   able   to   help   the   client   discover   that   several   large   railway   operators   were   potentially   interested   in   this  new  composite  offering  for  certain  use  cases.  More  importantly,  however,  we  also   helped   them   discover   that   their   proposed   offering   was   insufficient   along   a   key   performance  dimension  and  therefore  would  not  be  adopted.  The  client  quickly  made   the   decision   to   cease   its   product   development   efforts,   and   we   were   told   that   this   saved  them  a  large  investment.   One  effective  way  to  manage  the  risk  associated  with  the  three  factors  discussed  above  is  to  seek   financial,  product  development,  and/or  commercialization  partners  who  can  share  the  risk  and  perhaps   quicken   one’s   time   to   market.   For   example,   Inovo   is   just   winding   down   a   project   in   which   we   are   helping   a   carbon   fiber   part   manufacturer   build   an   entirely   new   business   in   the   automotive   sector   by   identifying,   screening   and   developing   partnerships   with   OEMs   and   Tier   1   suppliers   who   are   well   positioned  to  co-­‐develop  their  first  trial  offerings.     The  second  caveat  regarding  the  translation  of  TLA  to  the  world  of  corporate  strategic  innovation   relates   to   the   real   cultural   and   business   process   differences   between   a   true   startup   (VC-­‐funded   or   bootstrapped)   and   a   large,   established   company.   As   Blank   likes   to   say,   “A   startup   is   not   a   small   version   of  a  big  company.”  Here  are  some  important  factors  that  explain  why  TLA  alone  may  not  be  adequate   to  the  task  of  large-­‐company  strategic  innovation.     1. Coordination:   Startups   are   simple   organizations   with   few   layers   and   shallow   functional  divisions.  Large  companies  are  complex  structures  with  many  layers  and   The Innovation Quarterly (Fall 2013) | 5  
  6. 6. deep  functionalization.  Orchestrating  a  ‘startup’  activity  within  a  large  company  is   a   complicated   task,   and   doing   it   quickly   enough   to   achieve   adequate   speed-­‐to-­‐ market   is   a   significant   challenge.   Inovo   has   found   that   constructing   cross-­‐ functional  and  multi-­‐level  teams  with  clear  accountabilities  and  active  involvement   is   the   key,   but   it   does   require   a   lot   of   effort   and   patience   to   manage   this   team   structure.   2. Decision-­‐making:   Startups   have   one   or   few   decision-­‐makers,   and   they   are   well   aligned  around  a  simple  goal.  Large  companies  have  many  decision-­‐makers,  often   in  a  matrix  consensus  structure,  and  goals  are  not  always  well  aligned.  Decisions  to   adjust,   pivot   or   proceed   within   a   large   company   are   difficult   to   make   effectively   while  maintaining  alignment.  At  Inovo,  we  have  found  that  the  use  of  online  voting   tools   to   gather   opinions   and   preferences   in   advance   of   decision-­‐making   events   are   a   great   way   to   cut   through   politics   and   posturing   and   also   to   broaden   the   conversation.   We   believe   that   for   strategic   innovation,   the   reliance   on   the   judgment   of   senior   executives   or   scientific   gurus   is   dangerous,   because   their   intuition   is   derived   from   the   existing   business.   It   helps   to   apply   decision   meeting   facilitation  tools  that  interrupt  the  normal  way  in  which  administrative  or  technical   seniority  tends  to  bias  the  outcome  of  conversations.   3. Portfolio   management:   Startups   generally   have   one   offering   and   one   business   model   to   manage.   Large   companies   have   many   offerings   and   possibly   multiple   business   models.   Large   companies   are   not   just   making   decisions   about   one   opportunity  but  must  make  ‘go/no  go’  decisions  across  multiple  opportunities  on   an   ongoing   basis.   Inovo   uses   an   objective   assessment   tool   to   rate   opportunities   along   two   axes   –   ‘should’   we   do   it   and   ‘could’   we   do   it   –   to   assist   in   making   portfolio   decisions   across   multiple   opportunities.   We   also   use   opportunity   canvases   to   monitor   and   maintain   a   healthy   distribution   of   opportunities   across   the  white-­‐space  domains  that  we  are  exploring.   As  with  the  business  model  caveat,  this  organizational  caveat  does  not  invalidate  the  application  of   TLA   for   large-­‐company   strategic   innovation.   However,   it   does   mean   that   TLA   alone   is   probably   inadequate.   In  summary,  having  a  Lean  set-­‐based  approach  such  as  TLA  is  an  important  component  of  a  large-­‐ company   strategic   innovation   capability,   but   it   is   not   sufficient.   It   requires   an   appreciation   for   the   wide   variety   of   business   model   types   that   exist   in   the   world   and   the   need   to   tailor   one’s   Test   &   Learn   program  with  the  proper  methods  and  tools.  It  also  requires  additional  methods  and  tools  to  allow  TLA   to  be  effective  in  a  complex,  large-­‐company  setting.   The Innovation Quarterly (Fall 2013) | 6  
  7. 7.           About  The  Inovo  Group   Founded   in   2001,   Inovo   is   an   innovation   consulting   firm   based   in   Ann   Arbor,   Michigan,   that   helps  the  world’s  leading  organizations  succeed  at  strategic  innovation.       About  the  Author   BRIAN   CHRISTIAN   is   President   of   The   Inovo   Group.   He   has   30   years   of   diverse   business   experience   in   product   development,   marketing,   sourcing,   and   information   technology   across  multiple  industries  and  geographies.  The  common  theme  throughout  his  career  has   been  the  development  of  new  technologies,  new  products,  and  new  capabilities.     Prior   to   the   formation   of   The   Inovo   Group,   Brian   was   founder   and   president   of   DASO,   a   consulting   firm   with   a   mission   to   help   companies   build   innovation   culture   and   capabilities.  Brian   also   worked   for   Whirlpool   Corporation   for   eleven   years   in   multiple   management   positions,   the   last   five   years   as   Vice   President   of   Global   Product   Development.  Earlier  in  his  career,  he  worked  for  the  management  consulting  firm  Booz  &   Company  in  their  Chicago  office.   Brian  holds  an  MBA  from  the  University  of  Chicago  and  a  Chemical  Engineering  degree  from   the  University  of  Michigan.             Copyright  ©  2013  by  The  Inovo  Group,  LLC   All  rights  reserved.   www.TheInovoGroup.com