1. One person at a timeFocus groups are a group-think, distraction-filled mess. Avoid them and only talk to one person at a time. If desired, you can bring someone with you to take notes — some UX designers like this approach. Personally, I tend to do one-on-one interviews because I think people loosen up and thus open up a bit more.2. Know your goals and questions ahead of timeHave your assumptions and thus learning goals prioritized ahead of time. Decide who you want to talk to (age, gender, location, profession/industry, affluence, etc), and target interviewees accordingly. Prep your basic flow and list of questions. You might veer off the plan to follow your nose, which is great, but go in prepared.3. Separate behavior and feedback in discussionDecide up front if your focus is going to be on learning a user’s behavior and mindset, and/or getting direct feedback or usability insights on a product or mockup. Do not mix the two in the discussion flow or things will get distorted.Put “behavior and mindset” first in your discussion flow. During this part, don’t let the interviewee go too deep in terms of suggesting features (some people can’t help it), but keep them focused on if they have a problem, how they think about the problem space, and if and how they have tried to solve it in past. Getting people to discuss their actual actions, not just opinions, is very useful.4. Get psyched to hear things you don’t want to hearIf you don’t do this, you might find yourself selling or convincing, or even hearing what you want to hear. Remember, the goal in this early stage is learning and validation/invalidation, not a sale.Unless, of course, you have set a sale or LOI as a goal. You might want to shoot for a commitment from the interviewee as a way to measure true demand. If so, keep it entirely out of the behavior/mindset portion of the discussion.5. Disarm “politeness” trainingPeople are trained not to call your baby ugly. You need to make them feel safe to do this. My approach was to explain that the worst thing that could happen to me was building something people didn’t care about, so the best way they could help me was absolute, brutal honesty.6. Ask open ended questionsDo not ask too many yes/no questions. For example, minimize such questions as “do you like Groupon?” Instead ask “what kinds of deals do you look for, if any?” “What motivates you to hunt for deals?” “How do you discover deals?” “Do you get frustrated with the deal sites out there?”7. Listen, don’t talkTry to shut up as much as possible, and try to keep your questions short and unbiased (i.e. don’t embed the answer you want to hear into the question). Don’t rush to fill the “space” when the customer pauses, because they might be thinking or have more to say.Make sure you are learning, not selling! (at least not until that part of the conversation, if relevant)8. Encourage but don’t influenceIf you stay *too* quiet, some folks might start getting uncomfortable, thinking that they are boring you or you are judging them. You can keep things rolling with little motions of encouragement, such as nods, “I see”, “interesting”, etc. But do not say things that might steer or influence the interviewee.9. Follow your nose and drill downAnytime something tweaks your antenna, drill down with follow up questions. Don’t be afraid to ask for clarifications and the “why” behind the “what”. You can even try drilling into multiple layers of “why” (see “Five Whys”), as long as the interviewee doesn’t start getting annoyed.10. Parrot back or misrepresent to confirmFor important topics, try repeating back what the person said. You can occasionally get one of two interesting results through this. In the first, they correct you because you’ve misinterpreted what they said. In the second, by hearing their own thoughts, they’ll actually realize that their true opinion is slightly different, and they will give you a second, more sophisticated answer.Another approach is to purposefully misrepresent what they just said when you parrot it back, and then see if they correct you. But use this technique sparingly, if at all.11. Ask for introductionsAt the end of every interview, see if you can get leads to another 1 to 3 people to talk to.12. Write up your notes as quickly as possibleThe details behind a conversation fade fast, so if you haven’t recorded the session, write up your notes and color commentary as soon as you can. I brain-dump into a shared Google Doc so the rest of the team can see it. (Note: I typically have not recorded sessions for fear of making interviewees more self-conscious or careful, but other entrepreneurs have said to me that, while it takes some rapport-building at the start, pretty soon people forget about a recorder.)Afterwards: Look for patterns and apply judgementCustomer development interviews will not give you statistically significant data, but they will give you insights based on patterns. They can be very tricky to interpret, because what people say is not always what they do.You need to use your judgement to read between the lines, to read body language, to try to understand context and agendas, and to filter out biases based on the types of people in your pool of interviewees. But it is exactly the ability to use human judgement based on human connections that make interviews so much more useful than surveys.Ultimately, you are better off moving fast and making decisions from credible patterns than dithering about in analysis paralysis.
The Lean LaunchPadLecture 5: Customer Relationships Steve Blank Jon Feiber Jon Burke http://i245.stanford.edu/
key activities value customer proposition relationships key customerpartners segments cost revenuestructure key streams resources channels 2 images by JAM
CUSTOMER RELATIONSHIPSwhat relationships are you establishing with each segment? personal? automated? acquisitive? Retentive??
WeCall Customer RelationshipsDemand Creation• Get, Keep and Grow• How will customers hear about your product?• How much will it cost to acquire a customer using these strategies?• How does market type impact my demand creation strategy?
Grow Customers Funnel - Web/Mobile Earned and Paid Media Grow Customers product updates Affiliate Programs Keep Customers Contests, events Blogs, RS LoyaltyProgra S, emails ms Viral Loop
• How many come through the first step?• How much does that cost?• What is the conversion between each level?• How much in revenues can you get out of each acquired customer?
Demand Creation by Market Type • Create, drive demand into your Existing sales channel • Educate the market about what‟sResegmented changed • Drive demand into channel • Educate the market New • Identify/drive early adopters into your sales channels Clone • Copy a business
Market Type Existing Resegmented New Customers Known Possibly Known Unknown Customer Needs Performance Better fit Transformational improvement Competitors Many Many if wrong, NoneHow does market type influence demand few if right creation? Risk Lack of branding, Market and Evangelism and sales and product re- education cycle distribution definition ecosystem Examples Google Southwest Groupon Market Type determines: Rate of customer adoption Sales and Marketing strategies Cash requirements
Team Deliverable by Next Week - Web Get a working web site and analytics up and running – Track where your visitors are coming from (marketing campaign, search engine, etc.) and how their behavior differs – What were your hypotheses about your web site results? • Actually engage in “search engine marketing” (SEM) • Spend $20 as a team to test customer acquisition cost. • Ask your users to take action, such as signing up for a newsletter. • Use Google Analytics to measure the success of your campaigning. • Change messaging on site during the block to get costs lower, team that gets the lowest delta costs wins. • If you assume virality • show viral propagation of your product and the improvement of your viral coefficient over several experiments • What is your assumed customer lifetime value? • Are there any proxy companies that would suggest that this is a reasonable number?
Team Deliverable by Next Week• For non-web teams: • Get prototype demo working. • Build demand creation budget and forecast. • What is your customer acquisition cost? • Did anything change about Value Proposition or Customers/Users? • What is your customer lifetime value? Channel incentives – does your product or proposition extend or replace existing revenue for the channel? • What is the “cost” of your channel, and it’s efficiency vs. your selling price?• Everyone: Update you blog/wiki/journal • What kind of initial feedback did you receive from your users? • What are the entry barriers? • Present and explain your marketing campaign. What worked best and why?
implantable drug infusion pumps with remote physician control for chronic pain patients at home “the right dose at the right time and place”Christian Gutierrez (EL), Ellis Meng (PI), Carol Christopher (IM), Tuan Hoang (FE)
Chronic Pain v4 FS Team Trade shows Faster relief Training Patients Formulary AcceptanceKOLs Clinical data Efficient patient FDAFoundations management and Clinicians Dosing flexibility Support CMS (Medicare)Advocacy Groups Access to high-valueOEMs therapies and Institutions IP pharmacoeconomics HospitalsWirelessDevelopers Proprietary pharmacoeconomics Pain clinics Payors/ICA knowledgeElectronic healthrecord providers Human Resources Product Dev Costs Unit sales Manufacturing Costs Support ServicesMarketing Costs FDA/Clinical Trials Bundled kits Electronic records
Getting outClinicians Dr. Stan Louie, Drug Formulation Expert (USC Pharmacy) Dr. Giovanni Cucchiaro, Anesthesiologist (CHLA)Institutions/patients Dr. Diana Hull, Physician (Group Health in Washington state, formerly at Kaiser California) Thomas Hsu, Insurance Specialist (Network Medical Management; a California ICA) Two chronic pain patients Pump user and creator of support forum User of oral narcotics and patchesRegulatory Dr. Frances Richmond (Director Regulatory Science Program, USC) Richard Hull (formerly at company selling Lapband)Entrepreneurs/Industry
Product flow/ChannelElectronic Partners/ Health Fluid Synchrony OEMS Records Electronic Support Pump + Bundled Records Services Controller Kits Hospitals (AnesthesiologistsPatients Neurosurgeons) Pain Clinic (Anesthesiologists Neurosurgeons)
Hospitals Channels (Direct)Pain Clinics • Direct to institutions • Some formularies involved in purchase decisions • Some doctors make purchase decision directly • Device company/Doctor relationship is key • Heavily influenced by : • Clinical study results • Regulatory approval • Reimbursement
Patient Care Flow (Proposed) Electronic Partners/ Health Fluid Synchrony OEMS Records Electronic Support Pump + Bundled Records Services Controller Kits Actionable feedback to doctors/institutions E-prescription / closing loop Hospitals Surgery/Rx/ (Anesthesiologists Patient Discharged reprogramming Neurosurgeons) Pain ClinicTrial period/ Scheduled (AnesthesiologistsHome setting follow-up Neurosurgeons) Weeks/months Days Key factors: Reimbursement , state regulations
Regulatory considerations PMA 510K Trial size 100’s of patients 20-100 Costs Up to $100,000 per patient $10-50 MM $1-10 MM Time ~ 3-4 yrs + post ~ 2-3 yrs approval follow-on• PMA approval with grouping of FDA approved drugs.• Clinical trials results used to obtain CMS (Medicare) approval• 510K restricts technology to predicate devices • Can be more difficult to market against incumbents• European CE mark is easier to attain (safety and performance only)
Take-aways• Channel is direct in this existing market • Channel for e-health is more complex and evolving• State-to-state regulations can impact incentives • Can pose problems as electronic records systems vary across the country Next Steps• Understand costs associated with reaching doctors/institutions directly• Understand structure of e-health channel• Develop regulatory pathway (timelines and cost profile)
Chronic Pain v4 FS Team Trade shows Training Faster relief PatientsKOLs Formulary Acceptance Clinical data Efficient patientFoundations FDA management and Clinicians Dosing flexibility SupportAdvocacy Groups Access to high-valueOEMs therapies and Institutions IP pharmacoeconomics HospitalsWirelessDevelopers Proprietary pharmacoeconomics Pain clinics Payors knowledgeElectronic healthrecord providers Human Resources Product Dev Costs Unit sales Manufacturing Costs Support ServicesMarketing Costs FDA/Clinical Trials Bundled kits Electronic records
“insero” = to plant ”gen” = gene Manufacturing platform for Lucas Arzola (EL) rapid, cost-effective, and scalable Karen McDonald (PI)production of therapeutics in tobacco Vasilis Voudouris (Mentor)
What We Know We have a novel technology platform with numerous market opportunities Our working hypothesis – that we can scale up and commercialize our platform for production of life-saving therapeutics Jon Feiber – “Since you are a platform technology, it makes sense to engage in „market discovery‟ and „customer discovery‟ at the same time during the next weeks” Challenging this hypothesis by speaking with as many experts and customers as we can This week: explored decision making and distribution channels in the case of a pandemic
The Business Model Canvas Target Product – seasonal & pandemic flu vaccinesTobacco Suppliers R&D Speed Long-Term U.S. GovernmentGene Synthesis Manufacturing Cost-Effectiveness Contracts with - CDCCompanies Regulatory Approval Robustness Government and - HHS BARDACMOs Licensing Scalability Vaccine - DOD DARPA- Purification Marketing Safety Manufacturers Foreign Governments- Fill & Finish Ease of Customization NGOs- Packaging U.S. Supply Vaccine Manufacturers- QA/QC -Established andCROs Emerging Biotech- Clinical TrialsFDA IP – Patents, Trade Secret Manufacturing Facility Distribution through Government and Pharma Companies Capital Investments Contract Manufacturing Manufacturing Costs Fully Integrated Manufacturing (Sales) Licensing Costs Licensing (Royalties) Marketing
Getting Out of the Lab!Cast a broad net by talking to many different experts and customers:(1) Executives from large companies Name Title Institution Michael Girard Sustainability Manager Aerojet Michael Jacobson Director of Corporate Responsibility Intel Joseph Kieren Director of Corporate Real Estate AT&T(2) Entrepreneurs and angel investors from Sacramento Name Title Institution Andrew Hargadon Professor of Management UC Davis WilAgatstein Professor of Management UC Davis LarryPalley Former General Manager Intel John Selep Operations Manager HP ThomasAlberts Consultant SBDC Cary Adams Head of MedStart Program SARTA
Getting Out of the Lab!(3) Experts in the commercialization of biotech platform technologies Name Title Institution Greg McParland Consultant DSM Ventures Fernando Garcia Senior Director Amyris(4) Experts in vaccine manufacturing Name Title Institution Ann Arvin PCAST Vaccinology Working Group Stanford (Key Opinion Leader on Vaccines) Misa Sugui Associate Scientist MedImmune Floro Cataniag Laboratory Manager MedImmune
Channels and DistributionConversation with Dr. Ann Arvin – Key Opinion Leader on vaccinesIn the case of a pandemic: Vaccine manufacturers have to be producing vaccines for seasonal flu – regulatory approval, QA, and validation need to be in place When a pandemic occurs, the government (BARDA) negotiates a manufacturing contract with vaccine companies – number of doses, formulation, price, and time are agreed upon CDC provides the elucidated strain to the manufacturer FDA considers the pandemic flu vaccine to be a variation of the seasonal flu vaccine – new regulatory approval is not necessary Vaccine manufacturers work with the new strain to ramp up production as quickly as they can – takes 4-6 months Sterility and quality testing is performed for the produced vaccines – some tests are done in-house and some are done by outside laboratories Vaccine is released
Channels and DistributionGetting the vaccines to the patients Vaccine manufacturers have contracts with wholesalers (i.e. McKesson Corp.) to distribute the vaccines – distribution is not a cost for the manufacturers, they hand over the product In the case of a pandemic, vaccines are also distributed through local contracts with the state health departments Theydistribute the vaccines to hospitals and clinics, where they can be administered to the patients
Organizational StrategyConversation with Greg McParland – Former CEO of biotech platform company: the virtual biotechnology company model“Starting out and for as long as you can, you should be a virtual company. You can have contracts to outsource the downstream part of the process (purification, fill and finish, packaging, etc.) ”“Keep your core technology and focus on using your manufacturing platform for protein production”.Common practice in biotechnology – almost every company has contracts with CROs, CMOs, marketing and distribution arrangements, etc.More flexibility – move quickly from failed avenues of research to more promising projectsStartups partner with big pharma companies to complete clinical trials and take product to market“If you build it, they will come” – but only build the essential core that lets you control your technology platform
More FeedbackConversation with Dr. Ann Arvin – Key Opinion Leader on vaccines Pain point: Reliability issues with traditional egg platform - willingness to move away to a different manufacturing platform Pain point: Current platforms are not fast enough, cannot have an impact in case of a pandemic - sense of urgency in finding a manufacturing platform that can produce vaccines faster and at a large scale Given this landscape, we still believe our technology can solve a significant problem in the vaccine marketConversation with Dr. Misa Sugui & Dr. Floro Cataniag – MedImmune Pain point: attenuated virus platform is harder to work with, safety measures are more stringent – would prefer recombinant subunit vaccines Wish: a faster process for vaccine production (our technology can help with this) Wish: a faster process for clinical trials and for approval of new drugs (this we can’t do anything about) MedImmune is a possible partner - always looking for new vaccine production technologies and new products to incorporate in their pipeline
More FeedbackConversation with Fernando Garcia – Amyris Biotech platform technology company First target product: drug for malaria, partnered with Sanofi to commercialize Change in strategy: they have transitioned into making biofuels Why have they made this transition? We will follow up with one of the founders of the company to find out
Next Steps We believe we have a good feel for our value proposition We need to better understand how we can sell to customers and how to establish these relationships, how partners’ decisions are made – meeting with Sanofi Head of External R&D Keep searching for a business model that will allow us to commercialize our technology – looking for meetings with companies that distribute/sell flu vaccine antigens for research and diagnostic use, trying to determine market size We need to talk to many more experts and customers…
Interviews Action MotionCustomer Interaction Meetings: Planned Customer Interaction1. Director of R&D of C/A partner Meetings:2. NETL Methane Hydrate RG 1. Jeff Farbacher, CEO Accutran3. Ed Faust, Global Marketing, Siemens 2. Ed Faust, Global Marketing,4. Former GE Employee Siemens5. Berkeley sensors group 3. Charles Noll, Marcellus Shale6. Tim Fogarty, Director of IW Energy CoalitionHypothesis Testing: Planned Hypothesis Testing:1. Ed Faust, Global Marketing, Siemens 1. Dr. GiladKusne, NIST 2. Ann Truschel, Corporate Insurance Broker 3. Tim Fogarty, Director of IW Energy
Chemical, Physical, Thermal …. Chemical - Every significant market segment has specific marketing agencies directed towards selling them goods Direct MarketingPossible Not Possible [Too expensive]
Chemical, Physical, Thermal …. Chemical - Every significant market segment has specific marketing agencies directed towards selling them goods Direct MarketingPossible Not Possible [Too expensive]
Direct sales to plants typically is a very hard wayChemical, to generate Physical, scalable business in the sensors market.Thermal ….Typically much better to bundle product into offerings from larger Chemical sensors businesses - Every significant market segment has specific marketing agencies directed towards selling them goods Agrees with current approach to this first market! Direct MarketingPossible Not Possible [Too expensive]
Org. Chart – Current C/A Partner CEO, CTO, CFO, etc. etc. etc. Global Director of R&D CEO, Director of R&D Director of R&D Director of Marketing Director of Product Service Engineers, etc. etc. etc.