Look Past Biases to Measure the Right Metrics by Ellynita Lamin - The Lean Startup Conference 12/11/14
•Download as PPT, PDF•
1 like•779 views
Recommended
Recommended
More Related Content
What's hot
What's hot (8)
Viewers also liked
Viewers also liked (20)
Similar to Look Past Biases to Measure the Right Metrics by Ellynita Lamin - The Lean Startup Conference 12/11/14
Similar to Look Past Biases to Measure the Right Metrics by Ellynita Lamin - The Lean Startup Conference 12/11/14 (20)
More from Lean Startup Co.
More from Lean Startup Co. (20)
Recently uploaded
Recently uploaded (20)
Look Past Biases to Measure the Right Metrics by Ellynita Lamin - The Lean Startup Conference 12/11/14
- 1. ee1111yynnee Look Past Biases to Measure the Right Metrics
- 2. 635,384 people displaced 3,415 schools destroyed 93, 285 missing 127, 720 killed 73,869 hectars of agricultural lands destroyed 119 bridges destroyed 2,618 kilometers of road destroyed
- 3. PPOOVVEERRTTYY 50% below poverty line HHOOWW WWOOUULLDD YYOOUU SSTTAARRTT??
- 4. Women Empowerment IINNIITTIIAATTIIVVEESS IIMMPPAACCTT Competency Development Adult Education Entrepreneurship Development | 2005 – 2009 | | USD150,000 | 3 Provinces Indigenous Knowledge Resource Regional Media Visibility Cross-Province Health Education Infrastructure Development
- 5. TTuurrnniinngg PPooiinntt:: SStteeppppiinngg OOuutt;; SStteeppppiinngg IInn Community Assessor Facilitator Community Community Facilitator Assessor Distrus Distrus Daily Surviva t Daily t Surviva l l Instinc t Instinc t Social Stigma Social Stigma
- 6. EEPPIIPPHHAANNYY The One that Goes Behind the Scenes
- 7. EEnnggaaggeemmeenntt VVaalluuee SShhiifftt Scale Revenue Reach Beneficiary Judging (Opinion) Sustainability Social Value Time Partner Iterating (Intention)
- 8. EEnnggaaggeemmeenntt MMeecchhaanniissmm Trust as Currency; instead of $$$ Indigenous Knowledge = Asset
- 9. EExxppeerriimmeenntt ##11:: RReehhaabbiilliittaattiinngg OOssttrraacciisseedd MMiilliittaannttss
- 10. EExxppeerriimmeenntt FFrraammeewwoorrkk Interest + Strength + Maintenance + Support = Income Conversations = Repeat Purchases = Social Acceptance = ∧$$ Skill-set A Skill-set B Skill-set C Skill-set D Skill-set E … | 1.5wk | | 1.5wk | | 1.5wk | | 1.5wk | | 1.5wk |
- 11. LLeessssoonnss LLeeaarrnntt Clearly Defined Mission as Anchor Focus Shorter Trial Cycles It’s ok not to follow formula/model Using indigenous resources increases engagement Bonus = decreased cost
- 12. EExxppeerriimmeenntt ##22:: AAlltteerrnnaattiivvee IInnccoommee ffoorr CCoonnfflliicctt VViiccttiimmss
- 14. LLeessssoonnss LLeeaarrnntt Community-centric programme forces agility and adaptability to continuously remain relevant Bonus = Communities Empowered as “Living Labs” Test, Measure, Continue/Change, Repeat & Scale
- 15. BBeeaauuddoohh BBeessaarreeee PPrrooggrraammmmee | Early Stage | | Middle Stage | | Growth Stage | RReebbuuiilldd TTrruusstt & VVaalluuee IImmmmeeddiiaattee IInnccoommee SSoolluuttiioonnss SSyysstteemmaattiicc RReedduuccttiioonn ooff NNeeggaattiivvee IIssssuueess
- 16. BBeeuuddoohh BBeessaarreeee MMoovveemmeenntt | 2009 – 2012 | | USD10,000 | IIMMPPAACCTT +5 Provinces +USD 80 per income per household per month International Aid Supplement Governance & Empowerment Education Regeneration
- 17. NNeexxtt SStteeppss Learn Innovation Accounting Turn Community Intuition into Replicable Process
- 18. Measure the Right Metrics By Falling Out of Love with Your Assumptions ee1111yynnee
Editor's Notes
- Aceh is one of out the 34 provinces in Indonesia with 5mil population. 73,869 = 182,534.27acres
- Legacy Psyche: Growth Barrier Formula-centric Wrong success measures Bloatedness Fell victim to confirmation bias Non-continuous engagement Wrong Skill-Community Fit When we understand the problems, we were able to test what specific metrics to use to measure progress or success (to ask ourselves what will be the measurable benefits of impact and how we can inculcate skills for the community to continuously be motivated to execute life long improvements?) Weight: Solutions target clear social problems with transformative growth. Not just target the symptoms of the problems Process & Cost driven – very embedded in the impact world that often look at the traditional process because it’s been proven to help with systematic change (one solution Smallness is notable: Scale vs sustainability – miniscule growth. The economic activities, which functioned primarily in medium and big towns during the conflict period now need to small towns in sub-districts and village. Had to inform the volunteer community managers that I needed to relook at the last few years because we had not stopped working but the impact ripple effect was reducing as time went on. That was a major breakdown for me but it helped me open up to many discoveries. One was which was Prof Anil Gupta’s TED talk on his works with harvesting grassroots innovation with the poor in India. He repeats many times over of how, “people who are economically poor are not poor here (heart) or here (mind)”. Then, it struck me. As much as I was immersed in the locals lives and Aceh had become home, I was not actually engaging them. I went in with a consultant’s opinion and focused on that, instead of an intention – the intention to have the poor better improve their quality of life. I forget that this doesn’t necessarily mean increase of revenue or reduction of cost. So, with this in mind, I went back to Aceh and relook at the learnings and feedback of the last five years. I was amazed at the insight I overlook because I had the traditional consultant’s measure filter ie time, financial, literacy, scale of success and health. I was so caught up in listening to experts in aid, that I forgot that the best persons to know what they need are the locals themselves. And them, because money is associated with knowledge and success, often do not speak against the financial aid that they desperately need. They align themselves to an “expert” vision. Over time, this conditioning makes them not know how to say out what they really need and/or want. They become beneficiaries, in every cents of the word. IT’S VERY EASY TO LOOK FOR SYSTEMIC, SYSTEMATIC SOLUTIONS RATHER THAN TACKLING CONCRETE PROBLEMS ONE AT A TIME The NGO or impact world traditionally automatically try to find “once and for all solutions” for these beneficiaries. Operational problems are almost always local and detailed and we’ll learn more by solving them concretely and specifically one by one rather than by looking for hyper-solutions to issues we don’t fully understand. But it’s easier to throw a huge chunk of cash at a new toy (new system, new plant, new program) than deal with the grittiness of real value-level problems. We become focused on the process and formula for aid that we forget to make it work! And the only way to do so, is to have the locals participate with the solutions development. At this point, I was determined to change the lens through which I was looking at them by discarding my own belief system and have the locals relook at the collected feedback of the past five years together with me. Our findings were surprising to all parties – which was nice! Because it meant we were learning together and that no one party is more of an expert than another. And I began to remember that I was an entrepreneur. So what if each community was seen as a startup, instead of clusters of beneficiaries? What do we do then? The moment all of us stop trying to fit the communities into an aid formula, we (including the locals) starting seeing them as startups with internal and external operational problems. Lessons Understanding the cultural dynamics to guide solutions development, we needed to throw away our assumptions on what social norm is -- if we envision communities as startups, any business model required would need to first understand the cultural dynamics of the communities Understanding that the communities usually do not verbally express what the root of the problems is because their culture implicitly asks them to filter out how they express themselves to outsiders. By immersing ourselves in local context, we are able to shadow and live the problems to help observe the different needs/problems at different stage/cycle. Often communities are more readily open with sharing what their desires or wants are because they assume aid are often financial. They also often hope that by being able to realise the desires, they can pacify what the real problems may be Lean process help guide growth of new thinking for my team and for the communities - how to map value streams What Did: Adapting skills-based training delivery for optimum individual community learning Implementing relationship-based programme for wider reach Don’t judge; iterate Being in the field What Didn’t: Measured the wrong things Different success values Did not validate assumptions Own emotional & professional bias Bloated programme Too heavy to stop In love with own solution/formula Fell victim to confirmation bias Asking wrong questions Wrongly directed empathy Not knowing what to pivot Focused instead on direction of pivot So what were the majorlearning insights and what needs to change? Customer Problem Pivot Market Segment Pivot Technology Pivot Product Feature Pivot Sales Channel Pivot Product vs Service Pivot Major Competitor Pivot Revenue Model Pivot
- When we understand the problems, we were able to test what specific metrics to use to measure progress or success (to ask ourselves what will be the measurable benefits of impact and how we can inculcate skills for the community to continuously be motivated to execute life long improvements?) Weight: Solutions target clear social problems with transformative growth. Not just target the symptoms of the problems Process & Cost driven – very embedded in the impact world that often look at the traditional process because it’s been proven to help with systematic change (one solution Smallness is notable: Scale vs sustainability – miniscule growth. The economic activities, which functioned primarily in medium and big towns during the conflict period now need to small towns in sub-districts and village. Had to inform the volunteer community managers that I needed to relook at the last few years because we had not stopped working but the impact ripple effect was reducing as time went on. That was a major breakdown for me but it helped me open up to many discoveries. One was which was Prof Anil Gupta’s TED talk on his works with harvesting grassroots innovation with the poor in India. He repeats many times over of how, “people who are economically poor are not poor here (heart) or here (mind)”. Then, it struck me. As much as I was immersed in the locals lives and Aceh had become home, I was not actually engaging them. I went in with a consultant’s opinion and focused on that, instead of an intention – the intention to have the poor better improve their quality of life. I forget that this doesn’t necessarily mean increase of revenue or reduction of cost. So, with this in mind, I went back to Aceh and relook at the learnings and feedback of the last five years. I was amazed at the insight I overlook because I had the traditional consultant’s measure filter ie time, financial, literacy, scale of success and health. I was so caught up in listening to experts in aid, that I forgot that the best persons to know what they need are the locals themselves. And them, because money is associated with knowledge and success, often do not speak against the financial aid that they desperately need. They align themselves to an “expert” vision. Over time, this conditioning makes them not know how to say out what they really need and/or want. They become beneficiaries, in every cents of the word. IT’S VERY EASY TO LOOK FOR SYSTEMIC, SYSTEMATIC SOLUTIONS RATHER THAN TACKLING CONCRETE PROBLEMS ONE AT A TIME The NGO or impact world traditionally automatically try to find “once and for all solutions” for these beneficiaries. Operational problems are almost always local and detailed and we’ll learn more by solving them concretely and specifically one by one rather than by looking for hyper-solutions to issues we don’t fully understand. But it’s easier to throw a huge chunk of cash at a new toy (new system, new plant, new program) than deal with the grittiness of real value-level problems. We become focused on the process and formula for aid that we forget to make it work! And the only way to do so, is to have the locals participate with the solutions development. At this point, I was determined to change the lens through which I was looking at them by discarding my own belief system and have the locals relook at the collected feedback of the past five years together with me. Our findings were surprising to all parties – which was nice! Because it meant we were learning together and that no one party is more of an expert than another. And I began to remember that I was an entrepreneur. So what if each community was seen as a startup, instead of clusters of beneficiaries? What do we do then? The moment all of us stop trying to fit the communities into an aid formula, we (including the locals) starting seeing them as startups with internal and external operational problems. Lessons Understanding the cultural dynamics to guide solutions development, we needed to throw away our assumptions on what social norm is -- if we envision communities as startups, any business model required would need to first understand the cultural dynamics of the communities Understanding that the communities usually do not verbally express what the root of the problems is because their culture implicitly asks them to filter out how they express themselves to outsiders. By immersing ourselves in local context, we are able to shadow and live the problems to help observe the different needs/problems at different stage/cycle. Often communities are more readily open with sharing what their desires or wants are because they assume aid are often financial. They also often hope that by being able to realise the desires, they can pacify what the real problems may be Lean process help guide growth of new thinking for my team and for the communities - how to map value streams What Did: Adapting skills-based training delivery for optimum individual community learning Implementing relationship-based programme for wider reach Don’t judge; iterate Being in the field What Didn’t: Measured the wrong things Different success values Did not validate assumptions Own emotional & professional bias Bloated programme Too heavy to stop In love with own solution/formula Fell victim to confirmation bias Asking wrong questions Wrongly directed empathy Not knowing what to pivot Focused instead on direction of pivot So what were the majorlearning insights and what needs to change? Customer Problem Pivot Market Segment Pivot Technology Pivot Product Feature Pivot Sales Channel Pivot Product vs Service Pivot Major Competitor Pivot Revenue Model Pivot
- What happens when we take money out of the picture? Co-designing with communities. Vanity metrics are not mainstream ones, we needed to redefine (how to qualify and quantify non-financial metrics) them at different stages of growth
- Vanity Metrics changes “better, faster or cheaper” isn’t always good Don’t fetishise models Experiments turn communities into living labs Rapid prototyping allows communities to experience changes and empowers them to create the change they want and/or need If you optimise purely for growth at the expense of achieving its founding mission, you can “lost your way” and g etcaught in “lost purposes”
- Vanity Metrics changes “better, faster or cheaper” isn’t always good Don’t fetishise models Experiments turn communities into living labs Rapid prototyping allows communities to experience changes and empowers them to create the change they want and/or need If you optimise purely for growth at the expense of achieving its founding mission, you can “lost your way” and g etcaught in “lost purposes”
- Early stage - rebuild trust e.g. talking to ostracised community members Middle stage - practising immediate solutions e.g. using donated machines to farm instead of selling for money (long term thinking vs daily survival mode), etc Later stage e.g. lower crime rates (less bombings etc), etc
- For me, is all about telling stories. We tell stories to learn. We tell stories to connect, we tell stories to remember, we tell stories to explain why something needs to happen, and tell stories to get it there. We also make miracles happen on a shoestring or less, but it sometimes takes a bit longer because, well, we get caught up in the stories. Summary - Sustainability of impact is the more urgent need than scale of revenue at the early stages of developing underserved communities in volatile environments Rapid experimentation helped communities to have a better understanding and appreciation of how solutions may or may not facilitate their growth and rebuild trust amongst the different groups within their communities (faster and shorter cycles of prototyping skills or ideas help to test them against social acceptance of the community). Lean principles help to ease communities to accept and create change Going through the above -- it makes me wonder, could lean principles systematically change culture faster?