Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

When Remembering REALLY Matters - DevLearn 2014 presentation


Published on

Trying to improve business results via training? One-hit wonders will sabotage your efforts. Use research-proven strategies to yield long-term retention of your training.

Published in: Recruiting & HR, Business
  • Be the first to comment

When Remembering REALLY Matters - DevLearn 2014 presentation

  1. 1. when remembering REALLY matters Presented by Sharon Boller, President & Chief Product Officer
  2. 2. WHO AM I? Game-lover, learner, instructional designer, product owner, game designer, dog-lover and owner, Mom, wife, cyclist. Oh…and president and chief product officer, Bottom-Line Performance.
  3. 3. OUR GAME PLAN 1 The requirements to learn something The challenge of remembering… and the cost of 2 forgetting 3 The research on remembering…and forgetting 4 Strategies for remembering…and for learning 5 Two case studies that use the strategies
  4. 4. What’s Required to LEARN? Motivation Relevant Practice Specific, timely feedback Ability to retrieve later Right?????
  5. 5. MEMORY TEST #1 1 x 12 = ____ 2 x 12 = ____ 3 x 12 = ____ 4 x 12 = ____ 5 x 12 = ____ 6 x 12 = ____ 7 x 12 = ____ 8 x 12 = ____ 9 x 12 = ____ 10 x 12 =____ 11 x 12 =____ 12 x 12 =____ Times Tables Raise your hand if you feel VERY CONFIDENT that you can answer this set of multiplication problems without pausing.
  6. 6. MEMORY TEST #2 Spelling Words Write down 5 words as I say them…don’t think hard. Just write the words. Receive Believe Experience Neighbor Friend
  7. 7. MEMORY TEST #3 1. What year did the Civil War begin? 2. Where did Robert E. Lee surrender, ending the Civil War? 3. When did Abraham Lincoln deliver the Gettysburg address? History Three American history questions; write your answers to each one. Do NOT use your cell phone, iPad, or any other device to answer. Use your brain only.
  8. 8. So…why did you remember some of this stuff, but not other stuff?
  9. 9. Some stuff you need to know “cold” Other stuff you can “find/ locate”
  10. 10. Does Memory HERE ? Matter You are a customer support engineer for a medical device company. You train lab technicians how to operate and troubleshoot the $250,000 piece of machinery their hospital purchased from your company. Will the hospital be okay if its lab techs have to find/ locate info on using the equipment – or does the hospital expect lab techs to know common procedures “cold?”
  11. 11. Does Memory HERE ? Matter You are a rep in a sales meeting. Your customer just shared her needs, and she wants a response. You need to mentally think through your company’s eight product lines, select the RIGHT ONE, and then share the appropriate product benefits and features. She’s waiting…. Can you Google that? How long will she wait while you find/locate info? Or…does she expect you to formulate a fairly immediate response based on what you ALREADY KNOW AND CAN RECALL?
  12. 12. Does Memory HERE ? Matter You are a “skip tracer.” Your job is to recover cars if their owners have defaulted on a loan. There’s a lot of defaults, and the skip tracing process contains numerous steps! How efficient will you be at doing your job if you are constantly finding/locating? What’s okay to look up… and what do you simply need to know and know how to do from memory? What assets go unrecovered if you are slow at the job?
  13. 13. Does Memory HERE ? Matter You are a lab technician. You just dropped a vial of blood being tested for HIV. It just went SPLAT! You KNOW the blood poses a risk. You take annual training for this…but that was months ago. You recall some steps, but you are not sure of all of them. With blood splattered can you look up the procedure or do you need to be able to recall it and apply it to the situation at hand?
  14. 14. PEOPLE NEED TO REMEMBER A LOT OF STUFF IN THE WORKPLACE Product Knowledge Industry Facts Policies & Procedures Research Data Sales Messages What else do people need to “know cold” (i.e. from memory?)
  15. 15. Per ATD, we’re spending about $164 BILLION on training per year. Are we getting results? Time saved… or not wasted in the first place? Money saved…or money gained? People producing more, faster, more accurately or feeling less stressed?
  16. 16. WHAT’S REMEMBERED? During training? A week later?
  17. 17. How much forgetting? 30% in 3 to 6 days’ time 50% in 3 to 6 days’ time 70% in 3 to 6 days’ time 90% (or more) in 3 to 6 days’ time A B C D
  18. 18. IT DEPENDS
  19. 19. Let’s be REALLY generous and assume 30% forgetting.
  20. 20. Which 30% is the business okay with?
  22. 22. FOUR STRATEGIES TO USE Spaced intervals – 1 not a single “glop” 2 Repetition – several instances of it 3 right after making mistake 4 Stories Feedback – with requirement to do it
  23. 23. Spaced Intervals & Repetition Micro Spacing Macro Spacing Single Session Single Session Single Session or Day 1 Day 8 Day 16
  24. 24. Too Much Nothing
  25. 25. SPACING RESEARCH Spaced Repetition in the New York Times: Learning Research by Annie Murphy Paul: Distributed Prac9ce and Spaced Repe99on h<p://­‐research-­‐annie-­‐murphy-­‐paul-­‐distributed-­‐prac9ce-­‐ repe99on/ Spaced Repetition: What Research tells us by Dr. Karl Kapp: guru-research-tells-us/ Learning vs. Retrieval UCLA Bjork Learning and ForgeIng Lab: h<p://
  27. 27. STORIES Light up our whole brain Elicit POWERFUL Empathic response Personalize. Contextualize.
  28. 28. when remembering REALLY matters Things I have learned in this session:: 1. What
  29. 29. Case Study #1 – ExactTarget Product Launch (2012 case) 1. SaaS: Mul9ple Product Lines and Mul9ple Product Launches • 9 dis9nct product lines within organiza9on • Product line releases each month 2. Employees, clients, and partners had training overload; company needed a way to “mix it up.” 3. MobileConnect was one of the largest product launches in company history. Success was cri/cal.
  30. 30. LEARN + REMEMBER Motivation Relevant Practice Specific, timely feedback Ability to retrieve later Spaced Intervals, repetition Story
  31. 31. MOTIVATION Intrinsic is about person realizing learning will help them in their jobs. Extrinsic can couple with – but not replace – intrinsic. In this instances, motivation was prizes AND recognition.
  32. 32. MOTIVATION
  33. 33. MOTIVATION Meet Paul from Australia Earned 9tle as THE “Mobile Connect Guru” via game play. “I have never felt more valued in company that I have since becoming the Mobile Connect Guru”
  34. 34. STORY as challenge
  35. 35. STORY - context
  36. 36. STORY - context
  37. 37. Spaced Iterations, Relevant Practice, Feedback + REPETITION
  38. 38. Repetition – Interation 1
  39. 39. Repetition – Interation 2
  40. 40. Repetition – Interation 3
  41. 41. Feedback + Re-do
  42. 42. MORE Spacing, Repetition
  43. 43. Business Results Quicker pipeline built: “Of all the launches done in the two years previous to the MobileConnect launch, the sales team built one of the quickest pipelines for this product.” Dramatic increase in first-call support resolution. Greatly increased contract values.
  44. 44. Case Study #2 Dialysis Treat-ment Business Problems: 1. Patient post-training drop rate very high. 2. Time req’d to train a single patient longer than desired. 3. Only 1 patient per month could be trained. As a result, less marketshare and growth in marketshare than desired. Training Goal: “Patients can safely and confidently perform ongoing treatments at home.”
  45. 45. They had GLOP
  46. 46. We created spacing
  47. 47. Multiple repetitions
  48. 48. Multiple Repetitions
  49. 49. Practice, Feedback
  50. 50. Stories for empathy Characters in the eLearning told stories about their own situations; these mirrored the realities of the patients going through training. “One of biggest hurdles I had to face was learning to insert my own needles. I don’t care how brave you are, having to stick two needles in my arm every time I did a treatment was scary.” “I was worried about hurting David when I had to stick him with needles…I was not confident I could do it.”
  51. 51. RESULTS? • Yes! 6% reduction in “patient drops” in patients who received NxSteps training over those who were trained via another method. (Training for a single patient can typically cost $30K or more to do so 6% is a big deal.) • A major NxStage customer who used NxSteps program viewed it as so successful in improving bottom-line $ results for company that it is working to replicate program on peritoneal side of business.
  52. 52. Steps You Can Take In Design Ask: “What business problem is this learning solution solving?” To solve the problem, consider what is ESSENTIAL for people to know, do, believe. Distinguish between need to be able to “find/locate” from need to “know it cold” 1)
  53. 53. Steps You Can Take In Design If SMEs & stakeholders insist on too much content, ask: “What 30% - 90% are you okay with people forgetting…or never learning in the first place?” 2)
  54. 54. Steps You Can Take In Design Think beyond “one-hit wonder.” If remembering REALLY matters, then spacing and repetition HAVE to be part of your design. 3) Spaced Intervals, repetition
  55. 55. 4) DESIGN FOR: LEARN + REMEMBER Motivation Relevant Practice Specific, timely feedback Ability to retrieve later Spaced Intervals, repetition Story
  56. 56. Too Much Nothing 5) Remember…
  57. 57. Want to find me? Email: sharon@ Twitter: @Sharon_Boller