Hwap pres__w bri cugelman nov2010


Published on

Overview of the Heart&Stroke Healthy Weight Action Plan; Lessons Learned from Developing this online tool, and early evaluation learning.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • As context, I am going to give you a whirlwind tour of the toolWill focus on the process and our learningif time, take you through the learning on the from the livetool.
  • 12 weeks of lessons to walk user through key steps of behavior change. Virtual counsellor is providing the structure.
  • Demonstrations of personal feedback and interactivity
  • Based on age and gender (carried forward from UA) and activity level
  • User is taught appropriate level of calorie restriction to lose weight,Based on our reco’s and what they have learned, user selects their target weightApproach: try to educate. Not just formula’s and cells to fill in.
  • Feedback to help user set up goals
  • IN several subsequent lessons, user will get a chance to create various components of their plan such as challenges, rewards, support.Example: challenge & solution
  • Can add and edit goals
  • Section to allow tracking
  • My plan- place to consolidate whole plan, and make edits
  • Vendor that did the lit search, went on to develop the toolSuch a big decision needs active debate at every go/no go stage
  • Context:Med size org with modest budgetsHealth promo staff, not software development or website design experts.
  • 6 month delay
  • Extensive data—show a preview just to give you a sense of what is possible
  • lesson 2 - only 23% completing the lesson (of those that started lesson 2). This is a very long lesson.How to solve this, without compromising the science
  • Next we did usability
  • Dr. Bob Bensley as Project lead: a Professor of Community Health Education; PI on over $1 million in grants focused on Internet-based behaviour changeBA in Kinesiology and Physical Education from Wilfrid Laurier University and a MA in Kinesiology and Health Science at York University. Obesity guide, food guide, PA guide, DRI’sMr. Rivas has degrees in community health education, exercise science, and public administration.
  • Hwap pres__w bri cugelman nov2010

    1. 1. My Healthy Weight Action PlanLessons Learned<br />November , 2010<br />
    2. 2. Agenda<br />Why an etool?<br />Tour of the tool<br />Process steps and learning<br /><ul><li>“If I were to do it again”</li></ul>4. Post launch live learning<br /><ul><li>Usability
    3. 3. User data and feedback</li></ul>5. Enhancements<br />
    4. 4. Why an Healthy Weight etool?<br />Weight was a strategic focus<br />Needed to strengthen HSF consumer information in this area<br />Have had immense success with three prior etools:<br /><ul><li>Blood Pressure Action Plan
    5. 5. Heart&Stroke Risk Assessment
    6. 6. E Mail Coach Program</li></ul>Objective:<br /><ul><li>Incorporate most promising practices into our health information
    7. 7. Increase engagement and interactivity of our health information</li></li></ul><li>Program Tour<br />
    8. 8.
    9. 9.
    10. 10.
    11. 11.
    12. 12.
    13. 13. Weekly Lessons<br />
    14. 14.
    15. 15. 12 Lessons<br />Monitoring<br />Goal setting<br />Tracking<br />Overcoming challenges<br />Rewards<br />Setbacks<br />Support system<br />Behaviour contract<br />Time management<br />Stress<br />Plateaus<br />Maintaining/new beginning<br />
    16. 16. Lesson 2- A Key Lesson<br /><ul><li>Feedback on week of self-monitoring
    17. 17. Recommended caloric intake, number of servings
    18. 18. Learn to set goals
    19. 19. Personalized recommendations re what to work on
    20. 20. VERY LONG</li></li></ul><li>
    21. 21.
    22. 22.
    23. 23.
    24. 24.
    25. 25.
    26. 26.
    27. 27.
    28. 28.
    29. 29.
    30. 30.
    31. 31. Tool Sections and Features<br />
    32. 32.
    33. 33.
    34. 34.
    35. 35.
    36. 36.
    37. 37.
    38. 38.
    39. 39.
    40. 40.
    41. 41.
    42. 42.
    43. 43.
    44. 44.
    45. 45.
    46. 46.
    47. 47. Process Steps and Learning<br />Phase 1 literature search and environmental scan<br />Phase 2 Design and Build<br /><ul><li>Vendor selection
    48. 48. Contract negotiation
    49. 49. Design and build process</li></li></ul><li>Lit Search & Env’t Scan<br />Purpose:<br />What tools are out there?<br />What is available to buy? License?<br />How effective are they? What is the research evidence?<br />Process:<br />1. RFP- to researchers and academics active in consumer eHealth tools<br />2. Criteria: <br />Expertise: behavior change; online interventions; technology; <br />Price and timing<br />Potential for involvement in future phases<br />No conflict of interest. (i.e. able to provide a unbiased review)<br />Commitment to the project. <br />Creativity<br />3. Cross Foundation Review Panel <br />
    50. 50. Lit Search & Env’t Scan- Conclusions<br />Hundreds of diet/weight e-tools<br /><ul><li> most are highly commercials; generally either:
    51. 51. simple counters & assessments or
    52. 52. tools that inundate with features and gadgets
    53. 53. none theory based; no structure</li></ul>Nothing suitable to buy<br />->Developed recommended practices <br />->Decision to proceed<br />
    54. 54. If I were to do it again….<br />Would stipulate at lit search RFP stage, that if we proceed to build, the author of the lit search cannot take a lead role (researchers have vested interested)<br />2. Would challenge conclusions a bit more.<br />
    55. 55. Design and Build- Vendor Selection Process<br />RFP <br /><ul><li>to researchers, academics, commercial vendors in consumer eHealth
    56. 56. internal and external experts and stakeholders to assess proposals</li></ul>Weighted Criteria:<br /><ul><li>Project Approach & Plan – 35%
    57. 57. Qualifications of the vendor- 30%
    58. 58. Quality, clarity and responsiveness of proposal – 10%
    59. 59. Financial stability of vendor – 5%
    60. 60. Cost quotation and IP needs – 20%</li></ul>Contract: <br />Did not give one vendor full budget including design and programming (ie. have them subcontract) due to concerns regarding contract size.<br />Vision: Product design vendor would lead the selection of design and programming vendors, and HSFO would hold the contract. <br />
    61. 61. If I were to do it again….<br />1. Keep the formal criteria and workgroup for vendor selection<br /><ul><li>Helped with the quality and effectiveness of decision process, as well as buy in
    62. 62. Decision of this size not be made by one person or one department,
    63. 63. Provides transparent process </li></ul>2. Would give one vendor full project responsibility- from concept design, to graphic design through to implementation <br /><ul><li>having 3 separate contracts allowed for vendors to avoid/debate responsibility “ie. that is their job”
    64. 64. Define deliverables and roles in detail. </li></ul>3. Seek a vendor with full capability (from concept – implementation) <br /><ul><li>Particularly when client does not have internal expertise or capacity
    65. 65. Ensure vendor has expertise on projects of same scale</li></ul>4. Look more closely at the commercial sector. <br /><ul><li>More likely able to provide the IA and technical services required; subject matter knowledge can be brought in easily. </li></li></ul><li>Contract Discussions and IP Rights<br />Contract Discussions: <br /><ul><li>Took 4 months; $20K in legal fees; IP rights as a sticking point
    66. 66. Vendor perceived IP rights where we thought there was none
    67. 67. Agreed royalty free rights for the life of the program</li></ul>If I were to do it again…<br /><ul><li>Increase clarity at the RFP stage re IP ownership
    68. 68. Be aware of Universities drive for IP ownership
    69. 69. If IP becomes barrier at contract stage: be firm, willing to walk away.</li></li></ul><li>Timing and Stages<br />Design and Copy Phase: Fall 2007<br />Wireframes and copy<br />Selected graphic design and programming vendors<br /><ul><li>Specs written by lead vendor
    70. 70. December: Handing off to Design Vendor </li></ul>Graphic and interface design: December-January 2008<br />Full detail (architecture and specs) not outlined or tested at time graphic and interface design was developed<br />
    71. 71. Timing & Stages<br />Final Rules Development:<br />Lead vendor was to apply “final rules” to design prior to hand off to programmer<br /><ul><li> Questions re how best to move forward lead us to engage web developer to review work</li></ul>Conclusion From Review Process:<br /><ul><li>IA insufficiently defined
    72. 72. Architecture and navigation needs simplification
    73. 73. Need to pare back scope of tool</li></ul>Spring-Summer 2008 spent defining IA<br /><ul><li>Programming started Fall 2008
    74. 74. Product tested and completed June 2009</li></li></ul><li>If I were to do it again….<br />1. Ensure roles are exhaustively defined and scoped out. Revisit them frequently.<br /><ul><li> Role definition is that much more urgent if you work with multiple vendors.</li></ul>2. Engage a dedicated Project Management expertise and capacity. PLUS, plan on extensive staff involvement for strategic guidance.<br />3. Do not proceed to interface design, until the wireframe and requirements are fully defined .<br />
    75. 75. If I were to do it again….<br />4. Need detailed wireframes or designs for every unique page and every page that is interactive.<br />5. Invest the time and money to test a paper prototype.<br />6. Keep scope modest so that you can finish design and programming in that time frame.<br /><ul><li>Too large undermines the ability to generate momentum
    76. 76. Too overwhelms and drags down morale
    77. 77. Market changes</li></li></ul><li>Post Launch Live Learning<br />User data<br />Usability research<br />
    78. 78. Participation and Attrition<br />Great interest in the tool. <br />Key attrition areas identified for attention<br /><ul><li>Landing page to risk assessment; risk assessment to registration
    79. 79. Only 38% of users to landing page proceeded to RA
    80. 80. 64% or users finishing RA we on to register for HWAP
    81. 81. Tradeoff :value of data versus the extra effort and time required of user. </li></li></ul><li>User Demographic<br />Gender: 90% are female<br />Age: 76% are 35-64<br />Ethnicity: Majority (90%) are Caucasians; <br />1% Chinese, 3% S Asian, 2% African Heritage, 2% Aboriginal<br />BMI: 30% have overweight BMI 25-29.9;<br /> 49% obese BMI 30+<br /> <br />
    82. 82. Participation over time<br />
    83. 83. Lesson Feedback- “How did you find the lesson?”<br />Majority responded positively suggesting the contents of the lesson itself are positive. <br />Negative feedback fell into several categories<br />Nothing new/expected more: (25%)<br />Concerning motivation (e.g., “didn’t do it, as not inspired,” “it doesn’t keep me motivated”): (15%)<br />Navigation or technical issues: (14%)<br />Program did not meet needs/unmet need: (10%)<br />Negative comments (e.g., “boring”, “didn’t work”, “too short,” “): (8%)<br />Weight management is a personal responsibility : (5%)<br />
    84. 84. Article Feedback & Journaling<br />Articles: viewed an average of 1.6 times per user, <br /><ul><li>a small proportion (4%) of users who opened an article rated it
    85. 85. those that did rate, tended to rate the articles as being good</li></ul>Less than 3% of users journalled<br /><ul><li>total of 624 journal entries.
    86. 86. Majority made only 1-2 entries; a few used journal function 6 times+ </li></ul> <br />Themes:<br /><ul><li>Recognition of the importance of tracking
    87. 87. The food tracker made users feel discouraged (too many frowns)
    88. 88. The benefits of making healthier choices (“feeling happy and energetic”)
    89. 89. Frustration that “doing good” does not necessarily translate to the scale
    90. 90. Description of challenges to making healthier choices
    91. 91. Recognition of the problems of emotional eating
    92. 92. Health concerns “Cardiologist told me…” </li></li></ul><li>Goal Setting Practices<br /><ul><li>2600 users created goals.
    93. 93. Poor understanding of what constituted a SMART goal or what was meant by a “task.”
    94. 94. Did not use the goal system as intended (either due to confusion or choice)
    95. 95. Many picked up on HW PLAN concepts in creating their goals</li></li></ul><li>Usability Evaluation<br />The value proposition of the site is unclear.<br />Not immediately clear what the value of using this site is, and why one should use it rather than another online weight loss site.<br /> Recommendation:<br /><ul><li>Ensure clarity: what HW Plan offers versus other online weight loss tools.
    96. 96. Improve execution and communication of the program “Point of Difference”.</li></ul> <br />2. The use of the counselor (in its execution) has caused usability issues.<br />The reason for the presence of the counselor is unclear, and her lack of identity makes her presence and lesson language uncomfortable, phony and cumbersome. Static execution of the counselor is not consistent with internet trends/capabilities<br /> Recommendation:<br />Remove the counselor from the site entirely. <br />
    97. 97. Usability Evaluation<br />3. The linear format of lessons does not translate well to the web.<br />The general premise of the HW Plan, which directs users through twelve distinct online lessons in sequence, does not work well. Users are forced through content that is not interesting or relevant to them, and must make a decision: continue forward, waiting for later content up that does look interesting, or leave the site, to find more interesting content elsewhere.<br />Recommendation:<br />Remove the rigid approach of twelve linear sessions that restrict the user from key concepts that may be helpful at the outset. <br />4. Nothing is done to encourage the user to return on a weekly basis.<br />Although users are told they should return to the site weekly, nothing is done to promote this behaviour. <br />Recommendation:<br /> If the metaphor of twelve sessions is kept, design techniques for getting users to return to the site, e.g. a reminder email once a week.<br /> <br />
    98. 98. Usability Evaluation<br />5. Navigating around the site is difficult.<br />The structure of the site is confusing. Several items can be found in multiple areas of the site. Because concepts are spread out over so many pages of the site, it can be difficult to remember where to find something. The labels used in the navigation do not help. <br /> Recommendation:<br /> Revisit the information architecture of the site. Keep concepts like “goals” in specific sections, rather than having them spread out over a number of pages<br /> <br />Assess whether labels are sufficiently intuitive. <br /> <br />6. There is no way to communicate with other participants of the site.<br />Today, the internet is all about communication. In the world of “Web 2.0”, site visitors are used to creating their own content, and interacting with others. By not giving program participants any way to communicate with each other, sharing challenges, successes and tips, the site is missing out on a great opportunity. Recommendation:<br /> Build increased social networking opportunities into the program.<br /> <br /> <br /> <br />
    99. 99. Usability Evaluation<br />9. Line lengths are extremely long, making text harder to read.<br />In typography, the optimal length of a line of text is 66 characters<br />Recommendation:<br />In redesign, ensure line length is consistent with usability best practices.<br /> <br />
    100. 100. Current Status and Plans<br />Revising architecture since last May, working with usability experts<br /><ul><li>Iterative, including copy writing</li></ul>Expert review in September with behavior change experts sent us back to the drawing board.<br />Revised architecture and wireframes and copy.<br />Just completed consumer testing.<br />Programming quotes and go/no go decision.<br /> <br />
    101. 101.
    102. 102. Development team and science basis<br /><ul><li>Project design lead by WMU
    103. 103. Team of 2 Canadian RD’s; reviewed by HSF RD
    104. 104. PA content reviewed by:
    105. 105. a certified personal trainer with experience in community recreation,
    106. 106. a volunteer with post grad in Kinesiology
    107. 107. Content experts:
    108. 108. Review by CBT and Obesity Expert
    109. 109. Product design consultation provided by consultant with expertise in e intervention design and research
    110. 110. Existing Canadian guidelines and available research
    111. 111. Expert Advisory Group: High level advice</li></ul>; <br />