As context, I am going to give you a whirlwind tour of the toolWill focus on the process and our learningif time, take you through the learning on the from the livetool.
12 weeks of lessons to walk user through key steps of behavior change. Virtual counsellor is providing the structure.
Demonstrations of personal feedback and interactivity
Based on age and gender (carried forward from UA) and activity level
User is taught appropriate level of calorie restriction to lose weight,Based on our reco’s and what they have learned, user selects their target weightApproach: try to educate. Not just formula’s and cells to fill in.
Feedback to help user set up goals
IN several subsequent lessons, user will get a chance to create various components of their plan such as challenges, rewards, support.Example: challenge & solution
Can add and edit goals
Section to allow tracking
My plan- place to consolidate whole plan, and make edits
Vendor that did the lit search, went on to develop the toolSuch a big decision needs active debate at every go/no go stage
Context:Med size org with modest budgetsHealth promo staff, not software development or website design experts.
6 month delay
Extensive data—show a preview just to give you a sense of what is possible
lesson 2 - only 23% completing the lesson (of those that started lesson 2). This is a very long lesson.How to solve this, without compromising the science
Next we did usability
Dr. Bob Bensley as Project lead: a Professor of Community Health Education; PI on over $1 million in grants focused on Internet-based behaviour changeBA in Kinesiology and Physical Education from Wilfrid Laurier University and a MA in Kinesiology and Health Science at York University. Obesity guide, food guide, PA guide, DRI’sMr. Rivas has degrees in community health education, exercise science, and public administration.
Agenda<br />Why an etool?<br />Tour of the tool<br />Process steps and learning<br /><ul><li>“If I were to do it again”</li></ul>4. Post launch live learning<br /><ul><li>Usability
User data and feedback</li></ul>5. Enhancements<br />
Why an Healthy Weight etool?<br />Weight was a strategic focus<br />Needed to strengthen HSF consumer information in this area<br />Have had immense success with three prior etools:<br /><ul><li>Blood Pressure Action Plan
Design and build process</li></li></ul><li>Lit Search & Env’t Scan<br />Purpose:<br />What tools are out there?<br />What is available to buy? License?<br />How effective are they? What is the research evidence?<br />Process:<br />1. RFP- to researchers and academics active in consumer eHealth tools<br />2. Criteria: <br />Expertise: behavior change; online interventions; technology; <br />Price and timing<br />Potential for involvement in future phases<br />No conflict of interest. (i.e. able to provide a unbiased review)<br />Commitment to the project. <br />Creativity<br />3. Cross Foundation Review Panel <br />
Lit Search & Env’t Scan- Conclusions<br />Hundreds of diet/weight e-tools<br /><ul><li> most are highly commercials; generally either:
none theory based; no structure</li></ul>Nothing suitable to buy<br />->Developed recommended practices <br />->Decision to proceed<br />
If I were to do it again….<br />Would stipulate at lit search RFP stage, that if we proceed to build, the author of the lit search cannot take a lead role (researchers have vested interested)<br />2. Would challenge conclusions a bit more.<br />
Design and Build- Vendor Selection Process<br />RFP <br /><ul><li>to researchers, academics, commercial vendors in consumer eHealth
internal and external experts and stakeholders to assess proposals</li></ul>Weighted Criteria:<br /><ul><li>Project Approach & Plan – 35%
Cost quotation and IP needs – 20%</li></ul>Contract: <br />Did not give one vendor full budget including design and programming (ie. have them subcontract) due to concerns regarding contract size.<br />Vision: Product design vendor would lead the selection of design and programming vendors, and HSFO would hold the contract. <br />
If I were to do it again….<br />1. Keep the formal criteria and workgroup for vendor selection<br /><ul><li>Helped with the quality and effectiveness of decision process, as well as buy in
Decision of this size not be made by one person or one department,
Provides transparent process </li></ul>2. Would give one vendor full project responsibility- from concept design, to graphic design through to implementation <br /><ul><li>having 3 separate contracts allowed for vendors to avoid/debate responsibility “ie. that is their job”
Define deliverables and roles in detail. </li></ul>3. Seek a vendor with full capability (from concept – implementation) <br /><ul><li>Particularly when client does not have internal expertise or capacity
Ensure vendor has expertise on projects of same scale</li></ul>4. Look more closely at the commercial sector. <br /><ul><li>More likely able to provide the IA and technical services required; subject matter knowledge can be brought in easily. </li></li></ul><li>Contract Discussions and IP Rights<br />Contract Discussions: <br /><ul><li>Took 4 months; $20K in legal fees; IP rights as a sticking point
Vendor perceived IP rights where we thought there was none
Agreed royalty free rights for the life of the program</li></ul>If I were to do it again…<br /><ul><li>Increase clarity at the RFP stage re IP ownership
Be aware of Universities drive for IP ownership
If IP becomes barrier at contract stage: be firm, willing to walk away.</li></li></ul><li>Timing and Stages<br />Design and Copy Phase: Fall 2007<br />Wireframes and copy<br />Selected graphic design and programming vendors<br /><ul><li>Specs written by lead vendor
December: Handing off to Design Vendor </li></ul>Graphic and interface design: December-January 2008<br />Full detail (architecture and specs) not outlined or tested at time graphic and interface design was developed<br />
Timing & Stages<br />Final Rules Development:<br />Lead vendor was to apply “final rules” to design prior to hand off to programmer<br /><ul><li> Questions re how best to move forward lead us to engage web developer to review work</li></ul>Conclusion From Review Process:<br /><ul><li>IA insufficiently defined
Architecture and navigation needs simplification
Need to pare back scope of tool</li></ul>Spring-Summer 2008 spent defining IA<br /><ul><li>Programming started Fall 2008
Product tested and completed June 2009</li></li></ul><li>If I were to do it again….<br />1. Ensure roles are exhaustively defined and scoped out. Revisit them frequently.<br /><ul><li> Role definition is that much more urgent if you work with multiple vendors.</li></ul>2. Engage a dedicated Project Management expertise and capacity. PLUS, plan on extensive staff involvement for strategic guidance.<br />3. Do not proceed to interface design, until the wireframe and requirements are fully defined .<br />
If I were to do it again….<br />4. Need detailed wireframes or designs for every unique page and every page that is interactive.<br />5. Invest the time and money to test a paper prototype.<br />6. Keep scope modest so that you can finish design and programming in that time frame.<br /><ul><li>Too large undermines the ability to generate momentum
Participation and Attrition<br />Great interest in the tool. <br />Key attrition areas identified for attention<br /><ul><li>Landing page to risk assessment; risk assessment to registration
Only 38% of users to landing page proceeded to RA
64% or users finishing RA we on to register for HWAP
Tradeoff :value of data versus the extra effort and time required of user. </li></li></ul><li>User Demographic<br />Gender: 90% are female<br />Age: 76% are 35-64<br />Ethnicity: Majority (90%) are Caucasians; <br />1% Chinese, 3% S Asian, 2% African Heritage, 2% Aboriginal<br />BMI: 30% have overweight BMI 25-29.9;<br /> 49% obese BMI 30+<br /> <br />
Lesson Feedback- “How did you find the lesson?”<br />Majority responded positively suggesting the contents of the lesson itself are positive. <br />Negative feedback fell into several categories<br />Nothing new/expected more: (25%)<br />Concerning motivation (e.g., “didn’t do it, as not inspired,” “it doesn’t keep me motivated”): (15%)<br />Navigation or technical issues: (14%)<br />Program did not meet needs/unmet need: (10%)<br />Negative comments (e.g., “boring”, “didn’t work”, “too short,” “): (8%)<br />Weight management is a personal responsibility : (5%)<br />
Article Feedback & Journaling<br />Articles: viewed an average of 1.6 times per user, <br /><ul><li>a small proportion (4%) of users who opened an article rated it
those that did rate, tended to rate the articles as being good</li></ul>Less than 3% of users journalled<br /><ul><li>total of 624 journal entries.
Majority made only 1-2 entries; a few used journal function 6 times+ </li></ul> <br />Themes:<br /><ul><li>Recognition of the importance of tracking
The food tracker made users feel discouraged (too many frowns)
The benefits of making healthier choices (“feeling happy and energetic”)
Frustration that “doing good” does not necessarily translate to the scale
Description of challenges to making healthier choices
Recognition of the problems of emotional eating
Health concerns “Cardiologist told me…” </li></li></ul><li>Goal Setting Practices<br /><ul><li>2600 users created goals.
Poor understanding of what constituted a SMART goal or what was meant by a “task.”
Did not use the goal system as intended (either due to confusion or choice)
Many picked up on HW PLAN concepts in creating their goals</li></li></ul><li>Usability Evaluation<br />The value proposition of the site is unclear.<br />Not immediately clear what the value of using this site is, and why one should use it rather than another online weight loss site.<br /> Recommendation:<br /><ul><li>Ensure clarity: what HW Plan offers versus other online weight loss tools.
Improve execution and communication of the program “Point of Difference”.</li></ul> <br />2. The use of the counselor (in its execution) has caused usability issues.<br />The reason for the presence of the counselor is unclear, and her lack of identity makes her presence and lesson language uncomfortable, phony and cumbersome. Static execution of the counselor is not consistent with internet trends/capabilities<br /> Recommendation:<br />Remove the counselor from the site entirely. <br />
Usability Evaluation<br />3. The linear format of lessons does not translate well to the web.<br />The general premise of the HW Plan, which directs users through twelve distinct online lessons in sequence, does not work well. Users are forced through content that is not interesting or relevant to them, and must make a decision: continue forward, waiting for later content up that does look interesting, or leave the site, to find more interesting content elsewhere.<br />Recommendation:<br />Remove the rigid approach of twelve linear sessions that restrict the user from key concepts that may be helpful at the outset. <br />4. Nothing is done to encourage the user to return on a weekly basis.<br />Although users are told they should return to the site weekly, nothing is done to promote this behaviour. <br />Recommendation:<br /> If the metaphor of twelve sessions is kept, design techniques for getting users to return to the site, e.g. a reminder email once a week.<br /> <br />
Usability Evaluation<br />5. Navigating around the site is difficult.<br />The structure of the site is confusing. Several items can be found in multiple areas of the site. Because concepts are spread out over so many pages of the site, it can be difficult to remember where to find something. The labels used in the navigation do not help. <br /> Recommendation:<br /> Revisit the information architecture of the site. Keep concepts like “goals” in specific sections, rather than having them spread out over a number of pages<br /> <br />Assess whether labels are sufficiently intuitive. <br /> <br />6. There is no way to communicate with other participants of the site.<br />Today, the internet is all about communication. In the world of “Web 2.0”, site visitors are used to creating their own content, and interacting with others. By not giving program participants any way to communicate with each other, sharing challenges, successes and tips, the site is missing out on a great opportunity. Recommendation:<br /> Build increased social networking opportunities into the program.<br /> <br /> <br /> <br />
Usability Evaluation<br />9. Line lengths are extremely long, making text harder to read.<br />In typography, the optimal length of a line of text is 66 characters<br />Recommendation:<br />In redesign, ensure line length is consistent with usability best practices.<br /> <br />
Current Status and Plans<br />Revising architecture since last May, working with usability experts<br /><ul><li>Iterative, including copy writing</li></ul>Expert review in September with behavior change experts sent us back to the drawing board.<br />Revised architecture and wireframes and copy.<br />Just completed consumer testing.<br />Programming quotes and go/no go decision.<br /> <br />