Hwap  pres__w bri cugelman nov2010
Upcoming SlideShare
Loading in...5

Hwap pres__w bri cugelman nov2010



Overview of the Heart&Stroke Healthy Weight Action Plan; Lessons Learned from Developing this online tool, and early evaluation learning.

Overview of the Heart&Stroke Healthy Weight Action Plan; Lessons Learned from Developing this online tool, and early evaluation learning.



Total Views
Views on SlideShare
Embed Views



2 Embeds 146

http://www.moflow.ca 144
http://feeds.feedburner.com 2


Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • As context, I am going to give you a whirlwind tour of the toolWill focus on the process and our learningif time, take you through the learning on the from the livetool.
  • 12 weeks of lessons to walk user through key steps of behavior change. Virtual counsellor is providing the structure.
  • Demonstrations of personal feedback and interactivity
  • Based on age and gender (carried forward from UA) and activity level
  • User is taught appropriate level of calorie restriction to lose weight,Based on our reco’s and what they have learned, user selects their target weightApproach: try to educate. Not just formula’s and cells to fill in.
  • Feedback to help user set up goals
  • IN several subsequent lessons, user will get a chance to create various components of their plan such as challenges, rewards, support.Example: challenge & solution
  • Can add and edit goals
  • Section to allow tracking
  • My plan- place to consolidate whole plan, and make edits
  • Vendor that did the lit search, went on to develop the toolSuch a big decision needs active debate at every go/no go stage
  • Context:Med size org with modest budgetsHealth promo staff, not software development or website design experts.
  • 6 month delay
  • Extensive data—show a preview just to give you a sense of what is possible
  • lesson 2 - only 23% completing the lesson (of those that started lesson 2). This is a very long lesson.How to solve this, without compromising the science
  • Next we did usability
  • Dr. Bob Bensley as Project lead: a Professor of Community Health Education; PI on over $1 million in grants focused on Internet-based behaviour changeBA in Kinesiology and Physical Education from Wilfrid Laurier University and a MA in Kinesiology and Health Science at York University. Obesity guide, food guide, PA guide, DRI’sMr. Rivas has degrees in community health education, exercise science, and public administration.

Hwap  pres__w bri cugelman nov2010 Hwap pres__w bri cugelman nov2010 Presentation Transcript

  • My Healthy Weight Action PlanLessons Learned
    November , 2010
  • Agenda
    Why an etool?
    Tour of the tool
    Process steps and learning
    • “If I were to do it again”
    4. Post launch live learning
    • Usability
    • User data and feedback
    5. Enhancements
  • Why an Healthy Weight etool?
    Weight was a strategic focus
    Needed to strengthen HSF consumer information in this area
    Have had immense success with three prior etools:
    • Blood Pressure Action Plan
    • Heart&Stroke Risk Assessment
    • E Mail Coach Program
    • Incorporate most promising practices into our health information
    • Increase engagement and interactivity of our health information
  • Program Tour
  • Weekly Lessons
  • 12 Lessons
    Goal setting
    Overcoming challenges
    Support system
    Behaviour contract
    Time management
    Maintaining/new beginning
  • Lesson 2- A Key Lesson
    • Feedback on week of self-monitoring
    • Recommended caloric intake, number of servings
    • Learn to set goals
    • Personalized recommendations re what to work on
  • Tool Sections and Features
  • Process Steps and Learning
    Phase 1 literature search and environmental scan
    Phase 2 Design and Build
    • Vendor selection
    • Contract negotiation
    • Design and build process
  • Lit Search & Env’t Scan
    What tools are out there?
    What is available to buy? License?
    How effective are they? What is the research evidence?
    1. RFP- to researchers and academics active in consumer eHealth tools
    2. Criteria:
    Expertise: behavior change; online interventions; technology;
    Price and timing
    Potential for involvement in future phases
    No conflict of interest. (i.e. able to provide a unbiased review)
    Commitment to the project.
    3. Cross Foundation Review Panel
  • Lit Search & Env’t Scan- Conclusions
    Hundreds of diet/weight e-tools
    • most are highly commercials; generally either:
    • simple counters & assessments or
    • tools that inundate with features and gadgets
    • none theory based; no structure
    Nothing suitable to buy
    ->Developed recommended practices
    ->Decision to proceed
  • If I were to do it again….
    Would stipulate at lit search RFP stage, that if we proceed to build, the author of the lit search cannot take a lead role (researchers have vested interested)
    2. Would challenge conclusions a bit more.
  • Design and Build- Vendor Selection Process
    • to researchers, academics, commercial vendors in consumer eHealth
    • internal and external experts and stakeholders to assess proposals
    Weighted Criteria:
    • Project Approach & Plan – 35%
    • Qualifications of the vendor- 30%
    • Quality, clarity and responsiveness of proposal – 10%
    • Financial stability of vendor – 5%
    • Cost quotation and IP needs – 20%
    Did not give one vendor full budget including design and programming (ie. have them subcontract) due to concerns regarding contract size.
    Vision: Product design vendor would lead the selection of design and programming vendors, and HSFO would hold the contract.
  • If I were to do it again….
    1. Keep the formal criteria and workgroup for vendor selection
    • Helped with the quality and effectiveness of decision process, as well as buy in
    • Decision of this size not be made by one person or one department,
    • Provides transparent process
    2. Would give one vendor full project responsibility- from concept design, to graphic design through to implementation
    • having 3 separate contracts allowed for vendors to avoid/debate responsibility “ie. that is their job”
    • Define deliverables and roles in detail.
    3. Seek a vendor with full capability (from concept – implementation)
    • Particularly when client does not have internal expertise or capacity
    • Ensure vendor has expertise on projects of same scale
    4. Look more closely at the commercial sector.
    • More likely able to provide the IA and technical services required; subject matter knowledge can be brought in easily.
  • Contract Discussions and IP Rights
    Contract Discussions:
    • Took 4 months; $20K in legal fees; IP rights as a sticking point
    • Vendor perceived IP rights where we thought there was none
    • Agreed royalty free rights for the life of the program
    If I were to do it again…
    • Increase clarity at the RFP stage re IP ownership
    • Be aware of Universities drive for IP ownership
    • If IP becomes barrier at contract stage: be firm, willing to walk away.
  • Timing and Stages
    Design and Copy Phase: Fall 2007
    Wireframes and copy
    Selected graphic design and programming vendors
    • Specs written by lead vendor
    • December: Handing off to Design Vendor
    Graphic and interface design: December-January 2008
    Full detail (architecture and specs) not outlined or tested at time graphic and interface design was developed
  • Timing & Stages
    Final Rules Development:
    Lead vendor was to apply “final rules” to design prior to hand off to programmer
    • Questions re how best to move forward lead us to engage web developer to review work
    Conclusion From Review Process:
    • IA insufficiently defined
    • Architecture and navigation needs simplification
    • Need to pare back scope of tool
    Spring-Summer 2008 spent defining IA
    • Programming started Fall 2008
    • Product tested and completed June 2009
  • If I were to do it again….
    1. Ensure roles are exhaustively defined and scoped out. Revisit them frequently.
    • Role definition is that much more urgent if you work with multiple vendors.
    2. Engage a dedicated Project Management expertise and capacity. PLUS, plan on extensive staff involvement for strategic guidance.
    3. Do not proceed to interface design, until the wireframe and requirements are fully defined .
  • If I were to do it again….
    4. Need detailed wireframes or designs for every unique page and every page that is interactive.
    5. Invest the time and money to test a paper prototype.
    6. Keep scope modest so that you can finish design and programming in that time frame.
    • Too large undermines the ability to generate momentum
    • Too overwhelms and drags down morale
    • Market changes
  • Post Launch Live Learning
    User data
    Usability research
  • Participation and Attrition
    Great interest in the tool.
    Key attrition areas identified for attention
    • Landing page to risk assessment; risk assessment to registration
    • Only 38% of users to landing page proceeded to RA
    • 64% or users finishing RA we on to register for HWAP
    • Tradeoff :value of data versus the extra effort and time required of user.
  • User Demographic
    Gender: 90% are female
    Age: 76% are 35-64
    Ethnicity: Majority (90%) are Caucasians;
    1% Chinese, 3% S Asian, 2% African Heritage, 2% Aboriginal
    BMI: 30% have overweight BMI 25-29.9;
    49% obese BMI 30+
  • Participation over time
  • Lesson Feedback- “How did you find the lesson?”
    Majority responded positively suggesting the contents of the lesson itself are positive.
    Negative feedback fell into several categories
    Nothing new/expected more: (25%)
    Concerning motivation (e.g., “didn’t do it, as not inspired,” “it doesn’t keep me motivated”): (15%)
    Navigation or technical issues: (14%)
    Program did not meet needs/unmet need: (10%)
    Negative comments (e.g., “boring”, “didn’t work”, “too short,” “): (8%)
    Weight management is a personal responsibility : (5%)
  • Article Feedback & Journaling
    Articles: viewed an average of 1.6 times per user,
    • a small proportion (4%) of users who opened an article rated it
    • those that did rate, tended to rate the articles as being good
    Less than 3% of users journalled
    • total of 624 journal entries.
    • Majority made only 1-2 entries; a few used journal function 6 times+
    • Recognition of the importance of tracking
    • The food tracker made users feel discouraged (too many frowns)
    • The benefits of making healthier choices (“feeling happy and energetic”)
    • Frustration that “doing good” does not necessarily translate to the scale
    • Description of challenges to making healthier choices
    • Recognition of the problems of emotional eating
    • Health concerns “Cardiologist told me…”
  • Goal Setting Practices
    • 2600 users created goals.
    • Poor understanding of what constituted a SMART goal or what was meant by a “task.”
    • Did not use the goal system as intended (either due to confusion or choice)
    • Many picked up on HW PLAN concepts in creating their goals
  • Usability Evaluation
    The value proposition of the site is unclear.
    Not immediately clear what the value of using this site is, and why one should use it rather than another online weight loss site.
    • Ensure clarity: what HW Plan offers versus other online weight loss tools.
    • Improve execution and communication of the program “Point of Difference”.
    2. The use of the counselor (in its execution) has caused usability issues.
    The reason for the presence of the counselor is unclear, and her lack of identity makes her presence and lesson language uncomfortable, phony and cumbersome. Static execution of the counselor is not consistent with internet trends/capabilities
    Remove the counselor from the site entirely.
  • Usability Evaluation
    3. The linear format of lessons does not translate well to the web.
    The general premise of the HW Plan, which directs users through twelve distinct online lessons in sequence, does not work well. Users are forced through content that is not interesting or relevant to them, and must make a decision: continue forward, waiting for later content up that does look interesting, or leave the site, to find more interesting content elsewhere.
    Remove the rigid approach of twelve linear sessions that restrict the user from key concepts that may be helpful at the outset.
    4. Nothing is done to encourage the user to return on a weekly basis.
    Although users are told they should return to the site weekly, nothing is done to promote this behaviour.
    If the metaphor of twelve sessions is kept, design techniques for getting users to return to the site, e.g. a reminder email once a week.
  • Usability Evaluation
    5. Navigating around the site is difficult.
    The structure of the site is confusing. Several items can be found in multiple areas of the site. Because concepts are spread out over so many pages of the site, it can be difficult to remember where to find something. The labels used in the navigation do not help.
    Revisit the information architecture of the site. Keep concepts like “goals” in specific sections, rather than having them spread out over a number of pages
    Assess whether labels are sufficiently intuitive.
    6. There is no way to communicate with other participants of the site.
    Today, the internet is all about communication. In the world of “Web 2.0”, site visitors are used to creating their own content, and interacting with others. By not giving program participants any way to communicate with each other, sharing challenges, successes and tips, the site is missing out on a great opportunity. Recommendation:
    Build increased social networking opportunities into the program.
  • Usability Evaluation
    9. Line lengths are extremely long, making text harder to read.
    In typography, the optimal length of a line of text is 66 characters
    In redesign, ensure line length is consistent with usability best practices.
  • Current Status and Plans
    Revising architecture since last May, working with usability experts
    • Iterative, including copy writing
    Expert review in September with behavior change experts sent us back to the drawing board.
    Revised architecture and wireframes and copy.
    Just completed consumer testing.
    Programming quotes and go/no go decision.
  • Development team and science basis
    • Project design lead by WMU
    • Team of 2 Canadian RD’s; reviewed by HSF RD
    • PA content reviewed by:
    • a certified personal trainer with experience in community recreation,
    • a volunteer with post grad in Kinesiology
    • Content experts:
    • Review by CBT and Obesity Expert
    • Product design consultation provided by consultant with expertise in e intervention design and research
    • Existing Canadian guidelines and available research
    • Expert Advisory Group: High level advice