Successfully reported this slideshow.
Your SlideShare is downloading. ×

Open Inclusion Research London a11y Meetup May 2018

Ad

Top down and bottom up:
Gaining insights from the
people who know best
London Accessibility Meetup - GAAD 2018
17 May 2018...

Ad

Open’s essence
We identify and remove hurdles or friction
between customers and product or service
providers so that both ...

Ad

Real stories plus real numbers can drive real change

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Check these out next

1 of 42 Ad
1 of 42 Ad

More Related Content

Slideshows for you (19)

Similar to Open Inclusion Research London a11y Meetup May 2018 (20)

Open Inclusion Research London a11y Meetup May 2018

  1. 1. Top down and bottom up: Gaining insights from the people who know best London Accessibility Meetup - GAAD 2018 17 May 2018 Tom Pokinko Christine Hemphill Open Inclusion
  2. 2. Open’s essence We identify and remove hurdles or friction between customers and product or service providers so that both can achieve to the fullest of their ambitions.
  3. 3. Real stories plus real numbers can drive real change
  4. 4. Jane’s story • Jane is a very independent, professional working woman with a family. She has MS • During a particularly bad spell she was unable to get out of bed for a few months • Her usual supermarket didn’t offer an online experience inclusive enough given her limited dexterity and sight at the time • She tried a new provider. Beside an easier online experience, they also did delivery to the kitchen rather than to the front door
  5. 5. Being able to use my phone to order my groceries when I was bed bound was a great service. I could choose what I wanted, and didn’t have to disturb family and friends. I’m a fan! Features that I found so accessible when I was ill have led to me recommending this service to other people in a range of different situations.” “
  6. 6. Jane’s story is one experience. n = 1 How many people / experiences does it represent? • This story relates to a person with Multiple Sclerosis (MS). • Over 100,000 people in the UK have MS • It is a fluctuating condition that can variably impact energy, dexterity, mobility, sight, speech and/or mental health Mobility 19% impaired 10.4m 15% disabled 6m 19%
  7. 7. Jane’s story is one experience. n = 1 How many people / experiences does it represent? • This story relates to a person with Multiple Sclerosis (MS). • Over 100,000 people in the UK have MS • It is a fluctuating condition that can variably impact energy, dexterity, mobility, sight, speech and mental health Dexterity 10% impaired 5.7m 8% disabled 4.5m 10%
  8. 8. Jane’s story is one experience. n = 1 How many people / experiences does it represent? • This story relates to a person with Multiple Sclerosis (MS). • Over 100,000 people in the UK have MS • It is a fluctuating condition that can variably impact energy, dexterity, mobility, sight, speech and mental health Stamina / breathing 16% impaired 8.6m 11% disabled 6.0m 16%
  9. 9. This story is not unusual in that the customer has multiple needs Impairments are far more common in multiples than just one. Only 34% of people with an impairment have only one. More than 40% have three or more impairments. 8 7 6 3.9% 5 6.6% 4 11.2% 3 18.8% 2 22.9% 1 33.9% % of people classed as disabled wit h mult iple impairment s 0.2% 33.9% % Of Disable..
  10. 10. Individual experience can engage hearts and minds across a business – especially those tasked with designing and delivering brand experience Ongoing research (regular stories) in combination with broader statistics allows these rich stories to become valuable decision support insights
  11. 11. Customers know best…
  12. 12. But despite our best efforts, do we ask them enough? “ I don’t know what it is. It seems deaf people often fall through the net when it comes to accessibility. Why don’t they [companies] just ask us? ~ Ed Open Panel Community Lead Hearing Impairment
  13. 13. So how can we generate personal stories and insight? There are many different ways to ask
  14. 14. (image: Steve Krug, Don’t Make Me Think) Knowing how to ask (and when)
  15. 15. Q: When should you do a “reality check” with users?
  16. 16. Get user insights early to avoid costly changes COSTTOMAKECHANGES PROJECT ENDPROJECT START
  17. 17. Case studies 1. Paper prototype stage 2. Component testing 3. Prototype 4. Beta and live stage
  18. 18. Case study #1 (Paper prototype) SURVEY WHAT WE TESTED: Prototype iOS app that presents stock market graphs in sound form, so blind traders can understand patterns and trends in data
  19. 19. Case study #1 (Paper prototype) SURVEY “I think this is a fabulous idea because I can see it in my head.” “I was skeptical at first. Personally, I’ve never invested directly in the stock-market, but I could see that this would be a useful way to help spot trends in stock market variation.”
  20. 20. Case study #2 (Components testing) SURVEY WHAT WE TESTED: Accessibility and usability of specific UI components • Back ‘button’ - which gesture for blind users? • Home – how to get to home page?
  21. 21. Case study #2 (Components testing) SURVEY
  22. 22. Case study #3 (Prototype testing) IN-CONTEXT WHAT WE TESTED: Real-time assistance functionality using an iPhone app for booking assistance during rail journeys in England
  23. 23. Case study #3 (Prototype testing) IN-CONTEXT “The app feels personal and friendly. I wasn’t expecting to feel that closeness to the staff, because it’s not needed strictly for the journey. But actually emotionally it is needed.” “If people like me assigning passengers had the app, that would save me going upstairs or radioing to my colleagues to find out who to expect and when…” (assistance staff)
  24. 24. Case study #3 (Prototype testing) IN-CONTEXT What was your favourite feature of the app?
  25. 25. Case study #4 (Beta/live testing) USABILITY TEST WHAT WE TESTED: The European and UK websites of a major electronics retailer
  26. 26. Case study #4 (Beta/live testing) USABILITY TEST “I couldn’t get through to buy the product. It’s good there are pictures but it feels too much like a promotions feed.” “I can’t get past the cookie message!” “I don’t like ‘link find out more’ - find out more about what?” “Fonts too small, poor contrast. What does the hand icon mean?”
  27. 27. Quantitative Research Data driven intelligence Leading to better decisions and better outcomes
  28. 28. Research can be done at many levels of scope and styles specific to resolving the question at hand. • Product level insight • Organisation level – business maturity • Leading practice reviews • Market / industry volumes and value • Customer segment volumes and value • ROI modelling
  29. 29. It’s not what you don’t know that gets you in trouble. It’s what you know that just ain’t so. Mark Twain “ Learning or unlearning
  30. 30. We have a robust, actionable dataset with a relatively complete picture of the prevalence and value of access needs in the UK. In God we trust All others bring data. “ W. Edwards Deming
  31. 31. Quantitative research • How many people in your user base are impacted by impairments – By category and severity – By adaption approach – By co-morbidity • Value of this group – overall and sub-segments • Digital usage and device preference • Age, gender, other demographics • Align the data to your customer database or target market
  32. 32. Where can we get good primary data that we can trust? Study Source Style Breadth (n) Last released Family Resources Survey DWP / ONS, GOV.UK Annual (since 1994) Cross-sectional not longitudinal 19,000 households Adults 16 y.o. + March 2017 Understanding Society ISER, Uni. of Essex Longitudinal Annual (since 2009) 40,000 households Wave 7 November 2017 Life Opportunities Survey DWP / ONS, GOV.UK Longitudinal from 2009-2014 in 3 waves 19,950 households No longer run Completed 2014 Labour Force Survey DWP / ONS, GOV.UK Annual (since 1973) Rotating panel each for 5 quarters 89,000 individuals in 37,400 households February 2017
  33. 33. Myth 1 – it’s a small market 34% of the UK adult population have an impairment That is 1 in every 3 or 18.6m adults
  34. 34. Myth 1 – it’s a small market 24% of the UK adult population are classified as “disabled” That is 1 in every 4 or 12.8m adults
  35. 35. Understanding each impairment is important 1 in 5 19% All mobility impaired 10.4m Mobility as a disability 8.2m 1 in 7 15%
  36. 36. Myth 2 – it’s not such a valuable segment £265 billion In 2016 The ”Purple Pound” is worth £265 billion That is +15% of all UK discretionary income This is based on all impairment categories • Physical: mobility, dexterity • Sensory: sight, hearing • Cognitive: memory, learning & mental health
  37. 37. Myth 2 – it’s not such a valuable segment Given it is so big and so valuable, why is is this not better known? Complexity, visibility and value perceptions all limit more general understanding of this massive market segment.
  38. 38. National averages can be useful. Population BY CATEGORY Impaired Disabled Impaired Disabled Sight Impaired 280,916 203,653 £6,149,822,697 £3,965,202,162 Hearing Impaired 303,917 223,304 £6,842,989,594 £4,677,933,098 Mobility Problems 961,770 755,214 £21,205,112,769 £15,355,014,068 Dexterity Problems 524,095 416,015 £11,746,005,985 £8,631,486,758 Learning Difficulties 219,508 179,313 £5,022,339,333 £3,832,279,934 Memory Difficulties 319,101 250,770 £6,930,512,786 £5,072,583,627 Mental Health problems 477,424 363,986 £10,774,504,042 £7,324,844,804 Stamina/ Breathing problems 793,399 559,377 £18,730,566,746 £11,635,031,932 Social / Behavioural Problems 97,807 80,836 £2,348,142,432 £1,824,307,981 5,000,000 Population Value ofHH income
  39. 39. Myth 3 – they’re not my target customers / staff / users Age wealth bias: Disability is incurred increasingly with age. In the UK Baby Boomers have the highest assets and disposable income. Older people, especially younger old, are increasingly opting for digital channels. Design for us all: Mobile technology means we all use inclusive technology when we can’t see, hear, touch or otherwise use our devices due to situational impairments. We are getting more used to having and adopting user-defined interaction options. Employment is higher than you know: 1 in 5 employees have an impairment. In most workplaces only 3-5% share that with employers.
  40. 40. How do I use inclusion data along with user insights to make better product decisions? Get insight Prioritise Improve Measure
  41. 41. Qualitative rich stories plus quantitative numeric insight Can drive real change in customer experience and value
  42. 42. For more information, please contact: christine@openinclusion.com tom@openinclusion.com better experiences for all

Editor's Notes

  • I’m Tom and this is Christine. Today we’d like to talk to you about when to use qualitative and quantitative approaches in inclusive research to help improve business decisions and outcomes.
  • We gather feedback from real disabled and older users to gauge where products or organisations are on their inclusion journey, that allows us to provide insight and find efficient solutions to improve experiences

    Essentially we help map out the speed bumps along various customer journeys
  • Detailed qualitative rich stories + perspective from volumes and value of people that may have similar experiences = strong engaging prioritised insight

    That can drive powerful change momentum in business, increasing value driven by real change in satisfaction, loyalty and value for your customers
  • For example, Jane is someone with whom we work closely on projects. She’s a member of our panel that helps deliver insights to businesses.

    By using a service that accommodated her needs, she became a strong brand advocate.

    Jane has MS. She had a bad period when she couldn’t get up and about easily so swapped to doing her grocery shopping online
    She swapped from her previous standard brand as their website wasn’t able to support her needs effectively
    The company she switched to also provided a better quality, more reliable and friendly home delivery service than she had experienced before
    After she recovered from that spell she stuck with the new brand, and has singing their praises every since
  • They helped her at a time she appreciated it most. That is great customer service.

    She also found other suppliers annoying when she couldn’t use their app or their service punctuality was not as good…

  • Source Family Resources Survey 2015/16 (Released April 2017)

    Impaired = “illness, disability or impairment which causes substantial difficulty with day-to-day activities”
    Disability = Long standing (expected to or has already lasted for more than 12 months)

    This is NOT the social model of disability but it is in line with the Equality Act 2010 definition and in line with the questions asked of survey participants.

    10.4m = More than the populations of London + Birmingham + Manchester

  • 5.7m = More than the UK households subscribing to Netflix
  • 8.6m = just more than the 8.1m viewers that tuned in the UK last Saturday night during the height of Eurovision finals in Portugal
  • Multiple Sclerosis by its nature has multiple, fluctuating impacts that generate co-occurring access needs
    Treat people as whole people don’t simplify them into single access needs boxes. Be supportive of multiple adaption options

    The reality is many people have a mix of co-occurring needs. The needs here are just long term ones, even more true once you add situational needs.
    Therefore design that solves for one need in isolation is likely to be insufficient
    We have just heard from Molly who is a smart, tech savvy lady with Usher’s Syndrome. She has product design needs that take both hearing and sight adaptations into account.

    Some of the most common co-morbidities are:
    mobility along with stamina, dexterity, memory, hearing then sight
    Then Dexterity and memory
    Memory and stamina
    Mental health and stamina
    Mobility and learning difficulties
  • A large crowd at a festival in Germany. A diverse group.
    Within the crowd are older people who don’t identify as disabled, younger neuro-diverse and other non-visibly disabled such as people with speech or mental health conditions, people with temporary and permanent physical or sensory impairments etc. Also carers, family and friends of these people. Many people within the crowd with many varying needs, preferences and adaption approaches to a single environment.
  • Because most disabled users have been solving access problems throughout their lives, they are uniquely placed to give useful feedback on problems.
  • Unfortunately, not enough organisations ask disabled people for their input.
  • Research we provide includes behavioral and attitudinal approaches, quantitative and qualitative styles. There are many research formats within these ranges.
  • Don’t mistake Attitudinal for Behavioural research! Use the right research tool at the right time.
  • Get input throughout the project development cycle. There are numerous opportunities throughout, at each stage.
  • A good rule of thumb is to get user insights early…

    At Open, we know the strategies for getting insight quickly, effectively, relatively cheaply. Insight now = savings tomorrow (avoiding costly changes)

    2% to over 100% of total project costs depending on when you start to consider it
  • So now I’d like to share with you a few case studies that illustrate how we’ve managed to capture real user insights from disabled people at various stages throughout product development cycle, and in that way help determine the most inclusive business decisions and outcomes.
  • HOW WE TESTED IT:
    We described the interface and main user journey to blind members on our research panel
    Asked questions about accessibility, usability and the general concept behind the app
    We also asked them to download a similar app with less functionality from the App store
  • Feedback from disabled users allowed us to confirm that this was a good idea. Most users were unaware they might need this kind of app, but they quickly realised its potential and then helped suggest ways to make the UI even better for them.

    10-20 users helped us realise that sonification in banking apps has legs – and now the app we created has received a commendation from the Design for All Foundation and is up for an award.

  • HOW WE TESTED IT:
    Described the activity and VoiceOver output; asked would you prefer to swipe right to go back, swipe left, etc?
    Provide a link. “Please explain what you clicked to get to the homepage.”

    Back button – which gesture is best for blind users using VoiceOver?
    Home button - do older and neurodiverse users prefer to click on a company logo to go to the homepage, or do they prefer a Home link in the main nav?
  • We found swiping left to go back was a more popular option than swiping right. Some users also liked the 2 finger upward swipe. On the home button issue, in general our findings supported the idea that you should have a separate home link in the main nav rather than relying on a logo to serve as the link to home.
  • HOW WE TESTED IT:
    In-context usability testing of app at rail stations and on train
    Accompanied testing (contextual enquiry)
    Interviews with station staff
  • We found the accessibility of the app was already quite good, but key touchpoints in the customer journey made or broke the user experience. Staff needed to be more aware of the app and different levels of staff needed to work together to help create the best customer experience using the app. Also, a key suggestion we got from staff while we were testing the prototype in context of real rail journeys was: we all have smartphones. Put the app on our phones to cut out the middle man!
  • We found the accessibility of the app was already quite good, but key touchpoints in the customer journey made or broke the user experience. Staff needed to be more aware of the app and different levels of staff needed to work together to help create the best customer experience using the app. Also, a key suggestion we got from staff while we were testing the prototype in context of real rail journeys was: we all have smartphones. Put the app on our phones to cut out the middle man!
  • HOW WE TESTED IT:
    In-lab usability testing with 5 disabled users
    Blind
    Visually impaired (screen magnifier)
    Older/less tech savvy
    Dyslexic
    Mobility/dexterity impairment
    Both desktop and mobile
  • Unfortunately, there was much in need of fixing both accessibility wise and from the perspective of user experience.
    But we could also confirm that the brand was respected and viewed positively by our disabled testers. The client was very grateful for our testing because we identified so many ways their site could be better.

    Steve Krug: “You can find more problems in half a day than you can fix in a month.”

    So these case studies illustrate some of the ways that probing deeply with real users – across attitudes and behaviours - allowed us to gather actionable insight that then helped improve products and services at all stages of product evolution.

  • Now I’d like to discuss some of the quantitative research that we do. So this is research with over 30 users.

    Quant is about market size and breakdown from overall to specific usable sub-segments
    A report that tells you a lot but doesn’t make decision making easier and better is useless
    Good quant research is designed, conducted and delivered to inform better decision making that will create more value for your customers and for you

    The goal is to use data and insights to:

    increase the quality of management decisions
    shorten the time to improvements and innovation

    I should add this is also the goal of qualitative research. They just go about it in a slightly different way.
  • Tom talked about the range of research options before – Quant / Qual, attitudinal and behavioural
    These are some of the approaches we provide for clients

    This is about the question it is looking to answer and the type of analysis that follows the research
  • MYTHS and misconceptions:
    - Disabled people are only a small group
    - That group is not so valuable
    - They are not my target audience

    Qualitative insight brings emotion and connection. It is great for informing design decisions, especially behavioural insight. Quant transforms that to broader business value.
    Challenge myths with real people (as Tom described earlier).

  • Turn of the century American statistician

    Another quote of his

    It is not necessary to change. Survival is not mandatory.
  • Category – eg sight, hearing, mobility, dexterity, speech, social, learning, memory and mental health
    Severity – Combination of time impacted - permanently, temporarily or situational along with completeness of impact eg Hard of Hearing vs Deaf
    Adaption approach – to follow through the haring impaired example BSL speaker, lip reader or both. For sight, uses Voiceover on mobile vs uses pinch zoom with inverted colour preferences
    Co-morbidity – solve for real, whole people with messy, genuine combinations of impairments

  • What are sources for this data? Where can we get good primary data that we can trust?

    This is for the UK. Family resources survey, Understanding society, etc.

    Other secondary sources can be good too. Eg the RNIB Sight Loss Tool covers sight loss but also falls, dementia, stroke, hearing impairment, dual sensory loss and learning impairments as well as visual.

    Despite these great sources, however, you are still trusting that their use and interpretation aligns to your use and how you would chose to interpret the raw data
    All these surveys focus on adults, defined as 16+
    Use people who understand the source, sampling method, populations and limitations.

    *** We have used reputable data science agencies where they provide the raw data feeds we need along with constraints of use and then we analyse it
  • So let’s take a look at some of the myths I identified earlier.
  • That is equivalent to the top 12 cities by population in the UK, from London to Fife
  • I won’t go through these in detail as we have them provided for you here today by the activity – with details of the relevant category for that activity

    Understanding each impairment is important; however, a single number is not enough. It is worth understanding age, gender, employment and severity of each category.

    This helps as without it it is hard to know how best to improve your service.

    If this sounds hard – get some help.
    Advocacy groups can help you to understand the group they advocate for. Businesses like ours work across all impairment categories.

    Other businesses and organisations such as many in this room, BBC, GDS and Barclays have been doing this for a while now and are happy to open source knowledge to the broader community,
    Meetups, podcasts and the like can all provide a good grounding too.
  • 2012/13 – initial survey £212b (household disposable income where at least 1 person in the household had a disability)
    2014/15 – restated by DWP to £249b
    2015/16 Latest stats done for Barclays last year £265b


  • Complexity –It’s not homogeneous This is a dynamic and multi-dimensional group. Fear is also involved (political correctness and legal risk) and not knowing how to get this right (get good insight, which will help you understand whole people and segments, get some support and get started)
    Visibility – Less than 1 in 3 impairments are visibly obvious and if you live in “bubble London” then it has the lowest proportion of people with disabilities in the country
    Value – negative assumptions and biases prevail around employment and earnings. However 22% of the workforce has an impairment. Even though only 3-5% tend to state it to their employers. New more flexible ways of working, enabled by digital and lower cost of entry to set up a business are making it increasingly easy for people to work as they need to from where they wish to and add great value – either self employed, as entrepreneurs or working in a larger business now offering more flexibility.
    Additionally impairments are acquired at increasing rates with age. The Baby Boomers have the highest holdings of assets and disposable income in the UK. They are discerning buyers with high service expectations, still spending and both use digital and adapt to new technology at a much higher rate than the last generation of retirees.

  • This is an example if a customer database was nationally representative and had 5 million customers.
    We can tailor this to align to specific known demographics such as age, gender and region

    Understanding the nuances behind the averages is important for major decisions.



  • +65 and +75 have the fastest increase in smartphone takeup / growth rate.

    75% of +65 have and use their mobile phone. Over half now use smartphones up from 1/3 just last year. Ofcom survey Adult Media Use and Attitudes

    Also the new old used technology in their working years and had to adapt to device and system change so are less afraid of “breaking the internet” than the older old.
    +40% now have a social media profile too.

    We have created breakdowns of the data
  • Spend a bit of time understanding how many people will appreciate the effort you make to create more inclusive designs and products.

    Get insight – quant and qual and align it to your specific customer or target customer base
    Align that to your target or current customer base
    Prioritise your improvements based on
    Cost and effort
    Depth of impact (how annoying is it?)
    Scale of impact (how many people are annoyed by it?)
    Make the changes
    Product level
    Organisational level – processes, tools and skills
    Measure the changes through analytics, reduced complaints and other metrics
  • Inclusive design is based on the principle that learning about diverse needs and designing for those needs helps to create better user experiences for everyone.
    Simply put, an included customer is a happy customer

    A happy customer is loyal and can also be a great advocate

    Eg one of our panel members who has MS. She had a bad period when she couldn’t get up and about easily so swapped to doing her grocery shopping online
    She swapped from her previous standard brand as their website wasn’t able to support her needs effectively
    The company she switched to also provided a better quality, more reliable and friendly home delivery service than she had experienced before
    After she recovered from that spell she stuck with the new brand, and has singing their praises every since

    They helped her at a time she appreciated it most. That is great customer service.
  • Open Inclusion. User research + insight + analysis = business intelligence + better decisions = better outcomes

×