CUTGroup 13 - mRelief Final Report

CUTGroup 13 – mRelief
In-person test on January 29, 2015
1
Introduction
For our thirteenth Civic User Testing session, we tested mRelief, a website that helps residents check
their eligibility for social services benefits in Chicago & Illinois.
Target Group:
On January 21, we sent out an email to all of our 834 CUTGroup participants. We wanted to know if
they would be available for an in-person test on January 29, 2015. When segmenting our testers we
were interested in:
 If they were currently enrolled in social services programs
 If they were not currently enrolled, if they would be eligible depending on their income and
number of household members
Test Type:
 In-person test with one-on-one proctors working with testers directly
Questions:
The mRelief team was interested in testing these things:
 Language: We heard that language is very important in understanding if someone is eligible for
social services. We wanted to see if testers understood terms such as: gross income (what to
and not to include), value of assets, household size, etc. The mRelief team also uses pop-overs
and wanted to see if testers use them and if they were helpful.
 SMS: We wanted to test the text-messaging platform to see if users found it easy to use and
how long each session took to receive and send messages
 Website usability: In addition to testing SMS, we wanted to see how testers liked the mRelief
website and how easy it was to use.

2
Part I Responses
Could you tell us your age?
Under 18 0%
18-24 27% (3)
25-34 0%
35-44 27% (3)
45-54 18% (2)
55-64 27% (3)
65 or Above 0%
How much time do you spend on the Internet a day?
1 – 2 hours 9% (1)
2 – 3 hours 18% (2)
4 or more hours 73% (8)
Do you SMS text to get information?
100% (11) – Yes
0% – No
How useful would it be for you to receive information via text?
5 Very Useful 82% (9)
4 Useful 0%
3 Neutral 18% (2)
2 Slightly Useful 0%
1 Not Useful 0%
Do you pay per text?
91% (10) – No
9% (1) – Yes
What is your cell phone service provider?
(3) – Sprint
(2) – AT&T
(2) – Boost Mobile
(2) – T-Mobile
(1) – Access Wireless
(1) – Metro PCS
Are you currently enrolled in social services?
73% (8) – Yes
27% (3) – No
If so, which one?
(8) – Food Stamps/SNAP/Link

3
(5) – Medicaid
(2) – SSI
(1) – Section 8
(1) – Home Health Care Works (DHS)
(1) – Subsidized Housing
Do you go on the internet to access social service benefits?
64% (7) – Yes
36% (4) – No
If so, can you tell us more about that experience?
3 out of 11 testers (27%) specifically mentioned checking their food stamp balance online.
3 out of 11 testers (27%) mentioned that they go online to specifically apply for benefits.
Mr_B (#7) says that he usually goes online to help people find social services, often in the
context of his job.
Meearene (#3) says, “I’m more tech savvy… Accessing by the internet is easier.”
Daily Computer User (#4) does not currently access social services benefits online and uses
email mostly. “It would be useful for me to access social service benefits online. I may have
applied for my Link.”
Review the homepage. What do you think this Web site does?
When reviewing the homepage, 6 out of 11 testers (55%) mentioned the graphics. 3 testers felt that the
icons were very intuitive and understood that they connected to the social services benefits. The other
3 testers did not think the icons were as intuitive.
Checking Eligibility (#11) says, “Building might be a school... Bottle might be for children… It gives
you transportation times, maybe check where the nearest grocery store is…”
In addition, testers Single One (#9) and Global Citizen (#10) thought the images were icons that
could be clicked on. In addition, when we asked testers to explore the first site and tell us what the first
thing to do, Very outgoing and ambitious (#12) and Checking Eligibility (#11) also tried to click
on the icons first.
Single One (#9) did not think the symbols were relevant or intuitive. Global Citizen (#10) says, “I
don’t like the icons – they look like they should be school. What is this bottle for? It looks like stuff for
kids.”
Explore the site. What's the first thing you do?
82% (9) – Clicked “Get Started”
18% (2) – Clicked “Programs”

4
Tell us more about the first thing you did.
The majority of testers clicked on the “Get Started” button, and saw it is the logical next step after
reviewing the homepage. Here are some responses from the testers:
Daily Computer User (#4) chose “Get Started” because “I was instructed to [by the
website]. I liked the experience, it was fine, friendly and moved me forward.”
Parent already has benefits (#1) chose “Get Started” because “it is the first button you
see.”
Global Citizen (#10) shared that “Clicking on Programs is the most logical way to find out
what the site is about.”
Checking Eligibility (#11) said, “I figure I would get started or click one of the icons.”
Go to the "Programs" tab and choose one that is relevant to your needs. Which
one did you choose?
(3) – Food Stamps
(3) – Rental Assistance
(2) – RTA Ride Free
(1) – Cash Assistance
(1) – Child Voucher
(1) – AABD
(1) – Medicaid
Go through the process of seeing if you are eligible and tell us about your
experience.
8 out of 11 testers (73%) thought that the process was easy, straight forward, and clear.
Advocate for the visually impaired (#2) says, “The form seemed like an easy to use form. It was
helpful to have all of the right questions there, and the calculator was helpful. It is accessible for disabled
people.”
Single One (#9) says “Follows the same flow as the paper questionnaire. [The] Flow of logic made
sense.”
We also saw that a lot of testers liked the idea of the calculator, but most testers did not use if because
they had already answered the questions before coming across the calculator.
Parent already has benefits (#1) went through the process for the Medicaid program, and since she
was not eligible she received links to Get Covered Illinois and Healthcare.gov. She mentioned that
“When it links you out it would be nice to link directly to relevant page/form instead of the Get
Covered Illinois home page.”
We also had a few experiences with testers when filling out the form that they did not notice variations
in the question regarding gross monthly, 90 day, or annual income. For example, Very outgoing and

5
ambitious (#12) put in her 30 day income instead of 90 day income, and only changed it when the
proctor mentioned it.
Did you click on pop-overs for additional information?
91% (10) – No
9% (1) – Yes
Was this process easy?
100% (11) – Yes
0% – No
Please tell us how you might improve the application questions.
7 out of 11 testers (64%) mentioned that the questions were easy to understand or straightforward.
Daily Computer User (#4) said, “It was surprisingly short, sweet, and efficient. Sometimes you get
frustrated filling out long forms. Basic questions should get to the point so that you can proceed or
stop.”
Single One (#9) said that the language was “not foreign” and she expected that it would be language it
would be familiar with.
Checking my status (Tester is disabled) (#6) thought that it was easy, but asked “where would I
go from here?” It took her to the Illinois Department of Aging, but it did not provide the tester with
instructions. She thought it would be better to go to this page instead:
https://www.illinois.gov/aging/BenefitsAccess/Pages/Ride-Free-Transit-Benefit.aspx
3 testers mentioned it would be better to update the income language in the application. Parent
already has benefits (#1) thought that the pop-over information should be moved over to the main
text. Meearene (#3) said, “What is the household gross monthly income?” would be a better way to
explain it.
Global Citizen (#10) wanted better information on the TANF form about relationship with children,
and was not sure if this included caring for godchildren.
Tell us about your eligibility and the resources provided. What is the next thing
you would do with this information?
7 out 11 testers (64%) said they would call the phone number to learn more.
3 out of 11 testers (27%) said they wanted to know more about why they were or were not eligible for
social services benefits.
3 out of 11 testers (27%) said they wanted to be taken directly to the application to apply for the social
services benefit. Global Citizen (#10) adds that “It would be nice to prepopulate the form with the
eligibility information entered into the Illinois State page.”

6
Check your eligibility via SMS: Text "hello" to 1-773-377-8946. Which program do
you find more information about?
(8) – Food Stamps
(3) – RTA Ride Free
Go through the process of seeing if you are eligible and tell us about your
experience.
5 out of 11 testers (45%) thought that the questions were “easy” or “straight-forward.”
4 out of 11 testers (36%) were frustrated that they could not go back after making a mistake. Simple
One (#9) also added that it was “annoying” that she did not know when the end was.
3 out of 11 testers (27%) mentioned that the process went really fast.
2 out of 11 testers (18%) experienced the last text message coming before the first one. In addition,
Meearene (#3) noticed that the word “Medicaid” also got split between messages.
Was this process easy? (Text)
100% (11) – Yes
0% – No
3 out of 11 testers (27%) specifically mentioned that the questions were easy or simple. Some testers
had suggestions to improve the process:
Daily Computer User (#4) suggested that “You might want to include forms about student
loans and paratransit… It would be great if you could do SMS for applications. It would be great
to do this in one move.”
Hardest worker (#5) said that it would be “great if there was a native app for this.”
Global Citizen (#9) did not like the way 10000 is formatted in text version.
Two testers discussed how being in college affects your eligibility:
Checking Eligibility (#11) says, “I am glad you included the question about college because a
lot of people don't know that you have to be working AND in college to get food stamps.”
Very outgoing and ambitious (#12) adds that mRelief should make it clear that being in a
college does affect your eligibility. She has never seen that before in the food stamps application
process.
Single One (#9) did not think it was intuitive as the website because of being unable to correct
answers. She also wanted to know how many questions would need to be answered through text
messaging since it took a couple of minutes for each text to be received.

7
Tell us about your eligibility and the resources provided. What is the next thing
you would do with this information?
5 out of 11 testers (45%) said they would call the number of the resources. On the other hand, 2 other
testers mentioned that they would prefer to have website links or other information to research first
before visiting or calling the location.
How important is it for you to know what documents you need to prepare for an
interview with a caseworker to receive public assistance?
5 Very Important 100% (11)
4 Important 0%
3 Neutral 0%
2 Slightly Important 0%
1 Not Important 0%
Is it important for you to know why you do or don't qualify for public assistance?
4 Important 18% (2)
3 Neutral 0%
1 Not Important 0%
How important is it for you to know how much money you will receive from public
assistance before going to file with a caseworker?
4 Important 9% (1)
3 Neutral 9% (1)
2 Slightly Important 18% (2)
1 Not Important 18% (2)
How important is it for you to know how to file an application for public
assistance?
4 Important 0%
3 Neutral 0%
Do you like mRelief?
100% (11) – Yes
0% – No

8
Would you use mRelief in the future?
100% (11) – Yes
0% – No
Do you like CUTGroup?
100% (11) – Yes
0% – No
Did you like this CUTGroup test?
100% (11) – Yes
0% – No
100% (11) – Yes
0% – No
Anything else to add?
Advocate for the visually impaired (#2) recommended, “Add ZoomText or Jaws to the survey
software for the visually impaired.
Single One (#9) says “What draws me to the CUTGroup is that there is a group that evaluates user
experience. That’s important.

9
Results
8 out of the 11 testers were currently enrolled in social services benefits, and all of these testers were
enrolled in Food Stamp benefits. The 3 testers who were not currently enrolled seemed to have
qualified for social services benefits based on their household size and income.
Language
We learned from this CUTGroup test that testers found the form(s) when checking their eligibility to
be a simple process. 7 out of 11 testers (64%) mentioned that the questions were easy to understand or
straightforward. Although most of the language was very clear, however, only 1 tester clicked on a pop-
over to find out more information. Therefore, any question that requires additional help text should be
added after the question.
In addition, some testers did not notice that some questions might ask for monthly income vs others
that ask for annual income. These are key pieces of information that should be made clearer in the
question format.
SMS
Out of 11 testers, 10 testers (91%) do not pay per text message.
Only 5 out of 11 testers (45%) thought that the questions were “easy” or “straight-forward” when going
through the text messages. The biggest thing we heard was that testers were "frustrated" or "annoyed"
that they were unable to fix a mistake and had to start from the beginning.
Lastly, some testers received messages out of order, and some terms were split up between messages
(ex: "Medicaid").
1 of our testers, Simple One (#9), experienced a delay between text messages that lasted anywhere
from 1 to 2 minutes. She thought it was "annoying" that she did know when the end of the questions
would be. 3 testers thought the process was pretty fast!
Website usability
When reviewing the homepage, 6 out of 11 testers (55%) mentioned the graphics. 3 testers felt that the
icons were very intuitive and understood that they connected to the social services benefits. The other
3 testers did not think the icons were as intuitive. 4 testers thought the graphics were icons that they
could click on.
Most of the testers had a clear direction after reviewing the homepage. The majority (9 testers) clicked
"Get Started" while the other 2 testers clicked on "Programs." It was great to see that testers had a
clear path in using the website that got them to the most important part -- checking their eligibility for
social services.
When checking eligibility, we noticed that a lot of testers liked the idea of the calculator to calculate
income, but most testers did not use if because they had already answered the questions before coming
across the calculator located at the bottom of the screen.

10
Documents & More Information
We wanted testers to tell us how important it would be for mRelief to provide additional information
about documents that they might need to bring, or money that they would receive from the benefit. We
hoped this would be a help for mRelief to decide what the next steps for the website might be. Here is
what we learned:
How important is it for you to know what documents you need to prepare for an interview with
a caseworker to receive public assistance?
4 Important 0%
3 Neutral 0%
1 Not Important 0%
4 Important 18% (2)
3 Neutral 0%
1 Not Important 0%
How important is it for you to know how much money you will receive from public assistance
before going to file with a caseworker?
4 Important 9% (1)
3 Neutral 9% (1)
2 Slightly Important 18% (2)
How important is it for you to know how to file an application for public assistance?
4 Important 0%
3 Neutral 0%

Tester #:
1
Tester Profile (given by tester):
Parent already has benefits
Proctored by:
Josh Kalov
18-24
What type of device are you using for this test?
Dell 6440
Is this your normal device
that you use everyday?
No
If not, what is your primary device?
Macbook Air
ABOUT
YOUR
INTERNET
AND
DEVICE
USE

4 or more hours
Do you SMS text to get
information?
Yes
How useful would it be for you to receive information via
text?
5 - Very Useful
No
AT&T
Are you currently enrolled
in social services?
Yes
If so, which one?
Food Stamps
Do you go on the internet
to access social service
benefits?
Yes
Check balance on the card at Illinois EBT website
WEBSITE
REVIEW


Check your eligibilty for benefits like link, food stamps, housing
Click "Get Started"
Click on Get Started first because it is the first button you see.
Go to the "Programs" tab and choose one that is relevant to your needs. Which one did
you choose?
Child Voucher
Go through the process of seeing if you are eligible and tell us about your experience.
Child Voucher - Straight forward. Medicaid - Filled out form, may not be eligible->went to
getcoveredillinois and healthcare.gov. When it links you out it would be nice to link directly to relevant
page/form instead of getcoveredillinois home page. All City Programs- longer but a one stop.
Did you click on pop-overs
for additional information?
No
Yes
For the monthly income form maybe move the pop-over info to main text.
Tell us about your eligibility and the resources provided. What is the next thing you would
do with this information?
Apply for the eligible ones and probably call for the unsure ones. For ones do not qualify check anyway
just incase "cause you never know"
SMS
REVIEW

Check your eligibility via SMS: Text "hello" to 1-773-377-8946. Which program do you find
more information about?
RTA ride free
It went fast. Straight forward questions

Yes
It might be useful to include website links in email in addition to the phone number.
OVERALL
REVIEW

How important is it for you to know what documents you need to prepare for an interview
with a caseworker to receive public assistance?
5 - Very Important
4 - Important
5 - Very Important
5 - Very Important
Yes
Yes
In what ways would you improve mRelief?
Add more direct links to benefits applications on 3rd party websites. Add more descriptions in
parenthesis

Yes
Yes
2015-01-29 16:19:50

Tester #:
2
Advocate for the visually impaired
Proctored by:
Genevieve
35-44
laptop
No
iPad
ABOUT
YOUR
INTERNET
AND
DEVICE
USE

1 - 2 hours
information?
Yes
text?
5 - Very Useful
No
Metro PCS
in social services?
Yes
If so, which one?
Medicaid, Food stamps
benefits?
Yes
Apply for benefits, but it is not accessible to ask questions. You can
only apply for services. You should be able to be contact - you
should be able to send an email or get in contact.
WEBSITE
REVIEW


Immediately clicked on Get Started. It is a website to pre-screen for eligibility for services
Click "Get Started"
Clicked Get Started hoping to have access to the programs offered
you choose?
RTA Ride Free - it relates to his disability - he already has it, but that is the program that is relevant for
him
The form seemed like an easy to use form. It was helpful to have all of the right questions there, and
the calculator was helpful. It is accessible for disabled people. He didn't click on the popovers
because he knew what they wanted and has filled out many government forms. Add a popover to the
zipcode question to enter you address and find your popover. He asked about a calculator for gross
annual income and did not see the calculator below.
No
Yes
Questions were easy to understand. So far, it looks clean.
He would move forward and process the application. It is good that it gives you a link to the website, so
that you can go directly and apply. Different colors would make it accessible for people with vision
loss - black background and blue or yellow text. Also, add an icon to enlarge a website text for the
vision impaired.
SMS
REVIEW

RTA Ride Free

Questions were easy to answer of text messaging. It's good.
Yes
Questions were good.
He would follow up with the number to get more information. Nothing to make it better.
OVERALL
REVIEW

5 - Very Important
5 - Very Important
5 - Very Important
5 - Very Important
Yes
Yes

Add an app for iPad and Android.
Yes
Yes
Add ZoomText or Jaws to the survey software for the visually impaired.
2015-01-29 16:22:32

Tester #:
3
Meearene
Proctored by:
Sonja
18-24
iPad
Yes
ABOUT
YOUR
INTERNET
AND
DEVICE
USE

2 - 3 hours
information?
Yes
text?
5 - Very Useful
No
Sprint
in social services?
Yes
If so, which one?
SNAP Food Benefits
benefits?
Yes
SNAP website, provide social, date of birth, pin, pending benefits, it's
easy. "I'm more tech savvy" Accessing by internet is easier
WEBSITE
REVIEW


"Check your eligibility benefits" -- graphic connects to the benefits, makes sense SNAP website is very
plain just asking for information.
Click "Programs"
What programs that this site offer Graphics were appropriate for the programs offered
you choose?
Food stamps
Questions are pretty clear and "straight cut" Self explanatory Gross monthly income -- net pay minus
taxes
No
Yes
"What's the household gross monthly income?" would be a better way to explain it.
"It's helpful to give you a place to go" Recognizes the location Phone # to call is good Information
about what to bring This is very useful Wants to know why I am not eligible, maybe they need further
information... "if they could tell me to confirm" "Something else in mind!"
SMS
REVIEW

Food stamps
The end of the message came before the beginning of the message. Samsung Galaxy S4 Mini "Medicaid"

got split between the messages "Ah this is useful!" Messages came right away Simple question
Yes
Simple
She would call them because she was unsure why she wasn't eligible after just answering 2 questions
(Are you in school? Zip code?). She does not understand why they can't determine eligibility.
OVERALL
REVIEW

5 - Very Important
5 - Very Important
5 - Very Important
5 - Very Important
Yes
Yes

When exploring: Likes that adding the zip code it shows resources nearby All of the assistance
programs Head Start is relevant (did not scroll all the way down) "you may be in luck!" -- goes to
website which is helpful (direct link, faced 404 errors in the past) Very helpful, simple questions Easy
numbers Calculator is helpful Might want to contact mRelief based on website, 1 option (looking for a
second option to go to if the first location is crowded or you are familiar with another org)
Yes
Yes
Confirming time slot, and sending reminder e-mail
2015-01-29 16:26:17

Tester #:
4
Daily Computer User
Proctored by:
Rose Afriyie
55-64
laptop
No
Desktop
ABOUT
YOUR
INTERNET
AND
DEVICE
USE

2 - 3 hours
information?
Yes
text?
5 - Very Useful
No
Boost Mobile
in social services?
Yes
If so, which one?
Link, Section 8, Medicaid, SSI
benefits?
No
Email mostly; it is reasonable; I don't have any complaints -- I know
how it is you have to wait. It would be useful for me to access social
service benefits online. I may have applied for my Link card online.
WEBSITE
REVIEW


This website is concerned with the basic needs of people housing, food and transportation, and health.
Family concerns and infants and education. I get this for the icons.
Click "Get Started"
I was instructed to (by the website) I liked the experience, It was fine, friendly and moved me forward.
you choose?
AABD
It is just fine! No problem. Noticed About the Questions: I wasn't too clear about AABD before this
and this site made me more familiar. She did not choose any of the popovers
No
Yes
For me it was fine. It was surprisingly short, sweet, and efficient. Sometimes you get frustrated filling out
long forms. Basic questions should get to the point so that you can proceed or stop.
I will certainly call the organization.
SMS
REVIEW

Food
Frustrated. I made some errors myself moving too quickly and making assumptions. It is easy to use --
but if you take your time. To get food stamps, I had to answer a lot more questions. But this is much

simpler. A lot of citizens complain about the time it takes to get through process. But it's important to
ask the basic questions and get to the bottom of it.
Yes
You might want to include forms about student loans and paratransit -- go beyond free ride. It would be
great if you could do SMS for applications. It would be great to do this in one move.
I will follow up with the information generated for me for AABD; I would use it if I didn't have it. I think
this is a useful tool. I really do. Had I used this, it would help me go through social services a lot simpler.
It helps to know if they qualify for certain services.
OVERALL
REVIEW

5 - Very Important
5 - Very Important
5 - Very Important
5 - Very Important
Yes
Yes

Including other concerns.
Yes
Yes
I have said it all!
2015-01-29 16:36:45

Tester #:
5
Hardest worker
Proctored by:
Rose Afriyie
18-24
laptop
Yes
Android phone
ABOUT
YOUR
INTERNET
AND
DEVICE
USE

4 or more hours
information?
Yes
text?
3 - Neutral
No
Boost Mobile
in social services?
No
If so, which one?
benefits?
Yes
I go online to access financial aid for school.
WEBSITE
REVIEW


It helps you check and see what you are eligible for. He read the icons and saw food and where you can
get those services.
Click "Get Started"
I knew that you would have more information if you would get started.
you choose?
Medicaid
Questions are simple and straight forward -- questions anyone can answer without becoming frustrated.
No
Yes
monthly income should specify that it pertains to all members in a household
I would call and get more information. I would also do a little more research.
SMS
REVIEW

Food
It's fast! It's so small on SMS. I would rather see it on a bigger screen. For me, it is hard to understand.

Yes
It would be great if there was a native app for this. It would be much more informative because it could
provide more information.
I would go online and research the organization and try to check my eligiblity online. But I would not go
there first.
OVERALL
REVIEW

5 - Very Important
5 - Very Important
2 - Somewhat Important
5 - Very Important
Yes
Yes
I think you should make the questions more specific. I like how you have a big box with the title. It is
quick. Adding more programs -- scholarships and community organizations that might get you to some

of theses services.
Yes
Yes
No complaints.
2015-01-29 17:25:50

Tester #:
6
Checking my status (Tester is disabled)
Proctored by:
Christopher Whitaker
55-64
iPad
No
PC or phone
ABOUT
YOUR
INTERNET
AND
DEVICE
USE

4 or more hours
information?
Yes
text?
5 - Very Useful
No
Sprint
in social services?
Yes
If so, which one?
Mediaide, SSI, Food Stamps, Thresholds, Home Health Care Works
(DHS), has subsidized housing that has a social worker that goes to
the office for her
benefits?
No
WEBSITE
REVIEW

I really want to push the "Get Started" button - immediately wanted to go after it. Noticed that she is
eligible for the Free Ride Program. (She's one it) (Tester pretty much went for it - w/o waiting for me
to ask to review it.)
Click "Get Started"
(Tester pretty much went for it - w/o waiting for me to ask to review it. - She's looking at the cash
assistance page. Seems to be flying through the application.) A lot of people are asking her how to get
to the ride free program. She's on the board for the CTA.
you choose?
Cash assistance
Really easy to use and navigate, big buttons, explains what it is. I (For the AABD misses the 62 or
above - it does say AND, but "Elderly WITH disabilities. It's not Elderly OR disabilities." Usually
forgets about the pop-overs - didn't know the (i) button meant a pop over. A lot of people would like
a phone number so they can call somebody for help. (Check touch id for the programs - took a
couple of times to click.) Tester is going through the ride free application. She's eligible for ride free.
No
Yes
It's easy - but where would I go from here? It takes her to the Illinois Department of Aging, but doesn't
give me instructions. It should take me to https://www.illinois.gov/aging/BenefitsAccess/Pages/Ride-Free-
Transit-Benefit.aspx
It should take me to the application. It's two clicks away from where it should be.
SMS
REVIEW


Food
I texted food, and it's asking me questions (What does college have to do with it? - didn't understand
why it was asking) [Maybe we should put a text explaining, we're asking screening questions] (Had
a question about if SSI was social security - there's different types of SSI) [[ There was a bug.. we
texted 'Yes' and it sent a 'typo' message. }] It also should have sent her to Bellwood and not King
Community Center. King is really far from her. (She lives in Oak Park.) (60302) - when she tried
entering another zipcode, it defaulted to the 'to check other programs text menu' It does give her
another number to call.
Yes
There's those two little things and that about it. (The location and then the many forms of SSI and
Veterans Affairs.) It was easy it was understandable. All of these are frosting on the cake questions.
I'd fill out the application and it would tell me how long I would wait until I get the benefits. (I assume).
[[ It may be useful for mRelief to give an estimate on time.]] HUD is a good example. It took 10 mins
to fill it out, and got an email at once.
OVERALL
REVIEW

5 - Very Important
4 - Important

3 - Neutral
5 - Very Important
Yes
Yes
That was perfect! Does screenreaders work with the program? That has to be able to talk what the
page says.
Yes
Yes
Can I keep the iPad (kidding)
2015-01-29 17:43:11

Tester #:
7
Mr_Broughton
Proctored by:
Daniel X. O'Neil
45-54
iPad
No
PC desktop
ABOUT
YOUR
INTERNET
AND
DEVICE
USE

4 or more hours
information?
Yes
text?
3 - Neutral
Yes
Access Wireless
in social services?
No
If so, which one?
benefits?
Yes
Usually helping people find social services, often in the context of his
job.
WEBSITE
REVIEW


Prefers email to text due to service plan (email is free)
Click "Get Started"
First thing was I read "check my eligibility" and I clicked "Get started". "The first page speaks for itself".
I'm looking at this site for the work that I do pertaining to my job. He is an outreach ambassador
providing resources to the community. Head Start, WIC, rental assistance, health care. Parent
ambassador for COFI.
you choose?
Rental assistance
I chose a temporary loss of income. One dependent (me). Zero income. 60628. "When they ask me
how many dependents I have, when I click on it, the keypad should show up automatically". He reads
that he likely does not qualify. "It should tell me why I don't qualify (what it is that keeps me from
getting it) When presented with "here are some additional resources from Purple Binder", none of the
items are clickable. "The font should have been switched" ("Something else in mind!" should be smaller
and the "not eligible" should be bigger).
No
Yes
Did not see any popovers. "This is what came up"
Wants to be able to find out more. "All it gives me is the phone number for a program that doesn't have
a name. I would be looking for a link" to find out more.
SMS
REVIEW


Ride
There might be confusion between "medicare" and "medicaid". "If you make a mistake"-- "what kind of
mistake would a person make? "My question is, what number would I text the word to? The same
number, a shorter version, like CTA 41411? Instead of texting that full number again, it would be good
to have an abbreviated number" It was a total of 6 messages. Response was, "Are you 65 or older"
No Predicted that the response would be "are you disabled" (true, this happened) No Predicted
that it would say he wasn't eligible (but that didn't happen, got zip code prompt)-- got an "incomplete
message" message, then indicated that he not eligible. Wondering if the number provided for more
info (non-emergency transportation prior approval) was text or phone. Default ringtone
Yes
But the phone vs. text was complicated.
"I would call the 877 number to see what's going on with that". Not sure what number to text to.
OVERALL
REVIEW

5 - Very Important
5 - Very Important
4 - Important

5 - Very Important
Yes
Yes
"what I've used so far, it's OK, yes"
Yes
Yes
"It's very helpful"
2015-01-29 17:44:33

Tester #:
9
Single One
Proctored by:
Sonja Marziano
55-64
iPad
No
Laptop HP
ABOUT
YOUR
INTERNET
AND
DEVICE
USE

4 or more hours
information?
Yes
text?
5 - Very Useful
No
Sprint
in social services?
Yes
If so, which one?
SNAP
benefits?
No
WEBSITE
REVIEW


Friendly homepage, not too busy Symbols are not relevant Not intuitive -- eye went to graphic first
Click "Get Started"
Tried to click on the graphic first and then clicked "Get started" underneath
you choose?
Food stamps
Follows the same flow as the paper questionnaire "Flow of logic" made sense Boxes were close
together
No
Yes
The application questions are "not foreign" expects people would be familiar with the language
Spanish speaking -- appreciates that it provides this information Familiar with the community service
center -- but knows it as Auburn Gresham not Englewood (Mispelling in Englewood) Online option
first before calling or going in
SMS
REVIEW

Food stamps
Split messages, 2nd one came before the first one 1 or 2 minutes Annoying because "you don't know
when the end is" Made a mistake and realized that you can't go back and correct the mistake "LAF

Englewod" error Expected to be kicked off after replying "No" to citizenship This response was
friendlier than being kicked off or saying "You're not eligible"
Yes
Not as intuitive as the website because of being unable to correct Wants to know how many questions
you will need to answer
Would go to the location because it is her "only option" She knows that she is eligible, but made a
mistake Tried RTA Ride Free via the website: Likes the calculator Missed the first question about
disabilities but still got a response Benefit Access tab was confusing, she did not know she had to click
to another website Was not familiar with IL Department on Aging website, never checked Felt
comfortable with mRelief directing IL Department on Aging website
OVERALL
REVIEW

5 - Very Important
5 - Very Important
1 - Not at all Important
5 - Very Important
Do you like mRelief? Would you use mRelief in the future?

Yes Yes
Could not figure out how to get home More contrast with text (mainly on the banner about texting)
Icons and list what they are for -- did not realize this was a graphic Likes the background
Yes
Yes
What draws me to the CUTGroup is that there is a group that evaluates user experience. That's
important.
2015-01-29 17:47:49

Tester #:
10
Global Citizen
Proctored by:
Genevieve
45-54
iPad
No
iPhone or MacBook Pro
ABOUT
YOUR
INTERNET
AND
DEVICE
USE

4 or more hours
information?
Yes
text?
5 - Very Useful
No
AT&T
in social services?
No
If so, which one?
benefits?
No
WEBSITE
REVIEW


Checking for benefits. "I don't know what benefits me. To me that means work benefits." "I don't like
the icons - they look like they should be school. What is this bottle for? It looks like stuff for kids."
Sometimes m is uppercase and sometimes lower. Branding is not consistent. When I see donate, I think
it is a non-profit. "I don't care about Ruby developers if I am checking for benefits." It is giving facts
about who built it as opposed to the why. Share more about what the site does. The About me is
written for the wrong audience. The icons look clickable, but they are not. What does the basket
mean?
Click "Programs"
Clicking on Programs is the most logical way to find out what the site is about. The subtitles on the
program pages are helpful. She could filter out that the programs do not apply. "All City Programs" is
misleading. All City looks nice in the grid, but the box should not be in the grid. It should stand out and
be a different color. It looks like another options as opposed to comprehensive.
you choose?
Interested in cash assistance programs, but they are not relevant. She chose RTA ride free for the sake
of this survey.
Likes that you provide the eligibility requirements up front, but they should be in bullets instead of full
sentences. List three bullet points of the requirements. The Q in qualify should be uppercase in the
title. Is there an income requirement? She would want to know that up front. The calculator is
cute, but people should know how much money they make if they are using this program. For our
audience, it is not relevant. She likes the qualifier language in the dependents question. Popover
icons should be larger. One click of the second popover the first should disappear. Numerical
keyboard should come up on numerical questions The footnote should be clearer. When you have an
asterix, there should be another on the page. The footnote has no reference. She likes that the
system refers to paralegals. Why would I want to tweet about applying for social services? Remove the
social service icons from this page. Mentioning the 'Benefits Access tab' on the RTA Ride Free
website is confusing. Find a really concise way to say that the 'apply button' will take you to a different
site. Let users know that they are leaving mRelief before they click the button. She suggested
removing the print button because there is no information to take with her.
Yes
Yes
Questions were straightforward. 'Describe the relationship to the child you are caring for.' - Delete

the word biological from the TANF form question. You are missing the guardian category. There is
too much left for interpretation on the TANF form - what about caring for godchildren? If this form
doesn't take care of that, it should be added.
The resources provided were not clear. She thought that it would prevent her from having to go to the
Illinois pages. It would be nice to prepopulate the form with the eligibility information entered into the
Illinois State page. What is the benefit from knowing that she "may be eligible?" She wants mRelief to
add her information automatically to forms. She likes the All Programs page because she learned
what the State of Illinois offers.
SMS
REVIEW

Food
First she said, make the texting announcement on the website clearer. "It gets lost in the sauce." Click
here or text. It currently looks like a warning. The language SMS is not accessible. She likes that the
question says 'Enter yes or no.' She likes that it says 'Enter a number.' If you enter, "y" that should not
be a typo. You should not have to start over if you make a mistake - you should not have to go back to
the beginning. The typo message that refers to "this question" is misleading.
Yes
She doesn't like the way 10000 is formatted.
Instead of answering all of the questions, she would like to just call the number from the start. The
response was anti-climatic. She would like to know up front that you will be referred to community
resources on the homepage - features and benefits should be stated on the homepage. Remove the
about us and list the features and benefits.
OVERALL
REVIEW

5 - Very Important
5 - Very Important
2 - Somewhat Important
5 - Very Important
Yes
Yes
"All City Programs" is misleading - get rid of the word all. She would like to know the universe of
programs that are available. Make Red Cross programs or other disaster programs accessible on
mRelief. We have to work on the marketing and promotion on mRelief. People need to know about
the text option - it should be publicized.
Yes
Yes
This was way more organized than it used to be.
2015-01-29 18:03:52

Tester #:
11
Checking Eligibility
Proctored by:
Rose Afriyie
35-44
laptop
Yes
ABOUT
YOUR
INTERNET
AND
DEVICE
USE

4 or more hours
information?
Yes
text?
5 - Very Useful
No
T-Mobile
in social services?
Yes
If so, which one?
Food Stamps, Medicaid
benefits?
Yes
It is pretty simple, they ask for your information and they have a log
in for county care so it is pretty helpful. I would rather just get the
number myself and do everything on the computer.
WEBSITE
REVIEW


Building might be a school Bottle might be for children It gives you transportation times, maybe check
where the nearest grocery store is
Click "Get Started"
I figure I would get started or click one of the icons.
you choose?
Rental Assistance
It was definitely easy to use.
No
Yes
Self explanatory
I would call and check it out.
SMS
REVIEW

Food
Respond back real fast! What do you put if you have no income? zero?

Yes
This is pretty much the same. I am glad you included the question about college. Because a lot of people
don't know that you have to be working AND in college to get food stamps.
I didn't know they did stamps there. I would check them out because now I know they have a few more
resources.
OVERALL
REVIEW

5 - Very Important
5 - Very Important
Yes
Yes
Add some graphics. Something that will stick up so you remember next time. You need some better
graphics.

Yes
Yes
Instead of just googling all over the place -- I can just go right here This is something that is very useful
Cuts down on the calling-around time -- And you can text -- even better!
2015-01-29 18:06:56

Tester #:
12
Very outgoing and ambitious
Proctored by:
Genevieve
35-44
iPad
Yes
ABOUT
YOUR
INTERNET
AND
DEVICE
USE

4 or more hours
information?
Yes
text?
5 - Very Useful
No
T Mobile
in social services?
Yes
If so, which one?
Food stamps, Medicaid
benefits?
Yes
Uses the Internet your food stamps balance. Looks for medical
providers for yourself and children.
WEBSITE
REVIEW


Looks like it tells you services that are provided in Chicago - information about schools, food and
transportation.
Click "Get Started"
It seemed like 'Get Started' was the best way to start. It was easy. She thought the icons might have
been clickable, but the Get Started button really caught her eye.
you choose?
Rental Assistance - this is a program that she could benefit from - she is also receiving two of the
programs listed.
She put in 4 for the dependents question - her and her children. She put in her 30 day income
instead of 90 day income, but I pointed it out and she changed it.
No
Yes
The questions were easy to understand. She liked the questions.
She would call the number to find out more information, and then maybe schedule an appointment if it
is a good fit. She would like to know the income range of eligibility, so that she is not wasting her
time by pursing the option. List the income guidelines on the response page. She wants to know the
guidelines so that she can tell other people.
SMS
REVIEW

Food

She likes it because it is self explanatory. She says that she is enrolled in school.
Yes
Make it clear that being in a college does affect your eligibility. She has never seen that before in the
food stamps application process.
She would call to find out more information and hopefully they could answer the questions. She wants
to know that calling is beneficial.
OVERALL
REVIEW

5 - Very Important
5 - Very Important
5 - Very Important
5 - Very Important
Yes
Yes

It is self-explanatory. It is great for cell phone people.
Yes
Yes
2015-01-29 18:27:58

3/31/2015 CUTGroup Test: mRelief
https://smartchicago2012.wufoo.com/forms/cutgroup-test-mrelief/ 1/6
CUTGroup Test: mRelief
Thanks for participating in our test of mRelief, a Web site that helps residents check their eligibility for social
services benefits in Chicago & Illinois.
About you
First, some preliminaries. Keep in mind that we are going to publish the results of this test, but will not associate your
name or email address with the results.
About your internet and device use
We are interested in learning a little bit more about what you do on the internet.
Proctored by: *
First Last
Name *
Email *
Give yourself a tester profile. *
Under 18
18‑24
25‑34
35‑44
45‑54
55‑64
65 or Above
Prefer Not to Answer
Could you tell us your age? *
What type of device are you using for this test? *
Yes
No
Is this your normal device that you use everyday? *

Website review
Let's review the website!
Less than an hour
1 ‑ 2 hours
2 ‑ 3 hours
4 or more hours
How much time do you spend on the Internet a day? *
Yes
No
Do you SMS text to get information? *
5 ‑ Very Useful
4 ‑ Useful
3 ‑ Neutral
2 ‑ Slightly Useful
1 ‑ Not useful at all
How useful would it be for you to receive information via text? *
Yes
No
Do you pay per text? *
What is your cell phone service provider? *
Yes
No
Are you currently enrolled in social services? *
If so, which one?
Yes
No
Not applicable
Do you go on the internet to access social service benefits? *

Click "Español"
Click "Get Started"
Click "How mRelief Works"
Click "Contact Us"
Click "About Us"
Click "Press Release"
Click "Programs"
Other
Explore the site. What's the first thing you do? *
Tell us more about the first thing you did. *
Go to the "Programs" tab and choose one that is relevant to your
needs. Which one did you choose? *
Go through the process of seeing if you are eligible and tell us
about your experience.
Did you click on pop‑overs for additional information? *

SMS Text Review
Yes
No
Yes
No
Was this process easy? *
Tell us about your eligibility and the resources provided. What is
the next thing you would do with this information?
Check your eligibility via SMS: Text "hello" to 1‑773‑377‑8946.
Which program do you find more information about?
Go through the process of seeing if you are eligible and tell us
about your experience.
Yes
No
Was this process easy? *

Overall Review
Tell us about your eligibility and the resources provided. What is
the next thing you would do with this information?
5 ‑ Very Important
4 ‑ Important
3 ‑ Neutral
2 ‑ Somewhat Important
1 ‑ Not at all Important
How important is it for you to know what documents you need to
prepare for an interview with a caseworker to receive public
assistance?
4 ‑ Important
3 ‑ Neutral
Is it important for you to know why you do or don't qualify for
public assistance?
4 ‑ Important
How important is it for you to know how much money you will
receive from public assistance before going to file with a case
worker?

Overall review of the CUTGroup experience
3 ‑ Neutral
4 ‑ Important
3 ‑ Neutral
How important is it for you to know how to file an application for
public assistance?
Yes
No
Do you like mRelief? *
Yes
No
Would you use mRelief in the future? *
In what ways would you improve mRelief? *
Yes
No
Do you like CUTGroup? *
Yes
No
Did you like this CUTGroup test? *

3/31/2015 New CUTGroup Opportunity: Test a Social Services Website
http://us5.campaign-archive2.com/?u=085247dea37361c266002462c&id=2450aca6ab&e=[UNIQID] 1/2
Make $20 for a test of a social services website.
Hi <<First Name>>,
We've got a new opportunity for you to make money in the Civic User Testing Group
(CUTGroup) by testing a social services website.
This website helps residents check their eligibility for benefits in Chicago & Illinois.
We want to know if this website serves your needs, and how it can be improved.
Are you available for a 30-45 minute in-person test on Thursday, January 29
between 3:30 - 7:30 PM? If so, please complete this form.
We are in the process of gathering responses, so we will be in touch to check
availability and confirm that you are right for this test. We are looking for about 20
testers.
For your participation, you will receive a $20 VISA gift card. You'll also help make
better software for Chicago.
Thanks for being a member of the CUTGroup! As usual, call or write with questions.
Remember: if you want to be a part of this test, please complete this form.
--
Daniel X. O'Neil
(773) 960-6045
doneil@cct.org
Subscribe Share Past Issues Translate

3/31/2015 New CUTGroup Opportunity: Test a Social Services Website
http://us5.campaign-archive2.com/?u=085247dea37361c266002462c&id=2450aca6ab&e=[UNIQID] 2/2
Facebook Twitter Website
Creative Commons Attribution 3.0 Unported License 2015 Smart Chicago Collaborative. unsubscribe from this list
update subscription preferences

3/31/2015 CUTGroup Test: Social Services Website
https://smartchicago2012.wufoo.com/forms/cutgroup-test-social-services-website/ 1/1
CUTGroup Test: Social Services Website
If you would like to participate in the next CUTGroup test, please complete this form. We will follow‑up to
check your availability and let you know if you qualify for this test!
Smart Chicago values your online privacy and is committed to safe guarding the data you share with us and
your online privacy. Our On‑Line Privacy Policy applies to data collected at www.smartchicago2012.wufoo.com
domain.
Email *
Yes
No
Are you available for an in‑person test on Thursday, January 29 for
30 minutes between 3:30 and 7:30 PM? *
Yes
No
Are you 18 years old or older? *
Where are you most likely to use the internet? *
Yes
No
Do you use a smartphone? *
Yes
No
Are you enrolled in any social services programs? *
What is the number of people living in your household including
yourself? *
What is your gross monthly income? *

3/31/2015 Details for next CUTGroup Test of a Social Services App
http://us5.campaign-archive2.com/?u=085247dea37361c266002462c&id=ece384e026&e=[UNIQID] 1/2
Let us know when you could join us on Thursday for this test.
Hi <<First Name>>,
Thanks for your interest in our CUTGroup test of a social services app. Now that we
have our location nailed down, we'd like to know if you can still make it next
Thursday and, if so, what time you want. Here are the details:
Thursday, January 29, 2015
Between 4:00 - 7:30 PM
Chicago Public Library King Branch
3436 S. King Drive
If you can still make it on this night, please let us know what time slot you want
by completing this form. For your participation, you will receive a $20 VISA gift card.
There are limited slots available for this night, and it's coming up fast, so
please let us know as soon as you can. We will get back to you with a
confirmation of your time if you are chosen to do this test.
If this location doesn't work for you, or if you just can't make it on this night after all,
no big deal. If you have any questions or comments, just hit "reply" and let me know
what you think.
Remember: use this form to tell us about your availability.
Thanks for being a member of the CUTGroup!
--
Subscribe Share Past Issues Translate

3/31/2015 Details for next CUTGroup Test of a Social Services App
http://us5.campaign-archive2.com/?u=085247dea37361c266002462c&id=ece384e026&e=[UNIQID] 2/2
Daniel X. O'Neil
(773) 960-6045
doneil@cct.org
Facebook Twitter Website
Creative Commons Attribution 3.0 Unported License 2015 Smart Chicago Collaborative. unsubscribe from this list
update subscription preferences

3/31/2015 CUTGroup Test: Social Services Availability
https://smartchicago2012.wufoo.com/forms/cutgroup-test-social-services-availability/ 1/1
CUTGroup Test: Social Services Availability
If you would like to participate in our CUTGroup test of social services app, please complete this form. The test
is Thursday, January 29 between 4:00 ‑ 7:30 PM at the King Library at 3436 S. King Drive.
Email *
Yes
No
Are you still available on Thursday, January 29, 2015 to attend this
test in person?
4:00 ‑ 4:30 PM ‑‑ This time slot is now full. Please choose
another one. Thank you!
4:45 ‑ 5:15 PM
5:30 ‑ 6:00 PM
6:15 ‑ 6:45 PM
Sorry, this day does not work for me!
Which of these time slots would work for you? Please select all that
may apply, and we will get back to you with your confirmed time. *
The test will last about 30 minutes.
We will send an e‑mail confirming
your time slot!

CUTGroup 13 - mRelief Final Report

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (6)

Similar to CUTGroup 13 - mRelief Final Report

Similar to CUTGroup 13 - mRelief Final Report (20)

More from Smart Chicago Collaborative

More from Smart Chicago Collaborative (20)

Recently uploaded

Recently uploaded (20)

CUTGroup 13 - mRelief Final Report