Final Report for CUTGroup #24 - OpenGrid

CUTGroup #24
OpenGrid
Report by
Nathalie Rayter & Sonja Marziano
Smart Chicago Collaborative

CUTGroup #24: OpenGrid
In-person test on Wednesday, April 20, 2016
Introduction
For our 24th Civic User Testing Group (CUTGroup) test, we tested OpenGrid, an open-source
interface developed by the City of Chicago that allows residents to search for, interact with and
visualize City of Chicago’s datasets.
“OpenGrid has an intuitive user interface that’s highly responsive and efficient. The
application is accessible from any devices (e.g., mobile, laptop, tablet or desktop). It’s
unique, functional, and an easy to use tool to find city data. It extracts city data and
displays it in an illustrative form which makes it easier for a user to search and
understand the information.” - City of Chicago Department of Innovation & Technology
staff
Target Group
On April 12, we sent out an email to 1,132 CUTGroup testers who lived in Chicago. We wanted
to know if they would be available for an in-person test on April 20. When segmenting our
testers, we were interested in:
● Testers who live within Chicago city limits because OpenGrid specifically shows City of
Chicago data
● A mix of testers with different types of mobile devices (Android/iPhone)
● Testers who had varying degrees of familiarity with what happened in their
neighborhoods
● Testers who did or did not use datasets in their personal and/or professional lives
Test Type
● In-person test component where testers would view different features on their own
device and complete tasks
● This test utilized A/B testing, where 13 of the testers (“A” Testers) began on
http://opengrid.io/ (OpenGrid’s homepage) and the other 10 testers (“B” Testers) on
http://chicago.opengrid.io/opengrid/ (the app interface).
● All testers were paired with a proctor, who asked them questions, observed and
recorded their interactions with OpenGrid and their responses and feedback.
Goals of Test
The main goal of the OpenGrid test was to measure whether Chicago residents are able to find
the information that they are looking for and whether it is easy to navigate datasets. Past
usability testing of the interface had been with highly skilled users or people who work at the
City of Chicago. We also wanted to learn how residents might use the information and data
displayed on OpenGrid in their personal or professional lives and if they find value in OpenGrid.
2

Responses
23 testers completed the OpenGrid CUTGroup test. In Group A, 8 testers used laptops, 3 used
smartphones, and 2 used tablets. In Group B, 5 testers used laptops and 5 used smartphones.
Can you tell us more about your experience, if any, using data or datasets?
3 testers described proficiency in accessing data in their professional and/or personal lives. 7
testers mentioned they had some familiarity with data and/or using the internet to find more
information about their neighborhoods. 3 testers gave responses that indicated they had
previously had little to no exposure to using data.
2 testers described proficiency in accessing data in their professional and/or personal lives. 6
testers mentioned they had some familiarity with data and/or using the internet to find more
information about their neighborhoods. 2 testers gave responses that indicated they had
previously had little to no exposure to using data.
Review the homepage and tell us about the three things that you notice
first.
Group A testers started on OpenGrid’s homepage (http://opengrid.io/). Here are A testers
pointed out when reviewing the homepage (number indicates number of times a topic of feature
was mentioned):
6 - “Launch”
6 - Background image
3 - “Code”
3 - Headings
2 - Contact Us
2 - Quick Search
2 - Video
2 - Informational text
2 - Expressed confusion
1 - Managed Queries
1 - Symbol
1 - Open Source
3

1 - Data
1 - “Customizable”
1 - City of Chicago
Group B testers began on the app interface (http://chicago.opengrid.io/opengrid/). Here are
what B testers pointed out when reviewing the app, presented to them as the homepage:
4 - Map
3 - “Queries”
2 - Features on map
2 - Quick search box
2 - Colors
2 - Menu
1 - Absence of call-to-action
1 - Submit button
1 - OpenGrid logo
1 - “Commonly Used Queries”
1 - Side menu
1 - Map icons
If you had to describe what this website does, what is your best guess?
After reviewing the homepage, the majority of Group A testers mentioned one of these four
topics: data, map directions for getting to places, information about places on the map, and
exploring neighborhoods. Here are key responses:
Charlie (#A9) says, “It's like Google Maps - put in your location, and it will give you
directions, information about what's on the map.”
Bronco (#A15) says, “[It’s for] Encouraging the developer community to develop
application from using the data on this website. These tools would be for residents.”
JG (#A14) says, “I probably wouldn't have an idea - "OpenGrid" isn't telling much, unless
you start scrolling down - and even then, 'exploring neighborhood' is vague.”
After reviewing the app launch page, the majority of Group B testers mentioned one of these
two topics: map directions for getting to places and information about places on the map. B
4

testers did not have the contextual information provided by the OpenGrid homepage, which
proved to be valuable to the testers in Group A. Here are key responses:
Make Believe (#B1) says, “[It’s for] traffic or how to get around city quicker.”
Paloma (#B8) says, “Traffic or how to get around city quicker.”
Who do you think this website is targeted to? Why?
7 testers said that they thought the site was either for Chicago residents. Here are some key
responses:
Charlie (#A9) says, “Focus is on travelers, drivers, bicyclists, public transit riders - it may
help these people know where they're going without asking other people.”
Heather (#A11) says, “Chicagoans who want to know more about their neighborhood.”
3 testers indicated that they thought the OpenGrid was targeted toward developers.
Chris (#A16) says, “Anyone who has interest in viewing the data; developers that would
like to integrate this into other projects.”
1 tester indicated uncertainty of who the site’s target audience is; another tester, Ms Boop
(#A6), says it’s for “anyone that can get online.”
There was much less consensus about who the target audience is among testers in Group B,
who mentioned a general audience, people moving through the city, residents, investors, and
government employees as possible audiences. Here are key responses:
Susan (#B5) says, “Targeted to the city of chicago's residents or any user. If you lived in
a place like Alaska you could still pull this up. Could be for residents or people investing
in property.”
Zachary (#B7) says, “[It’s targeted] To individuals who are not familiar with certain parts
of the city.”
5

After reviewing the homepage, what is the first thing you do or click?
When exploring the website the Group A, 7 out of 13 testers in Group A (54%) clicked
“Launch”, which opened the map interface; 2 of the 13 testers (15%) had already launched the
map, and proceeded to use the search tools.
7 - A: Clicked “Launch”
1 - A: Clicked “Explore”
1 - A: Clicked on video
1 - Other: tried to click on a building
1 - Other: Clicked on "Manage Queries" on map page
1 - Other: Clicked on “Advanced Search” on map page
0 - A: Clicked “Contribute”
Most testers in group B did not take an overwhelmingly clear first step or action. 3 out of 10
testers (30%) used quick search and 3 more testers (30%) used chose commonly used queries,
while the rest of the testers did a variety of other first actions.
3 - B: Used quick search
2 - B: Chose Commonly Used Query
2 - Other: Relocated map to a familiar area
1 - Other: Searched random addresses
1 - Other: Clicked advanced and commonly used queries
1 - Other: Clicked on the measure tool
1 - Other: Clicked on the menu
0 - B: Added datasets and searched
0 - B: Clicked on “Manage Queries”
If you searched, what was the search term you used?
Testers’ search terms included “2 bedroom apartment”, addresses, “Business License 60624”,
“Foodie”, buildings, and “restaurant passing initial inspection” and others from the commonly
used queries.
6

Tell us more about the first thing you did, and why you did it. What were
you expecting to happen?
Several testers from Group A said that they didn’t expect the “Launch” button to open up a map.
When looking at the map interface, a few testers some expressed confusion about why the map
was centered where it was and about what they should do next in the absence of prompts.
The tester who clicked on the video did so looking for explanation of the website.
Testers in Group B similarly expressed a desire for prompts that would teach them how to use
the interface.
Here are some responses from testers about the first thing they did:
Renee54 (#A4) says, “‘Launch’ sounds technical, new-age. Upon launching, I did not
expect to see a map of Chicago. I didn't ask for the map information, yet a map of the
west side is here. I would expect to see links of items of interest to residents of Chicago;
I don't see them here. Not sure what I'm looking at; the screen doesn't provide what I
asked, so I'm not sure what I'd inquire.”
Chris (#A16) knew he would see a map when he clicked “Launch” but thought he would
be prompted to do something. He felt that he was just “dropped" into the map view.
Make Believe (#B1) used quick search and typed ‘food’, but didn't see that the search
returned an error because the keyboard was blocking information. After some difficulty
clicking the “Help” button, they found that the instructions were targeted towards
technical audience, and ‘a little complicated.’ They couldn’t see the entire window on
help screen, and observed that the quick search function seems to be for a specific
location, not data sets.
Paloma (#B8) says, “I wanted to see what the computer [could] do instead of me
creating search criteria on my own. [I] wanted to see what it would do for me. [I] didn't
click on anything else because I wanted to see what that button would do. The result met
expectations; I wanted locations and the grid returned results.”
Find 311 service requests that are currently open in your neighborhood.
Tell us about your experience.
3 testers, Susan (#A5), Sigma1922 (#A13), and Gabi (#B12), were able to conduct searches
that yielded results without prompting from their proctors.
7

13 testers attempted to search “311”, their neighborhood or ZIP Code, or a combination of both
using the Quick Search bar, but these searches did not yield results. 2 other testers, Data User
(#A23) and Paloma (#B8), attempted to use the “Near Me” location filter to search for open 311
requests, but did not have initial success.
10 testers (only 3 of whom did not initially attempt a search - Tact (#B17), Community Interest
(#B20), and Family Safety Interest (#B21)) were prompted by their proctors to use the
Commonly Used Queries and/or Advanced Search options, which yielded results.
Here are some key observations of testers:
Cleveland54 (#B3) says, “It should [say] enter zip code here or enter here… [There’s]
no arrow no graphical element to tell me to enter data. It doesn’t stand out. Expected a
drop down. The up and down arrows expect to a droplist with a list of zipcodes. The
submit button is not noticeable.”
Renee54 (#A4) would have tried “advanced search” out of desperation, but would want
a quick search to get everything needed. The tester felt that “advanced' implies a
specific person or place, not general information about a neighborhood.
Susan (#B5) says she thought there should be a clearer place to enter addresses. She
wouldn’t put it where there is “Quick Search.”
Did this search meet your expectations?
Yes 23% (3)
No 77% (10)
Yes 60% (6)
No 40% (4)
Why or why not?
Here are some responses from testers who said “No” this search did not meet their
expectations:
Make Believe (#B1) says, “[It] is not intuitive. [It’s] for a technical audience [and it] needs
different UX to engage a non-technical audience”
8

Ms Boop (#A6) says, “It is hard to interpret the data. There are no keys.”
JG (#A14) says, “The options that are given to work with aren't clear. [It] needs to be
more specific yet easy to understand - [I] played with a bunch of options, yet nothing
showed up.”
Here are some responses from testers who said “Yes” this search did meet their expectations:
Paloma (#B8) says, “It produced something but [I’m] trying to understand the info in the
data points. Trying to guess what it is.”
Gabi (#B12) says, “Often when you try to map stuff it doesn't work the first time. I am
used to instructions but I know that isn't common with this stuff. My expectation wasn't
high. I will try this again on a computer.”
Sigma1922 (#A13) says, “Because when [I] was able to find the 311 request I was able
to find the information without doing additional search. It was right in front of me.”
How easy was it to find open 311 service requests in your neighborhood?
5 - Very easy 0%
4 - Easy 15% (2)
3 - Neutral 31% (4)
2 - Difficult 31% (4)
1 - Very difficult 23% (3)
5 - Very easy 10% (1)
4 - Easy 0%
3 - Neutral 10% (1)
How would you improve this search experience?
13 testers recommended that changes should be made to the OpenGrid search tools to make it
easier for users to find the information they are looking for. 7 of these testers thought that the
search bar should respond to addresses/ZIP codes or tags like “311”. 3 testers commented that
9

the existing search filters are too complicated, and that they should be simplified. Paloma (#B8)
says, “Commonly used queries was easy; all other filters made it difficult. Dataset and Standard
filters should be first; it would be easier.”
7 testers mentioned that they would improve the descriptions and instructions that orient the
user to the OpenGrid interface. 2 of these testers recommended adding instructions that
suggest how a user would interact with the page, such as suggesting a search or having sample
questions. For example: Cleveland54 (#B3) says, “The advanced search panel should be
pushed down with a note at top saying ‘welcome, you can search blah blah here.’ The landing
page should have description of the website and then launch to the site.”
2 other testers recommended improving the language used to label data points and information
on the map.
7 testers said that they would make changes to design elements including color, readability, and
map appearance to improve the search experience. 2 testers commented that the text and
symbols on the interface were small and might be difficult for some people to read. 2 other
testers (Zachary (#B7) on a laptop and Gabi (#B12) on an iPhone 5, mentioned that it was
difficult to see all of the search text on their screens.
2 other testers mentioned data point color as an area that could be improved. JG (#A14)
indicated that “Information should be more easy to find. For starters, the blue and white [on the
map] is not the best color option, too. As an additional walkthrough, tester chose street closures
within zip code value, and found 13 records. The records that are showing up, however, are not
reflected on the map -- tester requests highlighting streets or including more easily visible what
the closures are.”
Pretend that you are in the process of moving to a new part of Chicago;
use OpenGrid to find information about this area and let us know what you
find.
What did you learn using OpenGrid?
When seeking information about a new part of Chicago, 7 of the 23 testers initially entered in
neighborhoods or ZIP codes into the search fields without indicating the datasets, and their
searches yielded no results. JG (#A14) says, “Honestly - the site is not giving what I want, need;
information is confusing and not clearly presented. Not much has been learned.”
10

10 testers (6 from Group A and 4 from Group B) were able to successfully obtain information by
using the search function, but had varying degrees of difficulty. Here are a couple of
experiences we observed:
Paloma (#B8) searched 60638 with several different filters, including crimes. She says,
“This is telling me a lot about the neighborhood. [I] would be worried about seeing all
these crimes and thefts… for everyday citizens, try translating the style of writing more
user-friendly. It looks like the data was copied and pasted out of the inspector’s user
book and it will take time to understand the data.”
Tact (#B17) first went to add a ZIP code, then found data sets and started to scroll to
see options. The tester ended up looking at landmarks before settling on crime. Tact was
a little slow to press on result dots; they were looking for historical data but were only
getting data from the most recent year (although they did not enter a data range). Tact
recommended that the labels for filters and search criteria could be improved, for
example: ‘geospatial’ is not commonly used phrase.
Yoonie (#A18) could not search by community area "Pilsen" as expected, so the tester
decided to move the map to that general location and look up crime data from
2001-present. The results do not prepopulate as the tester expected, but they eventually
change the geospatial filters to “map extent.”
Some testers moved beyond the search bar. Charlie (#A9) tried multiple addresses through
quick search, but none provided a result; she found that moving the map around manually was
more effective than using the search bar. Tester Chris (#A16) did the same thing to find their
results.
Search for crime data from the last year in the neighborhood you live in.
8 of 13 A testers (62%) encountered difficulties in searching for crime data from their
neighborhoods. 2 of these testers initially attempted to search using the names of their
neighborhoods, which did not return results; using ZIP codes or ward numbers yielded data
points. 2 others were not able to return results. Here are some of the key observations:
Doc (#A2) tested OpenGrid on a smartphone and had a hard time finding crime and
filtering for a time period. He also lost his search criteria when he navigated back to the
map from the search panel. When he submitted his search query, the app returned with
an error, “Search is invalid,” without giving any explanation.
ChicaoJAE (#A10) started her search with location, then selected crime data. With a
little instruction, she found search filters, but had to manually enter the dataset. The
11

tester hit submit, and the app produced results on the grid. As she scrolled through, the
tester saw the icons but was more interested to read grid data.
9 out of 10 B testers (90%) had a difficult experience conducting this search. 4 of these testers
found it hard to select the correct data set from the listed options. 3 were not able to return
results. Here are some of the key observations:
Tact (#A17) used advanced search to enter a ZIP code into the boundary filter and
select crime data from last 15 years, with some help from the test proctor. On the first
attempt, the search produced error; then, the tester removed some empty filters and it
produced results.
Family Safety Interest (#B21) did not recognize the menus that they needed to pull up
the crimes. The tester would have liked to select a neighborhood from a drop-down
menu instead of a ZIP code. They found rectangle selection tool a little difficult to use,
but the polygon tool was friendlier.
Did this search meet your expectations?
Yes 46% (6)
No 54% (7)
Yes 70% (7)
No 30% (3)
Why or why not?
Here are some responses from testers who said “No” this search did not meet their
expectations:
Susan (#B5) says, “The language is not at a 6th grade level and is not at a level that
anyone can use.”
Ms Boop (#A6) says, “It was not easy to determine how to get the data and once I did it
was much more information that I needed. I just needed crime and address. I don't need
ward or district.”
12

Charlie (#A9) observed that it's not as helpful as it should be to find anything -- you
should only have to click once or twice to get the desired results. Charlie thinks that
OpenGrid is not designed for novices, and would require training to know how to use it.
Here are some responses from testers who said “Yes” this search did not meet their
expectations:
Paloma (#B8) says, “It got easier because I got use to practicing.”
Heather (#A11) says, “The data is good, but having the address more easier to find
would be nice.”
Chris (#A16) says, “Adding the rule took a second to figure out, and then it made
sense.”
How easy was it to find crime data from last year?
5 - Very easy 0%
4 - Easy 15% (2)
3 - Neutral 38% (5)
5 - Very easy 0%
4 - Easy 40% (4)
3 - Neutral 20% (2)
How would you improve this experience?
9 testers recommended making changes to the OpenGrid search experience. 2 of these testers
(Doc (#A2) and Charlie (#A9)) described the search experience as not too difficult, but only
after learning how to navigate it, indicating a need for more instruction around searching.
The other 7 testers had comments on how to improve the search selector fields. 2 testers
mentioned improvements to the way the search function handles location. Chris (#A9) feels that
the OpenGrid interface emphasizes the data set and then the location; he would swap that
13

around and make it more location-driven, such that users would first select location, then select
their desired data set. Cleveland54 (#B3) would prefer a drop-down menu of community areas
to having to manually input locations.
2 testers (Susan (#B5) and Gabi (#B12)) thought that it was difficult to search crime. Gabi
(#B12) recommends better defining the suggested queries and giving more options to refine the
results. Similarly, 1 tester, Tact (#B17), hope for the ability to search for specific events or
incidents within the crime data set.
2 testers (Paloma (#B8) and Downtown Closures (#B21)) both recommended reducing the
initial number of search fields to make it easier to search.
5 testers said that OpenGrid should be made more user-friendly. 2 of these testers (Renee54
(#A4) and Make Believe (#B1)) mentioned that the language currently used is too technical
Renee54 (#A4) says, “It needs to be user-friendly for all Chicago citizens - terms like
‘geo-spatial filters' don't mean anything to most users. It sounds deep, but gets people nowhere.
This isn't Star Trek over here.” 1 tester (JG (#A14)) said that clearly labeling tabs would be
helpful to orienting the user.
1 tester Community Interest (#B20) encountered challenges related to the design; when
searching for crime in their neighborhood on an Android smartphone, the gridview was up and
blocking the advanced search pane. The tester had to minimize the grid to pull-up the advanced
search pane.
5 testers recommended improvements related to the data returned by searches. 2 testers
(Zachary (#B7) and Family Safety Interest (#B21)) wanted more descriptive information on the
map data points, while 1 tester (Ms Boop (#A6)) would prefer a shortened description of the
crime in the data point.
Tester ChicagoJAE (#A10) recommends displaying both the grid and map results at the same
time.
Tester Bronco (#A15) had concerns about the quality of data returned, noting that they were
not able to obtain information from within the last year.
14

Next, find a dataset that you are interested in learning more about that is
relevant to you. What did you look for?
3 - Food inspections
2 - Filming locations
2 - Landmarks
2 - Potholes
2 - Building violations
1 - Schools
1 - Red light cameras
1 - Speed cameras
1 - Street closures
1 - 311 service requests
1 - Environmental complaints
1 - 911 reports
1 - Transportation
1 - Common Queries
Tell us about your experience. How did you search? Can you describe the
information you found?
9 testers were successful when searching for datasets they were interested in learning more
about.
Charlie (#A9) tried to search for “speed cameras” and “locations,” but that didn’t work;
the tester then tried “citywide,” which yielded 50-60 results. After using the website for a
while, it became slightly easier, and the tester saw the process as more useful over time.
Charlies says, “You need to know specifically what you want in order to get any value
out of [OpenGrid].”
Community Interest (#B20) was able to look at potholes to see where they were
reported. The tester mentioned it would be helpful to report 311 calls through the app.
5 testers were not able to successfully return results through their searches. Here are some of
the things we observed:
Renee54 (#A4) was not able to find information they were looking for, and would instead
just go to another website.They recommended that OpenGrid developers look at better
models out there for public information and improve it; the tester would not use this site
because it was unclear what kind of data is available in the first place. Without knowing
what information is there, the site is useless for now -- but it can be improved.
15

JG (#A14) tried to search “schools” using “quick search” at first, which yielded no results.
The tester then went to “advanced search,” but didn’t see their query under “commonly
used queries”; the tester then clicked on “datasets -> add dataset" and did not see it
there either. The tester concluded that schools are not included in the OpenGrid system.
The remaining 9 testers’ search outcome is unclear based on the test notes, but 5 of these
testers described encountering obstacles in their search processes.
13 testers (4 testers who had successful searches) described some difficulties during the
search. Here are some of their experiences:
Gabi (#B12) searched for “film locations,” thought that the filters were nice. Her search
yielded results from the whole city since she did not specify boundaries. She zoomed in
and found that the data points were from three years ago. “They only paid $25?! [It’s]
exciting there are so many of them.” She could not figure out how to limit the search by
date, and accidentally hit an arrow that opened the bottom table list. It was hard to close
the popup when it takes up most of the screen on a mobile device.
Family Safety Interest (#B21) found that once she got the hang of the query, it was
easier, but took a bit of effort to find the query in the menus. The custom location queries
took a number of taps on her smartphone to find.
Overall, how easy do you think it is to use the OpenGrid website?
5 - Very easy 0%
4 - Easy 23% (3)
3 - Neutral 23% (3)
5 - Very easy 0%
4 - Easy 20% (2)
3 - Neutral 40% (4)
16

What improvements might you make to make it easier to use?
4 testers recommended that changes should be made to the OpenGrid search tools to make it
easier for users to find the information they are looking for. Make believe (#B1) says, “Search
by natural feedback. More functionality in primary search.” Doc (#A2) says, “Saving information
so you don't have to reenter everything. Less filters. They are too broad.”
4 other testers mention improving the search-by-location mechanism, which tripped many of
them up during the test exercises. Charlie (#A9) says, “Get it to recognize addresses in quick
search query quicker.” Zachary (#B7) says that “Quick search should allow address.”
4 testers mentioned that language used on the OpenGrid interface was unclear and/or hard to
understand. Chris (#A16) feels that there is “too much technical language that could be more
user-friendly.”
4 testers recommend adding more explanation about elements of the app. Gabi (#B12) wants
“More explanation of the buttons and icons,” while Renee54 (#A4) suggested including “More
links, lists of what data is available, and more user-friendly design.”
2 testers mention the desire for instructions on how to use the app. Paloma (#B8) says that a
“Pop-up about the site would be useful on the landing page or quick tutorial about how to use
the site.”
2 testers suggest making changes to the look and feel of the app. ChicagoJAE (#A10) says,
“Larger font. Make the grey bar larger.” Cleveland54 (#B3) says that “adding graphical
elements” would make it better.
Overall, how easy do you think it is to understand the content and
information on OpenGrid?
5 - Very easy 15% (2)
4 - Easy 38% (5)
3 - Neutral 8% (1)
17

5 - Very easy 10% (1)
4 - Easy 40% (4)
3 - Neutral 0%
What improvements might you make for the information to be easier to
understand?
Language Accessibility
3 testers mentioned difficulty understanding the language used on the OpenGrid interface.
Paloma (#B8) didn’t understand the vocabulary used, and found that it had alot of technical
wording, while Bronco (#A15) found that searching was intuitive "once I got the language." Doc
(#A2) recommended use of “more plain/laymen terms.”
User Orientation
5 of 23 testers expressed a desire for more information about how to use OpenGrid when the
interface is first launched. 1 tester, Sigma1922 (#A13) said understanding would be improved
by including more detailed information and a tutorial on the main page on how to navigate. The
other 4 testers recommended giving the user more instruction about what kinds of information
are accessible through OpenGrid; Charlie (#A9) says, “Once you get inside and get a better
feel for it, the information is easy to understand - key is [the] user needs to understand the
information that they're looking for.”
Information Displayed
5 testers recommended various improvements to the format and content of the returned search
results. Here is what they said:
● “Each result on the map would have a link to click on for more information.” - Zachary
(#B7)
● “Have the address more predominant.” - Heather (#A11)
● “When I was scrolling through the text information about 311 calls, there are a lot of
fields there which is okay. Maybe there could be more space between fields so they
don’t all run together. I don't think I want the fields to go away but it is not easy [to
understand].” - Gabi (#B12)
● “[I] was more interested in the grid instead of the map.” - Community Interest (#B20)
● “[I] would like to see some more data sets. The grid search was also difficult to find.” -
Family Safety Interest (#B21)
18

What other types of information would you like to see in OpenGrid that
would be helpful to you as a user?
6 - Businesses
3 - Crime and safety
3 - Events
2 - Neighborhoods
2 - Parking and traffic
2 - Transportation
1 - 311 reporting function
1 - hospitals
1 - houses
1 - parks
1 - schools
After reviewing parts of the website, do you think the OpenGrid website is
targeted to you?
Yes 69% (9)
No 31% (4)
Yes 40% (4)
No 60% (6)
Why or why not? If not, who is the target audience?
Here are some responses from testers who said “Yes” they are the target audience:
Bronco (#A15) says, “More than I thought! If I just looked on the opening page and saw
references to open source and APIs and SQL like at the beginning, it would be targeted
to the developer group… but after exploring, maybe it is for end users!"
Community Interest (#B20) says, “Would be useful, but still too difficult to use.”
Here are some responses from testers who said “No” they are not the target audience:
19

Renee54 (#A4) says, “It's currently not targeted to anybody - not the citizens of Chicago;
it’s not even targeted to the writer of this program. It’s not easy to use or useful in its
current state.”
Gabi (#B12) says, “Not specifically, but with some practice I could learn how to use it
and it has some information that I am interested in.”
Do you like the OpenGrid website?
Yes 38% (5)
No 31% (4)
Not sure 31% (4)
Yes 60% (6)
No 30% (3)
Not sure 10% (1)
Why or why not?
Here are some responses from testers who said “Yes”:
Heather (#A11) initially didn’t like OpenGrid, but when she was able to find the data that
she wanted she was pleased with the information.
Gabi (#B12) says, “There is potential to access all that data and mash it up with the
spatial representation. That is very exciting and worthwhile.”
Here are some responses from testers who said “No” they do not like OpenGrid:
Renee54 (#A4) says, “Thumbs down - the page itself was beautiful, but beyond that,
that's all it was - it was not clear how I would use it or what I could get from it. It's very
disappointing.”
Susan (#B8) says that OpenGrid is “Too frustrating to use.”
Here are some responses from testers who said they are “Not sure” whether they like OpenGrid:
Ms Boop (#A6) says, “I am not sure how easy it will be to start using it as a brand new
person.”
20

Bronco (#A15) says, “I like the idea of it... I understand it is a work in progress. [But] It
needs a lot of work to plot the results.”
How likely are you to use the OpenGrid website in your personal life?
5 - Very likely 31% (4)
4 - Likely 8% (1)
3 - Neutral 31% (4)
2 - Not very likely 8% (1)
1 - Not at all likely 23% (3)
4 - Likely 40% (4)
3 - Neutral 10% (1)
What makes you say this?
Testers who said they would be “not very likely” or “not at all likely” to use the OpenGrid website
in their personal lives mention finding OpenGrid too difficult to use, or not having reason to use
it. Bronco (#A15) says, “In it's a current state ... it might be easier on the City's data portal. I can
get the data and know I'll get consistency of the results. I know what I am getting on the data
portal.” Another tester, Downtown Closures (#B22) said that they don’t access computers very
often.
Testers who responded “neutral” mention some difficulties in using the site, but also uncertainty
about how they would use it. Heather (#A11) says she would find OpenGrid useful in figuring
out what neighborhood to live, work, and be around during certain times in her life. JG (#A14)
says, “Based on the data choices available, things like neighborhood crime, 311 calls could be
useful for me from time to time.”
Testers who said they would be “likely” or “very likely” mention the appeal of relevant
information. Ms Boop (#A6) says, “Several of the common queries would be helpful as a
contributing neighbor to my neighborhood.” Gabi (#B12) says, “I am a curious person so looking
up crime or zoning code violations or whatever is in there. It might be amusing/interesting. It
might become vital if we decide to buy a house. Curious about future information that may
appear.”
21

How likely are you to use the OpenGrid website in your professional life?
4 - Likely 15% (2)
3 - Neutral 15% (2)
5 - Very likely 0%
4 - Likely 30% (3)
3 - Neutral 30% (3)
Testers who said they would be “not very likely” or “not at all likely” to use the OpenGrid website
in their professional lives gave either difficulty of use or lack of relevance as their reasons.
Charlie (#A9) says, “I'm not working in transportation, sanitation, other areas that apply to city
governance - might take a glance for traffic information related to work, but it doesn't do that
anyway.” Bronco (#A15) says that as a sociologist, they really see the importance in consistent
results.
Testers who responded “neutral” describe OpenGrid as out of line with their work. Doc (#A2)
says he might use it “only if it hand anything pertaining to restaurants or cooking jobs.”
Testers who said they would be “likely” or “very likely” mention the relevance of the tool. Gabi
(#B12) says that she is an “academic librarian so there are always students looking for data &
information (public health, social justice, geography, GIS program).”
How likely are you to share the OpenGrid website or its information with
your neighbors or community members?
4 - Likely 23% (3)
22

3 - Neutral 15% (2)
4 - Likely 30% (3)
3 - Neutral 0%
Testers who said they would be “not very likely” or “not at all likely” to use share the OpenGrid
website gave either difficulty of explanation and use or lack of relevance as their reasons.
Zachary (#B7) says his neighbors may be frustrated by OpenGrid and if he were to recommend
it, it would make him look bad. Make believe (#B1) was afraid they’d have to do too much
explaining.
Testers who responded “neutral” were split between the value of the information and the
challenges of using the tool. Yoonie (#A18) says, “Data seems really interesting - that would be
a 4. Accessibility is challenging is 2. There's potential to have this tool beat out all of the other
tools (crime map, food inspections stuff, public safety concerns).”
Testers who said they would be “likely” or “very likely” mention the relevance of the information
to their communities. Gabi (#B12) says, “Most neighbors are elderly or don't speak English.
Would share with son's friends who could share with parents.” Chris (#A16) says that his
community has a bunch of other data geeks that are interested in checking this out.
In what other ways would you improve the OpenGrid website?
Recommendations for further improvements were largely in the same categories as previous
suggestions. Here are some of the improvements that other testers suggested:
9 testers suggested making improvements to the OpenGrid search function, finding it difficult to
use quick search, especially when searching by location. Neighborhood Watch (#A20) says,
“[It] needs to improve navigation. I would also like to see neighborhood as a drop-down option…
[and] to be able to type in the ZIP code to see everything that's happening.” Tact (#B17) says,
“Improve the menu items by including dataset info.”
23

8 testers recommended making changes to the design elements of the OpenGrid app interface,
pinpointing several current features as obstacles to easy use. Cleveland54 (#B3) says,
“Graphic design information [is] needed to explain. There should be a floating help; the nav tools
are completely useless until I know what I’m actually doing.” Renee54 (#A4) says, “The user
should be able to figure out easily what they want to do. In its current state, it needs an
overhaul. It seems like there are great ideas embedded here, but the design did not let me get
to them.”
2 testers commented that the legibility of the app was poor. Yoonie (#A18) says, “Legibility is
difficult to read, not enough contrast for this search area… Make it a little bit easier to read.
Launch was terrific simple action. White on blue text.”
2 users recommended making improvements to the lexicon used in OpenGrid, commenting that
the language is too technical for non-developer audiences. Susan (#B5) says, “The language
was too difficult to understand.”
Was this your first CUTGroup test?
Yes 39% (9)
No 61% (14)
Do you like the CUTGroup?
Yes 100% (23)
No 0%
Did you like this CUTGroup test?
Yes 91% (21)
No 9% (2)
Anything else to add?
Susan (#B5) says, “No comment. Awful.”
Gabi (#B12) says, “This was the hardest test.”
Bronco (#A15) says, “This was fun! I like breaking databases!”
24

Yoonie (#A18) says, “This tool feels a lot less refined or finalized than other tools tested.
Seems like an earlier version.”
25

Make $20 for a test of a Chicago data website.
Hi <<First Name>>,
We've got a new opportunity for you to make money in the Civic User Testing Group
(CUTGroup) by testing a website where you can navigate Chicago data on your
computer or mobile device.
Are you available for a 30 - 45 minute in-person test on Wednesday, April
20 between 4:00 - 7:30 PM? If so, please complete this form.
We are in the process of gathering responses, so we will be in touch in a couple of
days to check availability, conﬁrm the location, and conﬁrm that you are right for this
test. We are looking for about 15-20 testers.
For your participation, you will receive a $20 VISA gift card. You'll also help make
better software for Cook County. Thanks for being a member of the CUTGroup! As
usual, call or write with questions.
Remember: if you want to be a part of this test, please complete this form.
--
Sonja Marziano
(312) 565-3274
smarziano@cct.org
Facebook Twitter Website
Subscribe Share Past Issues Translate

CUTGroup Test: Chicago Data Website
If you would like to participate in the next CUTGroup test, please complete this form. We will follow‑up to check your
availability and let you know if you qualify for this test!
Smart Chicago values your online privacy and is committed to safeguarding the data you share with us and your
online privacy. Our On‑Line Privacy Policy applies to data collected at www.smartchicago2012.wufoo.com domain.
Email *
Yes
No
Are you available for an in‑person test on Wednesday, April 20 for 30‑
45 minutes between 4:00 and 7:30 PM? *
Yes
No
Are you 18 years old or older? *
5 ‑ Very familiar
4 ‑ Somewhat familiar
3 ‑ Neutral
2 ‑ Not very familiar
1 ‑ Not at all familiar
How familiar are you with what is happening in your neighborhood? *
Tell us more.
If anything, what are you interested in learning about in your
neighborhood that you do not already know? *
Yes
No
Have you looked up information online about the neighborhood you live
in now? *
Yes
No
Do you use maps to explore your neighborhood? *
What characteristics would you look for when moving to a new
neighborhood? *
How would you search or ﬁnd that information? *

Yes
No
Have you ever used the City of Chicago’s data portal before? *
Yes
No
Do you work with or analyze datasets in your personal or professional
life? *

Let us know when you could join us on Wednesday for this test.
Hi <<First Name>>,
Thanks for your interest in our CUTGroup test of a website where you can navigate
Chicago data on your computer or mobile device. We'd like to know if you can still
make it next Wednesday and, if so, what time you want. Here are the details:
Wednesday, April 20, 2016
Between 4:00 - 7:30 PM
Chicago Public Library Legler Branch
115 S. Pulaski Road, Chicago IL 60624
If you can still make it on this night, please let us know what time slot you want
by completing this form. For your participation, you will receive a $20 VISA gift card.
We will get back to you with a conﬁrmation of your time.
If this location doesn't work for you, or if you just can't make it on this night, after all,
no big deal. If you have any questions or comments, just hit "reply" and let me know
what you think.
Remember: use this form to tell us about your availability.
Thanks for being a member of the CUTGroup!
--
Sonja Marziano
(312) 565-3274
smarziano@cct.org

CUTGroup Test: Chicago Data Website Availability
If you would like to participate in our CUTGroup test, please complete this form. The test is Wednesday, April 20
between 4:00 ‑ 7:30 PM at the Chicago Public Library Legler Branch, 115 S Pulaski Rd, Chicago, IL 60624.
Email *
Yes
No
Are you still available on Wednesday, April 20 to attend this test in
person? *
4:00 ‑ 4:30 PM ‑‑ This slot is now full! Please choose another.
6:15 ‑ 6:45 PM
7:00 ‑ 7:30 PM
Sorry, this day does not work for me!
Test time slot 7 remaining.
Which of these time slots would work for you? Please select all that may
apply, and we will get back to you with your conﬁrmed time. *
Yes
No
Would you be willing to bring your mobile device to this test to test
with? *

Thanks for being willing to help out.
Hi <<First Name>>,
Thanks for your interest in our CUTGroup test of a website that helps you navigate
Chicago data on your computer or mobile device. We really appreciate your
response to our call-out. We got a lot of responses for this test, and unfortunately,
you were not selected. We took a look at all of the responses we received especially
around the data analysis tools you use, how you've used the Chicago data portal,
how you get neighborhood information and more.
To keep you in the loop, the website we are testing is OpenGrid. If you are interested
in contributing this project or have feedback you can email the developer team
directly or contribute to their GitHub.
Even though you were not selected for this test, we have a lot more tests coming up.
We will send you info on our next test, and we'll deﬁnitely try to spread the tests
around so that everyone gets what they want out of the CUTGroup experience.
Thanks for being a member of the CUTGroup. As usual, call or write with questions.
--
Sonja Marziano
(312) 565-3274
smarziano@cct.org
Facebook Twitter Website

CUTGroup 24: OpenGrid
Thanks for participating in our test of OpenGrid, a website that lets you navigate Chicago data.
About you
First, some preliminaries. Keep in mind that we are going to publish the results of this test, but will not associate your name or
email address with the results.
Background review
First Last
Name *
Email or Phone Number *
"A" tester
"B" tester
Is this an "A" or "B" tester? *
You can check this on the tester sheet!
Give yourself a tester proﬁle name. *
Proctored by:
What device are you using for this test? *
You can check this on the tester sheet whether they should be using a laptop or their
mobile device.
Safari
Internet Explorer
Chrome
Firefox
Other
What browser are you using right now? *
Can you tell us more about your experience, if any, using data or
datasets? *

Homepage review
A testers should start on http://opengrid.io/
B testers should begin on http://chicago.opengrid.io/opengrid/
Testers have answered screening questions which should be a starting point to this
conversation.
Take a look at the tester sheet for more information!
Review the homepage and tell us about the three things that you notice
ﬁrst.
While you review, try not to click on the next page just yet, but feel free
to scroll up and down the page. *
If you had to describe what this website does, what is your best guess? *
Who do you think this website is targeted to? Why? *
A: Clicked "Launch"
A: Clicked "Explore"
A: Clicked "Contribute"
A: Clicked on video
B: Used quick search
B: Chose Commonly Used Query
B: Added datasets and searched
B: Clicked on "Manage Queries"
Other
After reviewing the homepage, what is the ﬁrst thing you do or click? *

Website review
If you searched, what was the search term you used?
Tell us more about the first thing you did, and why you did it.
What were you expecting to happen? *
Find 311 service requests that are currently open in your neighborhood.
Tell us about your experience *
Yes
No
Did this search meet your expectations? *
Why or why not?
5 ‑ Very easy
4 ‑ Easy
3 ‑ Neutral
2 ‑ Difficult
1 ‑ Very difficult
How easy was it to find open 311 service requests in your
neighborhood? *
How would you improve this search experience? *

Pretend that you are in the process of moving to a new part of Chicago,
use OpenGrid to find information about this area and let us know what
you find.
What did you learn using OpenGrid? *
Search for crime data from the last year in the neighborhood you live in.
*
Yes
No
Did this search meet your expectations? *
Why or why not?
5 ‑ Very easy
4 ‑ Easy
3 ‑ Neutral
2 ‑ Difficult
1 ‑ Very difficult
How easy was it to find crime data from last year? *
How would you improve this experience? *

Overall Review
Next find a dataset that you are interested in learning more about that is
relevant to you. What did you look for? *
Tell us about your experience. How did you search? Can you describe
the information you found? *
5 ‑ Very Easy
4 ‑ Easy
3 ‑ Neutral
2 ‑ Difficult
1 ‑ Very Difficult
Overall, how easy do you think it is to use the OpenGrid website? *
What improvements might you make to make it easier to use?
5 ‑ Very Easy
4 ‑ Easy
3 ‑ Neutral
2 ‑ Difficult
1 ‑ Very Difficult
Overall, how easy do you think it is to understand the content and
information on OpenGrid? *
What improvements might you make for the information to be easier to
understand?
What other types of information would you like to see in OpenGrid that
would be helpful to you as a user?

Yes
No
After reviewing parts of the website, do you think the OpenGrid website
is targeted to you? *
Why or why not? If not, who is the target audience?
Yes
No
Not sure
Do you like the OpenGrid website? *
Why or why not?
5 ‑ Very likely
4 ‑ Likely
3 ‑ Neutral
2 ‑ Not very likely
1 ‑ Not at all likely
How likely are you to use the OpenGrid website in your personal life? *
5 ‑ Very likely
4 ‑ Likely
3 ‑ Neutral
How likely are you to use the OpenGrid website in your professional
life? *
5 ‑ Very likely
4 ‑ Likely
3 ‑ Neutral
How likely are you to share the OpenGrid website or its information with
your neighbors or community members? *

Overall review of the CUTGroup experience
In what other ways would you improve the OpenGrid website? *
Yes
No
Was this your ﬁrst CUTGroup test? *
Yes
No
Do you like the CUTGroup? *
Yes
No
Did you like this CUTGroup test? *
Anything else to add?

Final Report for CUTGroup #24 - OpenGrid

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

Similar to Final Report for CUTGroup #24 - OpenGrid

Similar to Final Report for CUTGroup #24 - OpenGrid (20)

More from Smart Chicago Collaborative

More from Smart Chicago Collaborative (20)

Recently uploaded

Recently uploaded (20)

Final Report for CUTGroup #24 - OpenGrid