Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Final Report for CUTGroup #28 - City of Chicago Open Data Portal Homepage

3,043 views

Published on

For the 28th Civic User Testing Group (CUTGroup) test, Smart Chicago Collaborative tested the redesigned homepage of the City of Chicago’s Open Data Portal. The Open Data Portal allows users to find resources and various datasets regarding the city of Chicago. The City of Chicago Department of Innovation and Technology is working with Socrata to redesign the Open Data Portal, focused currently on the homepage, to be more user-friendly while representing multiple data and technology initiatives and applications created with open data.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Final Report for CUTGroup #28 - City of Chicago Open Data Portal Homepage

  1. 1. CUTGroup #28 Open Data Portal Report by Sonja Marziano, Project Coordinator Christopher Gumienny, CUTGroup Proctor Smart Chicago Collaborative
  2. 2. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 Introduction For the 28th Civic User Testing Group (CUTGroup) test, we tested the redesigned homepage of the City of Chicago’s Open Data Portal. The Open Data Portal allows users to find resources and various datasets regarding the city of Chicago. The City of Chicago Department of Innovation and Technology is working with Socrata to redesign the Open Data Portal, focused currently on the homepage, to be more user-friendly while representing multiple data and technology initiatives and applications created with open data. Target Group On September 22, 2016, we sent an email to 1,172 CUTGroup testers who lived in Chicago, and notification via text message to our testers who prefer to receive notification via text. We wanted to know if they would be available for an in-person test on September 28. When segmenting our testers, we were interested in testing with: ● Testers who use a variety of devices: Android, iOS, and laptops. We wanted to have an almost equal representation on all these devices types to better understand the homepage’s functionality on different devices. ● Testers who have used the Open Data Portal in the past and those who have never used the Open Data Portal before. We wanted to have people compare both data portals while including new, first impressions. ● Testers who have all levels of data experience to see how user-friendly the search functionality is. Test type ● In-person test proctored on both mobile and laptop devices. We included 6 testers on laptop devices, 6 testers on Android mobile devices, and 5 testers on iOS mobile devices (total of 17 testers). ● Testers were paired with proctors to record their interactions and comments or opinions about the City of Chicago Open Data Portal homepage. Goals of Test The main goal is to understand how testers who have some familiarity with the data portal (even minimum) respond to the changes made to the homepage. We want to capture how residents search and what is the best structure for the homepage to make searching easier. Lastly, we want to see how responsive testers are to the other content that relates to the programs and tech initiatives of the City of Chicago. 1
  3. 3. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 Responses 17 testers completed the in-person CUTGroup test of the City of Chicago’s Open Data Portal homepage at Woodson Regional Library, ​9525 S Halsted St, Chicago, IL 60628, ​on September 28, 2016. Background questions What successes or challenges have you had finding data that you were looking for or using the Open Data Portal? Tell us about any of your past experiences. 6 out of 17 testers had no previous experience with datasets or using the Open Data Portal. 8 testers expressed positive experiences using Chicago’s Open Data Portal. For example, CommunityResearcher 201601 (#11)​ said,“Somewhat successful. Used to research a paper a number of years ago, [and] was able to find what I needed on the elderly population.” 3 testers shared challenges using Chicago’s Open Data Portal. Syliva (#3) ​said,“It’s not that data is difficult to access, but the results are not relevant. If I'm looking up restaurant data I don't want school data for instance. Results are not always relevant.” Blessed (#6) ​said,“I usually look for crime in the immediate block. Theft/robbery, person or house. [It] wasn’t accurate.” We learned from this CUTGroup test that there was some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website. Some testers very clearly used the Open Data Portal, while a few testers shared experiences finding city services, which led us to believe that they were speaking about the official City of Chicago website. What are your expectations of what the City of Chicago Open Data Portal would be useful for? Here are some responses from testers about what they expect the City of Chicago Open Data Portal would be useful for (number indicates number of times a response was given as testers sometimes had multiple answers): 6 – Search for specific data 3 – Analyze data 2
  4. 4. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 3 – Find City services 2 – Creates transparency 2 – Find jobs 1 – Find doctors 1 – Mapping 1 – Develop apps using the data 1 – Pay parking tickets 1 – Communicate with Aldermen Gabireads (#2) ​said that the Open Data Portal is useful for “Mapping. Creating thematic maps. Census, crime, restaurant, neighborhood data to make maps and analyze data.” Batman (#14) ​said,“ To have data/info on various municipal systems, licenses, historical data on various going-ons.” M (#15) ​mentioned,“Great if everything that is not confidential was available. Would enable better social science and community involvement, services, security, etc.” Initial Homepage Review Review the homepage and tell us about your first impressions. While you review, try not to click on the next page just yet, but feel free to scroll up and down the page. When reviewing the redesigned homepage for the first time, 13 testers had initial positive impressions, 2 had critical impressions, and 2 had neutral impressions. Mobile users The majority of mobile testers had a positive reaction to the homepage with 5 testers mentioning that it seemed straight-forward. 3 testers had negative responses when viewing the homepage on their mobile device. 4 out of 11 testers specifically mentioned the categories, whereas only 1 tester mentioned the search bar. Other things testers mentioned: Divvy (3), “Service Tracker” (2), clear/readable font (2), specific datasets (2). Funky Burrito (#10) ​says, “Well laid out, straightforward. Right amount of pictures versus text content. Feels easy to navigate based on criteria of information.” VP (#12) ​thought that the homepage is “Well organized. Preview of what is to come is on the banner. I assume that I would be able to swipe but you cannot. Also lots of scrolling. Visually seems very cramped, not aligned.” 3
  5. 5. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 GPBlight (#4)​ believe that “Flashing promotions are unnecessary. It should focus on the top three items people search for, not what they are pushing" in relation to the Divvy advertisement. Laptop users 4 out of the 6 testers who viewed the homepage mentioned the categories in their initial review, while no one mentioned the search bar. Most testers had a positive reaction to the homepage, while 1 tester specifically said that there was too much information. Other things testers mentioned: Divvy (1), “Chicago apps” (1), “How to video” (1). Batman (#14) ​says, “Looks clean and modern. The old one wasn't easy to navigate and this page has broken up categories to find stuff more easily. Info is more visual.” TJ (#5) ​thought there is “Too much information and stuff on the homepage. ‘Digital Chicago’ section should be bigger.” Lhibbard (#7) ​says, “Not exactly what I expected. Didn't expect to see a map or information about Divvy. Thought it would highlight city services. Noticed the Map first. Looks more fresh and updated.” What are three actions that you know you can do just from reviewing this first page? Here are responses we have heard from testers about actions they can do just from reviewing the homepage (number indicates number of times a response was given as testers sometimes had multiple answers): 6 – Browse data catalog by choosing category 6 – Tutorials/Videos 4 – Learn about community 4 – Use Chicago apps 4 – Request city services (street sweeping, garbage pick-up, 311, etc) 2 – Find events 2 – Navigate crime data 2 – Click on menu 2 – Type information in search box 2 – Transportation 2 – Find health and human services 1 – Environmental data 4
  6. 6. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 1 – Review free maps Thrones (#1)​ mentioned,“Look at different events, coordinate routes, and find out what's going on in your community.” Syliva (#3)​ said,“Click on menu, can type in info in search box, or I can choose categories from below.” Lhibbard (#7) ​said,“Look up community, health and human services, look up information about transportation.” Who do you think this website is designed for? Why? 14 testers mentioned that this Open Data Portal is designed for the general public/residents to help find city services/resources (3), government transparency (1), and access data (1). Testers who said it was for residents said so because it is “easy to use and not laden with jargon” (​Gabireads #2​) and “to help people find resources in the City of Chicago” (​Thrones #1​). 3 testers specifically said the Open Data Portal is designed with people with moderate to high technical savviness. 1 tester mentioned it was for people interested in data analysis. 1 tester said it was built for business owners, while another said it was built for developers. After reviewing the homepage, what is the first thing you want to do or click? Clicked on a data catalog category 59% (10) Search to find a specific dataset 18% (3) Clicked on a "Digital Chicago" topic 6% (1) Clicked on a "Chicago App" 6% (1) Other: Clicked on menu on the upper right 6% (1) Other: Clicked on Divvy banner 6% (1) What did you search? Or what was the category you chose? Category: ​FOIA (3), Education (3), Events (2), Transportation (1), Health & Human Services (1) Search terms: ​Food (1), Building safety data (1), Community (1) Tell us more about the first thing you did, and why you did it. What were you expecting to happen? 5 testers expected to find more general information about what the website does or the type of information listed under a category. 4 testers had specific topics to query. 1 tester expected city service options. 5
  7. 7. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 Clicked on a data catalog category Mobile users (6) FOIA 50% (3) Education 33% (2) Events 17% (1) Laptop users (6) Human Health Services 32% (2) Transportation 17% (1) Community 17% (1) Education 17% (1) Events 17% (1) TJ (#5) ​chose the transportation category because he “wanted to find CTA maps dealing with train transportation” and it “met the expectation.” “Search to find a specific dataset…” Mobile users (1) “Food” 100% (1) Syliva (#3) ​searched the term “food” and said,“Food, because I enjoy fine dining. Was expecting to see a list of restaurants.” Laptop users (1) “Building safety data” 100% (1) “Other” Mobile users (4) Clicked on a "Chicago App": OpenGrid 25% (1) Clicked on a "Digital Chicago" topic 25% (1) Other: Clicked on menu on the upper right 25% (1) Other: Clicked on Divvy banner 25% (1) VP (#12) ​clicked on Divvy banner because the user “thought it would take me to a list of subcategories but it didn’t.” 6
  8. 8. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 Next, search for a dataset you're interested in learning more about. How did you search? Chose a data catalog category 59% (10) "Search to find a specific dataset..." box 35% (6) Used search in navigation 6% (1) Search term (if used) Animal control (1), CPS (1), Crime statistics for Lake View (1), Government (1), Lead testing schools (1), Nursing (1), Potholes (1) Category (if used) Buildings (2), Community (1), Events (1), FOIA (1), Historic Preservation (1), Sanitation (1), Service Requests (1), Transportation (1) Tell us more about your search experience. Did you find what you were looking for? 8 testers had difficulty finding datasets they wanted or said the “language” made it hard to find desired results. 2 testers expected city services such as employment or 311 service requests. 3 testers were still exploring the site generally trying to learn more about what is listed in certain categories. 2 testers mentioned that the data appeared outdated, while 2 other testers could not distinguish a rational order to the results. Miko (#13) ​typed in “public transportation and received “Open Spaces” results which were not relevant to her. She mentioned how this site uses “A certain type of lingo for certain type of people… not geared towards a regular Jane or Joe.” Did the search meet your expectations? Out of the 7 testers who searched: Yes 29% (2) No 71% (5) Out of the 10 testers who chose a category: Yes 30% (3) No 70% (7) Why or why not? When expectations were not met, a lot of the time it was because it took too long or too many steps for the tester to find their specific query. Some testers, again, expected city services or 7
  9. 9. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 employment information that could be found on the general City of Chicago website and not necessarily an Open Data Portal. Lhibbard (#7) ​did not find what they were looking for and chose the service requests category and said,“I found the news about 311 service requests. I thought I might be able to make a personal request directly to the alderman.” Thrones (#1) ​searched “nursing” and it did not meet expectations as tester was looking to review human resources or employment information. For those that had expectations that were met, testers were pleased to find and that the homepage provided more options and details. TJ (#5) ​found what they were looking for and chose a data category. They said they were “Looking for libraries and found locations/hours.” Batman (#14) ​searched for “potholes” and found what he expected, “Curious that they list recent pothole fixes. Wanted to see potholes patched recently and got what tester expected.” Search for the building permits dataset. How did you search? "Buildings" Category > "Building Permits" dataset 70% (12) "Search to find a specific dataset…." box 24% (4) "Buildings" Category > "Building Permits" data lens 6% (1) One tester, ​Miko (#13)​, first searched, but the results did not load on their mobile device, so the tester then clicked on the “Buildings” category. Search term (if used) Building permits (4) Anything else to note about this search experience? Most testers agreed this search was simple and straightforward but ​GPBlight (#4)​ suggested that in the data results there “should be an option for ward which is not on the map” and Batman (#14)​ thought it was “Interesting that some things go back 2 years, others 10 years. Might be useful if data can be accessed further back.” Next, search for a map of crime data. How did you search? "Public Safety" Category > "Crimes" map 35% (6) "Public Safety" Category > "Crimes - 2001 to present" dataset 23% (4) "Public Safety" Category > "Crimes - 2001 to present" data lens 18% (3) 8
  10. 10. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 “Search to find a specific dataset…” box 18% (3) Chose "Crimes - 2001 to present" link on header image 6% (1) Search term (if used) News (1), crime map (1), crime data (1) Anything else to note about this search experience? 6 of 17 testers had little to no difficulty finding the map, however, a couple of testers commented on the map functions and legend. For example, ​Gabireads (#2) ​zoomed in near her home and said that it’s not clear what it is a map of, nor the time period. 7 testers knew that crime data would be listed under the “Public Safety” category but chose either the dataset or data lens result, which appeared higher in the results list. These testers, therefore, did not find the crime data map as tasked. One tester, ​Miko (#13)​,​ ​reviewed the categories before typing in “News” into the search box. This did not come up with relevant results, and so she typed in “Crime data” in the search box and then chose the data lens option. Next, find the potholes that have been recently fixed. How did you search? “Service Requests” Category > Potholes Patched Last 7 Days 76% (13) “Search to find a specific dataset…” 18% (3) Other: “Sanitation Category” then “Search to find a specific dataset…” 6% (1) 6 testers out of the 13 who clicked the “Service Requests” category attempted to find this data under another category first. Search term (if used) Potholes (1), Potholes fixed (1), Pot holes (1) Anything else to note about this search experience? While almost all of the testers were able to find this dataset, a lot of testers had difficulty choosing the correct category at first. 6 testers expected that potholes dataset would be either in the Sanitation, Environment, Public safety, or Transportation categories. One tester, ​Thornes (#1)​, tried to find this information first in the “Sanitation” category, then in the “Public Safety” category, before finally switching to the search functionality and searching for “potholes.” 9
  11. 11. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 Next, search for active business licenses. How did you search? “Community Category > “Business Licenses - Active” dataset 65%(11) “​Search to find a specific dataset…” 35%(6) 10 out of 11 testers who eventually found the active business licenses dataset explored other categories first. 5 out of the 6 testers who used the search functionality had difficulties with other search terms. Search term (if used) Active business licenses (5), business licenses (1) Anything else to note about this search experience? 15 out of 17 testers had some level of difficulty in finding active business licenses either by choosing a category or finding search. 6 testers specifically tried the “Administration & Finance” category first, while 4 other testers suggested the “Business” category. Brittsky (#9) ​explored the “Buildings” category before searching “active business license.” Testers who searched had some difficulties with misspellings in their search terms. ​M (#15)​ first clicked on the “Administration & Finance” category and then searched “license” and could not find the correct data. Searching “Business licenses” led to a successful search. Find the City of Chicago's community boundaries map. How did you search? "Facilities & Geo. Boundaries" Category > "Boundaries - Community Areas (current)" map 65% (11) “Search to find a specific dataset…” 29% (5) Other: Chose a Digital Chicago article 6% (1) 4 out of the 11 testers who selected a category first chose a different category before choosing the “Facilities & Geo. Boundaries” one. Search term (if used) City of Chicago community boundaries map (1),community boundaries map (2), community boundaries (1), community areas (1). Anything else to note about this search experience? 4 testers expected the community boundaries map to be located under the “Community” category, and not only the “Facilities & Geo. Boundaries” category. ​TRobin312 (#17) ​even pointed out that the hover over help text of the “Community” said “boundaries” would be found in this category. The correct result, however, was not apparent in the many search results. 10
  12. 12. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 One tester, ​Miko (#13), ​was viewing the site on her mobile device and​ ​clicked the “Digital Chicago” map image for the article “Building Safety Data on the Open Data Portal” because this image appeared to contain a boundary map. She tried to search, but the magnifying glass button was not working and therefore she assumed her results were invalid and became frustrated with this experience. She also attempted searching in the “Parks and Recreation” category. The majority of testers had an easier time finding the correct results for this task than the last task of finding active business licenses. Overall, how easy was it for you to find these datasets? 5 – Very easy 35% (6) 4 – Easy 35% (6) 3 – Neutral 24% (4) 2 – Difficult 6% (1) 1 – Very difficult 0% Tell us more about your search experiences and how you would improve the site. While the majority of testers said it was “very easy” or “easy” to find datasets, there were clear improvements that should be made. There is also evidence in how testers completed the tasks that indicate that categories and search terms were not always intuitive and it sometimes took testers multiple tries to complete the task. 2 testers specifically told us improvements that were about city services and not open data. Mobile Testers on mobile devices discovered a bug with using the magnifying glass button that did not function. On mobile devices, we saw that filters were rarely used, and some testers found it easier to find a dataset using other options such as going back to the homepage and trying another category or using the “Search for a Specific Dataset…” box function. Laptop We learned that the search bar was a bit lost on the homepage especially on laptop devices. A couple of testers suggested to make the search box bigger and a higher contrast. 11
  13. 13. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 Next, explore the "Digital Chicago" section on the homepage. Tell us about the information you find. 10 out of 17 testers clicked on the Array of Things article, which is the first article under the “Digital Chicago” section. 4 testers exclaimed that they did not know what “Array of Things” means and did not find the descriptive text helpful. When reviewing the “Digital Chicago section, 2 testers mentioned that that the “Month in Review: July 2016” article was out of date. 2 testers tried to click on the “Digital Chicago” header, but it was not hyperlinked as expected. 2 testers also mentioned that they do not know what “Digital Chicago” means. One tester, who was viewing this on her mobile device, did not think that “Digital Chicago” was a section different from the rest of the page. Lhibbard (#7) ​mentioned that this section “seemed more friendly to city employees than city residents.” How relevant do you think the "Digital Chicago" section is to you? 5 – Very relevant 24% (4) 4 – Relevant 24% (4) 3 – Neutral 24% (4) 2 – Not very relevant 12% (2) 1 – Not at all relevant 12% (2) Not applicable 6% (1) Africa (#16) chose a dataset versus reviewing the “Digital Chicago,” and, therefore, is listed as “not applicable” in this question. Why or why not? Here is what some testers said when describing “Digital Chicago” as “relevant” or “very relevant” (8/17 testers): TJ (#5)​ felt this section,“Provides valuable info for the public dealing with buildings,etc.” Thrones (#1)​ was content and that the section was relevant but,“Could have more details. Kind of short.” For those saying it was not relevant (4/17 testers): Lacked descriptive and concise content. One user felt it might be more meaningful to city employees than residents. Some also felt it might be an interesting read from time to time but not necessary to know. 12
  14. 14. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 TRobin312 (#17) ​claimed, “[I] don’t understand what information I'm supposed to get in this section.” Next, explore the "How to Videos" section on the homepage. Tell us about the information you find. Mobile device testers experienced issues in viewing this section because it was not aligned properly on the page and cut off the video and the description. 4 testers mentioned that they thought “How to Videos” are a good way to provide instructions or information, another tester would not get information in this way. 2 testers mentioned that they liked the specific information found on the City of Chicago Department of Innovation & Technology’s YouTube page. Upon viewing the videos in the playlist, 3 testers mentioned that the videos were too long. Funky Burrito (#10) ​suggested breaking down the large videos into smaller modules. Both Blessed (#6) ​and ​Funky Burrito (#10) ​were surprised to be directed to YouTube without any notifications after clicking on “More videos.” How valuable do you think the "How to Videos" section is to you? 5 – Very valuable 41% (7) 4 – Valuable 24% (4) 3 – Neutral 18% (3) 2 – Not very valuable 12% (2) 1 – Not at all valuable 6% (1) Why or why not? Valuable Testers that found this section valuable thought that it was eye-catching and the videos provided nice overviews/tutorials of different subjects. ​Gabireads (#2) ​said,“ Easy way to get an overview of something… good way to provide instruction and make robust use.” Not valuable Testers that found this section not valuable felt that the videos should be shortened. 2 testers said there should have been a notification when leaving the main site for YouTube. ​Blessed (#6) ​also shared that they are “Not interested in [viewing] a video to find one piece of information.” 13
  15. 15. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 Next, review the "Chicago Apps" section on the homepage. Tell us about the information you find. Many testers (11) found out they can find info on when their streets would be plowed/swept and could request 311 services which they felt was useful. 1 tester on mobile felt “Chicago Apps” was not distinctly separated from rest of page. 2 testers tried to click on “Chicago apps” header. Were you familiar with any of these apps? If so, which one(s)? 15 testers were not familiar with the apps.1 tester was familiar with 311 service tracker app while another had heard of snow plow tracker app but never used it. How likely are you to use these "Chicago apps" in the future? 5 – Very likely 59% (10) 4 – Likely 17.5% (3) 3 – Neutral 17.5% (3) 2 – Not very likely 6% (1) 1 – Not at all likely 0% Why or why not? Most testers (13) said they would be likely to use these apps in the future as they are convenient, faster than calling a city employee, and, overall, useful to have as residents. VP (#12) ​said he was unlikely to use this “Chicago apps” section in the future. He mentioned that there’s “No need. Just do a Google search.” GPBlight (#4) ​was neutral because“In my neighborhood, the data is seriously flawed, which makes it inaccurate." Overall, how easy or difficult did you think it is to use the new redesigned Data Portal homepage? 5 – Very easy 41% (7) 4 – Easy 47% (8) 3 – Neutral 6%(1) 2 – Difficult 6%(1) 1 – Very difficult 0% What makes you say this? The majority of testers (15) found the new, redesigned homepage easy to use since they found what they were looking for even if it took testers multiple tries to complete tasks. Some did, 14
  16. 16. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 however, agree that the search “language” and function could be improved to make it give better results when trying to search. Categorization could also be improved as many users expected to find one thing in one area and not another. After reviewing the site, do you think you are the target audience for the Data Portal? Yes ​76% (13) No 12% (2) Not sure 12% (2) If not, who is the target audience? Blessed (#6) ​also did not feel they were the target audience and claimed it was for “rich people. M (#15) ​was unsure and said,“Broad target audience. At least some parts of the site are intuitive and do not require data science background but if you do have it, easy to find data.” How likely are you to use the City of Chicago's Open Data Portal in your personal life? 5 – Very likely 58% (10) 4 – Likely 18% (3) 3 – Neutral 18% (3) 2 – Not very likely 6% (1) 1 – Not at all likely 0% Why or why not? Out of the 13 testers who are likely to use it in their personal life, only 3 testers said they would use it specifically for data purposes. 2 out of these 13 testers mentioned that they would use if because of the apps. Additionally 2 testers said they would use it in their personal life because they are curious. 5 of the 13 testers would use it because they believe it will give them information about their community or connect them to City services. Those who are neutral or unlikely (4) to use in personal life claimed they don’t use city services or can just use Google to access what they need. How likely are you to use the City of Chicago's Open Data Portal in your professional life? 5 – Very likely 53%(9) 15
  17. 17. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 4 – Likely 29%(5) 3 – Neutral 12%(2) 2 – Not very likely 6%(1) 1 – Not at all likely 0% Why or why not? Many testers (14) are likely to use the data portal in their professional lives as they have a business, work a career requiring use of this data such as a librarian or human resources coordinator, or will use for educational/research purposes. Gabireads (#2)​ is an academic librarian and said,“I work a lot with students who need access to data in a range of areas.” Those who are neutral/unlikely to use it professionally said the info and data available is nice but don’t have a real need for them in their lives or careers. Funky Burrito (#10)​ said,“Perhaps not the datasets but the apps I am very likely to use.” [Question only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal? From our background question, we learned that 11 testers used the Open Data Portal in the past. As we previously mentioned, we understand that there might be some confusion around whether or not people had used the Open Data Portal before or the City of Chicago website. 8 testers (73%) reacted positively and said it is more user friendly due to the addition of categories and making search easier. Testers also shared that the new homepage looks modern and updated. 3 testers (27%) felt neutral or indifferent about the redesigned portal, and mentioned they only care about retrieving data and not design, or that it only targets specific people such as people who recently move here or residential/business developers. What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use, more valuable/important to you, and more likely for you to use it again? In this question, 8 testers did not mention any additional improvements. 3 testers mentioned to make the search bar more visible, 2 testers wanted to see simpler language used, and 2 testers reiterated that better categorization was needed. 1 tester wanted to see the icon for the “Community” changed from a shopping cart to something more relevant. Lastly, another tester 16
  18. 18. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 mentioned using a breadcrumb menu system as there is too much scrolling up and down and each section is not distinctly separated and it seems like a continuous newsfeed. All other improvements were shared when completing previous tasks or reviewing content. CUTGroup Review Was this your first CUTGroup test? Yes 41% (7) No 59% (10) Do you like the CUTGroup? Yes 100% (17) No 0% Did you like this CUTGroup test? Yes 94% (16) No 6% (1) Anything else to add about CUTGroup? “Love it, look forward to more surveys that are better to the city. No matter what age you are.” ​-Blessed (#6) “Learned more with this test. Thanks for coming to the Southside.” ​-Brittsky (#9) GPBlight (#4) ​asked, “When are they going to make the data itself user-friendly, not for those who use data professionally?” Gabireads (#2) ​says,“It was fantastic. Has to remember to keep looking at the site to get to play with these products and sharing them with other people. 17
  19. 19. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 Results Testers believed that the redesigned Chicago Open Data Portal homepage is designed for the general public and residents, but many expected access to city services. At the end of this CUTGroup test, we asked testers if they felt that they are the target audience for the new, redesigned Open Data Portal and 13 testers (76%) said “yes.” Our testers included individuals who had no, little, or some familiarity with the current data portal, and 14 testers mentioned that this site appeared to be designed for the general public and residents. 3 testers specifically said the Open Data Portal is designed for people with moderate to high technical savviness. 1 tester mentioned it was for people interested in data analysis. Another tester said it was built for business owners, while another said it was built for developers. One concern is that testers, even after review of the website, still expected more access to city services and resources that could be found on the official City of Chicago website. There are some possible solutions that would either provide this general service information to residents or better define the purpose of the website. Identify access points to city services or action steps for residents. There are many different levels of creating these access points to city services that residents expected when first viewing the Open Data Portal. A very simple option is to add a link to the City of Chicago website in the navigation bar or footer of the homepage. A second, more complex level is at the category level. For instance, if a user clicks on the “Events” category there could be a link to the City of Chicago events’ page before users even begin to explore datasets. This level of access would require reviewing all of the categories and understanding if users associate city services with those category types. The most complex level is at the dataset level. Since we conducted usability testing primarily on the homepage and the action of searching for datasets and reviewing information, this would require additional user feedback. The suggestion is to connect relevant datasets to the appropriate city services. Therefore, if a user is reviewing data about potholes, as one example, there could be a link to make a pothole service request. Utilize the header image to clarify purpose about the site. “What’s this sales thing? It makes it look like it’s advertising stuff,” ​GPBlight (#4)​ said in response to reviewing the header images. When the homepage first loaded, a bug was 18
  20. 20. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 immediately noticeable because the header images would stack and cause the page to jump upon load. While that is a detectable fix, we identified a larger opportunity to share information about what the website does. Very few testers reacted to the initial “data sale” image and those who did react had negative responses. The first header image, before rotating to either the crime or Divvy data, should define the purpose to the site while being as clear and direct as possible. The current “data sale” language and images confuse some users. Searching for datasets was not always intuitive when using search or categories and testers often had to try multiple times to complete a task. We asked testers to search for six different datasets; one was open-ended, while the rest proctors requested that they find specific datasets: building permits, map of crime data, recently fixed potholes, active business licenses, and Chicago’s community boundaries map. We were interested to see how testers would search and find the datasets and what issues they came across during the process. While 12 testers (70%) indicated that finding these datasets was “easy” or “very easy” we witnessed that it often took testers multiple tries to find the correct dataset and some testers could not find the proper result or chose a relevant, but not requested dataset. Categories were often chosen over search bar, but technical bugs and design influenced those choices. Only 1 tester out of 17 (6%) used the search bar consistently for all of the dataset search tasks, whereas 7 testers (41%) used categories to complete all dataset search tasks. The remainder of testers used both search and categories to find what they were looking for depending on the type of dataset we requested them to find. On mobile devices, the magnifying glass of the search bar was not working properly. Therefore, if testers attempted a search nothing would appear and testers thought the search was broken or that there were no search results. When doing the initial review of the page, only 1 tester mentioned the search bar, which indicates that it is not a prominent part of the page. When asked about what actions they knew they could do from reviewing their page, only 2 testers mentioned the search bar. When asked to take their first action, 3 testers (18%) searched whereas 10 (59%) clicked on a data catalog category. We understand that a lot of these testers did not have much experience with the data portal and the data catalog categories are useful for exploration purposes. Nonetheless, we realized that the search bar was not very prominent for testers. Suggested improvements included larger font, higher contrast, and a more prominent location on the page. 19
  21. 21. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 Categorization needs to be more intuitive and filtering a higher priority in the user experience or search needs to be more flexible. While testers knew where to find building permit and crime data, finding recently patched potholes, active business licenses, and community boundaries was more challenging. In one example, 10 out of 11 testers who found the active business licenses dataset had explored other categories first. Testers expected to find the potholes data in these categories: Public Safety, Environment, and Sanitation. Testers expected to find active business licenses in Administration and Finance. Lastly, testers looking for community boundaries looked in the Community category. Some testers did use the help text on the categories to make decisions about which category to choose, but it was difficult to decipher the order of results and whether a category contained the relevant dataset after choosing a category. Testers did not use the filters on the search results page, and upon reviewing a few datasets without identifying a relevant one, would often rely on search or finding another category. Providing users with an understanding of an order of the results and then the ability to filter easily to find the most appropriate results would be beneficial to this experience. Improving the filter functionality would make it easier to have datasets in multiple relevant categories and give users ways to find the data more quickly. We also witnessed that testers did not always distinguish between the type of data– whether it was a dataset, data lens, filtered view, or map they were choosing. This was evident when we asked testers to find a map of crime data and testers chose the dataset or filtered view results that were higher on the results list. A primary search method should be defined for the user. Currently, users can choose a category or search, but both have their challenges. Outlined above are results and improvements for the categories since that was a prominent way testers searched. For each task, 11 or more testers used the categories in completing their search. If search was more prominent on the page and functioned better on mobile, we could identify if search was a better way for testers to complete these tasks. We did learn that testers had difficulties with the search terms that they used and spelling errors caused stopping points in the experience. The bug on mobile also caused a stopping point that forced testers to use categories. If search becomes the primary method of finding datasets it should be flexible to account for spelling errors and find relevant resources. 20
  22. 22. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 Testers liked the “Digital Chicago” & “How to Videos” resources on the homepage but the “Chicago apps” were seen as being the most relevant. Testers reviewed the other sections of the homepage including the “Digital Chicago” section that showcases recent articles by the City of Chicago’s Department of Innovation and Technology, “How to Videos” that showcase how to use the data portal and other tools, and “Chicago apps” tools that use open data that let people better visualize data (OpenGrid) or is related to city services (311 service tracker). 48% of testers thought the “Digital Chicago” section was “relevant” or “very relevant,” but testers felt that “Digital Chicago” did not mean much to them (calling this section “news” could be an improvement), they had questions about the articles and what they meant, and this section lacked descriptive content for residents. 66% of testers thought the “How to Videos” section was “valuable” or “very valuable” and a lot of testers liked receiving information via video, although there was a consensus that they should be shortened and users should know that they will be directed to YouTube. 76.5% of testers were likely to use the apps under the “Chicago apps” section because they saw them as useful to have for residents. 15 of our 17 (88%) testers were not familiar with these apps prior to visiting the website. The “Chicago apps” section met expectations from testers who at the beginning searched for or were interested in city services. There was positive reaction to these tools and increased the likelihood of using the Open Data Portal in the future. For the resident user, we would suggest placing the “Chicago apps” section higher on the page because it relates with their needs. To avoid the confusion in distinguishing between the Open Data Portal and the City of Chicago website, adding a descriptive sentence about what “Chicago apps” is and why it’s on this page would be valuable. Sharing that these tools are using open data would describe the importance of the open data initiative while giving residents tools that they could use in their daily lives. Conclusion Most testers found the new, redesigned Open Data Portal as an accessible resource in finding information about Chicago. With different levels of familiarity to the current data portal, testers were able to explore datasets and provide feedback to improve the user experience of searching for data. Resources like the “Digital Chicago,” “How to Videos,” and “Chicago apps” is a welcomed addition for residents who thought these resources are valuable and relevant to their needs. Implementing the recommendations we learned from this CUTGroup test and 21
  23. 23. CUTGroup 28 - Chicago Open Data Portal In-person test on September 28, 2016 continuing to include residents in the process of building the Open Data Portal, will ensure a continued user-centric website. 22
  24. 24. Make $20 at a test of the open data portal! Hi <<First Name>>,   We've got a new opportunity for you to make money in the Civic User Testing Group (CUTGroup) by testing a new redesigned version of the City of Chicago's Open Data Portal homepage.    Are you available for a 30 ­ 45 minute in­person test in Chicago on Wednesday, September 28 between 4:00 ­ 7:30 PM?   If so, please complete this form.    We are in the process of gathering responses, so we will be in touch in a couple of days to check availability, confirm the location, and confirm that you are right for this test. We are looking for about 25 testers.    For your participation, you will receive a $20 VISA gift card. You'll also help make better software for Cook County. Thanks for being a member of the CUTGroup! As usual, call or write with questions.   Remember: if you want to be a part of this test, please complete this form. ­­ Sonja Marziano (312) 565­3274  smarziano@cct.org Subscribe Share Past Issues Translate
  25. 25. Facebook Twitter Website Creative Commons Attribution 3.0 Unported License 2016 Smart Chicago Collaborative. unsubscribe from this list    update subscription preferences    Subscribe Share Past Issues Translate
  26. 26. CUTGroup Test: Data Portal Redesign If you would like to participate in the next CUTGroup test, please complete this form. We will follow-up to check your availability and let you know if you qualify for this test! Smart Chicago values your online privacy and is committed to safeguarding the data you share with us and your online privacy. Our On-Line Privacy Policy applies to data collected at www.smartchicago2012.wufoo.com domain. Email * Are you available for an in-person test in Chicago on Wednesday, September 28 for 30-45 minutes between 4:00 and 7:30 PM? * Yes No How familiar are you with the City of Chicago's open data portal? * 5 - Very familiar 4 - Familiar 3 - Neutral 2 - Not very familiar 1 - Not at all familiar Have you ever used the City of Chicago's open data portal before? * Yes No Do you work with or analyze datasets in your personal or professional life? * Yes No How familiar are you with the City of Chicago's data and technology programs and initiatives? * 5 - Very familiar 4 - Familiar 3 - Neutral 2 - Not very familiar 1 - Not at all familiar Tell us more about the programs you are familiar with.
  27. 27. CUTGroup 28: Data Portal Redesign Thanks for participating in our test of the City of Chicago Data Portal redesigned homepage. About you First, some preliminaries. Keep in mind that we are going to publish the results of this test, but will not associate your name or email address with the results. Background First Last Name * Email * Give yourself a tester profile name. * Proctored by What device are you using for this test? * What browser are you using right now? * Chrome Firefox Internet Explorer Safari Other What successes or challenges have you had finding data that you were looking for or using the open data portal? Tell us about any of your past experiences. What are your expectations of what the City of Chicago Open Data Portal would be useful for?
  28. 28. Initial homepage review Review the homepage and tell us about your first impressions. While you review, try not to click on the next page just yet, but feel free to scroll up and down the page. * What are three actions that you know you can do just from reviewing this first page? * Who do you think this website is designed for? Why? * After reviewing the homepage, what is the first thing you want to do or click? * Search to find a specific dataset Clicked on a data catalog category Clicked on a "Digital Chicago" topic Clicked on "How to Videos" link Clicked on a "Chicago App" Clicked "Developer Docs" in navigation bar Clicked "Feedback" in navigation bar Clicked "Data Catalog" in navigation bar Other What did you search? Or what was the category you chose? Tell us more about the first thing you did, and why you did it. What were you expecting to happen? *
  29. 29. Website review Next, search for a dataset you're interested in learning more about. How did you search? * "Search to find a specific dataset..." box Chose a data catalog category Used search in navigation bar Other Search term (if used) Tell us more about your search experience. Did you find what you were looking for? * Did the search meet your expectations? * Yes No Why or why not? Search for the building permits dataset. How did you search? * "Search to find a specific dataset..." box "Buildings" Category > "Building Permits" dataset "Buildings" Category > "Building Permits" data lens "Data Catalog" in navigation bar > Search "Data Catalog" in navigation bar > "Buildings" Category Search box in navigation bar Other
  30. 30. Search term (if used) Anything else to note about this search experience? Next, search for a map of crime data. How did you search? * "Search to find a specific dataset..." box "Public Safety" Category > "Crimes - 2001 to present" data lens "Public Safety" Category > "Crimes - 2001 to present" dataset "Public Safety" Category > "Crimes" map Chose "Crimes - 2001 to present" link on header image Search in navigation bar Other Search term (if used) Anything else to note about this search experience? Next, find the potholes that have been recently fixed. How did you search? * "Search to find a specific dataset..." box "Service Requests" Category > "Potholes Patched Last Seven Days" "Service Requests" Category > "311 Service Requests - Pot Holes Requested" dataset Search in navigation bar Other Search term (if used) Anything else to note about this search experience? Next, search for active business licenses. How did you search? * "Search to find a specific dataset..." box "Community" Category > "Business Licenses - Active" Filtered View "Community" Category > "Business Licenses" Dataset
  31. 31. "Community" Category > "Business Licenses" Dataset Search in navigation bar Other Search term (if used) Anything else to note about this search experience? Find the City of Chicago's community boundaries map. How did you search? * "Search to find a specific dataset..." box "Facilities & Geo. Boundaries" Category > "Boundaries - Community Areas (current)" map Search in navigation bar Other Search term (if used) Anything else to note about this search experience? Overall, how easy was it for you to find these datasets? * 5 - Very Easy 4 - Easy 3 - Neutral 2 - Difficult 1 - Very Difficult Tell us more about your search experiences and how you would improve the site. * Overall, how easy was it for you to find these datasets? * 5 - Very Easy 4 - Easy
  32. 32. 3 - Neutral 2 - Difficult 1 - Very Difficult Next explore the "Digital Chicago" section on the homepage. Tell us about the information you find. * How relevant do you think the "Digital Chicago" section is to you? * 5 - Very relevant 4 - Relevant 3 - Neutral 2 - Not very relevant 1 - Not at all relevant Why or why not? * Next explore the "How to Videos" section on the homepage. Tell us about the information you find. * How valuable do you think the "How to Videos" section is to you? * 5 - Very valuable 4 - Valuable 3 - Neutral 2 - Not very valuable 1 - Not at all valuable Why or why not? * Next review the "Chicago Apps" section on the homepage. Tell us about the information you find. *
  33. 33. Overall Review Were you familiar with any of these apps? If so, which one(s)? How likely are you to use these "Chicago apps" in the future? * 5 - Very likely 4 - Likely 3 - Neutral 2 - Not very likely 1 - Not at all likely Why or why not? Overall, how easy or difficult did you think it is to use the new redesigned Data Portal homepage? * 5 - Very easy 4 - Easy 3 - Neutral 2 - Difficult 1 - Very difficult What makes you say this? After reviewing the site, do you think you are the target audience for the Data Portal? * Yes No Not sure If not, who is the target audience? How likely are you to use the City of Chicago's Open Data Portal in your personal life? * 5 - Very likely 4 - Likely
  34. 34. CUTGroup Review 4 - Likely 3 - Neutral 2 - Not very likely 1 - Not at all likely Why or why not? How likely are you to use the City of Chicago's Open Data Portal in your professional life? * 5 - Very likely 4 - Likely 3 - Neutral 2 - Not very likely 1 - Not at all likely Why or why not? [Only for testers who used the data portal in the past] How does this redesigned data portal homepage compare to what you remember of the current portal? What improvements would you make to City of Chicago Open Data portal homepage to make this site easier to use, more valuable/important to you, and more likely for you to use it again? * Was this your first CUTGroup test? * Yes No Do you like the CUTGroup? * Yes No Did you like this CUTGroup test? * Yes
  35. 35. Yes No Anything else to add about CUTGroup?

×