Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Final Report for CUTGroup #17 - Ventra Mobile App

2,426 views

Published on

For our seventeenth Civic User Testing Group (CUTGroup) session, we tested the Ventra Chicago mobile transit app. The Ventra app allows riders to manage their Ventra account, buy mobile tickets for use on Metra, get notifications about their account, and other features.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Final Report for CUTGroup #17 - Ventra Mobile App

  1. 1. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 Introduction For our seventeenth Civic User Testing session, we tested the new Ventra mobile transit app. The Ventra app allows you to manage your Ventra account, buy mobile tickets for use on Metra, get notifications about your account, and other features. Target Group: On June 28, we sent out an email to all of our 880 CUTGroup participants. We wanted to know if they would be available for a remote test within the timeframe of July 7 – 15, and then an in-person test either on July 15 or 16. When segmenting our testers we were interested in:  A mix of testers with different types of mobile devices (Android/iPhone)  Testers who did or didn’t have a Ventra card  Testers who add value in different circumstances  A mix of riders on the CTA bus, CTA trains, Metra, or Pace  Testers who commute on a regular basis Test Type:  Remote test component where testers would download the beta version of the Ventra app and use it in their normal day-to-day routine and answer brief questions about their experiences  In-person test component where testers would view different features on their own device and complete tasks  In addition, we also did some testing during the in-person test component on test devices so testers could purchase tickets and create new accounts  All testers were paired with a proctor, who asked them questions and took down their responses and actions Goals of Test: Here is what we heard from Tony Coppoletta, Manager, External Electronic Communications of the Chicago Transit Authority, about what the Ventra app team wanted to learn from this CUTGroup test: “The CTA has an incredibly varied set of use cases and we look forward to how all our riders can benefit from this app's feature set. In addition to some public outreach to invite people to test our app before public release, we want to try and the diverse and experienced testers the CUTGroup offers would be an excellent addition to our real-world user testing of the app. The focus is to trap bugs, ensure transactions happen as expected, good usability and a solid UX overall.” 1
  2. 2. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 Part I Responses 27 testers completed Part 1 of the CUTGroup test, which was a remote test where testers used the Ventra app in their normal day-to-day routine. When using the app for the first time which of these actions did you do? Logged in to an existing account 88% (24) Created a new account 4% (1) Registered a Ventra Card 8% (2) Purchased a Ventra Card 0% Other: 0% Have you added CTA transit value or passes to your account using the Ventra app? Yes 30% (8) No 70% (19) Have you purchased Metra tickets? Yes 19% (5) No 81% (22) If you have made a purchase, please tell us about that experience. Here are some comments from testers who did not make a purchase during the remote test: ChgoKC says, “I tried to add value to the account, but it asked for my Ventra password, which I entered (it's 4 digits), and it said the password had to be 8 digits. I tried my password for my bank account, but that was ‘incorrect.’ So not sure what password it's looking for.” Mischievous Metra maverick has not made a purchase and is “still waiting on function to upload existing Metra month pass into app. I purchase monthly Metra passes through a Conexis pre-tax commuter benefit program.” Gr8fl4CTA said, “I haven't made a purchase because I have a monthly pass. I do not like that you can only add $5 to the card. There should be an option to either fill out the amount- especially since most rides are $2.25 or $2.50. People shouldn't be forced to put $5 on their card if they don't need all of that.” Here are some responses from testers who made a purchase during the remote test: Brown line rider / occasional UP-N Metra rider says, “There was a little trouble with the Metra ticket purchase: my original Ventra password was too short. I attempted to change my password in the app, but was not successful. (Unbeknownst to me at the time, there are further restrictions on passwords beyond their minimum eight-character length.) I then logged into the 2
  3. 3. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 website and changed my password there, where the other password requirements (a numeral, a special character, etc) were made explicit. Once that was done, I was able to buy a Metra ticket through the app.” CTA/Metra Commuter/Evil Genius says, “Purchasing in general is easy. Multiple times, I've purchased a ticket while walking to the train. I love the ability to pay with my Ventra auto- reloading balance.” Frequent CTA / Occasional Metra says, “Purchasing the Metra tickets was very easy - I only purchased one-way tickets so far using the app, but it seemed very straightforward in choosing which stops and identifying the ticket type I wanted. The conductor that checked my mobile ticket on UP-N was educated on the system and he had no trouble accepting the ticket; that was my only concern about using the Ventra app to buy a Metra ticket.” Systems engineer CTA rider says, “I added a 30 day pass to an existing transit account and it went very smoothly. I'm also REALLY, REALLY impressed with how split tenders for payments were implemented. It's really intuitive and I really like that feature.” What are some features that you have used in the app? Check all that apply. Review account history 81% (22) Used Trip Tools 78% (21) Looked up Ventra Value 67% (18) Add Transit Value 22% (6) Buy Metra Tickets 15% (4) Add Autoload 8% (2) Add CTA Passes 4% (1) Here are some other features testers tried to use:  Tried to remove a Ventra card that wasn’t needed on the account, but the feature was not implemented yet  Looked up transit history  Removed old cards  Changed password Tell us about your overall experience using the Ventra app. 3 testers mentioned that sometimes the app was slow to load. Additionally, 3 testers either expected or wanted Near Field Communication (NFC) capabilities. Otherwise, testers had a wide range of responses when describing their overall experience using the app. Here are some improvements we heard:  Show password  Be able to transfer value from one card to another  Improve the to back button so it does not kick the user out of the app  Route planning in Trip Tools Blind Transit rider says, “I spent a bit of time trying to make my active reduced fare card for people with disabilities. I do not find the app or the website compatible. All the customer service agents are able to see my reduced fare permit/ventra card, but I cannot see my active 3
  4. 4. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 DRAFT card. I can only see my closed paratransit Ventra card. It is a closed account because I never activated it. I am trying to correct that issue, but in the meantime I have been using the card that is not visible to me in either the app, or the website. I was able to use the trip tools and found that to be extremely helpful and cover all the transit options I use.” Metra commuter says, “Overall it's pretty nice. I think it will definitely make commuting a lot easier. Especially with metra. If you're late and aren't able to buy a ticket in the station, you can always buy it in the app and avoid the insane fees. I like it a lot.” CTA Rider says, “It's a good, reliable app. One thing I noticed is that when you add value it appears instantly, versus the online site, where it usually doesn't update until you log off and log back in again, so that's a plus. I also like that Bus Tracker is built into the app, which saves time from having to search it on the internet.” How appealing do you find the visual design of the app? 5 – Very appealing 15% (4) 4 – Appealing 52% (14) 3 – Neutral 33% (9) 2 – Not very appealing 0% 1 – Not at all appealing 0% Tell us about what you like or do not like about the visual design of the app. How would you improve it? These responses are from testers who were neutral about the visual design of the app: Daily CTA Rider MW says, “I like it. It’s functional. It’s not eye candy but then, it doesn’t have to be.” Blind Transit rider says, “There are some buttons or links that do not have text labels, or alt text labels. This is important for accessibility.” CTA rider KB says, “Things are too close together in many places.” These are responses from testers who thought the visual design of the app is appealing, but had some suggestions for improving the design: Frequent CTA / Occasional Metra mentioned, “I have two Ventra cards registered to my account, and there are two semicircles that indicate two cards below the card name. This is a problematic design for an iPhone because shaded dots usually indicate something that a user can swipe on the screen, but in the Ventra app the user is expected to tap on the arrow on the far right to switch cards. I know the correct way to switch between them and I'm still doing it wrong.” CTA rider AC would change the color of the cursor from white to black or another darker color. CTA Rider AC says “It is really annoying that I can’t see where the cursor is.” 4
  5. 5. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 Occasional CTA rider says, “The alerts in the Trip tools are a very light font and not very easy to read.” Here are a couple of responses from testers who thought the visual design is very appealing: Gr8fl4CTA says, “Overall, I like the look and ease of using the website. I think that I would make the categories of ‘Transit value,’ ‘CTA & Pace passes’ and ‘Metra Mobile Tickets’ more bold or make the grey heading a little darker. The delineation between the categories needs to be clearer.” CTA Rider OP, says, “It's recognizable with the light blue of the website and it has a clean, streamlined interface which makes it easy to use.” How easy has it been to use the Ventra app? 5 – Very Easy 33% (9) 4 – Easy 48% (13) 3 – Neutral (not easy, not difficult) 11% (3) 2 – Difficult 4% (1) 1 – Very Difficult 4% (1) How would you improve this app to make it easier to use? During the remote test, testers gave a lot of suggestions for improvements to the Ventra app. There was not an overwhelming request for one feature over another and, therefore, here is a list of improvements we heard from our testers (the number indicates the number of times a suggestion was mentioned): 2 – Back button needs to be consistent with users’ expectations 2 – Add map features 2 – NFC capabilities 1 – Favorite stops should be shown first in Trip Tools 1 – Change requirements to reduce typing password multiple times within app 1 – Make icons more intuitive 1 – Hide Metra features if not a Metra rider 1 – Spanish option for bilingual speakers 1 – Add the ability to verify funds with bus drivers if card is lost Part II Responses 26 testers came to the in-person sessions (14 came on Day 1, while 12 came on Day 2). Two of these testers did not complete the remote questionnaire, but still provided great feedback during the in- person section. How often did you use the Ventra app while you have had it? Multiple times a day 19% (5) Once a day 23% (6) 5
  6. 6. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 A few times since I have had it 54% (14) Once for testing purposes 0% Other: Unable to login at home 4% (1) How easy was it to make a CTA ticket or pass purchase using the Ventra app? 5 – Very Easy 19% (5) 4 – Easy 8% (2) 3 – Neutral 4% (1) 2 – Difficult 0% 1 – Very Difficult 0% Not applicable 69% (18) How easy was it to purchase a Metra ticket using the Ventra app? 5 – Very Easy 11% (3) 4 – Easy 11% (3) 3 – Neutral 0% 2 – Difficult 0% 1 – Very Difficult 4% (1) Not applicable 73% (19) If you purchased a Metra ticket, did you use it on the train? Yes 15% (4) No 8% (2) Not applicable 77% (20) How was the experience using Metra tickets on the train? 4 out of 26 testers purchased a Metra ticket and used it on a train during the remote portion of the test. Here is what these testers said about using the Metra ticket on the train: Frequent CTA / Occasional Metra says, “Very cool. I was the first one this conductor saw and he was well-trained and knew how to activate it and use it.” SR thought it was excellent! The conductor was excited since it was the first time he got to see it and then had her tap the ticket and then also wanted to see the QR code. Daily CTA Rider (MW) says, "It was really cool. I wasn't sure what it'd be like and didn't want to activate at first, wasn't sure if should until saw conductor." Daily CTA Rider (MW) learned that he can activate when on board. He mentioned that there wasn't a round trip option, but liked that he was able to buy one A-E and E-A. 6
  7. 7. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 CTA/Metra Commuter/Evil Genius said that the conductors weren’t trained when he rode it on July 11. They were not sure about QR scanners. Go to "My Ventra Cards" and review the page. We want to find out if this page works for you and how to improve it. Testers, in general, had a good response to the “My Ventra Cards” page, but there were a lot of suggestions made to improve this page:  A few testers had trouble loading the Transit History page, and it generally, took multiple tries to get it to load. 6 testers mentioned liking the Transit History page when reviewing this page.  Frequent CTA Rider showed us on their LG phone that when the keyboard was up, they were unable to scroll on the page and could not see where they were typing.  Metra rider88 wants to be able to scan their credit card to add the payment information.  Frequent CTA / Occasional Metra thought that if autoload has already been set up for a card, the option should not be there. “I didn’t know what it meant since I already have it set up from online.”  Route 66 Book Stalker expected to see their frequently used Ventra card first, instead of the one they never use.  Testers had questions about why “Autoload” was listed twice, and one tester, Mischievous Metra Maverick mentioned that you do not get a warning on the autoload page and wants a pop-up explaining exactly what you are doing.  Occasional CTA Rider wanted to change the names of the card in the app.  ChgoKC noted that the back button exits the app from the “My Ventra Cards” page rather than going back to the menu. Metra rider88 never registered their card, but did it on the Ventra app because “it was right there!” How would you explain what the following features mean to you. Think about how these values are different from one another: "Add Transit Value" This was very explanatory to all testers, who understood immediately that this would add value to their Ventra cards. "Add Pass" 6 testers were not sure what “Add Pass” meant at first, and two of these testers specifically had to click “Autoload” to better understand the difference between pass and autoload. Daily CTA Rider thought passes should be an option under “Add Transit Value.” "Add Autoload" 6 testers were not sure exactly what the “Add Autoload” feature would do, and there was still some confusion between adding autoload and adding passes. 7
  8. 8. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 What are some websites or apps that you currently use to check on transit times? Here are the websites or apps that testers mentioned (number indicates the number of times an answer was mentioned): 8 – Google Maps 6 – CTA Bus Tracker 5 – Transit Chicago 5 – Transit Stop app 3 – RTA Trip Planner 2 – City Mapper 1 – Goroo 1 – Metra Rail Time Tracker 1 – Boothscoot 1 – Embark CTA 1 – iTrans Metra 1 – iTrans CTA 1 – Chicago Transit Tracker Lite 1 – Buster 1 – CTA Train tracker Visit the "Trip Tools" page on your device and search for transit information near you. Tell us about your experience. Here are improvements or suggestions testers made when searching for transit information near them: 3 testers mentioned that they were more likely to use other trip tools because it does not map out routes. 2 additional testers said that they were expecting a wayfinding feature since the name of the feature was called “Trip Tools.” 14 testers did not use the “Locate Me” feature when searching for transit information near them. Some of this was due to the fact that some testers were aware of the route they would need to take, but 6 testers did not see the “Locate Me” icon. Overall, 8 testers thought the icons (especially the Locate Me and Favorites icons) could be more intuitive. 4 testers mentioned the inconsistency or lack of the Back Button, and the experience that the Back button would sometimes kick the tester out of the app. Here are some other comments from testers: Systems engineer CTA rider would use Google Maps to see the scheduled routes and then use the app to determine when the next available time really is. Systems engineer CTA rider did not use the “Locate me” feature and assumed that stops were sorted by distance. In addition, this tester showed that the second line overlaps with direction information. CTA Savant had some questions about the stop names especially with similarities such as Michigan & South Water or Michigan/South Water. CTA Savant wants to clarify different stops and then provide them on the map to see if they actually located her on the map.” 8
  9. 9. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 DRAFT CTA Rider says, “Term is very misleading, it is actually a bus and train tracker. Trip Tools make me think of real time trip planner…” Step by step directions were expected based on the naming convention. How did you search? Searched by bus/train route 54% (14) Used "Locate Me" feature 46% (12) Searched by stop number 0% Went to favorites 0% Now search for another transit route near your home. How was this experience? 7 out of the 26 testers were able to find a transit route near their home very easily and quickly and did not come across any troubles during their experience. Here is a list of issues that the remainder of the testers faced when finding a transit route near their home (the number indicates the number of times a tester mentioned an issue): 7 – Back button/had to start over 6 – Looking for route planning/tracking that is similar to other apps (such as Google maps) 5 – Stop order/organization did not make sense or could be better 3 – Unfamiliar or reverse routes were difficult Back Button Here are some experiences testers faced with the back button during this portion of the test: Metra rider88 briefly looked at Trip Tools during the remote test, but notices and likes the bus routes. Metra rider88 mentioned that there was no back button on the train route options page, and had to go back to the main menu. CTA rider (KB) says, “it’d be nice if back button had a larger hot area” because it did not feel responsive enough. Systems engineer CTA rider liked searching for stop IDs, but did not like that he could not use the Android back button when using Trip Tools. Stop Order Organization Here are some experiences testers faced with the organization of the bus stops: CTA rider (AC) scrolled a lot to find the correct stop, and then later used A-Z sort. This tester was confused about the order of how stops were listed. CTA Rider (AC) prefers to have stops listed alphabetically, but thinks that order by station might be better for users that are from out of town. 9
  10. 10. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 Metra rider88 also had to scroll a lot to find the bus route, and did not see “Sort A-Z.” Tester refers to the bus routes by the name vs the number, but it is sorted by bus number. Route Tracking/Planning Testers shared with us by using the phrase “Trip Tools” they were expecting more route planning features similar to using Google Maps. For example, one tester specifically called out wanting to see a map during this part of the test. Testers thought this feature would help them choose bus and train routes to get to locations, not only tell when the next train or bus is coming. Changing the name of this feature might assist with people’s expectations of the tool. How did you search? Searched by bus/train route 92% (24) Used "Locate Me" feature 4% (1) Searched by stop number 4% (1) Went to favorites 0% How easy was it for you to find the correct stops you were looking for? 5 – Very Easy 35% (9) 4 – Easy 31% (8) 3 – Neutral 23% (6) 2 – Difficult 11% (3) 1 – Very Difficult 0% How would you improve this experience? Here are some changes that testers recommended for the Trip Tools page (the number indicates the number of times it was mentioned): 6 – Be able to search by name of stop/bus (ex: testers did not know bus number, but cross streets might be better) 4 – Add a map 3 – Improve the sorting of busses/stops 3 – Icons could make more sense (Locate me, Favorites,) 2 – Improve back button 1 – Make it easier to determine direction of bus 1 – Make the cursor black for text boxes to make it easier to see 1 – Make the alerts more noticeable 1 – Less steps to determine transit times How does this feature compare to other trip websites and apps that you have used in the past? 9 out of 26 testers found the Trip Planning feature to be similar to or on par with other websites or apps that they use. Here are some responses from testers: 10
  11. 11. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 Brown line rider / occasional UP-N Metra rider says that this is “Similar to what you’d get on the CTA website only this app looks more this decade. CTA rider (KB) says, “It's good, has about the same functionality that many other apps have, and the alerts here are easier to find” because they are shown more prominently. 9 out of 26 testers thought the Trip Planning features on Ventra were better than other websites or apps they currently use. Here are some comments: CTA rider (AC) says, "I trusted it more because it was affiliated with Ventra." Daily CTA Rider (MW) says, "Actually easier than my other apps! And I think that may be because of simplicity and consistency on the screen and not use of a lot of different colors." 6 out of 26 testers thought this feature was worse than other tools that they use. Here are their comments: Frequent CTA / Occasional Metra says, “This is stop driven. I think in terms of routes. The other apps think in terms of route and not stops. Other apps locate you on a map and tell you what is around. Helps you find the bus and route you need to take.” Riding the 20 every day says, “It's similar, but the lack of map function is a problem.” 2 testers did not provide enough of a thorough response to this question to be added to any of these groups. How would you register a new Ventra card if you wanted to add a card? Where would you find this? 17 out of 26 testers found the plus (+) sign quickly and understood that it meant adding a new Ventra card. 4 additional testers found it confusing, but eventually found it. Here are some responses from testers who had trouble or thought it was confusing: TA thought it was confusing, but went to “My Ventra Cards,” but mentioned that “a new one wouldn’t be ‘my’ Ventra Card, right? Also, you need to remember all of your account information to set this up.” RIDER was having a difficult time finding how to enter a second Ventra card, and accidentally closed the app several times using the hard back button. CTA Rider and Metra rider88 thought that adding a text description would be better. Did you create a new account during the remote test? Yes 38% (10) No 62% (16) 11
  12. 12. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 Create new account section On day 1 of testing, we decided we wanted testers to create a new account to see how that experience worked. We decided to remove the second on day 2 since we got a lot of great feedback from day 1 and did not think it was necessary on the second testing date. We asked testers how easy it was for them to create an account if they created one during the remote test OR if they created one on the test device on day 1 of testing. Please create a new account on the test device. Tell us about your experience as you go through it. Only 9 testers created a new account on the test device. 3 out of the 9 testers missed the password instructions and had to go back and re-do the password. CTA rider (KB) noted that the keyboard does not have a “next field” skip button. SR pointed out a couple of issues such as it being nice to indicate the formatting of the phone number. In addition the error for not entering the city was “Please add a(n) address locality” which could have been explained better. How easy was it for you to create a new account? 19 testers responded: 5 – Very Easy 26% (5) 4 – Easy 47% (9) 3 – Neutral 5% (1) 2 – Difficult 21% (4) 1 – Very Difficult 0% How would you improve this experience? While the majority of testers had an easy or very easy time creating a new account, here are some suggestions from testers on how to improve the experience: 2 – Make password rules more prominent 2 – Fewer questions 2 – Add arrows to move to the next field 1 – Show password as typing 1 – Alert when account is already created 1 – Wanted to add in zip code first and have it auto-populate 1 – Bring you to the error if you miss a field CTA Savant thought it was strange that the state field only had 10 states listed and they were not in alpha order. Daily CTA Rider and CTA Savant had trouble with the password because when the password/confirm password field is highlighted the characters are a different size so it is difficult to tell if the passwords are the same length. 12
  13. 13. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 How often do you purchase Metra tickets? Every day 0% 2 - 3 times a week 4% (1) Once a week 0% Once or twice a month 31% (8) Once every six months 19% (5) Once every year 27% (7) I have never purchased a Metra ticket before 19% (5) Anything else to add? Most testers did not have much to add, except that CTA Savant and Metra rider88 both used to be heavy Metra riders, but now live in the city and do not use it anymore. In addition, Frequent CTA / Occasional Metra buys 10 rides at one time and that usually lasts them a month. Please purchase a Metra ticket using the test device. Tell us about your experience as you go through it. Here are the top things that testers liked about the process of purchasing a Metra ticket. 7 – Split payment 3 – Using transit value as payment 2 – Billing info saved 2 – Stops are in order of location on the line. More information about testers’ experiences: SR would like to see a when selecting a ticket that indicates why Metra Monthly isn’t available during the part of the month when it’s not sold. CTA/Metra Commuter/Evil Genius likes that you can add items to the card once you are in it, but does not like the fact that the app cleared the cart if you go back and change the ticket type. Riding the 20 every day tried to buy the card using the test device, he forgot to confirm password and an error message popped up but did not indicate what caused the error. Other than that, Riding the 20 every day thought it was “easy, organized, logical” Occasional CTA Rider was not sure when they were supposed to activate the ticket. Occasional CTA Rider wants to say that ticket should be activated when on the train. How easy was it for you to purchase a Metra ticket? 5 – Very Easy 69% (18) 4 – Easy 23% (6) 3 – Neutral 8% (2) 2 – Difficult 0% 13
  14. 14. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 1 – Very Difficult 0% How would you improve this experience? Testers thought that purchasing a Metra ticket was pretty easy to do. 3 testers mentioned that it might be difficult if you were not sure where to go. Here are some other suggestions from testers, although most testers did not have anything to add: SR says, it “would be nice if screen could scroll if the keyboard is in the way of entering a password.” Route 66 Book Stalker wants to be able to search by station. CTA rider (KB) would like a round-trip option. CTA/Metra Commuter/Evil Genius wants the cart to be automatically cleared if you change your ticket type. Now look for the Metra ticket you just purchased. Look through this page and the tickets. What do you think about this page? 12 testers mentioned specifically that they liked this Metra ticket page. Here are some suggestions from other testers: CTA rider (AC) was confused about the time on the top of the ticket because it showed the current time, not the valid until time. In addition, CTA rider (AC) thought the design was a little much and was concerned about the battery life. Brown line rider / occasional UP-N Metra rider was unclear how to activate and when to activate the tickets you bought. “Can there be a hint screen or help screen?” He wouldn’t be confident using these tickets. CTA Savant didn’t know where her second ticket was at first. When opening up the QR code, she commented that the Metra logo is over the QR code and was not sure if that affected the ticket. Wireframe Section On day 2 of testing, the Ventra team wanted to ask a couple of questions about a wireframe to see if testers understood when the next train was coming. Therefore, we asked 12 testers to look at a screenshot to tell us if the train was coming in 17 minutes or at 5:46 PM. Please take a look at this screen, and tell us when the next train is. “At 5:46 PM” 50% (6) “In 17 min” 42% (5) Other: “Sort of Both” 8% (1) 14
  15. 15. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 Do you have any other comments or insight about this page? Here are a couple of comments from testers who said the next train is coming “at 5:46 PM:” Riding the 20 every day says, “It's off that it says both the time and the minutes on the right. Instead of the minutes on the right, maybe noting how early or late they are?” Cta my way thinks it should say “Delayed” if it coming later than the regularly scheduled time. Here are comments from testers who said the next train is coming “in 17 minutes:” CTA rider (AC) expects on the left to be actual arrival time. Time on right is saying that the next train leaves in 17 minutes. CTA Rider (AC) think it would help to say "Delayed." MC assumed the time (5:46) would update in real time. Gr8fl4CTA and Occasional CTA Rider both think the information is redundant and the same. Do you like the Ventra app? Yes 92% (24) No 8% (2) Why or why not? Here are the top reasons why testers liked the Ventra app (number indicates number of times the same reason was given): 7 – Easy to use 7 – Metra functionality 5 – Convenient 3 – Good visual designs Here are some additional responses from testers: Brown line rider / occasional UP-N Metra rider said that this app is going to replace the Ventra website for him. Metra rider88 says, “It was really convenient!” Normally Metra rider88 has to go down to the vending machine at the train station to add value via credit card and this can be very inconvenient due to the lines that form. Frequent CTA / Occasional Metra says, “I will ride Metra more often because of the app. I wouldn’t have usually considered it as an option.” 15
  16. 16. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 Blind Transit rider says, “It has the promise to provide me with access on the go to the information my Ventra pass has. [This app] can be more accessible and useful than any other available option as a blind transit rider. Do you believe the Ventra app fits into your day-to-day routine? Yes 69% (18) No 31% (8) Why or why not? Here are some responses from testers who said, “Yes” the Ventra app fits into their day-to-day routine: ChgoKC says, It'd be similar to the parking apps, where you'd keep it around and put on the home screen of your phone. CTA/Metra Commuter/Evil Genius has wanted this app for a long time. Mischievous Metra Maverick rides the Metra daily and says this “eliminates the need to look for a soggy ticket that is ruined halfway through the month.” Geek Sheek says, this will probably be “one of his top five apps.” Here are some responses from testers who said, “No” the Ventra app does not fit into their day-to-day routine: Frequent CTA Rider JH and Occasional CTA Rider are both looking for use of NFC technology so that they can use their phone at the turnstiles to get on the trains or busses. Frequent CTA / Occasional Metra says, “Not day to day. But that is because I don't ride Metra every day. If I did ride Metra every day, I would open it every day. I like the autoload option and like I don't have to think about it. But I do like having the app to check balance.” SR likes that the ticket is protected from loss, but worries about having phone charged or problems with wireless carrier services. CTA rider (KB) says “making it easier to get to [the arrivals page] more directly on open would potentially replace use of other apps, but would use it a lot more if loading fare manually or riding Metra daily.” How likely will you keep and use the Ventra app after this test? 5 – Very Likely 81% (21) 4 – Likely 8% (2) 3 – Not sure 8% (2) 2 – Unlikely 0% 16
  17. 17. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 1 – Very Unlikely 4% (1) What is your reason for keeping or not keeping the Ventra app? Only three testers were not sure or thought it was very unlikely that they would keep the Ventra app on their phones. Here is what they said: CTA Savant says that the app is not doing enough for her to keep it. She would use it if it uses NFC or if the Trip Tools worked better and did more than her other apps. Gr8fl4CTA is not sure because the trip planning does not work well enough. TA is not sure if she would keep it because she knows her daily value and would not need to use trip tools often. “It is handy for adding a value and getting a pass.” 9 of the 23 testers who said that it was likely or very likely that they would keep the Ventra app mentioned how “easy” or convenient” it was to use the Ventra app. Here are some other responses from tester: CTA Rider says they are using Ventra every day and this does not make them jump between apps. Frequent CTA Rider believes the Ventra app will be part of her daily activities and is happy that she will not need to go to the Ventra station to add value. CTA/Metra Commuter/Evil Genius is very likely to continue to use the Ventra app because it allows you to use Metra without a wallet and only ever carried cash before for Metra tickets. How would you improve the Ventra app? How should it be improved to make you more likely to use it? Testers shared a lot of improvements they would make throughout the rest of the test. 7 out of 26 testers wanted improvements to the Trip Tools, with 4 of these testers wanting a map incorporated. 6 out of 26 testers were interested in NFC mechanics so that they could use the app to pay for their bus/train fare without carrying around their Ventra card. 2 testers mentioned the biggest improvement would be fixing the back button. Was this your first CUTGroup test? Yes 73% (19) No 27% (7) Do you like CUTGroup? Yes 100% (26) No 0% 17
  18. 18. CUTGroup 17 – Ventra App Remote Test then In-person test on July 15 & 16, 2015 Did you like this CUTGroup test? Yes 100% (26) No 0% Anything else to add? CTA Rider says, “I love being able to help contribute to the development of this product.” Frequent CTA Rider says “Keep tests coming!” CTA/Metra Commuter/Evil Genius shared that people were very jealous that she had the app. Route 66 Book Stalker liked testing something that is important and impacts a lot of people. 18
  19. 19. Tester Profile (given by tester): Blind Transit rider Device (listed in initial screening email) iphone 5 Remote Test When using the app for the first time which of these actions did you do? Logged in to an existing account Have you added CTA transit value or passes to your account using the Ventra app? No Have you purchased Metra tickets? No If you have made a purchase, please tell us about that experience. I couldn't because my active card was not accessible to me. What are some features that you have used in the app? Check all that apply. Used Trip Tools Tell us about your overall experience using the Ventra app. I am blind and use voice over for this app. I spent a bit of time trying to make my active reduced faire card for people with disabilities. I do not find the app or the website compatible. All the customer service agents are able to see my reduced fare permit/ventra card, but I cannot see my active card. I can only see my closed paratransit Ventra card. It is a closed account because I never activated it. I am trying to correct that issue, but in the meantime I have been using the card that is not visible to me in either the app, or the website. I was able to use the trip tools and found that to be extremely helpful and cover all the transit options I use. How appealing do you find the visual design of the app? 3 - Neutral Tell us about what you like or do not like about the visual design of the app. How would you improve it? There are some buttons or links that do not have text labels, or alt text labels. This is important for accessibility. How easy has it been to use the Ventra app? 2 - Difficult How would you improve this app to make it easier to use? Make it fully accessible for blind voice over users. This means making sure that text is part of all the activating links, buttons, etc. 19
  20. 20. 7/15/2015 9:04:12 AM In-person Component Proctored by: Tony Tom How often did you use the Ventra app while you have had it? A few times since I have had it How easy was it to make a CTA ticket or pass purchase using the Ventra app? Not applicable - I did not make a CTA ticket or pass purchase How easy was it to purchase a Metra ticket using the Ventra app? Not applicable - I did not purchase a Metra ticket If you purchased a Metra ticket, did you use it on the train? Not applicable How was the experience using Metra tickets on the train? Go to "My Ventra Cards" and review the page. We want to find out if this page works for you and how to improve it. Registration T/C's created a big issue when creating an account. Not straight forward for limited sight user. How would you explain what the following features mean to you. Think about how these values are different from one another: "Add Transit Value" "Add Pass" "Add Autoload" What are some websites or apps that you currently use to check on transit times? Transit Stop (bad), nothing good that she's found. 20
  21. 21. Visit the "Trip Tools" page on your device and search for transit information near you. Tell us about your experience. Worked better than any other tool that she's found. How did you search? Searched by bus/train route Now search for another transit route near your home. How was this experience? N/A How did you search? Searched by bus/train route How easy was it for you to find the correct stops you were looking for? 3 - Neutral How would you improve this experience? The function itself is good. Getting there is difficult within the current version of the app. Likes that the information is provided in a list view and is easily read by voice over function. How does this feature compare to other trip websites and apps that you have used in the past? Much better than other options she has. Websites work but are not convenient. How would you register a new Ventra card if you wanted to add a card? Where would you find this? N/A Did you create a new account during the remote test? Yes Please create a new account on the test device. Tell us about your experience as you go through it. How easy was it for you to create a new account? 2 - Difficult How would you improve this experience? Date of birth field issue. Form is too long. Check box doesn't work and last box doesn't have a label of what it wanted. Lack of consistency in how all of the fields were presented. How often do you purchase Metra tickets? Once every six months Anything else to add? 21
  22. 22. Please purchase a Metra ticket using the test device. Tell us about your experience as you go through it. N/A How easy was it for you to purchase a Metra ticket? 3 - Neutral How would you improve this experience? Date of birth field issue. Form is too long. Check box doesn't work and last box doesn't have a label of what it wanted. Lack of consistency in how all of the fields were presented. Now look for the Metra ticket you just purchased. Look through this page and the tickets. What do you think about this page? Please take a look at this screen, and tell us when the next train is. Do you have any other comments or insight about this page? OVERALL REVIEW Do you like the Ventra app? Yes Why or why not? It has the promise to provide me with access on the go to the information my Ventra pass has. Can be more accessible and useful than any other available option as a blind transit rider. Do you believe the Ventra app fits into your day-to- day routine? Yes Why or why not? How likely will you keep and use the Ventra app after this test? 5 - Very Likely What is your reason for keeping or not keeping the Ventra app? Easier than other option. It allows me independence! How would you improve the Ventra app? How should it be imporved to make you more likely to use it? Everything is labeled with words which is rare. Don't add pictures! 22
  23. 23. Was this your first CUTGroup test? No Do you like CUTGroup? Yes Did you like this CUTGroup test? Yes Anything else to add? 7/15/2015 6:01:07 PM 23
  24. 24. Tester Profile (given by tester): Brown line rider / occasional UP-N Metra rider Device (listed in initial screening email) HTC One M8 Remote Test When using the app for the first time which of these actions did you do? Logged in to an existing account Have you added CTA transit value or passes to your account using the Ventra app? No Have you purchased Metra tickets? Yes If you have made a purchase, please tell us about that experience. There was a little trouble with the Metra ticket purchase: my original Ventra password was too short. I attempted to change my password in the app, but was not successful. (Unbeknownst to me at the time, there are further restrictions on passwords beyond their minimum eight-character length.) I then logged into the website and changed my password there, where the other password requirements (a numeral, a special character, etc) were made explicit. Once that was done, I was able to buy a Metra ticket through the app. My next use of the app will be applying my federal mass transit employee stipend to my Ventra account later this month. What are some features that you have used in the app? Check all that apply. Review account history Buy Metra Tickets Used Trip Tools Tell us about your overall experience using the Ventra app. Overall pretty useful. Changing the password (and setting forth the password requirements) could be a little more clear-cut. How appealing do you find the visual design of the app? 3 - Neutral Tell us about what you like or do not like about the visual design of the app. How would you improve it? It's certainly serviceable, but it does not yet have the polish of peer apps, such as Uber or Spotify. 24
  25. 25. How easy has it been to use the Ventra app? 4 - Easy How would you improve this app to make it easier to use? I feel navigation isn't always consistent. Sometimes I can use my phone's back button, other times I have to use an in-app menu button. 7/12/2015 10:32:23 PM In-person Component Proctored by: Denise Linn How often did you use the Ventra app while you have had it? Once a day How easy was it to make a CTA ticket or pass purchase using the Ventra app? Not applicable - I did not make a CTA ticket or pass purchase How easy was it to purchase a Metra ticket using the Ventra app? 4 - Easy If you purchased a Metra ticket, did you use it on the train? No How was the experience using Metra tickets on the train? Go to "My Ventra Cards" and review the page. We want to find out if this page works for you and how to improve it. Gets a federal transit benefit - normally adds that on and buys a pass with his own money. Hasn't done that yet, but he does he'll have to figure out how to do that. He knows how to do this on the website. For the app, it seems he can do the same process. Familiar. It has all the information he would want to see at a glance. How would you explain what the following features mean to you. Think about how these values are different from one another: "Add Transit Value" cash in the account 25
  26. 26. "Add Pass" fare "Add Autoload" automatically charging a card to predetermined amount What are some websites or apps that you currently use to check on transit times? CTA train tracker (notices they have one here and it seems better - more moden, auto uploads) Visit the "Trip Tools" page on your device and search for transit information near you. Tell us about your experience. Did this just before getting here since he doesn't work in this part of the Loop. He likes you can specify bus vs. rail. Intuitive. How did you search? Used "Locate Me" feature Now search for another transit route near your home. How was this experience? He had done that before. Seems clear cut how to do it. How did you search? Searched by bus/train route How easy was it for you to find the correct stops you were looking for? 5 - Very Easy How would you improve this experience? Give me all the information I would normally want/expect at a glance. The bookmarking function isn't as intuitive as it could be. Icons could make more sense. How does this feature compare to other trip websites and apps that you have used in the past? Similar to what you'd get on the CTA website only this app "looks more this decade." The back button doesn't get your back the main screen. Here you do the right swipe to get to navigation. This isn't all the clear at first. It has to be learned. How would you register a new Ventra card if you wanted to add a card? Where would you find this? My Ventra cards, then touch the plus sign, then register Ventra card. Did you create a new account during the remote test? No Please create a new account on the test device. Tell us about your experience as you go through it. 26
  27. 27. How easy was it for you to create a new account? Not applicable How would you improve this experience? His original passwork was only 4 characters long and he had trouble changing the password. Also, the password requirements weren't on the app. He had to change his password correctly online before he could use the app. How often do you purchase Metra tickets? Once every six months Anything else to add? specifically once every 2 months Please purchase a Metra ticket using the test device. Tell us about your experience as you go through it. Note: didn't have password for the dummy account so didn't finish purchase. How easy was it for you to purchase a Metra ticket? 4 - Easy How would you improve this experience? His original passwork was only 4 characters long and he had trouble changing the password. Also, the password requirements weren't on the app. He had to change his password correctly online before he could use the app. Now look for the Metra ticket you just purchased. Look through this page and the tickets. What do you think about this page? Unclear how to activate and when to activate the tickets you bought. Can there be a hint screen or a help screen? He wouldn't be confident using these. This is a new feature to users of CTA, so more help would be good. Please take a look at this screen, and tell us when the next train is. At 5:46 PM Do you have any other comments or insight about this page? What I would expect. It's intuative. OVERALL REVIEW Do you like the Ventra app? Yes Why or why not? Looking forward to paying with the cellphone, but overall very useful app. For him, it will replace going to the Ventra website. Do you believe the Ventra app fits into your day-to- Why or why not? 27
  28. 28. day routine? No Just because I don't need to go to the website day-to-day. How likely will you keep and use the Ventra app after this test? 5 - Very Likely What is your reason for keeping or not keeping the Ventra app? Likes the trip tools a lot. How would you improve the Ventra app? How should it be imporved to make you more likely to use it? The password issue was a one-time difficulty. Was this your first CUTGroup test? Yes Do you like CUTGroup? Yes Did you like this CUTGroup test? Yes Anything else to add? 7/16/2015 7:33:00 PM 28
  29. 29. Tester Profile (given by tester): ChgoKC Device (listed in initial screening email) Android Version 4.4.2 Remote Test When using the app for the first time which of these actions did you do? Logged in to an existing account Have you added CTA transit value or passes to your account using the Ventra app? No Have you purchased Metra tickets? No If you have made a purchase, please tell us about that experience. I tried to add value to the account, but it asked for my Ventra password, which I entered (it's 4 digits), and it said the password had to be 8 digits. I tried my password for my bank account, but that was "incorrect". So not sure what password it's looking for. What are some features that you have used in the app? Check all that apply. Review account history Add Transit Value Used Trip Tools Looked up Ventra Value Tell us about your overall experience using the Ventra app. not always easy to navigate. you couldn't hit a "back" key, or the back on the phone. Had to go completely out to the app, and then re-log in. Now the little bars at the top left are visible to get to the main menu, but it wasn't always visible. How appealing do you find the visual design of the app? 4 - Appealing Tell us about what you like or do not like about the visual design of the app. How would you improve it? I like the design of the app. How easy has it been to use the Ventra app? 3 - Neutral (not easy, not difficult) How would you improve this app to make it easier to use? works pretty well! adding $$ was my only issue. 29
  30. 30. 7/13/2015 11:45:44 AM In-person Component Proctored by: Tony Coppoletta How often did you use the Ventra app while you have had it? A few times since I have had it How easy was it to make a CTA ticket or pass purchase using the Ventra app? 4 - Easy How easy was it to purchase a Metra ticket using the Ventra app? Not applicable - I did not purchase a Metra ticket If you purchased a Metra ticket, did you use it on the train? Not applicable How was the experience using Metra tickets on the train? Go to "My Ventra Cards" and review the page. We want to find out if this page works for you and how to improve it. Noted that back button exits app from My Ventra Cards rather than going back to menu. Otherwise had no additional comments. How would you explain what the following features mean to you. Think about how these values are different from one another: "Add Transit Value" To add more value "Add Pass" Doesn't know "Add Autoload" Doesn't know What are some websites or apps that you currently use to check on transit times? n/a 30
  31. 31. Visit the "Trip Tools" page on your device and search for transit information near you. Tell us about your experience. No comments (worked as expected) How did you search? Used "Locate Me" feature Now search for another transit route near your home. How was this experience? Would like to be able to search for stops by name. How did you search? Searched by bus/train route How easy was it for you to find the correct stops you were looking for? 4 - Easy How would you improve this experience? Would like to be able to search for stops by name, or type in route by number or name. How does this feature compare to other trip websites and apps that you have used in the past? n/a How would you register a new Ventra card if you wanted to add a card? Where would you find this? Don't know, tried My Account--would probably just go online Did you create a new account during the remote test? No Please create a new account on the test device. Tell us about your experience as you go through it. How easy was it for you to create a new account? Not applicable How would you improve this experience? How often do you purchase Metra tickets? Once every six months Anything else to add? Please purchase a Metra ticket using the test device. Tell us about your experience as you go through it. 31
  32. 32. Understood transit value and chose to use that split with card. Very easy to use. How easy was it for you to purchase a Metra ticket? 5 - Very Easy How would you improve this experience? Now look for the Metra ticket you just purchased. Look through this page and the tickets. What do you think about this page? Very self-explanatory, says what it is, when it expires. Please take a look at this screen, and tell us when the next train is. Sort of both Do you have any other comments or insight about this page? Should be the clock time, could be clearer. OVERALL REVIEW Do you like the Ventra app? Yes Why or why not? I think it's cool. I love the whole idea of it. Everyone has a smartphone, and you don't have paper to lose and it's easy. Do you believe the Ventra app fits into your day-to- day routine? Yes Why or why not? It'd be similar to the parking apps, where you'd keep it around and put on the home screen of your phone. How likely will you keep and use the Ventra app after this test? 5 - Very Likely What is your reason for keeping or not keeping the Ventra app? On the weeks that she rides a lot she'd check every day. How would you improve the Ventra app? How should it be imporved to make you more likely to use it? Nothing additional. Thinks it looks nice. Was this your first CUTGroup test? 32
  33. 33. Yes Do you like CUTGroup? Yes Did you like this CUTGroup test? Yes Anything else to add? 7/16/2015 7:11:46 PM 33
  34. 34. Tester Profile (given by tester): Cta my way Device (listed in initial screening email) I phone Remote Test When using the app for the first time which of these actions did you do? Registered a Ventra Card Have you added CTA transit value or passes to your account using the Ventra app? Yes Have you purchased Metra tickets? No If you have made a purchase, please tell us about that experience. Did not purchase metra What are some features that you have used in the app? Check all that apply. Review account history Looked up Ventra Value Had to change password check balance Tell us about your overall experience using the Ventra app. It was not as user friendly as I had hoped. Can I use my phone as the ventra card instead of the actual card, like at Starbucks? How appealing do you find the visual design of the app? 3 - Neutral Tell us about what you like or do not like about the visual design of the app. How would you improve it? It was okay- nice color simple How easy has it been to use the Ventra app? 4 - Easy How would you improve this app to make it easier to use? It asked for the nickname of card and I had no idea what thAt was 7/13/2015 3:13:09 PM 34
  35. 35. In-person Component Proctored by: Sonja Marziano How often did you use the Ventra app while you have had it? Once a day How easy was it to make a CTA ticket or pass purchase using the Ventra app? 5 - Very easy How easy was it to purchase a Metra ticket using the Ventra app? 1 - Very difficult If you purchased a Metra ticket, did you use it on the train? Not applicable How was the experience using Metra tickets on the train? Go to "My Ventra Cards" and review the page. We want to find out if this page works for you and how to improve it. Had to reset the password and had to know the name of the card, but that was tough to remember Wants to be able to scan phone to get on transit (NFC) How would you explain what the following features mean to you. Think about how these values are different from one another: "Add Transit Value" Typically use this feature "Add Pass" Don't buy passes normally so these features are not as clear "Add Autoload" First autoload (likes the warning) is for the card. Second autoload might be for pass? seemed confused at first but then made sense after clicking What are some websites or apps that you currently use to check on transit times? CTA/RTA search for "Find my bus route" Visit the "Trip Tools" page on your device and search for transit information near you. Tell us about your experience. 35
  36. 36. Trip Tools "CTA Busses" was set automatically alphabetical order but Sort A-Z was not highlighted. Would make more sense to use your own location. Searched by route and stop, never saw the Locate me button or the heart button. Would be better to say "Nearby locations" or "Nearby stops" How did you search? Searched by bus/train route Now search for another transit route near your home. How was this experience? Searches by route, but wants to go back to the back screen "Find new stop" makes you start over "Show all routes at this stop" Was curious about the alert button, would click that normally since it looks like a message for me. How did you search? Searched by bus/train route How easy was it for you to find the correct stops you were looking for? 3 - Neutral How would you improve this experience? Click on the icons to know what they meant, wasn't intuitive. Stops: were not clear that it was alphabetical How does this feature compare to other trip websites and apps that you have used in the past? Easier than searching CTA/RTA site, but likes having the map of the stops. Would like to see this feature in Ventra app. How would you register a new Ventra card if you wanted to add a card? Where would you find this? Went straight to "Ventra cards" searched around the screen, almost pressed "Add Pass" and then clicked the + symbol. Did you create a new account during the remote test? No Please create a new account on the test device. Tell us about your experience as you go through it. How easy was it for you to create a new account? Not applicable How would you improve this experience? How often do you purchase Metra tickets? I have never purchased a Metra ticket before Anything else to add? 36
  37. 37. Please purchase a Metra ticket using the test device. Tell us about your experience as you go through it. Went through it very quickly, very smooth process. Used transit value, liked that feature How easy was it for you to purchase a Metra ticket? 5 - Very Easy How would you improve this experience? Now look for the Metra ticket you just purchased. Look through this page and the tickets. What do you think about this page? This seems very clear, no questions. Doesn't ride the Metra Please take a look at this screen, and tell us when the next train is. At 5:46 PM Do you have any other comments or insight about this page? "Delayed" if it's coming later. Shopping cart wasn't clear: thought it might show you your value. OVERALL REVIEW Do you like the Ventra app? Yes Why or why not? User-friendly Do you believe the Ventra app fits into your day-to- day routine? Yes Why or why not? Take the bus more than the train, no machines to put the money in. Usually uses to vending machine and has to go out of the way How likely will you keep and use the Ventra app after this test? 5 - Very Likely What is your reason for keeping or not keeping the Ventra app? Ease of adding value How would you improve the Ventra app? How should it be imporved to make you more likely to use it? Trouble "Find a Retail Location" on test device but works on her own iPhone -- says she doesn't really need that. 37
  38. 38. Was this your first CUTGroup test? Yes Do you like CUTGroup? Yes Did you like this CUTGroup test? Yes Anything else to add? Easy location from work! I got interrupted a lot, she mentioned that. 7/16/2015 6:18:34 PM 38
  39. 39. Tester Profile (given by tester): CTA Rider Device (listed in initial screening email) LG G2 Remote Test When using the app for the first time which of these actions did you do? Logged in to an existing account Have you added CTA transit value or passes to your account using the Ventra app? Yes Have you purchased Metra tickets? No If you have made a purchase, please tell us about that experience. There were no problems. I added value to my Ventra using a credit card, just as I do when using the website. It was very simple. What are some features that you have used in the app? Check all that apply. Review account history Add Transit Value Used Trip Tools Tell us about your overall experience using the Ventra app. It's a good, reliable app. One thing I noticed is that when you add value it appears instantly, versus the online site, where it usually doesn't update until you log off and log back in again, so that's a plus. I also like that Bus Tracker is built into the app, which saves time from having to search it on the internet. How appealing do you find the visual design of the app? 5 - Very appealing Tell us about what you like or do not like about the visual design of the app. How would you improve it? It's recognizable with the light blue of the website & it has a clean, streamlined interface which makes it easy to use. How easy has it been to use the Ventra app? 5 - Very Easy How would you improve this app to make it easier to use? It's pretty simple to use @ the moment so there's nothing really that I can think of, that could make it any simpler. It's pretty straightforward. 39
  40. 40. 7/13/2015 8:32:33 AM In-person Component Proctored by: Tony Tom How often did you use the Ventra app while you have had it? Once a day How easy was it to make a CTA ticket or pass purchase using the Ventra app? 5 - Very easy How easy was it to purchase a Metra ticket using the Ventra app? Not applicable - I did not purchase a Metra ticket If you purchased a Metra ticket, did you use it on the train? Not applicable How was the experience using Metra tickets on the train? Go to "My Ventra Cards" and review the page. We want to find out if this page works for you and how to improve it. Very streamlined. Related to the website, found it easy. Unlike website though you can remove a card, this feature is not yet implemented in the app so she had to go back to the website and use it. Transit History bogs down at times which can be frustrating. Overall, it's been better than the website because it's instantaneous. How would you explain what the following features mean to you. Think about how these values are different from one another: "Add Transit Value" Seemed self explantory "Add Pass" Seemed self explanatory "Add Autoload" Seemed self explanatory What are some websites or apps that you currently use to check on transit times? 40
  41. 41. CTA Trip Planner, Google Maps Visit the "Trip Tools" page on your device and search for transit information near you. Tell us about your experience. Term is very misleading. Is actually a bus and train tracker. Trip Tools make me think of real time trip planner, so all alerts, types of alerts didn't seem to go through. Step by step directions were expected based naming convention. How did you search? Searched by bus/train route Now search for another transit route near your home. How was this experience? Use bus/train tracker. Search stop. Search direction. How did you search? Searched by bus/train route How easy was it for you to find the correct stops you were looking for? 5 - Very Easy How would you improve this experience? Real-time alerts being more noticeable. More information would be needed on what a 'delay' means to make the alert functions more useable. How does this feature compare to other trip websites and apps that you have used in the past? Trip Planning is the big missing feature. 'Delay' statement could use more information to make it helpful. How would you register a new Ventra card if you wanted to add a card? Where would you find this? For an existing user, need to add a description to the Plus button in the top right. Had difficulty finding the button to add a card. Did you create a new account during the remote test? Yes Please create a new account on the test device. Tell us about your experience as you go through it. She already had a web account so did not create an account during her testing. We set up account on the device. Issue with birthdate field. How easy was it for you to create a new account? 4 - Easy How would you improve this experience? Make the year in birthdate two digits. 41
  42. 42. How often do you purchase Metra tickets? Once ever year Anything else to add? Please purchase a Metra ticket using the test device. Tell us about your experience as you go through it. Easy. How easy was it for you to purchase a Metra ticket? 5 - Very Easy How would you improve this experience? Make the year in birthdate two digits. Now look for the Metra ticket you just purchased. Look through this page and the tickets. What do you think about this page? Please take a look at this screen, and tell us when the next train is. Do you have any other comments or insight about this page? OVERALL REVIEW Do you like the Ventra app? Yes Why or why not? Flexibity and ease of use. Was very dependable and accurate. Almost instantaneous. Do you believe the Ventra app fits into your day-to- day routine? Yes Why or why not? Because I'm a commuter. How likely will you keep and use the Ventra app after this test? 5 - Very Likely What is your reason for keeping or not keeping the Ventra app? Already using Ventra every day. Makes me not have to jump between apps. How would you improve the Ventra app? How should it be imporved to make you more likely to use it? 42
  43. 43. Make Trip Tools more than a bus tracker. Real-time updates need to provide more info around 'delays'. Provide access to 24-hour support for Ventra. Put the card on the phone. Was this your first CUTGroup test? Yes Do you like CUTGroup? Yes Did you like this CUTGroup test? Yes Anything else to add? Loved being able to help contribute the development of this product. 7/15/2015 4:17:03 PM 43
  44. 44. Tester Profile (given by tester): CTA rider AC Device (listed in initial screening email) Nexus 4 Remote Test When using the app for the first time which of these actions did you do? Logged in to an existing account Have you added CTA transit value or passes to your account using the Ventra app? Yes Have you purchased Metra tickets? No If you have made a purchase, please tell us about that experience. It went seamlessly. The system remembered my credit card info from my earlier purchases and could not really have to make any effort to add value. What are some features that you have used in the app? Check all that apply. Add Transit Value Used Trip Tools Looked up Ventra Value Tell us about your overall experience using the Ventra app. It seems a bit slow to load but otherwise it works fine. How appealing do you find the visual design of the app? 4 - Appealing Tell us about what you like or do not like about the visual design of the app. How would you improve it? I would change the color of cursor from white to black or something dark. It is really annoying that I can't see where the cursor is. How easy has it been to use the Ventra app? 4 - Easy How would you improve this app to make it easier to use? Some icons are not very telling. I had to click on them to figure out what they are for. 7/13/2015 9:16:21 PM 44
  45. 45. In-person Component Proctored by: Sonja Marziano How often did you use the Ventra app while you have had it? A few times since I have had it How easy was it to make a CTA ticket or pass purchase using the Ventra app? 5 - Very easy How easy was it to purchase a Metra ticket using the Ventra app? Not applicable - I did not purchase a Metra ticket If you purchased a Metra ticket, did you use it on the train? Not applicable How was the experience using Metra tickets on the train? Go to "My Ventra Cards" and review the page. We want to find out if this page works for you and how to improve it. Last four digit of the card number doesn't seem necessary (only has one card though) PhD student so adds value during the summer Transit History takes multiple tries before it shows up. Attempted 5 times before it showed Wanted to know the timeframe of the Ventra cards Also wanted to add a second card. Was confused about "Register a Ventra Card" because she had two cards under two different user names. It seems to allow her to register a second card that is under a different user name. How would you explain what the following features mean to you. Think about how these values are different from one another: "Add Transit Value" just value "Add Pass" Monthly pass "Add Autoload" a pass and autocharge; Does not think that autoload needs to be a separate option but instead part of the pass option What are some websites or apps that you currently use to check on transit times? Chicago Transit Tracker Lite 45
  46. 46. Visit the "Trip Tools" page on your device and search for transit information near you. Tell us about your experience. It's easier to use on the Ventra app. Less clicks to find the correct stop. Uses the bus stop number when she is nearby the stop Locate me feature, first but then went back has background knowledge about which line to ride Has pink line in her favorites How did you search? Used "Locate Me" feature Now search for another transit route near your home. How was this experience? Lots of scrolling through trying to find the stop, and then went to A-Z sort after a little while Was confused about the order of how stops were listed Prefers to have stops listed alphabetically, but thinks that order by station might be better for users that are from out of town How did you search? Searched by bus/train route How easy was it for you to find the correct stops you were looking for? 4 - Easy How would you improve this experience? Cursor is white on the white background when typing is required Locate me icon did not make sense until she clicked it, thought text might be helpful. How does this feature compare to other trip websites and apps that you have used in the past? "I trusted it more because it was affiliated with Ventra" More "design-y" The "Locate me" function was not part of the old app The enter stop number also did not exist How would you register a new Ventra card if you wanted to add a card? Where would you find this? Found it during the remote test but had confusion about what "Register a Ventra Card" meant specifically Did you create a new account during the remote test? No Please create a new account on the test device. Tell us about your experience as you go through it. How easy was it for you to create a new account? Not applicable How would you improve this experience? How often do you purchase Anything else to add? 46
  47. 47. Metra tickets? Once ever year Please purchase a Metra ticket using the test device. Tell us about your experience as you go through it. Not familiar with Metra at all, so didn't know which stops to pick Ordering of stops are interesting but could not figure out the logic. Cursor should be black How easy was it for you to purchase a Metra ticket? 5 - Very Easy How would you improve this experience? Now look for the Metra ticket you just purchased. Look through this page and the tickets. What do you think about this page? Time on the top of the ticket is the current time, not the valid until time. Tester liked sharing the valid until time, but only shown in QR code. Design is a little much, concerned about battery life. Explained why it was moving and she seemed to feel better about the design element. Please take a look at this screen, and tell us when the next train is. In 17 min Do you have any other comments or insight about this page? Expects on the left to be actual arrival time. Time on right is next train leaves in 17 minutes. Would help to say "Delay" OVERALL REVIEW Do you like the Ventra app? Yes Why or why not? "Really easy to put money on your card." Used to put money on card online. Likes its the first thing you see if the amount left on the card. Do you believe the Ventra app fits into your day-to- day routine? Yes Why or why not? How likely will you keep and use the Ventra app after this test? 5 - Very Likely What is your reason for keeping or not keeping the Ventra app? easy to put money on card, and trip tools 47
  48. 48. How would you improve the Ventra app? How should it be imporved to make you more likely to use it? Showed me error to add a New Card: "Webpage not available". Keyboard is large and overlaps with the fields and "Save" and "OK" buttons. Seems to be an Android issue. Was this your first CUTGroup test? No Do you like CUTGroup? Yes Did you like this CUTGroup test? Yes Anything else to add? 7/16/2015 5:01:47 PM 48
  49. 49. Tester Profile (given by tester): CTA rider KB Device (listed in initial screening email) iPhone 6 Remote Test When using the app for the first time which of these actions did you do? Logged in to an existing account Have you added CTA transit value or passes to your account using the Ventra app? No Have you purchased Metra tickets? No If you have made a purchase, please tell us about that experience. N/A What are some features that you have used in the app? Check all that apply. Review account history Used Trip Tools Looked up Ventra Value Tell us about your overall experience using the Ventra app. Good, and much easier than the mobile app, which I used to use frequently. It is sometimes slow to load though. How appealing do you find the visual design of the app? 3 - Neutral Tell us about what you like or do not like about the visual design of the app. How would you improve it? Things are too close together in many places. How easy has it been to use the Ventra app? 4 - Easy How would you improve this app to make it easier to use? Many pages are missing a back button, making navigation difficult. 7/15/2015 10:46:43 AM 49
  50. 50. In-person Component Proctored by: Tony Coppoletta How often did you use the Ventra app while you have had it? A few times since I have had it How easy was it to make a CTA ticket or pass purchase using the Ventra app? Not applicable - I did not make a CTA ticket or pass purchase How easy was it to purchase a Metra ticket using the Ventra app? Not applicable - I did not purchase a Metra ticket If you purchased a Metra ticket, did you use it on the train? Not applicable How was the experience using Metra tickets on the train? Go to "My Ventra Cards" and review the page. We want to find out if this page works for you and how to improve it. Thinks there's a lot on the page. Noted difference between slider for transit value autoload but a button for adding autoload under pass. Likes that balance is clear and what's regular value and benefit value, as someone who has both in their account. Might like more to highlight what things do, though not sure exactly what that would be. Might like to be able to see both value and passes in one place rather than have separate flows to get to each and have it more explicit that there are no passes in queue when there aren't any. Transit history takes a while to load but not evident that when tapped it is loading, so tapped multiple times not realizing the tap had been accepted. Would like to be able to see further back in history. How would you explain what the following features mean to you. Think about how these values are different from one another: "Add Transit Value" Will add value "Add Pass" Will add a pass (7-day, 30-day) "Add Autoload" Will add autoload for value or pass (unclear) What are some websites or apps that you currently use to check on transit times? 50
  51. 51. TransitStop, CityMapper (but felt too much info, just needs quick stop info) Visit the "Trip Tools" page on your device and search for transit information near you. Tell us about your experience. Experience was good. Wants there to be a functioning back button from results screen and felt list of nearby stops could present what stop has what services more clear (direction per stop, for bus, for example). How did you search? Used "Locate Me" feature Now search for another transit route near your home. How was this experience? During stop selection, it'd be nice if back button had a larger hot area (doesn't feel responsive enough). Train info can be more intuitively sorted. Would be nice to be able to see, when selecting stops, which one is closest. Section headers on main screen seem restating what's implicit, perhaps unnecessary. How did you search? Searched by bus/train route How easy was it for you to find the correct stops you were looking for? 3 - Neutral How would you improve this experience? Make it easier to find that feature and get to it from more places within the Trip Tools section. How does this feature compare to other trip websites and apps that you have used in the past? It's good, has about the same functionality that many other apps have, and the alerts here are easier to find (shown prominently). Navigation between pages could be easier, though. Though the default area is My Ventra Cards, would make a better substitution for other transit apps used if the favorites/nearby were easy to find throughout the app/right up front. Would like link to view full schedule, or otherwise determine first bus/last bus. If no results shown, would be nice to pull to refresh. How would you register a new Ventra card if you wanted to add a card? Where would you find this? Used (+) button. Noted might be confusing to some that you can see balance in My Ventra Cards or separately look up value. Noted that "I'm logged in but it wants me to log in" on being redirected to mobile website flow. User understands need for auth in mobile web experience, but noted it would be nice if would at least autofill username. Did you create a new account during the remote test? No Please create a new account on the test device. Tell us about your experience as you go through it. 51
  52. 52. Started with a simple password but saw requirements after due to the keyboard. Noted it asks for at least three of following but has four bullets (one bullet is combined). Noted keyboard doesn't have "next field" skip button. How easy was it for you to create a new account? 4 - Easy How would you improve this experience? Would like arrows to skip between fields (iOS), make password req's visible above keyboard. How often do you purchase Metra tickets? Once every six months Anything else to add? Please purchase a Metra ticket using the test device. Tell us about your experience as you go through it. When selecting a ticket type, asking for info on reduced fares took me further back than selection list when done (one step further than expected). Found "Buy More" slightly unintuitive next to the increment/decrement buttons in cart on tix. How easy was it for you to purchase a Metra ticket? 5 - Very Easy How would you improve this experience? Would like arrows to skip between fields (iOS), make password req's visible above keyboard. Now look for the Metra ticket you just purchased. Look through this page and the tickets. What do you think about this page? It's fine. Could improve "use ticket" text. Please take a look at this screen, and tell us when the next train is. Do you have any other comments or insight about this page? OVERALL REVIEW Do you like the Ventra app? Yes Why or why not? Didn't like to have to log into mobile web, likes not having to log in or remember un/pw. Metra thing is cool and last time had to pay more on board not having time to buy ticket, would be more likely. Do you believe the Ventra app fits into your day-to- Why or why not? 52
  53. 53. day routine? No Having value automatically loaded, making it easier to get to arrivals more directly on open would make it potentially replace use of other apps, but would use it a lot more if loading fare manually or riding Metra daily. How likely will you keep and use the Ventra app after this test? 5 - Very Likely What is your reason for keeping or not keeping the Ventra app? Because it could be useful to check value. How would you improve the Ventra app? How should it be imporved to make you more likely to use it? Faster access to Trip Tools. Was this your first CUTGroup test? Yes Do you like CUTGroup? Yes Did you like this CUTGroup test? Yes Anything else to add? 7/15/2015 7:18:05 PM 53
  54. 54. Tester Profile (given by tester): CTA Savant Device (listed in initial screening email) IPhone Remote Test When using the app for the first time which of these actions did you do? Logged in to an existing account Have you added CTA transit value or passes to your account using the Ventra app? No Have you purchased Metra tickets? No If you have made a purchase, please tell us about that experience. N/A What are some features that you have used in the app? Check all that apply. Review account history Used Trip Tools Tell us about your overall experience using the Ventra app. The app was not difficult to use. How appealing do you find the visual design of the app? 3 - Neutral Tell us about what you like or do not like about the visual design of the app. How would you improve it? The colors are pleasant and seem to match the existing Ventra brand. The "My Ventra Cards" section felt somewhat cluttered with sections for different types of cards. I would add a single button to Add a card, then that can bring up a new page where you select CTA, Pace, Metra and the type of pass (autoload, one-time, time interval, etc). Many riders, including myself, likely only have one card and would therefore benefit from a more minimalist card management interface. How easy has it been to use the Ventra app? 4 - Easy How would you improve this app to make it easier to use? I would not use this app in my daily commute until it was capable of serving as a mobile card, allowing me to tap my phone against the Ventra card reader to board trains and busses. As a commuter that 54
  55. 55. has payroll deductions for their transit benefits, I never have a need to add more funds to my card and never manage more than one. The balance checking and transaction history functionality alone do not merit using the app. Some additional improvements I would suggest would be to reformat the Trip Tools so that Stops that you have already marked as favorite automatically appear on the main Trip Tools Page. This would save the user an extra click as once you have added your favorite stops, you probably won't use the search functionality as often. It would also be nice if the Trip Tools also provided the arrival time. It currently provides ETA, which is helpful enough but I find that both are beneficial to keep track of arriving trains/busses. 7/13/2015 8:01:01 PM In-person Component Proctored by: Sonja Marziano How often did you use the Ventra app while you have had it? A few times since I have had it How easy was it to make a CTA ticket or pass purchase using the Ventra app? Not applicable - I did not make a CTA ticket or pass purchase How easy was it to purchase a Metra ticket using the Ventra app? Not applicable - I did not purchase a Metra ticket If you purchased a Metra ticket, did you use it on the train? Not applicable How was the experience using Metra tickets on the train? Go to "My Ventra Cards" and review the page. We want to find out if this page works for you and how to improve it. When raining, she takes the bus and used the Trip Tools. Was able to connect to Ventra account and monitor her activity. Has transit benefits that get autoload. Wants to use her device to get on the bus (taps it) Lots of activity on my Ventra cards. One button to add a pass or card and then it would send to dialogue page that chooses between passes or value. Thinks that a lot of people only have 1 card, so 55
  56. 56. thinks that this page could be simpler. How would you explain what the following features mean to you. Think about how these values are different from one another: "Add Transit Value" "Add Pass" Had to click on this one to see the difference between Autoload. Learned quickly on them to figure it out. "Add Autoload" What are some websites or apps that you currently use to check on transit times? TransitStop app Visit the "Trip Tools" page on your device and search for transit information near you. Tell us about your experience. Did not have much success with the "Locate Me" took it a little while, and then it wasn't very accurate. This would be helpful on a map especially for users who do not know the area. Vague idea where she is so she would look for the stop. Had some questions about the stop names. Similarities such as Michigan & South Water or Michigan/South Water. Wants to clarify different stops and then provide them on the map to see if they actually located her on the map. Favorites icon info could be on the first transit page. How did you search? Used "Locate Me" feature Now search for another transit route near your home. How was this experience? Probably would not use this app, would prefer Google Maps. How did you search? Searched by bus/train route How easy was it for you to find the correct stops you were looking for? 3 - Neutral How would you improve this experience? Wants to see how frequently the busses/trains are running especially later in the evening How does this feature compare to other trip websites and apps that you have used in the past? Likes on Transit Stop app shows the number of minutes when it will arrive as well as the ETA time. Ex: 6 min & 6:11 PM Likes the service alerts on this app 56
  57. 57. How would you register a new Ventra card if you wanted to add a card? Where would you find this? Went directly "My Ventra Cards" "I expect the + sign will do it" Did you create a new account during the remote test? No Please create a new account on the test device. Tell us about your experience as you go through it. Password zoom in/zoom out is weird because it appears that you incorrectly input the password How easy was it for you to create a new account? 4 - Easy How would you improve this experience? State, wanted to add in zip code first and have it auto populate. Only had 10 states that is weird (was not alpha order) How often do you purchase Metra tickets? Once ever year Anything else to add? Never really use it, used to be a heavy rider when living in the suburbs Please purchase a Metra ticket using the test device. Tell us about your experience as you go through it. "That's nice, you can split payment" Added a new card, did "Same as My Account Information" and liked that experience Side note: proactive notification about scheduling How easy was it for you to purchase a Metra ticket? 4 - Easy How would you improve this experience? State, wanted to add in zip code first and have it auto populate. Only had 10 states that is weird (was not alpha order) Now look for the Metra ticket you just purchased. Look through this page and the tickets. What do you think about this page? Didn't know where her second ticket was at first. Had to go back and show the second ticket. Wish it was displayed together for multiple tickets Metra logo is over QR code, not sure if that affects anything Please take a look at this screen, and tell us when the next train is. Do you have any other comments or insight about this page? 57
  58. 58. OVERALL REVIEW Do you like the Ventra app? No Why or why not? Can't get on the bus with the app, not doing anything more than what I am already doing. Not doing Trip Tools well enough to use it just for that. Do you believe the Ventra app fits into your day-to- day routine? No Why or why not? How likely will you keep and use the Ventra app after this test? 1 - Very Unlikely What is your reason for keeping or not keeping the Ventra app? Not doing enough How would you improve the Ventra app? How should it be imporved to make you more likely to use it? Recommend improving trip tools. Wants to use as NFC card. Was this your first CUTGroup test? Yes Do you like CUTGroup? Yes Did you like this CUTGroup test? Yes Anything else to add? "I guess so, it's been solid" 7/15/2015 6:31:17 PM 58
  59. 59. Tester Profile (given by tester): CTA/Metra Commuter/Evil Genius Device (listed in initial screening email) HTC One M9 Remote Test When using the app for the first time which of these actions did you do? Logged in to an existing account Have you added CTA transit value or passes to your account using the Ventra app? No Have you purchased Metra tickets? Yes If you have made a purchase, please tell us about that experience. Purchasing in general is easy. Multiple times, I've purchased a ticket while walking to the train. I love the ability to pay with my Ventra auto-reloading balance. What are some features that you have used in the app? Check all that apply. Review account history Buy Metra Tickets Used Trip Tools Looked up Ventra Value Other Activate/Use metra tickets - single ride and weekend, two different lines Tell us about your overall experience using the Ventra app. I love it. The only thing I'd like to see if NFC CTA use like you can do with Google Wallet, but I'm sure there are technical limitations there. How appealing do you find the visual design of the app? 4 - Appealing Tell us about what you like or do not like about the visual design of the app. How would you improve it? It's simple, which I really like. I think it could fit in more with the Material Design Craze to make it more consistent with the rest of my OS, but the layout is great. The font is a little too rounded for my taste, but that's also an issue I have with Droid Sans. 59
  60. 60. How easy has it been to use the Ventra app? 5 - Very Easy How would you improve this app to make it easier to use? Nothing immediately comes to mind. 7/13/2015 2:42:58 PM In-person Component Proctored by: Lynnette Ciavarella How often did you use the Ventra app while you have had it? Once a day How easy was it to make a CTA ticket or pass purchase using the Ventra app? Not applicable - I did not make a CTA ticket or pass purchase How easy was it to purchase a Metra ticket using the Ventra app? 5 - Very easy If you purchased a Metra ticket, did you use it on the train? Yes How was the experience using Metra tickets on the train? Conductors weren't trained - Rock Island - July 11 at 12:19p they were sure what to do not sure about the qr scanners would be used Go to "My Ventra Cards" and review the page. We want to find out if this page works for you and how to improve it. HTC OneM9 update Ventra card balance is now cut off on the top Ventra card order - how to change? How would you explain what the following features mean to you. Think about how these values are different from one another: "Add Transit Value" add money to card or pass "Add Pass" add a daily, weekly or monthly pass "Add Autoload" to make the card auto-reload 60
  61. 61. What are some websites or apps that you currently use to check on transit times? Google maps - real-time CTA tracking Visit the "Trip Tools" page on your device and search for transit information near you. Tell us about your experience. use the GPS icon Like to location feature, but like to cite the distance the stop is from you right now. gray bar appears appears under the near by stops before showing the list of routes How did you search? Used "Locate Me" feature Now search for another transit route near your home. How was this experience? liked the pull-down to refresh discovered the favorite buttons - liked this feature but the in-app arrow doesn't work CTA Red line - station sort is in a strange order - not stop or ABC's How did you search? Searched by bus/train route How easy was it for you to find the correct stops you were looking for? 4 - Easy How would you improve this experience? stops in right order does not know what a CTA train stop number is can't search for a CTA L stop - such as "Sheridan station" How does this feature compare to other trip websites and apps that you have used in the past? like google maps; but don't love it likes that it doesn't force you to have location on likes the ability to buy and use Metra tickets would welcome the ability to use NFC How would you register a new Ventra card if you wanted to add a card? Where would you find this? Going to my Ventra cards and clicking on the plus Did you create a new account during the remote test? Yes Please create a new account on the test device. Tell us about your experience as you go through it. How easy was it for you to create a new account? 5 - Very Easy How would you improve this experience? Problem is that most people have forgotten their Ventra password 61
  62. 62. How often do you purchase Metra tickets? 2 - 3 times a week Anything else to add? n/a Please purchase a Metra ticket using the test device. Tell us about your experience as you go through it. once in the cart like that you can add items to the cart doesn't like that the app clears the cart if you go back to change the ticket type How easy was it for you to purchase a Metra ticket? 5 - Very Easy How would you improve this experience? Problem is that most people have forgotten their Ventra password Now look for the Metra ticket you just purchased. Look through this page and the tickets. What do you think about this page? not sure if a non-Metra rider would know what ticket was go for what trip Please take a look at this screen, and tell us when the next train is. In 17 min Do you have any other comments or insight about this page? did not notice the cart on the screen in the app OVERALL REVIEW Do you like the Ventra app? Yes Why or why not? Happy to have a Ventra app; has wanted an app to purchase Metra tickets for a long-time Do you believe the Ventra app fits into your day-to- day routine? Yes Why or why not? she has wanted this for a long time How likely will you keep and use the Ventra app after this test? 5 - Very Likely What is your reason for keeping or not keeping the Ventra app? because it allows you to use Metra without a wallet; only carried cash for Metra tickets 62
  63. 63. How would you improve the Ventra app? How should it be imporved to make you more likely to use it? fix the back button; integrating NFC Was this your first CUTGroup test? Yes Do you like CUTGroup? Yes Did you like this CUTGroup test? Yes Anything else to add? people were very jealous about the fact she had the app 7/16/2015 6:49:07 PM 63
  64. 64. Tester Profile (given by tester): Daily CTA Rider Device (listed in initial screening email) iphone 6 Remote Test When using the app for the first time which of these actions did you do? Logged in to an existing account Have you added CTA transit value or passes to your account using the Ventra app? No Have you purchased Metra tickets? No If you have made a purchase, please tell us about that experience. I have not made a purchase yet. What are some features that you have used in the app? Check all that apply. Used Trip Tools Looked up Ventra Value Tell us about your overall experience using the Ventra app. So far the really like the app! I like that you can purchase both CTA and Metra tickets and that schedules for CTA, Metra and Pace are all available in one place. In the past I've had separate apps for CTA train vs bus vs metra. I also like that you can "heart" the stops you use frequently so you can easily pull up that list in the future. I have had some issues with the transit history feature not working and getting a "connectivity lost" message when trying to search for a train stop. It would also be nice to have a map feature within the app. i didn't see that - only lists of nearby stops instead of a map to show you where those stops are. How appealing do you find the visual design of the app? 5 - Very appealing Tell us about what you like or do not like about the visual design of the app. How would you improve it? I like the color choices and the font and format of the design makes it really easy to read. How easy has it been to use the Ventra app? 5 - Very Easy How would you improve this app to make it easier to use? Add in a map feature. 64
  65. 65. 7/13/2015 11:40:14 AM In-person Component Proctored by: Sonja Marziano How often did you use the Ventra app while you have had it? Multiple times a day How easy was it to make a CTA ticket or pass purchase using the Ventra app? Not applicable - I did not make a CTA ticket or pass purchase How easy was it to purchase a Metra ticket using the Ventra app? Not applicable - I did not purchase a Metra ticket If you purchased a Metra ticket, did you use it on the train? Not applicable How was the experience using Metra tickets on the train? Side note: Very surprised to see that Metra was part of this. Normally, you see these apps as separate, and to see them together was great. Likes that you can purchase on the app especially at stations that do Go to "My Ventra Cards" and review the page. We want to find out if this page works for you and how to improve it. Past remote experience: Surprised by how easy the Trip Tools is! Normally checks the schedule in the morning and evening Likes how the amount is one of the first things that you see. "I usually have a monthly transit pass, but it hasn't kicked in yet." and therefore, is using transit value. Likes that she can add the value as she walks up to the train. Takes awhile to load "Transit History," usually multiple tries. How would you explain what the following features mean to you. Think about how these values are different from one another: "Add Transit Value" "Add Pass" Should be under "Add Transit Value" therefore, you click add "Transit Value" and then it gives passes an option. "Add Autoload" 65

×