World Usability Day - Usability test results


Published on

Usability testing of the Indian Railways site done by a group of Usability enthusiasts from HFI, for the world usability day 2007

Published in: Technology, Design
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • World Usability Day - Usability test results

    1. 1. Acid Test Results UT of Indian Railways Website Presented by: Sarit Arora
    2. 2. The Acid Test <ul><li>A team of usability professionals have conducted usability testing of the Indian Railways Reservation site – </li></ul><ul><li>Incidentally, this site won the “Genius of the web” award in the category - The Best Government and PSU Site . </li></ul><ul><li>Lets see how this site fared in terms of user friendliness </li></ul>
    3. 3. Usability Testing - Method
    4. 4. Usability Testing – <ul><li>Usability testing of Indian Railways reservation Site revealed: </li></ul><ul><li>Useful tools and functions </li></ul><ul><li>The facility of online reservation, checking status </li></ul><ul><li>Different payment options ( Credit Card, Online Banking) </li></ul><ul><li>Ability to book tatkal ticket </li></ul><ul><li>Home delivery of tickets </li></ul><ul><li>Web site does not match the users mental model </li></ul><ul><li>The homepage does not give any clue on how to search and book a ticket </li></ul><ul><li>Forces user to register and then search </li></ul><ul><li>Search form expects the user to remember station codes. Getting station codes is not easy </li></ul><ul><li>Search results do not help the user to make a decision faster and book tickets </li></ul><ul><li>Booking form is confusing </li></ul>
    5. 5. Participants <ul><li>13 Participants – Bangalore </li></ul><ul><li>Students, Professionals, Senior Citizens </li></ul><ul><li>Gender – 4 Women, 9 Men </li></ul><ul><li>Novice, Experienced users </li></ul><ul><li>Participants were asked to complete three basic real life scenarios </li></ul><ul><li>Search for a train </li></ul><ul><li>Compare prices </li></ul><ul><li>Book a Ticket </li></ul>
    6. 6. Accessing None of the test participants opened from the search engine results The Reservation links opens in a new browser window.
    7. 7. Some users felt the website did not appear to be associated with Indian Railways. Users expected the Indian Railways logo Users expected atleast one picture depicting a train
    8. 8. Homepage ‘ Train Reservation’ launched a new window with a list of hyperlinks. Users finally resorted to Help but found that the link was “not working”. Other links explored by users. Some users could not make the “connection” between ‘Signup’ and ‘Register’. Login should come after searching the trains, when we want to book Airline booking does not require registration. So why should train booking? The Homepage provided no guidance to what the user must do. Users then explored other links Most users clicked here first as it seemed most obvious link to start searching for trains.
    9. 9. Task 1: Search Trains Home page Task Description Given a date, users were asked to find out if there were any trains to Chennai on that day. Search page View results User’s Mental Model Home page Search page View results Registered User? Login Register Y N Task Flow in the website
    10. 10. Registration - Highlights
    11. 11. Registration (1) All fields in the form are mandatory Users felt the Registration was too long and cumbersome. Website assumes all users have an ‘Office Address’ Registration often failed due to poor response time. Very lengthy process. Prefer to go to station and book tickets Only advance users can login. Layman would not be able to register Want registration like Yahoo or Gmail, that's easy Users are asked to fill gender, marital status, occupation, etc.
    12. 12. Registration (2) Users often missed this error message and repeatedly attempted to submit this screen When informed of the error, users felt the password and confirm password fields should not have been shown.
    13. 13. Search Most users entered city names in the ‘From’ and ‘To’ fields instead of station code. Despite having few fields, the Search feature proved to be a considerable challenge. Even after reading the “definitions”, most users did not know the difference in the Ticket Type. Most selected this randomly as it is mandatory. One interviewee could not spell “Chennai”. Felt the website was not usable. When prompted, users explored other links hoping to find the Station Codes This error message was a major stumbling block. Users has no idea what went wrong.
    14. 14. Selecting the Station Many users felt they had to copy the station name from the list of stations below The field is in fact a filter for City name. I do not understand Station Code. It is irritating to find the code. There is no help in the site. Lack of instructions made simple things confusing to the users.
    15. 15. Search - Highlights
    16. 16. The RIGHT search criteria Field length is not adequate. The station code is “hidden”. The field still contain more than 4 characters.
    17. 17. Task 2: Compare Rates Task Description Given a budget, users were asked to find the best option across trains and classes. “ Best Option” is based purely on the user’s patience in exploring and refining search results Submit search Compare results Select best option User’s Mental Model Select Class Select train View fare Task Flow in the website Check availability
    18. 18. Search Results Train information should be in tables with cost on same page to compare. Information should be evenly distributed. Lengthy and frustrating. Better to call station master Users felt all of the displayed trains had available tickets. Multiple windows allows me to compare. The result is displayed in ‘Days of the Week’ though the criteria is date based Users are expected to Book Ticket even though no “decision making” information (Fare & Availability) is provided in the results. All such information is distributed among different pop-ups and is not available collectively.
    19. 19. Task 3: Book Tickets Task Description Having identified an option, users were asked to book their ticket. Select Train Book Ticket User’s Mental Model Task Flow in the website Select Train Book Ticket
    20. 20. Booking Form The ticket booking screen continued challenging the users. Options for quota are different from the Search screen Users faced difficulty understanding the terms… “ Select Master passenger” and “ Select passenger from Master list”. ‘ ID Card Holder’ field listed numbers instead of names. This confused the users.
    21. 21. Payment Users did not have an option of going to the list of banks from this screen. Replan takes the user back to the default booking page.
    22. 22. Booking - Highlights
    23. 23. Usability Metrics - Satisfaction Scale 1 - Strongly Disagree 7 - Strongly Agree <ul><li>Users found the website was quite useful </li></ul><ul><li>However, it did not fare well on the ease of use. </li></ul>
    24. 24. Summarizing their experience… <ul><li>What users liked </li></ul><ul><li>Online booking facility, which users felt helped them avoid the trouble of going to the station or to an agent. </li></ul><ul><li>Ability to select Tatkal and Class before finalizing the ticket. </li></ul><ul><li>Detailed information on the train route with all stations and halts. </li></ul><ul><li>What users did not like </li></ul><ul><li>Having to register just to find information on train timings. </li></ul><ul><li>Lengthy registration process. </li></ul><ul><li>Process of selecting Station Codes. </li></ul><ul><li>Search results did not show availability or fares. </li></ul>
    25. 25. The Verdict <ul><li>The IRCTC site is extremely useful. </li></ul><ul><li>However, Novice users will rely on an ‘experienced’ user to book a ticket from </li></ul><ul><li>They may still prefer to go and book from a booking counter rather book rail ticket online on their own. </li></ul>
    26. 26. Contributing Team Members <ul><li>Ananya Roy </li></ul><ul><li>Pravin Roberts </li></ul><ul><li>Anand Kumar </li></ul><ul><li>Juhi Joseph </li></ul><ul><li>Madhura </li></ul><ul><li>Neha Srivastav </li></ul><ul><li>Vivek Dhage </li></ul><ul><li>Nisha </li></ul><ul><li>Kunal </li></ul><ul><li>Sudhindra </li></ul><ul><li>Vaibhav </li></ul>
    27. 27. Headquarters 410 West Lowe Fairfield, IA 52556 Phone: (800) 242-4480 (641) 472-4480 Fax: (641) 472-5412 B altimore 930 South Wolfe Street Baltimore, Maryland 21231 Tel: (410) 327-1012 / 1013 Fax : (410) 327-1014 Boston 1050 Waltham Street, Suite 410 Lexington, MA 02421 Tel: (781) 860-7200 Fax: (781) 860-7979 Chicago 8700 W. Bryn Mawr Avenue Suite 800 South Chicago, IL 60631-3507 Tel: (773) 714-2362 Fax: (773) 714-4910 Bangalore, India 310/6 HR Complex, 2nd Floor Koramangala, 5th Block Bangalore 560 095 Tel: +91 (80) 4150 7221/2/3 Fax: +91 (80) 4150 7220 Mumbai, India Unit 7, Shrishti Plaza, Next to Killick Nixxon, Off Saki-Vihar Road, Andheri (E), Mumbai 400072 Tel: +91 22 4017 0400 / 2847 5551 Fax: +91 22 2847 5554 Pondicherry, India Aurelec Premises, Prayogashala Kuilapalayam Village Auroville 605 101 Tel: +91 413 26232 95/96 Fax: +91 413 2623297 Singapore 9 Raffles Place, Level 58 Republic Plaza, Singapore 048619 Tel: +65 6823 1368 Fax: +65 6823 1377 China 407, No. 555, Nanjing Road West Shanghai, China 200041 Tel: +86-21-52132061 Fax: +86-21-52132062 Minneapolis 8400 Normandale Lake Blvd, Suite 920 Minneapolis, MN 55437 Tel: (952) 820-4442 Fax: (952) 921-2306 New York 95 Morton Street Ground Floor New York, NY 10014 Tel: (212) 905-3495 San Francisco 235 Montgomery Street Suite 810 San Francisco, CA 94104 Tel: (415) 765-0962 Fax: (415) 765-0961 London, UK Grosvenor Gardens House 35/37 Grosvenor Gardens London SW1W OBS UK Tel: +44 20 7953 4010 Fax: +44 20 7953 4011 Thank You