SACHDEVA_SHALEEN_1437822_Evaluation Report on IRCTC.com_CI7700

Evaluation Report on - IRCTC.com
K1437822@kingston.ac.uk 1
Evaluation Report
On
IRCTC.com
Shaleen Sachdeva (K1437822)
Kingston University
CI7700 User Experience Design (System)

Consistency is one of the most powerful usability
principles: when things always behave the same,
users don’t have to worry about what will happen.
Instead, they know what will happen based on
earlier experience.
— JAKOB NIELSEN

Project Brief & Project Aim
Introduction & Background
Analysis
Aim of Usability Test
Method
Results
Redesign Recommendations
Conclusion
Appendices
References
Content
4
5
15
17
18
26
38
40
41
49

Project Brief
&
Project Aim
Project Brief
The coursework is to analyse, usability test, redesign and prototype a web &
mobile based website and provide a multi channel user experience adopting a
user-centered approach.
Project Aim
The aim of the project is to evaluate the current IRCTC website (desktop &
mobile) and compare it with it’s competitor website ( Cleartrip ) and accordingly
improve its user experience and user interface.

Introduction
&
Background
About IRCTC
Indian Railway Catering & Tourism Corporation (IRCTC) - it is a collateral of
Indian Railways which handles tourism and online ticket booking for railways
and flights.
It offers easy rail and flight ticket booking.
It has schemes like Shubh Yatra where you can pay for the whole year and enjoy
the discounts the entire year and also offers adventure tourism packages.
IRCTC Official website: http://www.irctc.co.in

IRCTC Current Trends
The current trends carried out by alexa.com shows that IRCTC is ranked 48th
travel booking website in India and 517th on global chart.
The engagement rate of visitors visiting the site is 5.94 which has dropped by
3.57%. The average time user stay on the website is 10:26

Project Scope
The Scope of the project consist of analysing the IRCTC website, usability
testing and then redesign the website for multi-platform (desktop, tablet
and mobile).
The project will be complete by 19th of December 2014.
It will include a report, prototype of a website (low fidelity and hight fidelity)
and a multimedia presentation.
Target Users
Everyone who uses IRCTC to travel around India.
User Group
The users are divided into 3 groups
• Indian
• NRI’s
• Foreigners
Project Scope, Target Users
& User Group

The main focus of the usability test was to identify problems faced by the
users while checking the train availability or booking the tickets.
The areas of where participants can face problems are
• Journey arrival and destination column
• Train Time-Table
• Ticket Fare
• Train Coaches
• Seat Availability
• Book now
• Travel summary
Overall Problems

IRCTC - Current Website Interaction
The current IRCTC website is simple and can be navigated easily however there
are major issues with the layout, readability and eye movement which distracts
the user which results in delay in ticket booking process due to which user get
irritated and annoyed.
Login Page
The user is thrown by lots of information and adverts as soon as they land on the landing
page of the website.
The Login box has a Captcha code which irritates users every time they login which delays
the booking process. Also every time user need to login to check train availability or book
tickets which increase steps towards final output.

Plan My Journey
The main attention of this page is taken by the adverts and secondary information because
of which it is difficult to locate the primary information - Plan My Journey.
Also the Text field and buttons used for Plan My Journey are very small which decreases the
readability.

Train Time-Table
The table used, to display train timings and other related information is small and cluttered
which leads to readability issue and confuses the users.

Train Details

Summary

Expected Usability Issue
Even tough users may be able to book a train ticket but due to the poor
interface design, information architecture and inconsistent layout, a user may
get confused and frustrated and might just end up using any other website.
The website is aesthetically simple but because of bad user experience
design it may be difficult for users to memories or learn to use the website.

Hierarchical Task Analysis
Analysis
Personas
The main use of persona is to create a realistic representation of you key users
for your reference.
These personas are base on the user group defined.

Karan is a Business Man living
in Ahmedabad. He travels to
New Delhi three to four time in
a month via train for his business
meetings. He uses IRCTC to book
his train tickets.
Anu is a Non Residential Indian.
She lives and studies in London
and visits India every 1 or 2 years.
She loves to travel around India
via trains and uses IRCTC to book
train tickets.
Anu Chitkara
Uses IRCTC Website:
• To book train tickets
• Check train availability
Uses IRCTC Website:
Age
Gender
Age
Gender
Train User
Train Usage
Internet Savvy
Train User
Train Usage
Internet Savvy
34
Male
22
Female
YesYes
Wants from IRCTC Website:
• Easy to use
• Remember journey history
• Round trip option
• Quick links
• Easy to use
• Check train tickets without
signing in
Technology Skills:
• Uses internet at work
• Owns a smart-phone
Technology Skills:
• Uses internet for more than
20 hrs. /day
• Owns a smart-phone and an
iPad.
Dan is a traveller. He loves
travelling and exploring history
and culture of different places.
He wants to visit India and wants
to travel around India via trains.
He wants to use IRCTC for his
train tickets.
Uses IRCTC Website:
• Never used IRCTC
Age
Gender
Train User
Train Usage
Internet Savvy
28
Male
No
• Easy to navigate and under-
stand
• Find train routes on map
• Find trains without signing in
Technology Skills:
• Owns a smart-phone and uses
it to do his daily work
Dan BrownKaran Kapoor

Aim
Of
The Usability Test
The main purpose of the Usability Test is to understand the strength and
weakness of how easily and efficiently a user can book a train ticket on the
desktop and mobile website.
The main aim of the redesign was to provide user a flawless user experience
where they can book a train ticket in few steps without any difficulty.
Multiphase approach was used in the usability test.
Phase 1 - Testing competitor website
A competitor website was selected for the Usability Test in this case Cleartrip.
The same tasks were conducted on the website to understand the users
reaction.
Phase 2 - Testing with different devices.
Desktop and mobile websites was tested to understand the usability, the
responsiveness and ease of use on both the devices.

Experimental Design
To achieve the goals outlined in the aims, the usability study was conducted
with 6 participants form each user groups. The participants were told to
complete 4 tasks for each website - 3 for desktop website and 1 for mobile
website.
A Common Industry Format (CIF) usability test was carried out on both IRCTC
and Cleartrip to evaluate and compare the two websites.
During the task the session was recorded to analyse and identify potential
area of improvement to the website like
• Navigation choice
• Task completion rate
• Errors
• Comments
• Overall satisfaction ratings
Method

Participants
In total 6 participants were choose from each user groups for the usability
test, the participants were contacted through verbal and email contact.
2 participants from each type of persona were selected for the usability test.
There were 2 females and 4 male participants and were aged between 18-60.
Participants - Table

Tasks /Scenarios
Following task were given to the participants for completion.
Task 1 - Find 3 Trains which take minimum travel time.
Imagine you are traveling from New Delhi (NDLS) to Ahmedabad (ADI).
Using IRCTC find 3 trains which takes the minimum time from New Delhi to
Ahmedabad.
* Participants will face problem while viewing
the train time-table as the table is badly
designed and have readability issues.

Task 2 - Check the availability of the seats.
Once you have figured out the least time taking train from New Delhi to
Ahmedabad find out the availability of seats on 2nd, 3rd and 4th January
2015 for AC 2 tire (2A) coach and AC first Class (1A) coach for that particular
train.
* Participants will be lost and confused while
doing this task as the main information is not
shown upfront.

Task 3 - Book train tickets for 3 people.
Imagine you are going back from Ahmedabad to New Delhi with 2 of your
friends. Using IRCTC website book train tickets for AC 2 tire(2A) coach for
your friends and yourself for 31st December 2014.
* Participants will be annoyed as the layout of
passenger details form has a bad layout and is
time taking.

Task 2 - Check the availability of the seats. (Mobile Website)
Once you have figured out the least time taking train from New Delhi to
Ahmedabad find out the availability of seats on 2nd, 3rd and 4th January
2015 for AC 2 tire (2A) coach and AC first Class (1A) coach for that particular
train.
* Participants will be lost and confused while
doing this task as the main information is not
shown upfront.
* Also the mobile website is not responsive.

Usability Metrics
During the tasks the nominal data, ordinal data and ratio data was collected.
Nominal Data - for task success and error
Ordinal Data - CSUQ likert scale data, levels of task completion attempts.
Ratio data - Time-on-Task, mouse clicks, Assists.
Performance:
Task Success: the success level was examined during the task execution.
Number of Errors (effectiveness): number of errors committed was recorded
when the participants were working through the task.
Task Time (efficiency): time for each task was monitored to see how much time
a participant take to complete a particular task.
Time to Proficiency (learn-ability): the experience of the participant was
recorded as they worked through the tasks.
Assists (Efficiency):
An assist was given to the participants if they faced any problem while going
through the tasks or begin to stray off the tasks.
Satisfaction:
A post-task questionnaire was prepared to measure participants satisfaction.
Computer System Usability Questionnaire (CUSQ) was used to measure the
response.
Post-test probing questions regarding the user interface and functionality of
the website was also asked.
Observation:
An observation form was also prepared to collect participants behavior and
performance based on errors, assists and time per tasks.

Procedure
• A single moderator interacted with the participant and the tasks lasted
for approximate 45 minutes.
• Laptop and mobile device was set up for participants.
• Silverback and iGlasses was used for recording.
• Participants was talked through and asked to sign a consent form.
• www.irctc.co.in and www.cleartrip.com was opened on Safari.
• The participant then was provided with the tasks and was told to think
aloud during the task.
• During the tasks the moderator was tracking the time and recording the
task progress.
• After the tasks were completed post-questionnaire link was given to
participants.
In addition the moderator also asked the participants general questions
regarding the website.
Equipments Used & Location
Equipments Used:
• Apple Mac Book Pro - Retina 15inch
• Apple Iphone 4
• Silverback application
• IGlasses application
• Quick Time Player
• Safari
• Stopwatch on Moto X
Locations:
Usability test took place at 2 locations -
• Kingston University, Surrey - KT1 2EE
• Seething Wells Halls of Residence, Surrey - KT6 5PJ

All the participants attempted the 4 tasks given to them. The tasks were
completed with 100% success rate but overall the participants found the
website to be frustrating and confusing.
On the other hand the cleartrip website had a better interface and user
experience. Based on the qualitative information, the website was simple,
easy to use and was consistent and responsive across multichannel.
Participants comments when using IRCTC website.
“The homepage is too cluttered.” - Participant 2
“Website is very confusing.” - Participant 4
“There is so much of information on one single page. It’s really confusing” -
Participant 3
“I don’t feel like using the site. It’s is not trustworthy at all” - Participant 1
Highlighted Issues
“ ‘Plan my Journey’ area should be located in the center because that’s the
important thing” - Participant 1
“I never realised clicking on the class will show me the
train details - Participant 2
“ The train time table is too small and cluttered” - Participant 3
“I am having trouble finding the train availability” - Participant 5
“After using Cleartrip, I would choose Cleartrip over IRCTC.
It’s much easier to use” - Participant 6
Results

Comparison Between IRCTC &
Cleartrip
The result of tasks designed for IRCTC & Cleartrip were analyised and it
demonstrated that the Task on time (mean) for IRCTC website took more time
as compared to the Cleartrip website. Participant found the 2nd Task most
challenging. This shows that they were confused about the train detail page
and where to look for the train availability and thus got lost on the website.
According to the table below, the errors and the assists for IRCTC website
was also more compared to the Cleartrip website. This showed that because
of poor user experience and user interface of IRCTC the participants got
confused and frustrated when going through the website and required help.
On the other hand user experience and user interface of Cleartrip was better
which results in less errors and assists.
Comparison IRCTC & Cleartrip - Table

Finding per Task
Mesurable results were gathered through time per task and post-task
Questionnaire.
Time per Task
The time per task for all the participants was recorded and measured. The
table below shows the time taken (in seconds) by each participants for each
given tasks. The time on task show interesting results.
Task 1 - 3 out of 6 participants were able to complete the task within 2 minutes
the other 3 took a longer time as they were new to the website ans found the
train time table and website confusing.
Task 2 - Participants spend the maximum time on this task as it was the most
challenging task. The user was confused with the class link and the availability
table. Participant 6 was lost and almost gave up.
Time per Task - Table

Overall the participants were not happy and satisfied while doing the task.
They were mostly confused and faced lots of usability issues.
Task 3 - 3rd Task was sort of a continuation of Task 2 therefore it took less
time to complete than task 2. The maximum time was taken while entering
the passenger details and captcha code.
Task 4 - This task was replica of Task 2 but for mobile website. Due to the non
responsiveness of the website the task took a little more time than expected.

Mean & Median - Graph
Mean & Median - Table

Errors & Assists
While conducting each task the number of errors and assists were also
recorded. The result shows that the participant faced the maximum problem
when working on Task 2 as it was the challenging task. Total of 10 errors
were made and 8 assists were given
Task 4 was replica of Task 2 but for mobile website so the number of errors
and assists were comparatively less.
Task 1 was the simplest therefore the number of errors and assists were the
least.
Task 3 was almost similar to task 2 so there were less errors and assists.

Error & Assist - Graph
Error & Assist - Table

Overall Metrics
After the task completion the participants were provided with a post-task
questionnaire where they were told to rate the IRCTC website using ‘Likert
Scales’ ranging from ‘Completely Disagree’ to ‘Completely Agree’.
Post-Task Questionnaire - Table

Post-Task Questionnaire - Table
The questionnaire result shows that the participants
• believed that the home page was not at all attractive.
• the website was difficult to navigate.
• the user interface and the layout was not simple and consistent.
• was not satisfied with the way the train information was displayed.
• was not satisfied by the train summary information.
• would not recommend the website to anyone else
• would rather user other websites over IRCTC.

Overall Rating
On the basis of Usability, Efficiency, Attractiveness and Learnability the
participants rated the IRCTC website. The Table and Graph shows the overall
rating.
Usability
The participants were not satisfied with the usability of the website as all the
rating was below 4 out of 7.
Attractiveness
The rating for the attractiveness of the IRCTC website was below average.
The score was approximately 2 out of 7.
Efficiency
The participants were not happy with the efficiency of the website they rated
the website 2 out of 7 which is again below average.
Learnability
Participants found the website easy to learn because of the similar tasks.

Overall Rating - Table
Overall Rating - Graph

Probed Question Response
The qualitative feedback from the Usability test was very useful as it showed
the proposed changes of IRCTC website by the participants. The participants
gave their opinion about the website while working on the tasks and after the
tasks.

The table below shows the recommended changes that have been determined
by participants views, behavior, success rate, errors and assists made during
the usability test. The recommendation table includes the ‘Usability Problem’,
‘Redesign Recommendation’ and ‘Severity’ level - High, Medium and Low.
It is believed that the following recommendation will improve the overall
experience and will address the problem faced by participants.
Redesign
Recommendations

The Usability Test conducted for IRCTC website have underlined various usability
issues which have interfered users in achieving information or delayed in completing
tasks. The satisfactory rate of all the participants was average or below average and
had a bad experience on the site.
Most of the participants found IRCTC website to be messy and cluttered. They found
the website working but felt that it needs resonable improvement to provide a user
friendly experience.
Even though participants were able to finish the tasks but they got confused and
frustrated while doing the tasks or after completing the tasks.
Also the mobile website was not at all responsive.
Thus, if the above design recommendation are carried out the user interface and
user experience of the website will improve. The redesign will focus on improving the
website‘s usability across multichannel and will focus on website’s main functions -
• Plan my Journey
• Train Time Table
• Train Details
• Summary
I believe by making the changes the user experience and user interface of the website
would be improved thus making thw website easy to use.
Conclusion

Appendices

Tasks (IRCTC)

Competitive Intelligence (no date). irctc.co.in Site Overview. Available at: http://www.
alexa.com/siteinfo/irctc.co.in (Accessed: 24 November 2014).
IRCTC site analysis (no date). IRCTC site analysis. Available at: https://www.scribd.
com/doc/157869310/IRCTC-site-analysis (Accessed: 12 November 2014).
Krug, S. (2009) Rocket surgery made easy: the do-it-yourself guide to finding and
fixing usability problems. United States: New Riders Publishing.
Nielsen Norman Group (no date). Nielsen Norman Group: UX Training, Consulting,
& Research. Available at: http://www.nngroup.com (Accessed: 12 November 2014).
Quotes on Design (no date). Quotes on Design. Available at: http://quotesondesign.
com (Accessed: 2 January 2015).
Search flights (no date). Cal. Available at: http://cleartrip.com (Accessed: 5 November
2014).
Tullis, T. and Albert, B. (2008) Measuring the user experience: collecting, analyzing,
and presenting usability metrics. United States: Morgan Kaufmann Publishers In.
References

(no date a). : : Usability.org | Solving Problems That Matter | Contact Us @
+1.443.831.1351 or kath@usability.org : : Available at: http://usability.org (Accessed:
14 November 2014).
(no date b). IRCTC Next Generation eTicketing System. Available at: http://irctc.co.in
(Accessed: 4 November 2014).
Citation
(Tullis and Albert, 2008, pp. 65 – 97)
(IRCTC site analysis, no date)
(Competitive Intelligence, no date)
(Search flights, no date)
(Krug, 2009)
(Nielsen Norman Group, no date)
(Quotes on Design, no date)

SACHDEVA_SHALEEN_1437822_Evaluation Report on IRCTC.com_CI7700

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (18)

Similar to SACHDEVA_SHALEEN_1437822_Evaluation Report on IRCTC.com_CI7700

Similar to SACHDEVA_SHALEEN_1437822_Evaluation Report on IRCTC.com_CI7700 (20)

SACHDEVA_SHALEEN_1437822_Evaluation Report on IRCTC.com_CI7700