Evaluation Test On Power Meeting
Upcoming SlideShare
Loading in...5

Evaluation Test On Power Meeting



User evaluation test on Google's PowerMeeting an Ajax based groupware tool.

User evaluation test on Google's PowerMeeting an Ajax based groupware tool.



Total Views
Views on SlideShare
Embed Views



3 Embeds 9

http://www.linkedin.com 5
http://www.techgig.com 3
http://www.lmodules.com 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Evaluation Test On Power Meeting Evaluation Test On Power Meeting Document Transcript

  • Powermeeting Usability Evaluation – Group 2 Anastasia Tsoutsoumpi Billy Kwasi Yeboah Fatima Sabiu Baba tsoutsoumpi_anastasia@yahoo.com afrikaba1@hotmail.com fatibaba@hotmail.com Vishal Dixit Ijeoma Udejiofor vdixit84@gmail.com ij_udejiofor@yahoo.co University ID: ABSTRACT (IJEOMA UDEJIOFOR)  Financial such as personal finance and It cannot be over emphasized that the usability of any accounting. software system is a key factor that determines whether  Travel, weather information and map searching. the system is going to be adopted for use by its intended users. PowerMeeting is a web-based synchronous system  Books and articles searching. built on Ajax technology and Google web Toolkit.  Spreadsheet and word processing. The aim of this paper is to measure the usability of  Amusement and fun sharing. PowerMeeting. We have done this through evaluating the user interface and the functionality of the PowerMeeting  Start page modification. framework. We also evaluated the functionalities of the GroupCalendar, a tool in the system.  Calendars and planning. Through careful and result-oriented planning, we  Question answering and searching. developed user tests which were a combination of  Information exchanging and opinions sharing. standard methodologies effective for identifying and measuring user errors, and main usability issues.  Social networking and communication. In this paper, we have also made recommendations for  Employment, commerce and news. improving the interface and functionality of the system.  Language learning. These were deduced from a qualitative analysis of the results obtained during the tests.  Health and sports. INTRODUCTION (ANASTASIA TSOUTSOUMPI)  People searching. Nowadays, web applications constitute an important aspect of everyday life. The popularity of Yahoo mail,  Web analytics. Gmail and Facebook is a clear evidence of that. However, Three of the most famous Ajax web applications are Web applications are not only related to communication Google Maps, Google Suggest and Gmail. Other or social networking. In fact they can be classified in examples of Ajax applications are Google Groups, Yahoo categories according to their functionality. FrontPage, Yahoo Instant Search, Windows Live, Orkut Further to this classification, applications can also be (social networking), Zimbra (email), Writely, Zoho, characterized by the technology upon which they are gOffice and AjaxOffice (on-line word processors), Kiko built. Therefore, they are divided in three categories; the and Calendarhub (calendars). Also, PageFlakes, Netvibes, conventional web applications, the rich internet and Photopage (free start pages), Travbuddy (for creating applications (RIA) and the Ajax based web applications. travel journals and sharing travel experience), Digg (for Many developers and web applications experts consider technology news), Meebo (instant messenger), Amazon’s the latter to be synonym to RIAs. However, some others, A9.com (search engine), Yahoo Flickr (photo sharing). argue that this is not entirely the case and believe that Finally, some examples of Ajax based business Ajax is just one of the technologies used for the applications are Salesforce.com, Basecamp (project development of rich applications. management), Backpack (organizer). Types of Web Applications Positive User Experience The basic advantage of the Ajax based web applications As far as the functionality is concerned the following are the interactivity and richness of the web interface. categories can be noted. Also, in contrast to traditional web applications they are
  • faster. The reason is that only small amounts of data are accessibility, which focuses on users with disabilities. For being refreshed and there is no need for the whole page to instance, Ajax applications are not friendly for visually be reloaded. All the above aspects had led users to impaired users due to the fact that screen readers are consider Ajax based applications being similar to desktop absolutely incompatible with Ajax technology. applications. As far as the security of Ajax web applications is However, it is not only the desktop ‘look and feel’ of concerned, experienced computer users argue that the Ajax applications that makes them so popular. There are level of security is not enhanced when compared with also many examples which reinforce the fact that such traditional web applications. In reality, security issues applications vastly improve user experience. For instance, such as ‘cross-site scripting’ are common for all types of for the completion of web forms with data (such as email web applications. address or credit card number) users do not experience a Finally, despite the fact that Ajax applications do not need dead time during the validation of their data. Another browser plug-ins, users should have JavaScript enabled in example is that user experience is not slowed down when their browsers. However, this consist a daunting issue for moving from their email inbox folder to the ‘compose less experienced computer users. Also this category of mail’ option, or when they perform a big number of clicks users, cannot cope with network delay. They consider it to while searching for books in the Amazon website. The be a severe problem, unless it is clearly indicated to them reason is that requests to the server for the sending and - through a screen message - that what happens is a delay retrieving of the various components of the graphical user rather than an internal problem of the application. interface are absolutely transparent to the users of the application. METHODS (BILLY KWASI YEBOAH AND IJEOMA UDEJIOFOR) Furthermore, animation such as fading text displayed on In the recent past all the content of a web page had to be the screen which inform the user that an option has been reloaded for every HTTP client request made described as just saved, add up to the user friendliness of Ajax synchronous communication Fig 1. This is unnecessary applications. Moreover, along with friendliness, loyalty to since most pages of a web application contain common an application is a clear evidence of Ajax success. That content and led to considerable delays while pages were applies mostly to e-commerce. With the aid of Ajax loading resulting in a diminished user experience. A technology user interfaces are presented in a way that stirs group of client-side technologies, Asynchronous customers’ emotions, and unconsciously leads them to JavaScript and XML (AJAX) has been used to eliminate choose the option of online purchase instead of buying this effect resulting in more interactive and dynamic goods in a shop. applications. Negative User Experience This is achieved by seemingly introducing an extra layer, Traditional web applications support both the use of the Ajax engine, to the traditional communication model ‘Back’ button and the ‘Bookmarking’ functionality. On for web applications as shown in the diagram below. The the contrary, these features have been discarded in Ajax Ajax engine is responsible for asynchronous applications. Because of that, many users are either communication. annoyed or confused. An additional source of confusion is color blinking behind changes, which is frequently The Components of Ajax include: mentioned by users of ‘Yahoo! Finance’ application.  XHTML, HTML and CSS used for creating the Also, being accustomed to traditional web applications, GUI and styling the web pages people interacting with Ajax applications, rarely realize how the updating and refreshing process based on Ajax  The document Object Model (DOM) is used by technology works. Further to that, they claim that if JavaScript code to produce interactive information is given about whether a box is draggable or a applications text field is editable, could prevent them from confronting usability problems.  XMLHTTPRequest or XMLHTTP used to retrieve the data from the server Additional usability issues arise when users try to send their friends links to a web page they find interesting.  JavaScript used to create the Ajax engine More specifically, the receivers by following the link can Power Meeting only see the default content. Similar problems exist with search engines, as the clicking of a link displays again the PowerMeeting is a framework that makes use of the default content. This problem is widely known as SEO Google web Toolkit and provides a common foundation (search engine optimization) issue, and usually stems for developing a synchronous groupware application from the absence of a proper sitemap. Another issue of consisting of a set of plug-ins, i.e. groupware tools for major importance in Ajax applications performance is specific collaborative activities.
  • PowerMeeting is a framework that makes use of the evaluators in finding usability problems in a user interface Google web Toolkit and provides a common foundation (Nielsen and Molich, 1990; Nielsen 1994). for developing a synchronous groupware application The Usability Evaluation consisting of a set of plug-ins, i.e. groupware tools for specific collaborative activities. The usability evaluation was made up of three parts: Wang, the author of PowerMeeting lists its key features as 1. The Background Information Sheet:  Uses standard Web browsers, including AJAX- We used this to collect the back ground information enabled browsers running on mobile devices as of all the participants for both the user interface and front end making the system widely available Group Calendar Test. The information collected and eliminating the need for installation. includes the users’ computer literacy levels, computer  Supporting direct manipulation of shared programming experience especially with the artifacts and direct textual and voice components of Ajax technology (Cascading Style communication making real-time collaboration Sheets, Javax Swing Package and JavaScript), their and coordination possible. opinions of web applications and possible motivations to use or not use Ajax applications.  Supporting rich user experience through rich set of graphical widgets, fast feedback, and nature 2. The User Tests: interaction means, such as tele-pointing, drag- This was made up of quantitative and qualitative test and-drop, and gesture. of the functionality of PowerMeeting and the  Maintaining view-data dependency and data Groupcalendar. We adapted it differently for the user consistency across clients and server; and interface/functionality test and for the Group maintaining data and collaborative session Calendar. A brief description of these tests is given persistency essential for any document-based below: real time collaboration and for a smooth moving a. The User Interface/PowerMeeting Functionality between synchronous and asynchronous Test collaboration modes. We chose six functionalities randomly in other to  Making the development and integration of task- evaluate the user interface and functionality of specific groupware tools into the system easier. PowerMeeting. Its implementation technologies are:  Login and Logout functions  GWT GUI library and GWT RPC for the front  The Documentation end components.  The Agenda creation function  The RemoteServiceServlet class and the continuation technology provided by Jetty web  The Tele-pointer server for the back end components.  The Group chat  SQL database – for making objects persistent. Two users were evaluated concurrently. One was The general user interface layout of PowerMeeting asked to chair the meeting and the other was a (Figure 1 in Appendix B) has two main parts separated by normal participant. Below are the tasks asked to test a slide bar. The right side is a large working area for plug- the five chosen functions: ins such as shared gadget objects while the left side has a Function 1: Log in to the PowerMeeting menu bar which cannot be seen on the image captured (Wang, 2008). This menu has tabs, Create, Edit, Pointer, Task: Create a new task. Help and Logout. Beneath the tab “Create” is a list of Function 2: Test the documentation of Agenda (Agenda items), Current users (participants) and PowerMeeting. below this is the group chat which can be done be text or by Skype. Task: Read the documentation in five minutes. The PowerMeeting groupware tools include Pincard Function 3: Creating an Agenda board, MindMap tool, Presetation slide, Calendar tool etc Task: Create new agenda item called Heuristic evaluation was mainly used for the user CONTINENT interface because it is quick, cheap and easy evaluation of Task: Add in “categories” EUROPE user interface. It also effective and requires a small set of
  • Task: Add in “Idea”, country name France.  Task 3 - Change the time for your event Task: Put Ideas in their respective categories  Task 4 - View the events on the calendar through the day, week and month views Task: Replace “Idea” Nigeria with Kenya  Task 5 - Delete your event Function 4: Using the Telepointer. Task: Show and hide telepointer On the questionnaire, they were to respond to seven statements using the SUS scale type Function 5: Log out from the session chair account response and to four other questions using a binary and login as a user response – Yes or No. These set of questions Task: Log in to an already existing session, by focused on assessing the following metrics typing the Skype name.  Ease of use Function 6: Work on the group chat and on the  Efficiency voice conference with Skype  Main usability issues Task: Exchanging written messages For the next set of questions, users were to respond Task: Have a 2 minutes voice conference on the Group Calendar as a whole. These questions After the completion of the above mentioned tasks, were focused on obtaining user satisfaction participants were asked to respond to the questionnaire experience of the tool by using four questions from used for our assessment. the SUS. We included an extra question to test the users’ satisfaction with using a browser-based For assessing the Agenda, Telepointer, voice conference groupware. The responses were also an SUS scale tool using Skype and the textual chat, questionnaires were type response. design to respond to the usability heuristics discussed in Molich and Nielsen (1990). Participants were asked to We then went on to obtain qualitative data of the answer 14 questions using the SUS scale 1-5. user’s view of the usability issues they encountered in carrying out the tasks, their expectations from the tool and their For the Login session, we used heuristics (Recognition recommendations. rather than recall” and Flexibility and efficiency of use) 3. The Evaluators’ Guideline and Observation Form and also measured performance and satisfaction. According to Dumas and Redish (1999), there are two We understand that sometimes users’ comments aspects of the documentation which can be tested: may not give a true or exact picture of their whether users go to the documentation or how well the experience of the software while responding to a documentation works for them when they go to it. We questionnaire. We therefore decided to also make chose the latter because the former requires a large use of this form to record the observations we made number of participants. as the users carried out the tests. We recorded information such as the main usability issues faced For the assessment of the log in, log out and the by users, usability errors made by users, user documentation, another set of questions were asked to comments, facial expressions etc. This provided a obtain the necessarily data needed for the evaluation. solid basis for correct analysis of results and b. The Group Calendar User Test conclusion. After briefly and orally introducing Group The Participants Calendar as a group calendar tool to the users, we We chose to test students of the University of Manchester logged into PowerMeeting and selected the Group because they are a true representative of the target Calendar tool. We presented a type-written audience for PowerMeeting. They have varying levels of questionnaire to the users. We then asked them to exposure to computers and web applications. They are carry out the following tasks without providing assigned group tasks. They face the challenge of having to any documentation, user guide or assistance. meet again outside of the normal class schedule because  Task 1 - Add a new event of the varying locations where they live or work. They would therefore have to use web tools offering  Task 2 - View and modify the description of synchronous collaboration for effective communication in an event created by another user. (We order to achieve their goals. created this event before the test)
  • A total of twelve participants performed the evaluation. He sees some games based on web application as being We increased the number of users to improve the tiny and interactive. His top web applications include reliability of the data, analysis and conclusions made in MSN, Tencent, Blackboard, and Moodles. He thinks web the test as suggested by (Insert references for Observing applications are user friendly, fast, well designed and the User Experience). Six users evaluated the user secured. interface/functionality and six evaluated the Group He has used groupware tools such as skype and tencent Calendar. before. Factors that might motivate him to use an Ajax Users for Evaluation of PowerMeeting Framework based web application are its ease of use, speed, and Functionality/User Interface adaptability with the user interface. What might discourage him from using such an Ajax based web User 1 application is if it allows pop-ups and if it is insecure. Tim is a male MSc Informatics student. He describes both User 4 his computer literacy level and computer programming experiences as proficient. He has programming Abdullah is a male student in the HCI and User interface experience with JavaScript technology but none with CSS class. His level of computer usage and programming (Style sheets) and Javax.Swing Package. experiences is intermediate. He has some computer programming experience some experience with In general the use of web applications has improved his Javax.Swing Package technology. life because they are useful for providing information and social networking. His top five web applications include Web applications have affected his lifestyle by helping Facebook, Email, Windows Live Messenger, Twitter and him to communicate with the rest of the world. His top Web browsers which he uses for news and information. web applications are yahoo, google doc, facebook and adobe share. He characterizes web applications as He characterizes web applications as being user friendly, generally being user friendly, fast, well designed and fast, well designed and secured. Though he does work in secured. groups, he has never used any groupware tool before. He would use Ajax web applications if they were useful, easy He sometimes works in groups and has used skype and to use and if they did not have any faults and errors. He group documentation tools before. He might be motivated would be discouraged from using them if they did not to use an Ajax web based application if he finds it easy to improve his life. use, cope well with the user interface, similar to a desktop application, and if refreshing function is easy and quick. User 2 He might not use Ajax web based application if it has Xiaokun is a female student studying Information security and privacy issues. systems. She has an intermediate computer literacy level User 5 and is a beginner in computer programming. She has programming experience in JavaScript. Sayeeram is a male student in Msc Electrical Engineering. He describes his computer usage experience as She sees a web application like skype as being useful intermediate and his computer programming level as because it makes it easier to contact people from a beginner. He has no programming experience with Ajax distance. Her top five web applications are MSN, Skype, technology. Google search, Emails, and Blackboard. She believes web applications are user friendly, fast, well designed and Web applications have positively affected his lifestyle secured. because they allow him to keep in touch with the rest of the world. His top five web applications are facebook, She has used some groupware tools like google groups MSN, Email, twitter and Google. He characterizes web and skype. She is motivated to use web applications if applications as being user friendly, fast, well designed and they are easy to use and cope well with the user interface. secured. She might be discouraged from using an Ajax application if it is insecure and difficult to use. He works in groups but is yet to use an Ajax based web application. He would be happy to use one if useful and User 3 easy to use. He would be discouraged from using them if Ji is a male Informatics student. He is proficient in they did not improve his life. computer usage and intermediate in computer User 6 programming. Like User 2, he has no experience in CSS and Javax.Swing Package but has experience in Diwakar is a male student in MSc Healthcare JavaScript programming. Management. He is an intermediate computer user and a
  • beginner in computer programming and has none with Skype, Twitter and e-mail applications such as yahoomail Ajax technology. frequently and Skype for her group work. He sees web applications as being effective positively For her, web applications are generally user friendly, fast, because they allow him to communicate with friends and well-designed and secure. Ease of use, simplicity and family around the world. His favorite web based speed are her possible motivations to use Ajax applications are Google, Email, MSN Messenger, Skype, applications and a lack of these would demotivate her to and Facebook. He believes web applications are user use them. friendly, fast, well designed and secured. User 4 He has never used any groupware tool before. He would Bima is a 26-year-old male, Masters student of the use Ajax web applications if they had a positive impact on Construction Management Department. He is a proficient his life and if they were not complicated to use. He would computer user, has intermediate computer programming be discouraged from using them if they had limited experience but none with Ajax technology. impact on his life. Web applications have helped him in interacting with Users for Evaluation of Group Calendar common interest group members and in resolving and User 1 working on programs or projects. He mainly interacts with group members through the UoM Blackboard system Sana is a 22 year-old female studying Informatics. She is and Facebook Groups. He also uses YouTube, Facebook, computer literate, is a beginner programmer and has no Skype, Twitter, and MySpace. programming experience with Ajax technology. He thinks most web applications are user friendly and She uses Facebook, MSN, Skype, Blackboard, LinkedIn fast, not secure and that their designs could be improved. frequently. Group wares have improved her studying and His possible motivations to use Ajax applications would ability to carry out group work. She would be motivated be user friendliness and security while poor design and to use an Ajax-based application if it is easy to use and insecurity of the application would demotivate him to use has minimal features displayed on the user interface and them. discouraged from using it if it has a busy user interface. User 5 User 2 Ibrahim is a 23 year-old male, postgraduate student of the Deji is a 25 and is pursuing a Masters degree in Public Health department. He is a proficient computer Operations, Project and Supply Chain Management. He user with intermediate programming experience but none has intermediate computer literacy, is a beginner with Ajax technology. programmer with no experience with programming with Ajax technology. He uses Skype, Facebook, Gmail, Twitter and Google Maps. Google groups daily because they help him to He mostly uses web applications for sharing information, organize his social networks and make his activities faster social networking, and making job applications. He uses and easier. He uses mainly Google Groups and Google Facebook and recruitment web sites frequently and Calendar for his group work. mostly uses Google Groups for his group work. He thinks web applications are generally user friendly, He thinks most web applications are user-friendly, well fast, well-designed and secure. These features would also designed and fast but he is uncertain of their security. He motivate him to use Ajax technology. He has no possible would be motivated to use Ajax applications if they are demotivations about the technology. easy to use, offer real time communication, and secure and would not be motivated to use them if they are User 6 insecure. Waheeda is a 22-year-old female studying Information User 3 Security. She is computer literate, has intermediate programming experience and has programmed with Kerry is a 21-year-old female student of the Human Cascading Style Sheets (CSS), one of the components of Resource Management and Industrial Relations Ajax technology. department. She is a proficient computer user, has intermediate programming experience but none with Ajax She says web applications have made her life easier and technology. more fun but have also increased the pace of living. She uses Facebook, Google Groups, Windows Live Web applications have made life easier for her through Messenger, Gmail and Skype regularly for social improved means of social networking, task management networking, chatting and e-mail. For her group work, she and access to information. She uses Facebook, Google,
  • uses Google wave, Skype, Google Groups, and Google  The standard deviation is marginally over 73 calendar. seconds. She thinks that web applications are generally user  The confidence level at 95% shows that the friendly, fast, well-designed and secure. These are her average population completion time will be 91 possible motivations for her to use Ajax applications seconds. while excessive display of information would discourage her from using them. The following were observed for the Agenda: RESULTS  The average time taken for Agenda is 292 This section consists of the results obtained during the seconds with 33.33% of the participants Powermeeting evaluation for the user interface and group completing the tasks before the average calendar tool. completion time. Results for User Interface (Vishal Dixit)  The standard deviation is approximately 149 The results for the User Interface evaluation carried out seconds. are based on three metrics task success, errors and issues.  The confidence level at 95% shows that the Task Success: Table 1 in appendix B shows the average population completion time will be 156 percentage of participants who were successfully able to seconds. complete the tasks given for the User Interface tools The following were observed for the Tele-pointer: (Login, Documentation, Agenda, Tele-pointer, Voice Conference). To calculate the percentage of success the  The average time taken for Tele-pointer is 51 binomial distribution method has been used where 1 seconds with 33.33% of the participants indicates the task was a success and 0 indicates task completing the tasks before the average failure. The table also shows the 95% confidence interval. completion time. Graph 1 in the appendix shows the Tasks Success rates  The standard deviation is over 27 seconds. for all the tools in the User Interface:  The confidence level at 95% shows that the  All the participants were able to complete the average population completion time will be 28 tasks for login, documentation, agenda and tele- seconds pointer. The following were observed for the Voice Conferencing:  The task success rate for Voice Conference is 66.66%.  The average time taken for Voice Conference is 200 seconds with 83.33% of the participants As part of the analysis of the Powermeeting features, the completing the tasks before the average total completion time of all the tasks were measured for completion time. each of the users. Tables 2, 3, 4, 5 and 6 in appendix B show a summary of the tasks completion times for each  The standard deviation is approximately 82 user. seconds. The following were observed for Login feature:  The confidence level at 95% shows that the  The average time taken for Login is 75 seconds average population completion time will be 86 with 66.66% of the participants completing the seconds. tasks before the average completion time. Further there was a Chi-Square test conducted to see if  The standard deviation is marginally over 35 there is any significant difference in the task success seconds. between three different groups (novices, intermediate and experts). The participants were categorized in three  The confidence level at 95% shows that the groups by asking them their computer usage experience. average population completion time will be 37 Tables of the test results are displayed in appendix B as seconds. Table 7, 8, 9, 10 and 11. The following were observed for the Documentation:  The results show that there is no difference in the  The average time taken for Documentation is task success between the three groups for Login, 180 seconds with 50% of the participants Documentation, Agenda and Tele-pointer as the completing the tasks before the average task success rate is 100% for all the four tools. completion time. (Task success rate is shown in the bar graphs and also in the Fig. Task completion rate above)
  •  The result for Voice Conference however is The issues encountered are summarized in table 13 in different; the task success between the three appendix B. groups is shown with 0.6 distribution. Graph 7 in appendix B shows the percentage of Errors: This section concentrates on the errors participants who encountered issues in the user interface encountered in the tasks performed by the participants. tools. The aim here is to understand how the level of experience  In login 66.66% participants found the issues as can affect the errors made by the participants. high severity and 33.33% as medium. Table 12 and graph 2 in appendix B shows the number of errors encountered in the tasks performed for the user  For documentation, Tele-pointer and voice interface. conference the percentage of high severity was 100%, 83.33% and 50% respectively. Graphs 3, 4, 5 and 6 in appendix B show the relationship between the level of experience and the errors made by This indicates that the issues encountered for the participants. The scatter plots indicate a negative slope documentation and Tele-pointer needs to be addressed for between the two variables (months of experience and the next version of PowerMeeting. average errors per day). To understand it better the Apart from the evaluation test the participants were asked correlation coefficient was calculated. to make some comments about the user interface tools which are summarized in table 14 in appendix B.  The correlation coefficient for login is -0.70 which shows that there is strong negative Results for Group Calendar Tool (Fatima Baba) relationship between the two variables (as the The result from the evaluation of the tool focused mainly level of experience increases the errors on efficiency of the system and the ease of completing the decreases). It shows that the average expectancy tasks using the tool. The findings also identified some of errors is -0.7 for the participants having a issues the users faced and the severity of those issues. A month of experience with the tool. summary of the data gathered from the evaluation is given below.  The correlation coefficient for documentation is - 0.51. Efficiency: was measured in terms of the user’s success in completing the evaluation tasks and how long it took  The correlation coefficient for agenda is -0.06. them to complete the tasks.  The correlation coefficient is not applicable for In measuring the success and failure of each task, the Tele-pointer as there were no errors encountered. binary success rate was used where successful tasks were  The correlation coefficient for voice conference assigned 1’s and failed tasks were assigned 0’s. Table 15 = -0.06. in appendix B shows the averages and 95 percent confidence intervals of the binary success data. Issues: This section of the results concentrates on the issues that the participants encountered while carrying out Graph 8 in appendix B shows that: the tasks. The issues are categorized on individual tasks  Tasks 1 and 2 were successfully completed by all the done in tools (Login, Documentation, Agenda, Tele- users. pointer, Voice Conference) and further being classified to having high, medium and low severity.  Task 3 has a sample mean completion of 50%. Hence there is a 95 percent chance that the true mean will fall  High severity: Issues are classified as high between 6 and 94 percent. severity if it has been faced by many participants and also likely to make a major impact on the  The confidence interval is rather large as a result of usability. the small sample size used for the evaluation.  Medium severity: Issues are classified as  For Tasks 4 and 5 there is a 95 percent chance that the medium severity if it has been faced by many true mean will fall between 50 and 116 percent. participants having a minor impact on the As part of the analysis of the efficiency of the calendar usability or few participants but large impact on system, the total completion time of all the tasks was the usability. measured for each of the users. Table 16 in appendix B  Low severity: Issues are classified as low shows a summary of the completion time for each user. severity if it has been faced by few participants  The average evaluation time was 144.5 seconds and also has minor impact on the usability. with 50 percent of the users completing the tasks (Tullis & Albert, 2008). before the average completion time.
  •  The standard deviation is about 33 seconds. This Issues: with the tool were identified as the evaluation was can be attributed to the small sample size. being carried out and a documentation of these issues is Therefore estimating the true mean of the given in this section. The issues are categorized based on individual tasks in addition to being classified as having population will be less accurate if based on this high, medium or low severity. The severity classifications sample size. are the same as the ones given in the user interface issues.  The analysis also shows that with a 95 percent A summary of the issues identified are given in table 19 confidence, it can be inferred that the average in appendix B. population completion time will be 144.5 35 Graph 11 in appendix B shows that three issues were seconds. identified in Task 1 and 60 percent of the users encountered problems in relation to the issue with the The experience level of users was obtained during the test highest magnitude. It also shows that Task 3 had only one and was measured by assigning numerical values to issue identified which all the users encountered. This different levels of experience as shown in table 17 in indicates that the issue in Task 3 is the most sever and appendix B. This was to enable us test for the relationship should be given priority when the issues are being between the users’ previous experience with calendar addressed by the design team of the Calendar tool. tools and the time it will take them to complete the tasks After the usability evaluation of the calendar tool, the for the Powermeeting calendar tool. users were asked some questions on the tool in general The scatter plot shown in graph 9 of appendix B indicates and were also asked to give general comments general. a negative slope between the two variables. For each Table 20 in the appendix summarizes the responses of the increase in level of experience there will be 0.65 seconds users. decrease in the completion time. DISCUSSION The correlation coefficient of -0.65 shows that there is a This section discusses the findings of the usability test for strong negative relationship between the level of the user interface and calendar tool. experience with other calendar systems and the time taken User Interface Discussion (Anastasia Tsoutsoumpi, to complete the tasks in the usability test. Vishal Dixit and Billy Kwasi Yeboah) Ease of Use: This was measured in terms of how easy it The user interface tool was evaluated on three metrics was for the users to complete the tasks. The users were task success, errors and issues. asked to rate the ease of use of the Calendar tool by completing a questionnaire which was based on the The findings of the evaluation suggest that there is no System Usability Scale (SUS) and has been presented in significant difference in errors among users with different the Methods section. The users’ responses which are the level of computer experience. This could be as a result of SUS scores (SUS scores are calculated by adding the time pressure the users where put under during the test rating figures and multiplying by 2.5) are presented as and the fact that they were not familiar with percentages in the table 18 and graph 10 in appendix B: PowerMeeting. For example, they had no reason to assume they needed to login as administrators as they have no prior knowledge of the fact that only  Easiest task - Task 4. (The task has an average administrators could create agenda items. ease of use rating of 66 percent with only 33 Calendar Tool Discussion (Fatima Baba) percent of the users rating below the average). Three main aspects of the calendar tool were measured during the usability evaluation. They are efficiency, ease  The most difficult task - Task 3. (The average of use and issues (problems). ease of task rating of this task is 55 percent with about 67 percent of the users rating below the average).  Overall the users did not find the tasks too easy to complete thus the average ease of use for all the tasks are 62 percent with the maximum efficiency of 75 percent.
  • Efficiency: The five tasks were measured for success and recognized during the test as well as from the statistical the means and confidence intervals were calculated in the results are outlined below. previous section. The confidence interval allowed us to estimate to what degree of accurateness the results from the sample can be used to generalize how successful the Recommendations (Anastasia Tsoutsoumpi and task completions will be when used in the real world. The Ijeoma Udejiofor) first two tasks were successfully completed by the users After analyzing the result of the usability test taking into but that does not mean everyone who will use the system consideration improvement ideas from the users the will be able to successfully complete the tasks. The small sample size has to be considered before any following recommendations are given in order to improve generalizations can be made. On the other hand, task 3 the Powermeeting system: had a really low average completion success and this  Connection for the user conference should be point out that there is a problem with the tool and if used made quicker in the real world a lot of people may not be able to use it to complete such a task. Tasks 4 and 5 only showed  The login should also be made quicker and moderate levels of difficulty in successful completion and should have a mechanism to remember this should also be addressed when improving the passwords. calendar tool.  The logout button should be clearly indicated on The negative slope of graph 9 shows that as the computer the screen. literacy level of the user increases, the time spent in completing the tasks decreases. To better understand the  Search mechanism should be built in the strength of the relationship of these variables, we documentation. calculated the correlation coefficient which gave a value of -0.71. This can be interpreted to mean that it is 71%  The recycle bin used to delete the agenda items probable that proficient computer users will find the should allow users retrieve deleted items system more usable than beginner computer users. Individuals, irrespective of their level computer literacy,  The design of the group calendar tool should be need to be to manage group events using a calendar tool. enhanced so that it is possible to easily edit the Hence, the general features of this tool should be enhanced to encourage even beginners to adopt it as their time for an event by entering a new time using group calendar tool. the blinking cursor in the pop-up dialogue box. This was a high severity issue and the general A standard deviation of 33 was obtained and this indicates a large variance between the completion times of the low usability results for task 3 echo this problem. different users. Although we can attribute this to the small sample size used for the test, it could also be indicative  Standard alternatives should be made for core or that it will take some classes of users far more time to commonly used functions. For example users complete tasks using the tool when compared to other should be able to select the “add”, ” edit” or classes of users. This could be as a result of several other “delete” event button by right clicking on the factors such as level of computer literacy as discussed mouse. Users should also be able to use simple above. Again, this indicates that the design should be key combinations on the keyboard, as an improved on to reduce the use time for all classes of users. alternative. Ease of use: from the results, it is clear again that there is  The users should get a notification of when an a problem with completing task 3. This confirms the event is added or edited by another group findings in the efficiency that showed task 3 as the least member and the identity of the person who efficient. In general all the users did not rate the ease of use of the calendar tool high, and this suggests that the added or edited the event. tool is not easy to use.  Currently in the group calendar, a deleted event Issues: with the calendar tool were identified during the cannot be restored. A means for restoring deleted usability test. These issues are valuable insights for the items should be added as a feature of the tool as developers and in order to improve the tool, the issues need to be resolved. Some recommendations for the this will greatly enhance its usability. improvement of the tool gathered from the issues
  •  When the mouse is placed on a clickable feature, As far as users attitudes towards web applications are a brief textual description of its function should concerned two basic points can be noted. Firstly, people be displayed to increase learnability and usability are always attracted by eye-catching user interfaces. However, the motivation to be ‘loyal’ in a web of the tool. application is its functionality. An additional point is that many users express their concerns about the security and CONCLUSION (ANASTASIA TSOUTSOUMPI) Firstly, the evaluation of the Power Meeting web the speed of web applications. That is a clear indication of application, gave us the opportunity to gain a good their demand for high quality standards in contemporary understanding of Ajax technology. Further to that, we had software products. the opportunity to familiarize ourselves with very Finally, the success of an application in the ‘web market’ important human computer interaction concepts. The is based exclusively on users’ acceptance of it. But they experience gained, through the preparation of the usability are the researchers and the professional software test and through the analysis of users’ feedback, will developers, who lead the technology one step further into improve our skills in the domain of software the future. development. The main reason for that is that we will be able to approach software design from the user’s point of view. 10. Giglio, Jason. "AJAX: Highly Interactive Web REFERENCES 1. Ajax Patterns. Whats Ajax. 25 March 2010. 1 April Applications." 2009. 2010 <http://ajaxpatterns.org/Whats_Ajax>. 11. INTERNET METHODOLOGIES JOURNAL AND 2. Arlekar, Sagar G. The Role of AJAX in enhancing NEWS. Are there Usability Issues with AJAX? 1 the user experience on the Web. 1 June 2006. 6 April 2010. 3 April 2010 March 2010 <http://www.roseindia.net/ajax/ajax- <http://www.imjan.com/internet-www/are-there- user-interface.shtml>. usability-issues-with-ajax/>. 3. Avangate. Usability Friends: Ajax. 29 October 2007. 12. Itura. AJAX SECURITY: ARE AJAX 1 March 2010 APPLICATIONS VULNERABLE TO HACK <http://www.avangate.com/articles/ajax-usability- ATTACKS? 2009. 05 March 2010 110.htm>. <http://www.itura.net/training/16-ajax-security-are- ajax-applications-vulnerable-to-hack-attacks.html>. 4. Brookes, J. SUS - A Quick and Dirty Usability Scale. 2009. 4 March 2010 13. Keely, Pavan. Using Ajax. 18 January 2006. 2 March <http://www.usabilitynet.org/trump/documents/Susch 2010 apt.doc>. <http://keelypavan.blogspot.com/2006/01/using- ajax.html>. 5. Bruno, Vince, Audrey Tam and James Thom. "CHARACTERISTICS OF WEB APPLICATIONS 14. Kluge, Jonas, Frank Kargl and Michael Weber. "THE THAT AFFECT USABILITY: A REVIW." EFFECTS OF THE AJAX TECHNOLOGY ON Proceedings of OZCHI 2005,. Canberra: CHISIG, WEB APPLICATION USABILITY." WEBIST 2007 2005. 2-4. International Conference on Web Information Systems and Technologies. 2007. 289-294. 6. Dumas, J.S. and J.C. Redish. A practical guide to Usability Testing. Exeter : Intellect Books, 1999. 15. "Ajax." Java Jazz Up 8 April 2008: 1-79. 7. Eernisse, Matthew. Build Your Own Ajax Web 16. Lerner, Reuven M. Ajax Application Design. 1 Applications. 28 June 2006. 5 March 2010 December 2006. 1 April 2010 <http://articles.sitepoint.com/article/build-your-own- <http://www.linuxjournal.com/article/9295>. ajax-web-apps>. 17. MacKay, Tara. Ajax Usability Concerns. 25 8. Garrett, Jesse James. Ajax: A New Approach to Web December 2007. 2 April 2010 Applications. 18 February 2005. 7 March 2010 <http://www.notesondesign.net/resources/web- <http://experiencezen.com/wp- design/ajax-usability-concerns/>. content/uploads/2007/04/adaptive-path-ajax-a-new- 18. Molich, R and J. (1990) Nielsen. "Improving a approach-to-web-applications1.pdf>. human-computer dialogue." Communications of the 9. Gibson, Becky. Ajax Accessibility Overview. 1 April ACM 33. 1990. 338-348. 2006. 1 April 2010 <http://www- 03.ibm.com/able/resources/ajaxaccessibility.html>.
  • 19. Molich, R. and J. and Nielsen. "Improving a human- <http://www.sitesecuritymonitor.com/ajax- computer dialogue." Communications of the ACM application-attacks/>. 33. 1990. 338-348. 27. SPOOL, JARED M. Five Usability Challenges of 20. Nielsen, J. and R Molich. "Heuristic evaluation of Web-Based Applications. 4 December 2007. 8 March user interfaces." Proc. ACM CHI'90 Conf. Seattle, 2010 1990. 249-256. <http://www.uie.com/articles/usability_challenges_of _web_apps/>. 21. Nielsen, J. "Finding usability problems through heuristic evaluation." Proceedings ACM CHI'92 28. Tullis, Tom and Bill Albert. Measuring the User Conference. CA: Monterey, 1992. 378-380. Experience. Burlington: Morgan Kaufmann, 2008. 22. —. Usability Inspection Methods. New York: John 29. Wang, W. PowerMeeting on CommonGrounds: Web Wiley & Sons, 1994. based synchronous groupware with rich user experience. 2008. 20 March 2010 23. Osborn, A. F. Applied Imagination. New York: <http://sites.google.com/site/meetinginbrowsers/weig Scribner, 1957. ang-wang-s-work>. 24. S.Dumas, Joseph and Janice C.Redish. A Practical 30. Web Aim. What is AJAX? 1 March 2010. 6 March Guide to Usability Testing. n.d. 2010 <http://www.webaim.org/techniques/ajax/>. 25. Sarwate, Amol. Hot or Not: Ajax Vulnerabilities. 19 31. Wood, John. Usability Heuristics Explained. 18 September 2007. 28 March 2010 January 2004. 28 March 2010 <http://www.scmagazineus.com/hot-or-not-ajax- <http://iqcontent.com/publications/features/article_32 vulnerabilities/article/35698/>. />. 26. Site Security Monitor. Ajax Application Attacks. 2010. 1 March 2010
  • APPENDIX A Acronyms and Definitions Asynchronous communication - the user’s interaction with the application happens independently of the application’s communication with the server. AJAX – Asynchronous JavaScript and XML CSS – Cascading Style Sheets GUI – Graphical User Interface GWT – Google Web Tool Kit HTTP – HyperText Transfer Protocol RPC – Remote Procedure Call SQL – Structured Query Language XHTML – Extensible HyperText Markup Language XML – Extensible Mark-up Language Statements and Usability Metric Tested for Agenda and Tele-pointer Statement Usability metric used 1 I have used similar tools before. Ease of use 2 It found it easy to create an agenda item. Ease of use 3 I found it easy to create a category. Ease of use 4 It was easy to create ideas. Ease of use 5 I found it easy to delete and replace ideas. Ease of use 6 The terms used were easy to understand. Ease of use 7 The various tabs were clearly visible and easy to find. Efficiency and ease of use 8 I found it easy to move from one task to another. Ease of use 9 The tasks were clearly different from each other. Efficiency 10 I made errors while navigating through the individual tasks. Ease of use 11 It was easy to find the telepointer. Ease of use 12 Telepointer navigation from one item to another was smooth. Efficiency and satisfaction 13 I find the telepointer an important tool to use. Satisfaction 14 There is a consistent icon design scheme and stylistic treatment across the system Satisfaction
  • Statements and Usability Metric Tested For Voice Conference and Chat No. Questions Usability Metric used 1. The tool is easy to use for the tasks given. Ease of use 2. The tool is efficient for voice conference and chat. Efficiency The tool needs to be used several times to get Ease of use 3. accustomed with. In relation to other tools I have used, this tool is easy. Ease of use 4. The fields/buttons are well presented and organized. Satisfaction 5. It is easy to understand the functions of the Ease of use and satisfaction 6. fields/buttons. The tools (PowerMeeting and Skype) are well integrated. Efficiency 9. I found the navigation around the tool easy. Ease of use 10. The following questions were then asked with expected binary response of YES/NO to gather qualitative data. Did you encounter any problem while connecting with Skype? If you answer YES please briefly mention some of them. Please make any comments on Power Meeting Voice conference (Skype) tool. Would you recommend others to use this voice conference tool? Answer by (YES/NO). Would you recommend others to use Power Meeting for group chat? Answer by (YES/NO). Questions for Assessment of Documentation 1. Compared to other web applications that you use, how would you describe the registration process of the PowerMeeting? Choose one of the following options and put it in a circle. You may consider selecting more than one answer. a. It is really confusing for the average user b. Very poorly designed mechanism c. Rather straightforward 2. Did you encounter any difficulties to log on the system and create a new session? Answer by (YES/NO) 3. If your answer is NO describe in a short sentence the basic difficulty you encountered. 4. Would you prefer it if Power Meeting included a mechanism to remember passwords? (YES/NO) 5. Are you convinced of the security which is provided by the Power Meeting during the log in process? Please consider mostly the case where you need to log in with your skype id. Answer by (YES/NO). 6. On any PC, it is impossible to log in on the Power Meeting by using the same Web Browser. How would you comment on that? Answer with a short sentence. 7. How would you characterize the overall design of the user guide? Your options are the following and you should put your answer in a circle.
  • a. very bad b. neither bad/nor good c. good d. good but corrections are needed e. fascinating 8. Do you believe that the description of the sessions in the user guide was helpful to you? (YES/NO/INDIFFERENT) 9. Are you satisfied by the organization of the user guide? (YES/NO) 10. Do you believe that the content of the user guide is accurate and to the point? (YES/NO) 11. Could you manage to communicate via Skype through the Power Meeeting tool without reading the session of the user guide describing the voice conference with Skype? 12. Do you believe that you would have been able to perform better in the agenda task if the user guide had included an illustrated presentation of this function? Answer by (YES/NO). 13. Suggest any improvement in a new version of Power Meeting’s documentation. Questions For Assessing the Group calendar and Corresponding Metrics Tested No Questions On Individual Tasks Type Of Usability Metric 1 I found the tool easy to use for this task Ease of Use 2 I found the tool efficient for this task Efficiency 3 I would need to use the tool several times before I get accustomed to performing this task. Ease of Use 4 The experience I have of previous tools increased my ability and speed of performing this task Ease of Use in the Groupware calendar tool. 5 I understood the text descriptions of buttons on the user interface of the Group calendar. Ease of Use 6 The text description on the buttons aptly describe their functionality Ease of Use 7 The steps for each task followed a natural and logical order. Efficiency 8 I felt confident and very much in control of the tool while performing this task. Ease of Use 9 I made an error on this task Ease of Use 10 I found it easy to retrace my steps when I made an error while carrying out this task. Ease of Use 11 I felt I needed to check the online user documentation for this task Ease of Use
  • General Questions on the Group Calendar No User Metric Tested General Comments on Power Meeting Group Calendar For 13 I found certain features of the tool unnecessary and distracting. Ease of Use 14 I found the various functions in the Power meeting calendar tool well integrated. Ease of Use 15 I would prefer to use a browser-based groupware calendar. Ease of Use 16 I would likely use this tool frequently as my group calendar tool. Ease of use/ Satisfaction 17 I would imagine that most people would learn to use this system very quickly Ease of Use (Questions 1, 13, 14, 16 and 17 are adapted from the System Usability Scale listed by Brookes J. but developed at the Digital Equipment Corporation). 18. Please briefly describe any problems you encountered in carrying out the tasks 19. List some ways you think the calendar tool would help you work better in groups 20. What other features did you expect to see in the group calendar tool? 21. Please give recommendations on the improvement of this tool. The Group Calendar Evaluators’ Guideline and Observation Form We used this form to get our own assessment of the main user metrics we set out to test. The data was recorded for each task and for each user. The table below lists out the guidelines we followed in order to effectively record our observations. No Guidelines Possible Data Values User Metric Tested For 1 Completion time for task Value in seconds Efficiency 2 Task success Yes or No Efficiency 3 No of unsuccessful attempts Maximum Number of Efficiency Attempts was 3 4 Does user display signs of discomfort? Descriptive text of Ease of use discomfort(if any) 5 Identify User’s Errors While Carrying Type of error Issue-based Out Tasks (In a measurable form) 6 Main Usability Issues Faced By this Measured description of Issue-based User error faced by user 7 Please Record User’s Comments per Exact comments Efficiency/Ease of use/ Main Issues (They Task confirm the other observations made above). 8 Please record any other observations Miscellaneous Efficiency/Ease of use/ Main Issues (They made confirm the other observations made above).
  • APPENDIX B – Graphs and Tables Figure 1 -The general user interface layout of PowerMeeting Documenta- Voice Login tion Agenda Telepointer Conference 1 1 1 1 1 User 1 1 1 1 1 1 User 2 1 1 1 1 1 User 3 1 1 1 1 0 User 4 1 1 1 1 1 User 5 1 1 1 1 0 User 6 100% 100% 100% 100% 67% Average Confidence Interval 0% 0% 0% 0% 41% (95%) Table 1 Showing percentage of users who completed tasks Graph 1 showing percentage of users task completions
  • Evaluation Evaluation User User Time (Secs) Time (Secs) 120 66 User 1 User 1 60 90 User 2 User 2 60 50 User 3 User 3 120 60 User 4 User 4 40 18 User 5 User 5 50 22 User 6 User 6 Average 75 Average 51 Table 2 showing user completion times for login Table 5 showing user completion times for tele-pointer Evaluation Evaluation User User Time (Secs) Time (Secs) 240 180 User 1 User 1 180 180 User 2 User 2 60 120 User 3 User 3 240 360 User 4 User 4 180 180 User 5 User 5 240 180 User 6 User 6 Average 180 Average 200 Table 3 showing user completion times for Table 6 showing user completion times for voice Documentation conferencing Evaluation Group Observed Expected User Time (Secs) Novice 2 2 300 User 1 Intermediate 2 2 336 Experts 2 2 User 2 TOTAL 6 6 540 Chi-test 1 User 3 Table 7 Login Chi-test 300 User 4 Group Observed Expected 130 User 5 Novice 2 2 147 Intermediate 2 2 User 6 Experts 2 2 Average 292 TOTAL 6 6 Table 4 showing user completion times for Agenda Chi-test 1 Table 8 Documentation Chi-test
  • Relationship between level of experience and errors made(Login) Group Observed Expected 1.2 Average Errors per Day Novice 2 2 1 Intermediate 2 2 0.8 0.6 Experts 2 2 R2 = 0.5786 0.4 TOTAL 6 6 0.2 Chi-test 1 0 Table 9 Agenda Chi-test 0 2 4 6 8 10 12 14 16 Months of Experience Group Observed Expected Graph 3 Novice 2 2 Intermediate 2 2 Relationship between level of experience and errors Experts 2 2 made(Documentation) TOTAL 6 6 1.2 Average Error per Day Chi-test 1 1 Table 10 Tele-pointer Chi-test 0.8 0.6 2 R = 0.3223 0.4 Group Observed Expected 0.2 Novice 1 2 0 Intermediate 2 2 0 2 4 6 8 10 12 14 16 Months of Experience Experts 1 2 TOTAL 4 6 Graph 4 Chi-test 0.606531 Table 11 Voice conference Chi-test Relationship between level of experience and errors made(Agenda) 1.2 Tools Errors 1 Average Errors per Day Login 3 0.8 Documentation 2 0.6 R2 = 0.0357 Agenda 1 0.4 Telepointer 0 0.2 Voice 0 Conference 1 0 2 4 6 8 10 12 14 16 Table 12 Number of errors Months of Experience Voice Graph 5 Conference, 14% Telepointer, 0% Relationship between level of experience and errors made(Voice Login, 43% Conference) Agenda, 14% 1.2 Average Errors per Day 1 0.8 0.6 Documentation, R2 = 0.0357 29% 0.4 0.2 Login Documentation Agenda Telepointer Voice Conference 0 0 2 4 6 8 10 12 14 16 Graph 2Percentage of Errors in User Interface tools Months of Experience Graph 6
  • Tools High severity Medium severity Low severity Login Takes long time to log in. Powermeeting does not remember passwords Documentation User guide does not illustrate the tasks for agenda. Agenda Cannot retrieve deleted items from the trash. Telepointer Cannot understand the use of tele-pointer. Voice Conference Takes too long to connect to Skype. Table 13 Issues encountered with the User Interface tools %participants encountered issues 120.00% 100.00% %participants 80.00% 60.00% 40.00% 20.00% 0.00% Login Agenda Voice Conference High severity Medium severity Low severity Graph 7 Percentage of participants who encountered issues Login Documentation Agenda Telepointer Voice Conference P1 Delays while logging in when Can’t Delays while sending there are too many understand text messages. participants logged in. how to highlight fields. P2 Should be a System freezes when search function there are too many to find relevant people logged in documents. PowerMeeting. P3 Would prefer PowerMeeting Takes too much time to remember passwords. to connect. P4 More security consideration and privilege rights assigned to certain participants P5 The logout process should be made easy perhaps making the logout button more prominent. P6 The administrator and client A bit more Should allow more login should be explained information on than three how to create participants to do the agenda. conference. Table 14 comments on the user interface tools
  • Task 1 Task 2 Task 3 Task 4 Task 5 User 1 1 1 1 1 1 User 2 1 1 0 0 0 User 3 1 1 1 1 1 User 4 1 1 1 1 1 User 5 1 1 0 1 1 User 6 1 1 0 1 1 Average 100% 100% 50% 83% 83% Confidence Interval 0% 0% 44% 33% 33% (95%) Table 15 showing success rate for calendar tasks Graph 8 showing task success with confidence intervals Evaluation Usage of Experience User Time (Secs) calendar tool level User 1 Daily 10 105 User 2 Twice a week 8 186 User 3 Weekly 6 180 User 4 Once in two 4 120 weeks User 5 150 Monthly 2 User 6 126 Never 0 Average 144.5 Table 17 showing experience levels Table 16 showing task completion times for each user
  • Task High severity Medium Low severity ID severity Task 1 Difficulty adding a Cannot modify The use of the new event (it is not event end time term ‘Event’ to clear that the refer to an calendar date should activity was not be clicked to add new clear (event was event and no mistaken for an alternative way to agenda item) carry out task is available) Task 2 Event Graph 9 showing the relationship between the completion time and description can general computer usage experience only be modified by Task 1 Task 2 Task 3 Task 4 Task 5 double clicking to bring up the User 1 43% 35% 43% 48% 33% dialogue box. It User 2 73% 73% 53% 63% 75% is not explicit that user User 3 70% 70% 60% 70% 70% should double click event to User 4 73% 73% 68% 75% 75% see the User 5 53% 63% 53% 73% 70% description User 6 58% 68% 53% 68% 68% Task 3 Event time can only be changed by Average 61% 63% 55% 66% 65% dragging and Table 18 showing Data from Calendar Tool Ease of Use dropping event to a Evaluation different time slot. Alternative way to change the time is through the dialogue box but this cannot be done as the text- box which contains the time is not editable Task 4 Cannot change event date from the daily and monthly views Task 5 Cannot delete an event without double clicking Graph 10 showing percentage ease of use of calendar tool the event. No alternative way of deleting event is provided Table 19 showing issues identified during usability evaluation of Calendar tool
  • Graph 11 showing percentage of users who encountered issues User How can the calendar tool What other features did you Other comments help you work better in expect to see in the group groups? calendar tool? User 1 Group deadlines would be To do list easier to clarify if each person was set an individual deadlines Show you which group member entered an event User 2 Allow members of a group Uploading documents share work schedules User 3 Saves cost : if you are away Priority feature will help and need to meet up with your organize events better group members Saves time User 4 Other users’ events should be Progress of event can be shown differently on my page incorporated User 5 Deadline and meetings can be The time in the dialogue box The ease of use of the easily communicated should be made editable like tool exceeded my the description expectation User 6 Allow access to uniform The textbox displaying time schedule of group members should either be made editable or changes to display the time as a label Table 20 summary of further details gathered during usability evaluation of Calendar tool