Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Sample Website Usability Analysis


Published on

Published in: Education

Sample Website Usability Analysis

  2. 2. INTRODUCTION Bowling Green State University offers an online degree completion program that is designed for working adults who wish to complete a Bachelor of Science degree in Technology. The Advanced Technological Education (ATE) program consists of nine core courses that teach students how to integrate technological concepts into their working environments. Students can choose between two courses of study: Technology in Training and Development or Information Studies. All of the courses are online offering convenience and flexibility for working students. An applied associate degree is required for the program, as well as work experience. Prospects wishing to pursue a degree online will tend to look for information online. It is critical that the ATE website be enticing, informational and usable. Therefore, a usability test was conducted on the ATE website to determine if prospects and existing students are able to easily retrieve key information. In addition, the process in which prospects contact ATE personnel to receive additional information was also analyzed, as it is an integral component of the website and recruiting process. In order to encompass all components of the website and the information process, this usability analysis consists of the following sections: • Pre-analysis • Methodology • Results • Conclusions and Recommendations
  3. 3. PRE-ANALYSIS In order to develop productive test questions, it was important to determine relevant facts about the ATE program. For instance, what do prospects and students want and need to know? Who is the target audience? What types of tasks do they want to perform from the website? What are the objectives of the website and key points it should contain? What is the purpose of the website? Currently, the purpose of the site is to inform prospective students about the ATE program. A list of questions usually asked by prospects was compiled. Answers to the following frequently asked questions should be found easily and quickly: • How long will it take me to finish? • What is required to be eligible for the program? • Do I need an associate degree? • How much does it cost? • What can I do with an ATE degree? • What does ATE stand for? • What will I learn in the ATE program? • What courses will I be required to take? • Can I take courses elsewhere and transfer them to the ATE program? • Will my degree say “online?” • When do classes begin? • What if my employer pays my tuition? The four questions below were found to be questions asked not of prospects, but of those that have already applied. Therefore, it was discovered that the purpose of the website needs to expand to providing information to those that have just applied as well. • When will I know I’ve been accepted into the ATE program? • I’ve applied – what happens next? • I don’t think a course transferred correctly. What do I do? • What is a transfer evaluation vs. an official audit? Who is the audience? The website can be accessed by both prospects and current students. However, for the most part, the website is currently designed for future students. Is it not normally used as a tool for students already
  4. 4. in the program. However, for the purposes of the usability test, the future and current student audiences are defined. Future Students Users will be most likely between the ages of 25 and 60. Computer skills will vary from beginner to expert. Most will have some college experience. Connection speed will vary from dial up to high speed. Current Students Users will most likely be between the ages of 25 and 60. Computer skills will vary from beginner to expert. All will have some college experience. Connection speed will be high speed. What tasks need to be performed from the website? Students and prospects viewing the ATE website will need to have access to the following links: • Financial aid • Course Descriptions • All online courses available • Course Schedules • Online application • Tuition deferral program • Ability to request more information • Access to a sample online classroom • Video/Audio of ATE explanation Information and features that should be present: • Explanation of the enrollment process including time involved • Application instructions • Curriculum • Course Descriptions • Class Schedule • Faculty Information What are the measurable usability objectives? In order to create test questions that would provide meaningful information, it was important to define the following measurable objectives: • Were all of the users’ questions answered? • Was the user able to request more information easily?
  5. 5. • Was the user able to quickly identify the correct links to answer their immediate questions? • Was the user able to easily print the information they wanted to save? • Did the user have an emotional reaction to the site? Was the user excited about starting the ATE program or was he/she discouraged, confused or indifferent? What are the key points? The following key points are important components and should be included in the website: • Course Descriptions • Costs • Completion Time • Current Promotions • Accreditation METHODOLGY Participants Five testers were selected for this usability test. Three out of the five were female and all five were between the ages of 40 and 45 which is also the approximate average age of most ATE prospects. All use the internet and email to some degree for work and/or pleasure. Hours of usage vary from 10 hours to 40 plus hours per week. Test Questions The following information, directions and questions were provided to each participant. Usability Test Instructions and Questions I would like you to view a website for me and answer a few questions. The process will probably take approximately 15 minutes. • Please know that I am not testing you, I am testing the site. You can’t make any mistakes during the test. • I want you to be completely honest – you won’t hurt my feelings. The goal is to make the site better and easier to use.
  6. 6. • As you think about each page and decide where to click, please do me a favor and include any thoughts you have in your answers so I can understand your thought processes during the test. Before you look at the website, I have a few questions for you, OK? 1. What is your occupation? 2. How many hours a week do you spend using the internet, including email? 3. What types of tasks do you use the internet for? What kinds of sites do you like to visit? 4. What is your age? 18 to 25, 26 to 39, 40 to 59, 60 to 74 Now its time to open the website. Go to 5. Have you ever seen this website before? a. If so, why and when? 6. Just from looking at this site, who do you think the site is designed for? b. Why do you think that? 7. By glancing at the website but without clicking on anything, can you tell me 4 types of information the site is intended to provide? 8. Click on a link you think you would click on first if you were accessing this website for the first time at home. Why did you choose that link? 9. By just glancing at this page, what information is provided? Scenario 1 You want to know the eligibility requirements for the ATE program. You’ve heard an associate’s degree is required, but you don’t have one. You want to know if you can still apply. Can you find the answer to your question? Not completed Completed with difficulty or help Easily completed Scenario 2 You would like to know how much it will cost to complete the program. Can you find the answer to your question? Not completed Completed with difficulty or help Easily completed
  7. 7. Scenario 3 You have an associate degree but want to know how many courses will transfer to the program. Can you find the answer to your question? Not completed Completed with difficulty or help Easily completed Scenario 4 You want to know what courses you are required to take as a part of the ATE program. Can you find the answer to your question? Not completed Completed with difficulty or help Easily completed Scenario 5 You’ve heard some employers don’t hold online degrees in as high esteem as other degrees. However, BGSU is a reputable and highly regarded by employers. You want to know if your degree will indicate that you attended online or if it is the same type of degree you would receive if you attended on campus. You also would like to know ATE’s accreditation. Can you find the answers to your questions? Not completed Completed with difficulty or help Easily completed Scenario 6 You want more information about the program and would like more information. Can you request more information? Not completed Completed with difficulty or help Easily completed Complete the request for more information form on the website 10. Did you feel uncomfortable about providing any of the information asked of you on the request for information form? Open the email sent to you in response to your request for more information 11. Do you feel you’ll read the entire email? 12. What information are you inclined to look at first? 13. Is the information too much information, not enough information, or just enough? 14. Is the information organized and easy to read? 15. Are you confused by anything indicated in this information? 16. Would you open each and every document?
  8. 8. Open the documents you would open in a real situation 17. Can you now answer any of the scenario questions you couldn’t answer before? 18. What is your overall impression to the ATE website and request for information process? 19. What did you like best? 20. What did you like least? 21. What is the first thing you would do to improve the site? 22. Do you have any other comments and/or questions? Rate the website and the request for information process: Strongly Strongly disagree agree 1. Overall I found the information I needed. 1 2 3 4 5 2. I found the information unnecessarily complex 1 2 3 4 5 3. I thought the information was easy 1 2 3 4 5 to use 1 2 3 4 5 4. I found the various links in the website were well integrated 1 2 3 4 5 5. I thought there was too much inconsistency in this website 1 2 3 4 5 6. I would imagine that most people would learn to navigate this website quickly 1 2 3 4 5 7. I found navigating the website very 1 2 3 4 5 cumbersome 8. I felt very confident using the 1 2 3 4 5 Website and the request for information 1 2 3 4 5
  9. 9. Test Procedures The ATE program is online, so there is a presumption that individuals requesting information about the program have access to the internet and email. Therefore, all informative documents about the ATE program are emailed to prospects. Statistics show that a majority of prospects find the ATE website and then complete the Request for Information Form. In order to simulate this procedure as accurately as possible, testers were emailed the test instructions. They were asked to complete the tasks and answer the questions without a test administrator present. Part of the test was to utilize the “Request for Information” feature on the ATE website, and then read an email with 5 attached documents sent to them in response. Participants were asked questions about both the website and the emailed information. Testers recorded their answers electronically and submitted their results via email.
  10. 10. RESULTS Test Answers
  11. 11. Tester Information Tester 1 Tester 2 Tester 3 Tester 4 Tester 5 Secretary/Medical Recruiting Coordinator & 1 Occupation IT/Contractor Transcriber/Student Secretary Administrative Secretary Database Administrator How many hours a week do you spend using the internet, 2 including email? 10 hours 10-15 hours A lot 20-40 hours a week 40+ hours Email at work, internet for work procedures. I rarely surf for pleasure but frequently visit What types of tasks do my bank site and sometimes will view you use the internet Banking, Investments, Google calendar, weather, or diet and exercise sites. I have a for? What kinds of sites Shopping, Email, Email, online classes, news, paycheck, bursar bill, business website that I check and sometimes Google, databases, email, 3 do you like to visit? News shopping book orders research 4 What is your age? 40 to 59 40 to 59 40-59 40-59 40-59 Website Information - Have you ever seen this I thought I had been to this website but it 5 website before? no No No looks different than I remember. no At the very first glance, I would think the site is for any BGSU student. It references the degree name and that it is a bachelor of 20-somethings who science. Of course, knowing about the didn't go to a 4-year program, I know it is for adult students and it Just from looking at this school. It never says To get info about a degree in is an online program, but you wouldn't know site, who do you think Designed for adults anywhere it's for adult Tech Education. I think that that unless you read the first three bulleted Students wanting to earn a the site is designed for? that want a four year learners or non- because it says "Bachelor of (sort of) lines under the description. I t think Bachelor of Science in Advanced 6 Why do you think that? degree traditional students Technological Education." that is because of the template Tech Education By glancing at the website but without Four things I should be able to get are clicking on anything, information about the program, the specific Phone number, email address, can you tell me 4 types Contact information if curriculum to finish the degree, info about qualifications: need an associate of information the site they are interested in the faculty and contact information to sign degree & work experience, can 7 is intended to provide? learning more Not really Contact info up. take classes online It is a degree from BGSU info about the degree You can find more details by the links on the left It is an online degree (although that isn't terribly obvious) Click on a link you think you would click on first if you were accessing About the Program - I clicked on About the I clicked on About the Program this website for the first because there isn't a Program because I because I know nothing about About the Program - I would choose it About the program - want to time at home. Why did lot of information on wanted to figure out the program and figured that because I would want to know what the learn more about it, plus first link 11 you choose that link? the home page. what it was about. was a good starting point. program is about and if it would interest me. in list
  12. 12. Results Analysis • None of the participants had seen the ATE website before completing the test. • At first glance, the participants were asked what the website was designed for. The testers all indicated the website provided information about a bachelor degree. When asked to identify four types of information provided on the website’s home page, one of the five reported what they should be able to find – not what they did find. Another tester said they could not identify four types of information. The remaining three all agreed they could locate contact information and information about the degree. Two participants said they identified the program was online, though one reported that it was not necessarily obvious. • All participants agreed they would be inclined to click on the “About the Program” link first. • One participant did not read the information provided on the “About the Program” page. All remaining participants indicated that the “About the Program” page provided more information, who the program is designed for, and how it works. One participant appeared to be a bit confused as this was the first time “degree completion” was mentioned, indicating that it should have been included on the home page. • When asked if the testers could find out if they could apply without an associate degree, one of the four said they could easily complete that task. The remaining four testers could not find this information. It is unclear how one tester found this information as it is not provided on any of the ATE website pages. • When asked if the participants could locate costs associated with the program, only one reported finding the information but with difficulty or help. All others could not locate cost information. • When asked to locate information about transferring courses to BGSU, the answers varied. Two completed the task with difficulty or help, one could not complete the task, and one easily completed the task. One tester experienced an error on the transfer credit page. • All participants agreed they could easily locate the courses required to complete the ATE program.
  13. 13. • When asked to find out what type of degree and accreditation the program held, one participant reported they completed the task with difficulty or help, three easily completed the task and one reported they found the information on the “Recent Updates” page and it should be placed on the “About the Program” page as well. • All five participants easily requested more information about the program. One tester suggested being able to ask specific questions on the Request for Information form and felt the contact information and the request for information pages should be combined. • All five testers felt comfortable providing information asked of them on the Request for Information form. • One participant felt they would only read the first few paragraphs of the email sent to them in response to their Request for Information. The remaining testers said they would read the entire email. One tester felt a lot of the information in the email duplicated the website information. • When asked which information in the email they were inclined to look at first, one reported they would like to see course compatibility, though that information is not currently included in the email. Another tester reported they would click on the link to the web page if they hadn’t already viewed it. Another said they would first read about what the program is and then view the curriculum. The remaining participant said they would read the email first. • What asked if the email provided too much information, not enough information or just enough, two participants felt it was just the right amount of information. One participant remarked that the highlighted and bulleted information was helpful. One participant reported it was not enough information and more transfer information needed to be provided. Two remarked that the letter repeats too much information from the web page. The remaining tester reported there is never too much information. • All agreed the information was organized and easy to read. One participant remarked they did not understand what some of the information meant, however.
  14. 14. • When asked if any of the information confused them, three participants reported they were not confused. One reported they were confused, and the remaining tester indicated the requirements should be better explained. • When asked if they would open every document attached to the email, three participants reported they would and two reported they would not. • Only one out of the five participants said they found answers to earlier test questions they could not answer after reading the email and opening an attachment. • The overall impression of the website was positive. The available links, contact information, the appearance and navigation capabilities were reported as the best features of the website. The small font size, the look of the Recent Updates and the Request for Information form, the number of attachments on the email, and the ability to understand the information were the least liked features of the website and request for information process. • Suggested improvements were: o Make the heading smaller and provide more information on the home page. o Provide more links to transfer information o Change the color of the Request for Information form and add a text box for specific questions o Increase the font size Conclusions/Recommendations Conclusions The ATE website and request for information process scored in the middle range overall. The average rating for positive features ranged between 3.25 and 4, with 1 being the lowest score and 5 the highest. However, the pre-analysis and testing processes revealed information that is missing and areas of the website that are in need of improvement.
  15. 15. Website Information The website appears to be informative in a limited capacity. The way in which the information is presented could be improved. For instance, key information should be accessible at the top of the page where users are most likely to focus their attention. This should include that the program is online, is for adults, is a degree completion program, and is accredited. In addition, the following information is missing from the website: • Financial Aid (although this is included in the emailed information) • Costs • Available online courses • Class Schedule • Online Application • Tuition Deferral Program • Tutorial • Enrollment process (although this is included in the emailed information) • Completion Time • Accreditation Agency • Current Promotions • Options for individuals without an associate degree • Detailed transfer information • Video/Audio files verbally describing the program • FAQ’s Overall, there was not the impression from the test results that the website generated an emotional reaction. If anything, one user’s answers indicated frustration; though it appears that could be from a previous bad experience. The Request for Information page did not seem to intimidate the users so it’s possible to conclude that the information requested from the prospect is not perceived as being invasive. For those using smaller computer screens, the graphical heading at the top is too large. In addition, some felt the font was too small and was bothersome.
  16. 16. The website does appear to navigate well. Users were inclined to select the “About the Program” link from the home page which would be the logical choice. In addition, the courses required for the program were easily obtained by all participants. Emailed Information It appears most testers read the entire email or at least skimmed the information. It was not apparent which attached emailed document was opened first. The question pertaining to this subject should have been worded differently in order to determine which document was of most interest to the tester. The amount of emailed information did not appear to overwhelm a majority of the testers, which was an initial concern. Most found the email easy to read and navigate, though it may be necessary to provide clearer explanations. Recommendations Based on the pre-analysis information, the test results, and the conclusions the following changes to the website and emailed information are recommended: • The home page should provide links to: “Future Students”, “Current Students”, and “I’ve just applied. What Next?" Each will direct the user to a page which has information specifically relevant to their situation. • Links that need to be added to the website in their appropriate category(s) are:  ATE Community  Class Schedule  MyBGSU  Online Classes Available  FAQ (different questions for future, current, or just applied)  Application Instructions  Tuition Deferral Application  ATE Checksheet  Video/Audio describing ATE  Online tutorial  Online Application • The website needs to include a printer-friendly version for all pertinent information and documents.
  17. 17. • The following information should be at the top of the home page where users will quickly identify it:  The program is online  The program is for adults  ATE is a Bachelor Degree Completion Program  How to retrieve more information/contact information  FAQ • One of the FAQ should be “What if I don’t have an Associate Degree?” A page should be dedicated to information specific to those individuals. • A text box should be added to the Request for Information form allowing users to enter specific questions. A link to the Future Students FAQ would be helpful from this page. • The email attachments should be converted to websites and links should be provided to the websites as opposed to attachments.