Final initial feedback commissioner usability study.docx


Published on

Published in: Design, Sports, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Final initial feedback commissioner usability study.docx

  1. 1. Fantasy Football Commissioner – Usability Study November 2008 Initial Feedback Overview: A total of 10 participants took part in a usability study of the Fantasy Football Commissioner product on November 18-19, 2008. Approximately half of the participants were experienced Fantasy players (2+ years using a Fantasy product of some sort) the other half had little or no experience but were interested in Fantasy sports. The goal of this study was to examine the user experience of visiting, finding the Fantasy Football product offerings, creating a Commissioner League, and then setting up and managing teams.Usability Rating # Top Level Observations and Findings 1. Registration: Participants regarded general site Registration as simple and straightforward – standard. Note: One of the main errors encountered during registration was choosing a site ID – one that was not already used by another user. Providing a “Check Availability” option would be helpful. Also, field by field error checking could be beneficial. Error prompting on page reloads (the action invoked by the user after completing the registration form) received some negative reaction from participants. 2. Home Page Clutter: A general theme observed during usability sessions was that the Home page was cluttered - especially for a first time visitor. It takes mental and visual effort to absorb the Home page and determine the lay of the land. Users have a point of comparison about Home page layout and given the exposure to other Sports web sites, it is important to examine how to improve the first time visit on in the future. Comments from two of the participants: Question: Your first impression? Answer: “It’s pretty cluttered… there’s a lot of things going on at the same time.” … “But [the site] is comprehensive about all the sports”. Question: Your first impression? Answer: “There’s a lot going on … little pictures that, a lot of text, giant ads, like… I’m trying to decipher what is an ad and what is not… for the first time there’s a lot on the page” … “hard to tell content from advertising.” 3. Fantasy Products Realization on Home Page: Participants quickly ascertained that CBSSports offers Fantasy products. The button labeled “Fantasy” near the top left of the Home page was frequently the first navigation option taken by participants. Large product ads were also noticed. Nonetheless, some participants mistakenly thought that the Fantasy News section was the main Page 1
  2. 2. Fantasy Football Commissioner – Usability Study November 2008 area for Fantasy products. In terms of organization, Fantasy, News, Products, etc. do not have a consistent home base and adds to the mental labor a user must perform in order to distinguish page organization. The page fantasy home page above ( has the same labels (e.g., Fantasy Football) to describe Fantasy News and Fantasy Products. The centered, vertical menu for Fantasy product is counter-intuitive Some ideas: Move vertical, centered menu to the far left Perhaps, label items in menus differently and specifically (e.g., ff news vs ff products) or integrate the menus so there would be one menu for fantasy assets. Page 2
  3. 3. Fantasy Football Commissioner – Usability Study November 2008 4. Commissioner Robustness vs. Price: The Commissioner product was seen by participants as having a lot of features and flexibility. Participants liked the mobile component and live scoring (something noted as distinguishing features for CBSSports in comparison to other products). However few, if any participants, deemed these features worth the price given what they can do for Free elsewhere with other products. True product differentiators should probably be strongly incorporated (in some fashion) into the Creative for the Commissioner product. It’s not easy to see the differences between products in the ff products above. For example, it is hard to tell the differences between the top three ff products, except 14-day free trial for ff commissioner vs free for ff free. For commissioner product, the current setup is to click on a product to get more details of the product (can’t see the price—used to be ~ $149), which then have link to free trial. Once in the detailed page, if viewers want more full details of the product, they need to look around (would find under features and new features in resources section) Issues: 1. Hard to differentiate f products 2. $149.95 is viewed as a very high price for ff commissioner 3. Not easy to get to learn all features of ff commissioner. Some effort to find its full feature list. 14-day trial period may be too short Some ideas: 1. Provide succinct descriptions to differentiate ff products in image above. Make the product you want to promote (commissioner) more salient. Include a link to take viewers to product side-by-side comparison table that allows them to see key differences between products. 2. In the product landing page ( source=MKTG_FFBC08_FSPOETH&refcode=FFBC08PP_FSPOE_TH), provide a link to a page that show Page 3
  4. 4. Fantasy Football Commissioner – Usability Study November 2008 all features of the product instead of having viewers search around in Fantasy Resource section. This link may be placed before “Start a Trial Now” button 3. For interested viewers who are new to ff commissioner, they will need to try out the product if they want to know it full potential. Yet, 14-day trial period may be too short for a football season. May consider lengthening it to 3-4 weeks. 4. $149.95 price is viewed as very high. May need to consider alternative pricing scheme. Perhaps a small fixed price per owner instead of a single charge to commissioner. The current price will probably be the biggest hurdle to getting new users. Setup page: League Rules vs Scoring System Page Page 4
  5. 5. Fantasy Football Commissioner – Usability Study November 2008 The Leagues Rules and Scoring System pages contain many different settings. Yet, they are organized differently. Leagues Rule partition items under different tabs, whereas Scoring System (has a lot more info than being shown above) is divided under different headings. Not consistent way of organizing info. It could be better if they are organized consistently. The tab organization in League Rules is preferred by viewers. Would be of minor importance as users can still work with both. Page 5
  6. 6. Fantasy Football Commissioner – Usability Study November 2008 Submit Payment In Submit Payment, it is not clear what function “Security Question”, “Security Answer”, and “Password” serve. Perhaps, they are all required to validate accounts, but there is no instruction to inform users. Why three? They introduce more obstacles for us to get $. The goal should be to make it very easy to user to submit payment. Should require as little inputs as possible. Those fields should not be there, if possible. Page 6
  7. 7. Fantasy Football Commissioner – Usability Study November 2008 5. Navigation on SPOE page: When asked to go to specific Fantasy products on the SPOE page, some participants mistakenly clicked on the hyperlinks above the main product selection area thinking that this was the navigation path to Fantasy products (e.g., Fantasy Football). Participants were surprised that it was a Fantasy News area for the selected sport. Competing horizontal navigation bars play a part in this confusion. Some participants did not immediately understand that navigation to the different Fantasy sports could be accomplished by clicking on any sport just to the right of the main splash image. Once pointed out, it was deemed simple. It was a case of once experienced it was a learned behavior. 6. Multiple Horizontal Navigation: The dual and triple horizontal navigation bars interfere with usability. Multiple horizontal navigation options compete for attention and potentially confuse or slow down users as they carry out tasks. There is some redundancy in the navigation as well (Add/Drop) and other actions that can effectively be provided in one feature navigation area. Better organization of action buttons and navigation is warranted. Page 7
  8. 8. Fantasy Football Commissioner – Usability Study November 2008 7. Add/Drop Players: While it was easy for participants to find the link to Add/Drop players, manipulation of the fields to actually add or remove a player presented a problem. The labeling and orientation of buttons/links confused some participants. The “OK” button carries out the operation for both “Add” and “Drop” yet not all participants realized this to be the case. The “clear” hyperlink adds to the confusion. Several participants thought this was the way to invoke adding or dropping players and that the “OK” button merely accepted the actions taken on the page. Page 8
  9. 9. Fantasy Football Commissioner – Usability Study November 2008 While not discussed by participants, the “Add Unlisted Player” and “Edit Service Time” features lower on the Add/Drop page seem curiously color coded and positioned. Why are these links red – a color choice normally reserved for error conditions. And, given that the links are below the “OK” button, they largely go unnoticed despite their coloring. Player pool is a long list of names, many of which may not mean much to viewers in terms of their past performance. May want to provide some stats on each viewer when viewer mouse over the player, so that viewers can quickly see association between players and their performances. Same feature may be considered for swap player section as well. Page 9
  10. 10. Fantasy Football Commissioner – Usability Study November 2008 8. Initial Fantasy Team Page: Once logged into the site and a Fantasy team selected, the page presented was not expected by all participants. When clicking on a Fantasy team, there was an expectation to see the “My Team” page rather than the general Fantasy News page. Page participant expected Page participant got 9. Help Text and Hints The Help text and Hints (light bulbs) provided on most screens were considered to be very helpful by all participants. During most tasks, participants read and seemed to rely on the hints in order to proceed or to confirm assumptions about the information or action they were about to take on the screen. 10. Ease of Setup vs. Competition: In terms of setting up a Fantasy League, CBSSports may exhibit an edge over the competition in terms of ease of use. One participant specifically mentioned that the Commissioner product was much easier to setup in comparison to EA Sports and Yahoo. Much of the success was due to helpful hints along the way. Page 10