CBS Sports Analytics, Usability, and Research Roadmap


Published on

Published in: Design, Sports, Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

CBS Sports Analytics, Usability, and Research Roadmap

  1. 1. Analytics, Usability & Research Research & Analytics
  2. 2. Y/Y and W/W growth across all metrics Y/Y Y/ Week 1 Growth Week 2 GrowthAvg. Daily Unique Visitors X,xxx,xxx 9% X,xxx,xxx 13%Visits Xx,xxx,xxx 4% Xx,xxx,xxx 9%Page Views Xxx, ,xxx,xxx 11% Xxx,xxx,xxx 17%Total Mins Xxx, ,xxx,xxx 12% Xxx,xxx,xxx 18%Bounce % 21% -5% 20% -9% Minutes/Visit 19.6 7% 20.1 6% Page Views/Visit 9.9 7% 10.3 7% Search engine referrals +55% Y/Y Referrals from other websites (non (Google +78% Y/Y). During Week 1 search) increased 63% Y/Y. ‘search’ accounted for 12.2% of all, yahoo,, AOL and referrals (9.9% last year) round up the top 5 Source: Omniture
  3. 3. Double-digit growth through 2 weeks Current Year Last Year Week 1 Sunday Week 2 Sunday Season to Date Y/Y Growth 4,000,000 Daily UVsAvg. Daily 3,000,000 11% Daily UVsUnique Visitors X,xxx,xxx 2,000,000 1,000,000 7,000,000 6,000,000 5,000,000 VisitsVisits X,xxx,xxx 6% 4,000,000 Visits 3,000,000 2,000,000 1,000,000 150,000,000 Total Minutes Total Minutes 100,000,000Minutes X,xxx,xxx 15% 50,000,000 0 30.0 Minutes/ VisitMinutes/ Visit XX.X 19.9 6% Daily UVs 20.0 Pages/Visit lack ‘Sunday spike’ evident in other metrics. AJAX-like 10.0 technology refreshes scoreboard, but not the page. Thus, traffic spikes on weekdays as fantasy players prepare for the coming week . 15.0 Pages/Visits Pages/ VisitPages/ Visit 10.1 XX.X 7% 10.0 5.0 Source: Omniture
  5. 5. Fantasy Football Today explodes*• Live, streaming fantasy football show Week 1 Week 2 W/W Growth featuring expert advice & analysis Streams 2x,xxx Xxx,xxx 2527%• Auto-played Minutes 2xx,xxx 1,xxx,xxx 362%• Streams from 11:00am – 12:30pm, EST each Minutes/Stream XX.X X.X -82% Sunday on FFT Microsite & Commissioner league home pagesFFT Microsite Commissioner League Home • Few streams (new show, low traffic • Not available in Week 1 page, developing audience) • Lots of streams (high traffic page) • High engagement. Users come to watch • Low engagement (users come to set video; they watch longer lineup, not watch video) • Possible bad user experience (forced to turn off video if not interested) Source: Akamai
  6. 6. FFT: the numbersContent Starts 9/7/2008 9/14/2008 W/W Growth The addition of the FantasyFFT Main Page Xx,xxx Xx,xxx 32% Football Today video onFFT League Home 0 Xxx,xxx -FFT Total Xx,xxx Xxx,xxx 2527% league home pages increased streams to Xxx,xxx in week 2Minutes 9/7/2008 9/14/2008 W/W Growth (+2527%)FFT Main Page Xxx,xxx Xxx,xxx -17%FFT League Home 0 Xxx,xxx -FFT Total Xxx,xxx X,xxx,xxx 362% Streaming minutes alsoEstimated Viewers 9/7/2008 9/14/2008 W/W Growth increased to over 1MM inFFT Main Page Xx,xxx Xx,xxx 24% week 2 (+362%)FFT League Home 0 Xxx,xxx -FFT UV Total * Xx,xxx Xxx,xxx 2172% Minutes perMins/Content Start 9/7/2008 9/14/2008 W/W GrowthFFT Main Page XX.X X.X -37% stream, however, decreased -FFT League Home - X.X - 82% W/W. This is likely theFFT Total XX.X X.X -82% result of users coming to their leagues to set their*UV total aggregates viewers from the FFT main page & the League HomePage (i.e. assumes no duplication) lineups, not watch video Source: Akamai
  7. 7. MOBILE
  8. 8. Slow start gives way to strong week 2Mobile Week 1 Y/Y Growth Week 2 Y/Y GrowthAvg. Daily Unique Visitors Xx,xxx -15% Xx,xxx 22%Visits Xxx,xxx 5% Xxx,xxx 13%Page Views X,xxx,xxx 0% X,xxx,xxx 35%Total Mins X,xxx,xxx N/A X,xxx,xxx N/A Minutes/Visit XX.X N/A XX.X N/A Page Views/Visit XX.X -5% XX.X 19% Minutes per visit Avg. daily unique visitors Though flat week 1, increased from XX.X in decreased 15% Y/Y in page views during week week 1 to XX.X in week week 1, but rebounded two reached X.X MM 2. Minutes per visit on +22% Y/Y in week 2 (+35% Y/Y) Sundays were nearly 2X that of other days Source: Omniture
  9. 9. 08/31/08 09/01/08 09/02/08TV 09/03/08 09/04/08 Thursday 09/05/08 Season Opener 09/06/08 09/07/08Mobile Week 1 09/08/08 Sunday, Monday 09/09/08 09/10/08 Football forges cross platform synergy 09/11/08 09/12/08Internet 09/13/08 09/14/08 Week 2 09/15/08 Sunday, Monday
  11. 11. Outline of Contents Purpose of the study Participants Methodology Observations Usability survey results Next steps 11
  12. 12. Purpose of the study…• To establish a usability and user experience baseline of• To establish a foundation for a method of continued observation and evaluation• To discover strengths and weaknesses of the current website• To provide recommendations for design alternatives 12
  13. 13. Outline of Contents Purpose of the study Participants Methodology Observations Usability survey results Next steps 13
  14. 14. Participants• 7 participants (all male) were recruited by CBS Television City, NV based on their interest in Sports and Fantasy products and membership.• 4 Fantasy users and 3 non-Fantasy users were recruited.• Fantasy memberships on sports sites included:  Yahoo (3 participants)  CBS (2 participants)  ESPN (1 participant)  Fox (1 participant)• All participants were comfortable and competent using the Internet.• Average age: 31 (min=21, max=43)• Education: Post Graduate (2); College (2); Some College (1), High School (1) 14
  15. 15. Outline of Contents Purpose of the study Participants Methodology Observations Usability survey results Next steps 15
  16. 16. Methodology• Participants were greeted by a moderator and given a verbal overview of the study.• To begin, each participant was given an opportunity to freely explore and interact with the site – while doing so they were asked to talk aloud about their thoughts and reactions.• Next, participants were directed to complete a set of tasks, for example:  Choose and read a story  Choose and play a video  Interact with the video player controls  Do a Search for a topic of interest  Get Sports information (team, schedule, scores, player stats)  Complete registration  Engage Community features• At the end of the session, a usability survey was given to each participant asking them to rate certain design statements about 16
  17. 17. Outline of Contents Purpose of the study Participants Methodology Observations Usability survey results Next steps 17
  18. 18. Observations – Key Takeaways• Participants who are Fantasy oriented (be it on CBS or somewhere else) rated the site much higher than non-Fantasy oriented participants.• Non-Fantasy participants tended to be negatively impacted by the amount of Fantasy information on the site. These participants were simply interested in getting general information about their team or sport – especially true on the Video page.• Video content (main thumbnail and page links) were not readily apparent to all participants. The Presence of a consistent pre-roll ad clearly a buzz kill but accepted as standard practice.• Player controls responded as expected and were generally very easy to use. 18
  19. 19. Observations – Key Takeaways, cont…• Participants liked the prominent stack of headlines on Home page.• Main story thumbnail navigation confused some participants.• Most participant assumed that clicking on the big picture would open the full story – not so. Others thought the story would open simply on one click.• Some confusion occurred when a participant clicked on the small thumbnail and the big picture did not match. “Did I click on the right picture?”• The link for the full story is hidden in the text description – hard to find/see. 19
  20. 20. Observations – Key Takeaways, cont…• Basic registration was considered straightforward and typical.• But, after registering; participants often stumbled through the process for: – Setting up and using My Quicklinks – Customizing the site to highlight Your teams and players• The presence of Community was readily apparent to participants.• Orange comment callouts were understood but participants thought they should be clickable.• Navigating to Community aspects was relatively easy. 20
  21. 21. Observations – Key Takeaways, cont…• Search may be an area for improvement as you will see by the usability survey ratings that follow.• Depending on the query entered, Search returned credible results.• A participant’s initial keyword search often under-delivered; however, subsequent searches were much better - trial and error learning.• Non-Fantasy participants were somewhat perplexed (sometimes put off) by the amount of Fantasy results returned by a query. 21
  22. 22. Outline of Contents Purpose of the study Participants Methodology Observations Usability survey results Next steps 22
  23. 23. Usability Survey Results Survey Ratings: Fantasy vs. Non-Fantasy ParticipantsCompletely 7 Fantasy and non-Fantasy participants Agree rated very differently! 6 On average, non-Fantasy ratings across all survey metrics are 1.5 points 5 lower compared to Fantasy ratings. Average Rating Good News: Visitors that are Fantasy 4 oriented are served fairly well. 3 Bad News: Visitors to the site looking for general information (not Fantasy) are not being served nearly as well. 2Completely Non-Fantasy participants Disagree 1 rated the apparentness of Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 video, navigation, and the Fantasy 5.5 6.8 5.5 6.3 4.5 6.0 6.5 6.8 6.5 6.0 5.5 5.3 5.0 6.3 organization of the site the Overall 4.9 5.9 4.3 5.7 4.1 5.1 5.3 6.0 5.9 5.6 5.1 4.7 4.6 5.7 lowest. Non-Fantasy 4.0 4.7 2.7 5.0 3.7 4.0 3.7 5.0 5.0 5.0 4.7 4.0 4.0 5.0 Fantasy participants also Survey Questions: 1. The amount of information on the Home page is just right. rated navigation the lowest 2. The design of Home page enables me to easily get the Sports information I want. but not as severely (maybe 3. Videos on the Home page were readily apparent to me. due to site familiarity). 4. The Video player controls responded exactly as I expected. 5. I had no difficulty navigating the web site. 6. The website is interesting and engaging to me. Overall participants rated the 7. The organization of information on the Home page is just right. design of the Home 8. It was easy for me to get information on my favorite Sport. 9. It was easy for me to get information on my favorite Team. page, video player 10. Getting the score for my favorite Team was easy. controls, and ease of getting 11. Getting stats for my favorite Player was easy. favorite sports team 12. The Search feature enabled me to get the information I wanted. information the highest. 13. The Search results returned (based on my keyword/phrase) matched my expectations. 14. The amount of advertising on the Home page is just right: 23
  24. 24. Usability Survey Results, cont… Survey Question: Thinking about your interests and needs in Sports Survey Question: Based on the experience you had on the web site you entertainment, how does this web site fit you? just used, would you recommend it to a friend? Completely 7 Would definitely 7 Fits me 6.5 recommend 6 5.8 5.6 6 5.6 5.3 Average Rating Average Rating 5 5 4.3 4 4 3 3 2 2 Completely Would definitelyDoes not fit me 1 NOT recommend 1 Fantasy Non-Fantasy Overall Fantasy Non-Fantasy Overall • Again, the difference between Fantasy • Participants in general might or would and non-Fantasy participants probably recommend demonstrates that is a better fit for Fantasy users. • The difference between Fantasy and non-Fantasy participants is not as strong regarding the propensity to recommend 24
  25. 25. Outline of Contents Purpose of the study Participants Methodology Observations Usability survey results Next steps 25
  26. 26. Next Steps• Review findings with the Sports team.• Prepare a more detailed analysis and presentation• Discuss strategy of website – how do we improve user experience?• Discuss design alternatives.• Set a schedule for iterative Usability tests. 26
  28. 28. 0 to 30 days• Interview key sports personnel Kint, Herde, Ruhl, Swanson, etc. to understand current reporting, must have data and future needs.• Reach out to Sales (Calacci & Fixer) to determine immediate needs. Research team participating in weekly Sales calls. Move towards proactive deliverables (industry one sheets, website & product strengths/weaknesses comparisons, etc.) Leverage Magid industry study findings to close deals (Magid presenting next week)• Quick, meaningful win. Commissioner league conversion project31-90 days• Revamp current report suite(s) to better-align w/ BU reporting needs.• Primary research around community• Primary research around usability (is the redesign working as intended. Test new concepts)91 days +• User segmentation project. Understand how users migrate from segment to segment in the value chain