UX Results Through Research: Penn State Web Conference

426 views

Published on

Jackie S. Zajac of American University's presentation on UX Results Through Research at Penn State, June 2013.

Published in: Technology, Education, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
426
On SlideShare
0
From Embeds
0
Number of Embeds
60
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Title Pages
  • Title Pages
  • Text1
  • 5,000+ graduate students currently.
  • Will be doing landing pages, email newsletters, mobile this month
  • At no point were there 100% unanimous results.
  • There were times they were reasonably satisfied, but did not think it was as easy.
  • More mixed results…
  • Title 2
  • Title 2
  • Title Three
  • “MS Applicants,” “MBA Applicants”
  • Tuition & Fees
  • Title Three
  • Time to move forwardSome staff weren’t happy – Global Learning
  • Title Three
  • Global Learning scuffleKLAB scuffle
  • Text, 2 Column
  • Text, 2 Column
  • Mark Clark not pleased
  • Text, 2 Column
  • Text, 2 Column
  • Text, 2 Column
  • Text, 2 Column
  • Text, 2 Column
  • Graduate Programs index page – DOWN 37% - eliminates one click for the user. Application Guide down 3%
  • Graduate Programs index page – DOWN 37% - eliminates one click for the user. Application Guide down 3%
  • Text, 2 Column
  • Blank
  • UX Results Through Research: Penn State Web Conference

    1. 1. Good UX Work ThatWon’t Go to WasteA Case Study of American.Edu/Kogod
    2. 2. Jackie ZajacDirector, Strategic Communications• Journalism —> Marketing & Communications• 4 years at AU/Kogod• Never really heard of Usability until 2012
    3. 3. Inspiration• PSUWeb12• User Interactions with Information Systems• Declining graduate enrollment• Bickering over web strategy
    4. 4. School Background• Kogod School of Business• 1,200 undergraduate, 600graduate students• 120 Faculty & Staff• 8 Graduate Programs• 40,000+ visits/month
    5. 5. “1,000 More Grad Students in 10 Years”
    6. 6. Reporting Structure8 Faculty ProgramDirectors6-person AdmissionsTeamDean, Provost,University Marketing AdvisoryCouncil, Web Steering Committee...
    7. 7. Study Parameters• Time (< 3 weeks)• Modest budget ($250)• Technical limitations due to AU’s CMS• Only tested American.Edu/Kogod
    8. 8. Usability Test
    9. 9. Tools!1. Silverback ($69.95)2. Treejack (Free version)3. Audio recorder4. Pen + paper5. Gift cards
    10. 10. Test DesignTasks (6)• Specific to Prospective Graduate StudentsUsers (5)*• Demographic matches*Jakob Nielsen said it was OK!
    11. 11. Test DesignTest outline• Homepage Impressions• Tasks• Post-Task Completion Survey• Final Thoughts?• 30 Minutes
    12. 12. Tasks1. How many credits are in the MS in Accounting program?2. Find the average GMAT score for a Full-time MBA student3. What is the cost of the Part-time MBA program?4. When is the next Information Session being held?5. Find the graduate admissions application6. Find the graduate admissions’ office phone number*Abbreviated wording
    13. 13. Completion SurveyAfter each task, the user was asked:1. Did you complete the task? (Yes/No)2. How satisfied are you, on a scale of 1 to 5? (5 beingvery satisfied)3. How easy was it, on a scale of 1 to 5? (5 being veryeasy)
    14. 14. Test Participants• 5 users• Washington, D.C.-based• Well-educated• Bachelor’s Degree Minimum• Age match (average 28.8 y/o)• 4 Domestic, 1 International Background
    15. 15. 40%60% 60% 60%80% 80%100%80% 80%60%80% 80%Find the numberof credits in theMS of Accounting.Find the averageGMAT score for aFull-Time MBAstudent.Find the cost of theProfessional (Part-time) MBAprogram.Find when thenext informationsession is beingheld.Find the graduateadmissionsapplication.Find the graduateadmissions’ officephone number.Task completion rateTask completed from testers perspectiveTask completed from users perspective
    16. 16. 00.511.522.533.544.55Satisfaction and Ease from User PerspectiveTask completionsatisfaction rate fromusers perspective (1 to5)Easiness rate from usersperspectives (1 to 5)
    17. 17. 0 1 2 3 4Site is attractive and has agood balance of graphics/text.Site was easy to navigate.Information is easy to read.I always felt I knew what waspossible to do next.My mistakes were easy tocorrect.Number of user responsesUsers’ Final ImpressionsMaybeNoYes
    18. 18. Key Finding:Too Many Navigational Options
    19. 19. User Comments“My eyes are drawn immediately to the big large picture. Itfeels like there’s a lot of stuff framing it, going around…it’skind of confusing.”“FTMBA…I’m not really sure what that acronym means.”“Would I give up right now? If this was a school I wanted togo to, No. …But it’s not easy to find.”
    20. 20. Page Load Time
    21. 21. Page Load Time• Average homepage load time = 6.1 seconds• Slower than 76% of tested websites• Industry standard = 4 seconds or less (June 2012)• Many Javascript and CSS files built into the design• Some homepage images were >500kb
    22. 22. Card Sort
    23. 23. Card Sort• 22 participants in a closed card sort• Median age 25-35• 28 cards, 4 categories• Graduate Admissions• Graduate Programs• Contact Us• About Kogod
    24. 24. Card Sort Results• The majority of users had similar ideas P• 3 cards consistently sorted in different categories• Frequently Asked Questions• Information Sessions• Visiting Campus
    25. 25. Card SortResults—22participants
    26. 26. Kogod HomePageAboutContact UsDeans BlogFacts & RankingsKogod AdvisoryCouncilKogod BrochureSchool HistoryWelcome Fromthe DeanVisiting CampusGraduateAdmissionsAdmissionsDeadlinesApplication GuideEnrollmentApplicationGraduate DegreeRequirementsInformationSessionsInternationalApplicantsMilitaryApplicantsTest ScoresRequest MoreInfoTuition and FeesVisiting CampusFAQsGraduateProgramsFull Time MBAFull Time MBACareer OutcomesProfessional MBAGraduateCertificatesMS in AccountingMS in FinanceMS in Real EstateMS inSustainabilityManagementMS in TaxationProposed IA
    27. 27. Kogod HomePageAboutContact UsDeans BlogFacts & RankingsKogod AdvisoryCouncilKogod BrochureSchool HistoryWelcome Fromthe DeanVisiting CampusGraduateAdmissionsAdmissionsDeadlinesApplication GuideEnrollmentApplicationGraduate DegreeRequirementsInformationSessionsInternationalApplicantsMilitaryApplicantsTest ScoresRequest MoreInfoTuition and FeesVisiting CampusFAQsGraduateProgramsFull Time MBAFull Time MBACareer OutcomesProfessional MBAGraduateCertificatesMS in AccountingMS in FinanceMS in Real EstateMS inSustainabilityManagementMS in TaxationProposed IAX
    28. 28. What’d We DoWith All This?
    29. 29. Recommendations• Add an Admissions tab to the top nav (Duh.)• Link directly to each graduate program from thehomepage• Restructure Admissions nav• Create a new, central MBA “landing” page• Allows users to view both formats before click
    30. 30. Recommendations• Decrease jargon• Global Learning vs. Study Abroad• FTMBA vs. Full Time MBA• Make functional info clearer + easier to find• Breadcrumb navigation?• Social media more prominent• Better process for Information Sessions• Reduce image file sizes• Delete homepage touts
    31. 31. For example
    32. 32. All Aboard!
    33. 33. All Aboard!Dear all,You may have noticed some changes to the Kogod website recently. They are theresult of a usability study of the website we conducted this summer. The focus of ourtesting was to determine how easy the website is for our primary audience—prospective students—to use. Based on the studys results, weve implementedseveral changes to the websites layout, navigation and terminology. A few morechanges lie ahead.We expect that this process will make it easier for our prospects to find theinformation theyre looking for, and to take action. A follow-up study will be conductedin 2013 to determine whether further improvements are needed.If you have any questions, do let me know.
    34. 34. Your BFFs <31. Data2. Admissions3. School Goals
    35. 35. Results @ 6 WeeksAdmissions Homepage: 7,637 pvs (+3%)MS Finance: 3,819 pvs (+50%)MS Accounting: 3,115 pvs (+60%)MS Sustainability Mgmt : 1,325 pvs (+5%)MS Taxation: 1,106 pvs (+84%)MS Real Estate: 1,095 pvs (+63%)Renamed “Tuition & Fees”: 2,147 pvs (+150%)New MS Applicants page: 2,515 pvs, 02:19 TOP
    36. 36. Results @ 6 WeeksNew MBA “landing” page: 2,617 pvs (n/a)FTMBA page: 2,621 pvs (-10%)PMBA page: 1,733 pvs (+11%)
    37. 37. Results @ 6 WeeksNew MBA “landing” page: 2,617 pvs (n/a)FTMBA page: 2,621 pvs (-10%)PMBA page: 1,733 pvs (+11%)
    38. 38. Results: 6 Weeks
    39. 39. Then Came…More ChangesRedesigned Degree Pages• Clearer Calls to Action• ConsistencyRedesigned Graduate Programs index page
    40. 40. Remember:Always send a status update to Program Directors.Don’t forget SEO.
    41. 41. Now @ 9 MonthsOverall: No substantial change in visitorsBut we’re DOWN:• Pageviews -6.7%• Avg visit duration down (slightly)• Pages per visit -7.3%
    42. 42. Now @ 9 MonthsAdmissions homepage: +3.8% over same time periodMS Finance: +58.8%MS Accounting: +48.9%MS Sustainability Mgmt: +12.4%MS Taxation: +72.4%MS Real Estate: +74%Tuition & Fees: +190.7%
    43. 43. Now @ 9 MonthsNew MBA landing page: 18,451 pvs (n/a)FTMBA degree page: -2.5%PMBA degree page: +7%
    44. 44. What’s Next?Summer 2013• Usability testing with same tasks• 2-3 additional tasks• Test ad landing pages• Test on desktop + mobile devices
    45. 45. Questions?@jackiesauter

    ×