Web navigation design with respect to cognitive science


Published on

Web navigation design with respect to cognitive science

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Web navigation design with respect to cognitive science

  1. 1. Web Site Navigation Design with Respect to Cognitive Load<br />Aimee Maree Forsstrom<br />Supervisor Dr Graham Cooper<br />Honours Presentation Southern Cross University<br />
  2. 2. Interdisciplinary research<br />Computer Science<br />Educational Psychology<br />Cognitive Load and Human Computer Interaction<br />Does web site navigation design effect how we take in information<br />How can we apply Cognitive Load Theory to web site navigation design to assist the intake of information<br />The story so far…<br />
  3. 3. Timeline - Key Dates<br />Honours Thesis started in February 2008<br />Original plan to run cognitive load tests on Australian Learning and Teaching Council June 2008<br />First Honours presentation July 2008<br />Honours update presentation October 2008<br />Plan changed to be run on a mock up University Faculty website January 2009<br />Plan changed to be run on a online game presenting random facts to the user May 2009<br />Ethics received in August 2009<br />Experiment ran first two weeks of September 2009<br />Final Honours presentation October 2009 <br />
  4. 4. Research in the Social Network era<br />Participants informed online<br />Tweets placed on www.twitter.com<br />Message sent to friends on www.facebook.com<br />Link posted on www.facebook.com profile page<br />Email sent to all Southern Cross University staff/students<br />Email sent to all Programming Society members (UTS)<br />202 participants played the game<br />124 meet needed requirements<br />118 participants in the final clean data set <br />59 in control group<br />59 in treatment group<br />Participants in both groups balanced on age/gender/comp use<br />
  5. 5. What we learnt<br />Social networking is a great way to reach large number of participants<br />Participants numbers coming to the site will peak and then slowly die off<br />Online game was a great tool to drive people through website navigation<br />People don’t mind playing a game and answering some short questions<br />Expect a percentage of drop out’s and possible technical glitches (40% error rate encountered)<br />Track mouse paths through all stages of the site to ensure proper human activity (not a robot)<br />Data must be thoroughly cleaned and results checked to ensure there are no anomalies from technical issues or cheating<br />
  6. 6. Experiment Design<br />
  7. 7. Playing a Game can be fun<br />Online fact learning game design chosen<br />15 questions presented in random fashion<br />Answers where found by navigating through the web site menu<br />5 top levels of categories in menu<br /> 4 lower level menu options per category<br />Using a game we can drive the participant through the menu <br />maximum use of menu 15 clicks with short time frame 10min<br />Engage participant in web site material and appeal to wide audience<br />
  8. 8. Experiment Components<br />Research online check<br />Random fact game play <br />10min, 15 questions<br />Effort survey <br />Fill in facts, participant to fill in missing words from 10 facts presented in game<br />Menu rebuild, participant asked to rebuild the menu to show if they acquired the menu schema<br />Usability Survey<br />
  9. 9. Random Game start page presented<br />Participant asked to scroll to bottom of page and roll dice<br />After the dice is rolled participant is given a question and asked to locate the section in the menu that will hold the answer<br />The menu sections are named after key words in the question asked<br />If the participant clicks on the wrong menu they are asked to re-read the question and try again<br />Once the final answer is read the participant is directed to answer questions about the game<br />Experiment Steps<br />
  10. 10. Data Analysis Methods<br />
  11. 11. Statistics<br />Non Parametric and parametric statistics used<br />Distribution was heavily skewed so Parametric statistics could not be used on most questions<br />Qualitative and Quantitative data analysed<br />
  12. 12. Survey asked on Effort<br />Two questions<br />Questions answered by 7 point likert scale <br />Mann Whitney U Test<br />Descriptive Statistics<br />
  13. 13. Survey asked on Usability<br />Two questions<br />Questions answered by 7 point likert scale <br />Mann Whitney U Test<br />Descriptive Statistics <br />Three open Ended questions<br />Linear regression on positive and negative word count<br />Linear regression and on word categories<br />
  14. 14. Fill in Facts Questions<br />10 Fill in fact questions grouped by easy, medium, hard<br />Mann Whitney U test on overall score average<br />ANOVA <br />Question type score and group interaction<br />Variables Easy, Medium, Hard questions, Factor group id<br />Chi-Square test for each individual question <br />Group comparison of the number of participants who got the question right<br />Wilcoxon Ranked Sum Test <br />Difference in individual question answers between groups<br />Group Average for Fill in facts question completion time<br />
  15. 15. Menu Schema Rebuild<br />Menu Rebuild, 18 menu answers captured<br />Mann Whitney U test on overall score average<br />ANOVA <br />Menu top, lower menu rebuild and group interaction<br />Variables top, lower level menu sections, Factor group id<br />Chi-Square test for each individual menu element <br />Group comparison of the number of participants who got the question right<br />Wilcoxon Ranked Sum Test <br />difference in individual menu section answers between groups<br />Group Average for Menu rebuild time<br />
  16. 16. Mouse path collected for each group (screen click point captured)<br />Comparison of mouse click path used in game<br />Overall random facts game time<br />T-test of game time means<br />Random Facts navigation time between each of the 15 questions<br />T-test of individual navigation steps<br />Mouse Paths and Game Time<br />
  17. 17. Preliminary Results<br />
  18. 18. Effort Survey Likert Question One<br />Effort one question displays slight backward trend<br />How easy was it to use the menu to play the game<br />Treatment group mean 2.05 <br />Control group mean 1.85<br />1 = very easy<br />2 = easy<br />
  19. 19. Statistical Significance found for Effort question two<br />How distracting did you find the scrolling<br />Effort Survey Likert Question Two<br />Treatment group mean 2.71 <br />Control group mean 3.51<br />2 = very slightly distracting<br />3 = slightly distracting<br />4 = distracting<br />
  20. 20. Usability Survey Likert Questions<br />No difference in likert scale questions<br />After rolling the dice for each question locating the navigation menu was easy<br />Treatment group mean = 5.37<br />Control group mean = 5.34<br />I enjoyed playing the random facts game<br />Treatment group mean = 4.97<br />Control group mean = 4.97<br />4 = neither agree nor disagree<br />5 = slightly agree<br />6 = agree<br />
  21. 21. Usability Survey Open Ended<br />Qualitative data was analyzed for positive and negative words mentioned <br />Experiment Group had higher positive score<br />Treatment Group had higher negative score<br />45% of the treatment group asked to change the scrolling in the game <br />12% of the experiment group asked to change the scrolling in the game<br />
  22. 22. Fill in Facts Questions<br />Statistical significance between groups for overall question score<br />Treatment group outperformed the control group on overall test scores<br />Treatment group mean 13.56<br />Control group mean 12.05<br />Maximum Score for facts = 17<br />
  23. 23. Fill in Facts Questions<br />Statistical difference found for overall mark on question type and group id<br />Treatment group outperformed the control group on the medium to harder questions<br />
  24. 24. Fill in Facts Questions<br />Statistical difference found for individual question marks between group<br />Treatment group outperformed control group<br />
  25. 25. Menu Schema Rebuild<br />Statistical significance not found on overall scores<br />Difference in means 14.03 for treatment group and 12.02 for control group not significant <br />Treatment group mean 14.03 <br />Control group mean 12.02<br />Maximum Score for facts = 33<br />
  26. 26. <ul><li>Statistical significance found on number of correct answers between groups</li></ul>Experiment group outperformed control group<br />Menu Schema Rebuild<br />
  27. 27. Lessons Learnt<br />Need twice as many questions <br />30 instead of 15<br />keep participants engaged longer <br />game play average was 5min not 10min<br />More elements in menu schema rebuild <br />30 instead of 18<br />Need for more medium and heavy questions<br />10/15 instead of 6/10 easy/hard question ratio<br />Ensure wording of questions crosses borders<br />Include more survey questions <br />Perform more technical checks and pilot<br />
  28. 28. Thank you<br />