Usability Testing in the Wild :Mindset and Framework                                               David Walczyk          ...
situation & myths          /precepts & methods
situation & myths          /precepts & methods
situation & myths          /precepts & methods
OLD MYTHS              information is... scarce, linear, concrete, analog, surface, closed, non-visual, non-spatial, discr...
NEW MYTHS            information is... abundant, holistic, non-linear, deep,  abstract, computational, executed,   explora...
situation & myths          /precepts & methods
when media change people change.
new media are extensions & amputations.
new media create new environments.
preconceived figures lead to   unperceived grounds.
old methods confirm old ways and conceal               new ones.
insight and understanding  trumps a point of view.
situation & myths          /precepts & methods
MethodsFirst Thinking and Then Doing
Steve the Student        Heather the Hipster         Tom the Teacher            Adam the Architect          Helen the Hipp...
BFI Personas and Scenarios                                                                                                ...
Demographics                                  Comfort with Technology                                Needs                ...
BFI Personas and Scenarios                                                                                                ...
Usability Professionals              Make                                  Grist                       Space Collective   ...
Usability                  Make               Grist         Space Collective   Surfrider Foundation                       ...
BFI Comparative Analysis : SWOT Analysis for BFI.org                                                                      ...
MethodsFirst Thinking and Then Doing
GUERILLA USABILITYeffectiveness + efficiency + satisfaction    Guess(ability) vs. Learn(ability)
EFFECTIVENESSCan a task be completed.
EFFECTIVENESSmeasured in yes or no.
EFFICIENCYhow long to complete a task.
EFFICIENCYmeasured in time.
SATISFACTIONphysiological / social / psychological /              ideological
SATISFACTIONmeasured in affect.
Guessability vs. Learnability
Guerilla Testing   4 steps
1. Decide what to test...       A whole site?       Part of a site?        A process?
2. Decide how to test it...What are the tasks to complete?What are the probing questions?What are the post-test questions?
3. Test it...Recruit users Test Users
4. Analyze results and make     recommendations        Quantitative        Qualitative         Severity
dare to be naive.
Thanks.david@davidwalczyk.com
Usability in the Wild: Mindset and Framework
Usability in the Wild: Mindset and Framework
Usability in the Wild: Mindset and Framework
Usability in the Wild: Mindset and Framework
Usability in the Wild: Mindset and Framework
Usability in the Wild: Mindset and Framework
Usability in the Wild: Mindset and Framework
Upcoming SlideShare
Loading in …5
×

Usability in the Wild: Mindset and Framework

1,184 views
1,122 views

Published on

Presentation given at Columbia University on guerrilla usability by David Walczyk.

Published in: Design
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,184
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Usability in the Wild: Mindset and Framework

  1. 1. Usability Testing in the Wild :Mindset and Framework David Walczyk Client: Columbia University, NYC
  2. 2. situation & myths /precepts & methods
  3. 3. situation & myths /precepts & methods
  4. 4. situation & myths /precepts & methods
  5. 5. OLD MYTHS information is... scarce, linear, concrete, analog, surface, closed, non-visual, non-spatial, discrete,hierarchical, mutually exclusive, phonetic, etc...
  6. 6. NEW MYTHS information is... abundant, holistic, non-linear, deep, abstract, computational, executed, exploratory, behavioral, symbolic,recursive, networked, emergent, etc...
  7. 7. situation & myths /precepts & methods
  8. 8. when media change people change.
  9. 9. new media are extensions & amputations.
  10. 10. new media create new environments.
  11. 11. preconceived figures lead to unperceived grounds.
  12. 12. old methods confirm old ways and conceal new ones.
  13. 13. insight and understanding trumps a point of view.
  14. 14. situation & myths /precepts & methods
  15. 15. MethodsFirst Thinking and Then Doing
  16. 16. Steve the Student Heather the Hipster Tom the Teacher Adam the Architect Helen the Hippie Steve isnt looking for Heather is going to Toms primary focus is Adam will be Helen will be on BFI to much exploration want to explore the going to be collecting interested in read Design Science here. He wants a clear site. Her major ideas for his class – networking/exchanges News and to listen and path to the concerns are going to activities, lesson plans, with like minded watch lectures. information sought so be finding biographical audio/video materials. designers and their he can complete his information and how A secondary concern is projects. homework. to get involved. going to be networking with other professionals to learn Searching for Heather is visiting the about what worked for Tom is looking for Adam is looking for Helen is looking for biographical site to learn more them. curriculum ideas and networking the new issue of information about about Buckminster maybe some insight/ opportunities on the Design Science News. Buckminster Fuller Fuller. success stories from site. from an authoritative others in his field. Grew up source. with WWW. Very tech savvy . Very comfortable. Very comfortable. Can get by. Steve is coming to the Heather is coming to Tom is coming to the Adam will come to the Helen is looking for site for information the site for information site from a site to brush up on current news and about Bucky the man. about Bucky the man, sustainability Buckminster Fuller the updates to the archive. but is going to perspective. Hes going man, network with like explore: community to need education- minded professionals, content, current specific resources and and keep abreast of events, ways to get community content. sustainability news and Steve is going to Heather is going to involved, etc. Tom is going to expect Adam is events. going to Helen is going to expect to find familiar expect clear navigation logical navigation, and expect logical expect things to be navigation and web 2.0 features. grade-specific navigation, where she’s used to mechanisms, like educational resources. biographical and finding them. The dynamic menus, that historical resources, an structure of the site lead him to the easy registration may not make sense, information sought. process, and but she has learned community content. how to navigate it.BFI Personas and Scenarios 1
  17. 17. BFI Personas and Scenarios 4 Demographics Comfort with Technology Needs Age: 24 Computers: Native. • Biographical/ Education: liberal arts BA, thinking Web: Native. historical about grad school Social web: Belongs to all and a few you havent head information income: <$30,000 of. • Current events Bucky familiarity: Vague • Community content • Information on getting involved • Clear and logical Personal Background navigation Heather is tech savvy and hyper-connected like most of her lot. Shes into sustainability & is coming to the site to learn more about the man and his connection to the broader HEATHER sustainability movement, though she hasnt really heard of him beyond being "the dome guy". The Hipster After exploring the site, Heather is going to want to organize a community chapter. She Googled "Buckminster Fuller Dome" to get to the site. "Isnt he the dome guy?" Motivations Scenarios Site Features Behaviors on Current BFI Website Behaviors on Redesigned BFI Website To learn more Read about geodesic • Clear and logical Googles domes and finds what appears to be a Lands on content page from Google. Skims it. about Bucky domes and thought navigation. landing page for articles about domes at BFI. Realizes she wants to check out more. Using and his Bucky sounded familiar, However navigation out of that page to other bread crumbs she goes up a level to see what connection to so she Googled parts of the site is unclear. other content may be of interest to her. the "Buckminster Fuller sustainability Dome" and landed on a movement bfi.org content page Wants to get Inspired by her about domes. reading • Clear and logical Goes to BFI homepage via her delicious.com Goes to BFI homepage via her delicious.com involved the other day, she navigation. bookmarks. Clicks on the Community Content bookmarks. Selects the "Local Chapters" button. locally decides she wants to button, but is unsure as it also reads "Connect Taken to local chapter landing page where she contribute locally. She • Easy and secure with our network of designers." Clicks on the picks a chapter in her local region. Is excited by goes back to the BFI registration "Use our forums" page and is taken to a How to all the evens listed on the homepage, and wants homepage to find out process (doesnt use forums page. Clicks the back button. Sees a to participate. Clicks the "register" link at the about local chapters. ask for too much link in the red box in the upper right corner top right corner (where its usually found) to or too little info). labelled "BFI Forums". Clicks that and is taken to sign up for the email newsletter and to join the • Clear opt-out the forums. forum. options. Wants to keep Gets too much email • RSS/Atom Goes to the BFI website and clicks on the RSS Heather realizes she doesnt want to fill up her abreast of any and newsletters end up (customizable link. Shes unsure exactly what content she is inbox with emails, so she goes to the BFI events and spam. Worried about feeds). subscribing to. She clicks "More options homepage and clicks the RSS link in the upper news from missing an event at her (experts only)" and finds an overwhelming list right corner. She is taken to the RSS page where BFI. local BFI chapter. of potential RSS subscription options. She goes she chooses what content she wishes to back to the RSS page and subscribes to the subscribe to in her feed reader. basic feed.
  18. 18. Demographics Comfort with Technology Needs Age: 16 Computers: Grew up with them. • Access historical Education: High school sophomore Web: Doesn’t know a world without it. information about Income: None (parents >$90,000) Social web: Facebook is lame now that Mom’s joined. Bucky in order to Bucky familiarity: None complete an assignment Personal Background Steve is a 16-year-old high school student finishing his sophomore year. He is very comfortable with online environments and has the latest video game consoles. He has no familiarity with STEVE Bucky, and was assigned to write a paper about him for a class. He doesnt read web pages The Student thoroughly – hes more of a page scanner and usually clicks the first link. If he doesnt see something quickly he gives up. He is frustrated that he cant just use Wikipedia for his "I dont know, my teacher assignment. just assigned it." Motivations Scenarios Site Features Behaviors on Current BFI Website Behaviors on Redesigned BFI Website Learn He couldnt use • Clear path to Clicks the "Who is Buckminster Fuller?" button Steve clicks on the About Buckminster tab and historical Wikipedia as a source. biographical on the BFI homepage. Scrolls down to "Basic is taken to a landing page that includes a Bucky information Clicked on BFI.org as an information. Biography" clicks link -- finds a timeline. Clicks biography. Links to more in-depth resources about external link from Bucky the back button. Clicks on the "Introduction to via the Education page are included. Buckminster article. Buckminster Fuller" article. Is frustrated that the Fuller article has some biographical information but it is not complete.BFI Personas and Scenarios 3
  19. 19. BFI Personas and Scenarios 2 complex interactions ct r ite p ste ch Hi Ar t he th e er a th a m He Ad frequent Steve the Student interactions er infrequent a ch e Te interactions p pi e Hi th e m th n To le He simple interactions Key Usage of current BFI website Usage of redesigned BFI website Change in website usage
  20. 20. Usability Professionals Make Grist Space Collective Surfrider Foundation Association http://www.makezine.com/ http://www.grist.org/ http://spacecollective.org/ http://www.surfrider.org/ http://www.upassoc.org/ From the consistent browser Browser title, site ID, and An initial Flash animation is Evocative header images are As befits a website for a title, site title, and subtitle, it subtitle make clear that the difficult to read and potentially background to the professional association of is fairly clear that Make covers website provides annoying to users without the foundations logo and usability specialists, no initial do-it-yourself technology "environmental news and plug-in or who are just illustrate its motto, "Protecting ads detract from the clarity of projects. Ad placement commentary." But a large impatient. A large text block Our Beaches For Over 20 a Web standard top left logo unfortunately dilutes the banner ad is placed at top of about the Space Collective text Years." Throughout the page, and subsequent tabbed global message somewhat – banner the page, where many users then loads first in the middle the use of blue and gray navigation. Navigation labels and Google ads loom above expect to find the main of the page, signaling that this evokes the ocean and the use recognized syntax, such the global navigation – and a navigation. Instead, main site is "different." The text foundations focus. Global as "About UPA," so that no large ad for McDonalds does navigation is located on the outlines the purpose, but not navigation in a Web-standard, mouseovers are required to not instill initial trust in the left. Site hierarchy is displayed the nature, of the site, and top of the screen location uses understand them. Most other site. But global navigation through the use of section moreover does so in an clear, action-oriented labels to links are either standard follows Web standards, titles, but section titles manner that reinforces the orient the user and convey site underlined blue text or counteracting the profusion of themselves are not clickable sites arty/science-fiction contents. The sites appeal for clickable graphics. Text about ads: a logo home button is on and the list is so long that the identity: users are user action continues with the organization is the upper left and utilities user cannot scan what "terrestrials." Navigation is other command labeling: "Stay prominently placed in the such as search and RSS are on amounts to a summary of the difficult to pick out, as dark Informed," "Join Surfrider," center just below the global the upper right, with a row of site without scrolling. gray elements do not contrast "Support Surfrider." The center navigation, and is followed by multicolored labels across the Navigation labels such as with a black background of the page features a slide content that is current and page presenting navigation "News/Analysis" communicate (though some elements pop show that outlines the useful to the associations options to the six main a news-oriented site, but out in red on mouseovers). A foundations strategic membership: conferences, sections of the website other labels, such as "Victual login utility is in a Web initiatives and links to the events, and news. A short beneath. Labels such as "Blog," Reality" impart a kind of standard upper right location, "What We Do" area of the site, section then follows which "Videos/ Podcasts," and "hipness." Although the latter but a barely visible hover label and is followed by links to introduces usability concepts "Forum/Community" signal type of labels are opaque to states that membership is by current campaigns and to potential clients. Site that Make is a participatory the uninitiated, mouseovers invitation only. Navigation featured content. On the left utilities, including search, site. The labels are highlighted yield some clarification. A labeling is also an issue, both are site utilities such as an most visited links, and when moused over, and clearly labeled toolbox links to in design (small and low email newsletter signup box, a membership registration/log- remain so when selected, useful site utilities, and is contrast) and vocabulary (what search box, and membership/ in are offered at the left and acting as "breadcrumbs" to located where most users is a "timecapsule"?). However, chapter links. Membership right sides of the screen. orient the user. Other links are would expect it, at top right. the blog/channel, multimedia links are also grouped in and Above UPAs contact not visually uniform: some are The rest of the page follows a nature of the site is beneath a color-coded portion information and privacy policy indicated by boxes and others layout fairly typical to news immediately implied by the of the top menu on the right. at page bottom are what merely with Web standard organizations on the Web: label "recent posts" and the Beneath this second appear to be ads in the form underlined text. An option to several prominently featured obvious image gallery layout. membership area are links to of logos/links to UPAs "Suggest a Site!" does not images/videos plus other Also, higher contrast large the foundations blogs, corporate supporter websites, specify why, and simple footer areas corresponding to global labels effectively separate podcasts, and other but these logos establish navigation is preceded by yet navigation categories page areas and lead the eye publications, coastal news UPAs credibility by more Google ads. But the containing titles and links to down and across the page. items, and two vaguely labeled announcing its acceptance by home pages large featured the latest stories. The right Space Collectives invitation- but visually appealing links to these organizations. Clear content area reinforces Makes side of the screen contains only, members-only policy, "The Ring" and "SocialVibe." labeling, standard placement message, flashing features more featured content, both and its use of a black Overall, the sites clear of page elements, and an from the site and magazine, as internal and external to the background, slick layout, and organization and layout, unremarkable, business- well as recent blog posts. The site, including a section for arcane vocabulary convey imperative labeling, and friendly color scheme convey a design in general is not "this weeks most popular" "coolness." The site is judicious use of images sober dedication to usability particularly slick, which items, as is typical for news deliberately less than communicates the principles. reinforces the you-can-do-it, sites. Footer navigation welcoming, drawing users in foundations proactive nature, craft-oriented nature of the continues the problematic by creating a desire to be "in" efficient operation, and site. global navigation labels, the group and in the know. seriousness of purpose. unfortunately.BFI Comparative Analysis : Detailed Evaluations : Home Page 5
  21. 21. Usability Make Grist Space Collective Surfrider Foundation Professionals Association Home Page Design 4.0 4.5 4.0 3.5 3.5 Interior Page 5.0 Design 4.0 4.5 4.0 1.5 Search Functionali ty 4.5 3.5 2.5 1.0 1.0 Features Offered 5.0 4.5 4.5 4.0 2.0 Navigation 5.0 4.0 4.0 3.0 3.0 Interaction Design 5.0 4.0 4.0 1.5 1.0BFI Comparative Analysis : Summary Evaluations (1 = poor, 5 = 11
  22. 22. BFI Comparative Analysis : SWOT Analysis for BFI.org 4 Strengths Weaknesses Opportunities Threats • BFI has rich content • The BFI website does not • Sites such as Surfrider • Other sites well engineered addressing a wide range of fully promote R. Foundation have not yet page designs and potential uses Buckminster Fullers legacy fully developed interactive navigation mean that, in by making his life, designs, functionality to encourage spite of the presence of • Buckminster Fuller and his and ideas easily participation and a large advertising, users are more ideas and design could be understandable by a wide potential user base may still likely to visit those sites and enormously appealing for audience remain to be captivated/ thus spend less time at educational uses captured BFI.org • Information on the BFI • BFI already has a dedicated website is not organized by • Sites such as Space • Several sites offer the site community of fans of user-oriented topics Collective and Surfrider utilities users have come to Buckminster Fuller the man Foundation provide poor expect and BFI risks losing • Interactive functions are not search capabilities, and BFI potential users to sites • The BFI Challenge has clearly described or could improve in these offering easier ways to get created enormous goodwill distinguished from one areas to make its large pool involved or stay informed and exposure for the another so not all users who of information more organization might be interested in accessible and thus more • Other sites do a better job participating are reached attractive of stating or making clear • The Idea Index is unique their organizations and/or and could be leveraged to • Separate registrations are • Sites such as Space websites purpose. Space invite more discussion and required for BFI and the BFI Collective and Surfrider Collectives textual interaction Challenge Foundation do not offer description may be • Top level navigation is much in the way of site somewhat vague, but the utilities to engage users, design of its home page difficult to find, so users and BFI could work to clearly implies the nature of may be confused about how improve in this area and the sites contents to navigate the site increase user involvement • Labels use specialized • BFI risks losing potential • BFI hosts such a wealth of members to sites such as vocabulary that new users historical and current Space Collective who have may not understand information regarding already succeeded in • Tagging functionality has Fullers Ideas, Life, and generating active member not been fully exploited to Designs that its redesign engagement enable exploration and could establish BFI as the discovery of site content authority on information regarding the man • The use of the blog format results in posts and comments that appear to be orphaned and not actively moderated: the site appears to be a ghost town
  23. 23. MethodsFirst Thinking and Then Doing
  24. 24. GUERILLA USABILITYeffectiveness + efficiency + satisfaction Guess(ability) vs. Learn(ability)
  25. 25. EFFECTIVENESSCan a task be completed.
  26. 26. EFFECTIVENESSmeasured in yes or no.
  27. 27. EFFICIENCYhow long to complete a task.
  28. 28. EFFICIENCYmeasured in time.
  29. 29. SATISFACTIONphysiological / social / psychological / ideological
  30. 30. SATISFACTIONmeasured in affect.
  31. 31. Guessability vs. Learnability
  32. 32. Guerilla Testing 4 steps
  33. 33. 1. Decide what to test... A whole site? Part of a site? A process?
  34. 34. 2. Decide how to test it...What are the tasks to complete?What are the probing questions?What are the post-test questions?
  35. 35. 3. Test it...Recruit users Test Users
  36. 36. 4. Analyze results and make recommendations Quantitative Qualitative Severity
  37. 37. dare to be naive.
  38. 38. Thanks.david@davidwalczyk.com

×