An Introduction toInteraction Design g Mark Billinghurst HIT Lab NZ
MarkPhD Electrical Engineering University of WashingtonInteraction Design Museum experiences Tools for designersAugmented Reality Mobile AR, Evaluation AR Evaluation, Multimodal Interfaces, CollaborativeCollaboration Enhanced FtF and remote collaboration Social networking
Workshop Overview1. What is Interaction Design2.2 Understanding Interaction3. Understanding User Experience4. Design and Prototyping5.5 Evaluation6. Design in the Real World7. Resources
Source MaterialInteraction Design: Beyond Human pComputer-Interaction Rogers, Sharp, Preecehttp://www.id-book.com/htt // id b k /
1. What is Interaction Design? Mark Billinghurst Preece Chapter 1
What to do?ImagineI i You’re bringing a new product to market Your #2 competitor has been in the market for over a year, selling millions of units y g Your #1 competitor launches the same month Your technology is slower than your competitors Your technology is older than your competitors Your last product failed in the market
Do you compete on Price ?Do you compete on Technology ?Do you compete on Features ?Wrong: Compete on user experience !
Nintendo WiiCheap - $500Unique game play Wireless 3 DOF controller Position and orientation sensingAiming to broaden user base gCan play previous games/downloads
Interaction Design“Designing interactive products to support people in their everyday and working lives” lives Preece, J., (2002). Interaction Design Design of User Experience with Technology
Goals of interaction designDevelop usable products Usability means easy to learn, effective to use and y y , provide an enjoyable experienceInvolve users in the design process
Bill VerplankInteraction Design involves answering three questions: What do you do? - How do you affect the world? What do you feel? – What do you sense of the world? What do you know? – What do you learn? y y
Interaction Design Process I D P Identify needs/ establish requirements q (Re)Design Evaluate Build an interactive versionDevelop alternative prototypes/concepts and compare them Final productAnd iterate, iterate, iterate.... iterate iterate iterate
2. Understanding Interaction Mark Billinghurst
Understanding the problem space What do you want to create? y What are your assumptions? What are your claims? Will it achieve what you hope it will? If so, how?
Analysing the problem spaceAre there problems with an existing product oruser experience? pWhy do you think there are problems?How do you think your proposed design ideasmight overcome these?When designing for a new user experience howwill the proposed design extend or changecurrent ways of doing things? y g g
Conceptual modelA conceptual model i t l d l is:“a high-level description of how a system is organized and operates. operates.” (Johnson and Henderson, 2002, p. 26)Need to first think about how the system will appearto users (i.e. how they will understand it)Not a description of the user interface but a structureoutlining the concepts and relationships between themBenefitsB f Enables “designers to straighten out their thinking before they start laying out their widgets” (p. 28) y g g (p ) Provides a working strategy and a framework of general concepts and their interrelations
Interface metaphorsDesigned be i ilD i d to b similar to a physical entity b also has own h i l i but l hproperties e.g. e g desktop metaphor, search engine metaphorExploit user’s familiar knowledge, helping them to understand‘the unfamiliar’Conjures up the essence of the unfamiliar activity, enablingusers to leverage of this to understand more aspects of theunfamiliar f ti lit f ili functionalityPeople find it easier to learn and talk about what they aredoing at the computer interface in terms familiar to them
Example: The spreadsheet Analogous to ledger sheet Interactive and computational p Easy to understand Greatly extending what accountants and others could dowww.bricklin.com/history/refcards.htm
Why was it so good?ItI was simple, clear, and obvious to the users how to l l d b h huse the application and what it could do“it i j“i is just a tool to allow others to work out their l ll h k h iideas and reduce the tedium of repeating the samecalculations.calculations ”capitalized on user’s familiarity with ledger sheetsGot hG the computer to perform a range of different f f diffcalculations in response to user input
Another classic8010 S office system targeted at workers not Star ff d kinterested in computing per seSpent several person-years at beginning working outS l b i i kithe conceptual modelSimplified h lSi lifi d the electronic world, making it seem more i ld ki ifamiliar, less alien, and easier to learnJohnson et al (1989)
Benefits of interface metaphorsMakes lM k learning new systems easierHelps users understand the underlying p y gconceptual modelCan be innovative and enable the realm ofcomputers and their applications to be mademore accessible to a greater diversity of users
Problems with interface metaphors P bl h f h (Nielson, (Nielson 1990)Break conventional and cultural rules e.g., recycle bin placed on desktopCan constrain designers in the way they conceptualize a problemConflict with design principlesForces users to only understand the system in terms of themetaphorDesigners can inadvertently use bad existing designs and transferthe bad parts over h b dLimits designers’ imagination with new conceptual models
Summary pointsNeed to have a good understanding of the problem space specifying what it is you are doing, why, and how it will support p y g y g, y, pp users in the way intendedA conceptual model is a high-level description of a product what users can do with it and the concepts they need to understand how to interact with itDecisions about conceptual d i should be made beforeD ii b l design h ld b d b fcommencing any physical designInterface metaphors are commonly used as part of aconceptual model
3. Understanding User Experience Mark Billinghurst
Dangerously Undermining TaskMany p p in Palm Beach, Florida that wanted to vote for y peopleAl Gore, accidentally voted for Pat BuchananEstimated 4000 voted“wrong“ wrongAnother 19,000punched both holes(invalid vote)
Interaction Design is User Centered A methodology for ensuring good user experiences with products by getting feedback from users to inform the design.
1. User research and needs analysis Identify needs/ establish requirements i t (Re)Design Evaluate Build an interactive version Final p oduct a product
What, How and Why? •Why: Requirements definition: the stage where failure occurs most h f il t commonly Getting requirements right is crucial50 www.id-book.com
What, how and why?•What 1. Understand users, task, context 2. Produce a stable set of requirements•How: Data th i D t gathering activities ti iti Data analysis activities Iterate Expression as ‘requirements’ requirements•Why: y Requirements definition: failure occurs most commonly Getting requirements right is crucial
What Wh are the Requirements? h R ?Functional What should the system do? What has it done?DataEnvironment or Context of use Physical: dusty? noisy? light? heat? humidity? …. Social: sharing of files of displays work alone .. files, displays, Organizational: IT support, hierarchy, training..Users: Wh are they?U Who h ?Usability: learnability, throughput, flexibility..
Who REALLY are your Users/Stakeholders? •Not as obvious as you think: — those who interact directly with the product — those who manage direct users — those who receive output from the product — those who make the purchasing decision — those who use competitor s products competitor’s •Three categories of user (Eason, 1987): — primary: frequent hands-on p y q — secondary: occasional or via someone else — tertiary: affected by its introduction, or will influence its purchase
Who are the Stakeholders? Check-out operators • Suppliers • Local shop ownersManagers and owners Customers C t
Data Gathering Techniques (1)Questionnaires Looking for specific information Qualitative and quantitative results Good for getting data from a large dispersed group large,Interviews Good for exploring issues, using props Structured, unstructured or semi-structured But are time consuming and difficult to visit everyone
Data Gathering Techniques (2)Workshops or focus groupsW kh f Group interviews/activities Good G d at gaining a consensus view and/or i i i d/ highlighting areas of conflictObservations Spending time with users in day to day tasks Good for understanding task context requires time and can result in a huge amount of dataDocumentation Procedures and rules written down in manuals
A day in the Life of.. Cultural Probes.. Role Playing..
Some Basic Guidelines• Focus on identifying the stakeholders’ needs• Involve all the stakeholder groups• Involve more than one person from each group• Use a combination of data gathering techniques•S Support the process with props such as prototypes h ih h• Consider carefully how to record the data y
Why?Users can give you lots of f db k about what yourU l f feedback b hproduct does well and what it doesn’t do well.Users can give you insight into their lives and needshelpingh l i you understand the context in which your d d h i hi hproduct will be usedUsers can inspire design solutions you haven’tthought of. h h f
Interviewing TechniquesGood interviewing is a skill and needs to be done properly to ensure y maximize the p p y you opportunity you have with your usersTips for interacting with end users: 1. Listen 2. Watch 3. C t T t 3 Create Trust 4. Inform Designg
1. Listen Most important part of interviewing. You are not there to train the user or to demonstrate how much you know. You are interviewing an expert to gain knowledge. Treat them like a precious partner and remember they know a lot more about their work then you do.
2. Watch Remember users will tend to want to say what they think you want to hear. Create opportunities to observe users rather than ask users.
Case Study –A usability professional is interviewing a user:Professional: “Do you know how to set the margins?”User: “Oh yes, I do that all the time.”Professional: “Could you show me how to do it?”User: “Sure.” (user presses a series of buttons unrelated to setting margins – the button sequence is actually changing a different setting). “See it beeped so the margins are set ” See set.
3. Create Trust Users will be nervous that they will appear stupid or incompetent. “We are testing design, not you” To get good data, user must feel relaxed and trusting.
A user’s perspective:Well okay, today’s the day. I have to report to some building on 14th street. I must admit I m a bit nervous. When I spoke to the woman on the phone, she asked I’m me a whole lot of questions about my background and experience. She seemed particularly gleeful that I wasn’t competent using computers and equipment. I’m g q p glad she is happy but for me it’s a recurring p ppy g problem.I’ve always felt intimidated with electronics. She wants me to use something on the computer while some people watch me. Well, it’s an easy $50 bucks and seeing that I don’t know the people, it can’t be too embarrassing…On the other hand, what if I’m the first person in the world that doesn’t understand how to do whatever I’m supposed to do? What if I totally bomb? What if they ask me a question that is embarrassing and they find out how stupid I really am. Well, I’ll give it a go this time but I don’t think I can do this again.
4. Inform Design g User research does not dictate your design but rather informs you so that you design better. y y g
Woodblock Study Example y pUsers were asked to placestickers representing f ti k ti functions tiOn a block model.Resulting design did not copy wordfor word where the users placedbuttons.Resulting design was informed byhow users grouped buttons andby observation of users interactingwith the stickers.
Summary Four basic activities in the design process 1. Establishing requirements 2. Designing alternatives 3. Prototyping 4. 4 Evaluating User-centered design rests on three principles 1. 1 Early focus on users and tasks E l f dt k 2. Empirical measurement using quantifiable & measurable usability criteria 3. Iterative design75 www.id-book.com
Discovering Unmet NeedsDigital Cameras for AppleLate 80s – What would happen if we married a computer with a digital camera?Many studies of current photography use.Uncovering of unmet needs in traditional experience - not enough photos - not knowing whether photo good or not - ability t add sound bilit to dd d - ability to organizeFeatures determined through research and designedIteratively.
Other ProductsNotice all the iPod look-alikes?
Tools for Effective DesignPersonasScenariosStoryboards (comic strip, movie technique)Wireframes and Mock-ups ypPrototypes
Persona Technique P T hPersonas are a design tool to help visualize who you are designing for and imagine how this person will use the productA persona is an archetype that represents the behavior and goals of a group of usersBased on insights and observations from customer research
How to Develop a PersonaBrainstorming Imagining different people using the technologyDesign Research Ethnographic Studies Contextual Inquiry Field Studies Usability StudiesMarketing Research Focus Groups Sales Feedback
Gunther the Ad Guy Gunther is from Germany. He Travels extensively for work and As he is an advertising executive he needs to present concepts to clients quickly and easily. He is a person very well-versed in new technologies t h l i and wishes h had d i h he h d easier portable solutions for his presentations…..
How to use your PersonasUse them to find real people for usability testing.Use them as a foundation to discuss any designissues/feature issues that come up. /f hUse them to evaluate competitor’s products.Use them to keep you honest.Keep updating them as part of your continued userresearch.hUse them to find real customers you can have on-going relationships with with.
ScenariosUsage Scenarios are narrative descriptions of how the product meets the needs of a personaShort (2 pages max)Focus on unmet needs of personaConcrete storyCSet of stories around essential tasks, problems...Use to test ideas
Scenario TechniqueDevelop with personasPrioritize experiences to supportFocus on top three experiences but support other experiences
A business woman travels from SF to Paris on a business trip. On her way to the airport she narrowly misses a traffic delay She avoids delay. the jam because her Smartphone beeps and send her a warning text message on her route from the office to the airport.Upon arrival the location-sensitive Smartphone notifies the airline that she‘ll check in shortly and an airline employee finds her immediately and takes her baggage Her display shows that her baggage. flight is on time and provides a map to her gate. On the way she downloads tourist information (maps, events) for Paris. Symbian
Once found her seat, she begins t review th dO f dh t h b i to i the downloaded l d d information. She books a ticket for an opera she wants to see. Her Smartphone makes the booking using her credit card number p g g stored in memory. The security software of the Smartphone protects her against fraud.The Smartphone stores the opera booking along with emails written ith ritten on the plane. As soon as she steps off the plane, it makes the calls and sends the emails. As she leaves the airport, a map appears on the display and guides her to her hotel Symbian
StoryboardingSequence of sketches showing use of system in everyday use contextConcrete exampleEasier (faster) to grasp than text based storiesMeans of communication with users and system developers pSketches, not drawings...Use to test interaction and make sure design works
Turning Scenarios into Storyboards1. Photographs to match personas (can be used to communicate environment or profession)2. Drawings3. Draw-over acted photos4. Detail not important – communicate the experience rather than specifics specifics.
Wireframes & Mock UpsRoughly sketched sequences exploring and validating interaction design.Focus is on navigation and functionality.Usually explores one interaction but can also be used as the foundation for development of testing p prototypes. yp
Design and PrototypingD dP Identify needs/ establish requirements q(Re)Design Evaluate Build an interactive version Final product
Prototypes RedefinedPrototyping is common activity in productdevelopment.Traditional prototypes can take weeks or months.RAPID prototyping is a design techniqueto speed up d l d development of ideas. f dA rapid prototype can take as little as a few minutesto make.
Benefits of RAPID PrototypingFast and inexpensiveIdentifies problems before they re coded they’reElicits more and better feedback from usersHelps developers think creativelyGets users and other stakeholders involvedearly i th process l in theFosters teamwork and communicationAvoids i iA id opinion warsHelps decide design directions
Types of PrototypesLow Fidelity – quick and dirty, easy access materials like cardboard and paper.High Fidelity – more involved electronic versions similar in materials to final product.
Paper Prototyping (Low Fidelity)Quick and simple means of sketching interfacesUse ffiU office materials t i lEasier to criticize, quick to changeCreative process (develop in team)Can also use for usability test (focus on flow of interaction rather than visuals)Used a lot to test out concepts before real designbegins. g
Half-functional prototypes More hi-fi HTML ((or Powerpoint) ... Mostly precoded flow, but allows clicking and experiencing flow
Wireframing T l Wi f i ToolsPidoco (websites mobile, desktop) (websites, mobile No programming http://www.pidoco.comWireframeSketcher Plug-in for Eclipse http://wireframesketcher.com/ htt // i f k t h /Omnigraffle (Mac) Digramming tool http://www.omnigroup.com/products/omnigraffle/ http://www omnigroup com/products/omnigraffle/
Mobile Rapid Prototype ToolsAndroid: App Inventor http://appinventor.googlelabs.com/about/iPhone – Meta collection http://iphoneized.com/2009/11/21-prototyping- mockup-wireframing-tools-iphone-app-development/ mockup wireframing tools iphone app development/
Rapid PrototypingSpeed development time with quick hardware mockups handheld device connected to PC LCD screen USB phone keypad, Camera screen, keypadCan use PC development tools for rapid development Flash, Vi l Basic, Fl h Visual B i etc
‘Wizard-of-Oz’ Prototyping • The user thinks they are interacting with a computer, but a developer is responding to output rather than the system. • Usually done early in design to understand users’ expectations User >Blurb blurb >Do this >Why?116 www.id-book.com
Typical Iterations for HW/SW Product:1.1 Paper prototype to evaluate conceptual model2. Interactive computer-based prototype with rough screens to evaluate feature placement p3. Tethered prototype to evaluate button + screen interactions4. Real device prototyped with major features working5. Real device prototyped with all features working and graphic design implemented
Design and PrototypingD dP Identify needs/ establish requirements q(Re)Design Evaluate Build an interactive version Final product
ITERATIVE DESIGN PROCESS Design (redesign) PrototypeEvaluate Test Design with Users (Fix Issues)( i ) (Note problems)
MOBILE AUGMENTEDREALITY FOR SPATIAL NAVIGATION Sharon Brosnan 0651869 Bachelor of Science in Digital Media Design
BUNRATTY FOLK PARK Sharon Brosnan 0651869 Bachelor of Science in Digital Media Design
BUNRATTY FOLK PARKIrish visitor attraction run by ShannonHeritageg19th19 h century life is recreated lf dBuildings from the mid-west have beenrelocated t th 26-land surrounding Bunratty l t d to the 26 l d di B ttCastle30 buildings are set in a rural or village setting
AUGMENTED REALITYIn Bunratty Folk Park: Allows the visitor to point a camera at an exhibit, the device recognises its by it’s location and layers digital information on to f the display p y 3- dimensional virtual objects can be positioned with real ones on display Leads to dynamic combination of a live camera view and information
NAVIGATIONAL AIDSmartphone PlatformMost people carry mobile phones and are comfortable with themIdeal AId l Augmented Reality Technology d R li T h l Global tracking tools g Wireless communication capabilities Location based computing Large display for interaction
HUMAN CENTRED DESIGNGoal f the Navigational AidG l of th N i ti l Easy to use, clear and understandable Useful to visitors Creating interaction between the visitor and the aid through the user interface Engage the visitor E h iiTo ensure this… It is necessary to understand the visitor of a navigational aid in Bunratty Folk Park Identify visitor motives and goals while going through the Folk Park. P k
HUMAN CENTRED DESIGNUnderstanding the technology and Related Works Literature Similar Technologies g Electronic Tours in Museum Settings Interactions design in Outdoor Museums gUnderstanding the User over time Observations Ob Interviews te v ews
HUMAN CENTRED DESIGNFindings Most visitors do not use the map Most visitors have mobile phones Visitors want more information View the Folk Park at their own pace Information should be straight to the point Large social interaction within groups g g p
NEXT STEPS FROM RESEARCHDefine Visitors GoalsDefine Functionalities f theD fi F ti liti of th AidDevelop Personas – visitors who use the AidDevelop Scenarios – how the persona uses the p pnavigational aid in the Folk ParkDraw up Storyboards on scenarios
FUNCTIONALITYView Options Augmented Reality Camera View Features for Map View navigation List View Text Information 3D ObjectsSub OptionsSub-Options 3D Tour Guide Places 3D Placement of Events Buildings g Restaurants
ITERATIVE DESIGN PROCESSPrototyping and User Testing Low Fidelity Prototyping Sketches Paper Prototyping Post-It Prototyping PowerPoint Prototyping High Fidelity Prototyping Wikitude
INITIAL SKETCHES Pros: • Good for idea generation • Cheap • Concepts seem feasible Cons: • Not great feedback gained • Photoshop not fast enough for Photoshop not fast enough for making changes
POST IT PROTOTYPINGCamera View with 3DCamera View with 3DFirst Draft Second Draft Third Draft • Selection Selection • Home button added Home button added highlighted in blue for easy navigation to main menu
POWERPOINT PROTOTYPING Benefits Benefits • Used for User Testing • Interactive • Functionalities work when following the Functionalities work when following the story of Scenario 1 • Quick •E Easy arrangement of slides t f lid User Testing • Participants found • 15 minute sessions screen captured • ‘Talk Allowed’ technique used Talk Allowed technique used • Notes taken • Post Interview Post‐Interview
WIKITUDEPopular augmented realitybrowser for mobile devicesMappingPoint of Interest abilitiesMultiplatformM li l fShows the points of interestof Bunratty Folk Park Markers can be selected in and an information pop-up appears
WIKITUDEUser Testing Application well received Understandable Participants playful with the technology
FINAL DESIGN CONCEPTKey IssuesK I Fix issues found in previous sessions Design with guidelines in mind Appealing to the Mental Model Icon Design Aesthetic Design Colour/Font Buttons Look
VIDEO PROTOTYPEFlexible tool for capturing the use of an interfaceElaborate simulation of how the navigational aid will workDoes not need to be realistic in every detailevery detailGives a good idea of how the finished system will workfinished system will work
4. Evaluation Identify needs/ establish requirements i t (Re)Design Evaluate Build an interactive version Final product
What is evaluation?Evaluation is concerned withgathering d about the usability of h i data b h bili fa design or product by a specifiedgroup of users for a particular activitywithin a specified environment or workcontext
When to evaluate?Once the product has been developed p pros : rapid development, small evaluation cost p p , cons : rectifying problems redesign & design implementation evaluation reimplementationDuring design and development pros : find and rectify problems early fi d d if bl l cons : higher evaluation cost, longer development design implementation
Four evaluation paradigms‘quick a dirty’ qu c and tyusability testing (lab studies)field studiespredictive evaluation
Quick and dirty‘quick & dirty’ evaluation: informal feedbackfrom users or consultants to confirm that theirideas are in-line with users’ needs and are liked.Quick & dirty evaluations are done any time.Emphasis is on fast input to the design processrather than carefully documented findings.
Usability TestingRecording typical users’ performance on typical tasks incontrolled settings. Field observations may be used. g yAs the users perform these tasks they are watched & recordedon video & their key presses are logged.This data is used to calculate performance times, errors & helpexplain why the users did what they did.User satisfaction questionnaires & interviews are used to elicitusers’ opinions.
Laboratory-based studiesLaboratory-based studies can be used for evaluating the design, or the implemented system are carried out in an interruption free usability lab interruption-free can accurately record some work situations some studies are only possible in a lab environment some tasks can be adequately performed in a lab q yp are useful for comparing different designs in a controlled context
Portable equipment for the field154 www.id-book.com
Field StudiesField studies are done in natural settingsThe aim is to understand what users do naturally and yhow technology impacts them.In product design field studies can be used to:- identify opportunities for new technology- determine design requirements- decide how to introduce new technology- evaluate technology in use use.
Predictive EvaluationExperts apply their knowledge of typicalusers,users often guided by heuristics to predict heuristics,usability problems.Can involve theoretically based models.A key feature of predictive evaluation isthat users need not be presentRelatively quick and inexpensive
Characteristics of approaches Usability Field Predictive testing studiesUsers do task natural not involvedLocation controlled natural anywhereWhen prototype early prototypeData quantitative qualitative problemsFeed back measures & descriptions problems errorsType applied naturalistic expert
Evaluation approaches and methodsMethod Usability Field Predictive testing studiesObserving x xAsking x xusersAsking x xexpertsTesting xModeling x
Living labs People’s use of technology in their everyday lives can be evaluated in living labs. g Such evaluations are too difficult to do in a usability lab lab. Eg the Aware Home was embedded with a complex network of sensors and audio/video recording devices (Abowd et al., 2000).159 www.id-book.com
DECIDE: A framework to guide evaluation- Determine the goals the evaluation addresses.- Explore the specific questions to be answered.- Choose the evaluation p di and t h i Ch th l ti paradigm d techniques- Identify the practical issues.- Decide how to deal with the ethical issues.- Evaluate, interpret and present the data.
DECIDE FrameworkDetermine Goals:D G l What are the high-level goals of the evaluation? How wants the evaluation and why? H t th l ti d h ?Explore the Questions: Create well defined, relevant questionsChoose the Evaluation Paradigm Influences the techniques used, how data is analyzedIdentify Practical Issues How to select users, stay on budget & schedule How to find evaluators, select equipment
DECIDE FrameworkDecide on Ethical Issues Informed consent form Participants have a right to: -kknow th goals of th study and what will h the l f the t d d h t ill happen to the fi di t th findings - privacy of personal informationEvaluate, Interpret and Present Data , p- Reliability: can the study be replicated?- Validity: is it measuring what you thought? y g y g- Biases: is the process creating biases?- Scope: can the findings be generalized?- E l i l validity: is the environment influencing the results? Ecological lidit i th i t i fl i th lt ?
Key K points • Many issues to consider before conducting an evaluation study. • These include: goals of the study; involvment or not of users; the methods to use; practical & ethical issues; how data will be collected analyzed & collected, presented. • Th DECIDE f The framework provides a useful checklist k id f l h kli for planning an evaluation study.164 www.id-book.com
Usability Testing U bl TRecording performance of typical users doing typical tasks tasks.Controlled settings.Users are observed and timed.U b d d dData is recorded on video & key presses are logged.The data is used to calculate performance times, and toidentify & explain errors.User satisfaction is evaluated using questionnaires &interviews.Field observations can provide contextual understanding.166 www.id-book.com
Experiments & Usability Testing Experiments test hypotheses to discover new knowledge by investigating the relationship between g y g g p two or more things – i.e., variables. Usability testing is applied experimentation. y g pp p Developers check that the system is usable by the intended user population for their tasks. p p Experiments may also be done in usability testing.167 www.id-book.com
Usability Testing Research U bilit T ti & R h Usability testing y g Experiments for p research Improve products Discover knowledge Few participants Many participants Results inform design Results validated Usually U ll not completely l l statistically replicable Must be replicable Conditions controlled as Strongly controlled much as possible conditions Procedure planned p Experimental design p g Results reported to Scientific report to developers scientific community168 www.id-book.com
Pilot StudiesA small trial run of the main study. Can identify majority of issues with interface designPilot studies check:- that the evaluation plan is viable- you can conduct the procedure- that i h interview scripts, questionnaires, i i i i experiments, etc. work appropriatelyIron out problems before doing the main study.
Controlled experimentsDesigner of a controlled experiment shouldcarefully consider y proposed hypothesis selected subjects measured variables experimental methods data collection data analysis
SubjectsThe hTh choice of subjects is critical to the validity of f b l h ld fthe results of an experiment subjects group should be representative of the expected user populationIn selecting the subjects it is important toconsider things such as their g age group, education, skills, cultureThe sample size should be large enough (10+) tobe statistically representative of the population
Variables V i blExperiments manipulate and measure variablesunder controlled conditionsThere are two types of variables independent: variables that are manipulated to create different experimental conditions - e g number of items in menus, colour of the icons e.g. menus dependent: variables that are measured to find out the effects of changing the independent variables - e.g. speed of menu selection, speed of locating icons
“Other” VariablesControl variables e.g. room light, noise… if controlled => less external validityRandom variables (not controlled) ( ) e.g. fatigue more influence of random variable => less internal validity yConfounding variables practice previous experience
HypothesisA hypothesis is a prediction of theoutcome what will happen to the dependent variables when the independent variables are changed to show that the prediction is right - dependant variables don’t change by changing the independent variables - rejecting the null hypothesis (H0 )
Experimental methodsIt is important to select the right experimental pmethod so that the results of the experimentcan be generalizedThere are mainly two experimental methods between-groups: each subject is assigned to one experimental condition within-groups: each subject performs under all the different conditions
Experimental methodsBetween-groups g p Within-groups g p Subjects Subjects Randomly Randomly assigned assigned rimental tasks rimental tasks rimental tasks Condition Condition Condition erimental task 1 2 3 Condition Condition Condition Condition Condition Condition 1 2 3 2 1 1 Expe Exper Exper Exper Condition Condition Condition 3 3 2 data data data data data data Statistical data analysis Statistical data analysis
Within vs. Between Subjectsbetween subjects design each participant is tested on only one level/condition p p y a separate group of participants is used for each condition - one group uses HMD other group uses Handheld devicewithin subjects d i i hi bj design participant is tested on each level/condition - e.g. participants use Handheld device and HMD repeated measurement p
Between SubjectsSometimes a factor must be between subjects e.g. gender, age, experienceBetween subjects advantage: avoids interference effects (e.g. practice / learning effect) ( g p g )Between subjects disadvantage: Increased variability = need more subjectsImportant: randomised assignment to conditions
Within SubjectsSometimes a f t must be within subjectsS ti factor t b ithi bj t e.g. measuring learning effectsWithin subjects advantages less participants needed (all p p p ( participants in all conditions) p ) differences (variability) between subjects the same across test conditionsCounterbalance order of presenting conditions A => B => C B => C => A C => A => BThe order is best governed by a Latin Square
Latin Square Designeach condition occurs once in each row andcolumnNote: In a balanced Latin Square each conditionbothb th precedes and f ll d d follows each other condition h th ditian equal number of times
Data collection and analysisThe choice of a method is dependent on thetype of data that needs to be collectedIn order to test a hypothesis the data has tobe analysed using a statistical methodThe choice of a statistical method depends onthe type of collected data h f ll dd All the decisions about an experiment should p be made before it is carried out
Observe and MeasureObservations are gathered… manually ( y (human observers) ) automatically (computers, software, cameras, sensors, etc ) sensors etc.)A measurement is a recorded observationObjective metricsSubjective metrics
Typical objective metricstask completion time k l i ierrors (number, percent,…)percent of task completedratio of successes to failuresnumber of repetitionsnumber of commands usednumber of failed commandsphysiological data (heart rate,…)…
Typical subjective metricsuser satisfactionsubjective performanceratingsease of useintuitivenessjudgments…
Data TypesSubjective Subjective survey How easy was the task - Likert Scale, condition rankings Scale 1 2 3 4 5 Not very easy Very easy Observations - Think Aloud Interview responsesObjective Performance measures - Time accuracy, errors Time, accuracy Process measures - Video/audio analysis
Experimental Measures E erimental Meas res Measure What does it tell us? How is it measured?Timings Performance Via a stopwatch, or automatically by the device.Errors Performance, Particular sticking points in a task By success in completing the task correctly. Through experimenter observation, examining the route walked.Perceived Workload Effort invested. User satisfaction Through NASA TLX scales and other questionnaires. i iDistance traveled and route Depending on the application, these can be used Using a pedometer, GPS or othertaken to pinpoint errors and to indicate performance location-sensing system. By experimenter observation.Percentage preferred walking Performance By finding average walking speed,speed which is compared with normal walking speed.Comfort User satisfaction. Device acceptability Comfort Rating Scale and other questionnaires.User comments and User satisfaction and preferences. Particular Through questionnaires, interviews andpreferences sticking points in a task. think alouds. think-alouds.Experimenter observations Different aspects, depending on the experimenter Through observation and note-taking and on the observations
Statistical AnalysisOnce data is collected statistics can be used for analysisTypical Statistical Techniques yp q Comparing between two results - Unpaired T-Test (for between subjects – assumes normal distribution, interval scale, homogeneity of variances) - Paired T-Test (for within subjects – assumes normal distribution, etc.) - Mann Whitney U test (between subjects – if assumptions are not met) Mann–Whitney U-test Comparing between > two results - Analysis of Variance – ANOVA - Followed by post-hoc analysis – Bonferroni adjustment - Kruskal–Wallis (does not assume normal distribution)
Case Study: A Wearable Information Space Head Stabilized Body StabilizedAnA AR i t f interface provides spatial audio and visual cues id ti l di d i lDoes a spatial interface aid performance? –Task time / accuracy
Task PerformanceTaskT k find target icons on 8 pages remember information space b i f iConditionsA - head-stabilized pagesB - cylindrical display with trackballC - cylindrical display with head trackingSubjects Within subjects (need fewer subjects) 12 subjects used
Experimental MeasuresObjectiveOb spatial ability (pre-test) Many time to perform task information recall Different workload (NASA TLX) kl d MeasuresSubjective j Post Experiment Survey - rank conditions (forced choice) ( ) - Likert Scale Questions • “How intuitive was the interface to use?”
Post Experiment SurveyFor each of these conditions please answer: 1) How easy was it to find the target? 1 2 3 4 5 6 7 1=not very easy 7=very easyFor the head stabilised condition (A):For the cylindrical condition with mouse input (B):For the head tracked condition (C):Rank all the conditions in order on a scale of one to three1) Which condition was easiest to find target (1 = easiest, 3 = hardest) A: B: C:
ResultsBody St bili ti Improved P fB d Stabilization I d Performance search times significantly faster (One factor ANOVA)Head Tracking Improved Information recall no difference between trackball and stack caseHead tracking involved more physical work
Subjective Impressions 5 4.5 4 3.5 3 Find Target 2.5 Enjoyable 2 1.5 15 1 0.5 0 A B CSubjects Felt Spatialized Conditions ( j p (ANOVA): ) More enjoyable Easier to find target
Subjective Impressions 3 2.5 2 Easiest 1.5 Understanding Intuitive 1 0.5 0 A B CSubject Rankings (Kruskal-Wallis) Spatialized easier to use than head stabilized S l d h h d bl d Body stabilized gave better understanding Head tracking most intuitive
Field S d F ld Studies Field studies are done in natural settings settings. “in the wild” is a term for prototypes being used freely in natural settings settings. Aim to understand what users do naturally and how technology impacts them them. Field studies are used in product design to: - identify opportunities for new technology; - determine design requirements; - decide how best to introduce new technology;gy; - evaluate technology in use.196 www.id-book.com
ObservationDirect observation in the field Structuring frameworks Degree of participation (i id or outsider) D f ti i ti (insider t id ) EthnographyDirect observation in controlled environmentsD b ll dIndirect observation: tracking users’ activities Diaries Interaction logging
Ethnography (1)• Ethnography is a philosophy with a set of techniques that include participant observation and interviews• Debate about differences between participant observation and ethnography• Ethnographers immerse themselves in the culture that they study• A researcher’s degree of participation can vary along a scale from ‘outside’ to ‘inside’ ‘ d ’ ‘ d ’• Analyzing video and data logs can be time-consuming• Collections of comments, incidents, and artifacts are made
Ethnography (2)Co operationCo-operation of people being observed is requiredInformants are usefulData analysis is continuousInterpretivist techniqueQuestions get refined as understanding growsReports usually contain examples
Direct observation in a controlled setting g Think-aloud technique qIndirect observation Diaries Interaction logs Cultural C lt l probesb
Frameworks to Guide Observation - The person. Who? - The place. Where? - The thing What? thing. The Goetz and LeCompte (1984) framework: - Who is present? - What is their role? - What is happening? - When does the activity occur? - Wh i it happening? Where is h i ? - Why is it happening? - How is the activity organized?201 www.id-book.com
Predictive modelsProvide a way of evaluating products ordesigns without directly involving users. g y gLess expensive than user testing.Usefulness limited to systems withpredictable tasks - e.g., telephone answeringsystems, mobiles,systems mobiles cell phones etc phones, etc.Based on expert error-free behavior.
Fitts’ Law (Fitts 1954)Fitts’ Law predicts that the time to point at anobject using a device is a function of the distancefrom the target object and the object’s size.The further away and the smaller the object, the objectlonger the time to locate it and point to it.
GOMS ModelGoals hG l - the state the user wants to achieve e.g., find a h hi fi dwebsite.Operators - the cognitive processes and physical actionsneeded to attain the goals Eg moving mouse to select icon g gMethods - the procedures for accomplishing the goals, e.g.,drag mouse over icon, click on button.Selection rules - decide which method to select when there ismore than one.
GOMS Response Times (Card et al., 1983) Operator Description Time (sec) K Pressing a single key or button g g y Average skilled typist (55 wpm) 0.22 Average non-skilled typist (40 wpm) 0.28 Pressing shift or control key 0.08 Typist unfamiliar withthekeyboard with the keyboard 1.20 120 P Pointing with a mouse or other device on a 0.40 display to select an object. This value is derived fromFitts’ Law which is discussed below. P1 Clicking the mouse or similar device 0.20 H Bring ‘home’ hands on the keyboard or other 0.40 device M Mentally prepare/respond 1.35 R(t) The response time is counted only if it causes t the user to wait.
Expert InspectionsSeveral kinds.Experts use their knowledge of users and p gtechnology to review software usability.Expert critiques can be formal or informal reports. p q pHeuristic evaluation is a review guided by a set ofheuristics.Walkthroughs involve stepping through a pre-pplanned scenario noting potential problems. gp p
Nielsen’s heuristicsVisibility of system status status.Match between system and real world.User control and freedom freedom.Consistency and standards.Error prevention.ERecognition rather than recall.Flexibility and efficiency of use.Aesthetic and minimalist design.gHelp users recognize, diagnose, recover from errors.Help and documentation.
Three stages for doing heuristic evaluation Briefing session to tell experts what to do. Evaluation period of 1-2 hours in which: Each expert works separately; Take one pass to get a feel for the product; Take a second pass to focus on specific features. Debriefing session in which experts work together to prioritize problems. g p p
Advantages and problemsFew ethical and practical issues to considerbecause users not involved.Can be difficult and expensive to find experts.Best experts have knowledge of applicationdomain and users.Biggest problems: Important problems may get missed; Many trivial problems are often identified; Experts have biases.
Overview• Fi k i Five key issues of d gathering f data h i • Data recording • Interviews • Questionnaires • Observation • Choosing and combining techniques213 www.id-book.com
Five K I F Key Issues 1. Setting goals - Decide how to analyze data once collected 2. Identifying participants - Decide who to gather data from 3. Relationship with participants - Clear and professional - Informed consent when appropriate pp p 4. Triangulation - Look at data from more than one perspective 5. Pilot studies - Small trial of main study214 www.id-book.com
Data recording Notes, audio, video, photographs Notes plus photographs Audio plus photographs Video215 www.id-book.com
Interviews • Unstructured - are not directed by a script. Rich but not replicable. • St Structured - are ti htl scripted, often like a t d tightly i t d ft lik questionnaire. Replicable but may lack richness. richness • Semi-structured - guided by a script but interesting issues can be explored in more depth. Can provide a good balance between richness and replicability.216 www.id-book.com
Interview Q i Questions i • Two types: − ‘closed questions’ have a predetermined answer format, e.g., ‘yes’ or ‘no’ − ‘open questions’ do not have a predetermined format • Closed questions are easier to analyze • A id Avoid: − Long questions − Compound sentences - split them into two − Jargon and language that the interviewee may not understand − Leading questions that make assumptions e.g., why do you like …? − Unconscious biases e.g., gender stereotypes eg217 www.id-book.com
Enriching the interview process• Props - devices for prompting interviewee, e.g., a prototype, scenario218 www.id-book.com
Questionnaires• Key Points • Can be administered to large populations • Paper email and the web used for dissemination Paper, • Sampling can be a problem when the size of a population is unknown as is common online• Design • Provide clear instructions • Questions can be closed or open • closed easier to analyze • The impact of a question can be influenced by order 219 Will phrases be positive, negative or mixed • www.id-book.com
Encouraging a Good Response Make sure purpose of study is clear y y Promise anonymity Ensure questionnaire is well designed Offer a short version for those who do not have time to complete a long questionnaire Follow-up with emails, phone calls, letters Provide an incentive 40% response rate is high, 20% is often acceptable p g p220 www.id-book.com
Online Questionnaires• www.surveymonkey.com k• Good • Low cost • Responses are usually received quickly • Data automatically collected • Time required for data analysis is reduced• Bad • Sampling is problematic if population size is unknown • Preventing individuals from responding more than once 221 www.id-book.com
6. Design in the Real World Mark Billinghurst
Product Development Balancing Act Marketing g Software Hardware Industrial Design Interaction Design Usability Manufacturing
Obstacles to SuccessEveryone has an opinionE h i i Design by committeeMisunderstanding user research Too literal, not objectivePower struggles Blue vs. Red InterfaceLosing focus of the end user Feature CreepRelying on assumptions/mythsFeaturitis = Requirements not well established
Penny Wise = Dollar PoorSoftware/Hardware development rewarded gfor fast work and low budgets.Interaction design activities take some budgetand some time time.Investment at front end saves money overall.
Cost Justifying UsabilityCost of ChangesDesign Alternatives Req. Dev. Build Phase Phase Phase Saving Development Costs
Cost Justifying UsabilityIncrease Revenue “IBM’s Web presence has traditionally been made up of a difficult-to-navigate labyrinth of disparate subsidies, but a redesign made it more cohesive and subsidies user-friendly. According to IBM, the massive redesign effort quickly paid dividends. The company said in the month after re-launch that traffic to the Shop IBM online store increased 120 percent and sales went up 400%400%. (Battey, 1999)Increase Safety “Chapanis cites two independent studies that showed a 54% reduction In rear-end accidents with the use of human factors improvement: the centered high mount brake light on autos.” high-mount autos.
More Value for Interaction Design1. US trade magazines score products on usability along with features and cost.2. 1/3 of new products fail in market place. (PDMA)3. Some big electronic chain stores do their own usability review of products before they will stock them.4. Customers have become very sophisticated and DEMAND highly usable products (as do distributors, investors). distributors investors)5. Enables companies to develop on-going relationships with customers to keep them current. p
Stories from the Field Connectix for BoeingAssumption that users wouldprimarily be business travellerswanting to work.Studies showed primary userwere indeed business travellersbut wanting to RELAX.
BooksInteraction Design by Jenny Preece, YvonneRogers, and Helen Sharp g pAbout Face 2.0: The Essentials of InteractionDesign by Alan Cooper and Robert M. Reimann MMobile Interaction Design by Matt Jones andGary Marsden
Resources: More booksDesigning Interactions. Bill MoggridgeThe Inmates Are Running the Asylum: Why High-Tech Products Drive Us Crazy and How to Restore the Sanity. Alan CooperInterface Culture. Steven Johnson (history of the development of interfaces, well written, more on the ideas)The invisible Computer. D ld NormanTh i i ibl C Donald NRapid Contextual Design. A How-to Guide to Key Techniques for User- Centered Desi n Karen H lt blatt Design. HoltzblattDesign Research. Brenda Laurel
Resources: ConferencesOzCHICHI (Computer Human Interaction, ACM)NordiCHI (Nordic CHI, ACM)MobileHCI (focused on mobile interaction/devices, ACM)IDC (Interaction Design and Children, ACM)InteractHCI InternationalDIS (Designing Interactive Systems, ACM) - Design Research, open to different approaches t d i diff t h to designDPPI (Designing Pleasurable Products) - Product Design ConferenceDUX (Designing the User Experience, ACM) - more practicionersCSCW (Computer Supported Cooperative Work)UbiComp
Onlinehttp://www.interaction-design.org/ Interaction Design online p g g gencyclopediahttp://www.baddesigns.com/ scrapbook of illustrated examplesof things that are hard to usehttp://www.ixda.org/en/ Interaction Design Associationhttp://dev.uxmatters.com/ (User Experience Matters - blog andnewsletter) l )http://www.nathan.com/ed/index.html (Nathan Shedroff‘swebsite on Experience Design)Web-Based User Interface Evaluation with Questionnaires, byGary Perlman http://www.acm.org/~perlman/question.htmlhttp://usability.gov/ US Dh // bili / Dept. Of Health and H H l h d Human Service S ihttp://www.paperprototyping.com/ (adjunct homepage to bookon paper p p p prototyping) yp g)
InstitutionsD-School - Stanford University (USA) http://www.stanford.edu/group/dschool/IIT Institute of Design (USA) http://www.id.iit.edu/CMU School of Design (USA) www.design.cmu.edu/MIT Media Lab (USA) www.media.mit.eduInteraction Design Institute + Domus Academy (Italy)I i D i I i D A d (I l ) http://www.interaction-ivrea.it/Interaction Design, Royal Co ege of Art (UK) te act o es g , oya College o t (U ) http://www.interaction.rca.ac.uk/University of Queensland (Australia) http://www.itee.uq.edu.au/~id/ htt // it d / id/
More Information• Mark Billinghurst – mark billinghurst@hitlabnz org email@example.com• Website – www.hitlabnz.org