Beyond User Research

Beyond user researchBuilding an organizational brainLou Rosenfeld • Rosenfeld Media • rosenfeldmedia.com UX Hong Kong • 17 February, 2012

I have always disrespected silos

Where do insights live?

The reports from theuser research group

Query data gleaned from site search teamXXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XX X.XXX.X.104 HTTP/1.1" 200 8283 146 0.16XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxysty lesheet=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17

The logs from thecall center

The reports coming out of theanalytics applications

Insights fromVoice of the Customer research

The reports coming from theCRM applications

The insights coming out of theresearch center

One agency’suser mental model

Another agency’sbrand architecture research

The surveys behindNet Promoter Score

We’re tilting at silos It’s bad news. Organizations are... • Dramatically overpaying for research • Duplicating effort • Missing out on the combinatorial effect: the sum won’t be greater than the parts

A three-partcartographic challenge

A three-partcartographic challenge1. Fragmentation Where is an organization’s research?

A three-partcartographic challenge1. Fragmentation Where is an organization’s research?2. Differentiation What kind of research is there?

A three-partcartographic challenge1. Fragmentation Where is an organization’s research?2. Differentiation What kind of research is there?3. Synthesis How might we combine it effectively?

What Why

Methods employed:quantitative versus qualitative

Goals: help org or users Organizational goals Users’ goals

Measuring the world we know measur How they use data: world we know versus woExploring the world we don’t we don’t

Kind of data they use: statisticalvs. descriptive Statistical data Descriptive data

Lou’s TABLE OFOVERGENERALIZED Web Analytics User Experience DICHOTOMIES Users intentions and What they Users behaviors (whats motives (why those things analyze happening) happen) Qualitative methods for What methods Quantitative methods to explaining why things they employ determine whats happening happen Helps users achieve goals What theyre Helps the organization meet (expressed as tasks ortrying to achieve goals (expressed as KPI) topics of interest) Uncover patterns and How they use Measure performance (goal- surprises (emergent data driven analysis) analysis) Statistical data ("real" data Descriptive data (in smallWhat kind of data in large volumes, full of volumes, generated in lab they use errors) environment, full of errors)

Without a map, why bother?

We can learn fromeach other’s data

XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxy stylesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02

XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxy stylesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip =XXX.XXX.X.104 HTTP/1.1" 200 8283 146 0.16

XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxy stylesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip =XXX.XXX.X.104 HTTP/1.1" 200 8283 146 0.16 One question “Are we converting license plate renewals?”

XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxy stylesheet=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip =XXX.XXX.X.104 HTTP/1.1" 200 8283 146 0.16 One question “Are we converting license plate renewals?” Another question “What are people searching the most?”

We can improveeach other’s design tools

We can help telleach other’s stories

Google Analytics goes beyondreports

We can help solve each other’sdesign problems

Lands End and SKUs

Lands End and SKUs SKU: # 39072-2AH1

We can test each other’shypotheses

What wassearched

Searched What was searched

Searched What was searched Viewed

Searched What was searched Viewed Added to queue

The organization’s challenge:thinking with a whole brain

How might practitionersbuild a whole brain?Some recommendations

Get out of your silo

Establish what’s common KPI goals segments personasDave Gray’s boundary matrix: http://bit.ly/gWoZQm

Map itMaps, after all, are designedobjects

Landscape of User Research Methods Rohrer/Mulder/Yaar’s map Behavioral / Eyetracking Data Mining/Analysis A/B (Live) Testing Usability Benchmarking (in lab) / Data Source Usability Lab Studies Online User Experience Assessments (“Vividence-like” studies) Ethnographic Field Studies mix Text Diary/Camera Study Message Board Mining Participatory Design Customer feedback via email Focus Groups Desirability studies Intercept Surveys Attitudinal Phone Interviews Cardsorting Email Surveys mix Qualitative (direct) Approach Quantitative (indirect) Key for Context of Product Use during data collection Natural use of product De-contextualized / not using product© 2008 Christian Rohrer Scripted (often lab-based) use of product Combination / hybrid 20Christian Rohrer: http://bit.ly/eAlbe2 / Steve Mulder & Ziv Yaar, The User Is Always Right

Avinash Kaushik: http://bit.ly/cabRYmKaushik’s map

How might decision-makersdecide with a whole brain?Some recommendations

Blue sky it If you were going to build your organization’s brain-- its decision-making capability --from scratch-- what would it look like?

Ban it Useless, misunderstood terms impede progress • Product names: Omniture, SharePoint... • Methods: focus group, usability test... • Departments: market research, analytics... • Disciplines: business analysis, information architecture...

Lou’s TABLE OFOVERGENERALIZED Web Analytics User Experience DICHOTOMIES Users intentions and What they Users behaviors (whats motives (why those things analyze happening) happen) Qualitative methods for What methods Quantitative methods to explaining why things they employ determine whats happening happen Helps users achieve goals What theyre Helps the organization meet (expressed as tasks ortrying to achieve goals (expressed as KPI) topics of interest) Uncover patterns and How they use Measure performance (goal- surprises (emergent data driven analysis) analysis) Statistical data ("real" data Descriptive data (in smallWhat kind of data in large volumes, full of volumes, generated in lab they use errors) environment, full of errors)

Design it

Win it Companies that integrate their research will achieve true insight and destroy their competitors. You are the one who can make it happen.

Say helloLou RosenfeldRosenfeld Media www.louisrosenfeld.com | @louisrosenfeldwww.rosenfeldmedia.com | @rosenfeldmediawww.slideshare.net/lrosenfeld/beyond-user-research

http://louisrosenfeld.com	1408
http://ixdabh.org	1405
http://www.scoop.it	567
http://paper.li	89
http://www.louisrosenfeld.com	51
http://sprmario.tumblr.com	26
http://humansindesign.com	25
http://www.borjasantaolalla.com	23
http://sprmario.posterous.com	22
http://flavors.me	16
http://lanyrd.com	15
http://lieveld.nl	13
http://translate.googleusercontent.com	11
http://library.iasummit.org	11
http://humansindesign.tumblr.com	9
http://jbaeck.posterous.com	9
http://a0.twimg.com	8
http://www.slideshare.net	5
http://twitter.com	4
http://www.twylah.com	3
url_unknown	3
http://webcache.googleusercontent.com	3
http://us-w1.rockmelt.com	3
http://feeds.feedburner.com	2
http://www.ofertadeviaje.org	2
http://www.stereoprototype.com	2
http://www.tumblr.com	2
http://www.keytobiz.com	1
http://www.mongodb.org	1
http://tweetedtimes.com	1
http://www.hanrss.com	1
http://www.ofertadeviajes.eu	1
http://dashboard.bloglines.com	1
http://fanyi.youdao.com	1

Beyond User Research

by Louis Rosenfeld on Mar 07, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

34 Embeds 3,744

Statistics

Beyond User Research — Presentation Transcript