When Too Many is Just Enough - Jeff Pass at UXCamp DC 2013

It starts with a guy introducing himself… Hey, I’m Jeff Pass. I’m the one that mentioned male impregnation during my presentation at last year’s UX Camp DC… remember me?

Then he makes his pitch… I’m working on a presentation for the IA Summit. It’s called, “When Too Many Is Just Enough: Citizen Engagement and Federal Government Websites.” I’d like to workshop part of it with you. Here’s the (boring) summary: Statistical significance is an important consideration in usability studies. You need a certain minimum sampling to ensure valid results, but at the same time, too large a sampling creates more work without increasing significance or validity. Nevertheless, there is a case to be made for casting a wide net and engaging a larger audience in order to actively engage users.

Then he gets to the point... So today I want to talk about online card sorts. Open, closed, reverse, it really doesn’t matter… any kind of card sort you like as long as it is online. More to the point, I want to talk about how many participants you want/need.

So, you’ve got yourself a card sort…

How many participants do you need? (1) According to Optimal Workshop, for an open card sort you need:

How many participants do you need? (2) According to Optimal Workshop, for a closed card sort you need:

How many participants do you need? (3) And there are lots of other opinions…

So how many did we do? ???

Over 1109 (1110 to be precise)

But why? What were you thinking? The 2012 Digital Government Strategy calls for large-scale citizen engagement. Increased participation doesn’t increase validity (and can complicate analysis and reporting), but it does allow us to directly engage the wider public in usability studies and give them a voice in organizing the (federal) websites that are meant to serve them.

And how did you engage them? Directly… Social media was our recruiter A blog post was our screener OptimalSort and TreeJack were our test vehicles**And yes, we did include a free text comment option and yes, analyzing the results wasonerous, but the additional effort was well worth it because while quantitative data (the sortresults) had pretty well stabilized by 25-30 participants, every comment provided valuablequalitative data.

So what do you think?Share your thoughts and experiences about large-scale usability studies and direct user engagement But enough about me; what do you think about me? Err… I mean… what do you think about large-scale usability studies, direct user engagement, and too many being just enough?

Thanks for your time & participation! Jeffrey Ryan Pass Lead User Experience Consultant Aquilent (www.aquilent.com) jeff.pass@aquilent.com @jeffpass UX Camp DC 2013

http://lanyrd.com	40
http://informatica4c182013.blogspot.com.es	13

When Too Many is Just Enough - Jeff Pass at UXCamp DC 2013

by JeffreyRyanPass on Jan 14, 2013

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 53

Statistics

When Too Many is Just Enough - Jeff Pass at UXCamp DC 2013 Presentation Transcript