It starts with a guy introducing himself…             Hey, I’m Jeff Pass.             I’m the one that mentioned          ...
Then he makes his pitch…     I’m working on a presentation for the IA Summit.     It’s called, “When Too Many Is Just Enou...
Then he gets to the point...       So today I want to talk about online card sorts.       Open, closed, reverse, it really...
So, you’ve got yourself a card sort…
How many participants do you need? (1)   According to Optimal Workshop, for an open card sort you need:
How many participants do you need? (2)   According to Optimal Workshop, for a closed card sort you need:
How many participants do you need? (3)         And there are lots of other opinions…
So how many did we do?       ???
Over 1109 (1110 to be precise)
But why? What were you thinking?                 The 2012 Digital Government                 Strategy calls for large-scal...
And how did you engage them?                                      Directly…                           Social media was our...
So what do you think?Share your thoughts and experiences about large-scale usability studies and direct user engagement   ...
Thanks for your time & participation!          Jeffrey Ryan Pass          Lead User Experience Consultant          Aquilen...
Upcoming SlideShare
Loading in...5
×

When Too Many is Just Enough - Jeff Pass at UXCamp DC 2013

1,608

Published on

Discussion of why it is sometimes desirable to have more than the required (either in terms of validity or statistical significance) number of participants in card sorts and similar usability exercises. Specifically, how large-scale usability exercises allow usability professionals and site owners to reach out to a broad audience and include them in the usability lifecycle.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,608
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

When Too Many is Just Enough - Jeff Pass at UXCamp DC 2013

  1. 1. It starts with a guy introducing himself… Hey, I’m Jeff Pass. I’m the one that mentioned male impregnation during my presentation at last year’s UX Camp DC… remember me?
  2. 2. Then he makes his pitch… I’m working on a presentation for the IA Summit. It’s called, “When Too Many Is Just Enough: Citizen Engagement and Federal Government Websites.” I’d like to workshop part of it with you. Here’s the (boring) summary: Statistical significance is an important consideration in usability studies. You need a certain minimum sampling to ensure valid results, but at the same time, too large a sampling creates more work without increasing significance or validity. Nevertheless, there is a case to be made for casting a wide net and engaging a larger audience in order to actively engage users.
  3. 3. Then he gets to the point... So today I want to talk about online card sorts. Open, closed, reverse, it really doesn’t matter… any kind of card sort you like as long as it is online. More to the point, I want to talk about how many participants you want/need.
  4. 4. So, you’ve got yourself a card sort…
  5. 5. How many participants do you need? (1) According to Optimal Workshop, for an open card sort you need:
  6. 6. How many participants do you need? (2) According to Optimal Workshop, for a closed card sort you need:
  7. 7. How many participants do you need? (3) And there are lots of other opinions…
  8. 8. So how many did we do? ???
  9. 9. Over 1109 (1110 to be precise)
  10. 10. But why? What were you thinking? The 2012 Digital Government Strategy calls for large-scale citizen engagement. Increased participation doesn’t increase validity (and can complicate analysis and reporting), but it does allow us to directly engage the wider public in usability studies and give them a voice in organizing the (federal) websites that are meant to serve them.
  11. 11. And how did you engage them? Directly… Social media was our recruiter A blog post was our screener OptimalSort and TreeJack were our test vehicles**And yes, we did include a free text comment option and yes, analyzing the results wasonerous, but the additional effort was well worth it because while quantitative data (the sortresults) had pretty well stabilized by 25-30 participants, every comment provided valuablequalitative data.
  12. 12. So what do you think?Share your thoughts and experiences about large-scale usability studies and direct user engagement But enough about me; what do you think about me? Err… I mean… what do you think about large-scale usability studies, direct user engagement, and too many being just enough?
  13. 13. Thanks for your time & participation! Jeffrey Ryan Pass Lead User Experience Consultant Aquilent (www.aquilent.com) jeff.pass@aquilent.com @jeffpass UX Camp DC 2013
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×