User Zoom Webinar Monster Aug09 Vf
 

User Zoom Webinar Monster Aug09 Vf

on

  • 1,726 views

Presentation on how UserZoom conducts remote usability testing. Special guest Monster explains how they use our solution.

Presentation on how UserZoom conducts remote usability testing. Special guest Monster explains how they use our solution.

Statistics

Views

Total Views
1,726
Views on SlideShare
1,724
Embed Views
2

Actions

Likes
0
Downloads
13
Comments
0

1 Embed 2

http://www.slideshare.net 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • In this webinar, we will show you how you can remove face to face moderation, have hundred of users participate simultaneously from different locations in a cost effective and fast way. Userzoom is an on demand research service that will automatically capture and gather both customers attitudes and behaviors. After our demonstration, we will open up to Questions, but let’s start now by diving into UserZoom software as a service.
  • In this webinar, we will show you how you can remove face to face moderation, have hundred of users participate simultaneously from different locations in a cost effective and fast way. Userzoom is an on demand research service that will automatically capture and gather both customers attitudes and behaviors. After our demonstration, we will open up to Questions, but let’s start now by diving into UserZoom software as a service.
  • Thank you (Alfonso? Viginie?) I’m Denise Nangle, User Experience Research Manager at Monster. Today, I’d like to share with you how Userzoom helped Monster teams evaluate the success of our total redesign of the seeker experience. Before the launch of this complete redesign, Monster’s strategy was to make incremental changes to the user interface. Since REDUX was a complete web site redesign, with an entirely new user experience and new products, it offered us the opportunity to evaluate the entire experience, and not just one or two parts of the site. To do this, we needed the right tool that would allow us to evaluate both the strengths and weaknesses. So, we called upon Userzoom and started planning a remote, international online study. In late January 2009, we launched identical studies in the US, UK, France, Germany and the Netherlands.
  • We knew we wanted to measure several areas, including user satisfaction, task success and efficiency, ease of use and likelihood of using Monster in the future. But we also wanted to gather qualitative feedback on what our users liked and didn’t like about the new experience. Since we were going to interpret the results of this study in context with with other remote studies done prior to the launch of REDUX, we had to structure the study to obtain comparative measures. We recruited panels of 250 people in each of the 5 markets we wanted to evaluate. Each participant was given 4 tasks to complete. All participants completed tasks in the same order, since some tasks had dependencies that required completion of an earlier task. We followed each task with self-reported success and satisfaction questions. We also gave participants an opportunity to tell us what they liked and didn’t like about the experience. Finally, we captured clickstream data to generate heat maps. This gave us another opportunity to objectively evaluate user behavior. We could see how they navigated the site, which areas of the site they visited, where they clicked, and more.
  • We knew we wanted to measure several areas, including user satisfaction, task success and efficiency, ease of use and likelihood of using Monster in the future. But we also wanted to gather qualitative feedback on what our users liked and didn’t like about the new experience. Since we were going to interpret the results of this study in context with with other remote studies done prior to the launch of REDUX, we had to structure the study to obtain comparative measures. We recruited panels of 250 people in each of the 5 markets we wanted to evaluate. Each participant was given 4 tasks to complete. All participants completed tasks in the same order, since some tasks had dependencies that required completion of an earlier task. We followed each task with self-reported success and satisfaction questions. We also gave participants an opportunity to tell us what they liked and didn’t like about the experience. Finally, we captured clickstream data to generate heat maps. This gave us another opportunity to objectively evaluate user behavior. We could see how they navigated the site, which areas of the site they visited, where they clicked, and more.
  • When we use large sample sizes across different markets, we are able to quickly identify where we are doing things the right way and where we need to make improvements. Because we have captured related metrics in past studies, are always planning future studies with similar measures, we are able to show improvement over time. We also recognize that our findings must be viewed as part of our entire knowledgebase, so we review other sources of data to validate and explain what users tell us in our remote studies. When we share the bigger picture with our teams, it allows us to make decisions that are based on user data. Another interesting learning came from the qualitiative sections of our studies. We found that when we asked users what they like, they often respond with vague comments such as “it’s easy” or “ it’s fast.” But when you ask people what they don’t like, they tend to be very specific about what they want to see on the site. We are taking advantage of learning when we design additional studies. If users are happy, that’s wonderful to hear. If they are not happy, the value of the response is learning why they are not happy. Lastly, this study allowed us to look at the complete site, not just this area or that page or this functionality. In sharing the results with the entire team, we succeed in pulling all our colleagues together and create a stronger team. It doesn’t matter if they are visual designers, or interaction designers or prototypers - everyone had a role in building our site and when we present the findings, everyone knows they played a part in our success. Thank you. [Back to Alfonso]

User Zoom Webinar Monster Aug09 Vf User Zoom Webinar Monster Aug09 Vf Presentation Transcript

  • Quality products & services for Online User Experience Research Webinar, August 19th Special guest
  • Welcome! Here’s an overview of today’s webinar:
    • Brief UserZoom company presentation
    • About unmoderated remote UX research
    • About UZ Self-Serve Edition
    • Monster’s case study
    • Q&A + summary
    About 30 mins About 30 mins
  • In just 2 mins … Our ‘elevator pitch’
    • What is UserZoom and what do we do?
      • International online market research firm, specializing in user experience and usability.
      • Help brands to better understand their online users, boost website performance & improve business results.
    • How do we do it?
      • Through a unique web-based software application (called UZ Self-Serve Edition) that empowers UX researchers & consultants to conduct highly sophisticated online research.
      • Offer both a self-service solution and professional services.
    • Bottom line… Why should you work with us?
      • You’ll be able to capture & measure the critical and actionable data about your users’ online experience and behavior that you need to manage your business.
  • Highlighted clients
  • Our position in the market
    • What does the market say about web research & analytics?
      • Usability testing in the lab is great, but it has its limitations
      • Web analytics tells you a lot about the WHAT, not the WHY
      • Conducting international testing is expensive
      • Benchmarking and measuring UX is more important than ever
      • Other…
    Overall conclusion: Today’s Web Business Managers seek more sophisticated and innovative research and analytics products to help them compete Highlight #1
  • Strengths & weaknesses of some research methods Focus group Usability labs Online surveys Web analytics Test REAL tasks Large Samples Reason WHY Behavioural Data Natural Setting                     UZ Self-Serve Edition     
  • What is the value? How does it work?
    • Automated (or unmoderated) online user experience research*:
      • A researcher predefines a study script
      • No human moderation needed (moderation through software)
      • Hundreds of users participate simultaneously
      • You ask them to perform tasks and answer follow-up questions
      • Participation from geographically spread locations (local & int’l) and…
      • … in their natural context (using their own PCs)
      • Results : usability metrics, user behavior and web analytics
    Very cost-effective and fast research method Highlight #2
  • This is basically how it works 1. Define your study
    • Choose the type of study you want to conduct
    • Write your study script
    • Choose the recruitment method & audience
    • Pre-test before launch
    • Invite users to participate (or ask your visitors to do it)
    • Collect data automatically
    • Monitor the progress real-time
    2. Launch study, collect data & monitor progress
    • Perform advanced data analysis
    • Use filtering capabilities
    • Copy & paste results
    • Export automatically into Word or Excel
    3. Analyze results & generate a report
    • Option 1: International panels*
    • + 10 million users, spread across 37 countries
    • Wide variety of profiles, great segmentation capabilities
    • Strict, professional control and panel mgmt.
    • Option 2: Intercept method
    • Ask users who visit the website to participate in a study.
    • Insert a javascript in the page
    • Option 3: Private panels, mailing lists
    • Can also work with private databases
    What about user recruitment? Highlight #3 * Partners of
    • Our core: Online (remote) usability tests (task based)
    Versatility of UserZoom (research you can do)
    • Test any web-based interface:
      • Website
      • Intranet
      • Prototype
      • Multimedia
    Highlight #4
    • But also the tool allows for:
      • Online Card-sorting exercises
      • True Intent studies / online surveys
      • Free / open search studies
      • Abandon analysis
      • Mobile interface tests
      • And more…
  • Task based studies: Real users attempting real tasks Websites Task instructions are shown here to remind users what they need to do …
  • Let’s see a video clip of a user participating in a study
  • Results
    • Quantitative data:
      • Effectiveness, efficiency & satisfaction ratios, ease of use and other Likert results are analyzed for statistical significance and correlations
    • Qualitative data:
      • Lots of responses to open-ended questions, relative to each task
    • Behavioral data:
      • Clickstream data and click-maps: Where did the participants actually go? Where did they click to get there.
    • User profile data:
      • Who are my users? What do they come to our site for? How do they perceive us?
  • Quick tour of our tool: UZ Self-Serve Edition
    • 100% hosted / on-demand solution
    • It includes access to 2 apps:
      • UZ Manager (for project design management)
      • UZ Analytics (for data analysis)
    Let’s take a quick tour…
  • Easily create and manage your studies with UZ Manager Fake data shown
  • Fake data shown
  • Highlight #5
    • Many advanced features and capabilities
      • Task validation criteria
      • Questionnaires tool
      • Logic/branching & conditions
      • Etc.
    • Easy to use!
    About our scripting capabilities
  • Analyze results with UZ Analytics: UX ratios & responses Fake data shown
  • Analyze results with UZ Analytics: Survey results Fake data shown
  • Analyze results with UZ Analytics: Clickstreams per task Fake data shown
  • Analyze results with UZ Analytics: Clickmaps/heatmaps per page Fake data shown
  • UZ Analytics reporting & exporting capabilities
    • Lots of actionable data  combination of usability and behavior
    • Many advanced analysis features and capabilities
    • Generate automatic reports (.doc, .html future release)
    • Copy & paste capabilities
    • Export raw-data
    • Share results online
    Highlight #6
  • International user experience study Denise Nangle UX Researcher
  • Background
    • Historically, Monster delivered incremental changes to UI
    • Project REDUX was a redesign of the entire site, offering new products and a new user experience
    • We needed an objective method to evaluate the strengths and weaknesses of this new user experience
    • Solution? Userzoom remote, international usability study
    • Kick off for US, UK, France, Germany and Netherlands studies in January 2009
  • Objectives
    • Measure user satisfaction in 5 key markets
    • Measure task success and efficiency and identify usability issues as users complete common tasks
    • Evaluate ease of use and likelihood of using Monster in the future
    • Collect data on users’ likes as well as their pain points
    • Compare results with past remote studies and develop benchmarks
  • Methodology
    • Recruited 250 participants per channel (total = 1250)
    • Each participant completed 4 tasks; each task was followed by a series of questions and a final questionnaire
    • Collected clickstream data to generate heat maps, which provided additional data on user behavior
  • How Userzoom is helping Monster build a better user experience
    • Using large sample sizes across different markets, we identify opportunities for improvement in a very short period
    • We capture longitudinal metrics to demonstrate improvement over time
    • We triangulate results with other sources of data (marketing studies, web analytics, heuristic reviews, lab studies) to help product, development and UX teams make informed decisions based on actual data
    • It showed us that less satisfied users are more specific in describing what they’d like us to offer, and we take advantage of that information when designing other studies
    • When we bring a holistic approach to our research strategy and share the entire user experience, we are building a better UX team
  • Thank you, Denise! Let’s do a quick summary and Q&A session now…
  • Summary & highlights
    • Today’s Web Business Managers seek more sophisticated and innovative research and analytics products to help them compete
    • Very cost-effective and fast research method
    • Various options for user recruitment
    • Versatility of our tool, research you can do
    • Great scripting capabilities, lots of control for the researcher…
    • … as well as analytics, reporting & exporting capabilities
    Now on to the questions…
  • Would you like a demo?
    • Contact us and we’ll offer you a chance to see and try out our technology
    Alfonso de la Nuez Founder & Int’l Business Dev [email_address] Virginie Glaenzer Sales Manager, USA [email_address] Xavier Mestres Founder & CTO / R&D [email_address]
    • USA Office
    • 440 N. Wolfe Rd. - Sunnyvale, CA 94085 (USA) · [t] +1 (408) 524 7445
    • Spain offices
    • Av. Diagonal 419, 3º 2ª - 08008 Barcelona - T. +34 93 414 75 54 - F. +34 93 368 42 95
    • Gran Vía 62, 7º Izda. - 28013 Madrid - T. +34 91 542 62 52 - F. +34 91 298 08 92
    Here’s our contact information Members of Q&A session Also in London in Sep 09!