This presentation will look at ways for using the Amazon Mechanical Turk system for conducting UX Research, with an emphasis on Specialized Techniques, and how to work around some of Mechanical Turk's inherent limitations. The intended tone is to provide an "Insider's Guide" to using Mechanical Turk ethically and effectively.
The speaker will share his experiences, including both challenges and successes, in working with Amazon's Mechanical Turk, along with gleaned incites.
Amazon Mechanical Turk is an online tool for recruiting and paying human subjects for completing specific work tasks. User Experience Professionals have been using Mechanical Turk for data gathering activities. It has been designed to link to supplemental tools and resources, such as the Qualtrics Survey Management system.
This document provides an agenda and overview for a workshop on getting started with usability testing. The workshop will include introductions, presentations on measurements and testing, and activities for planning test logistics and interacting with participants. It will cover why usability testing is important, common myths, and tips for getting started. Participants will learn about planning and preparing for tests, recruiting profiles, scripts, test materials, facilitating sessions, and reporting results. The goal is to help attendees understand the basics of usability testing and conduct initial tests.
Mixed Methods Research in the Age of Big Data: A Primer for UX ResearchersUXPA International
What does UX research entail in what some are calling the “Age of Data Science?” Most would agree that some level of collaboration is needed -- Data Science results feeding UX Research and vice versa -- but can this be more meaningful than simply attending each other’s readouts?
In this session, you’ll hear some practical, approachable tips for qualitative UX Researchers to play a larger role in Big Data discussions. Stats expertise not required! These tips will help you break through the lexicon barriers between UX Research and Data Science, and provide a framework for collaboration that can lead to even more impactful research.
Mobile application use has grown exponentially. Testing mobile experiences brings additional challenges and opportunities because the context-of-use is not a static location like a workplace, but instead wherever a user happens to be with his or her phone. Field studies are useful because they help researchers collect more naturalistic data since they observe users in their context-of-us instead of a lab.
In this course you will learn:
The variety of methods and tools for field testing mobile experiences, and the pros / cons of each
How to develop a research plan for mobile field testing
Recruitment strategies for field testing
How to moderate a field test and record participants’ behaviors
You will learn by doing an actual field study of a mobile application near the conference location.
Usability testing involves having people complete tasks while thinking aloud to provide insight into their thought processes. It can be done using paper prototypes, websites, or existing sites. Benefits for designers include uncovering unexpected issues, testing assumptions, and getting stakeholder buy-in. Benefits for site owners are fixing problems early when cheaper, improving customer satisfaction, and increasing conversion rates. Guerilla or informal testing is quick, inexpensive, and provides qualitative insights that can immediately feed back into the design process. It involves recruiting your own test subjects and moderating the sessions yourself.
This document discusses the importance of usability in websites. It notes that over 83% of users will leave a website if it takes too many clicks to find what they are looking for, and $25 billion is lost every year due to usability issues. The document then examines existing tools to evaluate website usability, including tools that obtain feedback from experts, track real user behavior and actions, and check for compliance with usability guidelines. It also references a Netcraft report finding that only 29.3% of existing websites are currently active, suggesting that websites need to be usable or users will stop using them.
Kirk Doggett and Kate Lawrence presented on using the usertesting.com tool for unmoderated user testing. They demonstrated a test of Vistaprint's business card design page. They discussed that usertesting.com allows recruiting specific users, testing websites and prototypes, and getting results quickly. However, tasks must be very clear as tests are unmoderated, and it may not be suitable for complex or time-consuming tasks. They highlighted when usertesting.com is appropriate to use, such as for targeted tasks, and its benefits like testing from anywhere.
Conducting Usability Research with a Team of One [Revised: October 2009]cmk22
The document summarizes how to conduct usability research with a limited budget and team. It describes how the author conducted pre- and post-redesign usability research for a university website redesign project with minimal funding and assistance. Key hurdles included limited time, manpower, budget, and supervisor buy-in. The author overcame these hurdles by testing users during lunch breaks, using free tools, asking for donations, and illustrating the value of testing to gain support.
In the Emirates, the UX interview is always a surprise as we really never know what to expect! Sometimes our interviewer is not a UX Designer. But what if he or she is a UX Guru?
The goal of this presentation is to discuss the best way to make you ready and rock at your next UX interview!
In order to get there, we'll talk about:
• The UX Role and types of UX roles
• The interview and a few suggestions on do's & don'ts
• The Recruiter's point of view
• The Candidate's point of view
• What are you really looking for in a UX job?
This document provides an agenda and overview for a workshop on getting started with usability testing. The workshop will include introductions, presentations on measurements and testing, and activities for planning test logistics and interacting with participants. It will cover why usability testing is important, common myths, and tips for getting started. Participants will learn about planning and preparing for tests, recruiting profiles, scripts, test materials, facilitating sessions, and reporting results. The goal is to help attendees understand the basics of usability testing and conduct initial tests.
Mixed Methods Research in the Age of Big Data: A Primer for UX ResearchersUXPA International
What does UX research entail in what some are calling the “Age of Data Science?” Most would agree that some level of collaboration is needed -- Data Science results feeding UX Research and vice versa -- but can this be more meaningful than simply attending each other’s readouts?
In this session, you’ll hear some practical, approachable tips for qualitative UX Researchers to play a larger role in Big Data discussions. Stats expertise not required! These tips will help you break through the lexicon barriers between UX Research and Data Science, and provide a framework for collaboration that can lead to even more impactful research.
Mobile application use has grown exponentially. Testing mobile experiences brings additional challenges and opportunities because the context-of-use is not a static location like a workplace, but instead wherever a user happens to be with his or her phone. Field studies are useful because they help researchers collect more naturalistic data since they observe users in their context-of-us instead of a lab.
In this course you will learn:
The variety of methods and tools for field testing mobile experiences, and the pros / cons of each
How to develop a research plan for mobile field testing
Recruitment strategies for field testing
How to moderate a field test and record participants’ behaviors
You will learn by doing an actual field study of a mobile application near the conference location.
Usability testing involves having people complete tasks while thinking aloud to provide insight into their thought processes. It can be done using paper prototypes, websites, or existing sites. Benefits for designers include uncovering unexpected issues, testing assumptions, and getting stakeholder buy-in. Benefits for site owners are fixing problems early when cheaper, improving customer satisfaction, and increasing conversion rates. Guerilla or informal testing is quick, inexpensive, and provides qualitative insights that can immediately feed back into the design process. It involves recruiting your own test subjects and moderating the sessions yourself.
This document discusses the importance of usability in websites. It notes that over 83% of users will leave a website if it takes too many clicks to find what they are looking for, and $25 billion is lost every year due to usability issues. The document then examines existing tools to evaluate website usability, including tools that obtain feedback from experts, track real user behavior and actions, and check for compliance with usability guidelines. It also references a Netcraft report finding that only 29.3% of existing websites are currently active, suggesting that websites need to be usable or users will stop using them.
Kirk Doggett and Kate Lawrence presented on using the usertesting.com tool for unmoderated user testing. They demonstrated a test of Vistaprint's business card design page. They discussed that usertesting.com allows recruiting specific users, testing websites and prototypes, and getting results quickly. However, tasks must be very clear as tests are unmoderated, and it may not be suitable for complex or time-consuming tasks. They highlighted when usertesting.com is appropriate to use, such as for targeted tasks, and its benefits like testing from anywhere.
Conducting Usability Research with a Team of One [Revised: October 2009]cmk22
The document summarizes how to conduct usability research with a limited budget and team. It describes how the author conducted pre- and post-redesign usability research for a university website redesign project with minimal funding and assistance. Key hurdles included limited time, manpower, budget, and supervisor buy-in. The author overcame these hurdles by testing users during lunch breaks, using free tools, asking for donations, and illustrating the value of testing to gain support.
In the Emirates, the UX interview is always a surprise as we really never know what to expect! Sometimes our interviewer is not a UX Designer. But what if he or she is a UX Guru?
The goal of this presentation is to discuss the best way to make you ready and rock at your next UX interview!
In order to get there, we'll talk about:
• The UX Role and types of UX roles
• The interview and a few suggestions on do's & don'ts
• The Recruiter's point of view
• The Candidate's point of view
• What are you really looking for in a UX job?
Usability testing tips for UX DesignersDavid Hamill
This document provides tips for usability testing from a usability specialist at Skyscanner. Some key tips include:
- Conduct small, frequent usability tests with around 6 participants rather than large lab studies
- There is no single best way to do usability testing and you have to sacrifice aspects of the process
- Remember that participants are not like real users in the way they will use the product
- Expect that the initial design will have problems and be prepared to iterate based on test findings
In this tutorial for experienced practitioners you will learn how to manage work and make great experiences one sprint at a time. We'll look at common Agile methodologies such as Scrum and Kanban and what opportunities and risks are inherent for UX teams. We will look at team makeup, balancing longer-term research with production needs and strategies for making the most of design spikes. We'll also go through the pros and cons of a Sprint Zero and alternatives. We'll look at how Lean Startup practices are changing business development, and how your UX skills can be a key part in making that successful. Participants will come away with the tools they need to be successful in their Agile/Lean environment
This document provides an introduction to user experience (UX) design for startups. It discusses UX principles like viability, desirability and feasibility, and emphasizes starting with understanding people through research methods like interviews, surveys and analytics. Key phases of the design process are covered, including diverging to generate ideas, deciding on a solution, prototyping, and validating designs. A variety of UX design tools and techniques are also introduced.
This document discusses various methods for conducting UX research for startups, including contextual inquiries, surveys, focus groups, card sorts, usability studies, and A/B testing. It also discusses crowdsourcing options like Usabilla, CrazyEgg, and Amazon Mechanical Turk. Game mechanics and flow theory are covered as ways to track user satisfaction. The concept of Net Promoter Score is introduced for measuring user recommendation likelihood.
Best Practice For UX Deliverables - Eventhandler, London, 05 March 2014Anna Dahlström
The document discusses best practices for UX deliverables. It emphasizes that UX deliverables should be adapted to the intended audience and add value. Deliverables should have a clear narrative and tell a story. Creating visually engaging deliverables that keep the audience's attention is important, especially when presenting to clients who may not have a background in UX. The document also stresses that UX is about collaboration between different roles and that effective deliverables facilitate common understanding between teams.
UXPA DC UX 101 Workshop - Usability TestingUXPA DC
This document provides an overview of usability testing from Stephanie Pratt of UXPA DC. It defines usability and its importance, how to set up and conduct a usability test, and tips for moderating a test. Key points include:
- Usability is how easy a product is to use, defined by criteria like effectiveness, efficiency and ease of learning.
- Usability testing evaluates a product's usability by observing users complete tasks. It identifies problems and improves the product.
- To set up a test, identify tasks, recruit 5-10 participants, develop a script, and practice moderation skills. When moderating, keep questions neutral and let the participant think aloud.
1) The document discusses how organizations typically progress through stages of usability maturity from initial hostility to widespread reliance on user research.
2) It provides an overview of techniques an organization can use to embed usability practices, including conducting user research, usability testing, creating personas, and integrating UX processes into agile development.
3) The author details how their own organization has progressed from occasional usability surveys to having a dedicated UX team and standardized processes, highlighting the importance of starting small, gaining buy-in from stakeholders, and using data to demonstrate the value of usability
This document provides guidance on conducting usability testing to improve product design. It recommends: 1) Conducting speed tests with users to understand behaviors and issues; 2) Building a tester community from real users to provide feedback; 3) Testing frequently with small groups to incorporate feedback early. It outlines when and what to test, how to prepare by determining goals and tasks. It provides a testing protocol, including introducing yourself, having users think aloud, and asking questions without justifying designs. The key is to listen, take notes on positive and negative experiences, and use the insights to reinforce successes and address problems.
Introduction to Usability Testing: The DIY Approach - GA, London January 13th...Evgenia (Jenny) Grinblo
The slides from my General Assembly workshop on January 13th, 2013 (https://generalassemb.ly/education/introduction-to-usability-testing-the-diy-approach)
ABOUT THIS WORKSHOP
Usability testing can quickly uncover areas of an interface that frustrate users and hurt business goals but many teams put it off due to budget, time, or training concerns.
This workshops will take you through a do-it-yourself approach to usability testing. We'll cover the basics (benefits, recruiting, and how to plan a test), learn how to facilitate a test to get reliable results, and how to use the testing results to move usability improvements forward. You'll walk away with the tools to hold a complete usability testing right away.
TAKEAWAYS
Learn why and when to hold usability testing
Learn practical tools and methods to overcome time, budget or training concerns that block user testing from happening
Shift the conversation from opinions and hunches to proven usability problems that your team can solve together
The document discusses research conducted to understand workplace distractions and challenges with multitasking. Key insights from surveys and interviews found that multitasking is required but disruptive, and that channels, waiting, and changing priorities make completing tasks difficult. Competitive apps were reviewed and a persona was developed. An MVP feature list and information architecture were created. Early sketches and wireframes developed a hub-and-spoke layout. User testing provided feedback that improved the designs. The initial designs aim to help users focus on one task at a time while providing team context.
Elizabeth Snowdon is a senior usability analyst at Sage Software who has been conducting usability studies since 2003. She discusses remote usability testing, which involves testing sessions where the participant and moderator are not physically in the same place but communicate electronically. Some advantages of remote testing are that it expands recruitment populations, saves costs compared to in-person lab testing, and allows testing in users' natural environments. However, remote testing has disadvantages like not being able to see participants' screens and potential technical issues. Snowdon provides an example case study where remote testing saved $50,000 compared to in-lab testing.
The document discusses the importance of user testing in app development. It outlines different types of user testing including usability testing, concept testing, guerrilla testing, lab testing, contextual testing, and remote testing. User testing helps identify navigation errors, presentation errors, control usage problems, and understand how people use the service. The document provides an example of usability challenges encountered during testing of the mFriend app and how they were addressed. It emphasizes that user testing should be included in every stage of development to create successful products.
The document provides an overview of user experience (UX) research methods. It explains that research is done to answer questions, remove ambiguity, understand human behaviors and needs, and build empathy. Research methods include interviews, observations, surveys, usability testing and more. Both qualitative and quantitative methods are used depending on the questions being asked and stage of the project. Numbers from research don't tell the whole story and can sometimes be misleading.
This document summarizes a workshop on creating lean research techniques. The workshop covered challenges in research such as delivering insights faster and recruiting users. It discussed lean UX principles like design thinking, agile development, and collaboration. Techniques for lean user research included creating a consolidated source of insights, educating all employees on users, conducting weekly user interviews, and running rapid iterative user testing. Challenges of these techniques like startup costs and managing large panels were also addressed. The goal was to facilitate collaboration and sharing of experiences to discover solutions already tried or brainstorm new methods.
This document summarizes a presentation about building AI products. It discusses the typical AI product lifecycle including defining the problem, data preparation, model development, evaluation and deployment. It provides examples of different types of AI systems and how the role of AI evolves from assisting humans to being more autonomous. The presentation emphasizes understanding users, data and metrics to build ethical and unbiased AI. It also discusses ongoing learning after deployment and providing case studies of AI products from companies like Microsoft, Uber and Tesla.
UX Workshop introducing what UX is and why it is important. The audience may or may not be familiar with UX so the presentation focuses more on principles than a step-by-step how-to.
Micro Feedback – Maximum Insights? UX Camp Europe 2015, Berlin, #uxce15Julius Dietz
Talk and discussion at uxcamp Europe ux (un)conference in Berlin. Looking at examples of Microfeedback such as micro surveys around the web and a little case study from our own webapp: Brandwatch Analytics.
The usability testing document discusses various methods for conducting usability testing, including remote screen sharing, recording user interactions, and analyzing task completion times, errors, and user feedback. It describes card sorting, task elicitation, and enhanced analytics as alternatives if an evaluator prefers machines over people. The document also provides recommendations for open-ended questioning, eye tracking, and using prototypes during testing. Overall, the summary emphasizes different approaches to usability testing and gaining user feedback.
Beyond Mechanical Turk: An Analysis of Paid Crowd Work PlatformsMatthew Lease
The document summarizes a presentation about analyzing paid crowd work platforms beyond Mechanical Turk. It discusses how Mechanical Turk has dominated research on paid crowdsourcing due to its early popularity, but that it has limitations. The presentation conducts a qualitative study of 7 alternative crowd work platforms to identify distinguishing capabilities not found on MTurk, such as different payment models, richer worker profiles, and support for confidential tasks. It aims to increase awareness of other platforms to further inform practice and research on crowdsourcing.
Research tools such as Likert Scales can often be blunt instruments when it comes to tracking people’s emotional state over the course of the product experience especially when it stretches over multiple steps and over an extended time period. The soon to be released cloud-based Emotie (www.emotietool.com) provides researchers with a simple tool for tracking and communicating people’s emotional state.
Usability testing tips for UX DesignersDavid Hamill
This document provides tips for usability testing from a usability specialist at Skyscanner. Some key tips include:
- Conduct small, frequent usability tests with around 6 participants rather than large lab studies
- There is no single best way to do usability testing and you have to sacrifice aspects of the process
- Remember that participants are not like real users in the way they will use the product
- Expect that the initial design will have problems and be prepared to iterate based on test findings
In this tutorial for experienced practitioners you will learn how to manage work and make great experiences one sprint at a time. We'll look at common Agile methodologies such as Scrum and Kanban and what opportunities and risks are inherent for UX teams. We will look at team makeup, balancing longer-term research with production needs and strategies for making the most of design spikes. We'll also go through the pros and cons of a Sprint Zero and alternatives. We'll look at how Lean Startup practices are changing business development, and how your UX skills can be a key part in making that successful. Participants will come away with the tools they need to be successful in their Agile/Lean environment
This document provides an introduction to user experience (UX) design for startups. It discusses UX principles like viability, desirability and feasibility, and emphasizes starting with understanding people through research methods like interviews, surveys and analytics. Key phases of the design process are covered, including diverging to generate ideas, deciding on a solution, prototyping, and validating designs. A variety of UX design tools and techniques are also introduced.
This document discusses various methods for conducting UX research for startups, including contextual inquiries, surveys, focus groups, card sorts, usability studies, and A/B testing. It also discusses crowdsourcing options like Usabilla, CrazyEgg, and Amazon Mechanical Turk. Game mechanics and flow theory are covered as ways to track user satisfaction. The concept of Net Promoter Score is introduced for measuring user recommendation likelihood.
Best Practice For UX Deliverables - Eventhandler, London, 05 March 2014Anna Dahlström
The document discusses best practices for UX deliverables. It emphasizes that UX deliverables should be adapted to the intended audience and add value. Deliverables should have a clear narrative and tell a story. Creating visually engaging deliverables that keep the audience's attention is important, especially when presenting to clients who may not have a background in UX. The document also stresses that UX is about collaboration between different roles and that effective deliverables facilitate common understanding between teams.
UXPA DC UX 101 Workshop - Usability TestingUXPA DC
This document provides an overview of usability testing from Stephanie Pratt of UXPA DC. It defines usability and its importance, how to set up and conduct a usability test, and tips for moderating a test. Key points include:
- Usability is how easy a product is to use, defined by criteria like effectiveness, efficiency and ease of learning.
- Usability testing evaluates a product's usability by observing users complete tasks. It identifies problems and improves the product.
- To set up a test, identify tasks, recruit 5-10 participants, develop a script, and practice moderation skills. When moderating, keep questions neutral and let the participant think aloud.
1) The document discusses how organizations typically progress through stages of usability maturity from initial hostility to widespread reliance on user research.
2) It provides an overview of techniques an organization can use to embed usability practices, including conducting user research, usability testing, creating personas, and integrating UX processes into agile development.
3) The author details how their own organization has progressed from occasional usability surveys to having a dedicated UX team and standardized processes, highlighting the importance of starting small, gaining buy-in from stakeholders, and using data to demonstrate the value of usability
This document provides guidance on conducting usability testing to improve product design. It recommends: 1) Conducting speed tests with users to understand behaviors and issues; 2) Building a tester community from real users to provide feedback; 3) Testing frequently with small groups to incorporate feedback early. It outlines when and what to test, how to prepare by determining goals and tasks. It provides a testing protocol, including introducing yourself, having users think aloud, and asking questions without justifying designs. The key is to listen, take notes on positive and negative experiences, and use the insights to reinforce successes and address problems.
Introduction to Usability Testing: The DIY Approach - GA, London January 13th...Evgenia (Jenny) Grinblo
The slides from my General Assembly workshop on January 13th, 2013 (https://generalassemb.ly/education/introduction-to-usability-testing-the-diy-approach)
ABOUT THIS WORKSHOP
Usability testing can quickly uncover areas of an interface that frustrate users and hurt business goals but many teams put it off due to budget, time, or training concerns.
This workshops will take you through a do-it-yourself approach to usability testing. We'll cover the basics (benefits, recruiting, and how to plan a test), learn how to facilitate a test to get reliable results, and how to use the testing results to move usability improvements forward. You'll walk away with the tools to hold a complete usability testing right away.
TAKEAWAYS
Learn why and when to hold usability testing
Learn practical tools and methods to overcome time, budget or training concerns that block user testing from happening
Shift the conversation from opinions and hunches to proven usability problems that your team can solve together
The document discusses research conducted to understand workplace distractions and challenges with multitasking. Key insights from surveys and interviews found that multitasking is required but disruptive, and that channels, waiting, and changing priorities make completing tasks difficult. Competitive apps were reviewed and a persona was developed. An MVP feature list and information architecture were created. Early sketches and wireframes developed a hub-and-spoke layout. User testing provided feedback that improved the designs. The initial designs aim to help users focus on one task at a time while providing team context.
Elizabeth Snowdon is a senior usability analyst at Sage Software who has been conducting usability studies since 2003. She discusses remote usability testing, which involves testing sessions where the participant and moderator are not physically in the same place but communicate electronically. Some advantages of remote testing are that it expands recruitment populations, saves costs compared to in-person lab testing, and allows testing in users' natural environments. However, remote testing has disadvantages like not being able to see participants' screens and potential technical issues. Snowdon provides an example case study where remote testing saved $50,000 compared to in-lab testing.
The document discusses the importance of user testing in app development. It outlines different types of user testing including usability testing, concept testing, guerrilla testing, lab testing, contextual testing, and remote testing. User testing helps identify navigation errors, presentation errors, control usage problems, and understand how people use the service. The document provides an example of usability challenges encountered during testing of the mFriend app and how they were addressed. It emphasizes that user testing should be included in every stage of development to create successful products.
The document provides an overview of user experience (UX) research methods. It explains that research is done to answer questions, remove ambiguity, understand human behaviors and needs, and build empathy. Research methods include interviews, observations, surveys, usability testing and more. Both qualitative and quantitative methods are used depending on the questions being asked and stage of the project. Numbers from research don't tell the whole story and can sometimes be misleading.
This document summarizes a workshop on creating lean research techniques. The workshop covered challenges in research such as delivering insights faster and recruiting users. It discussed lean UX principles like design thinking, agile development, and collaboration. Techniques for lean user research included creating a consolidated source of insights, educating all employees on users, conducting weekly user interviews, and running rapid iterative user testing. Challenges of these techniques like startup costs and managing large panels were also addressed. The goal was to facilitate collaboration and sharing of experiences to discover solutions already tried or brainstorm new methods.
This document summarizes a presentation about building AI products. It discusses the typical AI product lifecycle including defining the problem, data preparation, model development, evaluation and deployment. It provides examples of different types of AI systems and how the role of AI evolves from assisting humans to being more autonomous. The presentation emphasizes understanding users, data and metrics to build ethical and unbiased AI. It also discusses ongoing learning after deployment and providing case studies of AI products from companies like Microsoft, Uber and Tesla.
UX Workshop introducing what UX is and why it is important. The audience may or may not be familiar with UX so the presentation focuses more on principles than a step-by-step how-to.
Micro Feedback – Maximum Insights? UX Camp Europe 2015, Berlin, #uxce15Julius Dietz
Talk and discussion at uxcamp Europe ux (un)conference in Berlin. Looking at examples of Microfeedback such as micro surveys around the web and a little case study from our own webapp: Brandwatch Analytics.
The usability testing document discusses various methods for conducting usability testing, including remote screen sharing, recording user interactions, and analyzing task completion times, errors, and user feedback. It describes card sorting, task elicitation, and enhanced analytics as alternatives if an evaluator prefers machines over people. The document also provides recommendations for open-ended questioning, eye tracking, and using prototypes during testing. Overall, the summary emphasizes different approaches to usability testing and gaining user feedback.
Beyond Mechanical Turk: An Analysis of Paid Crowd Work PlatformsMatthew Lease
The document summarizes a presentation about analyzing paid crowd work platforms beyond Mechanical Turk. It discusses how Mechanical Turk has dominated research on paid crowdsourcing due to its early popularity, but that it has limitations. The presentation conducts a qualitative study of 7 alternative crowd work platforms to identify distinguishing capabilities not found on MTurk, such as different payment models, richer worker profiles, and support for confidential tasks. It aims to increase awareness of other platforms to further inform practice and research on crowdsourcing.
Research tools such as Likert Scales can often be blunt instruments when it comes to tracking people’s emotional state over the course of the product experience especially when it stretches over multiple steps and over an extended time period. The soon to be released cloud-based Emotie (www.emotietool.com) provides researchers with a simple tool for tracking and communicating people’s emotional state.
Intro into the IGNITE session at UXPA 2015. During this ignite track you'll learn about some cutting edge UX tools and methodologies, as well as some fascinating design psychology topics that you can apply to your upcoming projects.
IGNITE: How to Accidentally Create a Viral UX Infographic - Jennifer AldrichUXPA International
The author accidentally created a viral UX infographic by sketching a simple diagram to explain the difference between UX and UI to their curious child. The child was able to understand the distinction based on the sketch, so the author tweeted it. The sketch was then featured on several "best UX infographic" lists and the author was approached about using it in other publications. This unexpected success taught the author that artistic skill is not needed to effectively communicate concepts through visuals, and that simplicity and clarity are more important goals for infographics than perfection.
Design Studios are a popular method for getting product teams together to focus on design. Design Studios are more than just getting people together to sketch and critique. In this workshop, Brian Sullivan, author of The Design Studio Method: Creative Problem Solving with UX Sketching, will share his secrets to planning, running, and leading successful Design Studios
In this workshop, you will learn:
Ways to creative and evaluate sketches quickly
See different tools to get you started
The 9 Steps of a Design Studio
Stories of success and failure in Design Studio
How to deal with difficult people/strong personalities
We will have plenty of time for your burning questions, too.
C(ollab) RITE: How to run impactful iterative studies in a fast paced environ...UXPA International
UXPA 2013 Annual Conference July 11, 2013 1:30 - 2:30pm by Helena Roeber, Jhilmil Jain
UX research professionals often desire most that their work has impact, which can be tricky in fast-paced industry environments with short development cycles. This talk describes C-RITE , a method that maximizes impact through cross-disciplinary collaboration on research observation, analysis, and design exploration within an agile, user-centered development framework. We showcase real-world tested techniques that Google’s Android and TV teams have developed and share tactics for reducing logistical overhead in a lean, iterative user-centered design process.
The Ubiquitous Eye Tracker: Eye tracking has become a key method to test the usability of websites and software. It provides researchers and practitioners with indisputable, objective and convincing data describing user behavior and usability problems. Eye tracking is also used to study user interaction with mobile devices and physical products. Eye tracking augments traditional usability methods, providing additional information that the test participant cannot report and the researcher cannot observe. Unique insights about first glance, search patterns, failed search, and much more offer guidance in how to solve different usability problems. Eye tracking can be used together with a variety of research methods, including observations, interviews and the retrospective think aloud (RTA) method. The past ten years, eye trackers have evolved from bulky, expensive and difficult to use devices, to a level of maturity that is soon good enough to be used in widespread consumer user experiences. The solutions of today enable more dynamic research setups and a wider range of uses, but what about tomorrow? What does it mean for us as UX professionals when gaze information from our users will be available all the time, everywhere?
Sounds simple... and it is! Put down your sketchbooks and grab your sharpies & post-its. Create a simple storyboard to quickly communicate your ideas to stakeholders before you start wireframing.
This is a new golden age for design. In business, where technology has dominated for decades, the balance of power is shifting. Lessons learned have thrown up new imperatives. The most exciting of these conversations explore new frontiers for business - empathy, design insight, disruptive innovation, big data, lean practice - and all point to the prize: human-centred business transformation. The vision is of a future that will be brilliantly designed rather than just cleverly engineered. Technology alone cannot deliver the experience. Who are the design leaders who will breathe life into this vision? Where will we find them? How will we recognize them? Which skills and qualities will define them? How will we motivate them? How will we partner with other professions? And how do we support them to find, foster and equip a global design elite that will rise up and play their role in changing the world?
What ISO standards can do for you, and you can do for ISO (Poster, Nigel Bevan)UXPA International
This document discusses several ISO standards related to usability and human-centered design. It outlines standards that define what usability information should be documented, how to conduct and report on usability tests, and how to implement user-centered design processes. The document recommends some key ISO standards, including ISO 9241-11 which defines usability, ISO 9241-210 which provides guidelines for human-centered design processes, and ISO/IEC 15288 which integrates user-centered design into systems engineering. It encourages contributions to ISO standards to help improve processes for user experience and accessibility.
How UX Can Drive the Vision of Future Products - Arttu NiskasaariUXPA International
Our existing B2B product has been developed for 15 years and the need for complete redesign was acknowledged in 2013.
Unusual for the software business in our country and field of business, this project for the brand new solution was driven by UX from the beginning. The main target was to introduce new level of collaboration between all company functions to formulate a shared vision for the future product.
It took us one year to move from user research to prototypes, and in the meantime our UX team grew from two to six persons. Hence, we will also talk a bit about organizing the work of the team to support several products and projects without sacrificing the long-term project.
In this session we will share our experiences and lessons-learned from working our way towards that vision with research based top-down approach.
The document describes a digital diary study conducted to understand how people use fitness trackers. 40 people who used fitness trackers participated in several diary studies over 8 months. The studies involved participants reporting on their tracker usage through short videos and answering questions. Over 170 diary entries were collected. The studies revealed patterns in how participants' tracker usage evolved and the role the trackers played in their health goals. Key findings included that fitness trackers help people learn accurate information about their eating and exercise habits that they previously were unaware of.
Tell Me What You Do - How Storytelling Can Transform Artifacts Into Engagemen...UXPA International
The document discusses how storytelling can make designers better at their jobs. It presents a "UX Story Wheel" model for crafting stories around user experience design projects. The wheel includes elements like the user world, project world, and design interventions. It also describes three common story patterns - detective stories, buddy stories, and lessons stories - that follow archetypes like questioning, investigation, and resolving issues. The presentation ends by sharing 12 "rules of the road" for using stories in UX design work.
1/7 of the UXPA 2014 Ignite session "Ethical Dilemmas in UX"
Ever wondered if you should ask THAT question? Join us for a series of passionate speakers sharing their thoughts on ethics, what difficult situations they have faced, what they did, and why. Topics will cover lab situations, field situations and business situations. You will gain tactics to use in the future when issues arise.
1/5 of the "Future of UX" Ignite session from UXPA 2014.
The purpose of this session is to get attendees dreaming about the possible, the likely, and the probable future of UX Design - and to inspire them to be a part of making these dreams a reality.
We will have at least 5 visionary speakers directed to talk about what they think is possible, likely, and probable 20 years, 50 years, and 100 years with regard to personal and organizational technology design.
Speakers will be asked to consider the following questions in their presentations:
What similarities and differences do you foresee in how people think about technology in the next 20, 50, and 100 years?
What trends will have come and gone?
What trends are lasting How will the UX profession change?
How will businesses , users and UX professionals collaborate on design challenges?
What would you like to see in the future of UX Design and why?
This document summarizes the challenges of a career in UX design over many years. It describes UX designers feeling like masochists due to developers not following instructions or product teams prescribing solutions rather than requirements. It also discusses problems at companies where UX teams are unable to have enough impact or use their preferred methodologies. However, it notes that unsatisfied UX professionals are usually unhappy with their company or role rather than the work itself. The document ends on a positive note by quoting surveys finding that UX designers feel their work helps people and makes a difference.
Win User Loyalty by Targeting Logic AND Emotion (Mike Donahue)UXPA International
Emotions are arguably the most powerful motivator for humans and yet most projects lack an implicit strategy to target emotions. This talk will present insight on how and why to use emotion as a strategic target for UX design. It will cover the 4 Stages of Accomplishing Goals, the factors that affect how strongly we feel about an experience, what causes us to focus more on the positive or negative aspects of an experience, what stops us dead in our tracks, and how to create an experience that satisfies both the logical and emotional parts of our brain.
1/7 of the UXPA 2014 Ignite session "Ethical Dilemmas in UX"
Ever wondered if you should ask THAT question? Join us for a series of passionate speakers sharing their thoughts on ethics, what difficult situations they have faced, what they did, and why. Topics will cover lab situations, field situations and business situations. You will gain tactics to use in the future when issues arise.
Have you reached an inflection point in your career? Not sure how to get to the next step – or even what the next step will be? In this hands-on session, you will get an overview of the hiring landscape and salary trends for UX professionals. You’ll hear about the most in-demand positions and skills that employers are willing to pay a premium for – and learn how you can target your own skill set to those opportunities. You’ll also participate in a few exercises to help actively identify new career directions, keep your digital skills relevant to employers, overcome job-hunting obstacles and, ultimately, forge a fulfilling professional path.
Adversarial to Harmonious: Building the Developer/UX ConnectionUXPA International
Ever worked on a project where Design and Development blended like oil and water? Whether you're on a UX team of one, or designing with the help of a whole department, the success of your work ends up in the hands of a developer.
Teams with specialized skillsets and certain cross-team cultures can put up walls between designers and developers. We will deconstruct these adversarial relationships from real-world examples, then learn how to convince, collaborate, and co-create.
Being stuck in a storming phase isn’t good for you, your product, and ultimately your users. Bringing harmony to your team is important to your success and your sanity. Hone your best expertise to build relationships, handle differences of opinion, and learn to speak geek to be heard!
Walk out with tools and techniques to stay efficient and deliver the best possible experience for the real human beings who will use it.
The document discusses strategies for improving IT recruiting by focusing on hiring top talent, conducting effective interviews that assess skills rather than just academic knowledge, and providing career growth opportunities for engineers through senior technical roles rather than forcing them into management. It also provides advice for job seekers on creating strong resumes, preparing for coding interviews, and negotiating salaries effectively.
15 tips for bullet proof requirements analysis on SharePoint projectsDocFluix, LLC
Success or failure of any significant SharePoint project depends on having well defined business and technical requirements before the project starts. With this presentation, learn concrete, repeatable techniques for ensuring that the requirements for your SharePoint project are well defined.
[DevDay2018] High quality mindset in software development - By: Phat Vu, Scru...DevDay Da Nang
In this topic, Phat will show what high quality mindset is, how important it is. He will try to bring as many examples as possible – not theory, but lesson-learned ones. His expectation is that he could encourage developers to have or refresh a mindset about doing high quality software.
AWS re:Invent 2016: Leverage the Power of the Crowd To Work with Amazon Mecha...Amazon Web Services
With Amazon Mechanical Turk (MTurk), you can leverage the power of the crowd for a host of tasks ranging from image moderation and video transcription to data collection and user testing. You simply build a process that submit tasks to the Mechanical Turk marketplace and get results quickly, accurately, and at scale. In this session, Russ, from Rainforest QA, shares best practices and lessons learned from his experience using MTurk. The session covers the key concepts of MTurk, getting started as a Requester, and using MTurk via the API. You learn how to set and manage Worker incentives, achieve great Worker quality, and how to integrate and scale your crowdsourced application. By the end of this session, you will have a comprehensive understanding of MTurk and know how to get started harnessing the power of the crowd.
A Proven Software Development Process for the Non Technical FounderFounders Workshop
Are you a non-technical founder with a great software idea? Ready to take the plunge but want the “secret” to successfully managing software development? Well, it's not a "secret" at all - it's a disciplined methodology we are going to share with you. This presentation is designed to provide entrepreneurs with a blueprint for successful software development and technology implementation.
The unfortunate reality is that quality software development and technology implementation is not readily available to most startups and small business entrepreneurs. Great entrepreneurs are met with small thinkers when searching for a development team via online freelancer sites, or the recommendation of a friend’s cousin who may code on weekends. Or they are faced with development companies that impose business models that do not align with the entrepreneurial spirit.
A Successful Hiring Process for Data ScientistsGreg Makowski
Discuss one successful hiring process for data scientists. The current "best" algorithms are constantly changing. Also, it is not uncommon to need to learn about a new vertical market for a DS application. From my DS hiring experience over 2010-2022, I have focused on hiring people that are good at learning and adapting.
The document provides tips for both hiring managers and candidates when it comes to hiring frontend engineers. It recommends using sources like GitHub, employee referrals, and meetups to find candidates. It suggests showing what the company has to offer in terms of technology, opportunities, and fast responses. For candidates, it advises keeping resumes concise and focused on contributions over technologies, and preparing for coding interviews which may involve homework, existing code reviews, or small onsite coding projects. It also discusses salary negotiation and interviewing potential employers.
9 Tips on How to hire Tech Talent when you are not a TechieInterview Mocha
Looking for the tech talent, without being a tech expert yourself can be difficult. Here are few ways on how to hire tech talent that can help you find the great developer.
This document discusses how to hire software engineers from both the employer and candidate perspectives. It provides tips for employers on where to find engineer candidates, such as through blogs, meetups, referrals and GitHub. It also discusses how to structure coding interviews with phone screens, homework assignments, and onsite coding tests. For candidates, it offers advice on crafting a strong resume that highlights contributions and impacts rather than just technologies. It also suggests how candidates can interview their potential employers using "The Joel Test" to assess workplace culture and practices.
This document provides advice on how non-profits can build strong software engineering teams through an effective hiring process. It discusses challenges non-profits face in hiring, such as lower salaries compared to industry. It recommends casting a wide net to find candidates, running a rigorous interview process to filter candidates, hiring for potential as well as experience, assessing organizational fit, and making strong offers that emphasize mission over salary alone. The document provides examples from the author's experience hiring a new tech team at MoveOn.org, highlighting the importance of an inclusive job description and interview process for finding diverse talent.
Basic overview of software test types, methodologies.
Explaining and reasons to test and common pitfalls with various testing methodologies.
Example scenarios for the viewer to think about test strategies.
Tips to avoid having to write tests in the first place.
Content created and presented by Nico Heidtke at the "Die Programmierer" meetup organized by Binary-Gears in Darmstadt, Germany at 02.07.2019.
It's Better To Have a Permanent Income Than to Be Fascinating: Killer Feature...Ultan O'Broin
Presented at Product Camp Dublin 2018. Presentation on picking the right thing to design, right. The Jobs To Be Done framework trumps UX profiles and personas. Keeping it simple, wire-framing best practices, and Lean Startup methodologies included!
How to make change happen in your organisation by talking your devs languageBuiltvisible
This document provides tips on how to improve communication between SEO and development teams to help ensure SEO recommendations are successfully implemented. It recommends delivering recommendations in-person with clear goals, context and prioritization. It also suggests setting up tools for collaboration, integrating SEO into the development workflow, and educating developers on how their work impacts SEO. The overall goal is to make SEO an ally and have recommendations implemented successfully and on time.
Jack Molisani discusses personal branding and career advancement for technical communicators. He provides examples of how to measure your value at work, build professional relationships, and position yourself as an expert who solves business problems. The talk recommends responding to market changes by reeducating yourself and rebranding your skills and services to stay relevant. Personal branding involves promoting your core strengths so people will ask "What should I ask you about?" to hire your specialized services.
Some well-regarded platforms and resources to find web developers include:
Toptal: Claims to represent the top 3% of freelance talent, including web developers.
GitHub: A platform where many developers store their code. You can find potential hires based on their contributions.
Stack Overflow Jobs: Many developers use Stack Overflow for problem-solving. They also have a job listing section.
Upwork & Freelancer: Freelance platforms with a vast pool of developers.
Remember, the "best" developer is subjective and largely depends on the specific needs and nuances of your project.
The document discusses effective Scrum teams and provides information on Scrum roles and practices. It describes the Product Owner, Scrum Master, and Scrum Team roles and their responsibilities. It also covers topics like user stories, estimating work, themes vs epics, and characteristics of good user stories using the INVEST mnemonic. The document aims to help readers understand how to effectively structure work in Scrum.
Elevating the Business Analyst with the lens of Human Centered Design Thinking.
Learn how Human Centered Thinking and User Experience Design can directly address common software pitfalls.
UXPA 2023: Start Strong - Lessons learned from associate programs to platform...UXPA International
Imagine creating experiences for your rookie designers’ first couple years that are rewarding, enriching, and full of learning — without taking all your time or energy to manage. We’ll share techniques any team leader can put into practice using real-life examples from associate programs, apprenticeships, and internships.
Topics include onboarding, varied work challenges, developing multiple capabilities, buddy systems, group sharing, guest speakers, time with executives, and mentorship. We’ll also share how to operationalize learning, soft skills like communication and collaboration, setting boundaries, time management, achieving deep work, and more skills we all wish we were explicitly taught early on.
We’ll focus on modern-day associate programs, but even if you can’t create a full-fledged program, you’ll leave this session with ideas to use with your fledgling professionals. The benefits go beyond efficiency; it’s a foundation for culture, camaraderie, autonomy, and mastery.
UXPA 2023: Disrupting Inaccessibility: Applying A11Y-Focused Discovery & Idea...UXPA International
Digital advances are being made at a rapid-fire pace, yet disability inclusivity continues to fall short of the digital revolution. As the number of people living with disabilities rises, the time to take digital accessibility to the next level is now. Let’s disrupt inaccessibility together! Come hear about a multi-part discovery research and ideation project informing foundational UX designs for our customers. You’ll get insights from our unique study, which are widely applicable across industries, and walk away with tips and inspiration to kick off your own accessibility-focused discovery and ideation. Only YOU can prevent inaccessibility – are you in?
The document discusses the role of user experience (UX) in helping organizations score well on the environmental component of their ESG score. It provides examples of UX practices that can improve an organization's environmental impact, such as advocating for renewable energy sources, optimizing interaction designs to reduce data usage, shortening journey maps to minimize data transmission, using vector graphics instead of heavy file formats, loading content on demand to reduce page load size and emissions, and publishing reports on sustainability practices and carbon emissions.
UXPA 2023 Poster: The Two Tracks of UX Under Agile: Tactical and StrategicUXPA International
The document discusses two sub-tracks for UX under Agile: tactical and strategic. The tactical track focuses on quick tasks and improvements from sprint to sprint, reaching delivery quickly. The strategic track takes a mid-to-long term view through exploratory research to inform product vision and objectives. It recommends doing both tracks simultaneously when possible and prioritizing strategy to balance short-term delivery and long-term planning.
User experience can be drastically elevated by combining data science insights with user-based insights from research. Data analytics on its own can make themes and correlations difficult to explain and to provide accurate recommendations. For example, themes identified via large global surveys and usage data can be better understood with UX insights from focused user research, such as user interviews and/or cognitive walkthroughs. This presentation will highlight the complimentary nature of data science and UX and will focus on the benefits of bringing the two disciplines together. This will be buttressed with practical examples of enterprise projects and applications that combined data and skills from the two disciplines, guidance on how the two disciplines can better work together, and the skills needed to improve as a UX professional when working with data science teams.
UXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden InsightsUXPA International
Users do not always accurately describe what they mean or feel. There are many reasons for this, ranging from politeness to poor introspection, to lack of sufficient technical vocabulary. Fortunately, UX researchers have tools in their trade to deduce what was really meant. We call this UX Fracking, a mixed methods approach that is optimized for extracting hidden user insights. We will illustrate the dangers of inadequate, superficial research, and how this may lead to outcomes incapable of addressing the users’ core issues. We will explore ways to avoid these pitfalls by leveraging mixed research methods to test hypotheses about the users’ intent and needs. This starts with a thorough understanding of who the user is, their goals, and how they work today, to an approach that combines surveys, interviews, and comment analysis with behavioral observation, and finally, validating the newly discovered user insights with the users themselves.
UXPA 2023 Poster: Are virtual spaces the future of video conferencing?UXPA International
Virtual spaces are simulated environments that can range from VR to 2D interfaces, touted as the future of video conferencing. However, they may pose accessibility issues and not be preferred over traditional non-spatial platforms. While virtual spaces could enhance social connection, their complexity risks excluding some users. A combined platform allowing choice of interface could provide an improved experience while maintaining inclusiveness.
UXPA 2023: Learn how to get over personas by swiping right on user rolesUXPA International
This session walks through the concept of user roles as an alternative to personas as a means to generate and disseminate user insights for product development teams. We will describe the tools and methods used to create a research database organized by user roles, along with examples and short exercises to help attendees think through user roles within their own context.
By the end of the session, attendees should be aware of tools and approaches for:
Organizing user research information in a database
Disseminating user role information to product and design teams
Managing a user roles database as part of a long term UX Research program
If you’re ready to ditch personas but don’t know how, this session is for you!
We will present a case study that details our approach for replacing user personas with user roles for a multi-national SAAS company. We will take the audience on a journey that starts with an executive request for personas, travels through the tribulations of realizing personas suck, and concludes with convincing others to accept a new and innovative way to understand the people who use the product. Our key message is that personas lack real value for organizations that already understand the importance of empathizing with users. Building user-centered products requires easily accessible and well organized user insights. We will discuss defining users through a process of stakeholder consultation and content review, and structuring data around Jobs to Be Done and product interactions. We will also discuss the dissemination of user roles in our organization using relational databases, interactive dashboards and online wikis. Spoiler alert, our stakeholders loved user roles!
UXPA 2023: Experience Maps - A designer's framework for working in Agile team...UXPA International
Agile Methodology refers to software design and development methodologies centered around the idea of iterative design and development, where requirements and concepts evolve through collaboration between self-organizing cross-functional teams. Thus, Agile enables teams to deliver value faster, with greater quality and predictability, and greater aptitude to respond to change. With evolving product features every design sprint, designers & researchers find it difficult to follow the design process. This sometimes leads to designs delivered in haste or sub-par design artifacts which result in UX debt. UX debt is accumulated when design teams take actions or shortcuts to expedite the delivery of a piece of functionality or a project which later needs to be refactored. It is the result of prioritizing speedy delivery of design to the development team over a perfect experience journey. Experience Maps is a great tool to practice UX in Agile as well as manage UX Debt.
UXPA 2023: UX Enterprise Story: How to apply a UX process to a company withou...UXPA International
How to build a UX Department from scratch, in an environment they think UX people do social media posters and posts! An agile implementation just started, and people are moving from a waterfall and ad-hoc mindset to agility. In this session, I will talk about my Journey to establish a UX Department for a company that is part of a global brand, but this local branch just started the digital transformation movement. Challenges like: spreading awareness and educating people about UX, hiring the right team, defining the right team structure, establishing workflow and day-to-day operations, and applying localization (non-western culture).
UXPA 2023: High-Fives over Zoom: Creating a Remote-First Creative TeamUXPA International
I started my current job in March of 2020. Many of us remember something clearly about the month that COVID started to shut things down. I remember being surprised to hear that my new on-site-only job would be starting in my living room over zoom. How do you lead a design team when none of the team members live near each other and creativity is highly collaborative? Taking from over a decade of working in HR software, I knew whatever I did needed to put people first. That what employees love about a job is often deeper than the work, it’s the culture, the relationships and people they work with. It’s the feeling that their work has value, and their contribution matters. In this talk I will walk though some of the rituals and best practices I have learned over the last two years building a remote-first creative team.
UXPA 2023: Behind the Bias: Dissecting human shortcuts for better research & ...UXPA International
As humans, we are biased by design. Our intricate and fascinating brains have developed shortcuts through centuries of human evolution. They reduce an unimaginable load of paralyzing decisions, keep us alive, and help us navigate this complex world. Now, these life saving biases affect how we behave with modern technology. Understanding some of the theories and reasons why these biases exist is the key to unlocking their power. In this workshop we will cover some theories around how the brain works. We will review some of our mental shortcuts, take a look at some common biases, and learn how they affect our users, our research, and our designs. Lastly we will review some advantages of biases, and ways to identify and reduce bias. This workshop is targeted for designers who do their own research, and researchers looking to learn more about removing bias from their studies.
UXPA 2023 Poster: Improving the Internal and External User Experience of a Fe...UXPA International
UXPA 2023 Poster: Improving the Internal and External User Experience of a Federal Government Legacy Application Using User Experience and Agile Principles
UXPA 2023 Poster: 5 Key Findings from Moderated Accessibility Testing with Sc...UXPA International
A moderated accessibility testing study conducted by UserTesting between 2021-2022 involved 25+ tests with screen reader users. The study identified 5 key findings about common issues: 1) Unexpected screen reader focus location on pages; 2) Missing alt text for images; 3) Lack of feedback when actions are performed; 4) Insufficient labeling of interactive elements; and 5) Unclear error messages. The study recommends conducting tests with 5 blind participants using the same screen reader, browser and device to standardize results and identify issues violating the Web Content Accessibility Guidelines.
Are you new to UX management, or thinking of getting into management? Then this talk is for you. After reading countless books, attending countless trainings, mentoring and being menteed, nothing quite prepared me for management like my first year. I’ll share with you what I wish they’d told me. I’ll also share my process for generating team research roadmaps, establishing team values, keeping employees motivated, and not burning out.
UXPA 2023: Redesigning An Automotive Feature from Gasoline to Electric Vehicl...UXPA International
This document summarizes the redesign of the Pro Power Onboard feature for electric vehicles from Ford. It discusses how the original gasoline-powered version used a radial dial interface but this would not work for an electric truck with more circuits. User research found the need for increased power and outlets in more locations. An iterative design process involved brainstorming, paper prototyping, and usability testing to create a horizontal gauge interface with on/off and range preservation settings. The final design was validated through testing truck prototypes up to production. Lessons included considering the user experience first and proactive stakeholder involvement.
Building Production Ready Search Pipelines with Spark and MilvusZilliz
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdfflufftailshop
When it comes to unit testing in the .NET ecosystem, developers have a wide range of options available. Among the most popular choices are NUnit, XUnit, and MSTest. These unit testing frameworks provide essential tools and features to help ensure the quality and reliability of code. However, understanding the differences between these frameworks is crucial for selecting the most suitable one for your projects.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slackshyamraj55
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
Driving Business Innovation: Latest Generative AI Advancements & Success StorySafe Software
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxSitimaJohn
Ocean Lotus cyber threat actors represent a sophisticated, persistent, and politically motivated group that poses a significant risk to organizations and individuals in the Southeast Asian region. Their continuous evolution and adaptability underscore the need for robust cybersecurity measures and international cooperation to identify and mitigate the threats posed by such advanced persistent threat groups.
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Main news related to the CCS TSI 2023 (2023/1695)Jakub Marek
An English 🇬🇧 translation of a presentation to the speech I gave about the main changes brought by CCS TSI 2023 at the biggest Czech conference on Communications and signalling systems on Railways, which was held in Clarion Hotel Olomouc from 7th to 9th November 2023 (konferenceszt.cz). Attended by around 500 participants and 200 on-line followers.
The original Czech 🇨🇿 version of the presentation can be found here: https://www.slideshare.net/slideshow/hlavni-novinky-souvisejici-s-ccs-tsi-2023-2023-1695/269688092 .
The videorecording (in Czech) from the presentation is available here: https://youtu.be/WzjJWm4IyPk?si=SImb06tuXGb30BEH .
Dive into the realm of operating systems (OS) with Pravash Chandra Das, a seasoned Digital Forensic Analyst, as your guide. 🚀 This comprehensive presentation illuminates the core concepts, types, and evolution of OS, essential for understanding modern computing landscapes.
Beginning with the foundational definition, Das clarifies the pivotal role of OS as system software orchestrating hardware resources, software applications, and user interactions. Through succinct descriptions, he delineates the diverse types of OS, from single-user, single-task environments like early MS-DOS iterations, to multi-user, multi-tasking systems exemplified by modern Linux distributions.
Crucial components like the kernel and shell are dissected, highlighting their indispensable functions in resource management and user interface interaction. Das elucidates how the kernel acts as the central nervous system, orchestrating process scheduling, memory allocation, and device management. Meanwhile, the shell serves as the gateway for user commands, bridging the gap between human input and machine execution. 💻
The narrative then shifts to a captivating exploration of prominent desktop OSs, Windows, macOS, and Linux. Windows, with its globally ubiquitous presence and user-friendly interface, emerges as a cornerstone in personal computing history. macOS, lauded for its sleek design and seamless integration with Apple's ecosystem, stands as a beacon of stability and creativity. Linux, an open-source marvel, offers unparalleled flexibility and security, revolutionizing the computing landscape. 🖥️
Moving to the realm of mobile devices, Das unravels the dominance of Android and iOS. Android's open-source ethos fosters a vibrant ecosystem of customization and innovation, while iOS boasts a seamless user experience and robust security infrastructure. Meanwhile, discontinued platforms like Symbian and Palm OS evoke nostalgia for their pioneering roles in the smartphone revolution.
The journey concludes with a reflection on the ever-evolving landscape of OS, underscored by the emergence of real-time operating systems (RTOS) and the persistent quest for innovation and efficiency. As technology continues to shape our world, understanding the foundations and evolution of operating systems remains paramount. Join Pravash Chandra Das on this illuminating journey through the heart of computing. 🌟
Best 20 SEO Techniques To Improve Website Visibility In SERPPixlogix Infotech
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Taking AI to the Next Level in Manufacturing.pdfssuserfac0301
Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as:
1. How quickly AI is being implemented in manufacturing.
2. Which barriers stand in the way of AI adoption.
3. How data quality and governance form the backbone of AI.
4. Organizational processes and structures that may inhibit effective AI adoption.
6. Ideas and approaches to help build your organization's AI strategy.
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on automated letter generation for Bonterra Impact Management using Google Workspace or Microsoft 365.
Interested in deploying letter generation automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Skybuffer SAM4U tool for SAP license adoptionTatiana Kojar
Manage and optimize your license adoption and consumption with SAM4U, an SAP free customer software asset management tool.
SAM4U, an SAP complimentary software asset management tool for customers, delivers a detailed and well-structured overview of license inventory and usage with a user-friendly interface. We offer a hosted, cost-effective, and performance-optimized SAM4U setup in the Skybuffer Cloud environment. You retain ownership of the system and data, while we manage the ABAP 7.58 infrastructure, ensuring fixed Total Cost of Ownership (TCO) and exceptional services through the SAP Fiori interface.
Introduction of Cybersecurity with OSS at Code Europe 2024Hiroshi SHIBATA
I develop the Ruby programming language, RubyGems, and Bundler, which are package managers for Ruby. Today, I will introduce how to enhance the security of your application using open-source software (OSS) examples from Ruby and RubyGems.
The first topic is CVE (Common Vulnerabilities and Exposures). I have published CVEs many times. But what exactly is a CVE? I'll provide a basic understanding of CVEs and explain how to detect and handle vulnerabilities in OSS.
Next, let's discuss package managers. Package managers play a critical role in the OSS ecosystem. I'll explain how to manage library dependencies in your application.
I'll share insights into how the Ruby and RubyGems core team works to keep our ecosystem safe. By the end of this talk, you'll have a better understanding of how to safeguard your code.
2. Disclaimer
• I have previously worked as a contractor at Amazon on multiple occasions
• This deck is in accordance with my various NDAs
• The intent of this talk is to share my experiences & nuances with:
1. Working with the Mechanical Turk front-end
2. Managing and working with Turkers
3. Managing live studies & reviewing their status
a. Some Thoughts on activities well-suited for the Mechanical Turk environment
4. Using Mechanical Turk as a supplemental tool for managing aspects of traditional
usability studies
• This deck is certainly not the “Gospel According to Marc”
• The talk is not intended to be either a recommendation or endorsement for
Mechanical Turk or third-parties who provide Mechanical Turk vendor services
• Also, not intended to be an “Intro to Using” or “How To Plan & Run Your Study” talk
• Lots of these already exist
2
3. My Background
• Have an MS in Computer Science, with a concentration in AI Natural
Language Computing
• Transitioned from “Classical AI” to complete a Ph.D. with a
concentration in Cognitive Psychology & Instructional Systems Design
• Implicitly, this is “Applied AI”
• Worked as a Tactical User Researcher for about 8 years
• Completed about 70 UX lab studies, and 35+ Mechanical Turk studies
• Quick Poll: How many people consider euthanasia a particularly
contentious topic? Show of hands, please.
3
4. The Difficulty With Disambiguation – AI is Hard
Originally used in a Benny Hill sketch and “borrowed” by SNL
4
5. Mechanical Turk – Brief Introduction to AAI
5
Amazon Mechanical Turk Login “Splash” Page
6. Mechanical Turk – The Basics: What it is
• Amazon’s “High Concept” premise is that Mechanical Turk is an internet-
based source for “Artificial Artificial-Intelligence”
• Provide a marketplace where Requesters can post tasks that Workers can perform
• Specifically, tasks a computer can’t handle
• Example: When I show you any legal English string, please tell me the number of syllables it has
• Requesters – Post “Human Intelligence Tasks” (AKA HITs) to be completed
for a denoted payment consideration upon completion
• Workers – Select HITs to perform and are paid upon submission and
Requester’s review & approval of completed work
• Workers are informally known as “Turkers”
6
7. Process & Mechanisms
• Front End – Mechanical Turk “Proper”
• Requesters: Create & Manage Tasks; Manage & Pay Workers; Manage Payment Account
• Workers: Select Tasks; Submit Verification Numbers; Manage Payment Receivables
• Back End – Link to external website where the “work” is completed
• For Requesters:
• Link to a web site created from scratch, or in conjunction with some third party service provider
• Important to Note: Mechanical Turk is NOT an end-to-end turn-key solution
• You the User Researcher must provide the activity back-end
• For Workers: The back end is the place to
• Either perform work or upload completed work
• Retrieve Task Verification Numbers (so that they are paid)
7
8. 1. Working With the Mechanical Turk Front-End
8
Root-Level Page for Creating a New Project – Clicking Create Project Button will display screen on Slide 10
9. Within the Mechanical Turk “Front End” UI
• Requesters Can
• Create a New Study Project Description
• Manage Worker Logs
• Manage Live Studies and Review Their Status
• Important to Note: The Front End Handles the Bookkeeping
• Actual study materials/protocol is completely external to the front end
• Your Project Description contains a link to the external Project Proper
• Upon completing the project, the worker is provided with a verification number
• The worker inputs that number into a form on the HIT Screen within the Worker UI front end
• The verification number is the underlying link between the front end Description and the
back end Project Proper
9
10. Creating a New Study Project Description
• Requesters Create a New Study Project by:
• Entering Study “Properties”
• Setting the “Design Layout”
• Previewing the “Back End” Link
10
The Enter Properties Screen – Above the Fold
11. Creating New Study Description – Properties
• Entering Study Properties Entails
• Providing a Title and Description of the Human Intelligence Task
• Important:
1. Lead with a Verb and clearly delineate the type of activity and the related topic/subject
2. Pretend your Editor is Jethro Bodine or Britney Spears, and write for a 6th Grade Education
• Example: Answer a brief survey about listening to music on a mobile device
• Determining the N; Time on Task; Gratuity Amount
• Strong Suggestion: Determine your Target N and add 15-20% to yield an Actual N
• Retain the first Target-N number of submissions for your analysis
• From personal experience, I typically had to throw away 15%+ of responses b/c they were unusable
• Examples: Rating everything a “5” (even the inverted questions); Selecting a corner or blank space
• Setting Worker Requirements – Some Recommendations
• Set Geo-Location to the country where your primary target segment(s) reside (E.G. USA)
• Do NOT require workers to be “Masters” (Requiring Masters is the default, so be sure to uncheck box)
• Set Visibility to either Private or Hidden (so that only qualified workers can preview HIT details)
11
12. • Setting the “Design Layout” entails
• Authoring Task Instructions
• Again, channel your inner 6th grader
• Be sure to clearly delineate any Restrictions or Qualifications – but use gentle language
• Examples:
• First Time Participants Only – If you’ve already taken one of our surveys, we might not accept and
pay you for your subsequent submission
• Please take our Qualification Survey if you would like to work on this task
• Providing the link to the external Project Proper
• Important: DO NOT DISPLAY THE URL as the link
• Click on the Source Button in the UI, and edit so the link displays a brief descriptor, but not the URL
• Displaying the URL is an open invitation to be gamed – cheats will go directly to your link
• Including a form to input the Survey Code
• Note: the Survey Code is the only proof that Worker actually competed the task
• If a worker leaves the form blank, always send a polite note about the importance of including the
Survey Code, and never approve submission for payment to prevent gaming
12
Creating New Study Description – Design Layout
14. • Previewing your project entails
• Checking that the link to the Project Proper is live and typo free
• Strongly Recommend: Doing a stakeholder cog walk prior to going live
• “Masters granted” is a default qualification unless you deselect it in the Enter Properties tab
• Checking that the Instructions section to the project is well written & typo free
• Recommend that you have 2-3 colleagues proof & review Instructions section
14
Creating New Study Description – Preview
The Preview Screen
15. Summary SWOT of Mechanical Turk
• Strengths
• Weaknesses
• Opportunities
• Threats
15
16. Mechanical Turk – Strengths
• Aside for some fundamental Right to Privacy restrictions and guards
to protect workers from known internet scams, your backend link can
go practically anywhere on the web, and your task in theory is only
limited to your imagination
• It is possible to assign piece-work at very low costs
• Provides a good forum to get “quick & dirty” data on the cheap
16
17. Mechanical Turk – Weaknesses
• Workers Might Not
• Represent your Targeted User Segments
• Be able to financially afford your product
• Possibility of a false positive/negative data submission because worker is non-customer
• Have any intrinsic interest in your product or service
• Might only be interested in doing what they need to earn your payment
• Workers Might
• Make choices and provide opinions based on what they think you want to hear
• Simply go through the motions, and provide derivative or random feedback
• Skip steps and rush through activities without any conscientious thought
• Intentionally provide disinformation, just to do it
17
18. Mechanical Turk – Opportunities
• Provides a venue for
• Testing early stage concepts quickly & inexpensively
• Conducting research about your competition “anonymously”
• Generating “Big Data” on the fly
• Example: Social Media companies can insert links within content to garner feedback
• Non-technical Requesters to obtain data via third party vendor provided tools
• Off-loading time intensive tasks such as phone screening (more on this later)
18
19. Mechanical Turk – Threats
• Using MT successfully requires more than a modicum of vigilance
• Garbage In Garbage Out – Requesters have to guard against:
• “Gamed” or otherwise “Bad” data
• Many workers will rush and skip steps if they think they can get away with it
• Publicity
• From personal experience, publicity attracts a lower-quality worker
• Getting any kind of reputation
• It’s almost always better to quietly pay for data that will be thrown out than to risk negative feedback
• False Positives & Satisfiers
• A “Satisfier” is a worker who tells you what he/she thinks you want to hear
• Example: “That’s awesome! I’d absolutely pay $300 for a narrated 3D video of my cat’s autopsy”
• Nothing is explicitly provided to prevent Turkers from multiple participation
• Creating Screening guards is not intuitive (will be discussed in a later slide)
19
20. 2. Managing and Working With Turkers
20Photo Source: http://www.wired.com/2008/12/anonymity-for-sale-on-mechanical-turk/
21. Typical Turker Characteristics
• It’s all about the Franklins
• Not uncommon for Turkers to be stay-at-home, students, or care-givers
• For most Turkers, MT provides a source for self-paced supplemental income
• Generally, Turker discussion forums tend to focus on HITs that are
• Paying good rates
• Easy to finish
• From requesters that pay promptly
• Note: I’ve never seen a thread call-out HITs that were “fun” or “stimulating”
• Important to remember that these individuals are literally working for
nickels and dimes
21
23. Sampling of Sites Where Turkers Congregate
• MTURKGRIND
• http://www.mturkgrind.com/
• TurkerNation
• www.turkernation.com/
• Reddit: Mechanical Turk Blog
• https://www.reddit.com/r/mturk
• Reddit: HITs Worth Turking For
• https://www.reddit.com/r/HITsWorthTurkingFor/
• Note: Reddit and their Blogs typically pay contributors for entries that receive
heavy traffic and heavy vote-up ranking promotions
• It’s possible to earn more for sharing a task link than actually doing the task
• Turkopticon
• https://turkopticon.ucsd.edu/
23
24. A Word About Turker Discussion Forums
• Rule of Thumb: ALL PUBLICITY IS BAD
• “Positive” Publicity Encourages Gamers & Satisfiers
• “Negative” Publicity can be Highly Alienating (can scare away “good” workers)
• Never “Block” a Worker
• Never Deny a Payment
• What to do when you get “junk” data
• Pay the Bad Worker(s) – Try to practice “just pay and put to sleep”
• Paid workers don’t Bitch & Moan on Public Forums
• Classify worker as a “BW = 1” (or whatever masked coding you prefer)
• If needed, run a “filler study” and take the first X viable/quality responses that get you to
your intended Target-N
• Example, If your Target N = 20 and you’ve tossed 5 submissions, you could then run a filler study
where you would retain the first 5 viable responses, which would then put your N back at 20.
Alternatively, bump-up your N by 20% to begin with, and just retain the first Target-N responses.
• As mentioned, expect to toss 15% to 20% of your data for every HIT 24
25. INSTRUCTIONS:
Please click on a page feature
and explain why you selected it.
Example of a
Bad Response
25Source: Synthesized from www.Disney.com/
Unusable Response on Next Slide
26. Explain why you selected your choice:
“It was interesting.”
Sample Bad
Data Response
26
Explanation: This is Bad Data because the
worker selected “dead air” outside of the
target area, and provided an explanation
that was devoid of any contextual value.
User Clicked Here
27. 3. Managing Live Studies & Reviewing Status
27
Source: http://journal.code4lib.org/articles/6004
28. Managing & Reviewing Studies – Potential Issues
• Payment Rate is a VERY contentious topic
• Some Turkers have tried organizing and lobbying for “fair market turking”
• Important to have a Screening tool to guard against repeated participation
• Workers will try to double-dip if they are allowed to
• Important to be very methodical with keeping Screener Categories up to date
• There is a constant need to check for bogus data
• Time factors related to when you launch a study can affect your data
28
29. Some Study Launch Considerations
• Launching studies to run over the weekend or holidays is risky
• Weekend Warriors are different – data tends to be all over the place compared to studies
that launch on a Tuesday Morning
• Launching late Friday afternoon will attract weekend warriors
• Studies set to run during legal holidays will also attract weekenders
• Launching on a Monday can be risky for the same reason
• Experience has shown that “Monday Turkers” were both more impulsive and more likely to
be satisfiers
• The time a study is launched can also affect the data distribution
• Launching early morning on West Coast is different than an early morning launch on the East
Coast
• Generally, the “best time” to launch is from 9AM to 10AM for a targeted Geo-Loc
• Best days are Tue, Wed, Thurs, (early Friday is okay, but you want the HIT finished by 4PM)
29
30. Suggested Pay Rates
• Objective: Pay well enough to encourage participation, but not so well
that you create a publicity stampede
• Rule of Thumb: 3 to 5 cents per click, or short-activity (E.G.S. Steps 2 & 3)
• Example: 15 Cent HIT with opportunity for 5 Cent Bonus
• Step 1. Click the Diagram You Preferred.
• Step 2. In 3 to 15 sentences please explain why you made your selection.
• Step 3. Please provide your level of experience or familiarity with <Context>.
• Bonus. Paid on case-by-case for high quality, thoughtful submissions
• Some approaches to setting rates
• Billing by estimated time-on-task
• Billing by number of interactions
• Utilizing Bonuses to reward quality work 30
31. Some Approaches to Setting Pay Rates
• Billing by Estimated Time
• $6.00/Hour = .10/minute (i.e., $1.00/10 Minutes – Baseline of “fair turking”)
• $15.00/Hour = .25/minute (Minimum Wage in Seattle; $2.50/10 Minutes)
• Billing by Interaction
• $6.00/Hour = 2 mills/click (i.e., .0016/click, rounded-up)
• $15.00/Hour = .025/click
• Using Bonuses (Token In-Exchange for Preferred Service)
• Reward for Quality Work (especially if it goes above & beyond)
• Don’t have to deny an initial, smaller payment to workers who skip steps or “phone it in”
• Well suited for a more involved “try it out in a store and tell us your thoughts” task
• Crucial that expectations for bonus are clearly delineated
• Modulo 10 Clustering – Try to group your workers into sets of 10
• Amazon’s 20% fee doubles when N > 10 (current pricing model)
• E.G. Bumped-Up N = 25 done as 3 studies where N = 8, 8, 9 (run serially; do Paired-T Tests)
31
32. Managing Turker Payments
• Try to approve payments the same day that your activity is completed
• Always review your data set before doing any payment approvals
• Also note that Study/Activity Data is accessed externally from the MT Front-End
• Every data set from a worker will have a corresponding survey code
• Survey Codes are visible in the MT Front-End within a Batch Detail Listing
• Accessed via clicking the Results Button for a given Study Batch
• Not uncommon to have data entries with missing or bogus survey codes
• Only retain data from workers who have survey codes that match the MT Batch Listing
• Only pay workers explicitly denoted in the MT Batch Listing
32
Batch Detail List
33. Screening Worker Candidates
• MT lets you create Categories for managing worker access to HITS
• To create a Category, click the Qualifications Types link within the Manage tab
• Potential Uses
• Screen for prior participation within a given study type
• Blacklisting for gaming behavior
• Profile-Qualifying workers
• Example: Want individuals proficient in Halo
• Administer 20-27 question pretest on gaming (Don’t share scoring criteria)
• Intersperse questions about Halo that only an advanced/proficient would know
• People who score 85%+ on the Halo questions would qualify
• Denote MVPs
33
34. Screening Candidates for Prior Participation
• Within the Manage tab
• Create a worker qualification column and label it “Opted-In <Study Type>”
• If a Worker has submitted a HIT for a given Study Type
• Then manually set the corresponding qualification column value to “1”
• When you run a HIT of <Study Type> require Opted-In equals 0 to qualify
• Workers with prior participation will be unable to access HITS of that Study Type
34
35. Managing HIT Access via Qualifications
• This approach for screening-out prior participants can also be used for
factors such as:
• Pre-Qualifying Workers (If Profile-Qualified <Study Type> = 1 Then Allow Access)
• Workers would complete a Screener Activity (like the Halo example on Slide 31)
• If they are successful, then set Profile-Qualified <Study Type> = 1 for those workers
• Denoting MVPs (Qualification MVP = 1)
• Blacklisting (If BL = 1 Then Deny Access)
• Since qualifications are public facing (to the individual worker), use a masked term
• Explicitly blocking someone might cause negative publicity about you
35
36. Summary: Some Approaches for Mitigating Risk
• Carefully crafting task instructions
• Use 6th Grade English
• Unchecking Require Masters rating
• Conduct an internal walkthrough to validate content & typo check
• Controlling Access to Your HITs
• Setting-Up an “Opting-In” qualification categories
• Setting HIT visibility to either Private or Hidden
• Some Things to Watch for When Reviewing Turker Submissions
• “Phoning it in”
• “Satisfiers”
• Workers skipping steps
36
37. 3a. Some Thoughts on Activities well-suited for
the Mechanical Turk Environment
37
Source: http://kernelmag.dailydot.com/features/report/4732/my-gruelling-day-as-an-amazon-mechanical-turk/
38. Activities potentially well-suited for MT
• Surveys
• First Impressions Feedback
• Supplemental Feedback to delivered content (Twitter does this)
• Script-Directed In-Store “Ethnography” (inherently risky)
• Go to a B&M; do activities & take notes; upload notes & take this survey
• Card Sorting and Affinity Diagramming (via Html5, or JavaScript)
• Each card would have a drop-down menu of numbers (# of cards/groups)
• User would set/reset the drop-down for each card, and then click submit
• For stack ranking, include a guard so a given drop-down number could only be selected once
• Reviewing Tutorial/Documentation Effectiveness (Pre/Post Treatment)
38
39. 4. Using Mechanical Turk as a Supplemental Tool
39
Source: Synthesized from a free-use image and utilized in a Participant Recruiting Flyer
40. Advanced Topics
• Using Mechanical Turk for supplemental and off-loaded tasks
• Recruiting – Participant-Directed Screeners
• As an alternative to conducting phone screeners to profile-qualify candidates to participate in
a UX study, the candidates can be directed to an online survey within MT
• MT Terms of Usage allow you to send specific people to a HIT
• Important: Set HIT visibility to hidden so only your recruits see the HIT
• Working with Remote Participants
• Participant-Directed protocols (recommend tightly choreographed script)
• Example: Flash-Based forward-chaining script where data was submitted in excel form to a Cold
Fusion-Based server (Data then retrieved via ftp)
• Facilitator-Directed protocols with remotely accessed questionnaire packets
• Example: Focus Group with a combination of local and remote participants
• You don’t want the remotes to physically possess your protocol script
• Implement survey elements of protocol as a SharePoint site, use MT to control/restrict access
• Managing Gratuities
• Importance of Modulo 10 clustering for optimal budgeting
• 20% “standard” fee becomes a 40% fee if N > 10
40
41. Using MT for Managing “Regular” UX Gratuities
• Not all companies have a Usability Central facility like Microsoft
• Side Effect: Principal Investigator is responsible for ensuring that Participant W9 and 1099
forms are accurately completed and properly submitted to the IRS
• Case Example
• At one of my assignments, my manager used an external vendor to provide lab space; schedule
participants; and distribute gratuities.
• It was cheaper to pay vendor than the Researcher to do the scheduling & phone screening
• More time & cost efficient for Manager to off-load tax form verification to vendor
• If something goes awry, the vendor deals with the Fed
• Approach
• Create a Category
• Set HIT visibility to Hidden
• Each participant is given a Group ID, Unique ID, and Pass-Phrase as identifiers
• Participants would be grouped across multiple HITs, modulo 10 (as needed)
• The HIT would involve inputting the identifier info
• Requester would carefully verify & validate each submission, and approve payment
• Amazon would collect a 20% fee for its troubles
Disclaimer: Neither the Presenter nor UXPA will be held responsible if you choose to use this method
41