This document discusses using an unmoderated online usability testing service called TryMyUI to test a website. It describes how TryMyUI provides a panel of participants to complete predefined tasks on a site while verbally commenting. The document outlines a test of dropdown menus on the BLS.gov site using TryMyUI, comparing 4 prototypes with 35 total participants. Analysis of task success rates, times, and open-ended feedback suggested prototype C as the best design. Advantages of TryMyUI included lower cost and time compared to in-person testing.
Sensible Specialist Service Responses to the Methamphetamine “Crisis”Uniting ReGen
APSAD 2014 presentation by Trevor King on current community concerns about methamphetamines and ReGen's experience in developing targetted treatment models for people affected by methamphetamine use.
Explores web usability and offers approaches to make web sites easy to use for an end-user, without requiring the user to undergo any specialized training. Creating websites that intuitively relate the performance actions needed on the web page to the user’s experience and expectations, the web designer/developer is able to present the information to the user in a clear and concise way, to give the correct choices to the users, in a very obvious way, to remove ambiguity regarding the consequences of an action and put the most important thing in the right place on a web page or a web application.
Sensible Specialist Service Responses to the Methamphetamine “Crisis”Uniting ReGen
APSAD 2014 presentation by Trevor King on current community concerns about methamphetamines and ReGen's experience in developing targetted treatment models for people affected by methamphetamine use.
Explores web usability and offers approaches to make web sites easy to use for an end-user, without requiring the user to undergo any specialized training. Creating websites that intuitively relate the performance actions needed on the web page to the user’s experience and expectations, the web designer/developer is able to present the information to the user in a clear and concise way, to give the correct choices to the users, in a very obvious way, to remove ambiguity regarding the consequences of an action and put the most important thing in the right place on a web page or a web application.
Red Hat JBoss BRMS and BPMS Workbench and Rich Client TechnologyMark Proctor
This is an overview video that shows the scope of work and technology used within the Red Hat JBoss BRMS and BPMS platforms.
The technology presented builds with GWT, Errai and UberFire as the foundation. Over 2015 we'll be working to make it for end users to consume the bits they need, paying for only what they use, so others can make power web platforms like BRMS and BPMS.
Materi yang ada pada slide ini berisi :
Android version
Platform architecture
App Component
Kotlin
Struktur Project
Activity Lifecycle
Melempar Nilai
Layouting LinearLayout
Layouting RelativeLayout
Look And Feel
Custom Buttom
----------------------------------------------------------------------
Keep in touch with me in :
Email : rizkiadamunikom@gmail.com
The potential in Drupal 8.x and how to realize itAngela Byron
As of Drupal 8.0.0, we've adopted a new release cycle that enables us to ship "minor" releases every 6 months with new, backwards-compatible features. This talk discusses possible implementation of this that attempts to strike a balance between agility and including all of the relevant stakeholders in feature decisions.
AwToolkit: Attention-Aware User Interface WidgetsAaron Quigley
AVI 2014 Best Paper: http://dx.doi.org/10.1145/2598153.2598160
See patent: Visual focus-aware techniques for visualizing display changes - https://patents.google.com/patent/US20150227334A1
Increasing screen real-estate allows for the development of applications where a single user can manage a large amount of data and related tasks through a distributed user interface. However, such users can easily become overloaded and become unaware of display changes as they alternate their attention towards different displays. We propose AwToolkit, a novel widget set for developers that supports users in maintaining awareness in multi-display systems. The AwToolkit widgets automatically determine which display a user is looking at and provide users with notifications with different levels of subtlety to make the user aware of any unattended display changes. The toolkit uses four notification levels (unnoticeable, subtle, intrusive and disruptive), ranging from an almost imperceptible visual change to a clear and visually salient change. We describe AwToolkit’s six widgets, which have been designed for C# developers, and the design of a user study with an application oriented towards healthcare environments. The evaluation results reveal a marked increase in user awareness in comparison to the same application implemented without AwToolkit.
Abstract 2: "Advanced testing in action on a Java project"
In 2019 we're all used to writing automated tests in Java projects. It's now time to move up the chain and learn how to implement more complex type of testing.
This talk will demonstrate advanced testing practices used by the XWiki open source project (http://xwiki.org), and using Java, Maven, Docker and Jenkins and more:
* Testing for backward compatibility with Revapi and an associated strategy
* Testing for coverage with Jacoco and defining a viable strategy for slowing improving the situation
* Testing the quality of your tests with Descartes Mutation testing
* Automatically enriching your test suite with DSpot
* Testing various configurations with Docker containers and Jenkins
Unleashing the Java Tooling in Eclipse IDE - Tips & Tricks!Noopur Gupta
With faster Java releases, it is an exciting time to be a Java developer. The new features in Java are changing the way you write code and to give it a spin, you have the tooling support in your favorite open-source Eclipse IDE ready at your disposal.
To be productive, mastering your IDE is as important as mastering your source code. In this session, I will unleash many tips and tricks that will make your experience more convenient and pleasant while working with Java in Eclipse IDE.
During this talk, you will have many "Ah, I didn't know Eclipse can do that!" moments. Some of the examples include determining who last modified a line of code and when, using trigger points and tracepoints while debugging, working with new constructs like lambdas and records.
Come and learn about the best Eclipse IDE capabilities that you might not be knowing yet. There will be something new for everybody.
Droidcon Spain 2016 - The Pragmatic Android Programmer: from hype to realityDaniel Gallego Vico
This presentation shows how to be a pragmatic Android programmer by showing real examples of applications/products developed in BQ attending to three important topics:
- Why and how to implement a pragmatic CLEAN architecture with a custom dependency injection framework and ReactiveX features.
- Why and how to implement unit, integration, view and smoke tests in every CLEAN layer and what tools should be used.
- Why and how to implement pragmatic Continuous
Development/Testing/Integration/Delivery by showing several tricks, plugins and snippets that you could use as a daily basis.
An application for regression testing of the PVS-Studio and CppCat analyzers.
Implementing the mechanism of regression testing of the static code analyzers PVS-Studio and CppCat.
Ensuring that tests are run on a large number of open-source C/C++ projects.
Ensuring testing of the analyzers’ operation under all the supported Visual Studio versions.
Providing convenient handling of the differences list, quick view feature, apply changes feature, and so on.
Advanced A/B Testing at Wix - Aviran Mordo and Sagy Rozman, Wix.comDevOpsDays Tel Aviv
While A/B test is a very known and familiar methodology for conducting experiments on production when you do that on a large scale it has many challenges in the organization level and operational level.
At Wix we are practicing continuous delivery for over 4 years. Conducting A/B tests and writing feature toggles is at the core of our development process. However when doing so on a large scale, with over 1000 experiments every month, it holds many challenges and affect everyone in the company, from developers, product managers, QA, marketing and management.
In this talk we will explain what is the lifecycle of an experiment, some of the challenges we faced and the effect on our development process.
* How an experiment begins its life
* How an experiment is defined
* How do you let non technical people control the experiment while preventing mistakes
* How an experiment go live, what is the lifecycle of an experiment from beginning to end
* What is the difference between client and server experiments
* How do you keep the user experience and not confuse them
* How does it affect the development process
* How can QA test an environment that changes every 9 minutes
* How can support help users when every user may be part of different experiment
* How can we find if an experiment is causing errors when you have millions of permutations [at least 2^(number of active experiments)]
* What are the effects of always having multiple experiments on system architecture
* What are the development patterns when working with AB test
At Wix we have developed our 3rd generation experiment system called PETRI, which is (will be) open sourced, that helps us maintain some order in a chaotic system that keep changing. We will also explain how PETRI works, what are the patterns in conducting experiments that will have a minimal effect on performance and user experience.
Bottom-up approach to relieve ViewController's from some responsabilities.
The first part is about how we can define a component and then how these components interact between them.
Breaking Dependencies to Allow Unit TestingSteven Smith
Unit testing software can be difficult, especially when the software wasn't designed to be testable. Dependencies on infrastructure concerns and software we don't control are one of the biggest contributors to testing difficulty. In this session, you'll learn the difference between unit tests and other kinds of tests, how to recognize and invert dependencies, and how to unit test your code's interactions with these dependencies without testing the infrastructure itself.
Experimenting on Humans’ (or, ‘everything you want from an a/b testing system’).
How do you know what 100 millions users like? Wix.com is conducting hundreds of experiments per month on production to understand which features our users like and which hurt or improve our business. At Wix we have developed our 3rd generation experiment system called Petri, which was open sourced. Petri helps us maintain some order in a chaotic system that keeps changing. We will also explain how it works and what are the patterns in conducting experiments that have a minimal effect on performance and user experience.
Red Hat JBoss BRMS and BPMS Workbench and Rich Client TechnologyMark Proctor
This is an overview video that shows the scope of work and technology used within the Red Hat JBoss BRMS and BPMS platforms.
The technology presented builds with GWT, Errai and UberFire as the foundation. Over 2015 we'll be working to make it for end users to consume the bits they need, paying for only what they use, so others can make power web platforms like BRMS and BPMS.
Materi yang ada pada slide ini berisi :
Android version
Platform architecture
App Component
Kotlin
Struktur Project
Activity Lifecycle
Melempar Nilai
Layouting LinearLayout
Layouting RelativeLayout
Look And Feel
Custom Buttom
----------------------------------------------------------------------
Keep in touch with me in :
Email : rizkiadamunikom@gmail.com
The potential in Drupal 8.x and how to realize itAngela Byron
As of Drupal 8.0.0, we've adopted a new release cycle that enables us to ship "minor" releases every 6 months with new, backwards-compatible features. This talk discusses possible implementation of this that attempts to strike a balance between agility and including all of the relevant stakeholders in feature decisions.
AwToolkit: Attention-Aware User Interface WidgetsAaron Quigley
AVI 2014 Best Paper: http://dx.doi.org/10.1145/2598153.2598160
See patent: Visual focus-aware techniques for visualizing display changes - https://patents.google.com/patent/US20150227334A1
Increasing screen real-estate allows for the development of applications where a single user can manage a large amount of data and related tasks through a distributed user interface. However, such users can easily become overloaded and become unaware of display changes as they alternate their attention towards different displays. We propose AwToolkit, a novel widget set for developers that supports users in maintaining awareness in multi-display systems. The AwToolkit widgets automatically determine which display a user is looking at and provide users with notifications with different levels of subtlety to make the user aware of any unattended display changes. The toolkit uses four notification levels (unnoticeable, subtle, intrusive and disruptive), ranging from an almost imperceptible visual change to a clear and visually salient change. We describe AwToolkit’s six widgets, which have been designed for C# developers, and the design of a user study with an application oriented towards healthcare environments. The evaluation results reveal a marked increase in user awareness in comparison to the same application implemented without AwToolkit.
Abstract 2: "Advanced testing in action on a Java project"
In 2019 we're all used to writing automated tests in Java projects. It's now time to move up the chain and learn how to implement more complex type of testing.
This talk will demonstrate advanced testing practices used by the XWiki open source project (http://xwiki.org), and using Java, Maven, Docker and Jenkins and more:
* Testing for backward compatibility with Revapi and an associated strategy
* Testing for coverage with Jacoco and defining a viable strategy for slowing improving the situation
* Testing the quality of your tests with Descartes Mutation testing
* Automatically enriching your test suite with DSpot
* Testing various configurations with Docker containers and Jenkins
Unleashing the Java Tooling in Eclipse IDE - Tips & Tricks!Noopur Gupta
With faster Java releases, it is an exciting time to be a Java developer. The new features in Java are changing the way you write code and to give it a spin, you have the tooling support in your favorite open-source Eclipse IDE ready at your disposal.
To be productive, mastering your IDE is as important as mastering your source code. In this session, I will unleash many tips and tricks that will make your experience more convenient and pleasant while working with Java in Eclipse IDE.
During this talk, you will have many "Ah, I didn't know Eclipse can do that!" moments. Some of the examples include determining who last modified a line of code and when, using trigger points and tracepoints while debugging, working with new constructs like lambdas and records.
Come and learn about the best Eclipse IDE capabilities that you might not be knowing yet. There will be something new for everybody.
Droidcon Spain 2016 - The Pragmatic Android Programmer: from hype to realityDaniel Gallego Vico
This presentation shows how to be a pragmatic Android programmer by showing real examples of applications/products developed in BQ attending to three important topics:
- Why and how to implement a pragmatic CLEAN architecture with a custom dependency injection framework and ReactiveX features.
- Why and how to implement unit, integration, view and smoke tests in every CLEAN layer and what tools should be used.
- Why and how to implement pragmatic Continuous
Development/Testing/Integration/Delivery by showing several tricks, plugins and snippets that you could use as a daily basis.
An application for regression testing of the PVS-Studio and CppCat analyzers.
Implementing the mechanism of regression testing of the static code analyzers PVS-Studio and CppCat.
Ensuring that tests are run on a large number of open-source C/C++ projects.
Ensuring testing of the analyzers’ operation under all the supported Visual Studio versions.
Providing convenient handling of the differences list, quick view feature, apply changes feature, and so on.
Advanced A/B Testing at Wix - Aviran Mordo and Sagy Rozman, Wix.comDevOpsDays Tel Aviv
While A/B test is a very known and familiar methodology for conducting experiments on production when you do that on a large scale it has many challenges in the organization level and operational level.
At Wix we are practicing continuous delivery for over 4 years. Conducting A/B tests and writing feature toggles is at the core of our development process. However when doing so on a large scale, with over 1000 experiments every month, it holds many challenges and affect everyone in the company, from developers, product managers, QA, marketing and management.
In this talk we will explain what is the lifecycle of an experiment, some of the challenges we faced and the effect on our development process.
* How an experiment begins its life
* How an experiment is defined
* How do you let non technical people control the experiment while preventing mistakes
* How an experiment go live, what is the lifecycle of an experiment from beginning to end
* What is the difference between client and server experiments
* How do you keep the user experience and not confuse them
* How does it affect the development process
* How can QA test an environment that changes every 9 minutes
* How can support help users when every user may be part of different experiment
* How can we find if an experiment is causing errors when you have millions of permutations [at least 2^(number of active experiments)]
* What are the effects of always having multiple experiments on system architecture
* What are the development patterns when working with AB test
At Wix we have developed our 3rd generation experiment system called PETRI, which is (will be) open sourced, that helps us maintain some order in a chaotic system that keep changing. We will also explain how PETRI works, what are the patterns in conducting experiments that will have a minimal effect on performance and user experience.
Bottom-up approach to relieve ViewController's from some responsabilities.
The first part is about how we can define a component and then how these components interact between them.
Breaking Dependencies to Allow Unit TestingSteven Smith
Unit testing software can be difficult, especially when the software wasn't designed to be testable. Dependencies on infrastructure concerns and software we don't control are one of the biggest contributors to testing difficulty. In this session, you'll learn the difference between unit tests and other kinds of tests, how to recognize and invert dependencies, and how to unit test your code's interactions with these dependencies without testing the infrastructure itself.
Experimenting on Humans’ (or, ‘everything you want from an a/b testing system’).
How do you know what 100 millions users like? Wix.com is conducting hundreds of experiments per month on production to understand which features our users like and which hurt or improve our business. At Wix we have developed our 3rd generation experiment system called Petri, which was open sourced. Petri helps us maintain some order in a chaotic system that keeps changing. We will also explain how it works and what are the patterns in conducting experiments that have a minimal effect on performance and user experience.
Similar to Unmoderated, Online Usability Testing for Web (Brandon Kopp & BIll Mockovak) (20)
Wireframing with Your Team in Mind (Susana Esparza & Jason Kolaitis & Jennife...
Unmoderated, Online Usability Testing for Web (Brandon Kopp & BIll Mockovak)
1. The Use of an Unmoderated,
Online Usability Testing
Service to Test a Website
Brandon Kopp & Bill Mockovak
Research Psychologist
Office of Survey Methods Research
UXPA-DC User Focus Conference
19 October 2012
2. TryMyUI
• Provides panel of participants that are presented with user
defined tasks
• You receive a video screenshot of the participant’s
computer with an audio voiceover of their comments
2
3. Many Alternatives
Userzoom.com
UserTesting.com
UTest.com
OptimalWorkshop.com
OpenHallway.com Loop11.com
4. The BLS.gov Dropdown Usability Test
• During earlier
usability
testing, several
users had
problems with
immediately
activating
dropdown menus
• Users would
accidentally click
menu links and
get lost
5. Prototypes
• Test design alternatives for mega-dropdowns
Type of Menu
Prototype Display Width List of Subject Areas
Control
Partial list of content, with
A Click to show/close Fixed, 3 columns
‘view all’ option
B Hover, no delay Fixed, 1 column Compact
C Hover, with delay Fixed, 3 columns Current content
D Hover, no delay Fixed, 3 columns Current content
6. Prototype A
• Need to click on
tab to open
• Click on X or
Arrow to close
• Abbreviated list
is initially
presented
• User clicks View
All to see entire
list
7. Prototype B
• Dragging mouse
over menu will
open it
• Menu closes
when you move
mouse off it (no
clicking is
necessary)
• Higher-level
categories are
displayed
8. Prototype C
• Current, ‘full’
list of topics
• Menu drops
after delay, with
mouseover
• Moving off
menu makes it
disappear
quickly
9. Prototype D
• Control Condition
• Current menu on
BLS.gov
• Menu drops
immediately (no
delay) with
mouseover
• Moving off menu
makes it disappear
quickly
10. Method
• Define Participant Criteria Prototype
# of
Participants
Gender: Any Age: 18-55 A 11
Country: U.S. Income: Any B 14
Education: Any Employment: Any C 10
Computer Experience: Beginner-Intermediate or Expert D 10
• Participants complete up to 10 tasks while
providing verbal feedback
11. Tasks
1. Find a publication called the Occupational
Outlook Quarterly. This is an online magazine
about jobs and careers.
12. Tasks
1. Occupational Outlook Quarterly
2. Mass Layoffs
3. International Unemployment
4. Green Jobs
5. Survey of Occupational Illness and Injury
6. Strikes and Lockouts
7. Students and Teachers
8. NY State Wages
13. What the Participant Sees
Task
Instructions
Browser /
Webpage View
Recording
Time
14
16. Task Success and Time To Complete
# of Tasks Successfully Completed Time to Complete Task
8 120
7.4
7 6.6 6.7
Time to Complete Task (in sec.)
96.9
# of Tasks Completed (out of 8)
6.2 100
87.8
6 83.8
77.8
80
5
4 60
3
40
2
20
1
0 0
A B C D A B C D
18. Method
• Define Participant Criteria Prototype
# of
Participants
Gender: Any Age: 18-55 A 11
Country: U.S. Income: Any B 14
Education: Any Employment: Any C 10
Computer Experience: Beginner-Intermediate or Expert D 10
• Participants complete up to 10 tasks while
providing verbal feedback
• Following testing, participants write responses
to 4 open-ended questions
19. Open-Ended Questions
• How easy were these tasks to complete? Were they very
easy, easy, neither easy nor difficult, difficult, or very difficult? How easy
do you think these tasks would be for an average American citizen using
this website?
80
% of Participants Selecting Rating
70 67 69
61
60
50 47
39
40 33
30 23
20
20 17 17
10 8
0
0
Easy/Very Easy Neither Easy Nor Difficult Difficult/Very Difficult
A B C D
20. Feedback/Recommendations
• Final Recommendation:
Prototype C
• Several participants were
surprised that the Home tab
had a menu
• No items under Workplace
Injuries subcategory on
Subject Areas menu
22. Advantages
• Cost ($)
Method Cost
Traditional, In-Lab $40 per participant
TryMyUI.com $35 per participant
UserTesting.com $39 per participant
UTest.com --
OpenHallway.com $49-$199 per month
Loop11.com $350 per project
23. Advantages
• Cost ($)
• Cost (Time)
Task In-Lab Web
20 minutes total; 30 minutes total;
Requesting participants explaining criteria to recruiter specifying test groups and criteria
0 minutes;
Screening participants 10 minutes per participant
done by TryMyUI
0 minutes;
Scheduling participants 15 minutes per participant
study done at participant convenience
60 minutes total;
Preparing for interviews 10 minutes per participant
setting up web survey and tasks
Total (for 45 interviews) 26.6 hours 1.5 hours
3 weeks; 1 – 2 days;
Data collection period based on interviewer schedule Depends on participant criteria
24. Advantages
• Cost ($)
• Cost (Time)
• Short data collection period
• Participants that can be selected based on criteria
• Videos can be shared
• Participants are skilled/trained
– At thinking aloud
– At dealing with problems
– At evaluating websites
• Can get replacements for unusable cases
25. Disadvantages
• No follow-up
• Cannot correct navigation errors
• Limited to 20 minutes*
• Task completion time, success, and other
quantitative methods have to be captured
manually*
• No rating scales*
• Panel participants may have selection biases that
make them different from a ‘typical’ user
* May be different on other testing services
26. Discussion
• What testing service have you used?
• What are the advantages/disadvantages of
that service? Loop11.com
UTest.com
Userzoom.com
UserTesting.com
TryMyUI.com
OpenHallway.com OptimalWorkshop.com
27. Contact Information
Brandon Kopp
Research Psychologist
Office of Survey Methods Research
www.bls.gov/osmr
202-691-7514
kopp.brandon@bls.gov
Editor's Notes
Feedback A, Task 8 (Students & Teachers): Reacts to Home as a dropdownFeedback D, Task 6 then jump to Task 7 (Injuries & Illnesses): Guy spends 5:20 looking for Workplace Injuries. Doesn’t find it until next task.