Traning workshop on ‘Designing an conducting user studies”
Module 1 - Methods and Techniques (Kristien Ooms)
@ ICC&GIS
June 15th, 2016
Albena, Bulgaria
The document summarizes the history and activities of the ICA Commission on Use, Users, and Usability Issues. It discusses how the Commission was established in 2005 and has continued holding meetings at subsequent ICC conferences. It outlines the Commission's goals of sharing knowledge on map use, users, and usability. It also describes the Commission's achievements, which include maintaining a website and bibliography, organizing workshops and conference sessions, and publishing papers. Finally, it presents the Commission's terms of reference and plans for 2015-2019, which involve expanding educational resources and collaborating with other groups.
This document summarizes user research conducted for the Feeler contextual inquiry case. The research included literature reviews, expert interviews, end-user interviews, focus groups, and workshops. The end-user interviews explored factors that impact student well-being and meaningful aspects to monitor to improve learning experiences. Focus groups involved a design game to discuss these topics. Workshops applied participatory design methods like concept mapping, prototyping, and idea generation to further explore learning, well-being, and how to quantify experiences. Co-design workshops with students then tested reflection on quantified self-data through a design game and prototyping.
This document provides an overview of methods for identifying and modeling user needs and goals for interactive systems. It discusses contextual design as a starting approach and outlines observation techniques and contextual interviews for identifying user needs. It also discusses focus groups. For modeling user needs, it covers scenarios and personas. Scenarios describe typical tasks by outlining the goal and initial conditions, while personas are archetypes of users defined by their goals and attributes to represent different user types.
User-Centered Design and the LIS Curriculum: Reflections on the UX Program at...craigmmacdonald
This document discusses the user experience (UX) concentration within the Master of Science in Library and Information Science (MSLIS) program at Pratt Institute. It provides an overview of the key components of the UX concentration, including the foundation, required courses, electives, student interest group, consulting group, and recent developments. It also reflects on lessons learned in the first year, including the importance of administrative support, faculty support, leveraging existing courses, student demand, and engaging students through extracurricular activities to help the program grow.
The document summarizes the history and activities of the ICA Commission on Use, Users, and Usability Issues. It discusses how the Commission was established in 2005 and has continued holding meetings at subsequent ICC conferences. It outlines the Commission's goals of sharing knowledge on map use, users, and usability. It also describes the Commission's achievements, which include maintaining a website and bibliography, organizing workshops and conference sessions, and publishing papers. Finally, it presents the Commission's terms of reference and plans for 2015-2019, which involve expanding educational resources and collaborating with other groups.
This document summarizes user research conducted for the Feeler contextual inquiry case. The research included literature reviews, expert interviews, end-user interviews, focus groups, and workshops. The end-user interviews explored factors that impact student well-being and meaningful aspects to monitor to improve learning experiences. Focus groups involved a design game to discuss these topics. Workshops applied participatory design methods like concept mapping, prototyping, and idea generation to further explore learning, well-being, and how to quantify experiences. Co-design workshops with students then tested reflection on quantified self-data through a design game and prototyping.
This document provides an overview of methods for identifying and modeling user needs and goals for interactive systems. It discusses contextual design as a starting approach and outlines observation techniques and contextual interviews for identifying user needs. It also discusses focus groups. For modeling user needs, it covers scenarios and personas. Scenarios describe typical tasks by outlining the goal and initial conditions, while personas are archetypes of users defined by their goals and attributes to represent different user types.
User-Centered Design and the LIS Curriculum: Reflections on the UX Program at...craigmmacdonald
This document discusses the user experience (UX) concentration within the Master of Science in Library and Information Science (MSLIS) program at Pratt Institute. It provides an overview of the key components of the UX concentration, including the foundation, required courses, electives, student interest group, consulting group, and recent developments. It also reflects on lessons learned in the first year, including the importance of administrative support, faculty support, leveraging existing courses, student demand, and engaging students through extracurricular activities to help the program grow.
The document provides information about a workshop on Living Lab methodology hosted by Botnia Living Lab. It discusses the agenda, which includes an introduction to Botnia Living Lab and the FormIT methodology. Participants will work in groups to discuss key Living Lab principles and present their discussions. The document also provides background on Botnia Living Lab and its approach, including the FormIT methodology, which involves three cycles of user involvement: concept design, prototype design, and innovation design. It lists several of Botnia's projects and describes the key principles and benefits of the Living Lab approach.
User Interface Design: Definitions, Processes and PrinciplesMoodLabs
An introduction to User Interface Design, often called UX / UI. Presented by David Little, User Interface Designer, DDH from King's College London Digital Humanities program.
Website Usability & Eye-tracking by Marco Pretorious (Certified Usability Ana...DrupalCape
Things to consider when designing a website to make your site visitor's life easier!!
Note: There were some videos which were show to illustrate a point, however the presentation provides sufficient information and suggestion so you will not miss them.
User interface design: definitions, processes and principlesDavid Little
This document provides an overview of user interface design, including definitions, processes, and principles. It defines a user interface as the part of a computer system that users interact with to complete tasks. User-centered design is discussed as an approach that focuses on research into user behaviors and goals in order to design appropriate tools to enable users to achieve their objectives. Design principles like simplicity, structure, visibility, consistency, tolerance, and feedback are outlined.
This document provides an introduction to usability, including:
- Defining usability as the ease of use and quality of user experience when interacting with something.
- Explaining the history of usability originated from human factors research during WWII to ensure users could operate weapons and technology.
- Detailing common definitions of usability focus on effectiveness, efficiency and satisfaction of users achieving goals in a particular environment.
- Outlining key aspects of usability like being effective, efficient, engaging, error tolerant and easy to learn.
AccessU 2018 - Introduction to User ResearchKate Walser
AccessU 2018: “It’s too expensive to do testing.” “We don’t have time.” “We can’t find any users.” If that these sound like something your team might say, this session’s for you! Come learn how - with a little creativity and planning - you can find and learn from users in time for your next release.
Learn about different user research methods (e.g., interviews, personas, testing, and more) exist for your project
Learn how and when to apply them to your project
Learn how to find and engage users of all abilities
Learn how logistics change for these different situations
Inclusive Usability Techniques in Requirements Analysis of Accessible Web App...Grupo HULAT
Authors: Lourdes Moreno, Paloma Martínez, Belén Ruiz-Mezcua
IWWUA 2007: 1st International Workshop on Web Usability and Accessibility in conjuntion with the 8th International Conference on Web Information Systems Engineering, (January 2007, Nancy, France).
This document provides an overview of user-centered design (UCD). It discusses how UCD focuses on the end user, putting them at the center of the design process. The key aspects of UCD covered are its goal of optimizing the user experience, philosophy of adapting the system to the user, principles of early focus on users and empirical testing, and iterative design process. The UCD process involves analysis, design, evaluation, and iteration. UCD also incorporates disciplines like usability, user experience, accessibility, and information architecture to holistically consider the user.
The ICA Commission on Use, Users, and Usability Issues was established in 2005 to exchange knowledge on map use, users, and usability. The Commission maintains an online bibliography, organizes workshops and conference sessions, and supports young researchers. Its goals are to share information, facilitate discussion, and advance scientific research in this area through 2015-2019.
Cognitive Science Perspective on User eXperience!Hamed Abdi
How can we use research and knowledge about the brain, the visual system, memory and Emotion to design more effective products, services and systems?
In this presentation i explain how we can synthesize two different disciplinary to reach to more effectiveness in developing services, products and systems.
How user research shaped the thinking towards developing our institutions cen...Brendan Owers
The document summarizes how user research shaped the redesign of a university's central web portal. Through activities like card sorting, usability testing, and online surveys, researchers learned students found the original portal disorganized and difficult to navigate. User feedback informed the new portal's information architecture, navigation structure, and streamlined homepage. The research ensured the redesign centered the student experience. It demonstrated how user research is critical to developing digital services that users truly value.
The document discusses various user-centered approaches to interaction design. It describes the difficulty in bridging the gap between software developers and end-users. Traditional methods of communication have been one-directional. User-centered design aims to make development driven by users' needs and goals rather than just technology. The document then covers several approaches: ethnography involves long-term observation of users; participatory design actively involves users in development; contextual design uses targeted observations and interviews; and work modeling represents knowledge collected about users' work. Each approach has different benefits and drawbacks depending on factors like user involvement levels and project timelines.
1. The document discusses learning goals around user-centered design (UCD), including appreciating that user error stems from poor design, distinguishing UCD from design as art, and listing the major steps of the UCD process.
2. It provides an overview of the UCD process, which involves requirements analysis, conceptual design, production, launch, and iterative user research and evaluation.
3. Examples of affordances and how they help make products more intuitive by triggering mental models of how to use objects are discussed.
Investment in UCD Pays off
Bringing the users into every stage of the design process is an investment of effort and other resources of the design team, which makes understanding the benefits of a user-centered design approach relevant (IDF - www.interaction-design.org)
Purposeful UX Research given the Constraints from a PandemicIndu Kaila
Indu Kaila discusses conducting user experience (UX) research during the constraints of a pandemic. Some key challenges include being unable to travel for in-person participant sessions or team collaboration. However, remote research methods like moderated online user testing and collaboration tools allow UX work to continue. Ethics around recruiting frontline workers must also be considered. Overall, the constraints have sparked creativity in new remote research approaches and ways of working that may continue post-pandemic.
ICC2017 Washington http://icc2017.org/
6205.2
How hard is it to design maps for beginners, intermediates and experts?
Gáspár Albert
Eötvös Loránd Univerity
ICC2017 Washington - http://icc2017.org/
5504.1
Introducing MapStudy: An Open-Source Cartographic Research Tool
Carl Sack
University of Wisconsin-Madison
Robert Roth
Department of Geography, University of Wisconsin-Madison
Kristen Vincent
Department of Geography, University of Wisconsin-Madison
More Related Content
Similar to 2016 iccgis module1_methods_andtechniques
The document provides information about a workshop on Living Lab methodology hosted by Botnia Living Lab. It discusses the agenda, which includes an introduction to Botnia Living Lab and the FormIT methodology. Participants will work in groups to discuss key Living Lab principles and present their discussions. The document also provides background on Botnia Living Lab and its approach, including the FormIT methodology, which involves three cycles of user involvement: concept design, prototype design, and innovation design. It lists several of Botnia's projects and describes the key principles and benefits of the Living Lab approach.
User Interface Design: Definitions, Processes and PrinciplesMoodLabs
An introduction to User Interface Design, often called UX / UI. Presented by David Little, User Interface Designer, DDH from King's College London Digital Humanities program.
Website Usability & Eye-tracking by Marco Pretorious (Certified Usability Ana...DrupalCape
Things to consider when designing a website to make your site visitor's life easier!!
Note: There were some videos which were show to illustrate a point, however the presentation provides sufficient information and suggestion so you will not miss them.
User interface design: definitions, processes and principlesDavid Little
This document provides an overview of user interface design, including definitions, processes, and principles. It defines a user interface as the part of a computer system that users interact with to complete tasks. User-centered design is discussed as an approach that focuses on research into user behaviors and goals in order to design appropriate tools to enable users to achieve their objectives. Design principles like simplicity, structure, visibility, consistency, tolerance, and feedback are outlined.
This document provides an introduction to usability, including:
- Defining usability as the ease of use and quality of user experience when interacting with something.
- Explaining the history of usability originated from human factors research during WWII to ensure users could operate weapons and technology.
- Detailing common definitions of usability focus on effectiveness, efficiency and satisfaction of users achieving goals in a particular environment.
- Outlining key aspects of usability like being effective, efficient, engaging, error tolerant and easy to learn.
AccessU 2018 - Introduction to User ResearchKate Walser
AccessU 2018: “It’s too expensive to do testing.” “We don’t have time.” “We can’t find any users.” If that these sound like something your team might say, this session’s for you! Come learn how - with a little creativity and planning - you can find and learn from users in time for your next release.
Learn about different user research methods (e.g., interviews, personas, testing, and more) exist for your project
Learn how and when to apply them to your project
Learn how to find and engage users of all abilities
Learn how logistics change for these different situations
Inclusive Usability Techniques in Requirements Analysis of Accessible Web App...Grupo HULAT
Authors: Lourdes Moreno, Paloma Martínez, Belén Ruiz-Mezcua
IWWUA 2007: 1st International Workshop on Web Usability and Accessibility in conjuntion with the 8th International Conference on Web Information Systems Engineering, (January 2007, Nancy, France).
This document provides an overview of user-centered design (UCD). It discusses how UCD focuses on the end user, putting them at the center of the design process. The key aspects of UCD covered are its goal of optimizing the user experience, philosophy of adapting the system to the user, principles of early focus on users and empirical testing, and iterative design process. The UCD process involves analysis, design, evaluation, and iteration. UCD also incorporates disciplines like usability, user experience, accessibility, and information architecture to holistically consider the user.
The ICA Commission on Use, Users, and Usability Issues was established in 2005 to exchange knowledge on map use, users, and usability. The Commission maintains an online bibliography, organizes workshops and conference sessions, and supports young researchers. Its goals are to share information, facilitate discussion, and advance scientific research in this area through 2015-2019.
Cognitive Science Perspective on User eXperience!Hamed Abdi
How can we use research and knowledge about the brain, the visual system, memory and Emotion to design more effective products, services and systems?
In this presentation i explain how we can synthesize two different disciplinary to reach to more effectiveness in developing services, products and systems.
How user research shaped the thinking towards developing our institutions cen...Brendan Owers
The document summarizes how user research shaped the redesign of a university's central web portal. Through activities like card sorting, usability testing, and online surveys, researchers learned students found the original portal disorganized and difficult to navigate. User feedback informed the new portal's information architecture, navigation structure, and streamlined homepage. The research ensured the redesign centered the student experience. It demonstrated how user research is critical to developing digital services that users truly value.
The document discusses various user-centered approaches to interaction design. It describes the difficulty in bridging the gap between software developers and end-users. Traditional methods of communication have been one-directional. User-centered design aims to make development driven by users' needs and goals rather than just technology. The document then covers several approaches: ethnography involves long-term observation of users; participatory design actively involves users in development; contextual design uses targeted observations and interviews; and work modeling represents knowledge collected about users' work. Each approach has different benefits and drawbacks depending on factors like user involvement levels and project timelines.
1. The document discusses learning goals around user-centered design (UCD), including appreciating that user error stems from poor design, distinguishing UCD from design as art, and listing the major steps of the UCD process.
2. It provides an overview of the UCD process, which involves requirements analysis, conceptual design, production, launch, and iterative user research and evaluation.
3. Examples of affordances and how they help make products more intuitive by triggering mental models of how to use objects are discussed.
Investment in UCD Pays off
Bringing the users into every stage of the design process is an investment of effort and other resources of the design team, which makes understanding the benefits of a user-centered design approach relevant (IDF - www.interaction-design.org)
Purposeful UX Research given the Constraints from a PandemicIndu Kaila
Indu Kaila discusses conducting user experience (UX) research during the constraints of a pandemic. Some key challenges include being unable to travel for in-person participant sessions or team collaboration. However, remote research methods like moderated online user testing and collaboration tools allow UX work to continue. Ethics around recruiting frontline workers must also be considered. Overall, the constraints have sparked creativity in new remote research approaches and ways of working that may continue post-pandemic.
Similar to 2016 iccgis module1_methods_andtechniques (20)
ICC2017 Washington http://icc2017.org/
6205.2
How hard is it to design maps for beginners, intermediates and experts?
Gáspár Albert
Eötvös Loránd Univerity
ICC2017 Washington - http://icc2017.org/
5504.1
Introducing MapStudy: An Open-Source Cartographic Research Tool
Carl Sack
University of Wisconsin-Madison
Robert Roth
Department of Geography, University of Wisconsin-Madison
Kristen Vincent
Department of Geography, University of Wisconsin-Madison
ICC2017 Washington - http://icc2017.org/
5702-2
User Studies in Cartography: A Collaborative Research Agenda
Robert Roth; Department of Geography, University of Wisconsin-Madison
Arzu Çöltekin; UZH
Luciene Delazari; Federal University of Paraná
Homero Fonseca Filho; University of Sao Paulo
Amy Griffin; UNSW Canberra
Andreas Hall; Esri Sweden
Jari Korpi; Aalto University
Ismini-Eleni Lokka; University of Zurich
Andre Mendonça; Universidade do Estado do Amazonas
Kristien Ooms; Ghent University
Corné van Elzakker; University of Twente, Faculty ITC
1) The study tested how expert users interact with animated versus non-animated maps to understand the narrative of floods impacting a railway network.
2) The results found no significant difference in efficiency between map types, but animated maps influenced user confidence and hesitation. Information retrieval tasks were easier than analyzing causality.
3) Two questions had ambiguous wording, influencing the results. Additionally, question complexity likely had a greater impact than map type.
4) Future work could involve controlling task types by using standardized sets from other domains to improve reproducibility and comparison across studies.
ICC2017 Washington - http://icc2017.org/
5104.3
Researching the usability of a GeoVisual Analytics environment for the exploration and analysis of different data
Irma Kveladze
Aalborg University
Menno-Jan Kraak
University of Twente
Corné P.J.M. Van Elzakker
University of Twente
ICC2017 Washington - http://icc2017.org/
5604.2
GeoFARA: design, development and evaluation of a mobile human geography fieldwork application
Barend Köbben
ITC – University of Twente, Netherlands
Xiaoling Wang
PhD student
Faculty of Geo-Information Science and Earth Observation (ITC), University of Twente
Corné van Elzakker
University of Twente, Faculty ITC
Menno-Jan Kraak
Univerity of Twente
ICC2017 Washington - http://icc2017.org/
6205.1
Exploring the possibilities of eye tracking and EEG integration for cartographic context
Merve Keskin
Istanbul Technical University
Kristien Ooms
Ghent University
A. Ozgur Dogru
Istanbul Technical University
Philippe De Maeyer
Universiteit Gent
ICC2017 Washington - http://icc2017.org/
6105.4 Efficiency and effectiveness in case of disaster: a visual damage assessment tests
Kristien Ooms, PhD
Ghent University
Julia Åhlén
University of Gävle
Stefan Seipel
University of Gävle
Traning workshop on ‘Designing an conducting user studies”
Module 2 - Qualitative vs Quantitative User Research
(Robert Roth)
@ ICC&GIS
June 15th, 2016
Albena, Bulgaria
🔥🔥🔥🔥🔥🔥🔥🔥🔥
إضغ بين إيديكم من أقوى الملازم التي صممتها
ملزمة تشريح الجهاز الهيكلي (نظري 3)
💀💀💀💀💀💀💀💀💀💀
تتميز هذهِ الملزمة بعِدة مُميزات :
1- مُترجمة ترجمة تُناسب جميع المستويات
2- تحتوي على 78 رسم توضيحي لكل كلمة موجودة بالملزمة (لكل كلمة !!!!)
#فهم_ماكو_درخ
3- دقة الكتابة والصور عالية جداً جداً جداً
4- هُنالك بعض المعلومات تم توضيحها بشكل تفصيلي جداً (تُعتبر لدى الطالب أو الطالبة بإنها معلومات مُبهمة ومع ذلك تم توضيح هذهِ المعلومات المُبهمة بشكل تفصيلي جداً
5- الملزمة تشرح نفسها ب نفسها بس تكلك تعال اقراني
6- تحتوي الملزمة في اول سلايد على خارطة تتضمن جميع تفرُعات معلومات الجهاز الهيكلي المذكورة في هذهِ الملزمة
واخيراً هذهِ الملزمة حلالٌ عليكم وإتمنى منكم إن تدعولي بالخير والصحة والعافية فقط
كل التوفيق زملائي وزميلاتي ، زميلكم محمد الذهبي 💊💊
🔥🔥🔥🔥🔥🔥🔥🔥🔥
This presentation was provided by Racquel Jemison, Ph.D., Christina MacLaughlin, Ph.D., and Paulomi Majumder. Ph.D., all of the American Chemical Society, for the second session of NISO's 2024 Training Series "DEIA in the Scholarly Landscape." Session Two: 'Expanding Pathways to Publishing Careers,' was held June 13, 2024.
Leveraging Generative AI to Drive Nonprofit InnovationTechSoup
In this webinar, participants learned how to utilize Generative AI to streamline operations and elevate member engagement. Amazon Web Service experts provided a customer specific use cases and dived into low/no-code tools that are quick and easy to deploy through Amazon Web Service (AWS.)
CapTechTalks Webinar Slides June 2024 Donovan Wright.pptxCapitolTechU
Slides from a Capitol Technology University webinar held June 20, 2024. The webinar featured Dr. Donovan Wright, presenting on the Department of Defense Digital Transformation.
Andreas Schleicher presents PISA 2022 Volume III - Creative Thinking - 18 Jun...EduSkills OECD
Andreas Schleicher, Director of Education and Skills at the OECD presents at the launch of PISA 2022 Volume III - Creative Minds, Creative Schools on 18 June 2024.
Philippine Edukasyong Pantahanan at Pangkabuhayan (EPP) CurriculumMJDuyan
(𝐓𝐋𝐄 𝟏𝟎𝟎) (𝐋𝐞𝐬𝐬𝐨𝐧 𝟏)-𝐏𝐫𝐞𝐥𝐢𝐦𝐬
𝐃𝐢𝐬𝐜𝐮𝐬𝐬 𝐭𝐡𝐞 𝐄𝐏𝐏 𝐂𝐮𝐫𝐫𝐢𝐜𝐮𝐥𝐮𝐦 𝐢𝐧 𝐭𝐡𝐞 𝐏𝐡𝐢𝐥𝐢𝐩𝐩𝐢𝐧𝐞𝐬:
- Understand the goals and objectives of the Edukasyong Pantahanan at Pangkabuhayan (EPP) curriculum, recognizing its importance in fostering practical life skills and values among students. Students will also be able to identify the key components and subjects covered, such as agriculture, home economics, industrial arts, and information and communication technology.
𝐄𝐱𝐩𝐥𝐚𝐢𝐧 𝐭𝐡𝐞 𝐍𝐚𝐭𝐮𝐫𝐞 𝐚𝐧𝐝 𝐒𝐜𝐨𝐩𝐞 𝐨𝐟 𝐚𝐧 𝐄𝐧𝐭𝐫𝐞𝐩𝐫𝐞𝐧𝐞𝐮𝐫:
-Define entrepreneurship, distinguishing it from general business activities by emphasizing its focus on innovation, risk-taking, and value creation. Students will describe the characteristics and traits of successful entrepreneurs, including their roles and responsibilities, and discuss the broader economic and social impacts of entrepreneurial activities on both local and global scales.
Gender and Mental Health - Counselling and Family Therapy Applications and In...PsychoTech Services
A proprietary approach developed by bringing together the best of learning theories from Psychology, design principles from the world of visualization, and pedagogical methods from over a decade of training experience, that enables you to: Learn better, faster!
How to Manage Reception Report in Odoo 17Celine George
A business may deal with both sales and purchases occasionally. They buy things from vendors and then sell them to their customers. Such dealings can be confusing at times. Because multiple clients may inquire about the same product at the same time, after purchasing those products, customers must be assigned to them. Odoo has a tool called Reception Report that can be used to complete this assignment. By enabling this, a reception report comes automatically after confirming a receipt, from which we can assign products to orders.
2. Overview of the day
Module 1
Kristien Ooms
University of Ghent, Belgium
Chair, ICA Commission on UUUI
Methods and techniques of use, user and
usability research in geo-information
processing and dissemination
Module 2
Robert Roth
University of Wisconsin Madison, USA
Vice Chair, ICA Commission on UUUI
Quantitative versus Qualitative user
research: selecting the right approach
Module 3
Kristien Ooms
University of Ghent
ICA Commission on Use and User Issues
When and how to apply Eye Tracking?
Module 4
Corné van Elzakker
University of Twente, The Netherlands
Past Chair, ICA Commission on UUUI
An Introduction to Thinking Aloud
2
3. ICA Commission
on Use, User, and Usability Issues
3
http://icaci.org
Rob Roth
Artemis
Skarlatidou
Alena Vondráková
Kristien Ooms
http://use.icaci.org
4. ICA Commission
on Use, User, and Usability Issues
• Small commission ‘core’
– website, bibliography, facilitation of the organization of workshops /
seminars and all relationships with ICA
• Corresponding Commission members
– Mailing list of > 125 people
– Through website
– Mainly from science, many PhD candidates
• Goals:
– Inform > Share & exchange > Facilitate
• Through:
– Workshops & Congresses
– Website & Publications
4
5. ICA Commission
on Use, User, and Usability Issues
• 20 August 2015 in Curitiba, Brazil- Joint Commission
workshop
– Designing and Conducting User Studies
– ICA Commission on Cognitive Visualization - Use and User Issues
• 21 August 2015 in Curitiba, Brazil- Joint Commission
workshop
– Envisioning the Future of Cartographic Research
– ICA Commission on Cognitive Visualization – Geovisualization -
Use and User Issues - Map Design
• 23 – 29 August 2015, Rio de Janeiro, Brazil
– Several sessions on Use and User Issues
5
6. ICA Commission
on Use, User, and Usability Issues
• 6th International Symposium on the History of
Cartography
– October 13-14, 2016
– Dubrovnik, Croatia
• International Cartographic Conference 2017!!!
– July 2-7, Washington, USA
– Pre-conference workshop; Open meeting
– Sessions
• Deadline submissions: 26 October 2016
• Conference theme T07
User studies; user experience and usability; user interface design.
6
http://icc2017.org/
7. ICA Commission
on Use, User, and Usability Issues
Website
– http://use.icaci.org
– (annotated) online bibliography
7
Planned:
– NEW WEBSITE
– online overview & FAQ
usability methods and techniques
aspects of experiment design
tips for analysis
why/when one would use the techniques
8. ICA Commission on
Use, Users, and Usability Issues
8
Terms of Reference 2015-2019
exchange of knowledge, news, and information on use, users, and
usability in cartography, geo-information processing, and related fields.
bibliography on use, user, and usability research in cartography and
related
fields. TOPICS:
use case studies with print, web-based, and mobile
maps
methods and techniques
for evaluating products
use of emerging mapping
technologies
review paper from bibliographical database
includes a final research agenda on map use, users, and usability for
Cartography.
online overview & FAQ
usability methods and techniques
aspects of experiment design
tips for analysis
why/when one would use the techniques
(replaces planned book publication)
9. ICA Commission on
Use, Users, and Usability Issues
Terms of Reference 2015-2019
special sessions on map use, users, and usability
at future ICA conferences
other regional conferences on cartography and related topics.
gather and communicate original scientific contributions
training workshops for non-specialists on use, user, and usability
issues in Cartography and related fields
participate in and contribute to workshops and seminars organized in
other disciplines (e.g. Computer Science, Geography, GIScience)
by other ICA Commissions
Involve & support
young (PhD) researchers
researchers from different cultural environments
resulting training materials commission’s website!
special journal issue with sibling ICA Commissions
promote scientific research
10. Workshop on
Designing & Conducting User Studies
Module 1:
Methods and techniques of use, user and usability research in
geo-information processing and dissemination.
11. Overview
• Content of the module:
“The first module will introduce you to use, user & usability research in
geo-information processing and dissemination. Reasons for doing such
research are to understand and explain how GI tools work for their users
and to design more usable tools. In this first module, the focus will be on
the latter. The concept of User Centered Design will be presented and an
overview will be given of the most commonly applied research techniques,
along with tips & tricks on how and when to use them. Experiences will be
shared in interactive sessions.”
11
12. Know your user
• Who are you?
– What is your background?
– Have you ever done user research?
– What are your expectations of this workshop?
– What do you hope to learn from this workshop?
– Do you have specific questions?
12
13. This module...
• use,
• user &
• usability research
• in geo-information processing
• and dissemination
• ???
13
14. 14
Map use research
More holistic, functional map use research
e.g. use of cycling maps, investigated by Dutch project group
15. 15
Types of map use research
Perceptual and cognitive research
Flannery, 1971
16. 16
Nowadays …
• Technological developments:
– integration
in the geodata collection
in geoinformation production processes
• Map users became also map producers
(and collectors of map data > crowd sourcing / VGI)
• Not only maps
also other ways of geospatial data dissemination
therefore….
17. 17
Not only MAP use issues …
But also use and user issues related to, e.g.:
– Data and databases
– Other output formats
– Software and information systems
– Hardware
– Interfaces
From map use research to use, user & usability research
in geo-information processing and
dissemination…..
18. 18
Aspects of usability (ISO 9241-11)
Guidance on Usability:
“the extent to which a product can be used by specified users
to achieve specified goals with effectiveness, efficiency and
satisfaction in a specified context of use”
Effectiveness:
the accuracy and completeness with which specified users can
achieve specified goals in particular environments
Efficiency:
the resources expended in relation to the accuracy and
completeness of goals achieved
Satisfaction:
the comfort and acceptability of the work system to its users
and other people affected by its use
19. 19
Not only usability …
In our workshop we are dealing with:
– uses (tasks, questions, purposes, applications, etc.)
– users (human beings)
– usability
…and particularly with:
methods and techniques of doing use, user
and usability research
20. 20
Not only usability …
Morville, 2004
http://semanticstudios.com/publications/semantics/000029.php
21. 21
Not only usability …
• Useful > able to be used for a practical purpose > are our
geoinformation products and systems useful? (~ effective)
• Usable > relative ease of use
• Desirable > emotional desirability (tempers efficiency)
• Findable > users should be able to find what they need
• Accessible > geoinformation, tools and systems should also
be accessible to people with disabilities (more than 10% of
the population)
• Credible > do users trust and believe the geoinformation
products and tools they see or have to work with?
• Valuable > value should be delivered to the sponsors, the
mission must be advanced (~ satisfaction & effectiveness)
Morville, 2004
http://semanticstudios.com/publications/semantics/000029.php
22. The 3 U’s
• usability-utility tradeoff
distinction between
• interfaces for experts
o great utility
o difficult to learn and use
• general-use interfaces
o transparently usable
o require little or no learning to use
o support only a small set of user tasks
23. 23
User Centered Design
• What is UCD?
– Origin: software engineering
• Product development: waterfall model
• Involve user in all stages of design process
24. 24
The UCD life cycle
Analyse
Requirements
Produce Design
Solutions
Conceptual
Design
& Prototype
Final Design
& Prototype
Interim
Designs &
Prototypes
Identify the
need for
user-centred
design
Design
satisfies
requirements
Evaluate Designs
Testing with
representative
users
input
Business
User
StakeholderOrganisational
Characteristics Preferences
Use contexts Tasks
25. 25
The UCD life cycle
Analyse
Requirements
Produce Design
Solutions
Conceptual
Design
Interim
Designs &
Identify the
need for
user-centred
design
Evaluate Designs
Testing with
input
Business
User
StakeholderOrganisational
Characteristics Preferences
Use contexts Tasks
Requirement analysis!!!
26. 26
The UCD life cycle
Produce Design
Solutions
Conceptual
Design
& Prototype
Final Design
& Prototype
Interim
Designs &
Prototypes
Design
satisfies
requirements
Evaluate Designs
Testing with
representative
users
Usability evaluation
Iterative process
26
27. 27
The UCD life cycle
27
10 “elements” of the usability engineering lifecycle
1. Know the User: Complete a needs assessment (also called a
task analysis or work domain analysis) with target users to
establish user profiles and use case scenarios;
2. Competitive Analysis: Critically compare existing interfaces
supporting similar use cases to determine how the proposed
interface can fill unmet needs;
3. Setting Goals: Use insight from the needs assessment and
competitive analysis to formalize a requirements document
of proposed functionality to guide design and development;
4. Participatory Design: Recruit a representative set of target
users to participate in the conceptual design of the interface;
28. 28
The UCD life cycle
28
10 “elements” of the usability engineering lifecycle
5. Coordinated Design: Coordinate design across the project team to
develop a consistent product identity (i.e., look and feel);
6. Guidelines and Heuristic Analysis: Recruit experts during design
and development to evaluate the interface according to guidelines
(generalized insights generated from the scientific investigation of
digital interfaces) and heuristics (well-accepted, overarching design
principles drawn from experience);
7. Prototyping: Create static or interactive mockups of the interface;
an early, partially-functional prototype is referred to as an alpha
release while a fully-functional, but unstable prototype is referred
to as a beta release;
29. 29
The UCD life cycle
29
10 “elements” of the usability engineering lifecycle
8. Empirical Testing: Recruit a representative set of target
users to evaluate the utility and usability of numerous
prototypes during their evolution; formative evaluation
describes the feedback solicited in the early to intermediate
stages of the project on the alpha and beta releases, while
summative evaluation is conducted on the full release of the
interface to determine if the usability and utility goals have
been achieved;
9. Iterative Design: Revise the interface based on feedback
from guidelines/heuristic analysis and empirical testing;
10. Collect Feedback from Field Use: Acquire feedback about the
interface after it is transitioned into the field to inform future
product releases.
31. 31
Many user research methods & techniques
Interviews Product analysis Literature / document
studies Observation Brainstorming Surveys /
questionnaires Thinking aloud Eye-tracking Heuristic
evaluation Personas Scenario development Screen logging
Snapshot studies Focus groups Stakeholder meeting
Participatory design Retrospection Card sorting Task
analysis and modelling Cognitive walkthrough Diary /
notekeeping (empathy probes) Introspection Usability
inspections Performance testing User satisfaction
measurement Expert evaluation
32. 32
Many user research methods & techniques
How many users?
Stage of development?
Requirement phase?
Prototype?
Objectives?
Research Questions?
Time frame?
Manpower?
Money?
End users: expertise?
End users: gender?
End users: age?
Statistical analyses?
Visual analyses?
33. 33
Important aspects of usability research
• Distinction between methods & techniques of data
collection and data analysis
• Keep in mind that not all methods can be applied in all
stages of the UCD
• Distinction between qualitative and quantitative research
• Working with experts or with (representatives of) “real”
end-users
• Is it a within- or between user study?
• Doing research in the proper context
(field-based vs. laboratory based research)
34. 34
Important aspects of usability research
Techniques e.g.:
Qualitative
Focus groups
Interviews
Observation
Thinking aloud
Questionnaires
Screen logging
Eye tracking
Performance analysis
Quantitative
Analyse
Requirements
Produce Design
Solutions
Conceptual
Design
& Prototype
Final Design
& Prototype
Interim
Designs &
Prototypes
Identify the
need for
user-centred
design
Design
satisfies
requirements
Evaluate Designs
Testing with
representative
users
input
Business
User
StakeholderOrganisational
Characteristics Preferences
Use contexts Tasks
Method = User Centered Design
(of geospatial data processing and dissemination systems)
35. 35
Important aspects of usability research
Requirements Analysis
Evaluate early design
Evaluate elaborate design
User feedback
Focus groups
Brainstorm sessions
Interviews
Surveys & questionnaires
Stakeholder meetings
Literature review
Card sorting
...
Paper prototype
Storyboarding
Heuristic evaluation
Automated evaluation
Wizard of Oz
Thinking aloud
Eye tracking
Card sorting
Cognitive walkthrough
....
Performance testing
User satisfaction
Heuristic evaluation
Cognitive walkthrough
Functional prototyping
Critical incidents
Logging user interactions
....
Questionnaires
User satisfaction
Logging user interactions
Observations
Diary keeping
...
36. 36
Important aspects of usability research
• Involving (representatives of) “real” users:
– Surveys & questionnaires
– Interviews
– Focus groups / stakeholder meetings
– Observation: logging user activities (including tracking, video, screen
logging)
– Eye-movement tracking
– Thinking aloud
– Participatory design
• Involving experts (not the users themselves):
– Heuristic evaluation / usability inspection
– Cognitive walkthrough
• Theory based
– Automated evaluation
– Scenario-based design
– Secondary sources
37. 37
Important aspects of usability research
Method Similar or Related Methods
Expert-based
heuristic evaluation rulesof thumb
conformity inspection feature inspection, consistency inspection, standards inspection,
guideline checklist
cognitive walkthroughs pluralistic walkthroughs, prototyping, storyboarding, Wizard of Oz
Theory-based
scenario-based design personas, scenarios of use, use case, context of use, theatre
secondary sources contentanalysis, competitive analysis
automated evaluation automated interaction logs, unmoderated user-based
methods
User-based
participant observation ethnographies, field observation, MILCs, journal/diary sessions,
screenshot captures
surveys questionnaires, entry/exit surveys, blind voting, cognitive workload
assessment
interviews structured interviews, semi-structured interviews, unstructured
interviews, contextual inquiry
focus groups supportive evaluation
Delphi
card sorting Q methodology, conceptmapping, aflnity diagramming, brainstorming
talk/think aloud verbal protocol analysis, co-discovery study
interaction study performance measurement,controlled experiments
38. 38
Important aspects of usability research
Good when there are multiple components of an interface requiring the same
look/feel, when diVerent teams of designers are working on diVerent compo-
nents of the application, when there are established design standards and con-
ventions, when a work domain analysis has been completed.
Poor when the evaluation goal is to brainstorm potential usability issues rather
than ensure the interface meets certain requirements, when the project consists
of only several people working together closely, when there is little or no prec-
edent on how a novel interface should look and behave.
Tip for Cartography: While there are few cartographic interface design stan-
dards, the Open Geospatial Consortium (OGC) has developed several guidelines
and standards for the use and dissemination of geographic information.
Conformity Assessment
Conformity assessment requires experts to determine if the inter-
face passes a set of requirements, such as a pre-determined feature
list, design standards and conventions, or in-house specifications.
Good when input/feedback is needed quickly, when only a small set of
experts are available, when used for several rounds of expert evaluation,
when designers/developers are interested in uncovering a broad range
of interface issues.
Poor when expert consultants are unavailable or expensive, when the
experts are part of the project team, when targeting a specific kind of
interface problem, when one or more of the heuristics is not relevant to
the goals of interface evaluation, when there is excessive subjectivity in
interpreting the heuristics.
Tip for Cartography: It is important that the experts performing the heu-
ristic evaluation have experience in both Interactive Cartography and
Usability Engineering; such restrictions may prohibitively narrow the pool
of available experts.
Heuristic Evaluation
Heuristic evaluation requires experts to summarize
potential usability and utility issues with an inter-
face according to a pre-determined set of heuris-
tics, or well-accepted cartographic interface design
principles.
Cognitive
Walkthroughs
A cognitive walkthrough is the most 'hands-on' of
the expert-based methods, as the expert must
assume the role of the user and complete a set of
tasks in the anticipated way that the user would
perform them.
Good when the characteristics/behaviors of
the targeted end users are well understood,
when the expert has a great deal of experi-
ence working with users, when there is not
enough time to study users firsthand, when
the tasks included in the walkthrough repre-
sent real-world work objectives well, when
multiple steps must be completed in order to
use the interface, when only paper mockups
are available.
Poor when experts are not familiar with
the user group, when the tasks are
ill-defined, open-ended, or have multiple
solutions, when the research design is
not informed by a study with users.
(1) Expert-based Methods
Expert-based methods solicit input and
feedback about a cartographic interface
from consultants with training and experi-
ence in cartographic interface design and
evaluation.
Tip for Cartography: It may be best to
include experts and users together (i.e., a
pluralistic walkthrough), as interactive
map use often is exploratory and open-
ended, making it diflcult for experts to
predict suboptimal interaction strategies
on their own.
39. Important aspects of usability research
Good when the goal is to improve and stablize source code, when
long-term interface support is needed after deployment, when the
fully-featured interface serves a large user community, when
resources are limited to complete multiple rounds of user-based
studies.
Poor when the interface is unique or novel, when the the usability
measures for a specific type of application are poorly established,
when the interface is simple and includes only several features.
Automated Evaluation
With automated evaluation, usability and utility mea-
sures are generated programmatically by applying
specialized computer algorithms
Secondary Sources
A secondary source is any piece of information not col-
lected by the party using the information, and includes
text documents, maps/images, and softwareapplications.
Good when the designers/developers know little about the application
domain, when a user-based work domain analysis cannot be com-
pleted, at the formative stage of design and development, when there
are a large number of competing applications that implement similar
functionality, when there were multiple previous versions of the inter-
face.
Poor when the interface is designed to support a wide variety of appli-
cation domains, when the interface is the first of its kind and has few
extant parallels for comparison, when a robust work domain analysis
already was completed, when at the final stages of development.
Tip for Cartography: Secondary sources are useful for understanding
what types of questions or tasks should be included in future user-
based studies (particularly for an initial work domain analysis).
Scenario-based
Design
Scenario-based design makes the expected
uses of an interface explicit through the de-
scription of a scenario, or story consisting of
a problem setting, the interfaces available
in the setting, and personas of the expected
users of the interfaces.
Good when a work domain analysis
cannot be completed due to limited
resources or poor user accessibility,
when actual use scenarios are well
known or validated through user
studies, when the interface needs to
support a diverse set of users or objec-
tives, when expert- or user-based
studies relying on tasks are conducted
at later stages in design/development,
when the project team is large
Poor when little is known about the
users, when the scenarios are not vali-
dated with user-based studies, when
the scenarios are overly simplistic or
include only a subset of the complete
set of potential users or objectives,
when an interface is in the final stages
of development.
(2) Theory-based Methods
Theory-based methods require the interface
designers and developers to evaluate the
cartographic interface themselves by apply-
ing theoretical frameworks and guidelines
established through scientific research.
Tip for Cartography: Background
knowledge in Geography is important
for good scenario-based design, as the
designers/developers need to under-
stand the geographic phenomena rep-
resented in the interactive map when
drafting the hypothetical scenario.
Tip for Cartography: When the cartographic interface is
designed for use by a diverse set of users, automated
evaluation can be used to initiate changes in the interface
according to past use; simple ex- amples of such adaptive
interfaces include ranking of items by 'most viewed' or
adjusting default tool parameters according to the most
frequent selections.
40. Important aspects of usability research
Focus Groups
The focus group is the
multi-person equivalent to
the interview, where a
group of targeted end
users (3-10) discuss topics
posed by a session
moderator.
Good when the user needs and expectations
are poorly known, when investigators have
access to a intermediate number of users and
stakeholders (more than required for interviews,
but much less than required for surveys), when
the focus of summative evaluation is user
satisfaction, when the investigators do not have
time to complete interviews.
Poor when the participants are not rep-
resentative of the target users, when
the user group is diverse, when investi-
gators have limited time to perform the
evaluation and analyze the results.
Tip for Cartography: The interviews
may benefit from having users provide
map examples from discussion during
the interviews or from having partici-
pants demonstrate their software to
clarify the interview discussion.
Interviews
An interview is a purposeful conversa- tion
between the evaluator and user in which the
user answers a series of structured and
unstructured questions.
Good when the user needs and expectations are poorly
known, when the software supports a small number of
highly-specialized users or stakeholders, when transition- ing an
interface to a new application domain.
Surveys
The survey method requires partici-
pants to respond to a series of prede-
termined, typically structured ques-
tions with no investigator interaction.
Good when input is required from a large number of
diverse users, when characteristics of the targeted
audience are not fully known, when the investigators
cannot be present physically to adminster the evalua-
tion, when the participants have very little time to pro-
vide feedback.
Poor when important design decisions are based
soley upon the results, when the investigators are
unfamiliar with the user objectives or expectations
and therefore do not know what questions to ask,
when access to end users is limited, when users are
asked to recall experiences or usage strategies
from a significant amount of time prior to taking
the survey.
Tip for Cartography: Surveys for cartographic
interface evaluation should include questions spe-
cific to both cartographic representation (i.e., the
design of the maps themselves) and cartographic
interaction.
daily work in order to generated an ethnog-
raphy, or comprehensive narrative of the
way in which the interface is used in practice.
Good when evaluators have excellent access to users, when
evaluators want to build a strong connection with a particular
set of users, when information is needed about how users cur-
rently work, when the project is large with design/development
spanning multiple years, when the interface or a previous ver-
sion of the interface already is in used.
Poor when access to users is limited, when users are diverse in
their characteristics or application domain, when users are dis-
persed geographically, when feedback is needed quickly, when
the interface is simple or supports few tasks.
Tip for Cartography: Participant observation may not be an efl-
cient use of resources, as cartographic interfaces are often a
single step in a larger workflow that includes many, non-
cartographic applications.
Participant Observation
In participant observation, evaluators watch
users interact with the interface during their
(3) User-based Methods
User-based methods solicit input and
feedback about a cartographic inter-
face from a representative sample of
targeted end users; administration of
user-based methods is fundamental to a
user-centered design approach and the
only way to ensure success of the
cartographic interface.
Poor when access to users is limited, when users are
diverse in their characteristics or application domain,
when users are dispersed geographically, when
feedback is needed quickly, when the interface is simple
or supports few tasks.
Tip for Cartography: Focus groups may benefit from
includ- ing a mixture of map mapmakers and map users
in each ses- sion.
41. Important aspects of usability research
In this method, users are guided through a series of
bench- mark tasks by a session moderator and
instructed to de- scribe verbally either what they are
doing (talk aloud) or what they are thinking (think
aloud).
Good when evaluators are interested in identifying a
broad range of usability issues, when feedback is
required quickly on only the most important
problems, when project resources are limited, when
there are multiple ways to com- plete the same
objective.
Poor when the tasks the interface should sup-
port are poorly known, when the participants
are not representative of the target audience,
when each task requires a large amount of
time to complete, when evaluators are more
interested in utility than usability, when par-
ticipants are already familiar with the interface.
Tip for Cartography: Because most people
interact with maps in an unstructured way, it is
important that the task protocol includes both
close-ended, simple tasks with a 'right answer'
as well as open-ended, exploratory tasks that
may lead to divergent solutions.
Card
Sorting
Card sorting is an activity in
which participants group a
given set of items, or cards,
according to their similar-
ity.
Good when there are a large set of
functions included in the interface
(between 30-200) or these func-
tions include a large set of param-
eters, when the optimal categori-
zation or structure is not currently
known, when an existing categori-
zation resulted in usability issues
and needs to be revised, when the
interface requires a large amount
of navigation among multiple
pages or menus.
Poor when the set of items is small (less
than 30) or extremely large (200+),
when the goal is to refine a single fea-
ture in the interface, when users can
customize the layout/organization of
the interface.
Tip for Cartography: The card sorting
technique is particularly promising for
hierarchically grouping feature types
and associated map symbols in an ex-
pandable, interactive legend.
Delphi
The Delphi method is a distrib-
uted approach to focus groups
that collects multiple rounds of
anonymous feedback, with
moderator summaries distrib
uted between rounds.
Good when the interfaces and the problems
they support are extremely complex, when the
users are geographically dispersed or the feed-
back needs to be collected online, when the
superiors of one or more users also are
included in the group communication.
Poor when the feedback is required quickly,
when there is not a team member available to
moderate the sessions, when users have little
personal investment in the tool (as there may
be a high drop out rate over time).
Tip for Cartography: It may be useful to
anchor the discussion threads to an interactive
map using asynchronous/diVerent-place geo-
collaboration and geodeliberation techniques.
Good when the project spans multiple years and includes itera-
tive rounds of interaction studies, when the kind of interface
evaluated has an established optimal score in the applied per-
formance measures, when evaluators are interested both in ex-
panding the understanding of interactive cartography broadly
as well as improving the cartographic interface specifically.
Poor when user objectives are not known, when time and
resources are lacking to collect and analyze the copious interac-
tion logs, when the performance measures poorly support the
evaluation goals, when evaluators are interested in capturing
subjective satisfaction.
Tip for Cartography: In addition to the interaction log, informa-
tion about the user’s eye movements also can be captured
during an interaction study.
Interaction Study
Interaction studies require users to complete a set
of benchmark tasks with the interface in a con-
trolled setting while their interactions are cap-
tured in an interaction log.
Talk-Aloud &
Think Aloud
(3) User-based Methods
User-based methods solicit input and
feedback about a cartographic inter-
face from a representative sample of
targeted end users; administration of
user-based methods is fundamental to
a user-centered design approach and
the only way to ensure success of the
cartographic interface.
42. Learning from each other...
• Input from audience
– Who did user research?
– Describe background
– Decisions you had to make (select technique, find participants,...)
– Issues you encountered
– ....
42
43. Important aspects of usability research
How many users?!
• Depending on technique
– Data = Qualitative or Quantitative?
n5 participants... n , SD, z, ẋ,...
• How many user groups?
– Evaluate user characteristics (between user)
• Multiple groups, different characteristics
– Evaluate different products (within user)
• Multiple groups, same characteristics
43
Qualitative analyses on...
Focus Groups
Thinking aloud
Observations
Statistical analyses on....
Performance measurements
Questionnaires Eye tracking
44. 44
Important aspects of usability research
• Use contexts: e.g. map use environment
– hard copy maps:
• as wall map, on the desk top, in the field, ….
– screen maps:
• on the desk top, on handheld mobile devices
• stand-alone or network application
(e.g. Internet)
• hard- and software available, interfaces
• use at one place or distributed collaboration …
44
45. 45
Important aspects of usability research
• New user environments
– World Wide Web
Web maps, WMS, distributed GIS, GDI’s
– Mobile & wireless devices
Smartphones, PDA’s, Pocket PC’s … with location aware
applications (LBS)
– small screens
– low resolution
– few colours
– small data files
45
46. Thank you for your attention!
Contact:
kristien.ooms@ugent.be
http://use.icaci.org