The document discusses a study that examined differences in motivation and computer proficiency between daily computer users. The study hypothesized that extrinsically motivated proficient users would have more difficulty with unfamiliar computer tasks compared to intrinsically motivated users. The study involved administering a motivation inventory to over 130 participants from various countries and ages. Based on inventory scores, 16 participants were observed performing unfamiliar computer tasks. The observations found that extrinsically motivated users stumbled, fell, persisted, quit, and resisted unfamiliar tasks significantly more than intrinsically motivated users. The study provides insights into how motivation styles impact adaptation to unfamiliar technologies.
Harriet just enoughcomputerusersfamilyhckingmtuedu
This study examined differences in motivation style and proficiency with unfamiliar computer tasks between intrinsically and extrinsically motivated proficient daily computer users. The researchers hypothesized that extrinsically motivated users would have more difficulty with unfamiliar tasks compared to intrinsically motivated users. They designed a study where participants completed a motivation questionnaire and were then observed performing unfamiliar tasks on a computer system. The observations found that extrinsically motivated users exhibited significantly more stumbles, falls, persistence and quitting behaviors compared to intrinsically motivated users. The study provides evidence that motivation style, rather than age, digital experience or perceived competence, impacts one's ability to adapt to unfamiliar computer tasks.
2013 MBAA/NAMS presentation, "Critical Thinking Advances the Theory and Practice of Business Management." Phyllis R. Anderson, Governors State University and Joanne R. Reid, Corporate Development Associates, Inc.
This video is incomplete. We need to share ideas for instructions on completing the following tasks: writing the formal outline, drafting the paper, creating internal documentations and avoiding plagiarism.
Technical writing is the presentation of information that helps the reader solve a particular problem.
Technical communicators write, design, and/or edit proposals, manuals, web pages, lab reports, newsletters, and many other kinds of professional documents.
This document summarizes a presentation on improving STEM education. It discusses 9 keys to improving STEM education: investigate community needs, focus programs, collaborate with partners, engage students through hands-on learning, enrich programs with activities like camps and competitions, design curricula around engineering challenges, inspire teachers with professional development, integrate STEM across subjects, and evaluate programs. It provides examples for each key and discusses barriers to STEM participation like attitudes, knowledge, and equity issues that programs should address.
Design Thinking for Requirements EngineeringDaniel Mendez
This document provides an overview of a tutorial on Design Thinking for Requirements Engineering (DT4RE). The tutorial aims to introduce basic principles and methods of Design Thinking and discuss how it can be integrated with Requirements Engineering. It outlines different approaches to integrating Design Thinking and Requirements Engineering based on the complexity of the project from pure Design Thinking to fully integrated approaches. The tutorial also discusses open research challenges around principles, artifacts, project influences, methods adoption, and operationalization of Design Thinking and Requirements Engineering processes.
This document discusses the design principles of advanced task elicitation systems. It begins with an introduction that outlines the motivation and challenges of manual task elicitation in software development. It then reviews related work on task elicitation systems and the need to evaluate their design principles empirically. The methodology section describes a design science research approach used to conceptualize and evaluate an artifact called REMINER. Evaluation results show that semi-automatic task elicitation and leveraging imported knowledge bases can significantly increase elicitation productivity compared to manual elicitation. The discussion covers limitations and opportunities for future research at the intersection of task elicitation and software development processes.
Harriet just enoughcomputerusersfamilyhckingmtuedu
This study examined differences in motivation style and proficiency with unfamiliar computer tasks between intrinsically and extrinsically motivated proficient daily computer users. The researchers hypothesized that extrinsically motivated users would have more difficulty with unfamiliar tasks compared to intrinsically motivated users. They designed a study where participants completed a motivation questionnaire and were then observed performing unfamiliar tasks on a computer system. The observations found that extrinsically motivated users exhibited significantly more stumbles, falls, persistence and quitting behaviors compared to intrinsically motivated users. The study provides evidence that motivation style, rather than age, digital experience or perceived competence, impacts one's ability to adapt to unfamiliar computer tasks.
2013 MBAA/NAMS presentation, "Critical Thinking Advances the Theory and Practice of Business Management." Phyllis R. Anderson, Governors State University and Joanne R. Reid, Corporate Development Associates, Inc.
This video is incomplete. We need to share ideas for instructions on completing the following tasks: writing the formal outline, drafting the paper, creating internal documentations and avoiding plagiarism.
Technical writing is the presentation of information that helps the reader solve a particular problem.
Technical communicators write, design, and/or edit proposals, manuals, web pages, lab reports, newsletters, and many other kinds of professional documents.
This document summarizes a presentation on improving STEM education. It discusses 9 keys to improving STEM education: investigate community needs, focus programs, collaborate with partners, engage students through hands-on learning, enrich programs with activities like camps and competitions, design curricula around engineering challenges, inspire teachers with professional development, integrate STEM across subjects, and evaluate programs. It provides examples for each key and discusses barriers to STEM participation like attitudes, knowledge, and equity issues that programs should address.
Design Thinking for Requirements EngineeringDaniel Mendez
This document provides an overview of a tutorial on Design Thinking for Requirements Engineering (DT4RE). The tutorial aims to introduce basic principles and methods of Design Thinking and discuss how it can be integrated with Requirements Engineering. It outlines different approaches to integrating Design Thinking and Requirements Engineering based on the complexity of the project from pure Design Thinking to fully integrated approaches. The tutorial also discusses open research challenges around principles, artifacts, project influences, methods adoption, and operationalization of Design Thinking and Requirements Engineering processes.
This document discusses the design principles of advanced task elicitation systems. It begins with an introduction that outlines the motivation and challenges of manual task elicitation in software development. It then reviews related work on task elicitation systems and the need to evaluate their design principles empirically. The methodology section describes a design science research approach used to conceptualize and evaluate an artifact called REMINER. Evaluation results show that semi-automatic task elicitation and leveraging imported knowledge bases can significantly increase elicitation productivity compared to manual elicitation. The discussion covers limitations and opportunities for future research at the intersection of task elicitation and software development processes.
This document provides a comparison of different online platforms for public managers to use for idea generation and public engagement. It describes the purpose and key features of 11 platforms: Bubble Ideas, Crowd Wise, Delib Dialogue App, Google Moderator, IdeaScale, Microsoft Town Hall, PubliVate, Salesforce Ideas, Spigit, UserVoice, and compares what each platform is well-suited for and its pricing information. The document aims to help public managers choose a platform that best fits their specific needs and objectives.
Games have the potential to transform learning by making it student-centered, complex, and intrinsically motivating. When designed well, games can engage students in solving real-world problems through interactive problem-solving and collaboration. Game-based learning approaches like project-based learning embed critical thinking, communication, and deeper learning within an authentic and engaging context. Educators are exploring how to apply game mechanics and principles of game design to better capture students' interests and promote active, self-directed, and collaborative styles of learning.
Information Systems design science researchRaimo Halinen
The document discusses various aspects of developing and evaluating information systems using a design science research approach. It provides guidelines for conducting design science research, including that the research should design an artifact to solve an identified problem, evaluate the artifact and process, and ensure rigorous and communicable results. The document also discusses various process models and approaches for design science research.
Abbott a more transparent interpretation of health club surveysDean Abbott
The document summarizes Dean Abbott's presentation on interpreting health club member surveys. It describes using factor analysis to reduce 57 survey questions into 10 factors, then regression analysis to identify the 5 most predictive factors for member satisfaction. Rather than using the full factors, the final model used the top-loading question from each factor for improved performance and interpretability. This provided actionable insights for understanding member satisfaction.
The document summarizes findings from a pilot workshop and doctoral course assignment that used activity systems analysis to help students understand design as a practice and share their experiences through narrative case studies. Key findings from student reflections included: 1) Realizing through writing their cases that they see themselves as designers, 2) Being able to consciously articulate and explain their design decisions through writing, 3) Finding that activity systems analysis helped provide a framework to assess tensions in the design process. Students concluded that engaging in analysis surprised them by revealing aspects of their design work they did not previously recognize and by claiming their identity as designers.
The document discusses assessing 21st century skills in students. It outlines 6 critical skills and provides indicators and evidence for measuring each skill:
1) Use real-world digital tools to access, evaluate, and apply information. Evidence includes student-created digital products and research tools rubrics.
2) Work independently and collaboratively to solve problems and accomplish goals. Evidence includes collaboration reflections and comments from teachers on student work.
3) Communicate information clearly using various tools in different contexts. Evidence includes student media products and their ability to tailor communication for audiences.
4) Demonstrate originality and inventiveness in work and understand progress in creative skills. Evidence includes student self-reflections and peer
This short document discusses an overview and conclusions. It appears to present some initial information on a topic and then ends with stating conclusions, but without providing any other context or details.
Report on Beautiful Evidence by E. Tuftehckingmtuedu
Edward Tufte's book Beautiful Evidence discusses how images can provide evidence and explanations when annotated with detailed mappings. Chapter 1 focuses on "mapped pictures" which are images annotated with credible, quantified, and contextualized details. Well-mapped images combine direct visual evidence with explanatory diagrams and text. They provide scale, location, and other contextual details to clarify why the image matters as evidence. Examples discussed include scientific illustrations, art analyses, and historical documents. Overall, Tufte argues that images should be thoroughly mapped like these examples to maximize their explanatory and evidential power.
The document discusses the importance of teaching vocabulary and background knowledge to improve reading comprehension. It describes several organized methods teachers can use before and after reading to activate students' background knowledge and help them synthesize new information, including the language experience approach, directed reading-thinking activity, and the survey-question-read-recite-review method. It also explains how text mapping, networking, and flowcharting involve visually representing the relationships between key ideas in a text.
The document discusses several web browsers:
1. Internet Explorer is a series of graphical web browsers developed by Microsoft and included in Windows operating systems starting in 1995. It has gone through several versions.
2. Firefox uses sandbox security and SSL/TLS encryption. It has features like tabbed browsing and extensions.
3. Google Chrome is developed by Google and based on the WebKit layout engine. It has automatic updates and is available for Windows, Mac OS, and Linux.
4. Opera is a full-featured browser available for free with features like tabbed browsing, zooming, and an integrated download manager. It has built-in security protections.
The document describes an adaptive planning process for humanitarian response. It includes the following key elements:
1) A strategic lens to guide decisions including values, standards, strategies, and frameworks like DADDs and CWBOs.
2) Identification of potential hazards/risks like earthquakes, tsunamis, conflicts through a context overview and scenarios.
3) Assessment of needs, response capacity, and risks to determine whether and how to respond to different events. The process involves acting, assessing, and adapting in an ongoing cycle.
The document discusses several web browsers:
1. Internet Explorer is a series of graphical web browsers developed by Microsoft and included in Windows operating systems starting in 1995. It has gone through several versions.
2. Firefox uses sandbox security and SSL/TLS encryption. It has features like tabbed browsing and extensions.
3. Google Chrome is developed by Google and based on the WebKit layout engine. It has automatic updates and is available for Windows, Mac OS, and Linux.
4. Opera is a full-featured browser available for free with features like tabbed browsing, mouse gestures, and an integrated download manager. It has built-in security protections.
This document outlines research areas and issues regarding human-computer interaction for older novice users. It discusses how the aging population is increasingly using computers and the Internet, but faces barriers like physical and cognitive decline, conceptual difficulties with technology, and negative attitudes. The document examines limitations in various areas and proposes solutions such as familiar control designs, tangible interfaces, voice/gesture recognition, and improved teaching methods. It stresses the need for future research on older users' motivations for adopting technology.
Action research for_librarians_carl2012srosenblatt
This document provides an overview of an action research workshop for librarians. The workshop aims to teach participants how to incorporate evidence-based research into their practice through action research. It discusses the action research cycle of planning, acting, observing, and reflecting. Participants will learn about generating research questions based on problems in their work, collecting and analyzing both quantitative and qualitative data, and sharing and applying the results to make changes and ask new questions. The workshop involves hands-on activities for participants to analyze sample datasets and plan their own action research projects to investigate issues in their own practice.
Action research for_librarians_carl2012srosenblatt
This document provides an overview of an action research workshop for librarians. The workshop aims to teach participants how to incorporate evidence-based research into their practice. It covers the basics of the action research process, including identifying a problem or question, collecting and analyzing data, reflecting on findings, and planning changes. The document outlines the learning outcomes, introduces the action research cycle, and discusses different research methodologies and tools for data collection and analysis that can be used, such as interviews, surveys, and Excel. Participants are guided through practicing these steps by analyzing sample datasets and are encouraged to begin planning their own action research projects.
Summary of ICSE 2011 Panel on "What Industry wants from Research". This is a summary of all the presentations from that panel that I presented in an invited talk at the CSER meeting in Toronto, November, 2011.
Messy Research: How to Make Qualitative Data Quantifiable and Make Messy Data...Gigi Johnson
This document discusses qualitative research methods for business. It addresses challenges in making qualitative data understandable for real decisions. It discusses why businesses conduct research, how to determine what data and analysis is needed, and issues with determining "truth" in business contexts. Finally, it discusses four types of qualitative data and focuses on making qualitative data more quantitative by addressing issues like validity, sample sizes, and coding consistency.
This document summarizes research on scientists' communication behavior and willingness to engage with the public. Key findings include:
- Scientists have negative views of the public and media, but want to be helpful. They lack training in public engagement.
- Willingness to engage online is predicted by younger age, higher efficacy beliefs, and a desire to contribute to debates.
- Defending science against misinformation is scientists' top priority for online engagement goals. Prioritizing strategic goals depends on attitudes, norms, and efficacy related to those goals.
This document discusses business analytics. It defines business analytics as using data, statistical and quantitative analysis, explanatory and predictive models to gain insights and support decision-making. The document outlines the typical business analytics process, including understanding the business objectives, assessing the situation, collecting and preparing data, developing analytic models, evaluating and reporting results, and deploying the outcomes. It provides examples of how analytics can be used to drive personalized customer services, optimize people management decisions, and conduct real-time sentiment analysis of social media data for an FMCG company. The document concludes with lessons learned, emphasizing the importance of continuous learning, gaining experience through projects and mentoring, and having confidence in one's abilities.
Classroom action research allows teachers to identify issues in their own classroom, collect and analyze data, and make improvements. It is a collaborative process where teachers develop research questions, study their own teaching methods, and use the results to benefit students. The key aspects are identifying problems, developing a plan of action, gathering and examining data, and creating new approaches based on the findings. Overall, action research is a reflective process that empowers teachers to evaluate their own practices and enhance student learning.
A Pragmatic Perspective on Software VisualizationArie van Deursen
Slides of the keynote presentation at the 5th International IEEE/ACM Symposium on Software Visualization, SoftVis 2010. Salt Lake City, USA, October 2010.
Online Orientation for non-Credit Instructorsdwmcnaughton
The document summarizes the development of an online orientation for non-credit instructors at a community college. It discusses research that found technology, student affairs, and teaching strategies were the most popular orientation topics. It also notes that part-time and adjunct faculty are increasing. The orientation was designed with online tools to provide help and required information. Challenges included coordinating between departments and changes in personnel. The implementation results showed the orientation received positive feedback for being easy to use and valuable, though some issues arose in migrating instructors to the new system.
This document provides a comparison of different online platforms for public managers to use for idea generation and public engagement. It describes the purpose and key features of 11 platforms: Bubble Ideas, Crowd Wise, Delib Dialogue App, Google Moderator, IdeaScale, Microsoft Town Hall, PubliVate, Salesforce Ideas, Spigit, UserVoice, and compares what each platform is well-suited for and its pricing information. The document aims to help public managers choose a platform that best fits their specific needs and objectives.
Games have the potential to transform learning by making it student-centered, complex, and intrinsically motivating. When designed well, games can engage students in solving real-world problems through interactive problem-solving and collaboration. Game-based learning approaches like project-based learning embed critical thinking, communication, and deeper learning within an authentic and engaging context. Educators are exploring how to apply game mechanics and principles of game design to better capture students' interests and promote active, self-directed, and collaborative styles of learning.
Information Systems design science researchRaimo Halinen
The document discusses various aspects of developing and evaluating information systems using a design science research approach. It provides guidelines for conducting design science research, including that the research should design an artifact to solve an identified problem, evaluate the artifact and process, and ensure rigorous and communicable results. The document also discusses various process models and approaches for design science research.
Abbott a more transparent interpretation of health club surveysDean Abbott
The document summarizes Dean Abbott's presentation on interpreting health club member surveys. It describes using factor analysis to reduce 57 survey questions into 10 factors, then regression analysis to identify the 5 most predictive factors for member satisfaction. Rather than using the full factors, the final model used the top-loading question from each factor for improved performance and interpretability. This provided actionable insights for understanding member satisfaction.
The document summarizes findings from a pilot workshop and doctoral course assignment that used activity systems analysis to help students understand design as a practice and share their experiences through narrative case studies. Key findings from student reflections included: 1) Realizing through writing their cases that they see themselves as designers, 2) Being able to consciously articulate and explain their design decisions through writing, 3) Finding that activity systems analysis helped provide a framework to assess tensions in the design process. Students concluded that engaging in analysis surprised them by revealing aspects of their design work they did not previously recognize and by claiming their identity as designers.
The document discusses assessing 21st century skills in students. It outlines 6 critical skills and provides indicators and evidence for measuring each skill:
1) Use real-world digital tools to access, evaluate, and apply information. Evidence includes student-created digital products and research tools rubrics.
2) Work independently and collaboratively to solve problems and accomplish goals. Evidence includes collaboration reflections and comments from teachers on student work.
3) Communicate information clearly using various tools in different contexts. Evidence includes student media products and their ability to tailor communication for audiences.
4) Demonstrate originality and inventiveness in work and understand progress in creative skills. Evidence includes student self-reflections and peer
This short document discusses an overview and conclusions. It appears to present some initial information on a topic and then ends with stating conclusions, but without providing any other context or details.
Report on Beautiful Evidence by E. Tuftehckingmtuedu
Edward Tufte's book Beautiful Evidence discusses how images can provide evidence and explanations when annotated with detailed mappings. Chapter 1 focuses on "mapped pictures" which are images annotated with credible, quantified, and contextualized details. Well-mapped images combine direct visual evidence with explanatory diagrams and text. They provide scale, location, and other contextual details to clarify why the image matters as evidence. Examples discussed include scientific illustrations, art analyses, and historical documents. Overall, Tufte argues that images should be thoroughly mapped like these examples to maximize their explanatory and evidential power.
The document discusses the importance of teaching vocabulary and background knowledge to improve reading comprehension. It describes several organized methods teachers can use before and after reading to activate students' background knowledge and help them synthesize new information, including the language experience approach, directed reading-thinking activity, and the survey-question-read-recite-review method. It also explains how text mapping, networking, and flowcharting involve visually representing the relationships between key ideas in a text.
The document discusses several web browsers:
1. Internet Explorer is a series of graphical web browsers developed by Microsoft and included in Windows operating systems starting in 1995. It has gone through several versions.
2. Firefox uses sandbox security and SSL/TLS encryption. It has features like tabbed browsing and extensions.
3. Google Chrome is developed by Google and based on the WebKit layout engine. It has automatic updates and is available for Windows, Mac OS, and Linux.
4. Opera is a full-featured browser available for free with features like tabbed browsing, zooming, and an integrated download manager. It has built-in security protections.
The document describes an adaptive planning process for humanitarian response. It includes the following key elements:
1) A strategic lens to guide decisions including values, standards, strategies, and frameworks like DADDs and CWBOs.
2) Identification of potential hazards/risks like earthquakes, tsunamis, conflicts through a context overview and scenarios.
3) Assessment of needs, response capacity, and risks to determine whether and how to respond to different events. The process involves acting, assessing, and adapting in an ongoing cycle.
The document discusses several web browsers:
1. Internet Explorer is a series of graphical web browsers developed by Microsoft and included in Windows operating systems starting in 1995. It has gone through several versions.
2. Firefox uses sandbox security and SSL/TLS encryption. It has features like tabbed browsing and extensions.
3. Google Chrome is developed by Google and based on the WebKit layout engine. It has automatic updates and is available for Windows, Mac OS, and Linux.
4. Opera is a full-featured browser available for free with features like tabbed browsing, mouse gestures, and an integrated download manager. It has built-in security protections.
This document outlines research areas and issues regarding human-computer interaction for older novice users. It discusses how the aging population is increasingly using computers and the Internet, but faces barriers like physical and cognitive decline, conceptual difficulties with technology, and negative attitudes. The document examines limitations in various areas and proposes solutions such as familiar control designs, tangible interfaces, voice/gesture recognition, and improved teaching methods. It stresses the need for future research on older users' motivations for adopting technology.
Action research for_librarians_carl2012srosenblatt
This document provides an overview of an action research workshop for librarians. The workshop aims to teach participants how to incorporate evidence-based research into their practice through action research. It discusses the action research cycle of planning, acting, observing, and reflecting. Participants will learn about generating research questions based on problems in their work, collecting and analyzing both quantitative and qualitative data, and sharing and applying the results to make changes and ask new questions. The workshop involves hands-on activities for participants to analyze sample datasets and plan their own action research projects to investigate issues in their own practice.
Action research for_librarians_carl2012srosenblatt
This document provides an overview of an action research workshop for librarians. The workshop aims to teach participants how to incorporate evidence-based research into their practice. It covers the basics of the action research process, including identifying a problem or question, collecting and analyzing data, reflecting on findings, and planning changes. The document outlines the learning outcomes, introduces the action research cycle, and discusses different research methodologies and tools for data collection and analysis that can be used, such as interviews, surveys, and Excel. Participants are guided through practicing these steps by analyzing sample datasets and are encouraged to begin planning their own action research projects.
Summary of ICSE 2011 Panel on "What Industry wants from Research". This is a summary of all the presentations from that panel that I presented in an invited talk at the CSER meeting in Toronto, November, 2011.
Messy Research: How to Make Qualitative Data Quantifiable and Make Messy Data...Gigi Johnson
This document discusses qualitative research methods for business. It addresses challenges in making qualitative data understandable for real decisions. It discusses why businesses conduct research, how to determine what data and analysis is needed, and issues with determining "truth" in business contexts. Finally, it discusses four types of qualitative data and focuses on making qualitative data more quantitative by addressing issues like validity, sample sizes, and coding consistency.
This document summarizes research on scientists' communication behavior and willingness to engage with the public. Key findings include:
- Scientists have negative views of the public and media, but want to be helpful. They lack training in public engagement.
- Willingness to engage online is predicted by younger age, higher efficacy beliefs, and a desire to contribute to debates.
- Defending science against misinformation is scientists' top priority for online engagement goals. Prioritizing strategic goals depends on attitudes, norms, and efficacy related to those goals.
This document discusses business analytics. It defines business analytics as using data, statistical and quantitative analysis, explanatory and predictive models to gain insights and support decision-making. The document outlines the typical business analytics process, including understanding the business objectives, assessing the situation, collecting and preparing data, developing analytic models, evaluating and reporting results, and deploying the outcomes. It provides examples of how analytics can be used to drive personalized customer services, optimize people management decisions, and conduct real-time sentiment analysis of social media data for an FMCG company. The document concludes with lessons learned, emphasizing the importance of continuous learning, gaining experience through projects and mentoring, and having confidence in one's abilities.
Classroom action research allows teachers to identify issues in their own classroom, collect and analyze data, and make improvements. It is a collaborative process where teachers develop research questions, study their own teaching methods, and use the results to benefit students. The key aspects are identifying problems, developing a plan of action, gathering and examining data, and creating new approaches based on the findings. Overall, action research is a reflective process that empowers teachers to evaluate their own practices and enhance student learning.
A Pragmatic Perspective on Software VisualizationArie van Deursen
Slides of the keynote presentation at the 5th International IEEE/ACM Symposium on Software Visualization, SoftVis 2010. Salt Lake City, USA, October 2010.
Online Orientation for non-Credit Instructorsdwmcnaughton
The document summarizes the development of an online orientation for non-credit instructors at a community college. It discusses research that found technology, student affairs, and teaching strategies were the most popular orientation topics. It also notes that part-time and adjunct faculty are increasing. The orientation was designed with online tools to provide help and required information. Challenges included coordinating between departments and changes in personnel. The implementation results showed the orientation received positive feedback for being easy to use and valuable, though some issues arose in migrating instructors to the new system.
This document discusses teaching computational thinking through technologies education. It emphasizes developing students' thinking skills like design thinking, computational thinking, systems thinking and futures thinking through project-based learning. The document outlines curriculum outcomes, contexts, challenges and expectations for developing solutions across different year levels. It also discusses integrating different models of thinking, evaluating solutions, and the importance of creativity, innovation and accepting failure in the learning process.
The document summarizes a study that examined how templated requirements specifications can inhibit creativity in software engineering. The study found that presenting requirements in a structured template format led designers to fixate on the specifications, showing fewer instances of critical thinking. Specifically:
- The study used verbal protocol analysis to examine the cognitive processes of designers given a task with templated requirements specifications.
- Analysis of the dialogues found designers showed more instances of fixation on the given requirements compared to critical thinking, suggesting the structured format hindered their creative thinking.
- More experienced developers were even more prone to fixation on the templated requirements.
- The study contributes a theory that requirement fixation, from being preoccupied with satisfying structured specifications
This document provides an overview of research methods and best practices for conducting reader research. It discusses that research involves formal curiosity and asking questions with a purpose. There are two main types of research: primary research involving new data collection, and secondary research using existing data. When writing questions, it is important to have clear goals and ask effective, easy to understand questions. Both quantitative and qualitative methods are covered, noting their different focuses and analysis approaches. The document also offers tips for increasing survey response rates, analyzing results, and conducting a class exercise to design a research plan with questions.
This document provides an overview of research methods and best practices for conducting reader research. It discusses that research involves formal curiosity and asking questions with a purpose. There are two main types of research: primary research involving new data collection, and secondary research using existing data. When writing questions, it is important to have clear goals and ask effective, easy to understand questions. Both quantitative and qualitative methods are covered, noting their different focuses and analysis approaches. The document also offers tips for increasing survey response rates, analyzing results, and conducting a class exercise to design a research plan with questions.
First Cycle CodingContent drawn from Johnny Saldana’s The .docxclydes2
First Cycle Coding
Content drawn from Johnny Saldana’s The Coding Manual for Qualitative Researchers.
David Lee — TIM 158, Spring 2019
Credit: YCombinator : How to Start a Startup
Recall
generating hypotheses
Needfinding is about Recall
Who / What How / WhyHello! Thanks!
Recall
Summary
• Go from what to how and why, why, why
• Develop a model of an individual
• setting, actions → thoughts, feelings → values, motivations
• Then reflect on needs
• also consider: what is top of mind? hacks and workarounds?
Recall
Deep, rich understanding of individuals
Hypotheses about narrow user segments
Hypotheses about solution concepts
Recall
dt+UX: Design Thinking for User Experience Design, Prototyping & Evaluation 7
Generate Evaluate Generate Evaluate
Recall
dt+UX: Design Thinking for User Experience Design, Prototyping & Evaluation 8
Questions Prototype Questions Prototype
Recall
dt+UX: Design Thinking for User Experience Design, Prototyping & Evaluation2018/10/08 9
Recall
Follow-up with a
30-min interview.
We’re still in hypothesis generating mode!
Recall
The next two weeks
• Revisiting unpacking
• Communicating your concept
The prototyping process
Generate questions
!Untested design thesis
!Risky design decisions
!Unobserved user behaviors
Rank questions
Which is most critical?
Build and test a prototype
Answer only the most critical question
Recall
Today
• Overview of Qualitative Analysis and Coding
• First Cycle Coding and Analytic Memos for HW #3
What is qualitative research?
So first…
Qualitative research
• When you’re trying to
• develop a rich understanding of a complex phenomena and the complex
interactions between factors
• communicate a holistic interpretation or narrative that helps readers experience
“being there”
• Ask → Collect → Organize → Analyze → Theory
• In our case: a model of the user segments and their context or experiences in
relation to our product (segment, setting, sequence, satisfaction)
How do you go from qualitative data
to patterns, concepts, and theories?
Unpacking so far…
dt+UX: Design Thinking for User Experience Design, Prototyping & Evaluation2018/10/01
Recall
dt+UX: Design Thinking for User Experience Design, Prototyping & Evaluation
KEEP A LIST OF
TENSIONS, CONTRADICTIONS, SURPRISES
say
do
think
feel
2018/10/01
USE TO FIND NEEDS & INSIGHTS
Empathy Map to Help Synthesize
dt+UX: Design Thinking for User Experience Design, Prototyping & Evaluation
INSIGHTS
I wonder if this means . . .
think
feel
TENSIONS,
CONTRADICTIONS,
SURPRISES
2018/10/08 20
USERS & NEEDS
dt+UX: Design Thinking for User Experience Design, Prototyping & Evaluation
combine to create a point of view
need insight
2018/10/01 …
SU
RP
ISE
D
TO
D
ISC
OV
ER
...
…
GA
ME
-C
HA
NG
ING
TO
…
user attribs.
WE MET . . .
(extreme user you are inspired by)
WE WERE AMAZED TO REALIZE . . .
(what did you learn that’s new? What is their need?)
IT WOULD BE GA.
1. Vita Beans provides concise neurobehavioral profiles of people by having them play online games and activities for 5-8 minutes. This captures both spontaneous and conscious behaviors that can't be manipulated.
2. Their apps help understand employees, compare job applicants to current employees, and segment customers to provide customized content.
3. The profiles generated can be reused over time and customized for different analysis purposes, unlike traditional questionnaires that require new testing.
Human-centered AI: how can we support lay users to understand AI?Katrien Verbert
The document summarizes research on human-centered AI and how to support lay users in understanding AI. It discusses various research projects that aim to explain model outcomes to increase user trust and acceptance. It explores how personal characteristics like need for cognition can impact the effectiveness of explanations. The research also looks at different application domains for AI like healthcare, education, agriculture and recommendations. It emphasizes the importance of user involvement, personalization and domain expertise in developing AI systems that non-experts can understand and trust.
Thriving in an Uncertain World: Designing Virtual Teams Across the Innovation...Sociotechnical Roundtable
The document describes a roundtable discussion on designing virtual teams across the innovation continuum. It took place in April 2013 in Denver and was supported by the National Science Foundation. The roundtable explored how to coordinate virtual teams effectively across different stages of research and development. It also discussed evolving principles for sociotechnical system design over three waves from the 1950s to present, with the current wave characterized by virtual, distributed knowledge work. The document provides examples of challenges for two virtual research sites: fundamental optical research conducted across several countries, and creating a uniform Alzheimer's disease dataset from multiple research centers.
Thriving in an Uncertain World: Designing Virtual Teams Across the Innovation...
Defense 20121130
1. Understanding “Just Enough”
Computer Users:
Motivation Style and Proficiency
By Harriet King
Masters Candidate in Computer Science
1 of 45
2. The Question
Why do some proficient daily
computer users, stumble over the
unfamiliar and others easily adapt?
EXAMPLE: More information and detail in Supplementary Slides More
Introduction Study Design Motivation Observations Future Work Conclusions 2 of 45
3. What Is a Just Enough (JE) User?
• Daily computer user
• Competent
• Extrinsic Motivation
Introduction Study Design Motivation Observations Future Work Conclusions 3 of 45
4. The Hypothesis
We hypothesize that
extrinsically motivated
proficient daily computer users
have difficulty with unfamiliar computer
tasks and skill transfer, whereas
intrinsically motivated daily users
accomplish unfamiliar tasks readily.
Introduction Study Design Motivation Observations Future Work Conclusions 4 of 45
5. Who Cares?
• Software designers
• Human Computer Interactions (HCI)
• Software Users
• Stakeholders for computer literacy
“Lest we wish to change our field’s name to
student-computer interaction we should make
effort to find more representative participants”
(Barkhuus and Rode 2012)
Introduction Study Design Motivation Observations Future Work Conclusions 5 of 45
6. Study Design Overview
OUTPUT
INVENTORY
scores & statistics
group descriptors
OBSERVATIONS
Coded & analyzed
attitudes & actions
Introduction Study Design Motivation Observations Future Work Conclusions 6 of 45
7. Richness of Data for Understanding
• Pre-questionnaire: daily users?
• Quantitative motivation inventory scores
• Demographic and interview questions
• Ethnographic observation methods:
– Think Aloud Protocol
– Observation recordings
– Researcher questions and follow up
• Quantify transcripts with coding
• Post-questionnaire and JE Users questionnaire
(Sim 1999; Rose, Shneiderman, Plaisant. 1995)
Introduction Study Design Motivation Observations Future Work Conclusions 7 of 45
9. Motivation Background
Motivation Styles, adapted from Ryan and Deci (2000) ‘Taxonomy of Human Motivation’.
Low interest and enjoyment are on the left ranging to high interest and enjoyment on the
right. (Pintrich 2003; Deci and Ryan 1991; Downey and Smith 2011; Martens et al. 2004;
Deci and Ryan 1985; Iyengar and Lepper 2000; Henderlong and Lepper 2002; Ryan and
Deci 2000; Ryan and Deci 2012; Oudeyer et al. 2007)
More
More information and detail in Supplementary Slides
Introduction Study Design Motivation Observations Future Work Conclusions 9 of 45
10. Motivation Inventory
Source Factors
Guay, Vallerand, Blanchard 1. Amotivation
(2000) 2. External Regulation
3. Identified Regulation
Ryan and Deci (IMI 2012) 4. Interest/Enjoyment
5. Perceived Choice
6. Perceived Competence
L to R: Richard Ryan and Edward Deci
(Photo: Adam Fenster, August 2010)
Introduction Study Design Motivation Observations Future Work Conclusions 10 of 45
11. Adapting Questions
Precedent: (Shroff and Vogel 2009). Confirmed Inventory with two pilot studies.
Introduction Study Design Motivation Observations Future Work Conclusions 11 of 45
12. Precedents for Scoring Inventory
Likert scale IS-An ordinal measure of ranking
“We did violate some mathematical assumptions in creating an
interval level of measurement index out of ordinal components,
but as previously indicated, this is common practice in the social
and behavioral sciences.” (Sirkin, R. M., 2006. “Statistics for the
Social Sciences.” 3rd edition, Sage Publications.
Precedent for averaging motivation inventory scores
1. Pavlas, Jentsch, Salas, Fiore, and Sims, 2012
2. Shroff and Vogel, 2009
3. McAuley, Duncan, and Tammen, 1989
Introduction Study Design Motivation Observations Future Work Conclusions 12 of 45
13. Who Took the Inventory? Everybody!
• Ages 13 to 87 from FIVE continents
• 9 countries: USA, China, Turkey, Australia, Sweden, U.K.,
South Africa, India, and France
• 130+ total completed questionnaire
• Used 66 for total respondents
• 16 participants observed (7 intrinsics, 9 extrinsics)
Community
Classmates
Faculty
Internet
Introduction Study Design Motivation Observations Future Work Conclusions 13 of 45
14. Required Correlation
Table 8: Pearson Correlation of
Interest/Enjoyment & Perceived Choice
Correlation of Interest/Enjoyment & Perceived Choice
Factors
n = 66 n =16
All Respondents All Observed
Correlation 0.602 0.815
Significance (2- p < 0.01 p < 0.01
tailed)
Introduction Study Design Motivation Observations Future Work Conclusions 14 of 45
15. Grouping Variables
Venn Diagram is
External Regulation > 4.0
intersecting Total and percent inventory
Interest/Enjoyment > 4.0 responses by group with n=66
Introduction Study Design Motivation Observations Future Work Conclusions 15 of 45
16. Inventory T Test Results
Significant Differences in Inventory
Scores, Age, & Digital Native
* Asterisk indicates non parametric Mann-Whitney U test
All other are Independent Samples T-test
Factor Different Significance
Age* NOT different p=0.396
Digital Native* NOT different p=0.166
Perceived Competence* NOT different p=0.071
Amotivation* Different p=0.012
External Regulation Different p<0.001
Interest/Enjoyment Different p<0.001
Perceived Choice Different p=0.001
More information and detail in Supplementary Slides More
Introduction Study Design Motivation Observations Future Work Conclusions 16 of 45
17. Intrinsics: Digital Native or Not
Side by side comparison of digital non-natives (3) on left and digital
natives (4) on right. Ordered from low to high competence
Perceived
Interest/ Enjoyment Perceived Choice
Digital native Competence
Digital native
Digital native
Non-native
Non-native
Non-native
6.43 5.86 6.33
5.00 6.71 4.71 5.14 2.67 6.67
4.14 6.14 4.14 5.29 3.00 6.83
5.57 6.57 4.29 4.86 5.17 7.00 More
Introduction Study Design Motivation Observations Future Work Conclusions 17 of 45
18. Digital Natives
not significantly different
Digital Natives All
70
66
60
50
Number of People
40
41%
30 27
37%
57% 22%
20
16
10
9
6 7
4
2
0
Observed Participants Extrinsics
Inventory Respondents Intrinsics
Groups
Introduction Study Design Motivation Observations Future Work Conclusions 18 of 45
19. Not Significantly Different
Age, Perceived Competence, & Digital Native or not
Mean Age with error bars for Mean Perceived Competence with
standard deviation error bars for standard deviation
90.00 7.00
Mean Perceived Competence
80.00
6.00
70.00 5.38
60.00 55.67 5.00
50.00 46.57
3.70
Age
4.00
40.00
30.00 3.00
20.00
2.00
10.00
0.00
1.00
9 Extrinsics 7 Intrinsics 9 Extrinsics 7 Intrinsics
Introduction Study Design Motivation Observations Future Work Conclusions 19 of 45
20. Mean Inventory Results
with error bars showing standard deviation
n=66 Respondents n=16 Observed n=9 JE Users n=7 Intrinsics
8
7
6
Likert Scale 1 - 7
with neutral at 4
5
4
3
2
1
Grouping Grouping
Variable Variable
0
More
Introduction Study Design Motivation Observations Future Work Conclusions 20 of 45
21. Data Screening
Extra High Perceived Choice
Mean Perceived Choice with standard
Extrinsic Molly = 5.57!? deviation error bars
2.3 standard deviations above
5.57
More
Introduction Study Design Motivation Observations Future Work Conclusions 21 of 45
24. Near Skill Transfer
More
Introduction Study Design Motivation Observations Future Work Conclusions 24 of 45
25. Participant Hesitation Wording
“uhhhh” looked in there” promising”
“I’m looking for a way to “I think I can just... click on [sigh]
do...” this here, and... that didn’t “no, that's not it”
“maybe if I go here” work” “maybe this”
“what’s this?” “ok, that didn’t work” “so, we're not doing that”
“I can’t...” “I looked at the bottom but “I wouldn't think it'd be
“ummm” there’s nothing there” under that”
“let’s go back here” “I saw this click to ... but “I'm going to try right click
[giggling] that isn’t it” again”
“aaaaannnnnnnd” “hmmm” “I forgot what you said to
“I could try like..” “contacts....contacts.... do”
“no I can’t drag that..” contacts” “this damn mouse”
“I’ll look in here, no I just “that doesn't look very
Introduction Study Design Motivation Observations Future Work Conclusions 25 of 45
26. For Prompting the Participant
“go ahead and tell me what you’re seeing”
“please tell me what you’re thinking”
“Are you trying to decide something, can you tell me about it?”
“did that work?”
“what seems odd about this?”
“what are you thinking?”
“you’re giggling, …you’re sighing…you sound angry, what are you
feeling?”
Introduction Study Design Motivation Observations Future Work Conclusions 26 of 45
27. Rubric for Coding Observations
CODE RULE
Stumble [action] >= 20 seconds
Fall [action] >= 1 minute
Persist [action] >= 3 minutes
Quit attitude towards a task
Resist attitude towards a task
Introduction Study Design Motivation Observations Future Work Conclusions 27 of 45
28. Transcript Example
stumble
persist
resist
quit
fall
time OLIVIA [action] “quote” (time on video) analysis
b 7:58 [while looking for spam, stumbles across trash 7:58 and says I’ll empty the trash 1 1 1
e 9:08 instead, I say go ahead] Participant: “I have no idea how to do that. It’s already IN the
trash” me: “Look around. ...you can empty the trash.” (8:10) Participant: “It’s already IN
trash. Where do you empty trash to? I’m thinking that I never empty my trash because
there’s no way to empty trash because it’s already trash.” (8:25) me: “no, there is a way
to empty trash.” Participant: “There’s no trash emptying.”
[ask about her agitation] Participant: “I’m not agitated at all. You’re just wrong. There’s
no trash emptying.” [ask what she’s feeling] Participant: “I think it’s dumb that the trash
doesn’t have an empty.” (8:40) me: “It does actually”
Participant: “I don’t see it. If I click on something in my trash, all I can do is trash
something in my trash, which is silly because it’s already in my trash” (9:08) me: “Ok,
we’ll come back to this. Let’s look at your spam” [so resistant that I stop this task on
test. Never does trash]
b 9:10 Participant: “I don’t know if I have spam” (9:10) me: “You do have spam.” “No. Really!? 1 1
e 9:45 I’m looking at all my folders and I do not have one called “spam”” (9:20) me: “Did you
find “more” at the bottom?” “There’s a more. Oh look at that, there’s spam.” (9:45)
b 9:50 [directed to delete all spam at once, (9:50), giving her hints] me: “It’s not that tricky, it 1 1
e 11:10 has words and I can see them, I’m looking at it right now” (10:37) (11:10) found “delete
all messages now”
b 11:20 [11:20 Go to address book] Participant: “I’m not fully sure where my address book is, I 1
e 12:10 think I have to go to my calendar”, then found contacts 12:10
Introduction Study Design Motivation Observations Future Work Conclusions 28 of 45
29. Inter Rater Reliability Results
• First Rater (HK)
• 2 outside raters (SK and PM)
• Outside raters reviewed 30% of transcripts
• Stumble, fall, and persist are time related
Rater 1 Rater 2
Stumble, Fall, Persist 100% agreement 100% agreement
Quit 99.13% agreement 97.73% agreement
Resist 96.52% agreement 97.73% agreement
Introduction Study Design Motivation Observations Future Work Conclusions 29 of 45
30. Occurrences for Each Code
• Asterisk indicates statistically significant difference for this code
between extrinsic and intrinsic. Total occurrences with percent of
total in parentheses.
• There was no significant difference between Unfamiliar Task
compared to Near Skill Transfer for either intrinsics or extrinsics.
Stumble* Fall* Persist* Quit* Resist
JE Users 91 56 15 9 13
(81%) (84%) (88%) (90%) (87%)
Intrinsics 21 11 2 1 2
(19%) (16%) (12%) (10%) (13%)
More
Introduction Study Design Motivation Observations Future Work Conclusions 30 of 45
31. All Occurrences of Stumble & Fall
Intrinsics on left and Extrinsics on right
stumble fall
Extrinsics
Intrinsics
20
15
10
5
0
More
Introduction Study Design Motivation Observations Future Work Conclusions 31 of 45
32. JE User vs. Intrinsic: Marsha & Rebecca
Exter: 4.5 Exter: 4.0
Int/En 2.57 Int/En 5.57
Similar:
1. both Amotivation = 1.0
2. Both digital non-native Different:
3. similar experience level 1. Performance
4. similar self rate and perceived competence 2. Different motivation styles
5. similar age
6. Appeared to cruise through unfamiliar tasks
7. Responsible community leaders More
8. Professional women
Introduction Study Design Motivation Observations Future Work Conclusions 32 of 45
33. Resist
• Only 5 out of 16 resisted 8
• 4 extrinsic & intrinsic Mike 7
• Olivia had 7 resists
Total Occurrences of Resist
6
1. Can’t empty trash
2. there is no spam 5
3. doesn’t “add” to group but 4
insists she did
4. says “check mail” button is 3
broken 2
5. won’t remove attachment,
6. says used wrong address but 1
was sent folder issue 0
7. says did not spell a word Lucy Mike* Miranda Marsha Olivia
correctly when did spell
correctly More
Introduction Study Design Motivation Observations Future Work Conclusions 33 of 45
34. Another Type of Resist
Marsha shares, “I never use the google calendar. I’m not telling them what I’m
doing every day. Forget that!”
“Passionate?...I am. I’m not MAD at them [MS Word], I’m frustrated with them.
… they’re leaving out the average person. And maybe that’s what open office is
for. I don’t know.”
Introduction Study Design Motivation Observations Future Work Conclusions 34 of 45
35. Quit
8 of 16 quit Quit Resist
8
* Asterisk indicate intrinsic
Total Occurrences of Quit
7
6
5
4
3
2
1
0
Screenshot of "Contacts" button
behind "Mail" in Gmail. Doesn't
look like a button with no rectangle More
or color change.
Introduction Study Design Motivation Observations Future Work Conclusions 35 of 45
36. Persist
6
Total Occurrences of
5
4
Persist
3
8 of 16 Persist
2
1
0
Mary Molly Lucy Olivia Mike* Marsha Alice Walter
Ann
Walter spent about 5 minutes (7:40 to 13:10)
using wrong password of “guest” and misspelled username trying
to login to gmail online.
He repeated the same behavior while expecting different results
Introduction Study Design Motivation Observations Future Work Conclusions 36 of 45
37. Lowest Interest & Choice
• Extrinsics Lilly and Olivia
• Opposite attitudes (shame vs. blame)
Lilly Olivia
7.00
6.00
5.00
4.00
3.00
2.00
1.00
0.00
Perceived
Enjoyment
amotivation
Competence
regulation
external
Perceived
Choice
Interest
More
Introduction Study Design Motivation Observations Future Work Conclusions 37 of 45
38. Just Enough User Alice (1/9)
“I don’t do ANYTHING that I’m not taught. And that is a big
drawback in my learning.”
“I know enough to get what I want, most of the time. And it
definitely is not a pleasure for me to try to figure out things on
my own. N-O-T AT A-L-L… Maybe everyone thinks they are a “Just
Enough” user.”
More
Introduction Study Design Motivation Observations Future Work Conclusions 38 of 45
39. Just Enough User Lucy (3/9)
“Why would I Google it? I wouldn’t, because it’s a bunch of
teenagers who can’t spell right, who don’t use punctuation, all
lower case.”
“I am fine using the computer only for what I need. I think they
are ruining the world quite frankly, and am slightly proud I find
them somewhat repulsive machines.”
More
Introduction Study Design Motivation Observations Future Work Conclusions 39 of 45
40. Just Enough User Walter (8/9)
“You are … confronting an unbelievably unfamiliar system, with all the
scariness of being surrounded by REAL fully paid, fully trained, card
carrying life member geeks … I got spooked by the surroundings. I got
intimidated by my high level of geekitude surroundings.”
“People do get on without a computer at all, so perhaps ‘No
Computer’ (or ‘The Computer They Make You Use At Work’) is the true
‘Just Enough Computer’.”
More
Introduction Study Design Motivation Observations Future Work Conclusions 40 of 45
41. Future Work
• So much data!
• Bigger sample
• “Just Enough” term?
• Gender, socioeconomic status, years of experience,
aversion to change?
• Separating work and play in motivation study
• Less frequent users?
• What if a “consequence” element?
• Hand held computers?
Introduction Study Design Motivation Observations Future Work Conclusions 41 of 45
43. Statistical Results
• Confirmed competency of JE users
• Extrinsic proficient daily users stumble, fall,
persist and quit significantly more than intrinsics
• AND it is not explained by age, perceived
competence, or being digital native
• JE users account for over 80% of performance
difficulties in our study
• Just Enough users exist in all age groups and
experience levels (18% in our sample)
Introduction Study Design Motivation Observations Future Work Conclusions 43 of 45
44. Observed Phenomena
• Impossible to differentiate JE user from any
other competent user, until faced with the
unfamiliar
• Just Enough users shed competencies as they
become unnecessary
• Wide range of attitudes and experience
related to exploring and performance
• Sense of “not belonging”
Introduction Study Design Motivation Observations Future Work Conclusions 44 of 45
45. A Haiku
Thank
Just Enough is cool
you! till unfamiliar and new
safe routine un-do.
45 of 45
46. Just Enough User Lilly (2/9)
When asked during the test about her feelings,
Lilly shares, “ohhh, why am I so stupid? How can
I not know how to do this? I dread asking one of
my kids because they have no patience.”
“I really want computers to be as unobtrusive in
my daily life as can be. Just Enough term sounds
a bit lazy.”
More
Introduction Study Design Motivation Observations Future Work Conclusions 46 of 45
47. Just Enough User Marsha (4/9)
Marsha says, “I like to sign out, because then they,
THEORETICALLY, aren’t watching me, but you know
they are because advertisements for something I
just looked at turn up on the *weirdest* pages.”
“My feelings are that I would like to be more than
that [JE user]. I would consider a "just enough" user
to be one who uses only email, or only cruises the
web for news, or only uses one application.”
More
Introduction Study Design Motivation Observations Future Work Conclusions 47 of 45
48. Just Enough User Mary Ann (5/9)
“When I’m at work, I’m so busy, that I don’t have
time to play around... I always have to do things
in the fastest way possible, which doesn’t allow
exploration.”
“My feelings are that I would like to be more
than that. I do not want to be a "dinosaur. I
sometimes can do a little more than just enough
if I get up my courage to try."
More
Introduction Study Design Motivation Observations Future Work Conclusions 48 of 45
49. Just Enough User Miranda (6/9)
“It seems stupid and why should I waste my
time staring at the computer.”
“My feelings are, why would I spend any more
time at the computer? I'd rather read a book or
take a walk. Just enough is a perfect name.”
More
Introduction Study Design Motivation Observations Future Work Conclusions 49 of 45
50. Just Enough User Molly (7/9)
“This all is stupid. This is ridiculous. I don’t know
why anyone uses computers. … I don’t really
care. I can basically do anything I need to do and
I have [IT worker] and if I can’t do anything I just
call [IT worker] and cry.”
“The term "Just Enough" is kind. I don't feel
judged or "less than" (stupid).”
More
Introduction Study Design Motivation Observations Future Work Conclusions 50 of 45
51. Just Enough User Olivia (8/9)
“[it] is really annoying not to be able to find
these things that you’re CLAIMING it’s on here.
And it’s like, how are you supposed to know
where it is.....[I’m] irritated.”
“Very proud that I can do it enuf [sic]. People
should make more things easy for us.”
More
Introduction Study Design Motivation Observations Future Work Conclusions 51 of 45
52. “Just Enough Users”, a poem
Just enough is satisficing, works out fine till new and strange.
Computer changes make life messy,
then it’s struggle stumble quit.
Those interest people cruise along,
probably nothing ever wrong.
Curse you easy flexing user. Why can’t I just find my cursor?
Just Enough left me so helpless,
when the web changed all my favorites.
I just want to stay so lazy, stay low interest, stay low effort.
OK sometimes then I stumble. Just Enough was not effective.
Who to blame and who to curse?
Designers! They must be the worst.
Conclusion
52 of 45
53. Intrinsic Motivation Characteristics
• Deeper involvement in activities; natural activity
• More curiosity; exploration
• Trying out more complex options
• Increased persistence
• Higher achievement of goals; improved
performance
• Less avoidance behavior
• Interest, excitement, and confidence
Back
(Martens et al. 2004; Oudeyer et al. 2007; Deci and Ryan 2000)
Motivation Inventory
53 of 45
54. Intrinsic Motivation: Supports & By Products
• Self-esteem and general well-being
• Competence
• Autonomy
• Adaptable
• Pros/cons of praise Back
• Reduced by external rewards
• Supported by seeing examples; having capability
(Pintrich 2003; Deci and Ryan 1991; Downey and Smith 2011; Martens et al. 2004;
Deci and Ryan 1985; Iyengar and Lepper 2000; Henderlong and Lepper 2002; Ryan and
Deci 2000; Ryan and Deci 2012; Oudeyer et al. 2007)
Motivation Inventory
54 of 45
55. Parametric or Non parametric?
Does data pass the 3 assumptions for parametric
statistical analysis?
1. Independence? Yes! All different humans
2. Homogeneity? (equal variance, Levene’s test)
3. Normality? (skewness & kurtosis < |1.95|)
Back
Does it pass for 66 respondents and 16 participants?
Motivation Inventory
55 of 45
56. Homogeneity of Inventory Factors
Levene’s Test for Levene’s Test for
Equality of Variances Equality of Variances
n = 16 Observed. n = 66 Respondents.
Significance Significance
Amotivation 0.053 0.002
Identified Regulation 0.802 0.546
External Regulation 0.572 0.822
Interest/Enjoyment 0.989 0.842
Perceived Choice 0.492 0.218
Perceived Competence 0.152 0.010
Motivation Inventory
Back
56 of 45
58. T Test Result Detail
Amotivation (Mann-Whitney U test) (U = 9.50,
p = 0.012).
Perceived Choice (independent Samples T test)
extrinsic (M=2.7, SD=1.3) and intrinsics (M=4.9,
SD=0.6); t(14)=4.306, p=0.001.
Back
58 of 45
59. Intrinsics Descriptive Statistics
N Min Max Mean Std. Dev Skewness Kurtosis
Std. Std.
Statistic Error Statistic Error
age 7 23 87 46.71 27.93 0.59 0.79 -1.96 1.59
amotivation 7 1.00 1.50 1.07 0.19 2.65 0.79 7.00 1.59
external 7 1.00 4.00 2.64 1.02 -0.19 0.79 -0.06 1.59
regulation
Interest/ 7 4.14 6.71 5.80 0.95 -0.96 0.79 -0.11 1.59
Enjoyment
Perceived Choice 7 4.14 5.86 4.90 0.59 0.32 0.79 -0.35 1.59
Perceived 7 2.67 7.00 5.38 1.84 -0.87 0.79 -1.30 1.59
Competence
Amotivation (Mann-Whitney U test) (U = 9.50, p = 0.012). Back
Perceived Choice (independent Samples T test) extrinsic (M=2.7,
Motivation Inventory intrinsics (M=4.9, SD=0.6); t(14)=4.306, p=0.001.
SD=1.3) and
59 of 45
61. Summary of Correlations
Back
n=66 Inventory Respondents & n=16 Observed Participants
Relationship Correlation Significance n R^2
External Regulation with - 0.821 p=0.001 16 67.40%
Interest/Enjoyment
- 0.397 p=0.001 66 15.76%
External Regulation with Perceived - 0.879 p=0.001 16 77.26%
Choice
- 0.785 p=0.001 66 61.62%
Amotivation with Perceived - 0.602 p=0.014 16 36.24%
Competence
- 0.339 p=0.005 66 11.49%
Age with Perceived Competence - 0.710 p=0.002 16 50.41%
Motivation Inventory
61 of 45
62. Summary of Correlations
n=66 Inventory Respondents & n=16 Observed Participants
Relationship Correlation Significance n R^2
External Regulation with - 0.821 p=0.001 16 67.40%
Interest/Enjoyment
- 0.397 p=0.001 66 15.76%
External Regulation with Perceived - 0.879 p=0.001 16 77.26%
Choice
- 0.785 p=0.001 66 61.62%
Amotivation with Perceived - 0.602 p=0.014 16 36.24%
Competence
- 0.339 p=0.005 66 11.49%
Age with Perceived Competence - 0.710 p=0.002 16 50.41%
Back
Motivation Inventory
62 of 45
63. Digital Native Correlations
Digital Native Significant Correlations for Observed Participants
Digital Native Correlation Significance n R^2
Relationship with...
...Age 0.536 p<0.001 16 28.73%
...Interest/Enjoyment 0.561 p=0.024 16 31.47%
...Perceived Choice 0.575 p=0.020 16 33.06%
...Perceived Competence 0.647 p=0.007 16 41.86%
...External Regulation -0.534 p=0.033 16 28.52%
Back
Motivation Inventory
63 of 45
64. Mean Occurrences of Codes
Mean Number of Code Occurrences for Extrinsics and Intrinsics
Extrinsics Intrinsics
stumble 10.11 3.00
fall 6.11 1.57
quit 1.00 .29
resist 1.11 .29
persist 1.67 .29
Back
Observations
64 of 45
65. Correlations for Extrinsics
Extrinsic Group Significant Relationships
Relationship To Correlation Significance n R^2
Age Persist 0.667 0.050 9 44.49%
Digital Native -0.728 0.026 9 53.00%
Amotivation -0.713 0.031 9 50.84%
External Perceived -0.699 0.036 9 48.86%
Regulation Choice
Back
Observations
65 of 45
66. Correlations for Intrinsics Back
Relationship To Correlation Significance n R^2
Stumble Fall 0.898 .006 7 80.64%
Age 0.823 .023 7 67.73%
Digital Native -0.832 .020 7 69.22%
Interest -0.861 .013 7 74.13%
Perceived Competence -0.917 .004 7 84.09%
Digital Native Age -0.866 .012 7 75.00%
External Regulation -0.874 .010 7 76.39%
Interest/Enjoyment 0.866 .012 7 75.00%
Perceived Choice 0.866 .012 7 75.00%
Perceived Competence 0.866 .012 7 75.00%
Age External Regulation 0.757 .049 7 57.30%
Perceived Competence -0.929 .003 7 86.30%
Perceived Fall -0.768 .044 7 58.98%
Competence Interest/Enjoyment 0.786 .036 7 61.78%
External -0.883 .008 7 77.97%
Regulation Perceived Choice
Observations
66 of 45
69. Phase 2 & 3 Extrinsic Stumbles
Back
Total stumble occurrences for each extrinsic participant in phase 2 (blue on left) and
phase 3 (orange on right)
Observations
69 of 45
70. Comparing Intrinsic Digital Native Inventory Scores
Ordered from Lowest Perceived Competence to Highest
Competence
Amotivation
Regulation
Enjoyment
Perceived
Perceived
External
Interest/
Choice
Experience Age
Beth 16-25 years 26 1.00 1.00 6.43 5.86 6.33
Jane 16-25 years 27 1.00 2.50 6.71 5.14 6.67
Roger 16-25 years 23 1.00 2.00 6.14 5.29 6.83
Peter 16-25 years 24 1.00 2.50 6.57 4.86 7.00
Observations Back
70 of 45
71. Comparing Intrinsic Digital Non-Native
Inventory Scores
Ordered from Lowest Perceived Competence to Highest
Competence
Amotivation
Regulation
Enjoyment
Perceived
Perceived
External
Interest/
Choice
Experience Age
Wilma 16 - 25 years 87 1.00 3.75 5.00 4.71 2.67
Mike 16 - 25 years 74 1.50 2.75 4.14 4.14 3.00
Rebecca 25+ years 65 1.00 4.00 5.57 4.29 5.17
Back
Observations
71 of 45
72. Using Help or Not Back
• Many had no experience
• Or old experience from 10 years ago when help was
notoriously bad
• Stumbling of intrinsic digital native Beth had different
quality because used help
“Because it’s going to have 50 pages of text that I have no
desire whatsoever to read about something that I use
rarely, and I don’t really care to know. I don’t read
instruction manuals, generally. And why would I Google
it? I wouldn’t, because it’s a bunch of teenagers who
can’t spell right, who don’t use punctuation, all lower
case,” answers Lucy.
Introduction Study Design Motivation Observations Future Work Conclusions 72 of 45
75. Intrinsics
Correlations
Relationship To
Stumble Fall +
Age + Extrinsics
Digital Native - Relationship To
Interest -
Age Persist +
Perceived Competence -
Digital Native Age - Digital Native -
External Regulation - Amotivation -
Interest/Enjoyment + External Perceived -
Perceived Choice + Regulation Choice
Perceived Competence +
Age External Regulation +
Perceived Competence -
Perceived Fall -
Competence Interest/Enjoyment +
External -
Regulation Perceived Choice More
Introduction Study Design Motivation Observations Future Work Conclusions 75 of 45
76. Test Effects
• Lowering of emotions
• Learning without any teaching
– “do it” = “you CAN do it”
– Expect researcher to fix any problems
• Performance hindrances
– Age (Mike, Wilma, Walter, Marsha)
– Eye strain (Walter, Wilma)
– Tiredness (Lucy, Miranda, Wilma, Walter)
– Distraction (Molly’s daughter, Walter)
Introduction Study Design Motivation Observations Future Work Conclusions 76 of 45
77. Test Effects: Stress
Max Stress Self Rating of Participant by intrinsic (left) and extrinsic (right) on
a scale of 1 to 10 with 10 being high and 1 is low. Ordered from low stress to
high for both groups.
Intrinsics max stress JE Users max stress
Beth 1 Lilly 2
Jane 1 Mary Ann 4
Rebecca 1 Molly 5
Roger 1 Olivia 5
Mike 2 Walter 5
Peter 6 Alice 6
Wilma 10 Lucy 7
Marsha 8
Miranda 10
Introduction Study Design Motivation Observations Future Work Conclusions 77 of 45
78. Proposed Solutions
• Limit unfamiliar tasks, software, or systems
(impractical)
• Teach big picture patterns and how they relate
from one situation to another
• Teach visual and vocabulary tools
• Give a sense of belonging
• Generate interest and choice
Introduction Study Design Motivation Observations Future Work Conclusions 78 of 45
79. JE User vs. Intrinsic: Walter & Mike
Similarities:
1. Same age Differences:
2. Both retired professors 1. Performance
3. Both persisting 2. Different motivation style
4. Similar competence
5. Similar experience level
6. Both agitated but say they are “fine” More
Introduction Study Design Motivation Observations Future Work Conclusions 79 of 45
80. Weaknesses
1. Sample was a convenient sample
2. Ordinal Likert scale results should not be
averaged
3. All participants had different tasks so they
are not easily comparable
4. Sample size was small
5. Pros and cons of qualitative ethnographic
techniques
6. Did not measure the quantity, rate, and type
of task success
Introduction Study Design Motivation Observations Future Work Conclusions 80 of 45
81. What could have been…
1. Randomized respondent selection for motivation inventory to
get an evenly distributed sample
2. Standardized tasks assigned to measure rate and type of
stumbling and success
3. Standardized unfamiliar and familiar system and software
4. Give written instructions instead of verbal
5. Keep researcher ignorant of motivation scores before
observations
6. Participant alone in a room with the observer outside the room
7. Possibly observing through one way glass or video camera and
screen capture
8. Eliminating researcher interaction with participants
Introduction Study Design Motivation Observations Future Work Conclusions 81 of 45
82. So Much Data…
Could be re-analyzed with other emphases
• Digital literacy
• Communication patterns
• Misinformation or ignorance of a novice
• Attitudes to life long learning
• Attitudes of a “refuser”
• And more…
Introduction Study Design Motivation Observations Future Work Conclusions 82 of 45
83. Bigger Sample
• What percent are extrinsic? Intrinsic?
• What percent are Low-Low or High-High?
• How to characterize Low-Low or High-High?
• What percent are digital natives and non-
natives?
• Do age, perceived competence, or being
digital native hold no difference across
intrinsic and extrinsic?
Introduction Study Design Motivation Observations Future Work Conclusions 83 of 45
84. The Flaws Were Also Strengths
• Rich and diverse insights into identifying JE users
• Diverse population of daily proficient users
• Successfully quantified failure
• Likert scale average is standard in Social Science
• Captured individual proficiency and tested
unfamiliar tasks, software, & system
• Captured motivation style
Introduction Study Design Motivation Observations Future Work Conclusions 84 of 45
85. Other Questions
• “Just Enough” term?
• Gender, socioeconomic status, years of
experience, aversion to change?
• Separating work and play in motivation study
• Less frequent users?
• What if a “consequence” element?
• Hand held computers?
• Food and sleep deprived?
Introduction Study Design Motivation Observations Future Work Conclusions 85 of 45
86. Statistical Analysis of Inventory
Do 3 assumptions hold for n=66 Respondents and
also for n=16 Participants?
Statistical Analysis Inventory Factors
PARAMETRIC: Passes 3 assumptions 1.external regulation
for parametric analysis 2.interest/enjoyment
3.perceived choice
4.identified regulation
NON-PARAMETRIC: Must be non- 1.amotivation
parametrically analyzed 2.perceived competence
3.age
4.digital native
More information and detail in Supplementary Slides More
Introduction Study Design Motivation Observations Future Work Conclusions 86 of 45
87. Significant Differences
between extrinsic and intrinsic
Extrinsics and Intrinsics Have Significant Differences in Phase 2,
Phase 3, and total occurrences.
Phase Stumble Fall Persist Quit
2 Different Different Not Significant Different
(p=0.003) (p=0.003) (p=0.127) (p=0.041)
3 Different Different Different Not Significant
(p=0.018) (p=0.025) (p=0.023) (p=0.470)
Both Different Different Different Different
Phases (p=0.004) (p=0.005) (p=0.030) (p=0.014)
Introduction Study Design Motivation Observations Future Work Conclusions 87 of 45
88. Task FAMILIAR Difference in UNFAMILIAR UNFAMILIAR
GMAIL webmail Work Flow Obscure Company webmail GMX webmail
compose click “compose” button, top obscure is more difficult to see click pencil/paper icon, top click “compose mail” button,
mail (Fig. left, contrasting color compose but all compose are in left between other icons, top left, same color
6) same area of screen same color, no words
open click “inbox” word on left NO DIFFERENCE click “inbox” word on left top, click “inbox” word on left top,
inbox top, same color same color same color
read mail click “[name of sender or Gmail replaces center panel, click “[name of sender or click “[name of sender or
participant]” of mail in others open side by side with participant]” of mail in center participant]” of mail in left half
center large panel, same inbox list either below or to the top half panel, same color, of center panel, same color,
color, opens by replacing right opens in bottom half of opens in right half of center
same center window center panel panel
reply to click “arrow” icon button on Gmail has two places, both click “reply” button with click “reply” button with
mail right at top of what reading, same color, one is word, one is picture and word, top icon picture and word, top icon
same color, no word, OR icon, one or both can disappear bar, first of 9 buttons with bar, 2nd of 7 buttons with
gray “reply” word link at with medium and bigger emails, words, same color words, same color
bottom, same color in floats on top of mail view so not
separate white box. NOTE: always visible. Both gmx and
if email is medium to large, obscure have one step, button
“arrow” icon button with word and icon, always
disappears into the header visible
and the second choice
disappears into the footer
forward click “drop down” arrow on Gmail has two places, one click “forward” button with click “forward” button with
mail “arrow” for reply to see requires two steps (select from picture and word, top icon picture and word, top icon
more options, same color, drop down), both ways are bar, 3rd of 9 buttons with bar, 3rd of 7 buttons with
then select “forward in drop same color, one is word, one is words, same color words, same color
down menu”, all on center icon, one or both can disappear
right at top of email with medium and bigger emails,
reading, or click gray on floats on top of mail view so not
white words in white box at always visible. Both gmx and
bottom (often not visible if obscure have one step, button
reading anything other than with word and icon, always
shortest email) NOTE: visible, both at top center area
same as for reply 88 of 45
89. Phase 2 & 3 No Difference
There was no significant different between Unfamiliar
Task compared to Near Skill Transfer for either
intrinsics or extrinsics.
Stumble Fall Persist Quit Resist
Extrinsic (p=0.370) (p=0.147) (p=0.738) (p=0.056) (p=0.494)
Intrinsic (p=0.784) (p=0.872) (p=0.317) (p=0.317) (p=0.317)
Introduction Study Design Motivation Observations Future Work Conclusions 89 of 45