The document discusses methods and techniques for analyzing, designing, and evaluating incentivized semantic applications. It describes game theory, mechanism design, and participatory design as methods for analysis and design. For evaluation, it recommends interviews, questionnaires, field experiments, and various types of reviews. It also provides examples of applying these methods in case studies of knowledge sharing platforms.
This document describes steps for designing effective incentive systems through experiments and mechanism design. It discusses analyzing user preferences, formalizing existing rewards, designing simple testable hypotheses, and fine-tuning systems. An experiment with students tagging images found that a "winner takes all" incentive of a single cash prize produced more and higher quality tags than per-click compensation. Next steps involve making the task more realistic and useful for workers to further refine the incentive system.
The document presents 10 guidelines for designing incentivized technology: (1) Make the app usable, (2) enjoyable, (3) promote visibility, (4) be sociable, (5) valuable, (6) explorable, (7) flexible, (8) design participatorily, (9) [guideline not presented], (10) [guideline not presented]. Each guideline is explained, with examples given of how companies like IBM, White House, AOL improved their designs over time to better follow the guidelines. Research on social factors that could influence user behavior is also summarized briefly to motivate the guidelines.
This document presents methods for evaluating the usability and sociability of systems, including usability testing, eye tracking, and standardized evaluation. It summarizes a case study applying these methods to evaluate an existing SEEKDA system and develop a revised prototype. Usability testing identified critical issues and design recommendations. A prototype was created and evaluated through an online participatory design workshop with users. The document concludes by describing how explorations and user walkthroughs can be used.
The document discusses human motivation and incentives for contributing to Web 2.0 platforms. It covers theories of motivation like need theories, job characteristics approach, and reinforcement theory. It also discusses intrinsic and extrinsic motivations as well as designing incentive systems using game theory and mechanism design. A case study is presented on designing incentives for semantic annotation at a research organization through workshops, interviews, and a lab experiment comparing incentive systems.
The document discusses methods and techniques for analyzing, designing, and evaluating incentivized semantic applications. It describes game theory, mechanism design, and participatory design as methods for analysis and design. For evaluation, it recommends interviews, questionnaires, field experiments, and various types of reviews. It also provides examples of case studies where these methods have been applied, including knowledge sharing platforms and virtual games.
This document outlines a tutorial on motivating users to contribute semantic content through incentives and game mechanics. It discusses analyzing incentive structures and designing semantic applications to harness human intelligence. The tutorial covers human contributions to semantics, methods for incentivized application analysis and design, guidelines for technology that utilizes incentives, and examples of semantic annotation games. The goal is realizing the Semantic Web by encouraging widespread end-user generation of semantic content.
How to kickstart your co-creation platform - 20 examples by @boardofinnoBoard of Innovation
This document summarizes 20 existing co-creation platforms. It describes different types of co-creation such as clubs of experts, crowds of people, coalitions of parties, and communities of kindred spirits. Key principles of successful co-creation are inspiring participation, selecting the best ideas and people, connecting creative minds, sharing results, and continuing development. Platforms are compared based on parameters like number of people involved, frequency of interaction, competition level, project duration, and return for participants. The goal is to learn from existing examples of co-creation between industries, governments, and consumers.
Crowdsourcing involves outsourcing tasks traditionally performed by employees to a large, undefined group of people through an open call. It has been used successfully by companies like Netflix, whose Netflix Prize competition improved their recommendation algorithm, and Cisco, whose I-Prize contest generated new business ideas. However, crowdsourcing also carries risks like producing useless or wasteful results if not implemented properly.
This document describes steps for designing effective incentive systems through experiments and mechanism design. It discusses analyzing user preferences, formalizing existing rewards, designing simple testable hypotheses, and fine-tuning systems. An experiment with students tagging images found that a "winner takes all" incentive of a single cash prize produced more and higher quality tags than per-click compensation. Next steps involve making the task more realistic and useful for workers to further refine the incentive system.
The document presents 10 guidelines for designing incentivized technology: (1) Make the app usable, (2) enjoyable, (3) promote visibility, (4) be sociable, (5) valuable, (6) explorable, (7) flexible, (8) design participatorily, (9) [guideline not presented], (10) [guideline not presented]. Each guideline is explained, with examples given of how companies like IBM, White House, AOL improved their designs over time to better follow the guidelines. Research on social factors that could influence user behavior is also summarized briefly to motivate the guidelines.
This document presents methods for evaluating the usability and sociability of systems, including usability testing, eye tracking, and standardized evaluation. It summarizes a case study applying these methods to evaluate an existing SEEKDA system and develop a revised prototype. Usability testing identified critical issues and design recommendations. A prototype was created and evaluated through an online participatory design workshop with users. The document concludes by describing how explorations and user walkthroughs can be used.
The document discusses human motivation and incentives for contributing to Web 2.0 platforms. It covers theories of motivation like need theories, job characteristics approach, and reinforcement theory. It also discusses intrinsic and extrinsic motivations as well as designing incentive systems using game theory and mechanism design. A case study is presented on designing incentives for semantic annotation at a research organization through workshops, interviews, and a lab experiment comparing incentive systems.
The document discusses methods and techniques for analyzing, designing, and evaluating incentivized semantic applications. It describes game theory, mechanism design, and participatory design as methods for analysis and design. For evaluation, it recommends interviews, questionnaires, field experiments, and various types of reviews. It also provides examples of case studies where these methods have been applied, including knowledge sharing platforms and virtual games.
This document outlines a tutorial on motivating users to contribute semantic content through incentives and game mechanics. It discusses analyzing incentive structures and designing semantic applications to harness human intelligence. The tutorial covers human contributions to semantics, methods for incentivized application analysis and design, guidelines for technology that utilizes incentives, and examples of semantic annotation games. The goal is realizing the Semantic Web by encouraging widespread end-user generation of semantic content.
How to kickstart your co-creation platform - 20 examples by @boardofinnoBoard of Innovation
This document summarizes 20 existing co-creation platforms. It describes different types of co-creation such as clubs of experts, crowds of people, coalitions of parties, and communities of kindred spirits. Key principles of successful co-creation are inspiring participation, selecting the best ideas and people, connecting creative minds, sharing results, and continuing development. Platforms are compared based on parameters like number of people involved, frequency of interaction, competition level, project duration, and return for participants. The goal is to learn from existing examples of co-creation between industries, governments, and consumers.
Crowdsourcing involves outsourcing tasks traditionally performed by employees to a large, undefined group of people through an open call. It has been used successfully by companies like Netflix, whose Netflix Prize competition improved their recommendation algorithm, and Cisco, whose I-Prize contest generated new business ideas. However, crowdsourcing also carries risks like producing useless or wasteful results if not implemented properly.
Synthesising JISC Institutional InnovationGeorge Roberts
This document summarizes a program synthesizing various projects related to lifelong learning and workforce development. It provides themes that have emerged from the projects, including open educational dialogue, technical enabling practices, and social and cultural enabling practices. It also discusses top-down themes from funders and how to analyze the projects, emergent themes, and given themes using a 3D matrix. Next steps discussed include using a 2-level synthesis focusing on projects and the wider program/context and considering tools, rules, communities, and roles within the system.
Building the PoliMedia search system; data- and user-drivenMaxKemman
Presentation at eHumanities group at Meerten's Institute (Amsterdam) on Thursday 18 April 2013.
Analysing media coverage across several types of media-outlets is a challenging task for (media) historians. A specific example of media coverage research investigates the coverage of political debates and how the representation of topics and people change over time. The PoliMedia project (http://www.polimedia.nl) aims to showcase the potential of cross-media analysis for research in the humanities, by 1) curating automatically detected semantic links between four data sets of different media types, and 2) developing a demonstrator application that allows researchers to deploy such an interlinked collection for quantitative and qualitative analysis of media coverage of debates in the Dutch parliament.
These two goals reflect the two perspectives on the development of a search system such as PoliMedia; data- and user-driven. In this presentation, Laura Hollink (VU) will present the data-driven perspective of linking between different datasets and the research questions that arise in achieving this linkage: how to combine different types of datasets and what kind of research questions are made possible by the data? Max Kemman (EUR) will present the user-driven perspective: which benefits can scholars have from linking of these datasets? What are the user requirements for the PoliMedia search system and how was the system evaluated with scholars in an eye tracking study?
(1) The document presents research on evaluating the quality of mobile learning activities (LAs) using tablets in science education in Europe compared to traditional teaching methods.
(2) It describes a methodology using multiple criteria decision analysis and fuzzy methods to evaluate LAs based on criteria like flexibility, engagement, interaction, and teaching methods.
(3) The results found that mobile LAs based on problem-solving, personalization, collaboration and flipped classroom scored higher in quality (82.8%) than traditional LAs(50%), and better suited different learning styles like activists and pragmatists.
- The document provides an overview of Amy Barnes' background and philosophy regarding social media. She views social media as one part of an overall communications plan and sees her skills lying at the intersection of strategy, planning, content creation, and community management. She considers good social media to provide value to audiences through contextually relevant communications aligned with goals and objectives. As a deliverable, she can provide strategic planning documents, research, messaging strategies, and analysis and optimization. Success is measured through organizational impact, thought leadership, and recognition of excellence. The document also provides an initial impression of the University of Michigan's existing strong social media presence and opportunities to apply learnings from word-of-mouth marketing and anthropology.
This document provides an overview of the Information Systems Analysis and Modeling module. The module aims to help students understand tools and techniques used in information systems development, analyze and design systems, and evaluate methodologies. It covers topics such as requirements determination, object-oriented analysis, conceptual and physical design, and comparing methodologies. Students will be assessed through an exam, critiques of readings, and a requirements analysis assignment. Reading materials include textbooks on information systems development and object-oriented software engineering.
This document discusses socio-organizational issues and stakeholder requirements in systems design. It describes how organizational factors can impact a system's acceptance and success. Stakeholders must be identified, including their needs within the organizational context. Socio-technical models examine both human and technical requirements, while soft systems methodology takes a broader view of organizational issues. Participatory design involves users directly in the design process, and ethnographic methods study users unbiasedly in their work context.
This document discusses socio-organizational issues and stakeholder requirements in systems design. It describes how organizational factors can impact system acceptance and the need to identify stakeholder requirements within the organizational context. Several socio-technical models are presented for gathering human and technical requirements, including soft systems methodology, participatory design, and ethnographic methods to understand users and contexts. Stakeholders beyond direct users must be considered in the design process.
This document discusses a project called INSEMTIVES that aims to increase user motivation for semantic content creation. The project will analyze semantic content authoring tasks, identify where human input is most valuable, define incentive models, and develop a methodology for semantic content creation that incorporates incentives. The work plan involves tasks analyzing content processes, developing the methodology, and defining incentive models over 36 months. Research methods will include literature reviews, usability testing, interviews and workshops. Findings from case studies on existing tools will also inform the models and guidelines.
User Required? On the Value of User Research in the Digital HumanitiesMaxKemman
This document discusses the value of user research in developing digital tools for humanities research. It describes user research conducted for two tools: PoliMedia, which links Dutch parliamentary debates to media items, and Oral History Today, a search interface for oral histories. The research identified user requirements for both tools, though some requirements were deemed out of scope. Common requirements included searching by time period and names/roles of people. The discussion concludes that while generalizing requirements is difficult, user research helps ensure tools are usable and support researchers' broader workflows.
HUman computer Interaction Socio-organizational Issues.pptJayaprasanna4
This document discusses socio-organizational issues and stakeholder requirements in systems design. It covers topics such as organizational conflicts that can impact system acceptance, identifying stakeholder needs in context, and socio-technical models to understand human and technical requirements. Methodologies covered include soft systems methodology to take a broader view, participatory design to involve users, and ethnographic research to study users unbiasedly.
HUman computer Interaction Socio-organizational Issues.pptJayaprasanna4
This document discusses socio-organizational issues and stakeholder requirements in systems design. It covers topics such as organizational conflicts that can impact system acceptance, identifying stakeholder needs in context, and socio-technical models to understand human and technical requirements. Methodologies covered include soft systems methodology to take a broader view, participatory design to involve users, and ethnographic research to study users unbiasedly.
The SmartZoos project aims to integrate zoos in the Central Baltic region by developing and implementing a cross-border service package for creative adventure learning using mobile devices. The project will design, develop, pilot, and market this service package, which includes an online tool for creating location-based interactive assignments and a mobile application for guiding learners through the zoos. This is intended to form a joint tourist attraction across the participating zoos in Estonia, Finland, and Sweden.
Strijker & Fisser (2019-06-26) A new curriculum for the netherlands including...Saxion
In 2018 the Netherlands started the development of a new curriculum framework for primary, lower and upper secondary education. New themes in curriculum are Digital Literacy, citizenship and a strong focus on 21st century skills. Digital Literacy is defined as a combination of ICT skills, media literacy, information literacy and Computational Thinking. Starting with a vision on a theme such as Digital Literacy and using this vision as a starting point for describing big ideas. For Digital Literacy eight big ideas were described: data and information, safety and privacy, using and controlling, communication and cooperation, digital citizenship, digital economy, applying and designing, and sustainability. Computational Thinking is integrated in each of these big ideas. The underlying framework for Computational Thinking is based on ISTE and CSTA and each big idea is specified as a learning trajectory describing learning objectives leading to a new legal curriculum framework in 2022.
A hands-on approach to digital tool criticism: Tools for (self-)reflectionMarijn Koolen
Digital tool criticism is a recent and important discussion in Digital Humanities research. We define digital tool criticism as the reflection on the role of digital tools in the research methodology and the evaluation of the suitability of a given digital tool for a specific research goal. The aim is to understand the impact of any limitation of the tool on the specific goal, not to improve a tool’s performance. That is, ensuring as a scholar to be aware of the impact of a tool on research design, methods, interpretations and outcomes. Our goal with developing digital tool criticism as a method is to help scholars better understand how research methods, tools and activities shape our interpretations. Based on our experiences with two hands-on workshops on digital tool criticism, we find that reflection on using digital tools and data in all phases of the research process is key.
Reflection urges scholars to consider digital data and tools as part of the overall research goals and design, and interdependent with other elements of research design, namely research questions and methods. As scholars go through their research process, assumptions on the research design and the connection between tools, data and questions are constantly challenged, forcing updates in the design and the interpretation of data and question.
- The document discusses degrees of openness in massive open online courses (MOOCs), including definitions of openness in cMOOCs and xMOOCs.
- It provides an overview of the openHPI platform, including technical details, course formats, and insights into developing an engaged learning community.
- Guidelines are presented for cultivating a culture of participation based on theories of meta-design, including supporting different engagement levels, human-problem interaction, emergent behaviors, and co-evolution of the community and resources.
This document provides instructions for a group project on designing an intelligent interface system. Students will work in groups of up to 5 people to identify a problem and design an innovative technology or system solution. They will illustrate their idea through interface design using prototyping tools and present their design. The project aims to stimulate creativity and problem solving. It covers tasks like conducting research, designing the interface, and disseminating the idea in a report. The document provides background on creativity, AI models, and expected results. It also includes the project format, submission instructions, assessment criteria and due date. Groups will be judged on criteria like how well their design achieves its mission and communicates. The best designs will receive higher marks.
This document provides an agenda and overview for a two-day workshop on leading research and innovation in expert organisations. Day one will cover topics such as culture and leadership, governance models, examples from other organisations, and new ways of working supported by social media. Day two focuses on managing innovation from idea to implementation, portfolio management, reward systems, and stakeholder relationships. Examples of innovative practices from companies like IBM, Google and Lego will be presented, as well as examples from universities and open source communities. The workshop aims to provide participants with tools and examples for enhancing innovation in research centres, governments and corporations.
Innovation Platforms for increasing impact of research in Mozambique & IndiaFood_Systems_Innovation
Michaela Cosijn (CSIRO) presentation to the 'John Dillon Fellows' Workshop in Canberra in March 2015 & the 'Australian Award Fellowship' in Sydney in May 2015 on how successful collaborations and partnerships using innovation platforms can increase the impact of research.
Synthesising JISC Institutional InnovationGeorge Roberts
This document summarizes a program synthesizing various projects related to lifelong learning and workforce development. It provides themes that have emerged from the projects, including open educational dialogue, technical enabling practices, and social and cultural enabling practices. It also discusses top-down themes from funders and how to analyze the projects, emergent themes, and given themes using a 3D matrix. Next steps discussed include using a 2-level synthesis focusing on projects and the wider program/context and considering tools, rules, communities, and roles within the system.
Building the PoliMedia search system; data- and user-drivenMaxKemman
Presentation at eHumanities group at Meerten's Institute (Amsterdam) on Thursday 18 April 2013.
Analysing media coverage across several types of media-outlets is a challenging task for (media) historians. A specific example of media coverage research investigates the coverage of political debates and how the representation of topics and people change over time. The PoliMedia project (http://www.polimedia.nl) aims to showcase the potential of cross-media analysis for research in the humanities, by 1) curating automatically detected semantic links between four data sets of different media types, and 2) developing a demonstrator application that allows researchers to deploy such an interlinked collection for quantitative and qualitative analysis of media coverage of debates in the Dutch parliament.
These two goals reflect the two perspectives on the development of a search system such as PoliMedia; data- and user-driven. In this presentation, Laura Hollink (VU) will present the data-driven perspective of linking between different datasets and the research questions that arise in achieving this linkage: how to combine different types of datasets and what kind of research questions are made possible by the data? Max Kemman (EUR) will present the user-driven perspective: which benefits can scholars have from linking of these datasets? What are the user requirements for the PoliMedia search system and how was the system evaluated with scholars in an eye tracking study?
(1) The document presents research on evaluating the quality of mobile learning activities (LAs) using tablets in science education in Europe compared to traditional teaching methods.
(2) It describes a methodology using multiple criteria decision analysis and fuzzy methods to evaluate LAs based on criteria like flexibility, engagement, interaction, and teaching methods.
(3) The results found that mobile LAs based on problem-solving, personalization, collaboration and flipped classroom scored higher in quality (82.8%) than traditional LAs(50%), and better suited different learning styles like activists and pragmatists.
- The document provides an overview of Amy Barnes' background and philosophy regarding social media. She views social media as one part of an overall communications plan and sees her skills lying at the intersection of strategy, planning, content creation, and community management. She considers good social media to provide value to audiences through contextually relevant communications aligned with goals and objectives. As a deliverable, she can provide strategic planning documents, research, messaging strategies, and analysis and optimization. Success is measured through organizational impact, thought leadership, and recognition of excellence. The document also provides an initial impression of the University of Michigan's existing strong social media presence and opportunities to apply learnings from word-of-mouth marketing and anthropology.
This document provides an overview of the Information Systems Analysis and Modeling module. The module aims to help students understand tools and techniques used in information systems development, analyze and design systems, and evaluate methodologies. It covers topics such as requirements determination, object-oriented analysis, conceptual and physical design, and comparing methodologies. Students will be assessed through an exam, critiques of readings, and a requirements analysis assignment. Reading materials include textbooks on information systems development and object-oriented software engineering.
This document discusses socio-organizational issues and stakeholder requirements in systems design. It describes how organizational factors can impact a system's acceptance and success. Stakeholders must be identified, including their needs within the organizational context. Socio-technical models examine both human and technical requirements, while soft systems methodology takes a broader view of organizational issues. Participatory design involves users directly in the design process, and ethnographic methods study users unbiasedly in their work context.
This document discusses socio-organizational issues and stakeholder requirements in systems design. It describes how organizational factors can impact system acceptance and the need to identify stakeholder requirements within the organizational context. Several socio-technical models are presented for gathering human and technical requirements, including soft systems methodology, participatory design, and ethnographic methods to understand users and contexts. Stakeholders beyond direct users must be considered in the design process.
This document discusses a project called INSEMTIVES that aims to increase user motivation for semantic content creation. The project will analyze semantic content authoring tasks, identify where human input is most valuable, define incentive models, and develop a methodology for semantic content creation that incorporates incentives. The work plan involves tasks analyzing content processes, developing the methodology, and defining incentive models over 36 months. Research methods will include literature reviews, usability testing, interviews and workshops. Findings from case studies on existing tools will also inform the models and guidelines.
User Required? On the Value of User Research in the Digital HumanitiesMaxKemman
This document discusses the value of user research in developing digital tools for humanities research. It describes user research conducted for two tools: PoliMedia, which links Dutch parliamentary debates to media items, and Oral History Today, a search interface for oral histories. The research identified user requirements for both tools, though some requirements were deemed out of scope. Common requirements included searching by time period and names/roles of people. The discussion concludes that while generalizing requirements is difficult, user research helps ensure tools are usable and support researchers' broader workflows.
HUman computer Interaction Socio-organizational Issues.pptJayaprasanna4
This document discusses socio-organizational issues and stakeholder requirements in systems design. It covers topics such as organizational conflicts that can impact system acceptance, identifying stakeholder needs in context, and socio-technical models to understand human and technical requirements. Methodologies covered include soft systems methodology to take a broader view, participatory design to involve users, and ethnographic research to study users unbiasedly.
HUman computer Interaction Socio-organizational Issues.pptJayaprasanna4
This document discusses socio-organizational issues and stakeholder requirements in systems design. It covers topics such as organizational conflicts that can impact system acceptance, identifying stakeholder needs in context, and socio-technical models to understand human and technical requirements. Methodologies covered include soft systems methodology to take a broader view, participatory design to involve users, and ethnographic research to study users unbiasedly.
The SmartZoos project aims to integrate zoos in the Central Baltic region by developing and implementing a cross-border service package for creative adventure learning using mobile devices. The project will design, develop, pilot, and market this service package, which includes an online tool for creating location-based interactive assignments and a mobile application for guiding learners through the zoos. This is intended to form a joint tourist attraction across the participating zoos in Estonia, Finland, and Sweden.
Strijker & Fisser (2019-06-26) A new curriculum for the netherlands including...Saxion
In 2018 the Netherlands started the development of a new curriculum framework for primary, lower and upper secondary education. New themes in curriculum are Digital Literacy, citizenship and a strong focus on 21st century skills. Digital Literacy is defined as a combination of ICT skills, media literacy, information literacy and Computational Thinking. Starting with a vision on a theme such as Digital Literacy and using this vision as a starting point for describing big ideas. For Digital Literacy eight big ideas were described: data and information, safety and privacy, using and controlling, communication and cooperation, digital citizenship, digital economy, applying and designing, and sustainability. Computational Thinking is integrated in each of these big ideas. The underlying framework for Computational Thinking is based on ISTE and CSTA and each big idea is specified as a learning trajectory describing learning objectives leading to a new legal curriculum framework in 2022.
A hands-on approach to digital tool criticism: Tools for (self-)reflectionMarijn Koolen
Digital tool criticism is a recent and important discussion in Digital Humanities research. We define digital tool criticism as the reflection on the role of digital tools in the research methodology and the evaluation of the suitability of a given digital tool for a specific research goal. The aim is to understand the impact of any limitation of the tool on the specific goal, not to improve a tool’s performance. That is, ensuring as a scholar to be aware of the impact of a tool on research design, methods, interpretations and outcomes. Our goal with developing digital tool criticism as a method is to help scholars better understand how research methods, tools and activities shape our interpretations. Based on our experiences with two hands-on workshops on digital tool criticism, we find that reflection on using digital tools and data in all phases of the research process is key.
Reflection urges scholars to consider digital data and tools as part of the overall research goals and design, and interdependent with other elements of research design, namely research questions and methods. As scholars go through their research process, assumptions on the research design and the connection between tools, data and questions are constantly challenged, forcing updates in the design and the interpretation of data and question.
- The document discusses degrees of openness in massive open online courses (MOOCs), including definitions of openness in cMOOCs and xMOOCs.
- It provides an overview of the openHPI platform, including technical details, course formats, and insights into developing an engaged learning community.
- Guidelines are presented for cultivating a culture of participation based on theories of meta-design, including supporting different engagement levels, human-problem interaction, emergent behaviors, and co-evolution of the community and resources.
This document provides instructions for a group project on designing an intelligent interface system. Students will work in groups of up to 5 people to identify a problem and design an innovative technology or system solution. They will illustrate their idea through interface design using prototyping tools and present their design. The project aims to stimulate creativity and problem solving. It covers tasks like conducting research, designing the interface, and disseminating the idea in a report. The document provides background on creativity, AI models, and expected results. It also includes the project format, submission instructions, assessment criteria and due date. Groups will be judged on criteria like how well their design achieves its mission and communicates. The best designs will receive higher marks.
This document provides an agenda and overview for a two-day workshop on leading research and innovation in expert organisations. Day one will cover topics such as culture and leadership, governance models, examples from other organisations, and new ways of working supported by social media. Day two focuses on managing innovation from idea to implementation, portfolio management, reward systems, and stakeholder relationships. Examples of innovative practices from companies like IBM, Google and Lego will be presented, as well as examples from universities and open source communities. The workshop aims to provide participants with tools and examples for enhancing innovation in research centres, governments and corporations.
Innovation Platforms for increasing impact of research in Mozambique & IndiaFood_Systems_Innovation
Michaela Cosijn (CSIRO) presentation to the 'John Dillon Fellows' Workshop in Canberra in March 2015 & the 'Australian Award Fellowship' in Sydney in May 2015 on how successful collaborations and partnerships using innovation platforms can increase the impact of research.
Innovation Platforms for increasing impact of research in Mozambique & India
Methods and techniques-comm (3/6)
1. Methods and techniques to
analyze and design incentivized
semantic applications
www.insemtives.eu
Roberta Cuel, University of Trento, IT and
Markus Rohde, University of Siegen, DE
ISWC 2010
2. 1/29/2015 www.insemtives.eu 2
Why is my app not as successful as
Facebook…
1/29/2015 www.insemtives.eu 2
• Motivation and incentives
– Reciprocity
– Reputation
– Competition
– Altruism
– Self-esteem
– Fun
– Money
• This session is about the methods
and techniques you can use
– To study your scenario prior to
application design
– To evaluate your incentives strategy
and adjust your application
3. Overview
• Basic notions and definitions
• Methods and techniques for analysis, design
and evaluation
• Case studies
1/29/2015 www.insemtives.eu 3
4. BASIC NOTIONS AND DEFINITIONS
Basic notions and definitions
Methods and techniques for analysis, design and evaluation
Case studies
1/29/2015 www.insemtives.eu 4
5. Starting with motivation…
• Basic tenets of organizational behavior
– Performance : f(ability*motivation)
1/29/2015 www.insemtives.eu 5
6. Motivation and incentives
• Intrinsic vs extrinsic motivations
• Incentives are ‘rewards’ assigned to performer
to make him/her commit
– Can be totally uncorrelated to the nature of the
task
– Need to be compliant with the values and beliefs
of the acting person
1/29/2015 www.insemtives.eu 61/29/2015 www.insemtives.eu 6
7. Example: DBPedia
The DBpedia community collaborates using
the following tools
– Mailing lists, Facebook group
– Blog, quality assurance and bug tracking
– Development
• “The framework is easily extended and we offer the possibility to do this
yourself, increasing the likelihood of your desired data being included
in the next DBpedia release.”
– DBpedia consulting (money)
1/29/2015 www.insemtives.eu 7
8. 1/29/2015 www.insemtives.eu 8
Typology of motivations
Motivations Internal
(embedded in structure,
e.g., task, tools)
External
(additional to structure,
external re-inforcements)
Intrinsic
(predispositioned in
person, e.g., drives,
needs, desires )
Fun, joy, gaming,
interest, satisfaction,
self-actualization, self-re-
inforcement
Social appreciation,
reputation, love, trust,
social capital, community
support
Extrinsic
(additional to personal
predispositions, extern re-
inforcements )
Usability, sociability,
Design-for-fun, curiosity,
community-building
support
Material/financial capital,
money, rewards, prices,
medals, credit points
Structure
Person
Example: FLOSS software (Ghosh & Prakash, in Lerner & Tirole, 2005)
9. METHODS AND TECHNIQUES FOR
ANALYSIS, DESIGN AND EVALUATION
Basic notions and definitions
Methods and techniques for analysis, design and evaluation
Case studies
1/29/2015 www.insemtives.eu 9
10. Overview of methods and
techniques
• To analyze your scenario and design your
application
– Game theory, mechanism design
– Participatory design and end-user development
• To evaluate an incentives strategy in its context
– Interviews and questionnaires and field experiments
– Heuristic evaluation, guideline reviews
– Cognitive and pluralistic walk-throughs
– Participatory/User evaluation
1/29/2015 www.insemtives.eu 101/29/2015 www.insemtives.eu 10
11. Game theory
• Game theory is a formal way to analyze interaction among a
number of rational agents who behave strategically
– The rational agents: players involved in the situation (best choice)
– A number of players: more than one
– Rationality/payoffs: what are the players’ preferences over the
outcomes of the game
– The interactions: one player’s behavior affects another
– The rules: who moves when, what do they know, what can they do
– The outcomes: what is the outcome of the game (for each move)
1/29/2015 www.insemtives.eu 11
12. Mechanism design
• Mechanism design is about how to translate game
theory in effective behavior
– To design rules such that a desired set of outcomes
happens
– Alignment of interests between parties and production of
maximum social welfare
• Relevant variables
– Structural and personal motivations
– Goal and nature of good produced
– Tasks and skills required
– Social structure
1/29/2015 www.insemtives.eu 12
13. Analysis matrix
• A multidimensional tool that allows for an
analysis of a particular task with respect to the
variables relevant for mechanism design
1/29/2015 www.insemtives.eu 13
Goal Tasks Social structure Nature of good
produced
Skill
variety/level
Communication
level
Variety of Hierarchy-
neutral
Private good Skilled ability
Participation
level
Specificity of Hierarchical Public good Competence
Clearness level Identification
with
Common
resource
Club good
14. 1/29/2015 www.insemtives.eu 14
The matrix in practice: semantic
annotation
• Goal of the annotation or ontology population exercise
• Task, or more typically, an ordered collection of tasks into
which the annotation exercise can be broken down
• Social structure, a stylized and simplified set of social
relationships among the subjects participating in the
exercise
• Nature of good, a stylized description, in game-theoretical
terms, of the relationship between what good is produced
and who consumes it
• Required skills of the agents to complete the annotation
task
1/29/2015 www.insemtives.eu 14
15. The matrix in practice: crowd-sourcing
of ontology evolution
• Goal: communication and participation level
– Coordination (free-open)
• Interests are already aligned (free riding)
• Private and public benefit coincide
• Task: variety vs specificity
• Social structure: hierarchy neutral (large vs small
groups)
• Nature of good: public good non-rival (use does not
limit use of others), non-exclusive (open access)
• Skill variety/level: Skilled ability
1/29/2015 www.insemtives.eu 151/29/2015 www.insemtives.eu 15
16. Participatory Design (PD)
• To develop software for the participation of end-users, we
propose a participatory way of designing these software
tools, integrating potential users by participatory design
methods
• Participatory Design
– Improvement of the participation of workers in software development
processes
– The cooperation between software developers and end-users
– Participate in IT development projects as experts of their own work
1/29/2015 www.insemtives.eu 16
17. 1/29/2015 www.insemtives.eu 17
Methods for PD
• Participatory-design projects combine
– Design-by-doing methods
– PD workshops
– Scenarios
– Different forms of prototyping (mockups, rapid
prototypes)
– Work organization games
– Ethnographic methods
18. 1/29/2015 www.insemtives.eu 18
End-User Development (EUD)
• EUD is about designing highly flexible systems that enable
users to participate during the use of the system by
adapting and modifying the tools according to their
needs/preferences
• Goal: Empower end-users to develop and adapt systems
themselves by designing these systems to be easy to
– Understand
– Learn
– Use
– Teach
19. 1/29/2015 www.insemtives.eu 19
How to design an incentivized
application
• Ideally: field desk lab field
• Analyze the domain and find yourselves in the matrixes
– Find the relevant point of that situation (goal and tasks)
– Focus on a small group of individuals (social structure)
– Analyze their motivation (internal/external intrinsic/extrinsic)
– Analyze the other relevant variables (nature of good being
produced, kill variety/level)
• Design a simplest possible model that can effectively
support contributors
• Test and get feedback
• Fine-tune the experiment and add other elements
21. CASE STUDIES
Basic notions and definitions
Methods and techniques for analysis, design and evaluation
Case studies
1/29/2015 www.insemtives.eu 21
22. Examples from our project
• Telefónica Investigación y Desarrollo (TID - Spain): Knowledge
Intranet Platform
• Pepper’s Ghost (PGP-UK): Virtual Games/Worlds
• Seekda! (Austria): Web Services Search Portal
1/29/2015 www.insemtives.eu 22
23. 1/29/2015 www.insemtives.eu 23
Field work: TID
• Domain analysis
– Site visit, semi-structured, qualitative interviews
• Communication processes
• Existing usage practices and problems
• Existing tools/solutions
• Semantic annotation solutions
– Tape recording, transcription
– Data analysis per ex-post categorization
• Focus group discussion
– Usability lab tests
– Expert walkthroughs
24. 1/29/2015 www.insemtives.eu 24
Field work: TID (2)
1/29/2015 www.insemtives.eu 24
• Find TID in the matrix
– Goal: communication and participation (contribute to the
knowledge portal)
– Social structure (various structures co-exist)
• Strongly hierarchical organization (control is an issue)
• working groups and community of experts
– Nature of good: public good vs. private, club goods
– Skill variety/level: Skilled ability (knowledge workers)
– Motivations: fun, visibility, reputation, promotion, money
25. 1/29/2015 www.insemtives.eu 25
Field work: PGP
• Domain analysis
– Data analysis (Benchmark: Galaxyzoo, Moonzoo)
– Usability lab tests and expert walkthroughs
– Collect data on users such as logs files
• Find PGP in the matrix
– Goal: participation level
– Task: specific
– Social structure: hierarchy neutral
– Nature of good: public good non-rival (use does not limit
use of others), non-exclusive (open access)
– Skill variety/level: non skilled ability
26. 1/29/2015 www.insemtives.eu 26
Field work: SEEKDA
• Domain analysis
– Interviews and workshops
– Replicating Telefonica model
• Find SEEKDA in the matrix:
– Goal: communication and participation level
– Task: variety vs. specificity
– Social structure: hierarchy neutral (mainly)
– Nature of good: public good non-rival (use does not limit
use of others), exclusive, or club good
– Skill variety/level: Skilled ability
Virtual communities have (on average) a huge number of non active users
Gnutella for instance in the 2000 only 25% pax 98% files and 66% users shared nothing … in the 2005 85% shared nothing
50-percent of social, hobby, and work mailing lists had no traffic over 4 months
What are the main characteristics that transform any appl. Into a KILLER one?
What do you need to know before jumping into design?
Both continuous learning and motivations are very relevant to perform better.
Also environment might affect performance and both motivation (crisis vs. development) and capacity (skills and knowledge)
In the real world intrinsic and extrinsic motivation are together and are difficult to analyze as in a theoretical model
2 is ex ante
3 is ex post
Nash and travelling salesman
Google auction, Dutch auction, etc.
Prices start at low reserve prices, and rise whenever there is “over-demand” for an ad-slot — i.e. a slot that is currently held by one bidder is desired by another. Such small price increases keep going on until there is no “over-demand”, at which point the auction closes.
How to transform individual preferences into social decision
Google auctions:
Goal: Efficiency, Revenue of auctioneer, Fairness, Incentive Compatibility: the bid information is given to us by advertisers.
Complications: google charges TV ads NOT according the number of bidders BUT according the number of TV watchers. Long Term Payments Crowd control (a single ad. (or of the same company) cannot appear twice in the same commercial break, )
Long vs. short ads. / auctions overlaps / lack of free disposal is not accepted … develop a bunch of filler ads.
From literature of psychology, social science, fiel experiment, experimental economics, knowledge management, HRM, organization behaviors, cognitive science….
Goal: participation
Task: variety
Social structure: hierarchical vs non
Nature of good: public or private
Required skills: specific …
One of the way to trasfrom into practice mechanism design