The document discusses graduate student perspectives on replicating studies in the field of computer science and artificial intelligence. It summarizes the results of a survey of 93 graduate students which found that 17% had replicated a study before but 83% had not, and 38% said they would replicate a study in the future but 62% said they would not. It explores some of the reasons students gave for not replicating studies, such as it not being seen as creative or novel work, reviewers criticizing submissions for not being novel if they are replications, and there being no reward or incentive structure for replicating previous work.
Better Startups Through Science - MIT Accelerator 2014Colin Kennedy
Exploring how small experiments can build better businesses faster.
- Science, and how it applies
- "Bad" & "Good" science
- Experiments and 'Little Bets'
- Applications in startups
- Where to go next
Presented at MIT Founders' Skills Accelerator program at The Martin Trust Center for MIT Entrepreneurship in July 2014.
Introduction to estimations in project management; role of deception versus delusion; selective perception in estimating; practical recipes.
For more information how to perform schedule risk analysis using RiskyProject software please visit Intaver Institute web site: http://www.intaver.com.
About Intaver Institute.
Intaver Institute Inc. develops project risk management and project risk analysis software. Intaver's flagship product is RiskyProject: project risk management software. RiskyProject integrates with Microsoft Project, Oracle Primavera, other project management software or can run standalone. RiskyProject comes in three configurations: RiskyProject Lite, RiskyProject Professional, and RiskyProject Enterprise.
#ATAGTR2018 Presentation "The Subtle Influence of Cognitive Biases on Testing...Agile Testing Alliance
Prabhakar Panditi who is an enterprise Agile Coach, Executive Coach Lean Agile and Product Development Consult has conducted a Game session on "The Subtle Influence of Cognitive Biases on Testing Professional."
please refer our linkedin post for session details
https://www.linkedin.com/pulse/game-session-prabhaker-panditi-subtle-influence-biases-alliance/
http://verraes.net
The project was of to a bad start: an inherited legacy codebase, a waterfall contract, and a projected loss. The promise of Kaizen or Continuous Improvement seemed very appealing. But when we tried to incorporate this into our process, it didn’t catch on. Biweekly retrospectives didn’t seem to expose any problems we could improve upon. The ceremonies we tried, like Deming’s Plan-Do-Check-Act cycles, added too much overhead. We were doing something wrong.
Continuous Improvement implies that you know exactly where to focus your efforts. Like scientists, we started to experiment, without deciding upfront what we expected the outcome to be. The rules? Make every experiment as small as possible. No meetings, no consensus, no cumbersome evaluation process. We let the results speak for themselves. This talk explores the successes and failures of a team that went from survival mode to learning mode over the course of a year.
Cognitive Behaviour Coaching For Young Leadersrenjmat
Cognitive Behavior Coaching has been a novel and promising mode of corporate training. Cognitive Behavior models were proved clinically as well as using brain Imaging studies. This model has been well adapted and incorporated to corporate psychological training. This programme could surpasses the shortfalls of most of the available corporate training programmes.
Better Startups Through Science - MIT Accelerator 2014Colin Kennedy
Exploring how small experiments can build better businesses faster.
- Science, and how it applies
- "Bad" & "Good" science
- Experiments and 'Little Bets'
- Applications in startups
- Where to go next
Presented at MIT Founders' Skills Accelerator program at The Martin Trust Center for MIT Entrepreneurship in July 2014.
Introduction to estimations in project management; role of deception versus delusion; selective perception in estimating; practical recipes.
For more information how to perform schedule risk analysis using RiskyProject software please visit Intaver Institute web site: http://www.intaver.com.
About Intaver Institute.
Intaver Institute Inc. develops project risk management and project risk analysis software. Intaver's flagship product is RiskyProject: project risk management software. RiskyProject integrates with Microsoft Project, Oracle Primavera, other project management software or can run standalone. RiskyProject comes in three configurations: RiskyProject Lite, RiskyProject Professional, and RiskyProject Enterprise.
#ATAGTR2018 Presentation "The Subtle Influence of Cognitive Biases on Testing...Agile Testing Alliance
Prabhakar Panditi who is an enterprise Agile Coach, Executive Coach Lean Agile and Product Development Consult has conducted a Game session on "The Subtle Influence of Cognitive Biases on Testing Professional."
please refer our linkedin post for session details
https://www.linkedin.com/pulse/game-session-prabhaker-panditi-subtle-influence-biases-alliance/
http://verraes.net
The project was of to a bad start: an inherited legacy codebase, a waterfall contract, and a projected loss. The promise of Kaizen or Continuous Improvement seemed very appealing. But when we tried to incorporate this into our process, it didn’t catch on. Biweekly retrospectives didn’t seem to expose any problems we could improve upon. The ceremonies we tried, like Deming’s Plan-Do-Check-Act cycles, added too much overhead. We were doing something wrong.
Continuous Improvement implies that you know exactly where to focus your efforts. Like scientists, we started to experiment, without deciding upfront what we expected the outcome to be. The rules? Make every experiment as small as possible. No meetings, no consensus, no cumbersome evaluation process. We let the results speak for themselves. This talk explores the successes and failures of a team that went from survival mode to learning mode over the course of a year.
Cognitive Behaviour Coaching For Young Leadersrenjmat
Cognitive Behavior Coaching has been a novel and promising mode of corporate training. Cognitive Behavior models were proved clinically as well as using brain Imaging studies. This model has been well adapted and incorporated to corporate psychological training. This programme could surpasses the shortfalls of most of the available corporate training programmes.
Ipsos MORI Political Monitor: January 2013Ipsos UK
The January Ipsos MORI Political Monitor shows the highest level of support Ipsos MORI has recorded for UKIP in this long running series of national voting intentions. Among those who say they are certain to vote at an immediate general election 9% say they would vote for UKIP. See http://www.ipsos-mori.com/researchpublications/researcharchive/3113/Ipsos-MORI-Political-Monitor-2013.aspx for more information.
Ipsos Global @dvisor 29: The economic pulse of the world: February 2012Ipsos UK
Around one in eight Britons (13%) rate the state of the economy as good while 87% describe it as bad according to new research from Ipsos’ Global @dvisor online survey conducted in 24 countries.
FeedMe: Enhancing Directed Content Sharing on the WebMichael Bernstein
To find interesting, personally relevant web content, people rely on friends and colleagues to pass links along as they encounter them. In this paper, we study and augment link-sharing via e-mail, the most popular means of sharing web content today. Armed with survey data indicating that active sharers of novel web content are often those that actively seek it out, we developed FeedMe, a plug-in for Google Reader that makes directed sharing of content a more salient part of the user experience. FeedMe recommends friends who may be interested in seeing content that the user is viewing, provides information on what the recipient has seen and how many emails they have received recently, and gives recipients the opportunity to provide lightweight feedback when they appreciate shared content. FeedMe introduces a novel design space within mixed-initiative social recommenders: friends who know the user voluntarily vet the material on the user’s behalf. We performed a two-week field experiment (N=60) and found that FeedMe made it easier and more enjoyable to share content that recipients appreciated and would not have found otherwise.
Ipsos MORI Political Monitor: December 2013Ipsos UK
Ipsos MORI’s latest Political Monitor indicates that George Osborne has taken a firm lead over Ed Balls as most capable Chancellor in the eyes of the public, having been neck-and-neck in August. Four in ten (40%) see Mr Osborne as the more capable Chancellor, an eleven percentage point lead over Mr Balls (29%). In August they were tied, with Mr Osborne on 36% and Mr Balls on 35%.
Survey of Britain's Captains of Industry 2013Ipsos UK
Ipsos MORI’s Captains of Industry report finds 20 year high in optimism for economy from the men and women running Britain’s largest companies. Justin King of Sainsbury’s named Britain’s most impressive businessman.
Eddi: Interactive Topic-Based Browsing of Social Status StreamsMichael Bernstein
Talk given at UIST 2010 by Michael Bernstein.
Twitter streams are on overload: active users receive hundreds of items per day, and existing interfaces force us to march through a chronologically-ordered morass to find tweets of interest. We present an approach to organizing a user's own feed into coherently clustered trending topics for more directed exploration. Our Twitter client, called Eddi, groups tweets in a user’s feed into topics mentioned explicitly or implicitly, which users can then browse for items of interest. To implement this topic clustering, we have developed a novel algorithm for discovering topics in short status updates powered by linguistic syntactic transformation and callouts to a search engine. An algorithm evaluation reveals that search engine callouts outperform other approaches when they employ simple syntactic transformation and backoff strategies. Active Twitter users evaluated Eddi and found it to be a more efficient and enjoyable way to browse an overwhelming status update feed than the standard chronological interface.
Scottish Independence Referendum: 200 days to goIpsos UK
The Scottish Independence referendum on 18 September 2014 will be a major political focus of the autumn. This presentation highlights how public opinion is shifting in Scotland and the rest of the UK, as well as the wider implications of a yes or no vote.
Science as Creativity, Science is CreativitythirstDC
Science's understanding of the creative process is much more thorough than people might believe. This is the full version of of lecture on how the creative process is currently modeled and told through a series of stupid pictures.
Sample slides from Bo Adams and Meghan Cureton re: curiosity-based learning and employing the Innovator's DNA traits as practices for deepening curiosity, enhancing innovation, and rethinking construction of school curriculum.
Well-facilitated retrospectives are a key ingredient in high-functioning, continuously improving teams. Come learn and share best practices for planning and executing high quality retrospectives that establish rhythm, build trust, and create breakthroughs. We'll discuss the necessary elements of any great retrospective, practice designing an agenda, and troubleshoot common facilitation problems.
Participants will leave with a "bag of tricks" for facilitating open, honest, and game-changing retrospectives for any kind of team or project.
The Past, Present and Future of ABM: How To Cope With A New Research Method Edmund Chattoe-Brown
This talk considers the challenges of developing a "canon" for ABM based on research (some of which has been forgotten), the present problem situation of many non comparable models and a possible future based on greater interdisciplinary and more systematic development of methodology.
This lecture is given in the First Year Program at Bowling Green State University, School of Art, in the Artist Methods and Practice workshop. This presentation is by Anthony Fontana, Instructor of Art and Learning Technologist.
The presentation by Klaus Gottlieb highlights human thinking tools that maintain advantages over AI, focusing on critical thinking, creativity, and problem-solving strategies. It showcases how these cognitive skills enable humans to interpret, innovate, and navigate complex scenarios more effectively than current AI capabilities, underscoring the importance of leveraging human intellect alongside technological advancements.
Keywords: Critical Thinking, Creativity, Problem-Solving, Human Intellect, Cognitive Skills, Innovation, AI Limitations.
How soon after a training session do employees start forgetting new information? Why does this happen and how can it change?
In a recent LinkedIn discussion, Charles Henderson asked the question, “In 10 words or less, why do you think learners forget what they’ve learned so quickly?” The question has since garnered over 900 responses.
Join Charles and our CEO Carol Leaman as they explore this question and also discuss what the latest in brain science has to offer on the problem of forgetting.
Get the Webinar recording here: www.axonify.com/forgetting
It's all a game: The twin fallacies of epistemic purity and the scholarly inv...Carl Bergstrom
My talk from the Feb 2016 Gaming Metrics workshop at UC Davis.
Video of the talk at http://bit.ly/1WwfMxY (begins 20:00)
For some reason the visual gradients (used to indicate gradients of behavior) have not rendered here.
My presentation to UT Engineering's KTE mentorship program on 4/22/2019. It's primarily a presentation of the ideas of "Designing Your Life" by Bill Burnett and Dave Evans, how I used those ideas in my own career transition, and how specifically it could apply in the context of UT Austin.
Disclaimer: I'm not associated with the book or its authors, and the views expressed may not reflect the views of the authors or UT Austin. All opinions are intended for educational purposes and do not replace professional judgement.
Ipsos MORI Political Monitor: January 2013Ipsos UK
The January Ipsos MORI Political Monitor shows the highest level of support Ipsos MORI has recorded for UKIP in this long running series of national voting intentions. Among those who say they are certain to vote at an immediate general election 9% say they would vote for UKIP. See http://www.ipsos-mori.com/researchpublications/researcharchive/3113/Ipsos-MORI-Political-Monitor-2013.aspx for more information.
Ipsos Global @dvisor 29: The economic pulse of the world: February 2012Ipsos UK
Around one in eight Britons (13%) rate the state of the economy as good while 87% describe it as bad according to new research from Ipsos’ Global @dvisor online survey conducted in 24 countries.
FeedMe: Enhancing Directed Content Sharing on the WebMichael Bernstein
To find interesting, personally relevant web content, people rely on friends and colleagues to pass links along as they encounter them. In this paper, we study and augment link-sharing via e-mail, the most popular means of sharing web content today. Armed with survey data indicating that active sharers of novel web content are often those that actively seek it out, we developed FeedMe, a plug-in for Google Reader that makes directed sharing of content a more salient part of the user experience. FeedMe recommends friends who may be interested in seeing content that the user is viewing, provides information on what the recipient has seen and how many emails they have received recently, and gives recipients the opportunity to provide lightweight feedback when they appreciate shared content. FeedMe introduces a novel design space within mixed-initiative social recommenders: friends who know the user voluntarily vet the material on the user’s behalf. We performed a two-week field experiment (N=60) and found that FeedMe made it easier and more enjoyable to share content that recipients appreciated and would not have found otherwise.
Ipsos MORI Political Monitor: December 2013Ipsos UK
Ipsos MORI’s latest Political Monitor indicates that George Osborne has taken a firm lead over Ed Balls as most capable Chancellor in the eyes of the public, having been neck-and-neck in August. Four in ten (40%) see Mr Osborne as the more capable Chancellor, an eleven percentage point lead over Mr Balls (29%). In August they were tied, with Mr Osborne on 36% and Mr Balls on 35%.
Survey of Britain's Captains of Industry 2013Ipsos UK
Ipsos MORI’s Captains of Industry report finds 20 year high in optimism for economy from the men and women running Britain’s largest companies. Justin King of Sainsbury’s named Britain’s most impressive businessman.
Eddi: Interactive Topic-Based Browsing of Social Status StreamsMichael Bernstein
Talk given at UIST 2010 by Michael Bernstein.
Twitter streams are on overload: active users receive hundreds of items per day, and existing interfaces force us to march through a chronologically-ordered morass to find tweets of interest. We present an approach to organizing a user's own feed into coherently clustered trending topics for more directed exploration. Our Twitter client, called Eddi, groups tweets in a user’s feed into topics mentioned explicitly or implicitly, which users can then browse for items of interest. To implement this topic clustering, we have developed a novel algorithm for discovering topics in short status updates powered by linguistic syntactic transformation and callouts to a search engine. An algorithm evaluation reveals that search engine callouts outperform other approaches when they employ simple syntactic transformation and backoff strategies. Active Twitter users evaluated Eddi and found it to be a more efficient and enjoyable way to browse an overwhelming status update feed than the standard chronological interface.
Scottish Independence Referendum: 200 days to goIpsos UK
The Scottish Independence referendum on 18 September 2014 will be a major political focus of the autumn. This presentation highlights how public opinion is shifting in Scotland and the rest of the UK, as well as the wider implications of a yes or no vote.
Science as Creativity, Science is CreativitythirstDC
Science's understanding of the creative process is much more thorough than people might believe. This is the full version of of lecture on how the creative process is currently modeled and told through a series of stupid pictures.
Sample slides from Bo Adams and Meghan Cureton re: curiosity-based learning and employing the Innovator's DNA traits as practices for deepening curiosity, enhancing innovation, and rethinking construction of school curriculum.
Well-facilitated retrospectives are a key ingredient in high-functioning, continuously improving teams. Come learn and share best practices for planning and executing high quality retrospectives that establish rhythm, build trust, and create breakthroughs. We'll discuss the necessary elements of any great retrospective, practice designing an agenda, and troubleshoot common facilitation problems.
Participants will leave with a "bag of tricks" for facilitating open, honest, and game-changing retrospectives for any kind of team or project.
The Past, Present and Future of ABM: How To Cope With A New Research Method Edmund Chattoe-Brown
This talk considers the challenges of developing a "canon" for ABM based on research (some of which has been forgotten), the present problem situation of many non comparable models and a possible future based on greater interdisciplinary and more systematic development of methodology.
This lecture is given in the First Year Program at Bowling Green State University, School of Art, in the Artist Methods and Practice workshop. This presentation is by Anthony Fontana, Instructor of Art and Learning Technologist.
The presentation by Klaus Gottlieb highlights human thinking tools that maintain advantages over AI, focusing on critical thinking, creativity, and problem-solving strategies. It showcases how these cognitive skills enable humans to interpret, innovate, and navigate complex scenarios more effectively than current AI capabilities, underscoring the importance of leveraging human intellect alongside technological advancements.
Keywords: Critical Thinking, Creativity, Problem-Solving, Human Intellect, Cognitive Skills, Innovation, AI Limitations.
How soon after a training session do employees start forgetting new information? Why does this happen and how can it change?
In a recent LinkedIn discussion, Charles Henderson asked the question, “In 10 words or less, why do you think learners forget what they’ve learned so quickly?” The question has since garnered over 900 responses.
Join Charles and our CEO Carol Leaman as they explore this question and also discuss what the latest in brain science has to offer on the problem of forgetting.
Get the Webinar recording here: www.axonify.com/forgetting
It's all a game: The twin fallacies of epistemic purity and the scholarly inv...Carl Bergstrom
My talk from the Feb 2016 Gaming Metrics workshop at UC Davis.
Video of the talk at http://bit.ly/1WwfMxY (begins 20:00)
For some reason the visual gradients (used to indicate gradients of behavior) have not rendered here.
My presentation to UT Engineering's KTE mentorship program on 4/22/2019. It's primarily a presentation of the ideas of "Designing Your Life" by Bill Burnett and Dave Evans, how I used those ideas in my own career transition, and how specifically it could apply in the context of UT Austin.
Disclaimer: I'm not associated with the book or its authors, and the views expressed may not reflect the views of the authors or UT Austin. All opinions are intended for educational purposes and do not replace professional judgement.
The top mistakes you're making in your Data Science interview - Omri AlloucheOmri Allouche
To be a great Data Scientist, you need to be a good mathematician, a curious analyst, a smart computer scientist and an expert in the problem domain. Furthermore, the field is moving so fast, you have to run at full speed just to stay in place. How should you balance these skills?
When interviewing candidates for Gong.io, we try to evaluate how well the candidate will tackle the large variety of research tasks we face, including Speech Recognition, Video and Audio analysis, NLP and statistical hypothesis testing. In this talk, I'll give an inside pick into our Data Science interview, and will list the top mistakes I see people make preparing for Data Science interviews, hoping to help you excel in your next interview and next position.
You can view a low-quality recording of the talk at https://www.youtube.com/watch?v=yu0HAudwGEA
Bio:
Omri Allouche heads the Research department at Gong.io, helping sales organizations improve their performance by providing actionable, data-driven insights using machine learning.
He also teaches Applied Data Science at Bar Ilan University, and was the founder and CEO of Page2site (acquired by Algomizer), an algorithms engineer at Elisra, and researcher at IDF's intelligence unit.
Omri holds a Ph.D. in Computational Ecology from the Hebrew University (cum laude). He won several academic awards and scholarships, including the Clore fund, and his research papers had been cited over 2,000 times.
What's the Science in Data Science? - Skipper SeaboldPyData
The gold standard for validating any scientific assumption is to run an experiment. Data science isn’t any different. Unfortunately, it’s not always possible to design the perfect experiment. In this talk, we’ll take a realistic look at measurement using tools from the social sciences to conduct quasi-experiments with observational data.
On practical philosophy of research in science and technologySeppo Karrila
An attempt to indoctrinate graduate students with some philosophy of science and good practices in their research. Some references are included to disturbing trends known from poor practices that appear common to some fields, to make clear the importance of reliable methods, in particular the Scientific Method. Trigger warning: not trying to be nice to everybody.
Quantifying the Invisible Audience in Social NetworksMichael Bernstein
Presented at CHI 2013
When you share content in an online social network, who is listening? Users have scarce information about who actually sees their content, making their audience seem invisible and difficult to estimate. However, understanding this invisible audience can impact both science and design, since perceived audiences influence content production and self-presentation online. In this paper, we combine survey and large-scale log data to examine how well users’ perceptions of their audience match their actual audience on Facebook. We find that social media users consistently underestimate their audience size for their posts, guessing that their audience is just 27% of its true size. Qualitative coding of survey responses reveals folk theories that attempt to reverse-engineer audience size using feedback and friend count, though none of these approaches are particularly accurate. We analyze audience
logs for 222,000 Facebook users’ posts over the course of one month and find that publicly visible signals — friend count, likes, and comments — vary widely and do not strongly indicate the audience of a single post. Despite the variation, users typically reach 61% of their friends each month. Together, our results begin to reveal the invisible undercurrents of audience attention and behavior in online social networks.
Paper at http://hci.stanford.edu/publications/2013/CrowdWork/futureofcrowdwork-cscw2013.pdf
Paid crowd work offers remarkable opportunities for
improving productivity, social mobility, and the global
economy by engaging a geographically distributed
workforce to complete complex tasks on demand and at
scale. But it is also possible that crowd work will fail to
achieve its potential, focusing on assembly-line piecework.
Can we foresee a future crowd workplace in which we
would want our children to participate? This paper frames
the major challenges that stand in the way of this goal.
Drawing on theory from organizational behavior and
distributed computing, as well as direct feedback from
workers, we outline a framework that will enable crowd
work that is complex, collaborative, and sustainable. The
framework lays out research challenges in twelve major
areas: workflow, task assignment, hierarchy, real-time
response, synchronous collaboration, quality control,
crowds guiding AIs, AIs guiding crowds, platforms, job
design, reputation, and motivation.
Analytic Methods for Optimizing Realtime CrowdsourcingMichael Bernstein
Collective Intelligence 2012
Realtime crowdsourcing research has demonstrated that it is possible to recruit paid crowds within seconds by managing a small, fast-reacting worker pool. Realtime crowds enable crowd-powered systems that respond at interactive speeds: for example, cameras, robots and instant opinion polls. So far, these techniques have mainly been proof-of-concept prototypes: research has not yet attempted to understand how they might work at large scale or optimize their cost/performance trade-offs. In this paper, we use queueing theory to analyze the retainer model for realtime crowdsourcing, in particular its expected wait time and cost to requesters. We provide an algorithm that allows requesters to minimize their cost subject to performance requirements. We then propose and analyze three techniques to improve performance: push notifications, shared retainer pools, and precruitment, which involves recalling retainer workers before a task actually arrives. An experimental validation finds that precruited workers begin a task 500 milliseconds after it is posted, delivering results below the one-second cognitive threshold for an end-user to stay in flow.
4chan and /b/: An Analysis of Anonymity and Ephemerality in a Large Online Co...Michael Bernstein
Paper presented at ICWSM 2011
http://projects.csail.mit.edu/chanthropology
We present two studies of online ephemerality and anonymity based on the popular discussion board /b/ at 4chan.org: a website with over 7 million users that plays an influential role in Internet culture. Although researchers and practitioners often assume that user identity and data permanence are central tools in the design of online communities, we explore how /b/ succeeds de- spite being almost entirely anonymous and extremely ephemeral. We begin by describing /b/ and performing a content analysis that suggests the community is dominated by playful exchanges of images and links. Our first study uses a large dataset of more than five million posts to quantify ephemerality in /b/. We find that most threads spend just five seconds on the first page and less than five minutes on the site before expiring. Our sec- ond study is an analysis of identity signals on 4chan, finding that over 90% of posts are made by fully anonymous users, with other identity signals adopted and discarded at will. We describe alternative mechanisms that /b/ participants use to establish status and frame their interactions.
Talk given at UIST 2010.
This paper introduces architectural and interaction patterns for integrating crowdsourced human contributions directly into user interfaces. We focus on writing and editing, complex endeavors that span many levels of conceptual and pragmatic activity. Authoring tools offer help with pragmatics, but for higher-level help, writers commonly turn to other people. We thus present Soylent, a word processing interface that enables writers to call on Mechanical Turk workers to shorten, proofread, and otherwise edit parts of their documents on demand. To improve worker quality, we introduce the Find-Fix-Verify crowd programming pat- tern, which splits tasks into a series of generation and review stages. Evaluation studies demonstrate the feasibility of crowdsourced editing and investigate questions of reliability, cost, wait time, and work time for edits.
1. RepliCan’t Graduate Student Perspectives Michael Bernstein MIT Computer Science and Artificial Intelligence Laboratory msbernst@mit.edu | @msbernst mit human-computer interaction
4. My Unassailable, Extremely ScientificData Collection Protocol: Survey of CHI student volunteers, CHI-students ACM listserv, and snowballed recruitment through Facebook and Twitter. N=93 responses
5. Have you ever replicateda study or system? 17% yes, 83% no
6. Do you ever plan to replicatea study or system? 38% “Hell yes”, 62% “Hell no”
7. Set Yourself Apart “ The point of research is to come up with exciting ideas that solve problems. Not copy others’ work. ”
9. There’s No Reward “ New studies confirming old studies have no chance of publication. ”
10. There’s No Reward Reviewers […] didn't feel replication was necessary even though the original study was specific to a single company. “ ”
11. There’s No Reward I very frequently see reviewers criticize submissions for presenting results that are “not novel” or “have already been shown”. “ ”
12. There’s No Reward I very frequently see reviewers criticize submissions for presenting results that are “not novel” or “have already been shown”. “ ”
13. Responding to Incentives Open access and replication. A true scientist’s ideals, but see:The grad student must conform. “ ”
14. Haikus studies should break ground replication wastes our timelet's find new problems “ ”
16. Haikus repeat to be sure we stand on giants’ shoulders but do so on faith “ ”
17. 83% have not 62% will not (But we’ll need to replicate the study to be sure.)
18.
19. Why? “ repeat to be sure we stand on giant's shoulders but do so on faith ”
20. Why? “ Replication is a critical component of scientific research, and it should be encouraged and rewarded. The lack of it is detrimental to the scientific soundness of our discipline. ”
21. Why? “ we lack the time for replication of studies just review strictly ”
22. Why? “ think analyzing CMC is tough? try it reproducibly! ”
23. Why? “ It seems like the best outcome for a replication, rather than success, is actually a refutation of the original study. ”
24. Why? “ Because its's a waste of time: HCI studies are so small, I know they surely WON'T replicate, so why bother! ”
25. Why? “ CHI is too cutting edge for things like replication, or good science, or careful analysis, or the humility to accept that other topics besides Fitts' Law deserve dozens of nearly-identical studies. ”
26.
27. Why? “ I do not intend on taking the risk of replicating some of my favorite works unless I see evidence that the CHI community supports such a thing. ”
28. There’s No Reward Case A. Confirmation of the earlier results (very boring)Case B. Conflict with earlier results (unpublishable problem) “ ”
Editor's Notes
Interfaces that embed crowd contributions to support complex and novel interactive systems.-------------Bring in the simplex? Thread through, come back to it from time to time.Soylent. Emphasize that we’re not throwing away AI, not throwing away the user, …Explain FeedMe better via the simplex?Insights from designing two different families of interfaces --- generalize lessons? Motivations. Design differences. Commonalities: small bits of effort? At end of talk.- small bits of effortMost of this has examined moving things toward crowd. But, we need a better sense of when to do that, and when not to. Feed back into traditional We need to understand in order to build. Not just the user interface, but the crowd.Embiggen the gray from data mining. Move data mining to future work?Introduce points one and two, these are the systems I’m going to talk about in this talk. Then, at the end, say there are two avenues for future work. 1) realtime, 2) data thing thing embiggen thing.Data-driven?Less cadence at the end of Soylent.Transition between Collabio and FeedMe needs work.Second half was too fast-paced? Video especially?Stupid PhD student.25 min soylent, 15 min Don’t make crazy animation w/ ShortnTry to save the Soylent demo for later. Figure out a way to convey the concept without the early handwaved demo.“Pa-nah-cea”Finish the transition. There are lots of applications where you can't use crowds. One of those is when you need socially sourced information. We have to develop techniques for this