This document appears to be a presentation discussing challenges and considerations around open data sharing and analysis. It covers topics such as the need for data cleaning and clarification, difficulties in searching for and representing data, and ensuring tools and systems are useful, usable, understandable and learnable for researchers. The presentation provides examples of data that could be shared and calls for additional data to analyze learning patterns. It also discusses designing systems to accommodate workarounds.

1. Tales from the Trenches
- OR –
Replicating things that have already been said at this meeting
Robert R. Hoffman, Ph.D.
“Data Coding, Analysis, Archiving, and Sharing for Open Collaboration:
From OpenSHAPA to Open Data Sharing,”
© 2011 Robert R. Hoffman All rights reserved

2. Me
Experimental Psychology
(Cognitive, Psycholinguistics)
U of Cinti, U of Minn
Expertise Studies
Cognitive Systems Engineering
Human Factors
© 2011 Robert R. Hoffman All rights reserved

3. Macrocognition as a paradigm in
Cognitive Systems Engineering
Gunnar Johansson
Mental simulation
1980 Problem detection
Cacciabue & Hollnagel
Coordination
1995 Re-planning
Klein et al
Expertise development
2002
IEEE Intelligent Systems
(2 seconds ?)
© 2011 Robert R. Hoffman All rights reserved

4. © 2011 Robert R. Hoffman All rights reserved

5. Communities of Practice
“Behavior”
Versus
Activity
Activity
Theory
Work
Analysis
Sociotechnics
© 2011 Robert R. Hoffman All rights reserved

6. Ancient History
“How fast was the car going when it
(bumped, crashed) into the other car?”
Barbara Tversky, et al.
© 2011 Robert R. Hoffman All rights reserved

7. That was then, this is….
then. . . . . .
© 2011 Robert R. Hoffman All rights reserved

8. Myth of the Normal Curve
Examples:
Sampling under a stopping rule
Traffic delays (lots of brief ones, rare long ones)
Achieving progressive criteria in pole vaulting
Errors in motor coordination tasks
Patil, G. P. (1960). On the evaluation of the negative binomial
distribution with examples. Technometrics, 2, 501-505.
Sichel, H. S. (1951). The estimation of parameters of a negative
binomial distribution with special reference to psychological
data. Psychometrika, 16, 107-127.
© 2011 Robert R. Hoffman All rights reserved

9. "Call For Data"
• Usability/Learnability analysis
• Performance at the very first trials
of learning any task; any DVs
• Exact modeling of discrete
non-Gaussian distributions
© 2011 Robert R. Hoffman All rights reserved

10. Learning geometrical patterns
Learning to use a cell phone by the elderly
Learning to operate an automotive GPS (route-finding)
Learning to recognize voices in auditory localization
Learning to control an avatar in a virtual world
Learning of the structure of biological categories
Learning to fly a cockpit simulator
9 data sets on hand, 9 more pending
© 2011 Robert R. Hoffman All rights reserved

11. ?
© 2011 Robert R. Hoffman All rights reserved

12. ?
© 2011 Robert R. Hoffman All rights reserved

13. ?
© 2011 Robert R. Hoffman All rights reserved

14. ?
© 2011 Robert R. Hoffman All rights reserved

15. Rule #1
Clean-up is always necessary
Retabbing
Fonts
Column widths
Etc.
© 2011 Robert R. Hoffman All rights reserved

16. Rule #2
You always have to go back and talk to the
researcher
What does "No OT" mean?
What did you really do?
Did I fix the tab delineations correctly?
Worse. . . .
People forget things about their
own data, even short-term
© 2011 Robert R. Hoffman All rights reserved

17. The Control Challenge
How do you cope with the consequences of
these Rules?
Do you impose constraints?
- OR -
Do you acknowledge that clean-up will always
be necessary, and figure out ways to make it
easier.
© 2011 Robert R. Hoffman All rights reserved

18. Requirements v. "Desirements"
("help" versus "impose")
Designing for kluges and work-arounds
Hoffman, R. R. & Elm, W. C. (2006, January/February).
HCC implications for the procurement process.
IEEE: Intelligent Systems, pp. 74-81.
Koopman, P. & Hoffman, R. R., (November/December 2003).
Work-Arounds, Make-Work, and Kludges.
IEEE: Intelligent Systems, pp. 70-75.
© 2011© Robert R. Hoffman All rights reserved
Robert R. Hoffman All rights reserved

19. The Search Challenges
Challenge #1
Finding data by data constraint (e.g., data type,
meaning)
Challenge #2
Finding data by design constraint (e.g., between
v. within, etc.)
© 2011 Robert R. Hoffman All rights reserved

20. The Representation Challenges
Challenge #3
Frame problems (a priori categories v. search
categories)
Challenge #4
Practicalities of representation - cryptograms,
acronyms, abbreviations
© 2011 Robert R. Hoffman All rights reserved

21. The Purpose Challenges
Challenge #5
Chiding resarchers re: design limitations,
methodological questions
Challenge #6
Topical research data v. Statistics itself as an
area of research
© 2011 Robert R. Hoffman All rights reserved

22. Having seen all the Tools & Systems. . .
Usefulness
Usability
These features are measurable
Understandability
Learnability
e.g., Data on Researcher time, effort, resources
“labor intensive”
“efficient data mining”
© 2011 Robert R. Hoffman All rights reserved

23. © 2011 Robert R. Hoffman All rights reserved

24. Thank you!
www.ihmc.us
www.ihmc.us/users/rhoffman/main
© 2011 Robert R. Hoffman All rights reserved