Against Digital Research Methodologies


Published on

This is a description of my approach to research, based on problems inherent in traditional descriptions of scientific method, and the ways I have adapted my own work to compensate. It is a research process more like discovery, more like learning a language, than it is about forming hypotheses and confirming theories. For audio and video please see

Published in: Education, Technology

Against Digital Research Methodologies

  1. 1. Against Digital Research MethodologiesStephen DownesMay 10, 2013
  2. 2. Caveats• This is a report, not aprescription• I’m not arguing – at best, I’mexplaining, but not in thesense that you can generalizefrom that
  3. 3. The traditional viewThe steps of the scientificmethod are to:• Ask a Question• Do Background Research• Construct a Hypothesis• Test Your Hypothesis by Doing an Experiment• Analyze Your Data and Draw a Conclusion• Communicate Your ResultsVia Science Buddies also:
  4. 4. Research MethodsThis model is pretty much thecore of most research mothods• Design Research• Observational Research• Qualitative Research / Grounded Theory
  5. 5. HD-MethodThis model is known as theHypothetico-Deductive Method• cf. mid-1800s• Updated by Carl Hempel as theDeductive-Nomological Model• “Inference to the Best Explanation”
  6. 6. I am an Empiricist• Observation and experienceare the foundation ofknowledge• There is no ‘synthetic apriori’Hume“Hume maintained that all knowledge, eventhe most basic beliefs about the natural world,cannot be conclusively established by reason.Rather, he maintained, our beliefs are more aresult of accumulated habits, developed inresponse to accumulated sense experiences.”
  7. 7. People Falling IntoHoles••• science is not based on believing there are no holes. Onthe contrary, my science is based on the realization thatthere’s always the possibility that the earth will open andswallow you up.
  8. 8. Two Dogmas ofEmpiricism• that there is a principleddistinction between analyticand synthetic propositions• that reductionism is true take: If this is the case, youcannot even state a theory,much less find oneQuine
  9. 9. The Fallacy of Theory• Elusive Truth – what distinguishessense from nonsense?• Theory-laden data – you see whatyou expect to see (gorilla video)• Incommensurability andparadigms• Empty consensus replacingrationality and truthScientific Method in BriefHugh G., Hugh G Gauch, Jr., pp. 53-66
  10. 10. Research Methods…Research methods, in a certainsense, presuppose their ownconclusions: they are silent oncomplex questions, for example,whether certain software ought tobe developed, which options oughtusers to be given, what subjectsought learners be taught to learn?
  11. 11. Against Method“Against Method explicitly drewthe “epistemological anarchist”conclusion that there are no usefuland exceptionless methodologicalrules governing the progress ofscience or the growth ofknowledge. The history of scienceis so complex that if we insist on ageneral methodology which willnot inhibit progress the only ‘rule’it will contain will be the uselesssuggestion: ‘anything goes’.” Feyerabend
  12. 12. Voltaire’s Bastards• Voltaire (and contemporaries) –thought (correctly) that reason wasthe best defense against arbitrarypolitical and religious authority• However, "Among the illusions whichhave invested our civilization is anabsolute belief that the solutions toour problems must be a moredetermined application of rationallyorganized expertise. The reality is thatour problems are largely the productof that application."
  13. 13. Dimensions of Change• Design is no longer based onresearch– Because the theory that will be‘observed’ is presupposed inthe theory• Users are no longer ‘subjects’– Because casting myself in therole of ‘expert’ rendersillegitimate the validexperiences of othersWittgenstein: meaning is use
  14. 14. Research-Led Design• Winter: “I had always assumedthat case studies, literaturereviews, and ethnographicresearch were necessaryprecursors to every well-informeddesign project I did.”• Vs. Design-Led Research
  15. 15. Design-Led ResearchLiz Sanders
  16. 16. Liz Sanders Research
  17. 17. Liz Sanders Research
  18. 18. Situated Maketools• They situated the study at theworkplace• Then grounded the designingin the workersʼexplanations• And scaffolded thedesigning, i.e. usedtemporary stuffSalu Ylirisku
  19. 19. Situating as Framing• To ‘situate’ is totheorize, only with asmaller universe ofdiscourse• Lakoff – ‘Framing’Lakoff
  20. 20. Beyond Theory• Design withouttheory is discovery• (And I recognizethat I am able tosample only theedge of a complexlandscape)
  21. 21. Reading the World• I don’t see the world as neatand ordered, like logic andmathematics – I see it asmessy and complex, like alanguage“We see the future in the same waythat we see the past, by reading thesigns”Wittgenstein: Meaning is Use
  22. 22. Method as LiteracyWhat we call ‘theory’ isjust one aspect of worldliteracy, and not even themost important one
  23. 23. A frame for understanding new mediaMorris, Derrida and a little Lao TzuSyntax CognitionSemantics ContextPragmatics ChangeWe need this frame because (as Jukes said) if we aren’tlooking for these things, we just won’t see them.
  24. 24. Theories / SyntaxForms: archetypes? Platonic ideals?Rules: grammar = logical syntaxOperations: procedures, motor skillsPatterns: regularities, substitutivity (eggcorns, tropes)Similarities: Tversky - properties, etcNot just rules and grammar
  25. 25. Semantics- Sense and reference (connotation and denotation)- Interpretation (Eg. In probability, Carnap - logical space;Reichenbach - frequency; Ramsey - wagering / strength of belief)- Forms of association: Hebbian, contiguity, back-prop, Boltzmann- Decisions and decision theory: voting / consensus / emergencetheories of truth /meaning / purpose /goal
  26. 26. Pragmatics• Speech acts (J.L. Austin, Searle) assertives, directives,commissives, expressives, declarations (but also - harmful acts,harassment, etc)• Interrogation (Heidegger) and presupposition• Meaning (Wittgenstein - meaning is use)use, actions, impact
  27. 27. Cognition• description - X (definite description, allegory, metaphor)• definition - X is Y (ostensive, lexical, logical (necess. & suff conds), familyresemblance - but also, identity, personal identity, etc• argument - X therefore Y - inductive, deductive, abductive (but also:modal, probability (Bayesian), deontic (obligations), doxastic (belief), etc.)• explanation - X because of Y (causal, statistical, chaotic/emergent)reasoning, inference and explanation
  28. 28. Context- explanation (Hanson, van Fraassen, Heidegger)- meaning (Quine); tense - range of possibilities- vocabulary (Derrida); ontologies, logical space- Frames (Lakoff) and worldviewsplacement, environment
  29. 29. Change- relation and connection: I Ching, logical relation- flow: Hegel - historicity, directionality; McLuhan - 4 things- progression / logic -- games, for example: quiz&points, branch-and-tree, database- scheduling - timetabling - events; activity theory / LaaN
  30. 30. 21st Century ScienceLanguagesThe ‘skills’ described by Jenkins –performance, simulation, appropriation, etc -are actually languages and should beunderstood in terms of these six dimensions
  31. 31. Discovery• You don’t learn a language,you discover it• To discover a language is to beimmersed in it, to speak it andlisten to people speaking in it• My scientific method (if it canbe called that) is to go to theoffice each day and immersemyself in the world – to trylistening, and to try speaking
  32. 32. I am the universe ofdiscourse
  33. 33. I’m not trying totheorize, I’m just tryingto do
  34. 34. • The ‘theory’ (notproperly-co-called)emerges from theinteractions betweenmyself and mycolleagues
  35. 35. The Disunity of theSciences
  36. 36.