Selecting Empirical Methods for Software Engineering


Published on

Presentation on how to write good Master and PhD dissertations.

Empirical Methods, Software Engineering, science, computer science, software, methods, positivism, epistemology, onthology, construtivism, critical theory, pragmatism, case study, research action, ethnography

Published in: Technology, Spiritual
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Selecting Empirical Methods for Software Engineering

  1. 1. Selecting Empirical Methods for SE Research Daniel Cukier
  2. 2. 5 classes of methodControlled experimentsCase studiesSurvey researchEthnographiesAction research
  3. 3. Exploratory questionsExistenceDescription and classificationDescriptive-comparative
  4. 4. Base-rate questionsFrequency and distributionDescriptive-process
  5. 5. Relationship questionsIs X related to Y?If X occurs, will Y also occur?
  6. 6. Causality questionsDoes X cause Y? Does X prevent Y?Causality-ComparativeDoes context affects causality?
  7. 7. Design questionsHow effectively achieve X?What helps?What does not help?
  8. 8. Empirical truth
  9. 9. epistemologythe nature of human knowledge, and how we obtain it. onthology the nature of the worldirrespective of our attempts to understand it.
  10. 10. Plato about knowledge to know something, you must believe it to be true, andhave a clear justification for believing it to be true
  11. 11. Constructivistswe cannot separate knowledgefrom the language we use to express it
  12. 12. The philosophical stancesPositivismConstructivismCritical TheoryPragmatism
  13. 13. PositivismAll knowledge comes from a set ofbasic observable facts.ReductionistSometimes comes to wrong conclusionsAssociated to controlled experiment
  14. 14. ConstructivismInterpretivismOften adopted in social sciencesQualitative data about human beingsexploratory case studies or surveyresearch
  15. 15. Critical TheoryPolitical actEmancipatory and advogacy rolee.g. Open source movement and AgileCommunityAction research and case studies
  16. 16. PragmatismAll knowledge is approximate andincomplete and depends on the methodsTruth is whatever works for meTruth is relative to observerValues practical knowledgeMixed methods
  17. 17. AbstractTo the positivist, science is the process of veri-fying theories by testing hypotheses derived fromthem.To the constructivist, science is the process ofseeking local theories that emerge from (and explain)the data.To the critical theorist, theoriesare assertions of knowledge (and therefore power), tobe critiqued in terms of how they shape that power.To the pragmatist, theories are the products of aconsensual process among a community of researchers,to be judged for their practical utility.
  18. 18. TheoriesTheory becomes a “lens” through which the world is observed
  19. 19. Theories Real-world phenomena are simply too rich andcomplex to study without a huge amount of filtering
  20. 20. Theories Software Engineering researchers havetraditionally been verypoor at making theories
  21. 21. Method SelectionResearch Design is theprocess of selecting amethod for a particular problem
  22. 22. Controlled ExperimentsIndependent variables effects over dependent onesPrecondition: a clear hypothesisDemonstrate hypothesis by testing it on arepresentative populationControl unwanted variablesIf critical variables are ignored, experimentmight not generalize real worldTheory-driven is both a strength and a weakness
  23. 23. Case StudiesExploratory or confirmatoryCritical caseMultiple caseWeakness: open to interpretation andresearcher biasAll philosophical stances
  24. 24. Survey ResearchQuestionnaires for data collectionRepresentative sample from welldefined populationChallenge: control for sampling biasHarder challenge: ensure thatquestions are designed in a way thatyields useful and valid dataPositivism tradition
  25. 25. EthnographiesGoal: study community of peopleResult: a rich description of itParticipant observation - possible ifresearcher is technical guyChallenge: avoid preconceptions ondata observation and analysisConstructivism
  26. 26. Action ResearchSolve real-problem while study it.Researcher is also a change agentEducation, information science fieldsLong term commitment, expensiveIs a new idea, immature methodIterative processNo attempt to create a control groupCritical theory
  27. 27. Mixed-MethodsSequential explanatory - quantitativefollowed by qualitativeSequential exploratory - qualitativefollowed by quantitativeConcurrent triangulation - whatpeople say is different from whatpeople doDifficult to resolve contradictions
  28. 28. Data CollectionEach technique has its strengths andweaknessIf different kinds of data supportthe same conclusions, it strengthensthe studyPilot-test the data collection
  29. 29. Empirical ValidityConstructInternalExternalReliability
  30. 30. Critical Positivism Constructivism Pragmatism Theory Controlled experiment R RCase studies R R R REthnographies R R Survey research R R R Action research R R
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.