Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Selecting Empirical  Methods for SE     Research      Daniel Cukier
5 classes of methodControlled experimentsCase studiesSurvey researchEthnographiesAction research
Exploratory       questionsExistenceDescription and classificationDescriptive-comparative
Base-rate questionsFrequency and distributionDescriptive-process
Relationship      questionsIs X related to Y?If X occurs, will Y also occur?
Causality questionsDoes X cause Y? Does X prevent Y?Causality-ComparativeDoes context affects causality?
Design questionsHow effectively achieve X?What helps?What does not help?
Empirical truth
epistemologythe nature of human knowledge,    and how we obtain it.         onthology  the nature of the worldirrespective...
Plato about knowledge to know something, you must  believe it to be true, andhave a clear justification for   believing it...
Constructivistswe cannot separate knowledgefrom the language we use to         express it
The philosophical     stancesPositivismConstructivismCritical TheoryPragmatism
PositivismAll knowledge comes from a set ofbasic observable facts.ReductionistSometimes comes to wrong conclusionsAssociat...
ConstructivismInterpretivismOften adopted in social sciencesQualitative data about human beingsexploratory case studies or...
Critical TheoryPolitical actEmancipatory and advogacy rolee.g. Open source movement and AgileCommunityAction research and ...
PragmatismAll knowledge is approximate andincomplete and depends on the methodsTruth is whatever works for meTruth is rela...
AbstractTo the positivist, science is the process of veri-fying theories by testing hypotheses derived fromthem.To the con...
TheoriesTheory becomes a “lens” through  which the world is observed
Theories Real-world phenomena are    simply too rich andcomplex to study without a huge amount of filtering
Theories Software Engineering    researchers havetraditionally been verypoor at making theories
Method SelectionResearch Design is theprocess of selecting amethod for a particular        problem
Controlled ExperimentsIndependent variables effects over dependent onesPrecondition: a clear hypothesisDemonstrate hypothe...
Case StudiesExploratory or confirmatoryCritical caseMultiple caseWeakness: open to interpretation andresearcher biasAll ph...
Survey ResearchQuestionnaires for data collectionRepresentative sample from welldefined populationChallenge: control for s...
EthnographiesGoal: study community of peopleResult: a rich description of itParticipant observation - possible ifresearche...
Action ResearchSolve real-problem while study it.Researcher is also a change agentEducation, information science fieldsLon...
Mixed-MethodsSequential explanatory - quantitativefollowed by qualitativeSequential exploratory - qualitativefollowed by q...
Data CollectionEach technique has its strengths andweaknessIf different kinds of data supportthe same conclusions, it stre...
Empirical ValidityConstructInternalExternalReliability
Critical                Positivism Constructivism          Pragmatism                                           Theory Con...
Upcoming SlideShare
Loading in …5
×

Selecting Empirical Methods for Software Engineering

1,876 views

Published on

Presentation on how to write good Master and PhD dissertations.

Empirical Methods, Software Engineering, science, computer science, software, methods, positivism, epistemology, onthology, construtivism, critical theory, pragmatism, case study, research action, ethnography

Published in: Technology, Spiritual

Selecting Empirical Methods for Software Engineering

  1. 1. Selecting Empirical Methods for SE Research Daniel Cukier
  2. 2. 5 classes of methodControlled experimentsCase studiesSurvey researchEthnographiesAction research
  3. 3. Exploratory questionsExistenceDescription and classificationDescriptive-comparative
  4. 4. Base-rate questionsFrequency and distributionDescriptive-process
  5. 5. Relationship questionsIs X related to Y?If X occurs, will Y also occur?
  6. 6. Causality questionsDoes X cause Y? Does X prevent Y?Causality-ComparativeDoes context affects causality?
  7. 7. Design questionsHow effectively achieve X?What helps?What does not help?
  8. 8. Empirical truth
  9. 9. epistemologythe nature of human knowledge, and how we obtain it. onthology the nature of the worldirrespective of our attempts to understand it.
  10. 10. Plato about knowledge to know something, you must believe it to be true, andhave a clear justification for believing it to be true
  11. 11. Constructivistswe cannot separate knowledgefrom the language we use to express it
  12. 12. The philosophical stancesPositivismConstructivismCritical TheoryPragmatism
  13. 13. PositivismAll knowledge comes from a set ofbasic observable facts.ReductionistSometimes comes to wrong conclusionsAssociated to controlled experiment
  14. 14. ConstructivismInterpretivismOften adopted in social sciencesQualitative data about human beingsexploratory case studies or surveyresearch
  15. 15. Critical TheoryPolitical actEmancipatory and advogacy rolee.g. Open source movement and AgileCommunityAction research and case studies
  16. 16. PragmatismAll knowledge is approximate andincomplete and depends on the methodsTruth is whatever works for meTruth is relative to observerValues practical knowledgeMixed methods
  17. 17. AbstractTo the positivist, science is the process of veri-fying theories by testing hypotheses derived fromthem.To the constructivist, science is the process ofseeking local theories that emerge from (and explain)the data.To the critical theorist, theoriesare assertions of knowledge (and therefore power), tobe critiqued in terms of how they shape that power.To the pragmatist, theories are the products of aconsensual process among a community of researchers,to be judged for their practical utility.
  18. 18. TheoriesTheory becomes a “lens” through which the world is observed
  19. 19. Theories Real-world phenomena are simply too rich andcomplex to study without a huge amount of filtering
  20. 20. Theories Software Engineering researchers havetraditionally been verypoor at making theories
  21. 21. Method SelectionResearch Design is theprocess of selecting amethod for a particular problem
  22. 22. Controlled ExperimentsIndependent variables effects over dependent onesPrecondition: a clear hypothesisDemonstrate hypothesis by testing it on arepresentative populationControl unwanted variablesIf critical variables are ignored, experimentmight not generalize real worldTheory-driven is both a strength and a weakness
  23. 23. Case StudiesExploratory or confirmatoryCritical caseMultiple caseWeakness: open to interpretation andresearcher biasAll philosophical stances
  24. 24. Survey ResearchQuestionnaires for data collectionRepresentative sample from welldefined populationChallenge: control for sampling biasHarder challenge: ensure thatquestions are designed in a way thatyields useful and valid dataPositivism tradition
  25. 25. EthnographiesGoal: study community of peopleResult: a rich description of itParticipant observation - possible ifresearcher is technical guyChallenge: avoid preconceptions ondata observation and analysisConstructivism
  26. 26. Action ResearchSolve real-problem while study it.Researcher is also a change agentEducation, information science fieldsLong term commitment, expensiveIs a new idea, immature methodIterative processNo attempt to create a control groupCritical theory
  27. 27. Mixed-MethodsSequential explanatory - quantitativefollowed by qualitativeSequential exploratory - qualitativefollowed by quantitativeConcurrent triangulation - whatpeople say is different from whatpeople doDifficult to resolve contradictions
  28. 28. Data CollectionEach technique has its strengths andweaknessIf different kinds of data supportthe same conclusions, it strengthensthe studyPilot-test the data collection
  29. 29. Empirical ValidityConstructInternalExternalReliability
  30. 30. Critical Positivism Constructivism Pragmatism Theory Controlled experiment R RCase studies R R R REthnographies R R Survey research R R R Action research R R

×