SlideShare a Scribd company logo

(ebook) Understanding and Evaluating Research (McGregor 2018).pdf

•
•
S
ssuserdfeb82

This document provides an overview of the book "Understanding and Evaluating Research: A Critical Guide" by Sue McGregor. The book is dedicated to the author's husband Peter and was written during their retirement without computers. It aims to teach readers to be critical consumers of research by understanding research methodologies, theories, designs, and how to evaluate the quality of research presented in journal articles and papers. The book contains eight parts covering topics such as philosophical paradigms, research methods, literature reviews, results and findings, and how to discuss and draw conclusions from research.

Education
1 of 937
Download to read offline
Understanding and Evaluating Research 2
This book is dedicated to my wonderfully intelligent husband, Peter. The idea and the book were conceived and written during the second two years of our retirement. Now we have the future together on our boat, without a computer in sight. 3
Understanding and Evaluating Research A Critical Guide Sue L. T. McGregor 4
FOR INFORMATION: SAGE Publications, Inc. 2455 Teller Road Thousand Oaks, California 91320 E-mail: order@sagepub.com SAGE Publications Ltd. 1 Oliver’s Yard 55 City Road London EC1Y 1SP United Kingdom SAGE Publications India Pvt. Ltd. B 1/I 1 Mohan Cooperative Industrial Area Mathura Road, New Delhi 110 044 India SAGE Publications Asia-Pacific Pte. Ltd. 3 Church Street #10-04 Samsung Hub Singapore 049483 Copyright © 2018 by SAGE Publications, Inc. All rights reserved. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher. Printed in the United States of America Library of Congress Cataloging-in-Publication Data Names: McGregor, Sue L. T., author. Title: Understanding and evaluating research : a critical guide / Sue McGregor. Description: Thousand Oaks : SAGE Publications, [2018] | Includes bibliographical references and index. Identifiers: LCCN 2017034708 | ISBN 9781506350950 (pbk. : alk. paper) Subjects: LCSH: Research—Evaluation. Classification: LCC Q180.55.E9 M34 2018 | DDC 001.4—dc23 LC record available at https://lccn.loc.gov/2017034708 This book is printed on acid-free paper. Acquisitions Editor: Helen Salmon Editorial Assistant: Chelsea Neve Production Editor: Bennie Clark Allen Copy Editor: Melinda Masson Typesetter: C&M Digitals (P) Ltd. Proofreader: Susan Schon Indexer: Robie Grant 5
Cover Designer: Anupama Krishnan Marketing Manager: Susannah Goldes 6
Brief Contents 1. Preface 2. Acknowledgments 3. About the Author 4. PART I • INTRODUCTION 1. Chapter 1 • Critical Research Literacy 5. PART II • PHILOSOPHICAL AND THEORETICAL ASPECTS OF RESEARCH 1. Chapter 2 • Research Methodologies 2. Chapter 3 • Conceptual Frameworks, Theories, and Models 6. PART III • ORIENTING AND SUPPORTIVE ELEMENTS OF RESEARCH 1. Chapter 4 • Orienting and Supportive Elements of a Journal Article 2. Chapter 5 • Peer-Reviewed Journals 7. PART IV • RESEARCH JUSTIFICATIONS, AUGMENTATION, AND RATIONALES 1. Chapter 6 • Introduction and Research Questions 2. Chapter 7 • Literature Review 8. PART V • RESEARCH DESIGN AND RESEARCH METHODS 1. Chapter 8 • Overview of Research Design and Methods 2. Chapter 9 • Reporting Qualitative Research Methods 3. Chapter 10 • Reporting Quantitative Methods and Mixed Methods Research 9. PART VI • RESULTS AND FINDINGS 1. Chapter 11 • Statistical Literacy and Conventions 2. Chapter 12 • Descriptive and Inferential Statistics 3. Chapter 13 • Results and Findings 10. PART VII • DISCUSSION, CONCLUSIONS, AND RECOMMENDATIONS 1. Chapter 14 • Discussion 2. Chapter 15 • Conclusions 3. Chapter 16 • Recommendations 11. PART VIII • ARGUMENTATIVE ESSAYS AND THEORETICAL PAPERS 1. Chapter 17 • Argumentative Essays: Position, Discussion, and Think-Piece Papers 2. Chapter 18 • Conceptual and Theoretical Papers 12. Appendix 13. References 14. Index 7
Detailed Contents Preface Acknowledgments About the Author PART I • INTRODUCTION Chapter 1 • Critical Research Literacy Introduction to the Book Research Literacy Defined Benefits of Research Literacy Critical Research Literacy Balanced Appraisals Judging Chains of Reasoning Skeptical and Critical Scrutiny Uncritical Readings of Research Misplaced Trust Overindulgent and Undiscriminating Surface-Level Reading Reading Deeper for Ideologies and Paradigms Ideologies Paradigms Critically and Uncritically Reading Research Chapter Summary Review and Discussion Questions PART II • PHILOSOPHICAL AND THEORETICAL ASPECTS OF RESEARCH Chapter 2 • Research Methodologies Introduction Conceptual Confusion, Slippage, and Clarity Research Paradigm Research Methodology Research Traditions Confusion Ensues Theory and Method Choices Methodological Approach Used in This Book Methodological Responsibility in an Ideal World Philosophical Axioms Positivism and Postpositivism Positivistic Research Paradigm Postpositivistic Research Paradigm 8
Empirical, Interpretive, and Critical Methodologies Habermas’s Theory of Communication Matching Methodology With Research Intent Quantitative, Qualitative, and Mixed Methods Methodologies Quantitative and Qualitative Methodologies Mixed Methods Methodology (Mixing Assumptions) Research Methodology and Research Question Alignment Writing the Research Methodology Section of a Paper Chapter Summary Review and Discussion Questions Chapter 3 • Conceptual Frameworks, Theories, and Models Introduction Role of Theory in Research Inquiries Descriptive Theory Typology Taxonomy Relational Theory Relational Research Explanatory (Predictive) Theory Qualitative Explanatory Concepts and Constructs Characteristics of a Good Concept and Construct Distinguishing Among Conceptual Frameworks, Theories, and Models Terminology Confusion Conceptual Frameworks Presenting a Conceptual Framework in a Research Paper Theories Overarching Types of Theories Scientific Theories Qualitative Theories Philosophical Theories Mathematical Theories and Theorems Disciplinary Theories Purposes of Theory in Research Deductive Versus Inductive Theory Elements of a Theory Assumptions Propositions Hypotheses Characteristics of a Good Theory 9
Writing the Theoretical or Conceptual Framework Section of a Paper Theory in Quantitative Papers Theory in Qualitative Papers Elucidate Essence of the Theory Justify Theory Selected for Study Use Theory to Interpret Data Critiquing Theory in a Paper Models Chapter Summary Review and Discussion Questions PART III • ORIENTING AND SUPPORTIVE ELEMENTS OF RESEARCH Chapter 4 • Orienting and Supportive Elements of a Journal Article Introduction Orienting Secondary Elements Supportive Secondary Elements Authors’ Names, Credentials, and Affiliations Article Title Characteristics of a Good Title Abstracts Formatting Abstracts Keywords References Plagiarism and Paraphrasing Citation Styles Footnotes and Endnotes Appendices Formatting Appendices Acknowledgments Biographies Chapter Summary Review and Discussion Questions Chapter 5 • Peer-Reviewed Journals Introduction Journal Editor, Editorial Boards, and Facilitating Editors Guidelines for Authors Roster of Peer Reviewers Editorial Board Journal Publishers Peer Review Process Types of Peer Review 10
Open and Extended Peer Reviews (Prepublication) Postpublication Peer Review Acceptance Rates Journal Impact Factors Journal Title Journal Volume, Issue, and Pagination Year of Publication and Article Placement in Issue Special Topic and Themed Issues Electronic Journals Open Access Journals Predatory Open Access Journals Chapter Summary Review and Discussion Questions PART IV • RESEARCH JUSTIFICATIONS, AUGMENTATION, AND RATIONALES Chapter 6 • Introduction and Research Questions Introduction Etymological Roots Lay Versus Research Questions Research Topics, Problems, Questions, Statements, and Objectives Research Topics Research Problems Quantitative Research Problems Qualitative Research Problems Mixed Methods Research Problems Casuist, Theoretical, and Practical Research Problems Research Goals Basic Versus Applied Research Sources of Research Problems Characteristics of a Good Research Problem Research Questions or Hypotheses Qualitative Research Questions Quantitative Research Questions and Hypotheses Mixed Methods Research Questions Research Statements Quantitative Research Statements Qualitative Research Statements Mixed Methods Research Statements Research Objectives Opening Sentences and Opening Points in an Introduction Six Approaches to Writing Introductions 11
5W Framing of an Introduction Three-Phase Approach (Niche) to Introductions Five-Paragraph Model for Introductions Four-Paragraph Model for Introductions Research Argument Model Problem Statement (Propositional) Model for Introductions Delimitations Chapter Summary Review and Discussion Questions Chapter 7 • Literature Review Introduction Etymology and Definition of Literature Review Purposes of a Literature Review When to Conduct a Literature Review Quantitative Literature Review Conventions Qualitative Literature Review Conventions Mixed Methods Literature Review Conventions Conducting a Literature Search Keywords and Search Engines Types and Sources of Documents Primary and Secondary Sources Steps for Searching and Finding Documents Selecting Studies for the Literature Review Ending the Search Critically Reviewing the Collection of Literature Writing the Literature Review Section Synthesis Synthesis Matrix Organizational and Presentation Strategies Historical Chronological Conceptual Theoretical Thematical Methodological (Meaning Methods) Major Ideas and Trends Argumentative Hypotheses Characteristics of a Good Literature Review Critical 12
Important Relevant Timely Coverage Point of View Rigorous, True Accounting Technical Elements of Writing Effective Literature Review Sections Person and Voice Tense Length of Literature Review Section Number of References Transitioning From a Summary, Critical Evaluation to a Gap Transitioning From One Major Point to Another Minimize Quotations, Responsibly Paraphrase Chapter Summary Review and Discussion Questions PART V • RESEARCH DESIGN AND RESEARCH METHODS Chapter 8 • Overview of Research Design and Methods Introduction Etymology and Definition of Methods and Research Design Research Design Research Inquiry and Research Design Articulating Research Purpose in Research Design Research Design as Logical and Logistical Quantitative Research Design Logic Qualitative Research Design Logic Mixed Methods Research Design Logic Most Common Research Designs Methods Methodology Versus Methods Purpose and Importance of the Methods Section Major Differences Between Qualitative and Quantitative Intellectual Inquiry Major Components (Report Subheadings) of Qualitative and Quantitative Research Integrity of Research Designs Integrity of Qualitative and Quantitative Research Designs Integrity of Mixed Methods Research Designs Technical Aspects of Reporting Methods Length Organizational Logic and Approaches Objective Versus Subjective Writing 13
Person, Tense, and Voice Person Tense Voice Chapter Summary Review and Discussion Questions Chapter 9 • Reporting Qualitative Research Methods Introduction Qualitative Emergent Research Designs Strategic Versus Tactical Research Designs Types of Qualitative Research Designs Interpretive Investigative Participatory Illuminative Instrumentation Sensitization Conceptualization and Theory Development Methods Appropriate for Qualitative Research Qualitative Site Selection and Access Qualitative Sampling Sample Size Researcher’s Role Qualitative Ethical Considerations Qualitative Data Collection Observations and Field Notes Individual Interviews Focus Group Interviews Document and Artifact Analysis Qualitative Data Management, Security, and Storage Integrity of Qualitative Research Designs Qualitative Data Analysis Data Interim Analysis Data Analysis Spiral Coding and Patterns Themes in Qualitative Data Computer Qualitative Software Programs Qualitative Study Limitations Chapter Summary Review and Discussion Questions 14
Chapter 10 • Reporting Quantitative Methods and Mixed Methods Research Introduction Quantitative Research Design Scientific Method Deductive Reasoning Types of Quantitative Research Designs Descriptive Correlational Comparative Experimental Quasi-experimental Predictive Exploratory Survey Design (Nonexperimental) Integrity (Rigor) of Quantitative Research Designs Quantitative Instrument, Procedures, and Apparatus Quantitative Variables Quantitative Sampling Probability Sampling Random Assignment Sample Size and Representativeness Quantitative Ethical Considerations Quantitative Data Collection Quantitative Validity and Reliability Quantitative Data Analysis Quantitative Study Limitations Mixed Methods Research Design Philosophical Quandary Mixed Methods Main Research Designs Mixed Methods Degree of Data Integration Order of Mixing Methods to Ensure Data Integration Priority of Qualitative and Quantitative Methods Rigor in Mixed Methods Studies Rationale for Mixing Methods Chapter Summary Review and Discussion Questions PART VI • RESULTS AND FINDINGS Chapter 11 • Statistical Literacy and Conventions Introduction History of Statistics Laws of Probability and Chance 15
Statistical Literacy Types of Statistical Data Nominal Data Ordinal Data Interval Data Ratio Data Measurement Scales Statistical Variables Dependent and Independent Variables Criterion and Predictor Variables Constants Controls Moderator Variables Intervening Variables Extraneous Variables Confounding Variables Spurious Variables Experimental and Nonexperimental Research Experimental Research Types of Settings Randomization Types of Experiments: True, Factorial, Quasi, and Single Case Nonexperimental Research Group Comparisons Correlational Surveys Case Studies Chapter Summary Review and Discussion Questions Chapter 12 • Descriptive and Inferential Statistics Introduction Basic Differences Between Descriptive and Inferential Statistics Descriptive Statistics Measures of Central Tendencies Measures of Variation Bell Curve Standard Deviation Measures of Relative Standing Measures of Relationships Contingency Tables 16
Regression Analysis Reporting Descriptive Data Association and Causation Causal Inference and Inference of Association Moving From Association to Causation Recommended Causation and Association Vocabulary Inferential Statistics Hypothesizing Parameters (Hypotheses Testing) Hypothesis Testing Parameter Estimations Single-Point and Interval Estimates p Value and the Level of Significance Type I and II Errors Statistical Power (and Statistical Significance) Statistical Significance Sample Size and Effect Size Most Common Inferential Statistical Tests Reporting Inferential Statistics Conclusion Chapter Summary Review and Discussion Questions Chapter 13 • Results and Findings Introduction Distinguishing Between Results and Findings Purpose of Results or Findings Section With or Without Interpretation Qualitative Papers Quantitative Papers Jewel and Story Metaphors Reporting Qualitative Findings Organizing the Presentation of Findings Descriptions Versus Interpretations Formats for Reporting Qualitative Findings Research Purpose and Reporting Findings Themes Major Topics Using Quotations How Many Quotes? Presenting Quotes Thick Descriptions 17
Relating Qualitative Findings to the Literature Review Using Numbers to Report Qualitative Findings Length Tense, Person, and Voice Reporting Quantitative Results Interpretation or Noninterpretation Comprehensiveness Negative Results Missing Data Organizing Quantitative Results Presenting Numerical Results Text Tables Symbolic, Visual Representations Transitioning From Results to Discussion Length of Results Section Person, Voice, and Tense Reporting Mixed Methods Results and Findings Principle of Integration Barriers to Integration Meta-inferences Integration via Connecting, Merging, or Embedding Connecting Merging Embedding Narratives Data Transformation Joint Data Displays Triangulation Fit of Data Integration (Coherence) Organizing Mixed Methods Results and Findings Section Number and Nature of Papers to Publish and Where Conclusions: Unique Challenges of Three Approaches Chapter Summary Review and Discussion Questions PART VII • DISCUSSION, CONCLUSIONS, AND RECOMMENDATIONS Chapter 14 • Discussion Introduction Discussion Defined Discussion Versus Conclusion 18
Discussion Relative to Results and Findings Purpose of Discussion Summarize Salient Results and Findings Explain Results and Findings Speculation and Hedging Examine Implications of Results and Findings Discussion and the Concept of Significance Dimensions of Research Significance Significance Versus Novelty Significant Versus Statistically Significant Theoretical Significance Fundamental Questions to Address in the Discussion Limitations Threats to Validity Internal Validity External Validity Temper Limitations Humility and Limitations Organizing the Discussion Section Summarize Key Outcomes, Then Systematically Discuss Organize From Specific to General Mixed Methods Studies Organize and Discuss by Hypotheses Research and Null Hypotheses Retaining the Null Hypothesis Discussing Unhypothesized Results Statistically Correct Language Organize by Research Questions or Themes Staying Focused and On Topic Adding New Literature in the Discussion Quantitative Research Qualitative Research Mixed Methods Research Adding New Results or Findings Patterns in the Data Stay on Point, Tangential Issues Ensuring Strong and Responsible Discussions Employ Alternative Interpretations Bring Alternative Interpretations to Unexpected Findings or Results Critical and Creative Perspectives 19
Technical Strategies for Discussion Sections Length of Discussion Section Shortening the Length Tense, Voice, and Person Chapter Summary Review and Discussion Questions Chapter 15 • Conclusions Introduction Conclusion Defined Relating Conclusions to Discussion and Summaries Stand-Alone Conclusion or Combined With Discussion Summaries Versus Conclusions Summary Conventions Purpose of Conclusions Explore Broader Implications Suggest New Research Place Research in Disciplinary Context Identify Practice and Policy Innovations Closing Strategies for Effective Conclusions Provocative Question Quotations Anecdotes Profound or Provocative Thought Drama Call for Action Warning Relevant Narrative Things to Avoid in the Conclusions Section Do Not Apologize Avoid Emotional Appeals No New Information Technical Strategies for Writing Effective Conclusions Length of Conclusions Tense, Person, and Voice Vocabulary (Move Statements) Monosyllabic Words Syntax Chapter Summary Review and Discussion Questions Chapter 16 • Recommendations 20
Introduction Recommend Defined Logical Connections Informing Recommendations Relating Recommendations to Conclusions Research Versus Nonresearch Recommendations Purpose of Recommendations Audiences for Recommendations Characteristics of Effective Recommendations Language for Communicating Recommendations Actionable Verbs Imperative Verbs Modal Verbs Modal Force Certainty Possibility Obligation Crafting and Organizing Effective Recommendations Scope of Recommendations Prose Versus List (Numbered or Bulleted) Options for Presenting Lists of Recommendations Stand-Alone and Supportive Statements Technical Strategies for Presenting Recommendations Length Active and Passive Voice Grammar Chapter Summary Review and Discussion Questions PART VIII • ARGUMENTATIVE ESSAYS AND THEORETICAL PAPERS Chapter 17 • Argumentative Essays: Position, Discussion, and Think-Piece Papers Introduction Argumentative Essays Western Logic Comparing Position, Discussion, and Think-Piece Papers Position (Opinion) and Discussion Papers Length, Person, Tense, and Tone Transition Words Think-Piece Papers Length and Format Format for Argumentative Papers Introduction/Thesis 21
Body/Antithesis Evidence Conclusion/Synthesis Persuasive Writing and Rhetorical Appeals Aristotle’s Three Forms of Rhetoric Rhetorical Triangle Ethos Pathos Logos Deductive and Inductive Argumentation Deductive Argument Inductive Argument Logical Fallacies Fallacies Chapter Summary Review and Discussion Questions Chapter 18 • Conceptual and Theoretical Papers Introduction Chapter Caveats Steps in Developing a Theory Choosing a Theory for a Study Double Meaning of Conceptual Framework Purpose of Conceptual Versus Theoretical Papers Purpose of Conceptual Papers Purpose of Theoretical Papers Structural Elements of Conceptual and Theoretical Papers Special Role of the Literature Review Theory Paper Literature Review Conceptual Paper Literature Review (Role of Integration) Elements of a Conceptual Paper Introduction Literature Review Presentation of the New Conceptualization Discussion and Implications Conclusions Elements of a Theoretical Paper Title and Abstract Introduction Context for the New Theory Presentation of the New Theory 22
Discussion Conclusions Stage of Theory Being Published in a Theoretical Paper Conceptual Development Operationalization Confirmation or Disconfirmation (Testing) Application Rigor in Conceptual and Theoretical Papers Modes of Reasoning y-Axis Judging Reasoning and Argumentation Chapter Summary Review and Discussion Questions Appendix Worksheet for Critiquing a Journal Article or Research Report References Index 23
24
Preface The inspiration for this book came while I was teaching graduate-level courses on research literacy (2010– 2014). I could not find a book that contained everything I thought should be taught in a course about how to critically read and use someone else’s scholarship. While teaching the course five times, I prepared a roster of detailed PowerPoint presentations about ideas not found in related texts. These topics especially included (a) the powerful link between research methodology and everything else in the research enterprise; (b) the far- reaching role played by conceptual frameworks, theories, and models in research; and (c) the criteria for judging argumentative essays and theoretical papers, which differ so much from a conventional research report. I could find “methods textbooks” in abundance (how to do research yourself), but less so textbooks about how to be research literate (how to critique someone else’s research and the final report). I remember thinking that if I, a seasoned academic with 30 years of experience, did not know this “stuff,” then others might not either. With this in mind, I approached SAGE with an idea for such a book, and signed off on the project mere weeks later. Kudos to Acquisitions Editor Helen Salmon for her confidence in this idea. The publisher realized that a book like this was needed. The result is this book titled Understanding and Evaluating Research, focused on helping people learn how to read, critique, and judge research reports prepared by others. The assumption is that one cannot critically read a research report without being aware of the conventions that authors are supposed to follow when conducting and reporting their research. As a caveat, the book does not strive to teach people to do their own research, but readers may learn more about how to do research by appreciating research conventions as they vary by methodology. 25
Part I The book comprises 18 chapters, divided into eight parts. In Part I, the Introduction, Chapter 1 (“Critical Research Literacy”) begins by describing the approach I took to the book. Each chapter is prefaced with learning objectives, a sort of chapter abstract. To help readers engage with and apply chapter content, critical review questions, called Review and Engagement, are dispersed throughout each chapter. The premise is that after readers complete a critical assessment for each research component in a research report, they will complete the Appendix to help them decide if they can confidently use the study and its conclusions in their practice. The chapters all conclude with a wrap-up summary, followed with Review and Discussion Questions. I recommend you pose your own reflection and discussion questions as well. Your chapter takeaways will not be the same as mine. Chapter 1 then focuses on what it means to critically and uncritically read a research report. An uncritical reading means accepting the study without questioning it. 26
Part II Part II is called Philosophical and Theoretical Aspects of Research. It begins with Chapter 2, which is the philosophical anchor for the book. Titled “Research Methodologies,” it reinforces the point that the philosophical underpinnings of research determine everything from the research question to the reporting format, and everything in between. After describing philosophical axioms, positivistic and postpositivistic research paradigms, and empirical, interpretive, and critical research methodologies, I confirm that the book is organized using the qualitative, quantitative, and mixed methods approach to categorizing methodologies. The chapter concludes with a discussion of the importance of aligning research methodology and research questions, and with some basic conventions for writing the research methodology section of a research report. Still in Part II, Chapter 3 is titled “Conceptual Frameworks, Theories, and Models.” It makes the case that without frameworks, theories, and models, research is weakened. The chapter discusses concepts and constructs, the building blocks of all three, followed with (a) a detailed discussion of conceptual frameworks, (b) an extensive coverage of theories, and (c) a shorter overview of models. The chapter also covers the reporting conventions pursuant to writing the conceptual and theoretical framework or model sections of a research paper. 27
Part III Part III is called Orienting and Supportive Elements of Research. These terms were coined for this book as a way to recognize the importance of their role in reporting research. They act as bookends for the major (primary) components of a research report (methodology, theoretical underpinnings, introduction and research questions, literature review, methods, results or findings, discussion, conclusions, and recommendations). The orienting elements bring the paper to people’s attention (authors’ names, credentials, and affiliations; article title; abstract; and, keywords). The supportive elements help authors acknowledgments, contributions to the research (references, footnotes and endnotes, appendices, acknowledgements, and biographies). Chapter 4 (“Orienting and Supportive Elements of a Journal Article”) provides a rich discussion of each of these nine research elements, which are characterized as secondary, meaning they are important but for different reasons than the primary elements. Chapter 5, titled “Peer-Reviewed Journals,” focuses on the most common publication venue for research reports. Critical readers often use elements of the journal to judge the rigor of the scholarship, whether it is wise to do so or not. Starting with the role of journal editors, other types of editors, and editorial boards, the discussion shifts to the peer review process, acceptance rates, and journal impact factors. The discussion then turns to the journal’s title; volume, issue, and pagination; year of publication and the paper’s placement in the issue; and special or thematic issues. The chapter deals extensively with the new publication model of electronic journals, and discusses the possible threats to quality and rigor of open access journals, especially predatory journals. 28
Part IV Part IV deals with issues pertaining to Research Justifications, Augmentation, and Rationales. Chapter 6 (“Introduction and Research Questions”) distinguishes between opening points (six types) and an introduction. Opening points lead the reader into the introduction, and the introduction leads people into the paper. The chapter covers research problems, questions, purpose (statements), and objectives. These are all discussed in extensive detail (as they pertain to qualitative, quantitative, and mixed methods) because savvy authors and critical readers need a clear understanding of how they differ yet are inherently interconnected. Next comes an extensive overview of six approaches that authors can use to develop an argument supporting what they are doing in their study. These approaches help authors convince people of the merit of their study, namely by critically reviewing literature related to the research problem, identifying what is missing (justifying their research question), and explaining how their study will fill the gap (research statement and objectives). Chapter 7 (“Literature Review”) covers how the literature review section of a research paper differs with qualitative, quantitative, and mixed methods research methodologies. Quantitative studies do the review before collecting the data, and qualitative studies tend to wait and do the review after the data are collected and are being analyzed. The chapter covers what is involved in critically engaging with previous work done about the research question, and explains in detail the process of synthesizing insights gained from this critical engagement. Nine organizational strategies are identified and discussed, showcasing the many choices authors have about how to present their review of the literature. 29
Part V The next part is titled Research Designs and Research Methods. It comprises three chapters. Chapter 8 (“Overview of Research Design and Methods”) focuses on how the research methodology deeply affects the research design and the methods employed in a study. After distinguishing between methodology (philosophy, introduced in Chapter 2) and method (tasks to sample, collect, and analyze data), the discussion turns to how method and research design are different (but often used interchangeably). Readers learn the differences between qualitative and quantitative inquiries, followed with a general overview of the basic steps involved in each of qualitative and quantitative research designs. These steps are then used to organize the extensive, wide-ranging, comprehensive profile of both qualitative and quantitative methods, including seven qualitative research designs (Chapter 9, “Reporting Qualitative Research Methods”) and seven quantitative research designs (Chapter 10, “Reporting Quantitative Methods and Mixed Methods Research”). Chapter 10 concludes with a discussion of the mixed methods approach, which involves using both qualitative and quantitative strands, informed by the principle of data integration. In addition to design rigor, mixed methods research has a criterion of interpretative rigor, concerned with the validity of the conclusions (strand-specific and meta-inferences). 30
Part VI Part VI, Results and Findings, has three chapters. After addressing what it means for authors and critical readers to be statistically literate, Chapter 11 (“Statistical Literacy and Conventions”) turns to the four main types of data (nominal, ordinal, interval, and ratio) and measurement scales, and then provides an overview of the nine most commonly used statistical variables. The chapter ends with the distinctions between experimental research designs (true, factorial, quasi, and single-case) and nonexperimental research designs (group comparisons, correlational research, surveys, and case studies). The criteria of randomization, variable manipulation, experimental and control groups, types of claims (causal and association), and types of research settings are used to differentiate between these two basic quantitative research designs. Chapter 12 (“Descriptive and Inferential Statistics”) starts with a discussion of the four types of measurements for descriptive statistics (i.e., central, variable, relative, and relationships), which includes a detailed overview of such basic conventions as standard deviation, the bell curve, contingency tables, chi-square, regression correlational coefficients, and scattergrams. The topic then shifts to inferential statistics, prefaced with a section on association and causation. Inferential statistics is concerned with (a) hypotheses and estimates, (b) p value and level of significance, (c) Type I and II errors, (d) statistical power (sample size and effect size), and (e) the most common inferential statistical tests. Chapter 13 (“Results and Findings”) discusses conventions for authors to follow when conducting and reporting the outcomes of their study. As with most other chapters in the book, it is organized by research methodologies: qualitative, quantitative, and mixed methods. Some challenges for qualitative research are balancing both describing and interpreting the data, providing supportive evidence for the existence of a theme, learning the protocols around using participants’ quotations, and providing thick and rich descriptions (so as to ensure high-quality findings that can be trusted). The section on quantitative results deals with issues of comprehensiveness (when answering the research question or hypothesis), missing data and negative results, organizational strategies for presenting results, and conventions for reporting numbers (text, tables, and figures). The chapter ends with an overview of reporting protocols and conventions for mixed methods studies, especially the principle of data integration, and the steps/strategies researchers can take to ensure integration before data collection, during analysis, or both. The importance of moving from strand-specific inferences to meta-inferences is discussed as are barriers to integration in mixed methods studies. The importance of addressing the fit of data integration is followed with various organizational approaches and a discussion of how and where to publish the study to reflect the degree of data integration. 31
Part VII Part VII, Discussion, Conclusions, and Recommendations, provides a chapter on each research element. Chapter 14 (“Discussion”) begins by distinguishing between discussion points and concluding remarks, followed with an overview of the different conventions for relating results and findings to a discussion of their import. Then, the three main functions of a discussion section are addressed (summarize, explain, and examine), as are the topics of hedging and speculation. Discussion sections should address three basic questions, dealing with how the study (a) relates to existing understandings in the literature, (b) brings new understandings, and (c) makes new connections. After discussing these in some detail, the conversation turns to study limitations and how authors need to explain how threats to validity affect their conclusions and implications. The conversation then focuses on five major approaches to organizing a discussion section, shifting to an overview of strategies to keep the discussion section on track and focused. After explaining the importance of employing alternative interpretations of the results and findings, it ends with the role of critical and creative thinking when preparing discussion points. Chapter 15 (“Conclusions”) begins by advising authors of the need to appreciate how concluding comments differ from, but are related to, discussion points, and how concluding comments relate to a summary of the paper. The focus then turns to the four main purposes of a conclusion section: explore broader implications; suggest new research; relate study to existing literature; and identify policy, practice, and theoretical implications. Next, the chapter profiles eight closing strategies for a conclusions section, ranging from provocative questions to warnings and personal narratives. Things to avoid when creating concluding remarks are also discussed, especially apologies and new information. If done well, the concluding thoughts make the paper resonate with readers, increasing the likelihood they will engage with the study in the future. Chapter 16 (“Recommendations”) deals with the section of a research report where authors call people to action. After distinguishing between conclusions and recommendations (and the different roles they play), the discussion turns to the nine purposes that recommendations can serve, followed with an overview of the eight characteristics of effective recommendations. The chapter then turns to the topic of verbs when writing recommendations, with a special focus on the 10 modal verbs and on modal force (the writer’s conviction when posing the recommendation). There are three modal forces (certainty, possibility, and obligation), and each is recognizable through the use of particular modal verbs. An extensive overview of strategies for crafting, presenting, and organizing a recommendation section is provided. The chapter ends with some basic grammatical conventions, especially the use of passive and active voice. 32
Part VIII Part VIII, the final section of the book, is called Argumentative Essays and Theoretical Papers. The two chapters in this section are separated from the rest of the book because papers reporting research related to positions, opinions, think pieces, new conceptual frameworks, and new theories do not fit with the basic reporting conventions previously discussed in Parts II and IV–VII. Chapter 17 (“Argumentative Essays: Position, Discussion, and Think-Piece Papers”) focuses on argumentative essays, which are judged by the ability of the author to develop a convincing and well-reasoned argument. The chapter begins by comparing and contrasting the three most basic forms of argumentative essays: position (opinion) papers, discussion papers, and think pieces. This clarification is followed with a detailed overview of the format for writing an argumentative essay: thesis, antithesis, and synthesis. The discussion then turns to the concept of persuasive writing and the key role played by Aristotle’s rhetorical appeals: ethos (character), pathos (emotions), and logos (logic). The chapter then presents a very detailed discussion of both deductive and inductive argumentation, ending with a brief introduction to logical fallacies in argumentative essays. Chapter 18 (“Conceptual and Theoretical Papers”) is the last chapter in the book. Whereas Chapter 3 is about the role conceptual frameworks and theories play in the research design, and how to use and report on their use in a paper, this chapter is about how to write a paper that reports on a new conceptual framework or a new theory. They both require different formats. The chapter especially addresses the special role that the literature review plays in each paper. It is an integrative instrument in a conceptual paper and a theoretical-step instrument in a theory paper. The chapter then profiles the key structural elements in both types of papers, distinguishing between each one, explaining that they become the organizational subheadings in the published paper. What constitutes a good theory is addressed as well. The conversation turns to Lynham’s idea of what to include in a theory paper, determined by how far along it is (ranging from the conceptual stage to being successfully applied in practice). The chapter concludes with a detailed discussion of what criteria to use to judge the rigor of a conceptual or a theoretical paper—mainly its supportive, red-thread argument—and the author’s mode of reasoning (analytical and generative). 33
Acknowledgments First and foremost, I want to thank my husband, Peter, for his intellect, deep patience, powerful critical listening skills, and ability to make me reflect and accept things when I am in pushback mode—all part of the creative process. Thanks to my longtime friends Liz Goldsmith, Anne MacCleave, Felicia Eghan, and Vicki Jeans for lending supportive and critical ears during this creative process. Nearer to the end of the writing process came Frank Miccolis and Chrisanne Moffatt Miccolis, new boating friends who were incredibly patient with me while I unloaded my stress and expressed my excitement! My mom, Ethel Tweedie, has always been there for me during every life endeavor, as has my nephew, Connor Hunter Tweedie Morton. This journey about research methodologies and their link to scholarship began with a doctoral student turned friend, Dr. Jenny Murnane. Our work formed the cornerstone of the book (see Table 2.1). It was followed with an opportunity to teach Focus on Research Literacy in our education graduate program. Thanks to Jim Sharpe, then the dean, for patiently waiting until I felt I was ready to teach it. And those students! They were something, approaching this overwhelming topic with anticipation, undaunted grace, and open-mindedness. I would not have written the book without them and their inspiration. Finally, thanks to the SAGE editors and their assistants. SAGE gratefully acknowledges the following reviewers for their kind assistance: Kristin Bodiford, Dominican University Kadir Demir, Georgia State University Scott Greenberger, Grand Canyon University Tanya Kaefer, Lakehead University Xyanthe Neider, Washington State University Carole S. Rhodes, Queens College, City University of New York Anita Rose, New York University 34
About the Author Sue L. T. McGregor (PhD, IPHE, Professor Emerita) is a Canadian home economist (nearly 50 years) recently retired from Mount Saint Vincent University in Halifax, Nova Scotia. She was one of the lead architects for the interuniversity doctoral program in educational studies, serving as its inaugural coordinator. She has a keen interest in home economics philosophy, transdisciplinarity, research methodologies, and consumer studies. She is a TheATLAS Fellow (transdisciplinarity), a Docent in Home Economics at the University of Helsinki, recipient of the Marjorie M. Brown Distinguished Professor Award (home economics leadership), a Rhoda H. Karpatkin International Consumer Fellow, and recipient of the TOPACE International Award (Berlin) for distinguished international consumer scholars, especially consumer educators. Affiliated with 16 professional journals, she has nearly 170 peer-reviewed publications, four books, 31 book chapters, and 12 monographs. She has delivered 39 keynotes and invited talks in 14 countries. Dr. McGregor is also the principal consultant for McGregor Consulting Group (founded in 1991). Visit www.consultmcgregor.com, or contact her at sue.mcgregor@msvu.ca. 35
Part I Introduction 36
1 Critical Research Literacy 37
Learning Objectives Clearly understand the unique focus of this book Describe the basic elements of a research report Define research literacy and describe the benefits of being research literate Explain the basic process of critiquing a research report: read, critically analyze, and assess (judge) Distinguish between research literacy in general and critical research literacy Become familiar with five things to look for when critically reading research reports Understand reasons why people may not critically read (judge) a research report Explain the power of ideologies and paradigms in research, and explain what deep reading means Compare and contrast critical and uncritical thinking and reading of research reports 38
Introduction to the Book This is an introductory-level text in the sense that introduce stems from Latin ducere, “to lead” (Harper, 2016). This book strives to lead people through the research process by using a critical lens—hence, the title Understanding and Evaluating Research: A Critical Guide. The book is intended to help people learn how to critique other people’s research so they can use it with confidence. It is written in third person, explaining what authors should do at each stage of the research process while they respect the research conventions pursuant to the methodological (philosophical) approach used in their study (see Chapter 2). As readers gain a deeper appreciation of the expectations placed on researchers and authors to conduct and report research of high standards, they will concurrently learn how to critique the resultant study and the research report. In effect, the book is not intended to teach readers how to do research on their own. They will learn about it by way of osmosis, which is the gradual, often unconscious absorption of knowledge or ideas through continual exposure rather than deliberate learning (Anderson, 2014). As a way to learn during this process, readers should choose a journal article or research report they want to critically read. As they read each chapter in this book, they should engage with its content by applying it to critique the research report. This book facilitates this critical reading process by providing Review and Engagement checklists, which are strategically positioned throughout the chapters, close to the respective content. For example, as people read and learn about the conventions authors should follow when preparing the Methods section of a research report (see Chapter 8), they would critically review the paper’s Methods section to see if it meets the recommended standards for high quality. To help bring readers to this assessment, each chapter ends with a question such as “Taking all of the Review and Engagement criteria into account, what is your final judgment of the Methods section of the paper you are critically reading?” Figure 1.1 summarizes the basic stages of critiquing a research report (often a journal article). These steps include reading it, identifying the elements used to organize it, rereading it while critically analyzing and judging each element, and then assessing (judging) the entire paper for its quality as scholarship. The Appendix (available for download at https://study.sagepub.com/mcgregor) contains a document developed to help readers track their critical judgment of each separate element of a research report, leading to a cumulative judgment of the quality of that entire study and its formal report. For example, if the keywords, title, and abstract were judged inadequate but the methods, the results, and their discussion were clearly and rigorously reported, it would make sense to confidently judge the paper as good quality. On the other hand, if the authors used a catchy title, chose keywords wisely, and had a solid reference list but the methods, the findings, and their discussion were inadequately documented, it would make sense to lower the judgment of the research report (and perhaps the research enterprise itself). All chapters conclude with Review and Discussion Questions, another tool for engaging with the content of the specific chapter so as to better apply it when critically reading a research report. Furthermore, noting the etymological roots, origins, and meanings of the basic research terms adds linguistic flavor to each chapter. To illustrate, consider the phrase critical research literacy, the title of this chapter. Critical is Latin criticus, “to pass judgment on literature.” Research is Old French recercher, “intensive seeking.” Literacy is Latin literatus, “the 39
quality or rank of being educated, or learned” (Harper, 2016). Being critically research literate thus means that someone has, through education, learned how to critically judge an effort to intensely seek and report something—in this case, new knowledge emergent from a study. Figure 1.1 Process of Critiquing a Journal Article This etymological example frames the main focus of this book, which is to help readers develop the ability to critically assess authors’ studies and research reports. By association, authors reading this chapter can gain insights into how to ensure their research study and the report meet the expected rigor of a critical perspective. This approach mirrors contemporary definitions of research literacy, which include being able to function, respectively, as critical readers and critical producers of research (Dryden & Achilles, 2004; Ingham- Broomfield, 2008; Shank & Brown, 2007; Williams, Mulkins, Verhoef, Monkman, & Findlay, 2002). To that end, this first chapter explains the concepts of (a) research literacy and (b) critical and uncritical readings of research. It serves as a preamble to separate chapters on each of the many elements of the research process—that is, what authors need to know and do when conducting and reporting research, and what critical readers need to know to judge its quality (see Figure 1.2). Separate chapters are provided for understanding the basics of statistical literacy (Chapters 11 and 12), for writing an argumentative essay (Chapter 17), and for writing a conceptual paper or a theoretical paper (Chapter 18). These types of papers do not follow the basic conventions for a research report. Figure 1.2 Basic Elements of a Research Report 40
41
Research Literacy Defined The term research literacy comprises two concepts, research and literacy. Research is Modern French rechercher, “to examine closely, to see, to look for.” As noted earlier, it also stems from Old French recercher, with re, “intensive,” and cercher, “seeking.” Research thus involves the intensive, concentrated search for knowledge. Literacy is Latin literatus, “educated, or learned.” In the word literacy, the suffix -cy means “quality or rank” (Harper, 2016). Literacy thus means having the quality of being knowledgeable or educated and exhibiting the ability to recognize and understand ideas, in this case, about research. Taken together, research literacy can be defined as the ability to locate, understand, critically evaluate, and apply scholarly works—that is, to become discerning and knowledgeable about research (i.e., the search for new knowledge) (Dryden & Achilles, 2004). Beyond these etymological definitions, research literacy is defined as understanding research language and conventions. It is a foundational block of research capacity, defined as “the ability to design and conduct [and report] research studies” (Williams et al., 2002, p. 14). Shank and Brown (2007) proposed that the term research literate refers to scholars being literate in how to conduct and report research. But it also refers to those who are reading the resultant reports. They need to be able to judge that scholarship before they use it. 42
Benefits of Research Literacy Achieving research literacy is important for several reasons. Foremost, it helps alleviate the fear of not being able to access and assess research and scholarly publications. The more research literate people become, the more confidence they gain in judging others’ work. Research literacy entails both criticizing and critiquing scholarship, privileging the latter. Both of these words have the same Latin root, criticus, “to pass judgment on literature” (Harper, 2016). But while (a) criticizing the research process means finding fault with it, (b) critiquing it means assessing the strength of arguments and their supportive evidence, including the authors’ interpretations and conclusions drawn from their analysis and their discussion of the data. This accrued reader confidence is possible because, with experience, people build up knowledge of methodologies, theories, methods, and entire bodies of literature. Informed with this knowledge, confident readers are less inclined to erroneously rely on flawed scholarship. Research literacy encourages (actually necessitates) people to become critical readers and thinkers of others’ scholarly work (Kurland, 2000; Suter, 2012). Conversely, engagement with research enhances critical thinking and predisposes people to critique the scholarship. Indeed, critical readers of research both understand and engage with the research (Kattiyapornpong, Turner, Zutshi, Hagel, & Fujimoto, 2011). If people are engaged with something, they become involved with it, and it holds their attention (Anderson, 2014). This engagement means people go beyond merely reading the research report (superficial); rather, they dig deeper, holding the scholarship to high standards (Hart, Poston, & Perry, 1980; Shank & Brown, 2007). The next sections expand on this idea, in conjunction with being uncritical. Review and Engagement Critical readers will □ Appreciate what it means to be research literate in general □ Appreciate what it means to be critically (and uncritically) research literate (see Table 1.2) □ Recognize the benefits of being critically research literate □ Distinguish between being critical of and critiquing research □ Understand that critical engagement with a study means paying attention to, and getting involved with, it (not just a superficial, lay reading) □ Have a fundamental understanding of the basic elements of a research report (see Figure 1.2) and the conventions involved in preparing it, appreciating that most elements contain some margin of error 43
Critical Research Literacy Critical research literacy adds the dimension of judging the value of a study’s contribution to theory, knowledge, and practice. Recall that critical is Latin criticus, “to pass judgment on literature” (Harper, 2016), so being critical means engaging in and expressing the merits and faults of literary works (i.e., formal writings) (Anderson, 2014). Being critical of research means carefully evaluating the scholarly work (i.e., looking for biases, unspoken assumptions, underlying ideologies, prejudices, quality, and rigor) and then judging the research, being able to defend one’s position (see Figure 1.3). People cannot maintain a critical perspective when reading research unless they can unpack what it means to be critical and uncritical (Shon, 2015; Suter, 2012). Figure 1.3 What to Look for When Critically Reading Research Reports 44
Balanced Appraisals In more detail, Lunsford and Lunsford (1996) claimed that scholars and professional practitioners have a duty to critically review the literature and the research in their discipline and profession. Becoming “educated readers and interpreters of professional research literature” (p. 24) requires gaining the necessary skills and competencies to critically analyze research; that is, they have to learn to read research reports from a critical stance. Kurland (2000) agreed, claiming that readers have a responsibility to themselves and to others to monitor their reactions to a research report and to strive to understand the author’s point of view. This understanding has to occur before the text can be critiqued. The essence of the successful critique of a research paper is a balanced appraisal, meaning readers look for both merits and demerits (strength and weaknesses), achieved using logic and objectivity (Harris, 2014; Ingham-Broomfield, 2008). A balanced appraisal is accomplished via a logical and systematic assessment of the paper, grounded in critical awareness (see Figure 1.1). This means to strive to “be more questioning; try to see more than one side of an argument; try to be objective rather than subjective; weigh the evidence; make judgements based on reason, evidence or logic; look at the meaning behind the facts; identify issues arising from the facts; and recognise when further evidence is needed” (Ingham-Broomfield, 2008, p. 103). 45
Judging Chains of Reasoning Once the research report has been critically assessed, readers are able to decide what to accept as true and useful. To do this, they must evaluate the evidence and the argument used by authors to reach their conclusions (Kurland, 2000). Kurland (2000) suggested that readers have to take control of the text they are reading (prepared by someone else) and become authors of their own understandings of the facts and their meanings as framed and argued by the authors. Only when readers fully understand a text can they truly critically evaluate the authors’ assertions. This understanding entails unearthing the authors’ purpose, persuasive arguments, and inherent biases. Authors choose content, language, writing styles, argumentative rhetorical styles, and logics for a reason. More than careful reading, critical reading of a research report involves actively recognizing and judging these linguistic elements (Kurland, 2000). From another perspective, Kurland (2000) posited that critical readers of research will deeply appreciate that the report reflects just one person’s portrayal of facts, one person’s take on the topic. That author prepared an argument, and chose specific evidence to support it, for a reason. A critical reader will suss out those reasons, which are normally informed by research paradigms, assumptions, biases, values, prejudices, opinions, agendas, and interests. Upon recognizing what a text says on the surface, a critical reader goes further and deeper, discovering what it means. The latter process involves critical interpretation of the research report so as to offer alternative, possible meanings or to challenge the evidence and arguments used by the author to make points and to draw conclusions (Kurland, 2000). A critical reading also entails stepping back and gaining distance from the research report. From this distance, readers can “launch into an intensive critical reading” (Knott, 2009, p. 1) of the document. They would be looking for (a) the authors’ central claims or purpose, (b) the background context and intended audience, (c) the kinds of reasoning and appeals used by the authors to develop their argument, and (d) their selection of evidence to support their arguments. Readers culminate this intensive critical reading with their evaluation of the strengths and weaknesses of the arguments comprising the report (Knott, 2009) (see Figure 1.1 and the Appendix). Once readers have critically assessed the research, they have to decide to what extent they will accept the authors’ arguments, opinions, and conclusions. This involves uncovering the authors’ (a) rationale for the evidence they selected and (b) construction of their arguments. Readers need to ask themselves several pertinent questions. “How well developed are the arguments leading to discussion points and conclusions? Is the author’s interpretation of the data convincing? Are the author’s conclusions supported by the preceding arguments? Does the author’s line of reasoning (or logic) make sense and hold together? Might there be alternative interpretations of data other than those proposed by the author? Are there any hidden assumptions that have to be questioned?” These questions constitute an engaged, critical reading of a research report (University of Leicester, 2009). Conscious readers of research will appreciate that all research contains a margin of error (an amount, usually small, that is allowed for) (Blackmore & Rockert, 2004). No study is perfect (Shank & Brown, 2007). Critical 46
readers will acknowledge that these imperfections exist, accept this fact, and then determine whether these imperfections undermine the study’s rigor, rendering the results/findings meaningless or even harmful (Croad & Farquhar, 2005). Critical readers will read between the lines, unpack the article, and pass judgment on it at many levels (Shank & Brown, 2007). They will look beyond the minor issues (like a weak title or irrelevant keywords) and take a broad view to ask the big questions. “Readers who are critically literate in . . . research never lose sight of this big picture, even when they are happily wallowing in the [method] and stylistic details” (Shank & Brown, 2007, p. 227) (see the Appendix). 47
Skeptical and Critical Scrutiny Knowing that all studies are imperfect, critical readers would bring a healthy dose of skepticism to the process of critically reading research reports. Mitzenmacher (2010) suggested that critical reading requires people to be suspicious. But skepticism would likely be more productive and fair because suspicious means distrustful while skeptical means being inclined to inquire and to question (Anderson, 2014). Critical reading involves “harder, more positive thinking” than does suspicion, which tends to tear something down or apart (Mitzenmacher, 2010, p. 1). If people are skeptical, they can confidently resist ideas presented by others instead of uncritically accepting them. Their intent is to inquire into the scholarship in order to find unspoken assumptions, fallacies, dogma, and, yes, outright errors in the methods or analysis (McGregor, 2006; Suter, 2012). Harris (2014) urged readers to find a balance between naïveté and cynicism by thinking critically about research. Taking this stance, they can “intelligently appraise research” (p. 106). “A little skepticism provides healthy protection against [missing] mistakes” (Locke, Silverman, & Spirduso, 2010, p. 69). Rather than getting bogged down by criticizing the minutiae, critical readers would focus on whether the scholarship is of high quality and meets the conventions for the research methodology employed in the study (Locke et al., 2010). For example, confirming long-held anecdotal assertions, Caldwell, Henshaw, and Taylor (2005) affirmed that people tend to use the quantitative criteria of reliability and validity to judge qualitative research (but not vice versa). They claimed the resultant criticism is unjustified because quantitative research should be judged using different criteria, including transferability, dependability, credibility, and confirmability. Critical readers would be able to evaluate and judge the quality of the evidence the authors presented, appreciating that different criteria are used for quantitative, qualitative, and mixed methods methodologies (Zardo & Pryor, 2012) (see Chapter 8, Table 8.5). In a lighthearted example, Suter (2012) shared this anecdote. A quantitative researcher critically evaluated a qualitative study, taunting the researcher with “What? Your conclusion is based on only one participant?” The qualitative researcher aptly responded with “What? Your conclusion is based on only one experiment?” Suter affirmed that this rivalry and disrespect is unjustified because both research “‘camps’ . . . value rigorous data collection and analysis coupled with sound, logical arguments that characterize scientific reasoning, namely a compelling chain of evidence that supports the conclusions. Both camps are keenly aware of rival . . . and alternative explanations of their findings” (p. 345). Locke et al. (2010) concurred, acknowledging “the paradoxical fact that there are pervasive similarities” between quantitative and qualitative research paradigms (p. 80). Wallace and Wray (2011) tendered a model for discerning readers that better ensures they critically scrutinize a study. They cautioned readers to remember that each study was conducted and the report written by particular authors, with a particular purpose and audience. These authors intentionally constructed an argument to convince that audience of the merit of their research. Critical readers have to try to discern what the authors were striving to achieve. They have to work out the structure of the argument and try to identify the main claims (see Chapter 6). As they do this, critical readers should adopt a skeptical stance, checking to make sure the evidence supports the claims. This involves considering any underlying values, assumptions, 48
paradigms, or biases guiding the authors and influencing their claims. Throughout this whole critical process, readers have to keep an open mind and be willing to be convinced (but not cajoled). Review and Engagement Critical readers will □ Recognize and understand the differences among biases, assumptions, prejudices, and agendas (see Figure 1.3) □ Appreciate that a critical reading of a paper necessitates a balanced appraisal (look for both strengths and weaknesses), doing so objectively and using logic □ Learn how to locate and evaluate the authors’ argument and how to evaluate any evidence they used to make their claims (their conclusions) □ Practice stepping back from a paper to gain critical distance, and only then pass judgment on its merits and demerits, at many levels (see Figure 1.1) □ Bring a healthy dose of skepticism to the process of critically reading a research report (i.e., confidently resist ideas while remaining open to inquiry and to questioning) 49
Uncritical Readings of Research Short of stating the obvious, uncritical reading of research means people accept it without challenging anything. Uncritical means not using one’s critical faculties (by choice or lack of ability). Un is Latin “not,” and critical is “to judge.” So, being uncritical means not judging, in that people accept something too easily because of being unwilling or unable to critique it (Anderson, 2014; Harper, 2016). People can be uncritical for several reasons (see Figure 1.4), to be discussed in more detail. Figure 1.4 Nuances of Uncritical Reading of Research 50
Misplaced Trust Some people may innocently approach research assuming it is true and valid. This misplaced trust is unfortunate because they may end up using a weak, flawed study (Locke et al., 2010). Valid is Latin validus, “strong” (Harper, 2016). It also denotes the condition of being true (Locke et al., 2010). Fundamentally, internal validity (strength) refers to whether the study truly dealt with what was being studied. The data gathered have to match the research question. External validity refers to the results being valid (i.e., remaining truthful or meaningful) beyond the confines of the study. Different research methodologies and traditions use different terminology for these two aspects of strength and truth/meaning (Hart et al., 1980; Locke et al., 2010) (see Chapter 8, Table 8.5). Readers should also look for logical validity, meaning whether the report holds together and makes sense. Called internal consistency, it refers to whether each element of the research design logically flows from the others. For example, do the conclusions reflect the discussion points, which should be anchored in the results or findings and interpreted using the literature and any theory or conceptual framework underpinning the study (see Chapter 16, Figure 16.2)? If not logically valid, the report may be suspect and not usable (Wiersma & Jurs, 2009). At the crux of the matter is that authors must not deliberately insert falsehoods into their research reports; any errors or flaws have to be unintentional (Shank & Brown, 2007). 51
Overindulgent and Undiscriminating Uncritically using flawed data, logic, or conclusions can have very negative consequences (Croad & Farquhar, 2005). Critical readers of research would know when to suspend their trust in a research report (Locke et al., 2010), but this is not always the case. Some people are relatively knowledgeable of the research process but are undiscriminating, meaning they do not consistently apply critical standards of analysis when reading studies. They run the risk of uncritically accepting the study as good research when it is actually compromised. They are too indulgent of the authors’ claims and conclusions, overlooking flaws or being too tolerant of sloppy or shoddy scholarship. It is irresponsible, let alone uncritical, to disregard the requirement that authors should be held accountable to accepted standards and conventions of sound scholarship. Reading research is a collaborative event, with both writer and reader bearing responsibilities to ensure that communications do not break down. Every article is a conversation and a potential dialogue (Locke et al., 2010; Shank & Brown, 2007). 52
Surface-Level Reading Locke et al. (2010) advised that if they are unable to detect a flaw yet sense something is amiss, critical readers should shift from reading for general content to examining details pertaining to the rationale for the study; variables or phenomena; settings, contexts, and samples; methods for data collection and analysis; discussion points; conclusions; and the logic used to present the study. Inadequacies in any of these basic research elements can set off alarms and warning bells. If people persist in reading the article at the surface level while opting not to critique the research (or being unable to), their takeaways from the study will lack intellectual depth or thoroughness. They will end up using cursory, one-dimensional, and shallow interpretations of the study when actually their research or practice context demands more than a superficial, uncritical reading. Their naïveté hinders critical readings of studies. If they are also gullible (easily deceived), their unsuspecting demeanor may cause them to rely on scholarship without examining the authors’ assumptions and any hidden agendas (see next section). 53
Reading Deeper for Ideologies and Paradigms At a deeper level, uncritical readers may approach a research report unaware of the power of ideologies and paradigms. The consequences of this can cut two ways: (a) Readers may be unaware of which paradigm they bring to the exercise (or which ideologies are steeped into their psyche), and (b) they may not be able to discern the authors’ worldview or appreciate that authors are favoring a particular ideology that is clouding their perspective. Indeed, some authors purposely promote an ideology dear to their heart rather than conduct an honest inquiry. On the other hand, readers may be very aware of the power of ideologies but choose to find fault with the research (criticize it) instead of critically reading it. Readers’ unfounded suspicion that the author is harboring ideological motives may actually cramp their ability or inclination to read critically (Locke et al., 2010). Ideologies In more detail, ideologies (a system of ideas and ideals held by a group) are orientations to the world that characterize the thinking of a group or a nation. Paradigms (patterns) are individual thought patterns influenced by the ideologies. Ideologies come before paradigms. Ideologies are understood to be cultural blueprints, while paradigms are thought patterns for interpreting the world shaped by this blueprint. Ideologies are the ruling ideas of the times. They are assumptions about what is worthy of belief and attention, accepted as true, and valued. They pertain to how society should work and be arranged, and they provide the rules deemed most appropriate for achieving this ideal societal arrangement (see Table 1.1, which portrays only the dominant ideologies and paradigms, for illustrative purposes. There is a cadre of contending ideologies and paradigms, but their discussion is beyond the scope of this chapter) (see Donovan, 2010; Eaton, 1996; Elgin & LeDrew, 1997; Heuerman & Olson, 1998; McGregor, 2008, 2013; McGregor, Pendergast, Seniuk, Eghan, & Engberg, 2008). Table 1.1 Dominant Ideologies and Paradigms Seeping Into Research Agendas Dominant Ideologies (cultural blueprints of desired society) Attendant Paradigms (perspectives on a world shaped by the ideologies) Capitalism: an economic orientation that values profit, wealth accumulation, growth, production, and technological progress • Control, mastery, management, and efficiency • Property ownership, dominance, profit, and competition • Mechanistic orientation, focused on parts (rather than holistic) • Transmissions and transactions 54
Patriarchy: a system that privileges and conveys power to men and marginalizes and disempowers women and other presumably weak people • Dualism about every aspect of the world (One side of the binary pair is desirable because it is powerful; the other is undesirable and is in a powerless or marginalized position.) • Hierarchies that divide and separate human beings into categories such as gender, class, economic status, and political power Neoliberalism: economic and political notions that favor business with nominal roles for governments • Relativism (quick fix, no absolute truth) • Basic tenets: individualism, privatization, decentralization, and deregulation Economic globalization (from the top down): corporate- and elite-led neoliberalism focused on integrating national economies into a global economy; concerned with the pace of integration • Newtonianism (disconnected and fragmented) • Causal relationships, linearity, predetermination • Reductionism (reduce everything to categories, specializations, micro analysis) • Mastery over nature, resources, and marginalized people (exploitation, extraction, control) Political ideology of conservatism: the idea that societies should maintain (conserve) the status quo of the ruling elites and accept change only reluctantly, at a very slow pace; things endure because they work • Favors business with nominal roles for governments; minimal regulation of the market • Social hierarchy and social inequality viewed as inevitable, natural, normal, even desirable • Expansive military policies and spending are assumed to protect the country and, by association, the family • Scarcity mentality and competition for 55
Social Darwinism: survival of the fittest; monies and support for elders, children, the sick, and people experiencing life transitions are considered to be wasted because these people are not economically productive members of society scarce resources • Evolutionism (natural selection—the strong survive and thrive) • Win–lose mentality • Elitism (privilege the wealthy and powerful, who should be rulers) • Division of labor based on gender Fundamentalism: strong maintenance of and adherence to any set of ancient or fundamental doctrines and beliefs (moral codes), usually in the face of criticism or unpopularity (especially religious beliefs and social and political movements) • Extremism (far outside the acceptable mainstream attitudes of society), especially religious extremism • Free-market fundamentalism (the market can solve social ills) • In-group and out-group distinctions must be maintained • Rejection of diversity of opinions • Moral intolerance and adherence to strict social conventions Consumerism: inculcates the values of the western consumer lifestyle on a global scale; society’s preoccupation with the accumulation of goods and the procurement of services formally self-performed • Materialism (using things to measure success, with little concern for spiritual or ethical matters) • Material gains and possessions privileged over relationships • Conspicuous consumption that may be unethical and immoral • Popular culture postmodernism (novelty, commodification, entitlement, communication technologies) Paradigms Paradigms are habits of thinking in a particular way or of making certain assumptions (others call this a worldview or a mind-set). Paradigms profoundly affect the way people perceive reality as it is lived within 56
society’s dominant belief systems (i.e., its ideologies). Paradigms affect how people respond to their perceptions of the world, including those of a research report. Ensconced in their paradigms, people use them to make sense of their world by giving meaning to their lived experiences (see Table 1.1). A common metaphor used to explain this connection is an ideological camp where people live out their lives. When they climb up into an observation tower and view life below in the camps, they are seeing the world through their paradigms. The camps reflect the cultural blueprint by which all people are supposed to live, and the paradigms are the way people think about and make sense of that life (Donovan, 2010; Eaton, 1996; Elgin & LeDrew, 1997; Heuerman & Olson, 1998; McGregor, 2008, 2013; McGregor et al., 2008). Example 1.1 A study inadvertently reinforcing patriarchy-informed policy Consider a study that explored the impact of social welfare policy on families. The policy said that once a man moves into a single woman’s household, she no longer qualifies for welfare assistance because it is assumed that the man will be working and contributing money to the household. Even if he is not working, the policy still holds because a man is there. Under this policy, welfare assistance is automatically cut off. Researchers conducted focus groups and interviews with women receiving this assistance. They reported undue financial hardship because of this policy, which opponents called draconian, meaning excessively harsh. Despite the profound insights gained from these data, the researchers concluded that the policy was sound, in effect reinforcing the patriarchy ideology. It holds that the male is the head of the household and the main breadwinner. If a male is present (any male), it is assumed that women will be taken care of and do not deserve financial assistance using public tax dollars. Government bureaucrats cited the study as justification for maintaining the policy, and actually made it even harder for women to regain assistance once the man left. When opponents to this policy approached the researchers, challenging their assumptions and conclusions, the research team revealed they had simply not seen it from that perspective before (standing from their observation tower) and regretted drawing their conclusions. They had not intended to harm women with their study, but their unexamined ideological lens had the same effect. In a research context, articles inherently reflect authors’ assumptions about the phenomenon under examination and any surrounding contexts. Those assumptions are informed by ideologies and paradigms, which, for most people, go unexamined at the best of times. They are so ordinary in everyday life, even in research life, that they become invisible and go unchallenged. Despite that authors should be transparent about any assumptions underpinning their research, this often does not happen (Neuman, 2000). Critical readers would be aware of this and be prepared to suss out and question those assumptions. Actually, most authors intend to do good research and intend for readers to access their ideas so their work can be understood and used. Critical readers of research will respect these intentions, appreciating the insidious power of ideologies and paradigms (Locke et al., 2010). Example 1.2 Dominant paradigm (thought pattern) in research A good example of a dominant thought pattern in research is the assumption that quantitative research is more valuable and 57
legitimate than qualitative research because the former is empirical and positivistic, grounded in measurable numbers and verifiable statistics. Qualitative research is often maligned and called soft science, meaning it is hard to quantify, assuming that all data have to be quantified to be true. People’s meanings, wisdom, and interpretations of their own lives are not measurable; hence, they do not matter. These assumptions reflect the deeply ingrained paradigms of Newtonianism and positivism. Newtonian thinking holds that objective reality comprises predictable, measurable, linear, cause-and-effect phenomena. Positivism assumes that the only way one can be positive that one has knowledge is if it is produced using the scientific method. Qualitative studies are often undervalued, minimized, and even dismissed as not good research because they cannot be judged as empirically valid and reliable, nor do they yield desirable proof of cause and effect (see Chapter 2). Review and Engagement Critical readers will □ With practice, come to recognize when they are being gullible, naïve, too trusting, innocent, and unsuspecting (see Figure 1.4) □ Conversely, know how to check for the internal consistency in authors’ logic, when to suspend their trust in the report, how to read beyond the surface level, and how to dig deeper to expose ideologies and paradigms □ Appreciate the persuasive power of ideologies and paradigms in research and learn to critically recognize when they are at play, affecting the authors’ message (intentionally or not) (see Table 1.1) □ Examine their own research paradigms and ideological awareness, thereby becoming responsible partners in the critical research conversation 58
Critically and Uncritically Reading Research In summary, “being a critical consumer of research is more than just reading academic papers; it involves thinking critically about the assumptions behind research, the methodologies [and methods] employed and the implications that research results [or findings] hold for practice” (Zardo & Pryor, 2012, p. 1). Suter (2012) compared the qualities of critical and uncritical readers of research (see pp. 10–11), summarized in Table 1.2. A critical reader of research will be able to navigate or move with confidence and competence through research material and reports in order to critically evaluate and effectively use the information and new knowledge (Locke et al., 2010). Lacking this critical navigational ability can lead to a compromised knowledge base as well as compromised practice, policy, theory development, and future research (Zardo & Pryor, 2012). Example 1.3 Uncritiqued research project shaping policy Croad and Farquhar (2005) took issue with an actual New Zealand study about the link between the quality of early childhood education and children’s future learning. Touted as a significant study, it has influenced government policies and educational practice in New Zealand. The authors critically analyzed the study and found several issues that severely compromised its real contribution to the topic of competent children. They discovered shortcomings around “the sampling technique and the composition of the sample, the research design, data analysis, and interpretations of the data. Assumptions made in the research about children’s competencies, quality in early childhood education and the measurement of quality therefore need to be questioned” (p. 18). They challenged the authors’ and the media’s uncritical claims about the study and admonished government officials, educators, and practitioners to critically examine and openly discuss it. They claimed that too much is at stake for people to uncritically use a study with so many unchallenged limitations. Both superficial reporting and using a study purported to be a significant policy resource are irresponsible and shortsighted. “Publically funded policy-driven research which is used to legitimate education policy and spending should not be exempted from scrutiny and critique” (p. 17). Table 1.2 Comparison of Critical and Uncritical Readers of Research Reports Critical Thinkers and Readers Uncritical Thinkers and Readers • Search for biases • Accept information without scrutiny • Recognize logical fallacies and inconsistencies • Overlook, cannot recognize, or fall victim to fallacies (flaws in logic) • Reason with clarity, precision, and relevance • Fail to recognize disjointed or irrelevant reasoning 59
• Look to see if authors considered the context when reaching conclusions • Accept stereotypes and overgeneralizations • Confirm that conclusions are justified (supported by data) and well reasoned • Accept conclusions without challenging the authors’ reasoning and the evidence • Assess the validity (strength and truth) of all claims • Accept claims unchallenged, without questioning their validity • Unearth and examine authors’ assumptions • Confuse assumptions about what is true (with no proof) with facts • Discern whether authors accounted for complexity when judging import of study • Upon reading a paper, come to snap judgments about the study (oversimplify things) • Determine if authors considered alternative explanations (and come up with their own counter thoughts) • Tend to confirm personal biases (i.e., favor only evidence that reinforces their own preexisting beliefs) On a concluding note, the rest of the book is focused on sharing detailed and nuanced discussions of accepted conventions and standards of conducting and reporting high-quality research. The intent is to help readers become better equipped at, and more predisposed to, being critical readers of research and scholarship. There are chapters on each of the basic elements of the research process (see Figure 1.2 and the Appendix), prefaced with a chapter that thoroughly discusses the main approaches to categorizing research methodologies (Chapter 2), which differ from yet inform research methods. Upon reading Understanding and Evaluating Research, readers should feel more comfortable assuming the mantle and deep responsibility of critiquing others’ research. Review and Engagement Critical readers will □ Be able to practice being a critical thinker and reader when critiquing research reports (see Table 1.2) □ Appreciate the necessity of being able to critically navigate a research report, with critical awareness and critical prowess Chapter Summary This chapter introduced readers to the intent of the book, which is to help people critically read and evaluate other people’s scholarship before they use it in their research or practice. Readers were introduced to the idea of research literacy and why it is important. The discussion then turned to the topic of critical research literacy, starting with what is involved in reading critically, followed with why some people may not critically read a research report (see Figures 1.1 and 1.2). The former included a discussion of the need for balanced appraisals, to judge the author’s chains of reasoning, and to be skeptical and engage in critical scrutiny. These strategies help critical readers deal with the author’s biases, (unspoken) assumptions, interests, and prejudices (see Figure 1.3). People may fail to critically judge a research report for several reasons, including misplaced trust, being overindulgent and undiscriminating, reading at just the surface level, and failing to account for the power of ideologies and paradigms in research (see 60
Figure 1.4 and Table 1.1). The chapter concluded with an overview of what constitutes critical and uncritical thinking and reading of a research report (see Table 1.2). 61
Review and Discussion Questions 1. There are many reasons why it is important to be research literate. Identify three that are most important to you, and explain why they are so important. 2. How is criticizing a research report different from critiquing it? Are both skills needed? Which of these should be privileged, and why (see Figure 1.1)? 3. Compare and contrast what it means to be a critical reader and an uncritical reader of a research report (see Table 1.2). 4. What are five key things critical readers should look for when reading a research report (see Figure 1.3)? 5. What does someone have to do to be considered an uncritical reader of research reports (see Figure 1.4)? 6. Why is it important to critically understand a research report? 7. How do ideologies and paradigms differ? Explain how they are interconnected. What role do ideologies and paradigms play in research (see Table 1.1)? 62
Part II Philosophical And Theoretical Aspects Of Research 63
2 Research Methodologies 64
Learning Objectives Appreciate the history of key methodological terms Recognize the necessity of being able to defend any methodological choices made at the interface between philosophy and methods (methodologically responsible) Distinguish clearly between methodology and methods (as used in this book) Become familiar with the conceptual confusion, slippage, and clarity needed around three common terms: research paradigm, research methodology, and research tradition Appreciate the methodological approach used in this book (see Table 2.1) Explain the construct of philosophical axioms (epistemology, ontology, logic, and axiology) Distinguish between positivistic and postpositivistic research paradigms Compare and contrast empirical, interpretive, and critical research methodologies Compare and contrast quantitative, qualitative, and mixed methods methodologies Explain why it is necessary to match research methodology with the research question Understand the conventions for writing the research methodology section of a paper 65
Introduction Research and inquiry are about creating new knowledge (Habermas, 1984). Philosophy is the study of the fundamental nature of knowledge, reality, and existence—its truths, principles, and assumptions (Anderson, 2014). This book is premised on the assumption that everything in research hinges on philosophical underpinnings. But making this point is challenging because of the proliferation of methodology-related terms arising in the late 1970s and peaking in the early 1990s. Egon Guba is credited with initiating the paradigm dialogue about quantitative and qualitative research (Donmoyer, 2008). Since then, researchers have witnessed the emergence of a dizzying array of jargon used by scholars trying to address this thorny but imperative aspect of research. This scenario is exacerbated by the fact that “many researchers lack experience [or expertise] in deliberating about methodological issues, and the esoteric and unfamiliar language of philosophy can be intimidating” (MacCleave, 2006, p. 9). This array of methodology-related terms includes research paradigms, methodologies, methods, philosophical axioms, quantitative, qualitative, mixed methods, positivism, postpositivism, empirical, interpretive, and critical (and one can add postmodernism, poststructuralism, constructivism, naturalistic inquiry, critical realism, and so on). Inconsistency in what these terms mean, alone and in relation to each other, is evident across all disciplinary literature (Cameron, 2011). Acknowledging this state of affairs, Locke, Silverman, and Spirduso (2010) sardonically noted that “the first tour through the research literature in your own area of interest is likely to reveal more variety than you would expect” (p. 80). They even coined the term paradigmatic subspecies (p. 80) to accommodate this diverse philosophical situation. The result of such philosophical diversity is terminological soup or, as Buchanan and Bryman (2007, p. 486) called it, “paradigm soup.” Actually, some of these terms have been in use for more than 400 years, adding to this linguistic and philosophical conundrum (see Figure 2.1) (Ary, Jacobs, & Sorensen, 2010; Denzin & Lincoln, 2011; Fox, 2008; Guba, 1990; Johnson & Christensen, 2012; Lockyer, 2008; Niglas, 1999; Paley, 2008; Smith, 1983). Nonetheless, researchers have the responsibility of explicitly identifying the methodological and paradigmatic underpinnings of their scholarship (Maxwell, 2013). To address this conceptual slippage, this chapter explains and justifies the approach used in this book (see Table 2.1), knowing that not everyone will agree with it. Regardless, researchers and authors have to “acknowledge the paradigm debate” and rigorously defend any methodological choices “made at the interface between philosophy and methods” (Cameron, 2011, p. 101). This due diligence is necessary because, to academics, these words can mean different things. Without conceptual clarity, the integrity of any academic conversation about the interface between philosophy, methodology, and methods is compromised. Figure 2.1 History of Methodologically Oriented Terms 66
Table 2.1 Overview of Main Research Methodologies, With Common Methods (used with permission) [Acknowledgment: Deep, deep thanks to Dr. Anne MacCleave, Professor Emerita (MSVU), for vetting and validating the core concepts contained in this table] Research Paradigm Positivistic Postpositivistic Sources: From McGregor & Murnane (2010) with permission from John Wiley & Sons. Sources used by the authors to develop the appendix included: Howe, 1992; Lather, 1994; Niglas, 2001; MacDonald et al., 2002; Khazanchi and Munkvold, 2003; Guba and Lincoln, 2005; Ponterotto, 2005; Salmani and Akbari, 2008). Acknowledgment and deep thanks to Dr.Anne MacCleave, Professor Emerita Mount Saint Vincent University (MSVU), for vetting and validating the core concepts contained in this table. 67
Conceptual Confusion, Slippage, and Clarity This section attempts the near impossible, to distinguish between the terms research paradigm, research methodology (compared to methods), and research traditions. All three terms are used in the academic world, leading to confusion because paradigm means thought patterns, methodology is linked with philosophy, and tradition refers to long-standing customs (see Figure 2.2). In truth, they all have some merit when trying to distinguish between (a) collecting new information (data) to answer a research question and (b) knowledge creation using interpretations of those data. On the other hand, the diverse language used to refer to this aspect of research has created a quagmire. This complex and difficult situation makes it hard for one scholar to talk to and understand another. But talk to each other they must, so this section briefly explains how the literature understands these concepts, settling on research paradigm and research methodology for this book (they mean different things). 68
Research Paradigm Paradigm is Latin paradigma, “patterns” (Harper, 2016). A paradigm is “a generally accepted explanation of things,” with the dominant paradigm providing “the focal point and measuring stick” for inquiry (Rohmann, 1999, p. 296). Paradigms are thought patterns that help people make sense of their world, regardless of whether they are engaged in research or not. Paradigms are habits of thinking in a particular way or of making certain assumptions about the world (others call this worldview or mind-set) (Donovan, 2010) (see Chapter 1 for a discussion of paradigms and ideologies). The term research paradigm, coined by Kuhn (1962), is understood to mean “patterns of beliefs and practices that regulate inquiry within a discipline, doing so by providing the lenses, frames and processes through which investigation is accomplished” (Weaver & Olson, 2006, p. 460). Johnson and Christensen (2012) defined a research paradigm as a “perspective about research held by a community of researchers that is based on a set of shared assumptions, concepts, values, and practices” (p. 31). These definitions make sense. After all, disciplines are groups or communities of people, and paradigms reflect a group’s commitment to a constellation of beliefs about viewing the world. They are a group-licensed way of seeing reality (Botha, 1989). Normally, the philosophical notion of axioms is reserved for the term research methodology, as is the case in this book. Some scholars, however, characterize research paradigms by distinctive axioms, namely ontology, axiology, epistemology, rhetoric, causality and logic, and methodology (by which is meant the identification, study, and justification of research methods) (Guba, 1990; Pruyt, 2006). Figure 2.2 Research Paradigm, Methodology, and Tradition 69
Research Methodology In many disciplines, the term methodology is used to refer to the methods used to collect, analyze, and report data (see Schneider, 2014; Trochim & Donnelly, 2007). This usage eschews the real meaning of methodology. Ology is Greek for a branch of knowledge or science. Method is Greek methodos, “the pursuit of knowledge” (Anderson, 2014; Harper, 2016). Taken together, methodology means a branch of science that studies the pursuit of knowledge. “The misuse of methodology obscures an important conceptual distinction between the tools of scientific investigation (properly methods) and the principles that determine how such tools are deployed and interpreted (methodology)” (American Heritage Dictionary, 2000). This chapter views methodology as the philosophical underpinnings of research intended to generate new knowledge and methods as tools and techniques to collect and analyze data (Lather, 1994; MacCleave, 2006) (see Figure 2.3). To that end, this chapter focuses on methodologies, and Chapter 8 focuses on methods (and research design). In particular, methodology refers to knowledge creation, including what counts as knowledge and knowing, reality, logic, and the role of values in knowledge creation (i.e., four axioms, to be discussed shortly). Two common approaches to describing research methodologies are (a) quantitative, qualitative, and mixed methods and (b) empirical, interpretive, and critical. These are discussed in more detail further on in the chapter. This book uses the former as its organizational framework. 70
Research Traditions Actually, some academics skirt the contentious issue of whether to use the term research paradigm or research methodology and instead use the term research traditions (Jacob, 1987; Schneider, 2014). A tradition is an inherited pattern of thought and a specific practice of long standing (Anderson, 2014). Kuhn (1970) said any research tradition differs along three dimensions: (a) its assumptions about nature and reality, (b) the foci of studies and major issues of interest about the phenomenon, and (c) methodology (by which he meant methods). He also noted that a tradition can occur either as an entire discipline or as a school within a discipline (e.g., subdisciplines and disciplinary specializations). For example, Jacob (1987) applied this approach to profile three subdisciplines within the discipline of education. The term tradition is the least commonly used in the literature, but it was important to acknowledge it in this chapter because authors may choose to use it when reporting their study, or they might encounter it when reading literature. Patton (2002) identified 10 qualitative research traditions including constructivism, symbolic interaction, semiotics, hermeneutics, systems, and chaos (nonlinear dynamics). This book views these as falling within qualitative and interpretive research methodologies (see Table 2.1). Figure 2.3 Methodology Compared to Method 71
72
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf
(ebook) Understanding and Evaluating Research (McGregor 2018).pdf

Recommended

Payroll coordinator performance appraisal
Payroll coordinator performance appraisalPayroll coordinator performance appraisal
Payroll coordinator performance appraisal
bobalex231
 
Mcdonalds team leader performance appraisal
Mcdonalds team leader performance appraisalMcdonalds team leader performance appraisal
Mcdonalds team leader performance appraisalsmalldonan
 
Fast food manager perfomance appraisal 2
Fast food manager perfomance appraisal 2Fast food manager perfomance appraisal 2
Fast food manager perfomance appraisal 2tonychoper3304
 
Accounting officer performance appraisal
Accounting officer performance appraisalAccounting officer performance appraisal
Accounting officer performance appraisal
michealtaylor431
 
Insurance billing specialist performance appraisal
Insurance billing specialist performance appraisalInsurance billing specialist performance appraisal
Insurance billing specialist performance appraisal
jayschwandt06
 
Import clerk performance appraisal
Import clerk performance appraisalImport clerk performance appraisal
Import clerk performance appraisalFrancescoTotti012
 
Branch supervisor performance appraisal
Branch supervisor performance appraisalBranch supervisor performance appraisal
Branch supervisor performance appraisal
bellalalle93
 
Production support manager performance appraisal
Production support manager performance appraisalProduction support manager performance appraisal
Production support manager performance appraisal
hayileyeliot
 

Recommended

Payroll coordinator performance appraisal
Payroll coordinator performance appraisalPayroll coordinator performance appraisal
Payroll coordinator performance appraisal
bobalex231
 
Mcdonalds team leader performance appraisal
Mcdonalds team leader performance appraisalMcdonalds team leader performance appraisal
Mcdonalds team leader performance appraisalsmalldonan
 
Fast food manager perfomance appraisal 2
Fast food manager perfomance appraisal 2Fast food manager perfomance appraisal 2
Fast food manager perfomance appraisal 2tonychoper3304
 
Accounting officer performance appraisal
Accounting officer performance appraisalAccounting officer performance appraisal
Accounting officer performance appraisal
michealtaylor431
 
Insurance billing specialist performance appraisal
Insurance billing specialist performance appraisalInsurance billing specialist performance appraisal
Insurance billing specialist performance appraisal
jayschwandt06
 
Import clerk performance appraisal
Import clerk performance appraisalImport clerk performance appraisal
Import clerk performance appraisalFrancescoTotti012
 
Branch supervisor performance appraisal
Branch supervisor performance appraisalBranch supervisor performance appraisal
Branch supervisor performance appraisal
bellalalle93
 
Production support manager performance appraisal
Production support manager performance appraisalProduction support manager performance appraisal
Production support manager performance appraisal
hayileyeliot
 
Account receivable clerk performance appraisal
Account receivable clerk performance appraisalAccount receivable clerk performance appraisal
Account receivable clerk performance appraisalRonaldKoeman789
 
Product marketing manager performance appraisal
Product marketing manager performance appraisalProduct marketing manager performance appraisal
Product marketing manager performance appraisal
campbelljonny439
 
School catering assistant performance appraisal
School catering assistant performance appraisalSchool catering assistant performance appraisal
School catering assistant performance appraisalTayeaiwo789
 
Citibank teller performance appraisal
Citibank teller performance appraisalCitibank teller performance appraisal
Citibank teller performance appraisal
thomasmason037
 
Exams officer performance appraisal
Exams officer performance appraisalExams officer performance appraisal
Exams officer performance appraisalUtadaHikaru012
 
Cost manager perfomance appraisal 2
Cost manager perfomance appraisal 2Cost manager perfomance appraisal 2
Cost manager perfomance appraisal 2tonychoper5504
 
Hotel general manager performance appraisal
Hotel general manager performance appraisalHotel general manager performance appraisal
Hotel general manager performance appraisal
lewisstewart770
 
Supply chain officer perfomance appraisal 2
Supply chain officer perfomance appraisal 2Supply chain officer perfomance appraisal 2
Supply chain officer perfomance appraisal 2tonychoper6204
 
Kg teacher performance appraisal
Kg teacher performance appraisalKg teacher performance appraisal
Kg teacher performance appraisal
jacehobbs2
 
Chief stewardess performance appraisal
Chief stewardess performance appraisalChief stewardess performance appraisal
Chief stewardess performance appraisal
griffinbrandon276
 
Accounting clerk performance appraisal
Accounting clerk performance appraisalAccounting clerk performance appraisal
Accounting clerk performance appraisal
evansaaron775
 
Invoice processor performance appraisal
Invoice processor performance appraisalInvoice processor performance appraisal
Invoice processor performance appraisal
sammywood963
 
Head concierge performance appraisal
Head concierge performance appraisalHead concierge performance appraisal
Head concierge performance appraisaladamliesen
 
Database engineer perfomance appraisal 2
Database engineer perfomance appraisal 2Database engineer perfomance appraisal 2
Database engineer perfomance appraisal 2tonychoper1004
 
Jollibee crew performance appraisal
Jollibee crew performance appraisalJollibee crew performance appraisal
Jollibee crew performance appraisal
hallbradley21
 
Hrm job analysis
Hrm job analysisHrm job analysis
Hrm job analysis
Izzah Noah
 
Lead software engineer performance appraisal
Lead software engineer performance appraisalLead software engineer performance appraisal
Lead software engineer performance appraisalEmileHeskey345
 
Senior cook performance appraisal
Senior cook performance appraisalSenior cook performance appraisal
Senior cook performance appraisaloclermartin015
 
Help desk manager performance appraisal
Help desk manager performance appraisalHelp desk manager performance appraisal
Help desk manager performance appraisal
remus853
 
Import export specialist performance appraisal
Import export specialist performance appraisalImport export specialist performance appraisal
Import export specialist performance appraisal
jessicaemily215
 
Creswell.pdf beharoonjaved
Creswell.pdf beharoonjavedCreswell.pdf beharoonjaved
Creswell.pdf beharoonjaved
HaroonJaved28
 
Research-Design-third-ed-John-Creswell.pdf
Research-Design-third-ed-John-Creswell.pdfResearch-Design-third-ed-John-Creswell.pdf
Research-Design-third-ed-John-Creswell.pdf
LaxmiKhatri5
 

More Related Content

What's hot

Account receivable clerk performance appraisal
Account receivable clerk performance appraisalAccount receivable clerk performance appraisal
Account receivable clerk performance appraisalRonaldKoeman789
 
Product marketing manager performance appraisal
Product marketing manager performance appraisalProduct marketing manager performance appraisal
Product marketing manager performance appraisal
campbelljonny439
 
School catering assistant performance appraisal
School catering assistant performance appraisalSchool catering assistant performance appraisal
School catering assistant performance appraisalTayeaiwo789
 
Citibank teller performance appraisal
Citibank teller performance appraisalCitibank teller performance appraisal
Citibank teller performance appraisal
thomasmason037
 
Exams officer performance appraisal
Exams officer performance appraisalExams officer performance appraisal
Exams officer performance appraisalUtadaHikaru012
 
Cost manager perfomance appraisal 2
Cost manager perfomance appraisal 2Cost manager perfomance appraisal 2
Cost manager perfomance appraisal 2tonychoper5504
 
Hotel general manager performance appraisal
Hotel general manager performance appraisalHotel general manager performance appraisal
Hotel general manager performance appraisal
lewisstewart770
 
Supply chain officer perfomance appraisal 2
Supply chain officer perfomance appraisal 2Supply chain officer perfomance appraisal 2
Supply chain officer perfomance appraisal 2tonychoper6204
 
Kg teacher performance appraisal
Kg teacher performance appraisalKg teacher performance appraisal
Kg teacher performance appraisal
jacehobbs2
 
Chief stewardess performance appraisal
Chief stewardess performance appraisalChief stewardess performance appraisal
Chief stewardess performance appraisal
griffinbrandon276
 
Accounting clerk performance appraisal
Accounting clerk performance appraisalAccounting clerk performance appraisal
Accounting clerk performance appraisal
evansaaron775
 
Invoice processor performance appraisal
Invoice processor performance appraisalInvoice processor performance appraisal
Invoice processor performance appraisal
sammywood963
 
Head concierge performance appraisal
Head concierge performance appraisalHead concierge performance appraisal
Head concierge performance appraisaladamliesen
 
Database engineer perfomance appraisal 2
Database engineer perfomance appraisal 2Database engineer perfomance appraisal 2
Database engineer perfomance appraisal 2tonychoper1004
 
Jollibee crew performance appraisal
Jollibee crew performance appraisalJollibee crew performance appraisal
Jollibee crew performance appraisal
hallbradley21
 
Hrm job analysis
Hrm job analysisHrm job analysis
Hrm job analysis
Izzah Noah
 
Lead software engineer performance appraisal
Lead software engineer performance appraisalLead software engineer performance appraisal
Lead software engineer performance appraisalEmileHeskey345
 
Senior cook performance appraisal
Senior cook performance appraisalSenior cook performance appraisal
Senior cook performance appraisaloclermartin015
 
Help desk manager performance appraisal
Help desk manager performance appraisalHelp desk manager performance appraisal
Help desk manager performance appraisal
remus853
 
Import export specialist performance appraisal
Import export specialist performance appraisalImport export specialist performance appraisal
Import export specialist performance appraisal
jessicaemily215
 

What's hot (20)

Account receivable clerk performance appraisal
Account receivable clerk performance appraisalAccount receivable clerk performance appraisal
Account receivable clerk performance appraisal
 
Product marketing manager performance appraisal
Product marketing manager performance appraisalProduct marketing manager performance appraisal
Product marketing manager performance appraisal
 
School catering assistant performance appraisal
School catering assistant performance appraisalSchool catering assistant performance appraisal
School catering assistant performance appraisal
 
Citibank teller performance appraisal
Citibank teller performance appraisalCitibank teller performance appraisal
Citibank teller performance appraisal
 
Exams officer performance appraisal
Exams officer performance appraisalExams officer performance appraisal
Exams officer performance appraisal
 
Cost manager perfomance appraisal 2
Cost manager perfomance appraisal 2Cost manager perfomance appraisal 2
Cost manager perfomance appraisal 2
 
Hotel general manager performance appraisal
Hotel general manager performance appraisalHotel general manager performance appraisal
Hotel general manager performance appraisal
 
Supply chain officer perfomance appraisal 2
Supply chain officer perfomance appraisal 2Supply chain officer perfomance appraisal 2
Supply chain officer perfomance appraisal 2
 
Kg teacher performance appraisal
Kg teacher performance appraisalKg teacher performance appraisal
Kg teacher performance appraisal
 
Chief stewardess performance appraisal
Chief stewardess performance appraisalChief stewardess performance appraisal
Chief stewardess performance appraisal
 
Accounting clerk performance appraisal
Accounting clerk performance appraisalAccounting clerk performance appraisal
Accounting clerk performance appraisal
 
Invoice processor performance appraisal
Invoice processor performance appraisalInvoice processor performance appraisal
Invoice processor performance appraisal
 
Head concierge performance appraisal
Head concierge performance appraisalHead concierge performance appraisal
Head concierge performance appraisal
 
Database engineer perfomance appraisal 2
Database engineer perfomance appraisal 2Database engineer perfomance appraisal 2
Database engineer perfomance appraisal 2
 
Jollibee crew performance appraisal
Jollibee crew performance appraisalJollibee crew performance appraisal
Jollibee crew performance appraisal
 
Hrm job analysis
Hrm job analysisHrm job analysis
Hrm job analysis
 
Lead software engineer performance appraisal
Lead software engineer performance appraisalLead software engineer performance appraisal
Lead software engineer performance appraisal
 
Senior cook performance appraisal
Senior cook performance appraisalSenior cook performance appraisal
Senior cook performance appraisal
 
Help desk manager performance appraisal
Help desk manager performance appraisalHelp desk manager performance appraisal
Help desk manager performance appraisal
 
Import export specialist performance appraisal
Import export specialist performance appraisalImport export specialist performance appraisal
Import export specialist performance appraisal
 

Similar to (ebook) Understanding and Evaluating Research (McGregor 2018).pdf

Creswell.pdf beharoonjaved
Creswell.pdf beharoonjavedCreswell.pdf beharoonjaved
Creswell.pdf beharoonjaved
HaroonJaved28
 
Research-Design-third-ed-John-Creswell.pdf
Research-Design-third-ed-John-Creswell.pdfResearch-Design-third-ed-John-Creswell.pdf
Research-Design-third-ed-John-Creswell.pdf
LaxmiKhatri5
 
Ma dissertation workshop
Ma dissertation workshopMa dissertation workshop
Ma dissertation workshopCarolina Matos
 
Chapter7
Chapter7Chapter7
Chapter7
Ying Liu
 
KV713 Session 3
KV713 Session 3KV713 Session 3
KV713 Session 3
kturvey
 
John W. Creswell-Research Design (2015).pdf
John W. Creswell-Research Design (2015).pdfJohn W. Creswell-Research Design (2015).pdf
John W. Creswell-Research Design (2015).pdf
JeribethJanuras
 
Good practice in researching: A qualitative and cross-disciplinary research
Good practice in researching: A qualitative and cross-disciplinary researchGood practice in researching: A qualitative and cross-disciplinary research
Good practice in researching: A qualitative and cross-disciplinary research
Richard Lalleman
 
KV713 Session 3
KV713 Session 3KV713 Session 3
KV713 Session 3
kturvey
 
KV713 Session 3
KV713 Session 3 KV713 Session 3
KV713 Session 3
kturvey
 
BUSINESS RESEARCH METHODS
BUSINESS RESEARCH METHODSBUSINESS RESEARCH METHODS
BUSINESS RESEARCH METHODS
MBAnotes4u
 
Dissertation Topic; Dissertation Outline
Dissertation Topic; Dissertation OutlineDissertation Topic; Dissertation Outline
Dissertation Topic; Dissertation Outline
PPI Greater Glasgow
 
Research design & proposal writing
Research design & proposal writingResearch design & proposal writing
Research design & proposal writing
HafizurRahman174
 
Effective research paper writing for scientific write-up, Btech+Mtech.pptx
Effective research paper writing for scientific write-up, Btech+Mtech.pptxEffective research paper writing for scientific write-up, Btech+Mtech.pptx
Effective research paper writing for scientific write-up, Btech+Mtech.pptx
Methusharma
 
English for research writing ppt.pptx
English for research writing ppt.pptxEnglish for research writing ppt.pptx
English for research writing ppt.pptx
Methusharma
 
Systematic Literature Reviews : Concise Overview
Systematic Literature Reviews : Concise OverviewSystematic Literature Reviews : Concise Overview
Systematic Literature Reviews : Concise Overview
youkayaslam
 
Session 1 Rev - ANI.pptx
Session 1 Rev - ANI.pptxSession 1 Rev - ANI.pptx
Session 1 Rev - ANI.pptx
amelia naim indrajaya
 
Mg 5615 8 h group presentation
Mg 5615 8 h group presentationMg 5615 8 h group presentation
Mg 5615 8 h group presentation
Seif Ziwar
 
Digital Dissertation Overview - Dissertation Top Gun
Digital Dissertation Overview - Dissertation Top GunDigital Dissertation Overview - Dissertation Top Gun
Digital Dissertation Overview - Dissertation Top Gun
LifeAfterTelevisionInc
 
Content and organization of a manuscript
Content and organization of a manuscriptContent and organization of a manuscript
Content and organization of a manuscript
Naila Mehr
 
Research proposal: How to Write a Research Proposal
Research proposal: How to Write a Research ProposalResearch proposal: How to Write a Research Proposal
Research proposal: How to Write a Research Proposal
M. A. Shahzad
 

Similar to (ebook) Understanding and Evaluating Research (McGregor 2018).pdf (20)

Creswell.pdf beharoonjaved
Creswell.pdf beharoonjavedCreswell.pdf beharoonjaved
Creswell.pdf beharoonjaved
 
Research-Design-third-ed-John-Creswell.pdf
Research-Design-third-ed-John-Creswell.pdfResearch-Design-third-ed-John-Creswell.pdf
Research-Design-third-ed-John-Creswell.pdf
 
Ma dissertation workshop
Ma dissertation workshopMa dissertation workshop
Ma dissertation workshop
 
Chapter7
Chapter7Chapter7
Chapter7
 
KV713 Session 3
KV713 Session 3KV713 Session 3
KV713 Session 3
 
John W. Creswell-Research Design (2015).pdf
John W. Creswell-Research Design (2015).pdfJohn W. Creswell-Research Design (2015).pdf
John W. Creswell-Research Design (2015).pdf
 
Good practice in researching: A qualitative and cross-disciplinary research
Good practice in researching: A qualitative and cross-disciplinary researchGood practice in researching: A qualitative and cross-disciplinary research
Good practice in researching: A qualitative and cross-disciplinary research
 
KV713 Session 3
KV713 Session 3KV713 Session 3
KV713 Session 3
 
KV713 Session 3
KV713 Session 3 KV713 Session 3
KV713 Session 3
 
BUSINESS RESEARCH METHODS
BUSINESS RESEARCH METHODSBUSINESS RESEARCH METHODS
BUSINESS RESEARCH METHODS
 
Dissertation Topic; Dissertation Outline
Dissertation Topic; Dissertation OutlineDissertation Topic; Dissertation Outline
Dissertation Topic; Dissertation Outline
 
Research design & proposal writing
Research design & proposal writingResearch design & proposal writing
Research design & proposal writing
 
Effective research paper writing for scientific write-up, Btech+Mtech.pptx
Effective research paper writing for scientific write-up, Btech+Mtech.pptxEffective research paper writing for scientific write-up, Btech+Mtech.pptx
Effective research paper writing for scientific write-up, Btech+Mtech.pptx
 
English for research writing ppt.pptx
English for research writing ppt.pptxEnglish for research writing ppt.pptx
English for research writing ppt.pptx
 
Systematic Literature Reviews : Concise Overview
Systematic Literature Reviews : Concise OverviewSystematic Literature Reviews : Concise Overview
Systematic Literature Reviews : Concise Overview
 
Session 1 Rev - ANI.pptx
Session 1 Rev - ANI.pptxSession 1 Rev - ANI.pptx
Session 1 Rev - ANI.pptx
 
Mg 5615 8 h group presentation
Mg 5615 8 h group presentationMg 5615 8 h group presentation
Mg 5615 8 h group presentation
 
Digital Dissertation Overview - Dissertation Top Gun
Digital Dissertation Overview - Dissertation Top GunDigital Dissertation Overview - Dissertation Top Gun
Digital Dissertation Overview - Dissertation Top Gun
 
Content and organization of a manuscript
Content and organization of a manuscriptContent and organization of a manuscript
Content and organization of a manuscript
 
Research proposal: How to Write a Research Proposal
Research proposal: How to Write a Research ProposalResearch proposal: How to Write a Research Proposal
Research proposal: How to Write a Research Proposal
 

Recently uploaded

Normal Labour/ Stages of Labour/ Mechanism of Labour
Normal Labour/ Stages of Labour/ Mechanism of LabourNormal Labour/ Stages of Labour/ Mechanism of Labour
Normal Labour/ Stages of Labour/ Mechanism of Labour
Wasim Ak
 
Synthetic Fiber Construction in lab .pptx
Synthetic Fiber Construction in lab .pptxSynthetic Fiber Construction in lab .pptx
Synthetic Fiber Construction in lab .pptx
Pavel ( NSTU)
 
A Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in EducationA Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in Education
Peter Windle
 
JEE1_This_section_contains_FOUR_ questions
JEE1_This_section_contains_FOUR_ questionsJEE1_This_section_contains_FOUR_ questions
JEE1_This_section_contains_FOUR_ questions
ShivajiThube2
 
MASS MEDIA STUDIES-835-CLASS XI Resource Material.pdf
MASS MEDIA STUDIES-835-CLASS XI Resource Material.pdfMASS MEDIA STUDIES-835-CLASS XI Resource Material.pdf
MASS MEDIA STUDIES-835-CLASS XI Resource Material.pdf
goswamiyash170123
 
Pride Month Slides 2024 David Douglas School District
Pride Month Slides 2024 David Douglas School DistrictPride Month Slides 2024 David Douglas School District
Pride Month Slides 2024 David Douglas School District
David Douglas School District
 
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
MysoreMuleSoftMeetup
 
special B.ed 2nd year old paper_20240531.pdf
special B.ed 2nd year old paper_20240531.pdfspecial B.ed 2nd year old paper_20240531.pdf
special B.ed 2nd year old paper_20240531.pdf
Special education needs
 
Chapter 3 - Islamic Banking Products and Services.pptx
Chapter 3 - Islamic Banking Products and Services.pptxChapter 3 - Islamic Banking Products and Services.pptx
Chapter 3 - Islamic Banking Products and Services.pptx
Mohd Adib Abd Muin, Senior Lecturer at Universiti Utara Malaysia
 
Model Attribute Check Company Auto Property
Model Attribute Check Company Auto PropertyModel Attribute Check Company Auto Property
Model Attribute Check Company Auto Property
Celine George
 
Digital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments UnitDigital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments Unit
chanes7
 
Operation Blue Star - Saka Neela Tara
Operation Blue Star - Saka Neela TaraOperation Blue Star - Saka Neela Tara
Operation Blue Star - Saka Neela Tara
Balvir Singh
 
How libraries can support authors with open access requirements for UKRI fund...
How libraries can support authors with open access requirements for UKRI fund...How libraries can support authors with open access requirements for UKRI fund...
How libraries can support authors with open access requirements for UKRI fund...
Jisc
 
The Challenger.pdf DNHS Official Publication
The Challenger.pdf DNHS Official PublicationThe Challenger.pdf DNHS Official Publication
The Challenger.pdf DNHS Official Publication
Delapenabediema
 
Francesca Gottschalk - How can education support child empowerment.pptx
Francesca Gottschalk - How can education support child empowerment.pptxFrancesca Gottschalk - How can education support child empowerment.pptx
Francesca Gottschalk - How can education support child empowerment.pptx
EduSkills OECD
 
1.4 modern child centered education - mahatma gandhi-2.pptx
1.4 modern child centered education - mahatma gandhi-2.pptx1.4 modern child centered education - mahatma gandhi-2.pptx
1.4 modern child centered education - mahatma gandhi-2.pptx
JosvitaDsouza2
 
South African Journal of Science: Writing with integrity workshop (2024)
South African Journal of Science: Writing with integrity workshop (2024)South African Journal of Science: Writing with integrity workshop (2024)
South African Journal of Science: Writing with integrity workshop (2024)
Academy of Science of South Africa
 
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Dr. Vinod Kumar Kanvaria
 
STRAND 3 HYGIENIC PRACTICES.pptx GRADE 7 CBC
STRAND 3 HYGIENIC PRACTICES.pptx GRADE 7 CBCSTRAND 3 HYGIENIC PRACTICES.pptx GRADE 7 CBC
STRAND 3 HYGIENIC PRACTICES.pptx GRADE 7 CBC
kimdan468
 
S1-Introduction-Biopesticides in ICM.pptx
S1-Introduction-Biopesticides in ICM.pptxS1-Introduction-Biopesticides in ICM.pptx
S1-Introduction-Biopesticides in ICM.pptx
tarandeep35
 

Recently uploaded (20)

Normal Labour/ Stages of Labour/ Mechanism of Labour
Normal Labour/ Stages of Labour/ Mechanism of LabourNormal Labour/ Stages of Labour/ Mechanism of Labour
Normal Labour/ Stages of Labour/ Mechanism of Labour
 
Synthetic Fiber Construction in lab .pptx
Synthetic Fiber Construction in lab .pptxSynthetic Fiber Construction in lab .pptx
Synthetic Fiber Construction in lab .pptx
 
A Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in EducationA Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in Education
 
JEE1_This_section_contains_FOUR_ questions
JEE1_This_section_contains_FOUR_ questionsJEE1_This_section_contains_FOUR_ questions
JEE1_This_section_contains_FOUR_ questions
 
MASS MEDIA STUDIES-835-CLASS XI Resource Material.pdf
MASS MEDIA STUDIES-835-CLASS XI Resource Material.pdfMASS MEDIA STUDIES-835-CLASS XI Resource Material.pdf
MASS MEDIA STUDIES-835-CLASS XI Resource Material.pdf
 
Pride Month Slides 2024 David Douglas School District
Pride Month Slides 2024 David Douglas School DistrictPride Month Slides 2024 David Douglas School District
Pride Month Slides 2024 David Douglas School District
 
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
 
special B.ed 2nd year old paper_20240531.pdf
special B.ed 2nd year old paper_20240531.pdfspecial B.ed 2nd year old paper_20240531.pdf
special B.ed 2nd year old paper_20240531.pdf
 
Chapter 3 - Islamic Banking Products and Services.pptx
Chapter 3 - Islamic Banking Products and Services.pptxChapter 3 - Islamic Banking Products and Services.pptx
Chapter 3 - Islamic Banking Products and Services.pptx
 
Model Attribute Check Company Auto Property
Model Attribute Check Company Auto PropertyModel Attribute Check Company Auto Property
Model Attribute Check Company Auto Property
 
Digital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments UnitDigital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments Unit
 
Operation Blue Star - Saka Neela Tara
Operation Blue Star - Saka Neela TaraOperation Blue Star - Saka Neela Tara
Operation Blue Star - Saka Neela Tara
 
How libraries can support authors with open access requirements for UKRI fund...
How libraries can support authors with open access requirements for UKRI fund...How libraries can support authors with open access requirements for UKRI fund...
How libraries can support authors with open access requirements for UKRI fund...
 
The Challenger.pdf DNHS Official Publication
The Challenger.pdf DNHS Official PublicationThe Challenger.pdf DNHS Official Publication
The Challenger.pdf DNHS Official Publication
 
Francesca Gottschalk - How can education support child empowerment.pptx
Francesca Gottschalk - How can education support child empowerment.pptxFrancesca Gottschalk - How can education support child empowerment.pptx
Francesca Gottschalk - How can education support child empowerment.pptx
 
1.4 modern child centered education - mahatma gandhi-2.pptx
1.4 modern child centered education - mahatma gandhi-2.pptx1.4 modern child centered education - mahatma gandhi-2.pptx
1.4 modern child centered education - mahatma gandhi-2.pptx
 
South African Journal of Science: Writing with integrity workshop (2024)
South African Journal of Science: Writing with integrity workshop (2024)South African Journal of Science: Writing with integrity workshop (2024)
South African Journal of Science: Writing with integrity workshop (2024)
 
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
Exploiting Artificial Intelligence for Empowering Researchers and Faculty, In...
 
STRAND 3 HYGIENIC PRACTICES.pptx GRADE 7 CBC
STRAND 3 HYGIENIC PRACTICES.pptx GRADE 7 CBCSTRAND 3 HYGIENIC PRACTICES.pptx GRADE 7 CBC
STRAND 3 HYGIENIC PRACTICES.pptx GRADE 7 CBC
 
S1-Introduction-Biopesticides in ICM.pptx
S1-Introduction-Biopesticides in ICM.pptxS1-Introduction-Biopesticides in ICM.pptx
S1-Introduction-Biopesticides in ICM.pptx
 

(ebook) Understanding and Evaluating Research (McGregor 2018).pdf

  • 1.
  • 3. This book is dedicated to my wonderfully intelligent husband, Peter. The idea and the book were conceived and written during the second two years of our retirement. Now we have the future together on our boat, without a computer in sight. 3
  • 4. Understanding and Evaluating Research A Critical Guide Sue L. T. McGregor 4
  • 5. FOR INFORMATION: SAGE Publications, Inc. 2455 Teller Road Thousand Oaks, California 91320 E-mail: order@sagepub.com SAGE Publications Ltd. 1 Oliver’s Yard 55 City Road London EC1Y 1SP United Kingdom SAGE Publications India Pvt. Ltd. B 1/I 1 Mohan Cooperative Industrial Area Mathura Road, New Delhi 110 044 India SAGE Publications Asia-Pacific Pte. Ltd. 3 Church Street #10-04 Samsung Hub Singapore 049483 Copyright © 2018 by SAGE Publications, Inc. All rights reserved. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher. Printed in the United States of America Library of Congress Cataloging-in-Publication Data Names: McGregor, Sue L. T., author. Title: Understanding and evaluating research : a critical guide / Sue McGregor. Description: Thousand Oaks : SAGE Publications, [2018] | Includes bibliographical references and index. Identifiers: LCCN 2017034708 | ISBN 9781506350950 (pbk. : alk. paper) Subjects: LCSH: Research—Evaluation. Classification: LCC Q180.55.E9 M34 2018 | DDC 001.4—dc23 LC record available at https://lccn.loc.gov/2017034708 This book is printed on acid-free paper. Acquisitions Editor: Helen Salmon Editorial Assistant: Chelsea Neve Production Editor: Bennie Clark Allen Copy Editor: Melinda Masson Typesetter: C&M Digitals (P) Ltd. Proofreader: Susan Schon Indexer: Robie Grant 5
  • 6. Cover Designer: Anupama Krishnan Marketing Manager: Susannah Goldes 6
  • 7. Brief Contents 1. Preface 2. Acknowledgments 3. About the Author 4. PART I • INTRODUCTION 1. Chapter 1 • Critical Research Literacy 5. PART II • PHILOSOPHICAL AND THEORETICAL ASPECTS OF RESEARCH 1. Chapter 2 • Research Methodologies 2. Chapter 3 • Conceptual Frameworks, Theories, and Models 6. PART III • ORIENTING AND SUPPORTIVE ELEMENTS OF RESEARCH 1. Chapter 4 • Orienting and Supportive Elements of a Journal Article 2. Chapter 5 • Peer-Reviewed Journals 7. PART IV • RESEARCH JUSTIFICATIONS, AUGMENTATION, AND RATIONALES 1. Chapter 6 • Introduction and Research Questions 2. Chapter 7 • Literature Review 8. PART V • RESEARCH DESIGN AND RESEARCH METHODS 1. Chapter 8 • Overview of Research Design and Methods 2. Chapter 9 • Reporting Qualitative Research Methods 3. Chapter 10 • Reporting Quantitative Methods and Mixed Methods Research 9. PART VI • RESULTS AND FINDINGS 1. Chapter 11 • Statistical Literacy and Conventions 2. Chapter 12 • Descriptive and Inferential Statistics 3. Chapter 13 • Results and Findings 10. PART VII • DISCUSSION, CONCLUSIONS, AND RECOMMENDATIONS 1. Chapter 14 • Discussion 2. Chapter 15 • Conclusions 3. Chapter 16 • Recommendations 11. PART VIII • ARGUMENTATIVE ESSAYS AND THEORETICAL PAPERS 1. Chapter 17 • Argumentative Essays: Position, Discussion, and Think-Piece Papers 2. Chapter 18 • Conceptual and Theoretical Papers 12. Appendix 13. References 14. Index 7
  • 8. Detailed Contents Preface Acknowledgments About the Author PART I • INTRODUCTION Chapter 1 • Critical Research Literacy Introduction to the Book Research Literacy Defined Benefits of Research Literacy Critical Research Literacy Balanced Appraisals Judging Chains of Reasoning Skeptical and Critical Scrutiny Uncritical Readings of Research Misplaced Trust Overindulgent and Undiscriminating Surface-Level Reading Reading Deeper for Ideologies and Paradigms Ideologies Paradigms Critically and Uncritically Reading Research Chapter Summary Review and Discussion Questions PART II • PHILOSOPHICAL AND THEORETICAL ASPECTS OF RESEARCH Chapter 2 • Research Methodologies Introduction Conceptual Confusion, Slippage, and Clarity Research Paradigm Research Methodology Research Traditions Confusion Ensues Theory and Method Choices Methodological Approach Used in This Book Methodological Responsibility in an Ideal World Philosophical Axioms Positivism and Postpositivism Positivistic Research Paradigm Postpositivistic Research Paradigm 8
  • 9. Empirical, Interpretive, and Critical Methodologies Habermas’s Theory of Communication Matching Methodology With Research Intent Quantitative, Qualitative, and Mixed Methods Methodologies Quantitative and Qualitative Methodologies Mixed Methods Methodology (Mixing Assumptions) Research Methodology and Research Question Alignment Writing the Research Methodology Section of a Paper Chapter Summary Review and Discussion Questions Chapter 3 • Conceptual Frameworks, Theories, and Models Introduction Role of Theory in Research Inquiries Descriptive Theory Typology Taxonomy Relational Theory Relational Research Explanatory (Predictive) Theory Qualitative Explanatory Concepts and Constructs Characteristics of a Good Concept and Construct Distinguishing Among Conceptual Frameworks, Theories, and Models Terminology Confusion Conceptual Frameworks Presenting a Conceptual Framework in a Research Paper Theories Overarching Types of Theories Scientific Theories Qualitative Theories Philosophical Theories Mathematical Theories and Theorems Disciplinary Theories Purposes of Theory in Research Deductive Versus Inductive Theory Elements of a Theory Assumptions Propositions Hypotheses Characteristics of a Good Theory 9
  • 10. Writing the Theoretical or Conceptual Framework Section of a Paper Theory in Quantitative Papers Theory in Qualitative Papers Elucidate Essence of the Theory Justify Theory Selected for Study Use Theory to Interpret Data Critiquing Theory in a Paper Models Chapter Summary Review and Discussion Questions PART III • ORIENTING AND SUPPORTIVE ELEMENTS OF RESEARCH Chapter 4 • Orienting and Supportive Elements of a Journal Article Introduction Orienting Secondary Elements Supportive Secondary Elements Authors’ Names, Credentials, and Affiliations Article Title Characteristics of a Good Title Abstracts Formatting Abstracts Keywords References Plagiarism and Paraphrasing Citation Styles Footnotes and Endnotes Appendices Formatting Appendices Acknowledgments Biographies Chapter Summary Review and Discussion Questions Chapter 5 • Peer-Reviewed Journals Introduction Journal Editor, Editorial Boards, and Facilitating Editors Guidelines for Authors Roster of Peer Reviewers Editorial Board Journal Publishers Peer Review Process Types of Peer Review 10
  • 11. Open and Extended Peer Reviews (Prepublication) Postpublication Peer Review Acceptance Rates Journal Impact Factors Journal Title Journal Volume, Issue, and Pagination Year of Publication and Article Placement in Issue Special Topic and Themed Issues Electronic Journals Open Access Journals Predatory Open Access Journals Chapter Summary Review and Discussion Questions PART IV • RESEARCH JUSTIFICATIONS, AUGMENTATION, AND RATIONALES Chapter 6 • Introduction and Research Questions Introduction Etymological Roots Lay Versus Research Questions Research Topics, Problems, Questions, Statements, and Objectives Research Topics Research Problems Quantitative Research Problems Qualitative Research Problems Mixed Methods Research Problems Casuist, Theoretical, and Practical Research Problems Research Goals Basic Versus Applied Research Sources of Research Problems Characteristics of a Good Research Problem Research Questions or Hypotheses Qualitative Research Questions Quantitative Research Questions and Hypotheses Mixed Methods Research Questions Research Statements Quantitative Research Statements Qualitative Research Statements Mixed Methods Research Statements Research Objectives Opening Sentences and Opening Points in an Introduction Six Approaches to Writing Introductions 11
  • 12. 5W Framing of an Introduction Three-Phase Approach (Niche) to Introductions Five-Paragraph Model for Introductions Four-Paragraph Model for Introductions Research Argument Model Problem Statement (Propositional) Model for Introductions Delimitations Chapter Summary Review and Discussion Questions Chapter 7 • Literature Review Introduction Etymology and Definition of Literature Review Purposes of a Literature Review When to Conduct a Literature Review Quantitative Literature Review Conventions Qualitative Literature Review Conventions Mixed Methods Literature Review Conventions Conducting a Literature Search Keywords and Search Engines Types and Sources of Documents Primary and Secondary Sources Steps for Searching and Finding Documents Selecting Studies for the Literature Review Ending the Search Critically Reviewing the Collection of Literature Writing the Literature Review Section Synthesis Synthesis Matrix Organizational and Presentation Strategies Historical Chronological Conceptual Theoretical Thematical Methodological (Meaning Methods) Major Ideas and Trends Argumentative Hypotheses Characteristics of a Good Literature Review Critical 12
  • 13. Important Relevant Timely Coverage Point of View Rigorous, True Accounting Technical Elements of Writing Effective Literature Review Sections Person and Voice Tense Length of Literature Review Section Number of References Transitioning From a Summary, Critical Evaluation to a Gap Transitioning From One Major Point to Another Minimize Quotations, Responsibly Paraphrase Chapter Summary Review and Discussion Questions PART V • RESEARCH DESIGN AND RESEARCH METHODS Chapter 8 • Overview of Research Design and Methods Introduction Etymology and Definition of Methods and Research Design Research Design Research Inquiry and Research Design Articulating Research Purpose in Research Design Research Design as Logical and Logistical Quantitative Research Design Logic Qualitative Research Design Logic Mixed Methods Research Design Logic Most Common Research Designs Methods Methodology Versus Methods Purpose and Importance of the Methods Section Major Differences Between Qualitative and Quantitative Intellectual Inquiry Major Components (Report Subheadings) of Qualitative and Quantitative Research Integrity of Research Designs Integrity of Qualitative and Quantitative Research Designs Integrity of Mixed Methods Research Designs Technical Aspects of Reporting Methods Length Organizational Logic and Approaches Objective Versus Subjective Writing 13
  • 14. Person, Tense, and Voice Person Tense Voice Chapter Summary Review and Discussion Questions Chapter 9 • Reporting Qualitative Research Methods Introduction Qualitative Emergent Research Designs Strategic Versus Tactical Research Designs Types of Qualitative Research Designs Interpretive Investigative Participatory Illuminative Instrumentation Sensitization Conceptualization and Theory Development Methods Appropriate for Qualitative Research Qualitative Site Selection and Access Qualitative Sampling Sample Size Researcher’s Role Qualitative Ethical Considerations Qualitative Data Collection Observations and Field Notes Individual Interviews Focus Group Interviews Document and Artifact Analysis Qualitative Data Management, Security, and Storage Integrity of Qualitative Research Designs Qualitative Data Analysis Data Interim Analysis Data Analysis Spiral Coding and Patterns Themes in Qualitative Data Computer Qualitative Software Programs Qualitative Study Limitations Chapter Summary Review and Discussion Questions 14
  • 15. Chapter 10 • Reporting Quantitative Methods and Mixed Methods Research Introduction Quantitative Research Design Scientific Method Deductive Reasoning Types of Quantitative Research Designs Descriptive Correlational Comparative Experimental Quasi-experimental Predictive Exploratory Survey Design (Nonexperimental) Integrity (Rigor) of Quantitative Research Designs Quantitative Instrument, Procedures, and Apparatus Quantitative Variables Quantitative Sampling Probability Sampling Random Assignment Sample Size and Representativeness Quantitative Ethical Considerations Quantitative Data Collection Quantitative Validity and Reliability Quantitative Data Analysis Quantitative Study Limitations Mixed Methods Research Design Philosophical Quandary Mixed Methods Main Research Designs Mixed Methods Degree of Data Integration Order of Mixing Methods to Ensure Data Integration Priority of Qualitative and Quantitative Methods Rigor in Mixed Methods Studies Rationale for Mixing Methods Chapter Summary Review and Discussion Questions PART VI • RESULTS AND FINDINGS Chapter 11 • Statistical Literacy and Conventions Introduction History of Statistics Laws of Probability and Chance 15
  • 16. Statistical Literacy Types of Statistical Data Nominal Data Ordinal Data Interval Data Ratio Data Measurement Scales Statistical Variables Dependent and Independent Variables Criterion and Predictor Variables Constants Controls Moderator Variables Intervening Variables Extraneous Variables Confounding Variables Spurious Variables Experimental and Nonexperimental Research Experimental Research Types of Settings Randomization Types of Experiments: True, Factorial, Quasi, and Single Case Nonexperimental Research Group Comparisons Correlational Surveys Case Studies Chapter Summary Review and Discussion Questions Chapter 12 • Descriptive and Inferential Statistics Introduction Basic Differences Between Descriptive and Inferential Statistics Descriptive Statistics Measures of Central Tendencies Measures of Variation Bell Curve Standard Deviation Measures of Relative Standing Measures of Relationships Contingency Tables 16
  • 17. Regression Analysis Reporting Descriptive Data Association and Causation Causal Inference and Inference of Association Moving From Association to Causation Recommended Causation and Association Vocabulary Inferential Statistics Hypothesizing Parameters (Hypotheses Testing) Hypothesis Testing Parameter Estimations Single-Point and Interval Estimates p Value and the Level of Significance Type I and II Errors Statistical Power (and Statistical Significance) Statistical Significance Sample Size and Effect Size Most Common Inferential Statistical Tests Reporting Inferential Statistics Conclusion Chapter Summary Review and Discussion Questions Chapter 13 • Results and Findings Introduction Distinguishing Between Results and Findings Purpose of Results or Findings Section With or Without Interpretation Qualitative Papers Quantitative Papers Jewel and Story Metaphors Reporting Qualitative Findings Organizing the Presentation of Findings Descriptions Versus Interpretations Formats for Reporting Qualitative Findings Research Purpose and Reporting Findings Themes Major Topics Using Quotations How Many Quotes? Presenting Quotes Thick Descriptions 17
  • 18. Relating Qualitative Findings to the Literature Review Using Numbers to Report Qualitative Findings Length Tense, Person, and Voice Reporting Quantitative Results Interpretation or Noninterpretation Comprehensiveness Negative Results Missing Data Organizing Quantitative Results Presenting Numerical Results Text Tables Symbolic, Visual Representations Transitioning From Results to Discussion Length of Results Section Person, Voice, and Tense Reporting Mixed Methods Results and Findings Principle of Integration Barriers to Integration Meta-inferences Integration via Connecting, Merging, or Embedding Connecting Merging Embedding Narratives Data Transformation Joint Data Displays Triangulation Fit of Data Integration (Coherence) Organizing Mixed Methods Results and Findings Section Number and Nature of Papers to Publish and Where Conclusions: Unique Challenges of Three Approaches Chapter Summary Review and Discussion Questions PART VII • DISCUSSION, CONCLUSIONS, AND RECOMMENDATIONS Chapter 14 • Discussion Introduction Discussion Defined Discussion Versus Conclusion 18
  • 19. Discussion Relative to Results and Findings Purpose of Discussion Summarize Salient Results and Findings Explain Results and Findings Speculation and Hedging Examine Implications of Results and Findings Discussion and the Concept of Significance Dimensions of Research Significance Significance Versus Novelty Significant Versus Statistically Significant Theoretical Significance Fundamental Questions to Address in the Discussion Limitations Threats to Validity Internal Validity External Validity Temper Limitations Humility and Limitations Organizing the Discussion Section Summarize Key Outcomes, Then Systematically Discuss Organize From Specific to General Mixed Methods Studies Organize and Discuss by Hypotheses Research and Null Hypotheses Retaining the Null Hypothesis Discussing Unhypothesized Results Statistically Correct Language Organize by Research Questions or Themes Staying Focused and On Topic Adding New Literature in the Discussion Quantitative Research Qualitative Research Mixed Methods Research Adding New Results or Findings Patterns in the Data Stay on Point, Tangential Issues Ensuring Strong and Responsible Discussions Employ Alternative Interpretations Bring Alternative Interpretations to Unexpected Findings or Results Critical and Creative Perspectives 19
  • 20. Technical Strategies for Discussion Sections Length of Discussion Section Shortening the Length Tense, Voice, and Person Chapter Summary Review and Discussion Questions Chapter 15 • Conclusions Introduction Conclusion Defined Relating Conclusions to Discussion and Summaries Stand-Alone Conclusion or Combined With Discussion Summaries Versus Conclusions Summary Conventions Purpose of Conclusions Explore Broader Implications Suggest New Research Place Research in Disciplinary Context Identify Practice and Policy Innovations Closing Strategies for Effective Conclusions Provocative Question Quotations Anecdotes Profound or Provocative Thought Drama Call for Action Warning Relevant Narrative Things to Avoid in the Conclusions Section Do Not Apologize Avoid Emotional Appeals No New Information Technical Strategies for Writing Effective Conclusions Length of Conclusions Tense, Person, and Voice Vocabulary (Move Statements) Monosyllabic Words Syntax Chapter Summary Review and Discussion Questions Chapter 16 • Recommendations 20
  • 21. Introduction Recommend Defined Logical Connections Informing Recommendations Relating Recommendations to Conclusions Research Versus Nonresearch Recommendations Purpose of Recommendations Audiences for Recommendations Characteristics of Effective Recommendations Language for Communicating Recommendations Actionable Verbs Imperative Verbs Modal Verbs Modal Force Certainty Possibility Obligation Crafting and Organizing Effective Recommendations Scope of Recommendations Prose Versus List (Numbered or Bulleted) Options for Presenting Lists of Recommendations Stand-Alone and Supportive Statements Technical Strategies for Presenting Recommendations Length Active and Passive Voice Grammar Chapter Summary Review and Discussion Questions PART VIII • ARGUMENTATIVE ESSAYS AND THEORETICAL PAPERS Chapter 17 • Argumentative Essays: Position, Discussion, and Think-Piece Papers Introduction Argumentative Essays Western Logic Comparing Position, Discussion, and Think-Piece Papers Position (Opinion) and Discussion Papers Length, Person, Tense, and Tone Transition Words Think-Piece Papers Length and Format Format for Argumentative Papers Introduction/Thesis 21
  • 22. Body/Antithesis Evidence Conclusion/Synthesis Persuasive Writing and Rhetorical Appeals Aristotle’s Three Forms of Rhetoric Rhetorical Triangle Ethos Pathos Logos Deductive and Inductive Argumentation Deductive Argument Inductive Argument Logical Fallacies Fallacies Chapter Summary Review and Discussion Questions Chapter 18 • Conceptual and Theoretical Papers Introduction Chapter Caveats Steps in Developing a Theory Choosing a Theory for a Study Double Meaning of Conceptual Framework Purpose of Conceptual Versus Theoretical Papers Purpose of Conceptual Papers Purpose of Theoretical Papers Structural Elements of Conceptual and Theoretical Papers Special Role of the Literature Review Theory Paper Literature Review Conceptual Paper Literature Review (Role of Integration) Elements of a Conceptual Paper Introduction Literature Review Presentation of the New Conceptualization Discussion and Implications Conclusions Elements of a Theoretical Paper Title and Abstract Introduction Context for the New Theory Presentation of the New Theory 22
  • 23. Discussion Conclusions Stage of Theory Being Published in a Theoretical Paper Conceptual Development Operationalization Confirmation or Disconfirmation (Testing) Application Rigor in Conceptual and Theoretical Papers Modes of Reasoning y-Axis Judging Reasoning and Argumentation Chapter Summary Review and Discussion Questions Appendix Worksheet for Critiquing a Journal Article or Research Report References Index 23
  • 24. 24
  • 25. Preface The inspiration for this book came while I was teaching graduate-level courses on research literacy (2010– 2014). I could not find a book that contained everything I thought should be taught in a course about how to critically read and use someone else’s scholarship. While teaching the course five times, I prepared a roster of detailed PowerPoint presentations about ideas not found in related texts. These topics especially included (a) the powerful link between research methodology and everything else in the research enterprise; (b) the far- reaching role played by conceptual frameworks, theories, and models in research; and (c) the criteria for judging argumentative essays and theoretical papers, which differ so much from a conventional research report. I could find “methods textbooks” in abundance (how to do research yourself), but less so textbooks about how to be research literate (how to critique someone else’s research and the final report). I remember thinking that if I, a seasoned academic with 30 years of experience, did not know this “stuff,” then others might not either. With this in mind, I approached SAGE with an idea for such a book, and signed off on the project mere weeks later. Kudos to Acquisitions Editor Helen Salmon for her confidence in this idea. The publisher realized that a book like this was needed. The result is this book titled Understanding and Evaluating Research, focused on helping people learn how to read, critique, and judge research reports prepared by others. The assumption is that one cannot critically read a research report without being aware of the conventions that authors are supposed to follow when conducting and reporting their research. As a caveat, the book does not strive to teach people to do their own research, but readers may learn more about how to do research by appreciating research conventions as they vary by methodology. 25
  • 26. Part I The book comprises 18 chapters, divided into eight parts. In Part I, the Introduction, Chapter 1 (“Critical Research Literacy”) begins by describing the approach I took to the book. Each chapter is prefaced with learning objectives, a sort of chapter abstract. To help readers engage with and apply chapter content, critical review questions, called Review and Engagement, are dispersed throughout each chapter. The premise is that after readers complete a critical assessment for each research component in a research report, they will complete the Appendix to help them decide if they can confidently use the study and its conclusions in their practice. The chapters all conclude with a wrap-up summary, followed with Review and Discussion Questions. I recommend you pose your own reflection and discussion questions as well. Your chapter takeaways will not be the same as mine. Chapter 1 then focuses on what it means to critically and uncritically read a research report. An uncritical reading means accepting the study without questioning it. 26
  • 27. Part II Part II is called Philosophical and Theoretical Aspects of Research. It begins with Chapter 2, which is the philosophical anchor for the book. Titled “Research Methodologies,” it reinforces the point that the philosophical underpinnings of research determine everything from the research question to the reporting format, and everything in between. After describing philosophical axioms, positivistic and postpositivistic research paradigms, and empirical, interpretive, and critical research methodologies, I confirm that the book is organized using the qualitative, quantitative, and mixed methods approach to categorizing methodologies. The chapter concludes with a discussion of the importance of aligning research methodology and research questions, and with some basic conventions for writing the research methodology section of a research report. Still in Part II, Chapter 3 is titled “Conceptual Frameworks, Theories, and Models.” It makes the case that without frameworks, theories, and models, research is weakened. The chapter discusses concepts and constructs, the building blocks of all three, followed with (a) a detailed discussion of conceptual frameworks, (b) an extensive coverage of theories, and (c) a shorter overview of models. The chapter also covers the reporting conventions pursuant to writing the conceptual and theoretical framework or model sections of a research paper. 27
  • 28. Part III Part III is called Orienting and Supportive Elements of Research. These terms were coined for this book as a way to recognize the importance of their role in reporting research. They act as bookends for the major (primary) components of a research report (methodology, theoretical underpinnings, introduction and research questions, literature review, methods, results or findings, discussion, conclusions, and recommendations). The orienting elements bring the paper to people’s attention (authors’ names, credentials, and affiliations; article title; abstract; and, keywords). The supportive elements help authors acknowledgments, contributions to the research (references, footnotes and endnotes, appendices, acknowledgements, and biographies). Chapter 4 (“Orienting and Supportive Elements of a Journal Article”) provides a rich discussion of each of these nine research elements, which are characterized as secondary, meaning they are important but for different reasons than the primary elements. Chapter 5, titled “Peer-Reviewed Journals,” focuses on the most common publication venue for research reports. Critical readers often use elements of the journal to judge the rigor of the scholarship, whether it is wise to do so or not. Starting with the role of journal editors, other types of editors, and editorial boards, the discussion shifts to the peer review process, acceptance rates, and journal impact factors. The discussion then turns to the journal’s title; volume, issue, and pagination; year of publication and the paper’s placement in the issue; and special or thematic issues. The chapter deals extensively with the new publication model of electronic journals, and discusses the possible threats to quality and rigor of open access journals, especially predatory journals. 28
  • 29. Part IV Part IV deals with issues pertaining to Research Justifications, Augmentation, and Rationales. Chapter 6 (“Introduction and Research Questions”) distinguishes between opening points (six types) and an introduction. Opening points lead the reader into the introduction, and the introduction leads people into the paper. The chapter covers research problems, questions, purpose (statements), and objectives. These are all discussed in extensive detail (as they pertain to qualitative, quantitative, and mixed methods) because savvy authors and critical readers need a clear understanding of how they differ yet are inherently interconnected. Next comes an extensive overview of six approaches that authors can use to develop an argument supporting what they are doing in their study. These approaches help authors convince people of the merit of their study, namely by critically reviewing literature related to the research problem, identifying what is missing (justifying their research question), and explaining how their study will fill the gap (research statement and objectives). Chapter 7 (“Literature Review”) covers how the literature review section of a research paper differs with qualitative, quantitative, and mixed methods research methodologies. Quantitative studies do the review before collecting the data, and qualitative studies tend to wait and do the review after the data are collected and are being analyzed. The chapter covers what is involved in critically engaging with previous work done about the research question, and explains in detail the process of synthesizing insights gained from this critical engagement. Nine organizational strategies are identified and discussed, showcasing the many choices authors have about how to present their review of the literature. 29
  • 30. Part V The next part is titled Research Designs and Research Methods. It comprises three chapters. Chapter 8 (“Overview of Research Design and Methods”) focuses on how the research methodology deeply affects the research design and the methods employed in a study. After distinguishing between methodology (philosophy, introduced in Chapter 2) and method (tasks to sample, collect, and analyze data), the discussion turns to how method and research design are different (but often used interchangeably). Readers learn the differences between qualitative and quantitative inquiries, followed with a general overview of the basic steps involved in each of qualitative and quantitative research designs. These steps are then used to organize the extensive, wide-ranging, comprehensive profile of both qualitative and quantitative methods, including seven qualitative research designs (Chapter 9, “Reporting Qualitative Research Methods”) and seven quantitative research designs (Chapter 10, “Reporting Quantitative Methods and Mixed Methods Research”). Chapter 10 concludes with a discussion of the mixed methods approach, which involves using both qualitative and quantitative strands, informed by the principle of data integration. In addition to design rigor, mixed methods research has a criterion of interpretative rigor, concerned with the validity of the conclusions (strand-specific and meta-inferences). 30
  • 31. Part VI Part VI, Results and Findings, has three chapters. After addressing what it means for authors and critical readers to be statistically literate, Chapter 11 (“Statistical Literacy and Conventions”) turns to the four main types of data (nominal, ordinal, interval, and ratio) and measurement scales, and then provides an overview of the nine most commonly used statistical variables. The chapter ends with the distinctions between experimental research designs (true, factorial, quasi, and single-case) and nonexperimental research designs (group comparisons, correlational research, surveys, and case studies). The criteria of randomization, variable manipulation, experimental and control groups, types of claims (causal and association), and types of research settings are used to differentiate between these two basic quantitative research designs. Chapter 12 (“Descriptive and Inferential Statistics”) starts with a discussion of the four types of measurements for descriptive statistics (i.e., central, variable, relative, and relationships), which includes a detailed overview of such basic conventions as standard deviation, the bell curve, contingency tables, chi-square, regression correlational coefficients, and scattergrams. The topic then shifts to inferential statistics, prefaced with a section on association and causation. Inferential statistics is concerned with (a) hypotheses and estimates, (b) p value and level of significance, (c) Type I and II errors, (d) statistical power (sample size and effect size), and (e) the most common inferential statistical tests. Chapter 13 (“Results and Findings”) discusses conventions for authors to follow when conducting and reporting the outcomes of their study. As with most other chapters in the book, it is organized by research methodologies: qualitative, quantitative, and mixed methods. Some challenges for qualitative research are balancing both describing and interpreting the data, providing supportive evidence for the existence of a theme, learning the protocols around using participants’ quotations, and providing thick and rich descriptions (so as to ensure high-quality findings that can be trusted). The section on quantitative results deals with issues of comprehensiveness (when answering the research question or hypothesis), missing data and negative results, organizational strategies for presenting results, and conventions for reporting numbers (text, tables, and figures). The chapter ends with an overview of reporting protocols and conventions for mixed methods studies, especially the principle of data integration, and the steps/strategies researchers can take to ensure integration before data collection, during analysis, or both. The importance of moving from strand-specific inferences to meta-inferences is discussed as are barriers to integration in mixed methods studies. The importance of addressing the fit of data integration is followed with various organizational approaches and a discussion of how and where to publish the study to reflect the degree of data integration. 31
  • 32. Part VII Part VII, Discussion, Conclusions, and Recommendations, provides a chapter on each research element. Chapter 14 (“Discussion”) begins by distinguishing between discussion points and concluding remarks, followed with an overview of the different conventions for relating results and findings to a discussion of their import. Then, the three main functions of a discussion section are addressed (summarize, explain, and examine), as are the topics of hedging and speculation. Discussion sections should address three basic questions, dealing with how the study (a) relates to existing understandings in the literature, (b) brings new understandings, and (c) makes new connections. After discussing these in some detail, the conversation turns to study limitations and how authors need to explain how threats to validity affect their conclusions and implications. The conversation then focuses on five major approaches to organizing a discussion section, shifting to an overview of strategies to keep the discussion section on track and focused. After explaining the importance of employing alternative interpretations of the results and findings, it ends with the role of critical and creative thinking when preparing discussion points. Chapter 15 (“Conclusions”) begins by advising authors of the need to appreciate how concluding comments differ from, but are related to, discussion points, and how concluding comments relate to a summary of the paper. The focus then turns to the four main purposes of a conclusion section: explore broader implications; suggest new research; relate study to existing literature; and identify policy, practice, and theoretical implications. Next, the chapter profiles eight closing strategies for a conclusions section, ranging from provocative questions to warnings and personal narratives. Things to avoid when creating concluding remarks are also discussed, especially apologies and new information. If done well, the concluding thoughts make the paper resonate with readers, increasing the likelihood they will engage with the study in the future. Chapter 16 (“Recommendations”) deals with the section of a research report where authors call people to action. After distinguishing between conclusions and recommendations (and the different roles they play), the discussion turns to the nine purposes that recommendations can serve, followed with an overview of the eight characteristics of effective recommendations. The chapter then turns to the topic of verbs when writing recommendations, with a special focus on the 10 modal verbs and on modal force (the writer’s conviction when posing the recommendation). There are three modal forces (certainty, possibility, and obligation), and each is recognizable through the use of particular modal verbs. An extensive overview of strategies for crafting, presenting, and organizing a recommendation section is provided. The chapter ends with some basic grammatical conventions, especially the use of passive and active voice. 32
  • 33. Part VIII Part VIII, the final section of the book, is called Argumentative Essays and Theoretical Papers. The two chapters in this section are separated from the rest of the book because papers reporting research related to positions, opinions, think pieces, new conceptual frameworks, and new theories do not fit with the basic reporting conventions previously discussed in Parts II and IV–VII. Chapter 17 (“Argumentative Essays: Position, Discussion, and Think-Piece Papers”) focuses on argumentative essays, which are judged by the ability of the author to develop a convincing and well-reasoned argument. The chapter begins by comparing and contrasting the three most basic forms of argumentative essays: position (opinion) papers, discussion papers, and think pieces. This clarification is followed with a detailed overview of the format for writing an argumentative essay: thesis, antithesis, and synthesis. The discussion then turns to the concept of persuasive writing and the key role played by Aristotle’s rhetorical appeals: ethos (character), pathos (emotions), and logos (logic). The chapter then presents a very detailed discussion of both deductive and inductive argumentation, ending with a brief introduction to logical fallacies in argumentative essays. Chapter 18 (“Conceptual and Theoretical Papers”) is the last chapter in the book. Whereas Chapter 3 is about the role conceptual frameworks and theories play in the research design, and how to use and report on their use in a paper, this chapter is about how to write a paper that reports on a new conceptual framework or a new theory. They both require different formats. The chapter especially addresses the special role that the literature review plays in each paper. It is an integrative instrument in a conceptual paper and a theoretical-step instrument in a theory paper. The chapter then profiles the key structural elements in both types of papers, distinguishing between each one, explaining that they become the organizational subheadings in the published paper. What constitutes a good theory is addressed as well. The conversation turns to Lynham’s idea of what to include in a theory paper, determined by how far along it is (ranging from the conceptual stage to being successfully applied in practice). The chapter concludes with a detailed discussion of what criteria to use to judge the rigor of a conceptual or a theoretical paper—mainly its supportive, red-thread argument—and the author’s mode of reasoning (analytical and generative). 33
  • 34. Acknowledgments First and foremost, I want to thank my husband, Peter, for his intellect, deep patience, powerful critical listening skills, and ability to make me reflect and accept things when I am in pushback mode—all part of the creative process. Thanks to my longtime friends Liz Goldsmith, Anne MacCleave, Felicia Eghan, and Vicki Jeans for lending supportive and critical ears during this creative process. Nearer to the end of the writing process came Frank Miccolis and Chrisanne Moffatt Miccolis, new boating friends who were incredibly patient with me while I unloaded my stress and expressed my excitement! My mom, Ethel Tweedie, has always been there for me during every life endeavor, as has my nephew, Connor Hunter Tweedie Morton. This journey about research methodologies and their link to scholarship began with a doctoral student turned friend, Dr. Jenny Murnane. Our work formed the cornerstone of the book (see Table 2.1). It was followed with an opportunity to teach Focus on Research Literacy in our education graduate program. Thanks to Jim Sharpe, then the dean, for patiently waiting until I felt I was ready to teach it. And those students! They were something, approaching this overwhelming topic with anticipation, undaunted grace, and open-mindedness. I would not have written the book without them and their inspiration. Finally, thanks to the SAGE editors and their assistants. SAGE gratefully acknowledges the following reviewers for their kind assistance: Kristin Bodiford, Dominican University Kadir Demir, Georgia State University Scott Greenberger, Grand Canyon University Tanya Kaefer, Lakehead University Xyanthe Neider, Washington State University Carole S. Rhodes, Queens College, City University of New York Anita Rose, New York University 34
  • 35. About the Author Sue L. T. McGregor (PhD, IPHE, Professor Emerita) is a Canadian home economist (nearly 50 years) recently retired from Mount Saint Vincent University in Halifax, Nova Scotia. She was one of the lead architects for the interuniversity doctoral program in educational studies, serving as its inaugural coordinator. She has a keen interest in home economics philosophy, transdisciplinarity, research methodologies, and consumer studies. She is a TheATLAS Fellow (transdisciplinarity), a Docent in Home Economics at the University of Helsinki, recipient of the Marjorie M. Brown Distinguished Professor Award (home economics leadership), a Rhoda H. Karpatkin International Consumer Fellow, and recipient of the TOPACE International Award (Berlin) for distinguished international consumer scholars, especially consumer educators. Affiliated with 16 professional journals, she has nearly 170 peer-reviewed publications, four books, 31 book chapters, and 12 monographs. She has delivered 39 keynotes and invited talks in 14 countries. Dr. McGregor is also the principal consultant for McGregor Consulting Group (founded in 1991). Visit www.consultmcgregor.com, or contact her at sue.mcgregor@msvu.ca. 35
  • 37. 1 Critical Research Literacy 37
  • 38. Learning Objectives Clearly understand the unique focus of this book Describe the basic elements of a research report Define research literacy and describe the benefits of being research literate Explain the basic process of critiquing a research report: read, critically analyze, and assess (judge) Distinguish between research literacy in general and critical research literacy Become familiar with five things to look for when critically reading research reports Understand reasons why people may not critically read (judge) a research report Explain the power of ideologies and paradigms in research, and explain what deep reading means Compare and contrast critical and uncritical thinking and reading of research reports 38
  • 39. Introduction to the Book This is an introductory-level text in the sense that introduce stems from Latin ducere, “to lead” (Harper, 2016). This book strives to lead people through the research process by using a critical lens—hence, the title Understanding and Evaluating Research: A Critical Guide. The book is intended to help people learn how to critique other people’s research so they can use it with confidence. It is written in third person, explaining what authors should do at each stage of the research process while they respect the research conventions pursuant to the methodological (philosophical) approach used in their study (see Chapter 2). As readers gain a deeper appreciation of the expectations placed on researchers and authors to conduct and report research of high standards, they will concurrently learn how to critique the resultant study and the research report. In effect, the book is not intended to teach readers how to do research on their own. They will learn about it by way of osmosis, which is the gradual, often unconscious absorption of knowledge or ideas through continual exposure rather than deliberate learning (Anderson, 2014). As a way to learn during this process, readers should choose a journal article or research report they want to critically read. As they read each chapter in this book, they should engage with its content by applying it to critique the research report. This book facilitates this critical reading process by providing Review and Engagement checklists, which are strategically positioned throughout the chapters, close to the respective content. For example, as people read and learn about the conventions authors should follow when preparing the Methods section of a research report (see Chapter 8), they would critically review the paper’s Methods section to see if it meets the recommended standards for high quality. To help bring readers to this assessment, each chapter ends with a question such as “Taking all of the Review and Engagement criteria into account, what is your final judgment of the Methods section of the paper you are critically reading?” Figure 1.1 summarizes the basic stages of critiquing a research report (often a journal article). These steps include reading it, identifying the elements used to organize it, rereading it while critically analyzing and judging each element, and then assessing (judging) the entire paper for its quality as scholarship. The Appendix (available for download at https://study.sagepub.com/mcgregor) contains a document developed to help readers track their critical judgment of each separate element of a research report, leading to a cumulative judgment of the quality of that entire study and its formal report. For example, if the keywords, title, and abstract were judged inadequate but the methods, the results, and their discussion were clearly and rigorously reported, it would make sense to confidently judge the paper as good quality. On the other hand, if the authors used a catchy title, chose keywords wisely, and had a solid reference list but the methods, the findings, and their discussion were inadequately documented, it would make sense to lower the judgment of the research report (and perhaps the research enterprise itself). All chapters conclude with Review and Discussion Questions, another tool for engaging with the content of the specific chapter so as to better apply it when critically reading a research report. Furthermore, noting the etymological roots, origins, and meanings of the basic research terms adds linguistic flavor to each chapter. To illustrate, consider the phrase critical research literacy, the title of this chapter. Critical is Latin criticus, “to pass judgment on literature.” Research is Old French recercher, “intensive seeking.” Literacy is Latin literatus, “the 39
  • 40. quality or rank of being educated, or learned” (Harper, 2016). Being critically research literate thus means that someone has, through education, learned how to critically judge an effort to intensely seek and report something—in this case, new knowledge emergent from a study. Figure 1.1 Process of Critiquing a Journal Article This etymological example frames the main focus of this book, which is to help readers develop the ability to critically assess authors’ studies and research reports. By association, authors reading this chapter can gain insights into how to ensure their research study and the report meet the expected rigor of a critical perspective. This approach mirrors contemporary definitions of research literacy, which include being able to function, respectively, as critical readers and critical producers of research (Dryden & Achilles, 2004; Ingham- Broomfield, 2008; Shank & Brown, 2007; Williams, Mulkins, Verhoef, Monkman, & Findlay, 2002). To that end, this first chapter explains the concepts of (a) research literacy and (b) critical and uncritical readings of research. It serves as a preamble to separate chapters on each of the many elements of the research process—that is, what authors need to know and do when conducting and reporting research, and what critical readers need to know to judge its quality (see Figure 1.2). Separate chapters are provided for understanding the basics of statistical literacy (Chapters 11 and 12), for writing an argumentative essay (Chapter 17), and for writing a conceptual paper or a theoretical paper (Chapter 18). These types of papers do not follow the basic conventions for a research report. Figure 1.2 Basic Elements of a Research Report 40
  • 41. 41
  • 42. Research Literacy Defined The term research literacy comprises two concepts, research and literacy. Research is Modern French rechercher, “to examine closely, to see, to look for.” As noted earlier, it also stems from Old French recercher, with re, “intensive,” and cercher, “seeking.” Research thus involves the intensive, concentrated search for knowledge. Literacy is Latin literatus, “educated, or learned.” In the word literacy, the suffix -cy means “quality or rank” (Harper, 2016). Literacy thus means having the quality of being knowledgeable or educated and exhibiting the ability to recognize and understand ideas, in this case, about research. Taken together, research literacy can be defined as the ability to locate, understand, critically evaluate, and apply scholarly works—that is, to become discerning and knowledgeable about research (i.e., the search for new knowledge) (Dryden & Achilles, 2004). Beyond these etymological definitions, research literacy is defined as understanding research language and conventions. It is a foundational block of research capacity, defined as “the ability to design and conduct [and report] research studies” (Williams et al., 2002, p. 14). Shank and Brown (2007) proposed that the term research literate refers to scholars being literate in how to conduct and report research. But it also refers to those who are reading the resultant reports. They need to be able to judge that scholarship before they use it. 42
  • 43. Benefits of Research Literacy Achieving research literacy is important for several reasons. Foremost, it helps alleviate the fear of not being able to access and assess research and scholarly publications. The more research literate people become, the more confidence they gain in judging others’ work. Research literacy entails both criticizing and critiquing scholarship, privileging the latter. Both of these words have the same Latin root, criticus, “to pass judgment on literature” (Harper, 2016). But while (a) criticizing the research process means finding fault with it, (b) critiquing it means assessing the strength of arguments and their supportive evidence, including the authors’ interpretations and conclusions drawn from their analysis and their discussion of the data. This accrued reader confidence is possible because, with experience, people build up knowledge of methodologies, theories, methods, and entire bodies of literature. Informed with this knowledge, confident readers are less inclined to erroneously rely on flawed scholarship. Research literacy encourages (actually necessitates) people to become critical readers and thinkers of others’ scholarly work (Kurland, 2000; Suter, 2012). Conversely, engagement with research enhances critical thinking and predisposes people to critique the scholarship. Indeed, critical readers of research both understand and engage with the research (Kattiyapornpong, Turner, Zutshi, Hagel, & Fujimoto, 2011). If people are engaged with something, they become involved with it, and it holds their attention (Anderson, 2014). This engagement means people go beyond merely reading the research report (superficial); rather, they dig deeper, holding the scholarship to high standards (Hart, Poston, & Perry, 1980; Shank & Brown, 2007). The next sections expand on this idea, in conjunction with being uncritical. Review and Engagement Critical readers will □ Appreciate what it means to be research literate in general □ Appreciate what it means to be critically (and uncritically) research literate (see Table 1.2) □ Recognize the benefits of being critically research literate □ Distinguish between being critical of and critiquing research □ Understand that critical engagement with a study means paying attention to, and getting involved with, it (not just a superficial, lay reading) □ Have a fundamental understanding of the basic elements of a research report (see Figure 1.2) and the conventions involved in preparing it, appreciating that most elements contain some margin of error 43
  • 44. Critical Research Literacy Critical research literacy adds the dimension of judging the value of a study’s contribution to theory, knowledge, and practice. Recall that critical is Latin criticus, “to pass judgment on literature” (Harper, 2016), so being critical means engaging in and expressing the merits and faults of literary works (i.e., formal writings) (Anderson, 2014). Being critical of research means carefully evaluating the scholarly work (i.e., looking for biases, unspoken assumptions, underlying ideologies, prejudices, quality, and rigor) and then judging the research, being able to defend one’s position (see Figure 1.3). People cannot maintain a critical perspective when reading research unless they can unpack what it means to be critical and uncritical (Shon, 2015; Suter, 2012). Figure 1.3 What to Look for When Critically Reading Research Reports 44
  • 45. Balanced Appraisals In more detail, Lunsford and Lunsford (1996) claimed that scholars and professional practitioners have a duty to critically review the literature and the research in their discipline and profession. Becoming “educated readers and interpreters of professional research literature” (p. 24) requires gaining the necessary skills and competencies to critically analyze research; that is, they have to learn to read research reports from a critical stance. Kurland (2000) agreed, claiming that readers have a responsibility to themselves and to others to monitor their reactions to a research report and to strive to understand the author’s point of view. This understanding has to occur before the text can be critiqued. The essence of the successful critique of a research paper is a balanced appraisal, meaning readers look for both merits and demerits (strength and weaknesses), achieved using logic and objectivity (Harris, 2014; Ingham-Broomfield, 2008). A balanced appraisal is accomplished via a logical and systematic assessment of the paper, grounded in critical awareness (see Figure 1.1). This means to strive to “be more questioning; try to see more than one side of an argument; try to be objective rather than subjective; weigh the evidence; make judgements based on reason, evidence or logic; look at the meaning behind the facts; identify issues arising from the facts; and recognise when further evidence is needed” (Ingham-Broomfield, 2008, p. 103). 45
  • 46. Judging Chains of Reasoning Once the research report has been critically assessed, readers are able to decide what to accept as true and useful. To do this, they must evaluate the evidence and the argument used by authors to reach their conclusions (Kurland, 2000). Kurland (2000) suggested that readers have to take control of the text they are reading (prepared by someone else) and become authors of their own understandings of the facts and their meanings as framed and argued by the authors. Only when readers fully understand a text can they truly critically evaluate the authors’ assertions. This understanding entails unearthing the authors’ purpose, persuasive arguments, and inherent biases. Authors choose content, language, writing styles, argumentative rhetorical styles, and logics for a reason. More than careful reading, critical reading of a research report involves actively recognizing and judging these linguistic elements (Kurland, 2000). From another perspective, Kurland (2000) posited that critical readers of research will deeply appreciate that the report reflects just one person’s portrayal of facts, one person’s take on the topic. That author prepared an argument, and chose specific evidence to support it, for a reason. A critical reader will suss out those reasons, which are normally informed by research paradigms, assumptions, biases, values, prejudices, opinions, agendas, and interests. Upon recognizing what a text says on the surface, a critical reader goes further and deeper, discovering what it means. The latter process involves critical interpretation of the research report so as to offer alternative, possible meanings or to challenge the evidence and arguments used by the author to make points and to draw conclusions (Kurland, 2000). A critical reading also entails stepping back and gaining distance from the research report. From this distance, readers can “launch into an intensive critical reading” (Knott, 2009, p. 1) of the document. They would be looking for (a) the authors’ central claims or purpose, (b) the background context and intended audience, (c) the kinds of reasoning and appeals used by the authors to develop their argument, and (d) their selection of evidence to support their arguments. Readers culminate this intensive critical reading with their evaluation of the strengths and weaknesses of the arguments comprising the report (Knott, 2009) (see Figure 1.1 and the Appendix). Once readers have critically assessed the research, they have to decide to what extent they will accept the authors’ arguments, opinions, and conclusions. This involves uncovering the authors’ (a) rationale for the evidence they selected and (b) construction of their arguments. Readers need to ask themselves several pertinent questions. “How well developed are the arguments leading to discussion points and conclusions? Is the author’s interpretation of the data convincing? Are the author’s conclusions supported by the preceding arguments? Does the author’s line of reasoning (or logic) make sense and hold together? Might there be alternative interpretations of data other than those proposed by the author? Are there any hidden assumptions that have to be questioned?” These questions constitute an engaged, critical reading of a research report (University of Leicester, 2009). Conscious readers of research will appreciate that all research contains a margin of error (an amount, usually small, that is allowed for) (Blackmore & Rockert, 2004). No study is perfect (Shank & Brown, 2007). Critical 46
  • 47. readers will acknowledge that these imperfections exist, accept this fact, and then determine whether these imperfections undermine the study’s rigor, rendering the results/findings meaningless or even harmful (Croad & Farquhar, 2005). Critical readers will read between the lines, unpack the article, and pass judgment on it at many levels (Shank & Brown, 2007). They will look beyond the minor issues (like a weak title or irrelevant keywords) and take a broad view to ask the big questions. “Readers who are critically literate in . . . research never lose sight of this big picture, even when they are happily wallowing in the [method] and stylistic details” (Shank & Brown, 2007, p. 227) (see the Appendix). 47
  • 48. Skeptical and Critical Scrutiny Knowing that all studies are imperfect, critical readers would bring a healthy dose of skepticism to the process of critically reading research reports. Mitzenmacher (2010) suggested that critical reading requires people to be suspicious. But skepticism would likely be more productive and fair because suspicious means distrustful while skeptical means being inclined to inquire and to question (Anderson, 2014). Critical reading involves “harder, more positive thinking” than does suspicion, which tends to tear something down or apart (Mitzenmacher, 2010, p. 1). If people are skeptical, they can confidently resist ideas presented by others instead of uncritically accepting them. Their intent is to inquire into the scholarship in order to find unspoken assumptions, fallacies, dogma, and, yes, outright errors in the methods or analysis (McGregor, 2006; Suter, 2012). Harris (2014) urged readers to find a balance between naïveté and cynicism by thinking critically about research. Taking this stance, they can “intelligently appraise research” (p. 106). “A little skepticism provides healthy protection against [missing] mistakes” (Locke, Silverman, & Spirduso, 2010, p. 69). Rather than getting bogged down by criticizing the minutiae, critical readers would focus on whether the scholarship is of high quality and meets the conventions for the research methodology employed in the study (Locke et al., 2010). For example, confirming long-held anecdotal assertions, Caldwell, Henshaw, and Taylor (2005) affirmed that people tend to use the quantitative criteria of reliability and validity to judge qualitative research (but not vice versa). They claimed the resultant criticism is unjustified because quantitative research should be judged using different criteria, including transferability, dependability, credibility, and confirmability. Critical readers would be able to evaluate and judge the quality of the evidence the authors presented, appreciating that different criteria are used for quantitative, qualitative, and mixed methods methodologies (Zardo & Pryor, 2012) (see Chapter 8, Table 8.5). In a lighthearted example, Suter (2012) shared this anecdote. A quantitative researcher critically evaluated a qualitative study, taunting the researcher with “What? Your conclusion is based on only one participant?” The qualitative researcher aptly responded with “What? Your conclusion is based on only one experiment?” Suter affirmed that this rivalry and disrespect is unjustified because both research “‘camps’ . . . value rigorous data collection and analysis coupled with sound, logical arguments that characterize scientific reasoning, namely a compelling chain of evidence that supports the conclusions. Both camps are keenly aware of rival . . . and alternative explanations of their findings” (p. 345). Locke et al. (2010) concurred, acknowledging “the paradoxical fact that there are pervasive similarities” between quantitative and qualitative research paradigms (p. 80). Wallace and Wray (2011) tendered a model for discerning readers that better ensures they critically scrutinize a study. They cautioned readers to remember that each study was conducted and the report written by particular authors, with a particular purpose and audience. These authors intentionally constructed an argument to convince that audience of the merit of their research. Critical readers have to try to discern what the authors were striving to achieve. They have to work out the structure of the argument and try to identify the main claims (see Chapter 6). As they do this, critical readers should adopt a skeptical stance, checking to make sure the evidence supports the claims. This involves considering any underlying values, assumptions, 48
  • 49. paradigms, or biases guiding the authors and influencing their claims. Throughout this whole critical process, readers have to keep an open mind and be willing to be convinced (but not cajoled). Review and Engagement Critical readers will □ Recognize and understand the differences among biases, assumptions, prejudices, and agendas (see Figure 1.3) □ Appreciate that a critical reading of a paper necessitates a balanced appraisal (look for both strengths and weaknesses), doing so objectively and using logic □ Learn how to locate and evaluate the authors’ argument and how to evaluate any evidence they used to make their claims (their conclusions) □ Practice stepping back from a paper to gain critical distance, and only then pass judgment on its merits and demerits, at many levels (see Figure 1.1) □ Bring a healthy dose of skepticism to the process of critically reading a research report (i.e., confidently resist ideas while remaining open to inquiry and to questioning) 49
  • 50. Uncritical Readings of Research Short of stating the obvious, uncritical reading of research means people accept it without challenging anything. Uncritical means not using one’s critical faculties (by choice or lack of ability). Un is Latin “not,” and critical is “to judge.” So, being uncritical means not judging, in that people accept something too easily because of being unwilling or unable to critique it (Anderson, 2014; Harper, 2016). People can be uncritical for several reasons (see Figure 1.4), to be discussed in more detail. Figure 1.4 Nuances of Uncritical Reading of Research 50
  • 51. Misplaced Trust Some people may innocently approach research assuming it is true and valid. This misplaced trust is unfortunate because they may end up using a weak, flawed study (Locke et al., 2010). Valid is Latin validus, “strong” (Harper, 2016). It also denotes the condition of being true (Locke et al., 2010). Fundamentally, internal validity (strength) refers to whether the study truly dealt with what was being studied. The data gathered have to match the research question. External validity refers to the results being valid (i.e., remaining truthful or meaningful) beyond the confines of the study. Different research methodologies and traditions use different terminology for these two aspects of strength and truth/meaning (Hart et al., 1980; Locke et al., 2010) (see Chapter 8, Table 8.5). Readers should also look for logical validity, meaning whether the report holds together and makes sense. Called internal consistency, it refers to whether each element of the research design logically flows from the others. For example, do the conclusions reflect the discussion points, which should be anchored in the results or findings and interpreted using the literature and any theory or conceptual framework underpinning the study (see Chapter 16, Figure 16.2)? If not logically valid, the report may be suspect and not usable (Wiersma & Jurs, 2009). At the crux of the matter is that authors must not deliberately insert falsehoods into their research reports; any errors or flaws have to be unintentional (Shank & Brown, 2007). 51
  • 52. Overindulgent and Undiscriminating Uncritically using flawed data, logic, or conclusions can have very negative consequences (Croad & Farquhar, 2005). Critical readers of research would know when to suspend their trust in a research report (Locke et al., 2010), but this is not always the case. Some people are relatively knowledgeable of the research process but are undiscriminating, meaning they do not consistently apply critical standards of analysis when reading studies. They run the risk of uncritically accepting the study as good research when it is actually compromised. They are too indulgent of the authors’ claims and conclusions, overlooking flaws or being too tolerant of sloppy or shoddy scholarship. It is irresponsible, let alone uncritical, to disregard the requirement that authors should be held accountable to accepted standards and conventions of sound scholarship. Reading research is a collaborative event, with both writer and reader bearing responsibilities to ensure that communications do not break down. Every article is a conversation and a potential dialogue (Locke et al., 2010; Shank & Brown, 2007). 52
  • 53. Surface-Level Reading Locke et al. (2010) advised that if they are unable to detect a flaw yet sense something is amiss, critical readers should shift from reading for general content to examining details pertaining to the rationale for the study; variables or phenomena; settings, contexts, and samples; methods for data collection and analysis; discussion points; conclusions; and the logic used to present the study. Inadequacies in any of these basic research elements can set off alarms and warning bells. If people persist in reading the article at the surface level while opting not to critique the research (or being unable to), their takeaways from the study will lack intellectual depth or thoroughness. They will end up using cursory, one-dimensional, and shallow interpretations of the study when actually their research or practice context demands more than a superficial, uncritical reading. Their naïveté hinders critical readings of studies. If they are also gullible (easily deceived), their unsuspecting demeanor may cause them to rely on scholarship without examining the authors’ assumptions and any hidden agendas (see next section). 53
  • 54. Reading Deeper for Ideologies and Paradigms At a deeper level, uncritical readers may approach a research report unaware of the power of ideologies and paradigms. The consequences of this can cut two ways: (a) Readers may be unaware of which paradigm they bring to the exercise (or which ideologies are steeped into their psyche), and (b) they may not be able to discern the authors’ worldview or appreciate that authors are favoring a particular ideology that is clouding their perspective. Indeed, some authors purposely promote an ideology dear to their heart rather than conduct an honest inquiry. On the other hand, readers may be very aware of the power of ideologies but choose to find fault with the research (criticize it) instead of critically reading it. Readers’ unfounded suspicion that the author is harboring ideological motives may actually cramp their ability or inclination to read critically (Locke et al., 2010). Ideologies In more detail, ideologies (a system of ideas and ideals held by a group) are orientations to the world that characterize the thinking of a group or a nation. Paradigms (patterns) are individual thought patterns influenced by the ideologies. Ideologies come before paradigms. Ideologies are understood to be cultural blueprints, while paradigms are thought patterns for interpreting the world shaped by this blueprint. Ideologies are the ruling ideas of the times. They are assumptions about what is worthy of belief and attention, accepted as true, and valued. They pertain to how society should work and be arranged, and they provide the rules deemed most appropriate for achieving this ideal societal arrangement (see Table 1.1, which portrays only the dominant ideologies and paradigms, for illustrative purposes. There is a cadre of contending ideologies and paradigms, but their discussion is beyond the scope of this chapter) (see Donovan, 2010; Eaton, 1996; Elgin & LeDrew, 1997; Heuerman & Olson, 1998; McGregor, 2008, 2013; McGregor, Pendergast, Seniuk, Eghan, & Engberg, 2008). Table 1.1 Dominant Ideologies and Paradigms Seeping Into Research Agendas Dominant Ideologies (cultural blueprints of desired society) Attendant Paradigms (perspectives on a world shaped by the ideologies) Capitalism: an economic orientation that values profit, wealth accumulation, growth, production, and technological progress • Control, mastery, management, and efficiency • Property ownership, dominance, profit, and competition • Mechanistic orientation, focused on parts (rather than holistic) • Transmissions and transactions 54
  • 55. Patriarchy: a system that privileges and conveys power to men and marginalizes and disempowers women and other presumably weak people • Dualism about every aspect of the world (One side of the binary pair is desirable because it is powerful; the other is undesirable and is in a powerless or marginalized position.) • Hierarchies that divide and separate human beings into categories such as gender, class, economic status, and political power Neoliberalism: economic and political notions that favor business with nominal roles for governments • Relativism (quick fix, no absolute truth) • Basic tenets: individualism, privatization, decentralization, and deregulation Economic globalization (from the top down): corporate- and elite-led neoliberalism focused on integrating national economies into a global economy; concerned with the pace of integration • Newtonianism (disconnected and fragmented) • Causal relationships, linearity, predetermination • Reductionism (reduce everything to categories, specializations, micro analysis) • Mastery over nature, resources, and marginalized people (exploitation, extraction, control) Political ideology of conservatism: the idea that societies should maintain (conserve) the status quo of the ruling elites and accept change only reluctantly, at a very slow pace; things endure because they work • Favors business with nominal roles for governments; minimal regulation of the market • Social hierarchy and social inequality viewed as inevitable, natural, normal, even desirable • Expansive military policies and spending are assumed to protect the country and, by association, the family • Scarcity mentality and competition for 55
  • 56. Social Darwinism: survival of the fittest; monies and support for elders, children, the sick, and people experiencing life transitions are considered to be wasted because these people are not economically productive members of society scarce resources • Evolutionism (natural selection—the strong survive and thrive) • Win–lose mentality • Elitism (privilege the wealthy and powerful, who should be rulers) • Division of labor based on gender Fundamentalism: strong maintenance of and adherence to any set of ancient or fundamental doctrines and beliefs (moral codes), usually in the face of criticism or unpopularity (especially religious beliefs and social and political movements) • Extremism (far outside the acceptable mainstream attitudes of society), especially religious extremism • Free-market fundamentalism (the market can solve social ills) • In-group and out-group distinctions must be maintained • Rejection of diversity of opinions • Moral intolerance and adherence to strict social conventions Consumerism: inculcates the values of the western consumer lifestyle on a global scale; society’s preoccupation with the accumulation of goods and the procurement of services formally self-performed • Materialism (using things to measure success, with little concern for spiritual or ethical matters) • Material gains and possessions privileged over relationships • Conspicuous consumption that may be unethical and immoral • Popular culture postmodernism (novelty, commodification, entitlement, communication technologies) Paradigms Paradigms are habits of thinking in a particular way or of making certain assumptions (others call this a worldview or a mind-set). Paradigms profoundly affect the way people perceive reality as it is lived within 56
  • 57. society’s dominant belief systems (i.e., its ideologies). Paradigms affect how people respond to their perceptions of the world, including those of a research report. Ensconced in their paradigms, people use them to make sense of their world by giving meaning to their lived experiences (see Table 1.1). A common metaphor used to explain this connection is an ideological camp where people live out their lives. When they climb up into an observation tower and view life below in the camps, they are seeing the world through their paradigms. The camps reflect the cultural blueprint by which all people are supposed to live, and the paradigms are the way people think about and make sense of that life (Donovan, 2010; Eaton, 1996; Elgin & LeDrew, 1997; Heuerman & Olson, 1998; McGregor, 2008, 2013; McGregor et al., 2008). Example 1.1 A study inadvertently reinforcing patriarchy-informed policy Consider a study that explored the impact of social welfare policy on families. The policy said that once a man moves into a single woman’s household, she no longer qualifies for welfare assistance because it is assumed that the man will be working and contributing money to the household. Even if he is not working, the policy still holds because a man is there. Under this policy, welfare assistance is automatically cut off. Researchers conducted focus groups and interviews with women receiving this assistance. They reported undue financial hardship because of this policy, which opponents called draconian, meaning excessively harsh. Despite the profound insights gained from these data, the researchers concluded that the policy was sound, in effect reinforcing the patriarchy ideology. It holds that the male is the head of the household and the main breadwinner. If a male is present (any male), it is assumed that women will be taken care of and do not deserve financial assistance using public tax dollars. Government bureaucrats cited the study as justification for maintaining the policy, and actually made it even harder for women to regain assistance once the man left. When opponents to this policy approached the researchers, challenging their assumptions and conclusions, the research team revealed they had simply not seen it from that perspective before (standing from their observation tower) and regretted drawing their conclusions. They had not intended to harm women with their study, but their unexamined ideological lens had the same effect. In a research context, articles inherently reflect authors’ assumptions about the phenomenon under examination and any surrounding contexts. Those assumptions are informed by ideologies and paradigms, which, for most people, go unexamined at the best of times. They are so ordinary in everyday life, even in research life, that they become invisible and go unchallenged. Despite that authors should be transparent about any assumptions underpinning their research, this often does not happen (Neuman, 2000). Critical readers would be aware of this and be prepared to suss out and question those assumptions. Actually, most authors intend to do good research and intend for readers to access their ideas so their work can be understood and used. Critical readers of research will respect these intentions, appreciating the insidious power of ideologies and paradigms (Locke et al., 2010). Example 1.2 Dominant paradigm (thought pattern) in research A good example of a dominant thought pattern in research is the assumption that quantitative research is more valuable and 57
  • 58. legitimate than qualitative research because the former is empirical and positivistic, grounded in measurable numbers and verifiable statistics. Qualitative research is often maligned and called soft science, meaning it is hard to quantify, assuming that all data have to be quantified to be true. People’s meanings, wisdom, and interpretations of their own lives are not measurable; hence, they do not matter. These assumptions reflect the deeply ingrained paradigms of Newtonianism and positivism. Newtonian thinking holds that objective reality comprises predictable, measurable, linear, cause-and-effect phenomena. Positivism assumes that the only way one can be positive that one has knowledge is if it is produced using the scientific method. Qualitative studies are often undervalued, minimized, and even dismissed as not good research because they cannot be judged as empirically valid and reliable, nor do they yield desirable proof of cause and effect (see Chapter 2). Review and Engagement Critical readers will □ With practice, come to recognize when they are being gullible, naïve, too trusting, innocent, and unsuspecting (see Figure 1.4) □ Conversely, know how to check for the internal consistency in authors’ logic, when to suspend their trust in the report, how to read beyond the surface level, and how to dig deeper to expose ideologies and paradigms □ Appreciate the persuasive power of ideologies and paradigms in research and learn to critically recognize when they are at play, affecting the authors’ message (intentionally or not) (see Table 1.1) □ Examine their own research paradigms and ideological awareness, thereby becoming responsible partners in the critical research conversation 58
  • 59. Critically and Uncritically Reading Research In summary, “being a critical consumer of research is more than just reading academic papers; it involves thinking critically about the assumptions behind research, the methodologies [and methods] employed and the implications that research results [or findings] hold for practice” (Zardo & Pryor, 2012, p. 1). Suter (2012) compared the qualities of critical and uncritical readers of research (see pp. 10–11), summarized in Table 1.2. A critical reader of research will be able to navigate or move with confidence and competence through research material and reports in order to critically evaluate and effectively use the information and new knowledge (Locke et al., 2010). Lacking this critical navigational ability can lead to a compromised knowledge base as well as compromised practice, policy, theory development, and future research (Zardo & Pryor, 2012). Example 1.3 Uncritiqued research project shaping policy Croad and Farquhar (2005) took issue with an actual New Zealand study about the link between the quality of early childhood education and children’s future learning. Touted as a significant study, it has influenced government policies and educational practice in New Zealand. The authors critically analyzed the study and found several issues that severely compromised its real contribution to the topic of competent children. They discovered shortcomings around “the sampling technique and the composition of the sample, the research design, data analysis, and interpretations of the data. Assumptions made in the research about children’s competencies, quality in early childhood education and the measurement of quality therefore need to be questioned” (p. 18). They challenged the authors’ and the media’s uncritical claims about the study and admonished government officials, educators, and practitioners to critically examine and openly discuss it. They claimed that too much is at stake for people to uncritically use a study with so many unchallenged limitations. Both superficial reporting and using a study purported to be a significant policy resource are irresponsible and shortsighted. “Publically funded policy-driven research which is used to legitimate education policy and spending should not be exempted from scrutiny and critique” (p. 17). Table 1.2 Comparison of Critical and Uncritical Readers of Research Reports Critical Thinkers and Readers Uncritical Thinkers and Readers • Search for biases • Accept information without scrutiny • Recognize logical fallacies and inconsistencies • Overlook, cannot recognize, or fall victim to fallacies (flaws in logic) • Reason with clarity, precision, and relevance • Fail to recognize disjointed or irrelevant reasoning 59
  • 60. • Look to see if authors considered the context when reaching conclusions • Accept stereotypes and overgeneralizations • Confirm that conclusions are justified (supported by data) and well reasoned • Accept conclusions without challenging the authors’ reasoning and the evidence • Assess the validity (strength and truth) of all claims • Accept claims unchallenged, without questioning their validity • Unearth and examine authors’ assumptions • Confuse assumptions about what is true (with no proof) with facts • Discern whether authors accounted for complexity when judging import of study • Upon reading a paper, come to snap judgments about the study (oversimplify things) • Determine if authors considered alternative explanations (and come up with their own counter thoughts) • Tend to confirm personal biases (i.e., favor only evidence that reinforces their own preexisting beliefs) On a concluding note, the rest of the book is focused on sharing detailed and nuanced discussions of accepted conventions and standards of conducting and reporting high-quality research. The intent is to help readers become better equipped at, and more predisposed to, being critical readers of research and scholarship. There are chapters on each of the basic elements of the research process (see Figure 1.2 and the Appendix), prefaced with a chapter that thoroughly discusses the main approaches to categorizing research methodologies (Chapter 2), which differ from yet inform research methods. Upon reading Understanding and Evaluating Research, readers should feel more comfortable assuming the mantle and deep responsibility of critiquing others’ research. Review and Engagement Critical readers will □ Be able to practice being a critical thinker and reader when critiquing research reports (see Table 1.2) □ Appreciate the necessity of being able to critically navigate a research report, with critical awareness and critical prowess Chapter Summary This chapter introduced readers to the intent of the book, which is to help people critically read and evaluate other people’s scholarship before they use it in their research or practice. Readers were introduced to the idea of research literacy and why it is important. The discussion then turned to the topic of critical research literacy, starting with what is involved in reading critically, followed with why some people may not critically read a research report (see Figures 1.1 and 1.2). The former included a discussion of the need for balanced appraisals, to judge the author’s chains of reasoning, and to be skeptical and engage in critical scrutiny. These strategies help critical readers deal with the author’s biases, (unspoken) assumptions, interests, and prejudices (see Figure 1.3). People may fail to critically judge a research report for several reasons, including misplaced trust, being overindulgent and undiscriminating, reading at just the surface level, and failing to account for the power of ideologies and paradigms in research (see 60
  • 61. Figure 1.4 and Table 1.1). The chapter concluded with an overview of what constitutes critical and uncritical thinking and reading of a research report (see Table 1.2). 61
  • 62. Review and Discussion Questions 1. There are many reasons why it is important to be research literate. Identify three that are most important to you, and explain why they are so important. 2. How is criticizing a research report different from critiquing it? Are both skills needed? Which of these should be privileged, and why (see Figure 1.1)? 3. Compare and contrast what it means to be a critical reader and an uncritical reader of a research report (see Table 1.2). 4. What are five key things critical readers should look for when reading a research report (see Figure 1.3)? 5. What does someone have to do to be considered an uncritical reader of research reports (see Figure 1.4)? 6. Why is it important to critically understand a research report? 7. How do ideologies and paradigms differ? Explain how they are interconnected. What role do ideologies and paradigms play in research (see Table 1.1)? 62
  • 63. Part II Philosophical And Theoretical Aspects Of Research 63
  • 65. Learning Objectives Appreciate the history of key methodological terms Recognize the necessity of being able to defend any methodological choices made at the interface between philosophy and methods (methodologically responsible) Distinguish clearly between methodology and methods (as used in this book) Become familiar with the conceptual confusion, slippage, and clarity needed around three common terms: research paradigm, research methodology, and research tradition Appreciate the methodological approach used in this book (see Table 2.1) Explain the construct of philosophical axioms (epistemology, ontology, logic, and axiology) Distinguish between positivistic and postpositivistic research paradigms Compare and contrast empirical, interpretive, and critical research methodologies Compare and contrast quantitative, qualitative, and mixed methods methodologies Explain why it is necessary to match research methodology with the research question Understand the conventions for writing the research methodology section of a paper 65
  • 66. Introduction Research and inquiry are about creating new knowledge (Habermas, 1984). Philosophy is the study of the fundamental nature of knowledge, reality, and existence—its truths, principles, and assumptions (Anderson, 2014). This book is premised on the assumption that everything in research hinges on philosophical underpinnings. But making this point is challenging because of the proliferation of methodology-related terms arising in the late 1970s and peaking in the early 1990s. Egon Guba is credited with initiating the paradigm dialogue about quantitative and qualitative research (Donmoyer, 2008). Since then, researchers have witnessed the emergence of a dizzying array of jargon used by scholars trying to address this thorny but imperative aspect of research. This scenario is exacerbated by the fact that “many researchers lack experience [or expertise] in deliberating about methodological issues, and the esoteric and unfamiliar language of philosophy can be intimidating” (MacCleave, 2006, p. 9). This array of methodology-related terms includes research paradigms, methodologies, methods, philosophical axioms, quantitative, qualitative, mixed methods, positivism, postpositivism, empirical, interpretive, and critical (and one can add postmodernism, poststructuralism, constructivism, naturalistic inquiry, critical realism, and so on). Inconsistency in what these terms mean, alone and in relation to each other, is evident across all disciplinary literature (Cameron, 2011). Acknowledging this state of affairs, Locke, Silverman, and Spirduso (2010) sardonically noted that “the first tour through the research literature in your own area of interest is likely to reveal more variety than you would expect” (p. 80). They even coined the term paradigmatic subspecies (p. 80) to accommodate this diverse philosophical situation. The result of such philosophical diversity is terminological soup or, as Buchanan and Bryman (2007, p. 486) called it, “paradigm soup.” Actually, some of these terms have been in use for more than 400 years, adding to this linguistic and philosophical conundrum (see Figure 2.1) (Ary, Jacobs, & Sorensen, 2010; Denzin & Lincoln, 2011; Fox, 2008; Guba, 1990; Johnson & Christensen, 2012; Lockyer, 2008; Niglas, 1999; Paley, 2008; Smith, 1983). Nonetheless, researchers have the responsibility of explicitly identifying the methodological and paradigmatic underpinnings of their scholarship (Maxwell, 2013). To address this conceptual slippage, this chapter explains and justifies the approach used in this book (see Table 2.1), knowing that not everyone will agree with it. Regardless, researchers and authors have to “acknowledge the paradigm debate” and rigorously defend any methodological choices “made at the interface between philosophy and methods” (Cameron, 2011, p. 101). This due diligence is necessary because, to academics, these words can mean different things. Without conceptual clarity, the integrity of any academic conversation about the interface between philosophy, methodology, and methods is compromised. Figure 2.1 History of Methodologically Oriented Terms 66
  • 67. Table 2.1 Overview of Main Research Methodologies, With Common Methods (used with permission) [Acknowledgment: Deep, deep thanks to Dr. Anne MacCleave, Professor Emerita (MSVU), for vetting and validating the core concepts contained in this table] Research Paradigm Positivistic Postpositivistic Sources: From McGregor & Murnane (2010) with permission from John Wiley & Sons. Sources used by the authors to develop the appendix included: Howe, 1992; Lather, 1994; Niglas, 2001; MacDonald et al., 2002; Khazanchi and Munkvold, 2003; Guba and Lincoln, 2005; Ponterotto, 2005; Salmani and Akbari, 2008). Acknowledgment and deep thanks to Dr.Anne MacCleave, Professor Emerita Mount Saint Vincent University (MSVU), for vetting and validating the core concepts contained in this table. 67
  • 68. Conceptual Confusion, Slippage, and Clarity This section attempts the near impossible, to distinguish between the terms research paradigm, research methodology (compared to methods), and research traditions. All three terms are used in the academic world, leading to confusion because paradigm means thought patterns, methodology is linked with philosophy, and tradition refers to long-standing customs (see Figure 2.2). In truth, they all have some merit when trying to distinguish between (a) collecting new information (data) to answer a research question and (b) knowledge creation using interpretations of those data. On the other hand, the diverse language used to refer to this aspect of research has created a quagmire. This complex and difficult situation makes it hard for one scholar to talk to and understand another. But talk to each other they must, so this section briefly explains how the literature understands these concepts, settling on research paradigm and research methodology for this book (they mean different things). 68
  • 69. Research Paradigm Paradigm is Latin paradigma, “patterns” (Harper, 2016). A paradigm is “a generally accepted explanation of things,” with the dominant paradigm providing “the focal point and measuring stick” for inquiry (Rohmann, 1999, p. 296). Paradigms are thought patterns that help people make sense of their world, regardless of whether they are engaged in research or not. Paradigms are habits of thinking in a particular way or of making certain assumptions about the world (others call this worldview or mind-set) (Donovan, 2010) (see Chapter 1 for a discussion of paradigms and ideologies). The term research paradigm, coined by Kuhn (1962), is understood to mean “patterns of beliefs and practices that regulate inquiry within a discipline, doing so by providing the lenses, frames and processes through which investigation is accomplished” (Weaver & Olson, 2006, p. 460). Johnson and Christensen (2012) defined a research paradigm as a “perspective about research held by a community of researchers that is based on a set of shared assumptions, concepts, values, and practices” (p. 31). These definitions make sense. After all, disciplines are groups or communities of people, and paradigms reflect a group’s commitment to a constellation of beliefs about viewing the world. They are a group-licensed way of seeing reality (Botha, 1989). Normally, the philosophical notion of axioms is reserved for the term research methodology, as is the case in this book. Some scholars, however, characterize research paradigms by distinctive axioms, namely ontology, axiology, epistemology, rhetoric, causality and logic, and methodology (by which is meant the identification, study, and justification of research methods) (Guba, 1990; Pruyt, 2006). Figure 2.2 Research Paradigm, Methodology, and Tradition 69
  • 70. Research Methodology In many disciplines, the term methodology is used to refer to the methods used to collect, analyze, and report data (see Schneider, 2014; Trochim & Donnelly, 2007). This usage eschews the real meaning of methodology. Ology is Greek for a branch of knowledge or science. Method is Greek methodos, “the pursuit of knowledge” (Anderson, 2014; Harper, 2016). Taken together, methodology means a branch of science that studies the pursuit of knowledge. “The misuse of methodology obscures an important conceptual distinction between the tools of scientific investigation (properly methods) and the principles that determine how such tools are deployed and interpreted (methodology)” (American Heritage Dictionary, 2000). This chapter views methodology as the philosophical underpinnings of research intended to generate new knowledge and methods as tools and techniques to collect and analyze data (Lather, 1994; MacCleave, 2006) (see Figure 2.3). To that end, this chapter focuses on methodologies, and Chapter 8 focuses on methods (and research design). In particular, methodology refers to knowledge creation, including what counts as knowledge and knowing, reality, logic, and the role of values in knowledge creation (i.e., four axioms, to be discussed shortly). Two common approaches to describing research methodologies are (a) quantitative, qualitative, and mixed methods and (b) empirical, interpretive, and critical. These are discussed in more detail further on in the chapter. This book uses the former as its organizational framework. 70
  • 71. Research Traditions Actually, some academics skirt the contentious issue of whether to use the term research paradigm or research methodology and instead use the term research traditions (Jacob, 1987; Schneider, 2014). A tradition is an inherited pattern of thought and a specific practice of long standing (Anderson, 2014). Kuhn (1970) said any research tradition differs along three dimensions: (a) its assumptions about nature and reality, (b) the foci of studies and major issues of interest about the phenomenon, and (c) methodology (by which he meant methods). He also noted that a tradition can occur either as an entire discipline or as a school within a discipline (e.g., subdisciplines and disciplinary specializations). For example, Jacob (1987) applied this approach to profile three subdisciplines within the discipline of education. The term tradition is the least commonly used in the literature, but it was important to acknowledge it in this chapter because authors may choose to use it when reporting their study, or they might encounter it when reading literature. Patton (2002) identified 10 qualitative research traditions including constructivism, symbolic interaction, semiotics, hermeneutics, systems, and chaos (nonlinear dynamics). This book views these as falling within qualitative and interpretive research methodologies (see Table 2.1). Figure 2.3 Methodology Compared to Method 71
  • 72. 72