Producing Data: Analyzing Material,Culture & DocumentsChapter 7 of Taylor, B. C., & Lindlof, T. R. (2010). Qualitative communication research methods. SagePublications, Incorporated.• Material culture• As well as documents are “mute evidence”• They are unable to respond to the researcher’s questioning.• They have a lot to say when they are read alongside theliving voices of informants and other social actors.• Researcher must be ready to pay close attention to all theways in which people interact with things and texts in orderto gather evidence about how the material world evokesmeaning.• Collecting, reading and interpreting• Any humanly produced artifact; the corporeal, tangibleobjects constructed by humans; and the materialmanifestations of the social realities understood to berelevant and powerful.• Things owe their existence to a human source• Homo faber – human beings are always about the business ofmaking things, maintaining a degree of dominion over the materialworld.
Objects also posses a degree of agency They make their influence felt as soon as they appear. They do not evoke meaning in isolation but only in itsrelation to the meanings of other objects. They have the power to reconnect people with the traditionsand myths of their distant past or to serve as sense-makingdevices for future generations. While material culture may be expressive, it cannot formulate acomplex idea in the way as spoken language. A material object is looked not as the entity that communicates but as anelement in the process of communication. The study of material culture has a long history Is not the synonymous of folklore studies. Folklorists study, and sometimes act as collectors of, the indigenousarts, crafts, tools and built environment of subcultures. Archeology, social anthropology, sociology and consumerbehavior have used it and contributed to it. Communication is a latecomer to the field. Tendency of the communication researchers to regard objects andthe built environment as a mere backdrop or staging area for speechacts. Perceived nature of the objects.
MAKING MATERIAL CULTURE VISIBLE Reasons rooted in human history and the ongoing humanrelationship with the material world.1. Antimateriality – the theme of the body and other material objects ashaving nonessential, impure, vulgar, or even false status. Define both humanity and its higher pursuits as in direct opposition to thevulgarity of the material because they do not depend on the provisions ofmaterial forms. People are generally less comfortable with the idea that their interests are defined by whatthey consume or that objects help create the need for security.2. Social actors overlook the material culture Inconspicuous quality – hails our attention without resorting to language andrarely calls attention to itself, which makes it easier to take their physicalcharacter for granted. Overfamiliarity with concrete elements of a scene tends to breed undersensitivity to theirpresence and rhetorical effects. Great propagandistic value in the creation of a world of meaning – operates on us by subterfuge,evoking our memory of the discourses and cultural associations that have built up around it overtime.3. People often engage with an object in terms of its fitness for a particularproject. Theory of affordances – an object has built into its design a range of “actionand possibilities,” which affords certain kinds of interactions with capablehuman actors. Extends the concept of affordance from perceptual psychology into the realm of social
Material items do become explicit and visible in their ownright when they malfunction, go missing, are singled out inconversation, or become problematic in social relationships. Interviewing tactics can be used to aid people in “waking up” towhat they know and feel about their surroundings. Steps for making material culture visible for analysis:1. Identifying one or more objects for study. One common strategy is to choose an object with strong symbolicvalue in a particular context.2. Evaluate the similarities and differences of the object of study inrelation to objects in the same context. They will arise from comparing the discursive references to theseobjects in conversations, speeches, textual materials, ads, etc. Important for three reasons:1. It sensitizes you to those features of an object that elicit the most powerful orprominent meanings.2. You will be on firmer ground arguing for the boundaries of a particular context.3. You start to develop interpretations grounded in multiple levels of meaning.3. Expand the interpretations made within larger frameworks ofhistory, ideology, biography, and/or theory. The goal is to develop a deeper, multidimensional understanding The specific choice of frameworks to apply depends (a) on the natureof the place or objects being studied, (b) on the interests brought to thestudy by the researcher, or (3) as the result of ongoing encounters.
DOCUMENTS The study of documents can be very useful and a vitalsource of field data. You should always be on the look for them in the predictableplaces of a cultural scene. Familiarization will give you a “leg up” in framing sensiblequestions. Four uses in qualitative communication research:1. Often useful to study the “career” of a document Can lead to insights about the ways in which documents helpcoordinate interpretations and behaviors.2. Official documents are a site of claims of power, legitimacy, andreality. It is important to study these texts in relation to the institutional settingsin which they are constructed, interpreted, and used. Emphasis in the spatial, temporal, and practical contingencies associatedwith the texts.3. Communication events are encoded and preserved as documents.4. Many organizations and groups create documents of some sort forpublic consumption.
TYPES OF DOCUMENTS Primary sources – comprises the testimony of eyewitnessesof the event described. Usually preferred when the facts of an event need to be establish. Secondary sources – based on indirect (hearsay) evidence Media stories about an event Expressed opinion of social actors affected by but not directlyinvolved in an event. Records – any written or recorded statement prepared by orfor an individual organization for the purpose of attesting toan event or providing an account. Personal document – any written or recorded material otherthan a record that was not prepared specifically in responseto a request from an inquirer. Less constrained by organizational demands and requirementsthat can be read as genres of self-expession. A text considered primary in one context could be asecondary source in another context.
ADVANTAGES AND DISADVANTAGES Informational richness – contextually relevant andgrounded in the contexts they represent. Richness of documents derive not only from the amount ofinformation but also the quality. Availability – documents are almost always available,on a low-cost or free basis. Transition from stand-alone, analogical media to networked,digital media has had a major impact. Physical and online archives are both potential sources of bias andare both at the same risk of being incomplete, difficult to search, orpurged. The entity that controls access may impose conditions onthe ability to obtain, photocopy, or quote them. Ethical and legal questions about how it can or should be used forresearch purposes. Researcher should try to get the originals, but photocopies will sufficefor most purposes. Describe the document’s origins and history, who issued it, when and
Nonreactivity of the data – Many documents came into beingat the end of a lively social process or as a result of a deliberateand/or creative thought process by an individual. May have gone through many iterations during it span of lifeand maybe even morphed into something different. In most cases by the time the research arrives at a scene thedocument is a relatively inert, stable object. Major advantage – endures unchanged across time and space. A document’s meaning may vary across times, places, and readers Truth value – To the extent that the information contained indocuments is vetted for accuracy, is used as a reliable basis fororganizational decisions and actions, and/or is validated byinternal or external authorities, it may be regarded as atrustworthy source. Unless a problem with a document’s accuracy is noted,challenged, and repaired, or unless the users are indifferentabout its truth value, it may still be regarded as a trustworthysource. The evaluation of a document’s truthfulness depends on your
The Method of HistorySmith, M. Y. (1989). The method of history. Research methods in mass communication. EnglewoodCliffs, NJ: Prentice-Hall, Inc.• The issue of quantification in history raises thefalse dichotomy of whether the historian mayproperly use quantitative methods.• Whether historians will indulge in impressionistic researchand the subsequent necessary empty rhetoric.• Impressionistic history – random note-taking withoutapplying system and rigor to the data gathering.• Whether communication historians will disciplinethemselves to the rigor of empirical study.• Empirical history – the application of system and rigor to thestudy of the past.• The empirical historian fully recognizes that a systematic accountof relationships among events and/or persons is only a history ofcommunication and not the history of communication.• Seeks a verisimilitude rather than objective truth.• David Fischer – An empirical historian is one who asks an open-endedquestion about past events and answers it with selected facts which arearranged in the form of an explanatory paradigm.
FORMULATING THE RESEARCH QUESTION Thorough knowledge of general history andcommunication history. Practical approach: Saturation in historical readings andimmersion in communication history data. Begins when individual interest leads a researcher to seematerial about a general communication subject and then,when fortified by a general understanding of events, immerseitself in a general topic. Who, What, When, Where, Why & How Pursuit of Evidence Historians cannot properly begin formulating aresearch hypothesis before the immersion processnarrows into “guided entry” The researcher focuses in a more specific part of thedata delimiting the data to be studied.
GUIDED ENTRY The process of general question forming begins. A question must be worth asking A historical question ought to be neutral Receptive to both yes and no answers. All terms in the historical research question must becapable of being operationally defined The operational definition establishes categories orclassifications. The language of the question ought to be explicit andprecise Questions should not be based on unproved assumptions. The research question ought to be flexible enough thatit can be refined as the study progresses. Questions should be conceived with imagination andhonesty to define clearly the relationship beingobserved within certain parameters of time and place in
AUTHENTICATING THE DATA An authentic document is the product of the eyewitness, not aforgery. Evidence must be affirmative. All evidence must be clearly established by the testimony of independentwitnesses. The historian works by accretion of evidence from primary sources. Primary sources are eyewitness testimony with expertise in the area – noneed to be original as long as is firsthand testimony – person or thingpresent at the events and a narrator of those events who lacks bias andmotives of self-interest. Records – intentional eyewitness testimony; may function to obscure theinterpretation of a historical event. The motivation for the recordkeeping must beexamined, as well as the content. Relics – although also eyewitness testimony, were recorded for purposes otherthan to form a database for future historians. The historian should first try to establish the date of the document anddetermine whether the materials (ink, paper, type of film) were available atthat time. The source’s style of writing should be analyzed. The historian should ask whether the views expressed in the document are in
Anachronistic references to events in the documentshould also be searched. The document should have come from a line of well-authenticated, reputable owners, or it should have beenfound where it logically ought to nave been. Once data have been authenticated, the historian mustask how much of the authentic evidence is credible andto what degree. “Credible testimony is not that it is actually what happened,but that it is as close to what actually happened as we canlearn from a critical examination of the best availablesources.” Louis Gottschalk History of American Revolution – George Washington’s papers History of Watergate – memoirs of Richard M. Nixon The researcher should consider the tone of the newsmedium and how that affects reporting. The news medium’s general adherence to reasonablyacceptable standards of ethical and professional journalism,and the degree to which the publisher controls thenewsroom.
A particular caution in weighing credibility of evidence is theeffect of language, which as any cultural phenomenon,changes over time. Primary documents must be studied in context. Texts tell us what an actor did, not necessarily what it thought.Therefore, subjective documents may intentionally obscure whatthe actor did (Behavioral evidence v. Ideational evidence) Circumstantial evidence Should be used with care – Historians need to consider carefullythe deceptive possibilities of circumstantial evidence Should be considered inferior to direct evidence. Secondary sources – can be useful in establishing thehistorical context for the research problem. Research question and hypothesis that might be tested. The techniques for recording evidence should be mentionedbriefly. A full bibliographic reference must accompany every shred ofevidence. The important aspects of any system of data recording areaccuracy, completeness, and permanency.
SELECTING ANALYTICAL STRATEGIES Decisions on data analysis are made in response to theresearch question. The decision should have been made whether the historian uses themore traditional documentary analysis or quantitative analysis or both. Ideally, multiple measures are used to reduce uncertainty in interpreting resultswhen the historian reaches the same general conclusions through severalmeasures. Historians can use social science methods as a tool only if they have masteredhistoriography first. In considering data the historian must consider the nature ofhistorical reasoning, the documentary approach to dataanalysis and its advantages with incomplete data, theapplicability of quantitative methods, forms of data presentation,and literary style. Historical reasoning process is a process of adductivereasoning in the simple sense that a satisfactory explanatory fitis obtained Adduction permits the historian to respond to research questions with
To arrive at a verisimilitude of the past, the historianapplies traditional documentary analysis: To evidence too skimpy to permit quantification. To parts of quantified evidence to enhance and illuminate thehistorical record. In the course of extracting the credible material from theauthentic data, the historian: Notes the primary source’s interpretation of the situation andthe source’s behavior in the situation. The historian’s analysis includes reports on the feedback to theprimary source It should be noted whether the feedback was the intended result of thesource’s activity or whether it disagreed with the source’s aims. Defines other observer’s views of the situation and tracesthe anticipated and the unanticipated consequences of thesource’s activity Noting the relationships among those views and the primary sourcesideational and behavioral aspects, as well as the feedback. The information collected is organized in meaningfulfashion, depending on the structure of the researchquestion.
Two cautions should be exercise by the historian: Evidence should be interpreted in terms of the postulatedpast time Not to fall into the excesses of psychohistorians Quantification complements the traditional documentarymethod of analysis injecting rigor and system into it tomake more reliable generalizations. Content analysis permits systematic and objectiveclassification of communication content into categoriesaccording to specified characteristics so that inferences canbe made from the data. Every historian makes some leap of faith betweenevidence and generalization. However, the historiancannot predict on the basis of generalizations becauseits legitimate inferences are incapable of being tested bymeans of the full hypothesis. Exception – Case study, where little generalization, if any, ispossible. Has only limited value as a pretest for further
In making that leap of faith the historian goes showhigh probability The balance of chances that, given such and suchevidence, the event it records happened in a certainway, or, in other cases, that a supposed event did not infact take place. The historian is the judge of what constitutes highprobability In a quantitative approach, this is indicated in advanceand tested statistically In either case, the reader should be able to accept orreject the judgment made by the historian from theevidence presented. Difficulties in dealing with causal antecedents ofhistorical events can be controlled in the question-forming process when operational definitions areexplicated.
REPORT AND DATA PRESENTATION Should reflect the rigor and precision applied to the data,without, however, losing the literary quality of the report. Case studies – limited range of data Article, series, monographs or books – broad range of data Should begin with a conceptualization of the problemand a clear statement of the research question, followedby a literary survey appropriately integrated into thenarrative and an explanation of the research method, fulland precise presentation of the evidence responsive tothe research question, and finally, the generalizations inresponse to the research question. “You tell’em what you are gonna tell’em; you tell’em; andthen you tell’em what you told’em Rigorous, systematic history does not have to be dullhistory Communication history should be written in the best literary
Evaluating Information: Historical/Rhetorical-Critical MethodologyChapter 5 of Hocking, J.E., et al., (2003) Communication Research. Allyn and Bacon. Pp. 104-121• Three traditional classifications of research methods:• Historical/Rhetorical critical – general way of studying communications.• Studies a concept, event or person in great detail.• It is used by all researchers at some stage.• Reflects the careful and systematic assembling of all the authentic facts that may have abearing on the target of study.• Descriptive• Experimental• Your ability to generalize and to establish or test scientific theory is expanded.• These methods can be divided into two groups:• Humanistic – Qualitative• Criticism = creative activity that relies on the interpretative and observational powers of thecritic.• Scientific – Quantitative• These methods function by examining communication in a way so as to pointout aspects of the communication that the reader should be aware of.• In choosing a type of research and method the researcher indicates a philosophicalstance toward the phenomenon studied.• The historian’s methodology is tied to how the information is gathered.
AN OVERVIEW OF HISTORICAL RESEARCH Requires following systematic rules andprocedures rigorously in order to orderknowledge in such a way as to pass severaltests of critical analysis. Correct Appropriate Builds upon known facts and evidence in sucha way as to understand better the quality ofwhat happened and how certain events mayhave contributed to that event.
UNDERSTANDING THE MEANING OF HISTORY The means by which the researcher deals withthe latent meaning of history. Must get as close to the original source ofinformation about the original event as possible. Chronology of events or reflection of an eventoccurring in time and space Traces the events as they occurred in some time pattern Road map that provides insights into cause-effect relationships butnot to understand what happened. Spatial dimension of history – the events had to occursomewhere Helps to better key in on the unique contributions of the person,event, or institutions and make analysis.
AREAS OF POTENTIAL HISTORICAL RESEARCH Historical study falls into seven established areas and two suggestedpotential areas of study:1. Biographical or biographical/critical studies – concentrates onthe life of a particular person exploring not only what this person hassaid or done but also its biographers.2. Movement or idea studies – center on the development of socialmovements with a more sociological focus.3. Regional studies – examine the impact of geographical location.4. Institutional studies – center on the history of particularinstitutions.5. Case study – center on specific events, institutions, or persons at aspecific point in time.6. Selective studies – examine one particular aspect of a complexprocess.7. Editorial studies – focuses on translation of texts or discovery ofnew texts.8. *Bibliographic study – focuses on documentary research and thecreation of an information base.9. *Study of sources – study of those with whom the person studied
ADVANTAGES AND DISADVANTAGES Advantages Ability to know how the event turned out. Can ascertain whether or not the source agrees or disagreeswith history as it happened. Certain patterns that may help to explain why others saw orreported the event or person differently may become evident. Disadvantages The problem of determining accuracy Records may be inaccurate, incomplete or lost The result of new sources of data acquisition andretrieval available. Rather than reading all the documents identified, we may relyon the services of others. We may wait forever to conclude
HISTORICAL METHODOLOGY Historical Facts The researcher is seeking the facts as they might havebeen known at that time, which may be different fromthose we believe or know today. Facts are created through human perception, perceptions thatmay change as times or locations change. Are derived directlyor indirectly from historical documents and verified by others. A historical fact is something the researcher sees and uses to makea particular event clear that is dependent upon verification. Historical Documents Any evidence used to establish or help explain thetarget of study. Something the researcher seeks that may add to its knowledgeof the target of study.
Historical Research Blends the data obtained from various sources into a lucid andflowing narrative in a manner that illuminates the communicationand its effects. Its major functions are:1. Set out to find all available information2. Scrutinize that information in terms of accuracy3. Get the story out Sources, Information, and the Test of Evidence Primary evidence – evidence from the source itself. Testimony, documents, and evidence that comes from those actually engagedin the activities of the period. Secondary evidence – evidence from those of the time periodwho may have observed but did not actually engage in theactivities in question. Sources whose credibility or testimony cannot be verified through primarysources. Researcher must establish the context of the source and should carefullycross-check it for accuracy. Tertiary evidence – evidence based on the accounts of theprimary sources. May yield important clues to securing secondary or primary evidence.However, is accepted less frequently than secondary sources. Therefore, itmust be approached with greater caution and must be viewed skeptically it.
External evidence Finding two or more forms of the same material is not uncommon. Authenticity could be established through scientific tests onthe documents found. Examination of handwriting, paper and ink used, and whether thelanguage reflects the period under study. The source’s training as an observer, and establish the source’srelationship to the event, person, or institution. Ascertain whether you are working with the source’s own words. Speeches – be sure that the text you are working with is the one actually presented. Speeches before the Congress – be sure that the indicated speaker actually deliveredthe speech. Internal evidence – evaluates the statements made within thematerial following external testing. Source’s literal meaning – credibility of the document. The researcher must delve into the symbols and slang of the periodto gain an understanding of what the source was trying to say. Must take into account ambiguity, the use of figurative language, statements taken outof context, whether the statement reflects the target of the study, and whether or not isbelievable. Believability is based upon confirmation
Skepticism The acceptance or rejection of evidence must beapproached with skepticism. Conflicting testimony and information, repression of the truthby the ones in power. Analytical Strategies Deductive approach – reasoning from an idea or thesis,looking for evidence to support a thesis that would leadto new ideas or thesis statements. Inductive approach – when a particular piece ofevidence derives thesis or ideas concerningrelationships or events. Adductive approach (Lateral thinking) – the process ofadducing answers to specific questions so that asatisfactory explanatory fit is obtained. Holistic approach that examines the interactive impact of allpossible causes rather than singling out for specific attention. When the RQ suggests it is appropriate to do so, thehistorian can turn from a humanistic, qualitative approachto a quantitative, social scientific, behavioral approach.
RHETORICAL/CRITICAL METHODOLOGY Critical method is part of the humanistic approach tohistorical research. Is also considered an extension of the tests of internal evidence orcriticism the researcher uses when examining the question of howwell the act was accomplished. In conducting a critical study the researcher shouldexamine at least four aspects of the communication:1. the results What was the purpose of the communication? What was the truepurpose of the communication? What connection can be foundbetween the communication and the results?2. the artistic standards Refer to the use of appropriate rhetorical methods and principlesto examine how effectively or gracefully the intended affect of thecommunication was accomplished.3. the basis of the idea4. the motives and ethics behind the communication.
RHETORICAL CRITICISM/CRITICAL DISCOURSE Farrell’s three methods of performance of discourse, eachslightly different in its approach to the communicationevent or person:1. Symptom criticism – approaches the communicationevent or person as a symptom of some larger socialconcern. Tends to be more sociological examining the social features orthemes underlying the culture of the communication.2. Didactic criticism – a more traditional model that followsthe notion that the study of the event or person is themodel. Interested in the particular communicator and how this usescommunication as a tool.3. Thematic criticism – is interested in how thecommunication is constructed.
CONDUCTING CRITICAL EXAMINATION OFCOMMUNICATION The researcher begins by describing what has happened.It focuses on two sets of factors tied to the rigor of themethod:1. Extrinsic factors – the researcher seeks to: Describe and analyze the period in which the communicationoccurred. Describe the audience both in terms of the intended and thereal audience. Describe and analyze the occasion for the speech The researcher describe and analyze the speaker givingattention to its background and training. The communication itself is analyzed.2. Intrinsic factors – analysis of the communication itselfand the evidence with which it was constructed. How the logic fits with the period which the communicationwas presented. Intrinsic features stress the individual’s skills at creating andpresenting the communication.
LIMITATIONS OF HISTORICAL RESEARCH The researcher must be prepared forfragmentary and incomplete records. Facts change as language and meaningchange. The researcher’s personal bias, selectiveperception of what is right and what is not, andfaulty interpretation.
Describing Information:Content AnalysisChapter 8 of Hocking, J.E., et al., (2003) Communication Research. Allyn and Bacon. Pp. 170-192• Content analysis is the systematic study andquantification of the content or meaning ofcommunication messages that involves thecreation of categories that are designed to allowa particular research question to be answered bycounting the instances of content within amessage that fall within a certain predeterminedcategory.• There must be a good reason to use it – it must helpus answer an important research question.• Can be used to add accuracy and precision toobservations.• Provides a method for quantifying the messages thatwere produced during the event.• Can be used as a starting point for establishing theeffects of particular messages.
STEPS IN CONTENT ANALYSIS The methods of follow directly from and are largely guided bythe research question. The steps are:1. A literature review, which would help to focus and clarify theresearch questions or hypothesis that guide the study. The research question delimits the content to be analyzed in such away as to make the research doable and valuable.2. Define exactly what messages are to be studied. Here your population or universe of messages is defined.3. The selection of a sample. Messages that are representative of the entire body of messagesavailable for analysis. Must be large enough to be representative yet small enough ti be studiedfeasibly with available resources.4. Define the units of analysis The thing that is actually counted and assigned to categories.5. Create categories into which each unit of analysis would beassigned.6. The coding of the messages Carefully, systematically, and objectively
DEFINING THE METHOD Two directions that the researcher can take:1. Manifest aspect of the communication – the contentexactly as it appears.2. Latent aspect of the communication – the deepermeanings that are intended or perceived. At the coding level the researcher must be limited to manifestcontent, the latent question comes in the question goesbeyond the pragmatic aspect of the messages. In preparing to begin a research study you must:1. Decide what you will be studying. What would be yourunit of analysis?2. How to sort the data. What would be your categorysystem?3. Establish a sampling procedure.4. Acquire the data.
Delimiting the population Establish certain boundary conditions that will limityour search. Which one is appropriate? This will limit the generalizability of the findings. Units of analysis What you actually count and assigned categories. Their definition depends on how the variable(s) understudy are operationally defined. Barelson’s five major units (Table 8.1 p.178):Unit Description ExampleWord Manifest content Nouns, Proper names, Utterances (for example, “uh-huh”)Theme Latent content An idea, concept or thesis (Self-esteem, Democracy)(Subthemes)Character An individual Male/Female, Occupation, Ethnicity (Communicativebehaviors)Time/Space Physical or temporalmeasureColumn inches, Type size, Air time provided a program
How to separate the units of analysis into meaningfulcategories. Five general guidelines – underlie the systematic andobjective nature of content analysis. Content analysis categories should:1. Reflect the purpose of the research2. Be exhaustive3. Be mutually exclusive All units fit into only one of the categories4. Allow for independence The assignment of a unit to one category should not influence the assignmentof other units.5. All categories should reflect one single classification principle Counting – how many The most obvious and simple quantification is to count the unitsbeing analyzed in each category. Availability of data – decisions should be considered atthe research question/hypothesis stage.CREATING CATEGORIES FOR ANALYSIS
SAMPLING Census – conducted when the population to bestudied is either not large or is composed by alimited number of messages. Random and Systematic Sampling Random sampling – research question wouldconcentrate on understanding the messages of thisparticular person. Limited application. Systematic sampling – involves compiling a lost ofmessages and the selection of a random startingpoint and a constant skip interval. Not concerned with change in messages over time.
QUANTIFICATION Decisions on how to quantify or “code” thecategories should be approached early in theplanning process. One of the major goals is that all data should bepotentially quantifiable. Nominal quantification - The researcher ismerely naming (nominal) the characteristics of thevariables. Ethnicity, gender, job, category, region of the country… Ordinal quantification – requires some decisionregarding ranking the data. Requires the content to be placed in categories that aremutually independent and ordered in some way.
CODING RELIABILITY Coders should always use a standardized codingform to ensure reliability and accuracy from coderto coder. Researcher must be prepared to establish that thecoders are consistently placing data into the correctcategories. Intercoder reliability – can be enhanced by making certain thatall coders understand the definition of the categories and howand why the units of analysis were chosen. Coders should not be able to know what the researcher is seeking -they could introduce their bias (even unintentionally) Coding reliability – is enhanced when the units of analysis areclearly defined and they are assigned to mutually exclusiveand exhaustive categories. Pilot-test phase of research Practice sessions are important – is helpful to have the categories andthe operational definitions tested as well as the coders trained and
RELIABILITY COEFFICIENTS Once the data have been gathered some estimate ofcoding reliability must be determined. Holsti’s reliability formula – an example of a handwrittenreliability coefficients. Most researchers want to achieve a coefficient if about 90 –This assumes that independent coders have been employed In some cases you may use only one coder. A drawback of this method is coding the message which canbe quite complex and requires both extensive sessions andchecking of the coding by the researcher.Reliability = __2M__Ni +NjM = Total items agreed uponNi = Total items i selectedNj = Total items j selected
CODING VALIDITY Whether a coding system measured what youwant it to measure. Three possible sources ofinvalidity: (1) Definition; (2) Category; and (3) Sample1. Definitional Sources of Invalidity When units of analysis are defined, a set of boundaryconditions are created which serve to limit what you will acceptin the analysis. Your theoretical perspective will serve as the guiding force indefinition – You may or may not reflect the same definitions otherswould have chosen.2. Category Sources of Invalidity Definition is closely tied to category selection Category systems should be exhaustive and mutually exclusive andmust reflect the purpose of the research.3. Sampling Sources of Invalidity If your sample represents the universe of the population fromwhich it was drawn and the analysis of the sample is valid,
Use of Computers in Content Analysis Can provide speed and precision not possible byhuman beings. It does not replace anything that the researcher could do,but it does make it easier to accomplish the choresassociated with coding the data – can’t establishmeaning. Advantages and Limitations of ContentAnalysis Time consuming and complex. Does not, and cannot, asses the effects ofcommunications messages. Provides the manifest content, which may lead tofuture research on the latent meaning of thatcontent.
Fallacies of Factual SignificanceChapter 3 of Fischer, D. H. (1970). Historians Fallacies. Pp.64-100. Harper Perennial.• To write and read history is to engage in an endless process ofselection• The process of selection and the criteria of factual significancecan and must be clarified• Sampling – selection of representative facts of a certainpredetermined kind, within a closed universe of investigation• Selection of the kinds of facts to be sampled – determination offundamental criteria of factual significance• The purpose of historical inquiry is not to vindicate a method butto discover what actually happened.• Every efficient means to this end is legitimate, but none alone can beerected into a standard of legitimacy.• Few criteria of factual significance are inconsistent with empiricism inhistory• The fallacy consists not in the criteria per se but rather in an attempt tocombine them with methods and objects of empirical inquiry.
CRITERIA OF FACTUAL SIGNIFICANCE A true standard of factual significance is one which isgenerated by a sound model of historical explanation. A historical explanation is an attempt to relate somehistorical phenomenon in a functional way to other historicalphenomena. Nothing is self-explanatory – a properly executed explanation relatesthe unknown to the known in a series of orderly inferences. A fact becomes significant in proportion to its relevance to anexplanation model. A significant fact is one which helps historians to make a case for theirexplanation and to communicate its nature to the reader. Criteria of factual significance should not bemethodological, but substantive in nature Grounded on the nature of the problem itself and not in thetools of problem solving It should be empirical, capable of fulfillment, must bemade explicitly, and must not violate what philosopherscall the principle of nonvacuous contrast.
THE HOLIST FALLACY Is the mistaken idea that a historian shouldselect significant details from a sense of thewhole thing. It would prevent a historian from knowing anythinguntil it knows everything. Its evidence is always incomplete Its perspective is always limited The thing itself is a vast expanding universe A historian who swears to tell nothing but the whole truth, wouldthereby take a vow of eternal silence The researcher who promises to find the whole secret for himslefcondemns himself to perpetual failure. The whole truth cannot ever be attained –Historians are bound to tell the best and biggest
THE FALLACY OF ESSENCES Begins with the old idea that everything hassomething deep inside, a profound inner corereality called essence. Facts are significant in the degree to which they displaythe essence of the entity in question. Supplies a sense of completeness and it encourages asense of certainty. The existence of essences cannot be disproved by any rationalmethod. What it is possible to demonstrate is that a belief in essencesinvolves an empiricist in certain difficulties Essentialism is very common in historical writings– closely related to holism. Knowledge of the essence of a thing implies knowledgeof the whole thing – Impossible
THE FURTIVE FALLACY Erroneous idea that facts of special significanceare dark and dirty things and that history itself isa story of causes mostly insidious and resultsmostly invidious. Causal reduction where reality is a sordid secretthing and history happens on the back – Reality isalways the underlying fact; always something more(or sometimes less) than meets the eye Combines a naïve epistemological assumption that thingsare never what they seem to be, with a firm attachment tothe doctrine of the original sin Mental illness commonly called paranoia Self-fulfilling quality – Men who believe it begin to act furtively Errors and distortions in interpretation and assumptions
THE MORALISTIC FALLACY Selects edifying facts – It would make history ahandmaid of moral philosophy. This fallacy is exceedingly difficult to define precisely; All historians must and would make value judgments in theirwork Some historians moralize upon past events in ways whichare inconsistent with empiricism Important to distinguish between acceptable moral judgment (a priori)and unacceptable moralizing. The historian can adjust its project to its values in such afashion as to neutralize or to control its moralpreferences: Make its values as fully explicit to himself and others aspossible Design a research problem in which its values allow an open
Selects immediately and directly useful facts inthe service of a social cause. How utility canfunction in history? An attempt to combine scholarly monographs andsocial manifestoes in a single operation. Doubled trouble : distorted monographs and dullmanifestoes Scholars who take a pragmatic view of theirtask and collect facts that are weapons for acause are faced with the problem that somefacts exist which are useful to their enemies.THE PRAGMATIC FALLACY
Selects beautiful facts, or facts that can be builtinto a beautiful story, rather than facts that arefunctional to the empirical problem at hand. An attempt to organize an empirical inquiry uponaesthetic criteria of significance, or conversely in anattempt to create an object of art by an empiricalmethod. Art ≠ history – for art external reality is irrelevantwhile history is an empirical search for externaltruths. Any attempt to conduct research according to aestheticstandards of significance is either to abandonempiricism or to contradict it. A historian who omits all the ugly “buts,” “excepts,” “perhaps,”THE AESTHETIC FALLACY
Idea that the facts which count best count most.Assumes that facts are important in proportionto their susceptibility to quantification. Not quantification per se – to quantify is merely tocount. Counting has always been useful and it will ever be. Historians should quantify everything they can, by thebest statistical method. Conclusions tend to be inaccurate and a littlesuperficial.THE QUALITATIVE FALLACY
Erroneous idea that facts which count best, countleast. Begins with the assumption that regularitiesdo not exist in history, or do not exist significantly Hostility to quantification – counting does not coexistwith ideological preconceptions Every historical event is unique Commits to holism and essentialism Most apparent among conservative scholars whereattendant metaphysical bias is most clearly visible No historian really treats all facts as unique; it treatsthem as particular Every fact and event require reference to commonalities beforethey acquire meaningTHE ANTINOMIAN FALLACY
Committed by any scholar who abdicates itsarduous responsibility of rational selection andallows the task to be performed for it by time andaccident. This would reduce scholarship to mere sciolism A smattering of superficial nuggets of knowledge without pointor plan or purpose Careless, casual impressionism Exceedingly ineffectual. The reader is diverted byit, but so is the historian, from its difficult obligationto select factual statements according to explicitcriteria of significance and to tell truths which areas clear and comprehensive as mortal intelligenceTHE FORTUITOUS FALLACY