Accessing the right information: on time! Time planner / scheduler (reminder): revise accordingly in light of difficulties; Lists to obtain idea of optimal sources (for all aspects of dissertation coverage) & what can be achieved on a week-by-week basis; Critical analytical note-taking to ensure focus (reminder of week 2 key-topic). Logbook record examining progress reading & researching.
Primary and Secondary evidence Evidence is often categorised as being ‘primary’ or ‘secondary’ in nature Secondary research Primary research ‘ Desk Research’ Where primary and secondary sources blur : the distinction between a primary & secondary source is not always clear-cut. It depends on the nature of your project. A secondary source written in the 1950s on 1930s Conservatism, for example, would become a primary source for someone writing a dissertation specifically on ‘Literature on 1930s Conservatism’. However, for someone writing more generally on Conservatism in the 1930s, it would remain a secondary source.
Establishing the correct research ‘mindset’: Be prepared: ‘If something can go wrong, it will go wrong!’ – meticulous planning of research, thinking in detail about what could go wrong and what can be done to minimise of prevent such. Open-minded, enquiring Research: many students submit poor work because they rely on writing what they believe to be so – their personal or acquired ‘truth’ – rather than keeping an open, impartial mind when engaging in their research. (Equally, even with an open mind, research can be poor for lack of adequate time & rigorousness of critical analysis). You may have clear opinions about an issue before you begin your dissertation, but it is important that you enter the research process with an enquiring, open mind. Be prepared to allow your views to be modified by the evidence you obtain. If you have chosen a topic about which your views are so strong that you know that no literature will sway you, this will stunt your capacity to engage the topic openly and fairly. Maybe, in the interests of getting good grades – and making the most out of this opportunity to engage in genuine, impartial in-depth research – it would be wise to pick a topic that you feel less passionate about!
Primary research – examining or generating original evidence Not all dissertations and projects demand that you undertake primary research. Some do; others – often shorter and/or single weighted projects – do not – or leave the option open to you. So how should one decide whether primary research should be a part of one’s project? Can you obtain all the information that you need from secondary materials, or is primary investigation necessary? Is there a gap in research that necessitates primary research of your own? The key point is that it is up to you to decide which combination of research methods are best for your project. A project by its nature will lend itself to certain types of research over others. Different expectations of primary research aspects: substantial (core) – experiment-based studies, predominantly survey-based studies, studies where there is little published evidence perhaps due to newness of topic; element (aspect) – predominantly secondary-research studies (largely works of synthesis) – if this is the nature of your project, is this acceptable to you supervisor & the Project Module? If so, fine!
Research methods / methodology discussion. What, in essence, is discussion of research methods and methodology all about? A section discussing your research methods can be a very useful component to your project, given a clear indication of your thinking and decision-making as your project evolves. If you have to produce such a section, it usually follows the Literature Review section. Even if it is not required, if you had to take reasoned decisions regarding chosen evidence (primary and possibly also choices with regard to secondary evidence), then it would probably be a good idea to discuss such decisions made, if not in its own section, then in the space of a few paragraphs during your Introduction section: for, in much the same way as reviewing the literature can substantially improve a project, so can discussion of research methods / methodology. Thus, the section of your work discussing research methods and methodology should address the following: Why you have chosen those particular methods ; Why you have rejected other methods as less suitable ( do not waste space on methods that are clearly unsuited to the topic ); the anticipated limits of what that research can possibly reveal (remember this is ahead of your actual conducting of research: in other words, your advance perception of difficulties and limitations, not what you know after conducting the research, which should appear in a later main body chapter); what you hope your research might prove or disprove . How you aim to prevent foreseeable problems that could compromise the validity & reliability of your results Even a purely secondary source-based project can merit discussion of choices made, sources prioritised, etc., when it came to prioritising the literature: types of source most valued, etc.
‘ Methodology aspect’: Research ‘Methods’ and ‘Methodology’ Many students use the terms methodology and methods interchangeably, but they are not the same, and should not be confused with one another. ‘ Methods’ are the particular techniques used to collect data and information, for example, interviews or questionnaires; ‘Methodology’ is the reasoning behind using a particular research method: the philosophical basis behind the adoption of particular methods. (White, 2000, p. 20). In other words, you engage in methodological discussion in your work when you are referring to the thinking that lies behind any research method(s). Therefore, One method of research (for example, interviews) is not a methodology; it is a research method . Several methods of research are not methodologies, but are research methods . The discussion of the relative helpfulness, viability, and practicality of various research methods is discussion of methodology . In discussing the viability of various research methods, just make sure you don’t state something like ‘Interviews, therefore, are the favoured methodology of this study’! Do say, ‘Interviews, therefore, are the favoured research method of this study’. The advantages of stepping back and thinking also in terms of ‘methodology’
Choosing appropriate research methods Choosing appropriate research methods is not a decision to be taken lightly. It should be entirely based on what is most ideal for the project, albeit within constraints of what is feasible in terms of time. It has already been stressed how it is important to consider research methods, and their viability, from the earliest: it is extremely risky to arrive at your chosen research methods as an afterthought. Many departments insist upon an outline of research methods in project proposals & it is sensible to address this even if it isn’t requested: knowing what research methods you would have to employ can be a very helpful contribution to determining the viability of any proposal at the first instance. The important question to ask, therefore, is ‘ What research technique is likely to yield the most effective, attainable results for my research? ’ Some Typical research methods: Some projects may best be served by analysing available (or generated by yourself) statistical data . This may or may not require specialist software to undertake complex co-relational analysis. Others may best be served by interviews – or questionnaires – or some form of observation-based research – or studying unpublished or rare document- or artefact-based evidence , perhaps located in an archive or other repository (private papers, government papers, company papers; published reports and other documents; primary media evidence, original works of art, etc., etc.).
But … when choosing, also take into account any departmental preferences for / prejudices against different types of research and approaches. Reliability, Validity, and Wider Applicability Any research method’s suitability has to be considered via assessment of its reliability , validity and, possibly – dependent on the nature of the research – wider applicability (or what Swetnam, 1997, p. 28, terms ‘generalisability’). If you are conducting an interview, this might involve careful consideration of the nature of the questions you intend to put to your interviewees. An ambiguous question, one that might mean something different to different interviewees, would be of doubtful reliability & validity. Likewise, a questionnaire which was distributed to thirty students in the Moorgate canteen and which aimed to be representative of the entire population would be of very questionable validity: yet alone not representing the entire population, such a low polling sample may not even be representative of the views of those Moorgate students who frequent the Moorgate canteen ! Testing for reliability : ‘Reliability is the extent to which a test or procedure produces similar results under constant conditions on all occasions.’ (Bell, 1993, p. 64). All sorts of factors can affect responses, and there are a number of systems designed to test and compensate for this: test-retest, alternate forms method, split-half method. Always consult your supervisor when devising questions for surveys & interviews, and think about and seek their advice on testing for reliability. Testing for validity : Validity ‘tells us whether an item measures or describes what it is supposed to measure or subscribe … a reliable item is not necessarily also valid.’ (Bell, 1993, p. 64) Again, consult your supervisor over this. It may be that they can offer suggestions that will improve both reliability and validity of your questioning by adjusting the way the question is framed, or by advising on the introduction of control variables. For advice on designing and implementing questionnaires, see Bell, 1993, ch. 7. ‘ Generalisability’: This is about assessing how applicable are your findings to other/wider situations. This seems particularly appropriate if your research is of a case-study nature, focusing on one or few examples of a particular phenomenon. Yet it also has a more general application, because every project will at some point be required to place its findings within a wider context.
Be aware that some of the best projects skilfully weave in two or more types of research , even if one method is predominant. In other words, it may not always be a case of deciding whether to have interviews or questionnaires, if both are suitable and helpful, but which would be the better to rely more heavily upon. The advantages of using more than one research method are clear: it can aid the testing of the accuracy of evidence obtained from one particular method. For example, archival evidence may throw up some questions you are unsure about; interviews with living participants may shed light on some of these uncertainties. Both processes need each other: interview alone – oral history – can be unreliable: memories not only fade, but recollections may be warped over time. Document-based evidence can throw up questions to be asked of which you might otherwise not have thought about. This process of cross-referencing to ensure corroboration of evidence is known as ‘ Triangulation ’ and it may occur within or across different research approaches. It involves ‘cross-checking evidence from one source or type of source against another ‘in order to produce as full and balanced a study as possible.’ (Bell, 1993, p. 64) Time constraints (& other factors) will probably limit the full potential range of sources at your disposal, but try to include as much triangulation in your work as possible as this will help to prove the reliability and validity of your research.
Quantitative versus Qualitative Research methods. Perhaps the most common (& useful) dichotomisation of research types. Quantitative research is that which is capable of being measured, in terms of percentages or ratios. It involves the collection and utilisation of numerical or quantifiable data. It is research involving or relating to considerations of amount or size. As well as the pure sciences, quantitative measures can be used for a full range of subjects, including politics, sociology, economics, history, business, & art and design. Qualitative research is that involving or relating to distinctions based on quality or qualities. It is non-mathematical in its procedural basis. It is no less concerned with objectivity, but believes that the measurement of quantities gives no understanding or explanation to a problem, as that problem is socially constructed, and that therefore answers are to be located in subjective assessment. The ‘qualitative’ critique of ‘quantitative’ is that the latter gives the statistics, but cannot tell us ‘why’.
Getting the balance between the two : outside of the pure sciences, it is misleading to see research as a pure choice between qual & quant methods. Many projects become most effective by combining aspects of both. Interviews mainly fall into qualitative research. That is, unless the interviewee has to respond according to a measurable scale, for example the Likert scale (see later). Then the interviewee’s responses can be numerically coded and statistically compared against the responses of other interviewees, provided that each interviewee faced exactly the same questions. Observational research is also qualitative in nature, as this involves the researcher immersing themselves in the environment they are researching, and assessing what they perceive. Questionnaires may be qualitative and/or quantitative, depending on the type of questioning, open-ended (inviting the respondent’s considered comment are clearly qualitative); closed questions (again involving tick-boxes, like the Likert scale) are quantitative. Well-designed questionnaires often use a careful combination of qualitative and quantitative questioning. Too many qualitative questions might make it too much effort for a respondent to bother to take part; an appropriate mix of qualitative and quantitative will allow for both statistical measurement (quantitative) and yet also allow for detailed comment (qualitative), e.g., reasons ‘why’ that lie behind statistically measurable responses. Therefore, choose qualitative, quantitative, or a combination of both methods: whichever best suits your study.
When using quantitative measures, be aware of the following distinctions: 1. Nominal (or Categorical) scales. For classification purposes. The data in each answer given to the question cannot be measured against the other answers. For example, ‘To what religion do you belong?’ Muslim/Christian/Hindu/ etc. Or Gender: M/F. Or: ‘the lowest level of measurement that involves assigning characteristics into categories which are mutually exclusive, but which lack any intrinsic order’ (Bath University ‘Project Gold’ site <http://www.bath.ac.uk/e-learning/gold/glossary.html>) 2. Ordinal scales. Here, the data has some order. There is ranking as well as classification. e.g.: a question asking the respondent to order items in terms of preference. ‘Project Gold’: these categories can be used to rank order a variable, but the intervals between categories are not equal or fixed (e.g. strongly agree, agree, neither agree nor disagree, disagree, strongly disagree; social class I professional, II semi-professional, IIIa non-manual, IIIb manual, IV semi-skilled, and V unskilled). 3. Interval scales. Interval scales have the same properties of ordinal scales, but the intervals between the points on the scale are equal. ‘Project Gold’: the categories are ordered and there are equal intervals between points on the scale, but the zero point on the scale is arbitrary so that a particular measure cannot be said to be 'twice as' large as another measure on the same scale (e.g. degrees Centigrade). 4. Ratio scales. Ratio scales have equal intervals and an actual zero point (which interval scales do not have). ‘Used for measuring characteristics such as length, time and weight they have higher mathematical and statistical potential than the others, but limited relevance to social scientists whose areas of interest involve human behaviour.’ (Swetnam, 1997, p. 67; White, 2000, p. 48-9). Project Gold: ‘ (e.g. measurement in yards, feet and inches or in metres and centimetres).’ Discreet quantitative data: involving whole numbers. Continuous quantitative data: involving any numbers.
Likert-scale questions : not everyone’s ‘very satisfactory’ will be the same, but the results are still measurable and quantifiable, so it counts as quantitative research. The Likert scale is a good example of an ordinal scale. T he categories do not necessarily have the same value to each respondent (e.g., one person’s ‘very satisfactory’ may be another’s ‘satisfactory’), but it still allows for a meaningful statistical comparison of responses.
Correlation : invariably complex and requiring the use of specialist analytical software, for example the social sciences analysis programme SPSS, where users subject data, seeking to establish correlation or otherwise between 2 or more variables, often involving ‘control’, and using the most appropriate formulae of measuring relationships, for example, Chi Square, Anova, etc.
An LDU short course presentationDissertations & Major Project Writing Week 3 of 5: Research 2: secondary & primary evidence making the most of London Met libraries / database & on-line searchingRobert Walsha, LDU City campus, Calcutta House, CM2-22
Dissertations & Major Project Writing week 3This week’s topics:• Accessing the right information: on time!• Secondary & primary evidence (choosing appropriate research methods, validity, reliability, etc.);• Making the most of London Met libraries• Effective use of databases / on-line searching
Accessing the right information: on time!• Time planner / scheduler (reminder);• Lists to determine optimal sources to absorb & what is achievable on a week-by week basis;• Critical analytical note-taking to ensure focus (reminder);• Research logbook to monitor progress.
Secondary & primary evidence ch ar ch f o seearmeeor r ishedd ee eessnaachh publ lishe rr a rc f thhother ren r m ll ub • Defining ‘primary’ & ‘secondary’ sas aall p rcah:atnheehrofevidence; h: aarcs s anh trtee of as o e e ••Prima e Prima y rese ee da gg g r tr o s of t rr resea ryys ese dd dinotutgthhyy foflels,of thhe inh you gen aac : rhis srea thon aa rnale rm u dd cih :tb u b aast y reseair reesysreest t meentxamppal s, u you gen ratcmv t t m apbiy mey me .redEoources. ’. ne h s oon e keeth s momb m E j xumn y’ e: s e, m a a ’ ec t d prt k er e lr m s r a se ja p a es SSeoe,eiraeiahthaviaacicoabyanessarurhrrtth y er:aso vita rn er psl b ay r eyr ndeo c a crta obseav••ou -,wortke pentnnerewr,mks isttihilriddch that observathh’ c . R eesser acicnu-wnd t, ragge rvi rwsooia e ci‘aoh sou erar io h uu uu r rry or pp s,qu t t clann r p n iee ook us tr onhbb rknda hriy io bbtiat seest‘ito r aires , o o r c vetir q t • R t t se yr s gcs ,oar a macc / se t rce eskk•Yereiseaaoc inegoondchdveaes stahhhr nnaires, y s• •DDes arYy tremaiy ddssec caed emata,s,earc , etc.. de atti r , ‘‘ y e it mfhnn rin b h addi d less oo h re/ h ( (s a nddar aphet aeralrublbsishedte icle p – w s fy e o a o ( ie e i e l sss aac var ic , ees. arcch aas a secooncerrhapspudteihhrubpeeviaren, e rilmaryy tc.sear c r ph h – whethhsee (le eri d tn’ cpa read r re e bd p d c se ealerch e uub ispea pp rm c. yy rim ad a l ade lplooltle hpp,ae.ig.,tar)r). ’,epalreary e tmarg e . res seaetitgrrt,,nic gomelwdd ,ne,rgmeditions ufhaavatmad e t s i n c u ewss d r i a . e r lmteihseoncccnalee y‘ ‘p. , .,tunttiloyoou diar e larg :: r e p meght ior oc l an er nt ee c di i nso h v ie rch.e r nap l t e y is alttetsprieeenbednn tmomntytaeeorrs, ntildywof deersesorrh. Why ppstebt ei i g copas artped u or ra ry’ r ia i earc ret m er t d t e •l •D t es isenldd o he e m r po ts,on a rdatr a e bm e s s a o nin c e s e h D or shouted notthbappitnd‘sec ornrda y’ a/ evid le* * fr ho a t i wiay pp i ndi‘x e f o aw dat ruule)r)frsomul seatdstway wethixsofca secondaa/ evidence o om a aw c l resou o a secon ry so ce en r ub ssub t t nt i l hsead ssaanitaal htatistical resourc datab dary sour e rce urc y is thi s? e databa e.as se. ce …hWh t is?
Secondary & primary evidence ch ar ch f o seearmeeor r ishedd • Where primary and secondarytherteessreaachlh publblishe h: r a rc f theo her nn r m l … sources blur u e e aarcs s anh trtee of a a t e rcah:anheehrofsas s all phe o‘mindset’: ••Prima rr esea ryys e eresearch tghtr o lels , o • Establishing rthe correctse dd aingtutgh ryy fof s,of f th res see dd go a u a n u Prima y ra h: rhiusrea i m on a rnale y reseair reesysreest thheentxamppal s yoprepared! nn a mh – Beougenera omdd cim :tbhiyb astbe . E jouurn s. y u gen estcomt tc eapb smey mmredExo mcey’ ’. coo vieakh et s slmooatry n sea r rt s e, era iak pa b c m bey. d j au h ey r••SS :at ndert enerrere lriwilks icasisdrouhgh’ a observathhre se,tvita rnrthavieacs yks raaelwraongt!’tat cce’ nuepe i tnev p aiml gon eo rcaat! ’: avaion-,worut pgeen rri rwo, o a esohwsd n crh t obseear t u uu d k c ego itpmqiu tgr‘tco lriroppr ro t s i ti an r o ee e orh - , o o ,oa canv i, irbbwqli r RRss r gycochbgsr owrrrhriy e,citssaatee r h nai on g o o kmYehinearicanwboknwacyngcawstetthustst‘itonnares, p.10) ung , rarr o m e , il e dd h t / s ‘Iffeso• eteeiin efay dinng c cr ad dmatrtanarhN, e c..n r1 s,2(, p a• •‘IDssk•rYythstsgaicadssgs on ed eveea,s,e&rc ewtton,,i1e992as.10) ‘D so met trem yn r sec ade i a s / ootch ‘ e daar r t ethfen ri o (a h(Rudclses a Newto r99 , s a y i i a ssblbac vRr idie e nnd rhaph mahe l lruebi(sishedtee lstan & h, ees. a cch (a w s puyhee o e er(riia tn a r ecoo pe rha whet beap rubleeevdrd cce rilmary r tc.sear s a ue e h s sec rch –– ps ot lthh p e d n, p ea ry re pe rr ertte ht onc dbils hpa , p ige,taadyi’ rir aav attl aade uud ispe pr m cry . sealaectgrs, iniclpluoeeledp,ae.,rgme)r)’tc,epaluehdy eamarge: e . eese l ti ers t r comwssd‘ ‘p. i..,t. . i ons m ady e m dg h d e ec e –r pim ambitious eweneee ed entiiabout diasiea lar he: Beegnee nccn l…mtyntrealistic ilno ou he r youh. . m h, in napland r tt f av s esp etsiebbeoirggocaplasyarnp,ortuunttloyyoofwhatie ror canWhy is di ttete rrete t d i mm butreteds, or sry’ diar es cc p •lalat espisenldd n the mmtar ports,onda rdatres ar r be nio cob scale D achieve (timec+ a s e hconstraints, y’ a ra a r •D or shouted nottho e ppitnd‘iseccornrdw etc)!/ evid o n e app aw data/ lee*r)frsoma t i wiay w i ndi‘x e o secon l om ruu–*)Think ul seatdstway weth xsoffa between primaryc& o fr h o a c best ou o a secon ary evidence en e r l hseathe t t nt i about d l esbalance ssaanitaal htatisticalrresourc datab darysourr e rce d ub ssubsecondary?material … s as e databa e. sou c e c …hWh y is thi se. t – s? and the best make-up of primary evidence + what i… make-up of secondary materials to best benefit your project’s investigation.
Secondary & primary evidence• Setting appropriate balance (primary vs. secondary): – Take into account: • the project module requirements / the guidance & expectations of module leader and/or your supervisor; and: • the specific requirements of your chosen topic. – Maybe primary research is unnecessary? – … or maybe, even if not deemed essential, your project (& thus grades) would still benefit; – If primary research is required, then what level is considered appropriate: • ‘core’ component vs. ‘sufficient levels’
Secondary & primary evidence• What, in essence, is discussion of research methods / methodology all about?; a comment upon: – Why you have chosen particular primary methods; – Why you have rejected other methods; – How you have designed your research (experiments, questionnaires, etc.) – and why that way; – Anticipated limits of what that research can possibly reveal; – How you aim to prevent foreseeable problems that could compromise the validity & reliability of your results; – What you hope your research might prove or disprove; – Choices made re: secondary evidence;
Secondary & primary evidence – + methodology discussion: if discussing methodology, start by discussing pluses / minuses of each research approach adopted, in general, then move to discussing how it applies specifically to your chosen subject;• Thinking about research ‘methodology’ helps to think about the usefulness of any particular research approach: ‘What research technique is likely to yield the most effective results for my research?’
Secondary & primary evidence• Choosing appropriate research methods: – Start from the start (e.g., give it most serious attention in your proposal); – Ask key questions: ‘What research technique is likely to yield the most effective, attainable results for my research?’; – Think about all possible (e.g., possibly viable) types of research from which the project could benefit – within constraints of time.
Secondary & primary evidence – … but take into account any departmental preferences for (or prejudices against) different types of research and approaches.• When choosing your research methods & designing your research, it is important to think in terms of (anticipated): – Reliability; – Validity; – & possibly: ‘Generalisability’ (wider applicability);
Secondary & primary evidence + The concept of ‘Triangulation’ & how Triangulation triangulating can benefit your research.
Secondary & primary evidence • Thinking Quantitative versus Qualitative research methods …Qualitative research --that involving or relating to Qualitative research that involving or relating toQuantitative research - or qualities. is capable ofdistinctions based on quality -that which is capable of Quantitative research or qualities.Non- distinctions based on quality that which Non-mathematical in itsin terms of percentagesno lessbeing measured, inprocedural basis, it is or ratios. It being measured, terms of percentages or ratios. It mathematical in its procedural basis, it is no lessconcerned with objectivity, but believes that the orinvolves the collection and utilisation of numerical concerned with objectivity, but believes that the or involves the collection and utilisation of numericalquantifiable data.measurement of quantities gives no understanding or quantifiable data. measurement of quantities gives no understanding orexplanation to a problem (i.e., cannot tell us ‘why’). explanation to a problem (cannot tell us ‘why’). • … and getting the best balance between the two.
Secondary & primary evidence • Thinking Quantitative versus Qualitative research methods …Qualitative research --that involving or relating to Qualitative research that involving or relating toQuantitative research - or qualities. is capable ofdistinctions based on quality -that which is capable of Quantitative research or qualities.Non- distinctions based on quality that which Non-mathematical in itsin terms of percentagesno lessbeing measured, inprocedural basis, it is or ratios. It being measured, terms of percentages or ratios. It mathematical in its procedural basis, it is no lessconcerned with objectivity, but believes that the orinvolves the collection and utilisation of numerical involvesprojects become utilisation of numerical Many projects becomemost effective by concerned with objectivity, but believes that the or the collection andmeasurement of quantities gives noeffective by or Manyquantifiable data. most understanding quantifiable data.quantities gives measurementaspects of both. no understanding or of combining a problemof both.explanation to aspects (i.e., cannot tell us ‘why’). combining explanation to a problem (cannot tell us ‘why’). • … and getting the best balance between the two.
Secondary & primary evidence• Differing forms of measurement for quantitative research: – ‘Nominal’ (or ‘Categorical’) scales; – ‘Ordinal’ scales; – ‘Interval’ scales; – ‘Ratio’ scales.
Secondary & primary evidence – Likert-scale questions; Assessing people’s opinion on a 5-point* scale.Q. How would you rate the Government’s handling of thelivestock ‘blue tongue’ disease outbreak? (please tick 1box) 1: very satisfactory 2: satisfactory 3: neither satisfactory nor poor 4: poor 5: very poor * (occasionally a 4-point scale, to purposefully prevent people ‘sitting on the fence’ in the middle category)
Secondary & primary evidence• Correlation; Literally: ‘co-relation’, this concerns analysing the relationship between 2 measured variables.• ‘A variable is well correlated with another if their values alter together, either in a positive or in a negative fashion. … A statistic called the correlation coefficient can be used to express the strength or degree of linear correlation between two variables.’ (McMillan & Weyers, How To Write Dissertations & Project Reports, 2007, p.116)