The Research Process ii)
         From Research Requirement to Methodology & Surveys
Thus far we have explored Title, Intr...
2.
                                                               3.
                                                     ...
2.2 Methodology.

Firstly, this is a big area and you must read about it. (I am not going to write the
equivalent here of ...
Effectively you will need to look at each and every ‘gap’ this way to be able to work out
the answers to the following que...
You must always make your questionnaires (and interviews) work as hard as they can
to enable you to gather not just the fi...
Both qualitative and quantitative research therefore have their challenges.

In your Methodology, you must not merely expl...
2.3 Questionnaire / Interview Design Issues

First of all it is worth saying that this is just the next logical stage in t...
NB. There is no point in rushing the survey design: take time to get it right – otherwise
you will go to a lot of trouble ...
Upcoming SlideShare
Loading in …5
×

R2 Methy To Surveys

446
-1

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
446
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
25
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

R2 Methy To Surveys

  1. 1. The Research Process ii) From Research Requirement to Methodology & Surveys Thus far we have explored Title, Introduction and Literature Review.... now we move on to the critical definition and specification of your primary ‘Research Requirement’, through Methodology and into Surveying, as follows: • Defining the 'Research Gap' / primary Research Requirement. • Methodological options and considerations to address ‘the Gap’ • Selecting of an appropriate methodology and establishing that the chosen option is the ‘best’ option: o (F)easibility: can it technically be done? o Effectiveness: is it fit for purpose? / Will it achieve the desired objectives? o Efficiency: is it the most rational, direct, simple way of delivering valid and reliable results? o Economy: can it be done within the ‘budgetary’ constraints operating (economic, time, distance) • The Quantitative and / or the Qualitative: pros and cons. • Questionnaires: compiling, evaluating, piloting, reflecting, learning, redesigning, undertaking. 2.1The Research Gap / Research Requirement A suggestion (actually a recommendation... a strong recommendation at that!) : again I know this is a little ‘mechanical’ but it will help you. Review each of your A&O and sub-objectives (if any) and identify exactly what you now do know (and particularly what you do not know) about it after the LR. Do this methodically and you really cannot go wrong. See hereunder:- Objective What I do know What I do NOT know KeyPoints % % Gap/Requirement 1. • 80% 20% 1. • 2. • • 2. • 50% 50% 1. • 2. • 3. 3 • 30% 70% 1. • 2. 3. 4. 4 • 10% 90% 1.
  2. 2. 2. 3. 4. 5. 6. The numbers are just illustrative, but inevitably, as one goes from things that are generally well known and written about towards things that are relatively unexplored, one finds less in the existing literature and has more to establish in the primary research programme. Essentially, then, if you take this approach, the right hand column becomes your Research Gap or Research Requirement and it is demonstrably clear (both to you and to your examiner/reader) how you have identified it and that you haven’t lost or missed anything en route. Just to keep you abreast of things from the examiner’s point of view, the principal critical failures we find up to this point are: • Unclear title (we don’t really know what you are doing) • Jumbled, incomplete and unclear A&O (we don’t know what you are doing or quite how you are going to do it....and it doesn’t look as if you are very clear about it either) • Literature that is cited but not critiqued and subsequently not synthesised (so we don’t know what you have learned from it) If you are not meticulous about doing a simple sum: What I need to know – minus - what I have found out from the LR  Research Gap Then you will be adding a fourth ‘failure’ to the list: • Unclear Primary Research Requirement (we are unconvinced that you really know what you should be researching to fill the gap in your knowledge). You must surely see how this ‘compounds’ – one failure ‘up the line’ cascades down through each subsequent stage. At this point, if there has been significant failure at each of the above stages, the researcher may be about to embark on some time-consuming and costly research which will not come close to getting to the heart of the title and aims and objectives he has set: already the die is cast [Alea iacta est: Ceaser, J (55/56 BC)] and much subsequent effort will be of little value. If you have now answered the WHAT and WHY in terms of your Research Requirement, logically, then, you now need to turn to the HOW (and subsequently the where, when and who): How are you going to generate the data to fill the ‘gap’ and meet the research Requirement? Put another way: what methodology will you use...............?
  3. 3. 2.2 Methodology. Firstly, this is a big area and you must read about it. (I am not going to write the equivalent here of a good Research Methods textbook: there are plenty around, some often directly related to your own subject areas and in many different languages). There are all sorts of ways of generating data: that in itself is all too easy – the trick is to get the ‘right’ data: data which is valid (technically correct / accurate) and reliable (assured, capable of being replicated with the same results, dependable) and which directly and comprehensively addresses and answers the unknowns you have identified. To get started I suggest that you consider again your ‘Research Requirement’ (b/f from previous section) and ask yourself initially some key questions in respect of each missing element of information on your ‘hitlist’: • What sort of information is it (qualitative and /or quantitative and existing in what form) • Who might have this information? • Where might they be? • What might be the best way of accessing and acquiring such information given the above and any other operative constraints (like time, money, distance etc). You might perhaps usefully contain this in a tabular approach, as hereunder: Gap Component What type? Who has it? Where is Best way to access? it? Objective 1 Gap 1. ..............     Gap 2................ Objective 2 Gap 1.     Gap 2. Gap 3. etc....... An example (from the ‘James Dean’ scenario earlier), Gap 1 might possibly be something like: Gap: How does one classify degrees of ‘rebellion’? What Type: Qualitative information based largely upon perception, socio-cultural and socio-psychological opinion of those ‘best qualified’ to judge this. Who: Authors / biographers on the studies of ‘rebels’. Clinical psychologists. Where: In (and possibly to be deduced from) biographies. In the heads of authors/ biographers. Possibly in clinical studies / journals concerning adolescence / prison recidivism even. In the heads of clinical psychologists. Ways: Reading biographies. Reading clinical studies. Talking (semi-structured interviews perhaps) with key authors. Depending upon the constraints, this ‘talking’ might be F2F, on the telephone, by email/letter dialogue.
  4. 4. Effectively you will need to look at each and every ‘gap’ this way to be able to work out the answers to the following questions: • Am I looking at just one survey with one group (usually called a population or a representative sample of a population in research terms) or might I have to consider perhaps a two-pronged approach: maybe qualitative F2F interviews with key people AND a quantitative questionnaire to a different population perhaps done on the street or by Email? • Am I doing something principally qualitative rather than quantitative? Because often the answer to that question tends to drive the form, nature and content of the survey and how it might best be administered. • How can constraints of money, time and distance be factored in – do they render certain activity impossible, difficult, and undesirable? What is the best that can be achieved under the circumstances? Having analysed your various ‘gaps’ this way, then considered them together, you should then have a reasonable idea of what sort(s) of research / surveys you might well be doing. 2.2.1 Quantitative Research Introduced. If your ‘Research Gaps’ seem to be suggesting that you are looking overall at quantitative analysis (ie you want to know the broad answer to the question: ‘how many?, the answer to which can be expressed in terms of numerical values or degrees on a scale) then this tends to suggest generally that relatively simple questionnaires with predominantly ‘closed’ questions (where you the architect of the questionnaire delimit the responses in a pre-prescribed form: a box to tick / a number to ring / Yes or No etc). SPSS. These responses can then be processed manually or via software programmes like SPSS (Statistical Programme for Social Sciences). Unless you are dealing with particularly high numbers of simple questionnaires, I would avoid such programmes as you risk becoming largely ‘uninvolved’ in/ detached from the data and you don’t tend to get a ‘feel’ for it and see the picture building up before your eyes (which you would of course see were you doing this manually using a simpler tool like Excel, for example). Most quantitative research is not, however, quite that simple: you may establish that 73% said ‘No’... but of course you are going to want to know WHY people say what they do: what is the underlying reason. You can of course ‘script’ this into a closed question: ‘Was this because of cause A, B, C or D?’, but there you risk forcing people into your preconceived ideas of their reasoning: perhaps they wanted to answer something completely different that doesn’t fit into your A,B,C,D options. In such cases you are moving away from pure numbers towards more qualitative responses (inevitably – because you need to understand what the figures actually mean!), and you may need to give people open-ended questions which allow them the space to say what they will, the way they want to say it. The focus should always be upon understanding the real meaning of figures and what underpins them. Let me illustrate: if I were to respond to your questionnaire that I have flown easyJet over 40 times and have never flown with Ryanair (which happens to be true, actually!) and you ask me nothing more, you might be tempted to think that I am a truly loyal easyJet customer who doesn’t like Ryanair. You would be wrong! I fly one route very frequently which is only served by easyJet and not Ryanair: I am ‘loyal’ perhaps to the lowest cost price on the route I want to fly..... and if it happens to be easyJet or Ryanair then so be it!
  5. 5. You must always make your questionnaires (and interviews) work as hard as they can to enable you to gather not just the figures but an understanding of them: in the absence of the latter you have only figures, and they allow you to say very little in fact. Some advice about how to achieve this follows later when I talk about questionnaire structuring, design, piloting etc. Qualitative questionnaires can be Face to Face (F2F), may be sent by mail / email and self-completed or possibly set up to work online upon one of the www’s myriad online research questionnaire management sites. There are, however, different implications that ‘run’ from these choices. Discussed later. 2.2.2 Qualitative Research Introduced. This is usually research of a non-numerical nature where you are seeking to understand what people think and do and why they do it. It may involve participant observation, focus groups, structured (or more likely semi-structured) interviews and the like. It tends to be rather more difficult in that it is dealing with ‘soft’ rather than hard data: perceptions, opinions, observations, reasonings, explanations, justifications, judgements. For that reason quite often ‘flat’, simple, closed-questioned, self-completion questionnaires are unlikely to be that helpful as they are not going to be able to ‘get at’ the real information you are after. Hence, this research tends to be done on the basis of observation and interaction with subjects. For example, if I wanted to find out about how the context of Elizabethan London or rural Stratford influenced Shakespeare in his writings, perhaps I could only find out so much from the analysis of his plays and from what has currently been written in learned academic journals – perhaps, then, I might find it helpful to do Delphic Oracle – type research and to speak to those in key positions, who are authorities on the subject: perhaps leading academic writers on Shakespeare, maybe Elizabethan period historians, perhaps even Royal Shakespeare company actors. Once your ‘population’ and an appropriate ‘sample’ has been identified you are going to want to speak to them: not least because, although you perhaps know some of the key questions to ask, you will perhaps be able to develop responses to comments and other lines of enquiry as you go. You will therefore need an over-arching structure to make sure you ask people the same questions, but at the same time something ‘fluid’ enough to enable respondents / interlocutors to take you down interesting avenues you never thought of. This is the ‘Semi-Structured Interview’: a general framework derived from the research gaps with key questions and sub-questions/prompts as appropriate. Clearly the difficulty here, then, is to be able to take away perhaps hours of dictated recordings and transcriptions and compare, contrast and analyse them, when they are so ‘free-form’. The same problem goes for things like observational studies of participant activity: it is the observer that can potentially (mis)interpret or (mis)construe action as his observations are driven by his perspective, expectations etc.
  6. 6. Both qualitative and quantitative research therefore have their challenges. In your Methodology, you must not merely explain what you did, but why, given your particular research gap(s) and your constraints, you chose to do what you did. That means you need to take us through the principal options, analyse their respective pros and cons, and demonstrate which best fit your purpose and your constraints and why. If there are other, better options left unexplored, the reader may be tempted to think why you had not thought of them or why you had discounted them. Talking, once again, of typical failures in research, here is a fifth one to add to this list: • Failing to explain and justify why you selected a particular methodology and how it is designed to deliver the valid and reliable results that you are looking for. 2.2.3 Populations and Samples ‘Population’ is simply the entire number of people who fall within the category you are wishing to survey because they hold the answers you need. ‘Sample’ is a representative subset / cross-section of the population. It needs to be big enough to enable you to produce something valid and reliable though. That, in turn depends upon your cross-tabulation. In a quantitative baseline survey, for example, do you just need to know how many people said ‘yes’...in which case you perhaps will not need a huge sample size.... or do you need to know how many people that said ‘yes’ were: married, with children, were under 35 years of age and who have never used a low cost airline flight! To find even one or two people in such a category would perhaps need a significantly larger sample size. Guidance on such things can be gleaned from research methods text books. [As a very rough rule of thumb, however, I have often thought that for a simple yes/no without cross- tabulation for a large population (genuinely randomly- sampled) 40 respondents may be enough...and then for each level of cross-tabulation beyond this, this figure needs to be doubled. So in my example above 80 to the ‘married level’; 2x80 = 160 to the 2+ children level, 160 x 2 = 320 to the married, with children, under 35 level.] Again, on sample sizes relative to populations: read a good Research Methods text. Of course, if you are talking about a total ‘population’ of say just 29 key politicians or academics... you may choose not to ‘sample’ at all, but to survey the entire population: it all depends! 2.2.4 Validity and Reliability. Again here textbooks will help you. Where you are dealing with quantitative data particularly there are a set of reasonably simple tests that you will perhaps need to apply to your data sets to establish whether you can say something definitive or a with some measure of ‘confidence’ about your data. If, for example, your data and evaluation were to be used as justification for the development of a new product (ie you have produced ‘evidence’ that there IS a market for it), then you have to be very sure that your data is ‘sound’ and may be relied upon! At this stage, then, a 6th potential failure raises its head: • Failure to sample correctly leading to non-valid and unreliable results.
  7. 7. 2.3 Questionnaire / Interview Design Issues First of all it is worth saying that this is just the next logical stage in the process: 1. Your title should be reflected in the A&O 2. The Literature Review should be driven by the A&O 3. Your Research Gap is produced by analysis of the outcome of the review of the literature 4. The Methodology is driven by the Research Gap 5. Your Questionnaires/Interviews will be shaped around the Research Gap identified and perhaps concepts and models arising out of the Literature Review. 6. In turn, subsequently, the structure of your data presentation/analysis chapter/section is likely to be determined by the ‘themes’ emerging from the questionnaires and interviews. Clearly the first point to mention is, as above, that you must conceive and design your questionnaire to achieve the objectives required: specifically the acquisition of data identified as being required by your ‘Research Requirement’/gap. Look to find a logical structure. If you are dealing with a process, perhaps even a chronological structure would help. Normally it helps to go: • from the general to the specific, • from the simple to the complex, • from the outside to the inside, if you like. This we find relatively natural in the way we work and relate to one another: one often needs to deal with the basics before we can move into any deeper level of understanding and we often look for structure and shape in dialogue to ‘see where all this is leading’. Draft your questionnaire / semi-structured interview to the best of your ability and then road-test the design upon yourself and ‘imagineer’ the likely answers: • Pretend you are an ‘awkward cuss’ who wants to say something really important but your questionnaire isn’t letting him • Literally give yourself a persona – your boss, yourself as a consumer (a ‘typical’ respondent within the sample) – then try to put it to the test by completing the questionnaire in this role. Could you complete it? Did you want to be asked other questions (like why and how, as opposed to merely what)? • Take your best guess at the average end-survey data (70% said no / 20% yes / 10% didn’t know…..17 violently disagreed / 29 agreed completely/ 13 sat on the fence) THEN…consider what this does or does not allow you to say in terms of your hypotheses / A&O and your research gap: o Would it cover all the ‘holes’? o Would it permit you to be conclusive? o Are any answers capable of multiple interpretations or in need of further explanation • THEN: Redesign your questionnaire in the light of the above, check it, THEN pilot it for real on someone (or two!) within your likely sample frame. • Redesign if necessary then launch.
  8. 8. NB. There is no point in rushing the survey design: take time to get it right – otherwise you will go to a lot of trouble collecting data that doesn’t really help you to fully address any of the outstanding issues within your Research Gap. For example, asking just Yes or No, can get you a simple % split result... but would a Likert scale with strength of feeling help you more???? ... Would it help to know WHY people said what they did? You need to make your questions really WORK hard for you. NOW: go and get your data: taking care NOT to introduce any personal BIAS into the proceedings.

×