Introduction to organisational research and case studies

1,960 views

Published on

Presentation on organisational research and case studies delivered to research students at the ESRC Scottish Doctoral Training Centre Information Science Pathway Training day, Glasgow, on 25th June 2014. ** An updated version of this presentation from 2016 is available at http://www.slideshare.net/HazelHall/introduction-to-organisational-research-and-case-studies-60657001 **

Published in: Education, Technology
1 Comment
2 Likes
Statistics
Notes
No Downloads
Views
Total views
1,960
On SlideShare
0
From Embeds
0
Number of Embeds
37
Actions
Shares
0
Downloads
26
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

Introduction to organisational research and case studies

  1. 1. ESRC Scottish Doctoral Training Centre Information Science Pathway Training day 25th June 2014
  2. 2. Introduction to organisational research and case studies Professor Hazel Hall Institute for Informatics and Digital Innovation/School of Computing
  3. 3. This session  Theme is organisational research  Context is Information Science  Mix of lecture material and short exercises  Session begins with consideration of the distinctiveness of organisational research, then moves on to case studies  But first…
  4. 4. SOME INTRODUCTIONS
  5. 5. Organisational research and case studies at doctoral level within the Centre for Social Informatics Organisational case study Organisational case study Case study Organisational research Organisational research
  6. 6. Research cited in this session Title Organisational research Business research Case study as output Setting Intranet implementation in a corporate environment X X X Professional services firm Blogs in the classroom X X Edinburgh Napier E-information roles X IM/KM Outsourcing research & information services X Business information services Research in Librarianship Impact Evaluation Study (RiLIES) X Information science research
  7. 7. ORGANISATIONAL RESEARCH
  8. 8. Organisational research  What makes organisational research “different/distinctive”?  Practical difficulties in accessing sites for data collection  Information sharing practice of drug dealers  Strategies for dealing with information security breaches in Company X  Legal and ethical issues when setting up studies  Power of the context in sites of data collection  Real-life organisations staffed by humans whose behaviours are influenced by range of factors – culture, politics, power struggles  Intangibility of phenomena under investigation  Knowledge, value, social capital, goodwill  Expectation of the organisation to derive value from the study
  9. 9. Information Science and organisational research  Borrows from other disciplines  because Information Science is concerned with range of organisational perspectives  technology, culture, functions…  Requirement to read widely  Sociology, anthropology, management science… even physics?  Intranet implementation: Galison, P. (Ed), (1997). Image and logic: a material culture of microphysics. Chicago: University of Chicago Press.  As an applied science, organisational partners may expect return on participation
  10. 10. CASE STUDIES
  11. 11. Understanding of the term “case study”  Case study is an approach to research  Empirical enquiry that investigates a contemporary phenomenon within a bounded, real-life context  especially when boundaries between phenomenon and context are not clearly evident • Intranet implementation: “The reasons why they don’t use the intranet to knowledge share [phenomenon] may be due to cultural issues [context]”  uses multiple methods including, but not exclusively, qualitative techniques, e.g. participant observation, interviews, document analysis • Intranet implementation: interviews; document analysis • RiLIES: interviews; citation “sketching”
  12. 12. Alternative understandings  The case study is the output of research  “Story/ies” of the case(s) investigated  The knowledge trap: an intranet implementation in a corporate environment (http://hazelhall.org/publications/phd-the-knowledge-trap-an-intranet- implementation-in-a-corporate-environment/)  Hall, H. & Davison, B. (2007). Social software as support in hybrid learning environments: the value of the blog as a tool for reflective learning and peer support. Library and Information Science Research, 29(2), 163-187. (DOI 10.1016/j.lisr.2007.04.007.)  Enhancing the impact of LIS Research projects  (Text book exercise)
  13. 13. Case study approach for real-life, contemporary research  Describe - explore – explain – illustrate – provide examples  Intranet implementation  “Here’s a real information-intensive distributed organisation that hoped an intranet would support knowledge sharing in the firm. I established that it did not to the extent anticipated, and propose reasons why with illustrations and examples.”  Blogs in the classroom  “We wondered if claims that blogs can encourage student reflection were exaggerated. We tested this by analysing the content of recent student blog postings in an educational setting, and demonstrated with examples that reflection is often limited.”  RiLIES  “These five case studies of real LIS research projects show how a range of factors can increase the impact of the research output on the practice of librarians.”
  14. 14. Case study approach for investigating “how” and “why” questions  RiLIES  How can LIS research projects be conceived, designed and implemented to increases the chances that their findings will influence the practice of librarians?  Intranet implementation  Why don’t staff in this corporate environment use the intranet for knowledge sharing?
  15. 15. Case study approach for triangulation  Collect data on specific cases to triangulate with other data collected  Case study (or studies) are just part of the project, e.g. RiLIES  Practitioner poll  Focus groups  Validation survey  and case studies
  16. 16. Single/multiple case studies as output  A study can include single or multiple cases  Intranet implementation: 1 (big) case study  Blogs in the classroom: 1 case study  RiLIES: 5 case studies  In case of multiple case studies, each should stand on its own
  17. 17. Rationale for single case study  A critical case – likely to have strategic importance for the general population  Intranet implementation: focus on culture  ‘If it is valid for this case, it is valid for all (or many) cases’.  See http://heim.ifi.uio.no/~in166/h00/criticalcase.pdf  An extreme or unique case  RiLIES: 5 case studies chosen were amongst the most frequently cited in the practitioner poll as having influenced practice  A new/revelatory case  Blogs in the classroom: no empirical studies conducted previously (although plenty of claims made!)  A prelude to further study, to test ideas  For example, a pilot case
  18. 18. Case study research design process  Five elements 1. Identify research questions to be explored 2. Determine propositions or hypotheses  Bearing in mind that case studies themselves often generate hypotheses and models to be tested in the future – by you, by other researchers 3. Select clear units of analysis 4. Analyse data in a logical fashion so that it can be tied back to propositions 5. Interpret findings
  19. 19. Case study research design process  Five elements: Intranet implementation 1. What is the role of an intranet in knowledge sharing? 2. External and internal organisational factors determine role  This proposition was based on an analysis of sociotechnical literature that dated back to the 1970s 3. Interviews and document analysis 4. Data analysed and reframed using actor-network theory (more on this later…) 5. Findings interpreted to uncover underlying explanations of practice
  20. 20. Case study research and “rigour”  Accusations of bias and lack of rigour in case study research because data from which findings derive belong to a specific context  Poor reliability  Can you be certain that you would report the same findings if you ran the same study at another time or location? • Intranet implementation: Perhaps not, but research protocol is such that the process could be repeated in another large information-intensive professional services firm (i.e. method is reliable)  Doubtful validity  How can this/these case(s) be generalisable to the wider context? To what extent is your case study “representative” of the population as a whole? • Intranet implementation: It can’t, but it does not seek to “generalise” • RiLIES: Multiple case studies can address this to an extent
  21. 21. Other “weaknesses” of case study research  Causal inferences cannot be made, and it’s not possible to “test” in a “traditional” sense  Chemistry would give you Na2O + 2HCl = H2O + 2NaCl  In case studies only associations and correlations can be made  Processes can be time-consuming and cumbersome  Organising access, non-disclosure agreements  Requirement to be on-site  Willingness of “participants” to participate  Labour in transcribing interviews…
  22. 22.  Events cannot be controlled Intranet implementation: access agreed first week of September 2001 for interviews to start 1st October 2001…
  23. 23. Value of case studies  In-depth studies  “Power of good example” derives from “rich” data  Intranet implementation  Particularly useful for new areas of research, where  there is little/no extant literature and previous empirical evidence is lacking  Blogging in the classroom  Generate new hypotheses for future testing  Blogging in the classroom  Often inexpensive  Depends on depth of study (and how you transcribe interviews)
  24. 24. Resources  Research methods textbooks in business and management are useful for organisational research in general  Most general research methods textbooks include chapters on case study research  Three particularly useful texts  Eisenhardt, K. (1989). Building theories from case studies. Academy of Management Review, 14, 532-550.  Flyvbjerg, B. (2001). Making social science matter. Cambridge: Cambridge University Press.  Yin, R.K. (2013). Case study research (5th ed.). London: Sage.
  25. 25. Flyvbjerg, B. (2001). Making social science matter. Cambridge: Cambridge University Press.
  26. 26. Yin, R.K. (2013). Case study research (5th ed.). London: Sage.
  27. 27. Intranet implementation: Flyvbjerg (2001) helpful to justify case study approach.
  28. 28.  What are the main “questions” you would need to ask?  Which methods could you use to collect data?  Who would you collect data from?  How will you organise and analyse the data that you have collected? An investigation into the impact of UK information science research Exercise
  29. 29. Analysis of data for organisational research Professor Hazel Hall Institute for Informatics and Digital Innovation/School of Computing
  30. 30. In this section  Data analysis as part of the research process  Data, evidence and findings  Role of coding data in data analysis  Coding exercise
  31. 31. Data analysis as a process  Design methods  gather evidence  present case  Move from description of elements, e.g. object, people, phenomena “observed” to explanation, i.e. analysis.  Output is tied to purpose of the research and related research questions, with scope for extension  discover what the research is really about  new research questions may emerge  example: intranets and information sharing  power issues and knowledge management
  32. 32. Refine & develop concepts – critical treatment Research Established “theory” Intranet implementation Poor understanding of knowledge sharing with technologies Blogs in the classroom Blogs promote reflective learning Contribution (Action) Knowledge sharing practice is local. Efforts to knowledge share is influenced by power bases (Adopt communities approach to KM in case study organisation) Blogs do not promote reflective learning to the extent reported (Pay attention to weekly blog hints to engineer reflection)
  33. 33. Refine & develop concepts – critical treatment Research Established “theory” Intranet implementation Poor understanding of knowledge sharing with technologies Blogs in the classroom Blogs promote reflective learning Contribution (Action) Knowledge sharing practice is local. Efforts to knowledge share is influenced by power bases (Adopt communities approach to KM in case study organisation) Blogs do not promote reflective learning to the extent reported (Pay attention to weekly blog hints to engineer reflection) Data analysis Data analysis
  34. 34. Belief in your research results  Research findings are expected  to be grounded in evidence  not to be based on speculation, nor on weak inference  Therefore decisions on data analysis are important  Example from e-information roles study: apparently more opportunity in the public and voluntary than the corporate sector. However:  less obligation in corporate sector to advertise posts  public and voluntary sector organisations could be playing “catch up” with the corporate sector We could not be confident that this finding was grounded in evidence because our data collection was not extensive enough
  35. 35. Inevitability of “too much” data  Assume it’s murder  safety net  Not all data collected will be analysed - data collectively “emphasised”  to serve as evidence  to build a case  Tension  present a set of understandable findings  yet acknowledge the complexities of the social world under investigation
  36. 36. Data, evidence and findings  Data + interpretation = evidence  Evidence = social product/artefact of work completed  Evidence  findings  Data cannot (normally) speak for itself, so  data ≠ evidence  evidence ≠ mere illustration  evidence is built from multiple data sets  research design should permit multiple collection of “same” data for triangulation purposes  obligation falls on the researcher to check alternative claims for the evidence collected
  37. 37. Links between findings & research design  Outcome of data analysis (findings) must be understood in the context of the methods adopted  Example from e-information roles study: globalisation the strongest driver in the creation of new job roles in the corporate sector. However:  Research design determined sample selection focused on large, multinational companies So we were confident that the finding was grounded in evidence, within the context of our sample  Obligation to provide detailed and comprehensive account of both findings and basis on which they were obtained
  38. 38. Data analysis: some examples Research Format of data Coding & analysis Intranet implementation Recorded interviews; interview notes; archive of company documents Interviews transcribed; interview data coded using Ethnograph; archive details organised into historical sequence & coded manually – “content analysis” Blogs in the classroom Students’ blog entries; survey Content analysis of blog entries; survey results not incorporated e-information roles Focus group notes; job adverts; job descriptions; survey; telephone interview notes Combined mind-mapping of focus group notes & job data; survey & telephone data analysed using Excel Outsourcing research & information services Interview notes; provider web sites Interview data coded & analysed manually – total of 11 data sets
  39. 39. Data analysis options  Analysis using software  Standard packages  Word  Excel  Access  Dedicated software  SPSS - http://www.spss.com/  Ethnograph - http://www.qualisresearch.com/  Nvivo http://www.qsrinternational.com/products_nvivo.aspx  Atlas.ti - http://www.atlasti.com/  Manual analysis
  40. 40. Use of Excel to analyse survey & interview data for the e-information roles project Relative ranking of the importance of employee backgrounds: computing, business, librarianship Column M records comments
  41. 41. Date Data Source 8 November 2001 History – investment Budget changes Named meeting minutes Use of Word to analyse document data for intranet implementation Source of information Date of activity/development Activity/development
  42. 42. H: Who controls 422 the Intranet content, is it 423 controlled by you in XX … rather than 424 from the centre, from the KM group …? 425 #-CONTROL $-RELS KMG P: Well, in terms of what tools and what 427 -#-$ facilities are made available to us, 428 | | that's obviously controlled by the 429 | | central group. But in terms of the XX 430 |-$ content and the XX presence, that's 431 | entirely controlled by me … simply 432 | because it wouldn't be relevant to go 433 | through a central group. 434 -# H: Yeh, OK. You've told me about 436 #-INT BUY-IN ownership. How … it sounds as if 437 -# you've got really good buy-in from 438 | your own set of people … 439 | | P: Absolutely. 441 -# #-INT BUY-IN H: What about the Intranet as a whole in 443 -# the UK? What are your perceptions of 444 | buy-in there? 445 | | $-KM SPONS P: I think it varies. I mean, I'm very 447 -#-$ fortunate in that I report into the 448 | KM partner, who's also one of the 449 | senior partners … 450 -$ H: Which, who …? 452 Use of Ethnograph to analyse interview data for intranet implementation
  43. 43. “Translating” data for analysis - coding  Coding  records instances of occurrence  organises data into categories  comprises part of the analysis stage in qualitative research
  44. 44. Attention to coding in research design  Design of research tool has determined predefined codes Indicate the best day of the week for team leader meetings: A. Monday B. Tuesday C. Wednesday D. Thursday E. Friday F. Don’t know G. No preference H. Not applicable Note also the importance of the last three options: there is a difference between not having a preference and not knowing; if this forms part of a survey of staff who have nothing to do with team leader meetings, there needs to be an option for their response. Attention to coding at the design stage can help with asking the “right” questions.
  45. 45. Coding down  Data is coded according to predefined categories  identified in range of work brought together in literature review  identified in a single piece of work  commonly deployed, e.g. age breakdowns used in national statistics
  46. 46. Dimension Code Interpretation Evidence Reflection C Content-free Comment makes no reference to points in the original entry. U Non-reflective (U=’unreflective’) Comment makes reference to the original blog entry, the module content or the general context in order to state an opinion, emotion or a point of fact or theory. R Reflective Comment addresses points from the main blog entry and demonstrates a consideration of the validity of the content, the process or the underlying premise. Propositional stance A Agree Comment actively supports the point made in the original entry. I Indifferent Comment neither supports nor challenges original entry. D Disagree Comment takes up a contradictory position to the original entry. Affective P Positive Comment is encouraging, approving, accepting, etc. E Even Comment appears affectively neutral. N Negative Comment is hostile, discouraging, dismissive, etc. Scheme based on Kember, D., Jones, A., Loke, A., McKay, J., Sinclair, K., Tse, H., Webb, C., Wong, F., Wong, M. & Yeung, E. (1999). Determining the level of reflective thinking from students’ written journals using a coding scheme based on the work of Mezirow. International Journal of Lifelong Education, 18(1), 18–30. Example coding down: blog posting data coding scheme
  47. 47. Coding up Data is coded according to categories suggested by the data  Revise codes as new insight is developed through the process of coding - further discovery of what the research is really about  Example from intranet implementation project: seven broad categories related the intranet under investigation  Content  Functionality  History  KWorld  Policy  Staffing  Uptake
  48. 48. Some data in this spreadsheet fits with predefined codes, i.e. in columns D-L. However comments need to be coded up. Relative ranking of the importance of employee backgrounds: computing, business, librarianship Column M records comments
  49. 49. H: Who controls 422 the Intranet content, is it 423 controlled by you in XX … rather than 424 from the centre, from the KM group …? 425 #-CONTROL $-RELS KMG P: Well, in terms of what tools and what 427 -#-$ facilities are made available to us, 428 | | that's obviously controlled by the 429 | | central group. But in terms of the XX 430 |-$ content and the XX presence, that's 431 | entirely controlled by me … simply 432 | because it wouldn't be relevant to go 433 | through a central group. 434 -# H: Yeh, OK. You've told me about 436 #-INT BUY-IN ownership. How … it sounds as if 437 -# you've got really good buy-in from 438 | your own set of people … 439 | | P: Absolutely. 441 -# #-INT BUY-IN H: What about the Intranet as a whole in 443 -# the UK? What are your perceptions of 444 | buy-in there? 445 | | $-KM SPONS P: I think it varies. I mean, I'm very 447 -#-$ fortunate in that I report into the 448 | KM partner, who's also one of the 449 | senior partners … 450 -$ H: Which, who …? 452 Value of software packages for coding and generating reports for analysis Speaker Code Line numbers Data coded
  50. 50. Advice pointers  Be disciplined and systematic when analysing data  especially important to keep records of what you do if you dip in and out of your research work  Be prepared to account for what you have done in the report of your work  another reason to keep good records  When designing data collection tools, look forward to data analysis  good decisions at this stage may save a lot of work at data analysis stage – as will be demonstrated in the class exercise!
  51. 51. The class exercise is based on the responses to questions 3.1, 3.2 and 3.3 in the e- information roles survey
  52. 52. Ability to align work activities to business strategy Ability to connect with developments Ability to cope with change Ability to see the big picture Ability to translate the needs of the business at all levels Abstracting Adaptability Analytic mind Business acumen Business awareness Business development Business focus Cataloguing Change management Classification Collaboration – non-technical Collaboration – technical Commercial awareness Communication Computer literacy Confidence Contract/supplier management Creativity Diplomacy E-learning facilitation Empathy Engaging audiences Enterprise content management Enthusiasm Evaluation of information sources Facilitation Flexibility Grammar Imagination Indexing Influencing Information analysis Information delivery Information governance Information literacy Information retrieval Innovation Integrity Intellectual property knowledge Intelligence Interviewing IT savvy Knowledge harvesting Knowledge of e-information arena, new technologies Knowledge of government policy Knowledge of information sources Knowledge of law Knowledge of public sector vocabulary Languages Leadership Literacy Management Management of individuals Management of teams Marketing Multi-tasking Negotiation Networking Numeracy Organisation Outgoing personality Political awareness Presentation skills Prince 2 Problem solving Professionalism Project management Records management Relationship building Relationship management Repackaging information Research Self-management Small business knowledge Social computing Spelling Stakeholder management Strategic thinking Synthesising information Taxonomy development Technical ability Time management Training Understanding of technical tools Validation of information sources Web authoring Web development Web usability testing Working under pressure Writing How would you group these responses for coding?
  53. 53. Analytical tools and frameworks for organisational analysis Professor Hazel Hall Institute for Informatics and Digital Innovation/School of Computing
  54. 54. In this section  Focus on tools and techniques for organisational and case study analyses through a consideration of:  Purpose and output of frameworks  Actor-network theory as a framework – with example of its application in the research into the intranet implementation  NB there is a wide range of tools and techniques for research in general. Some (or elements of some) are more applicable to organisational and case study research than others. Actor-network theory is just one example for illustration purposes here.
  55. 55. Purpose of frameworks  Frameworks  help make sense of data collected, and thus of phenomena (e.g. organisational dynamics) observed  act as a tool for diagnosis  and thus aid the processes of:  acquiring knowledge  reflection  action for change (if appropriate, for example in an action research setting)
  56. 56. Output of frameworks  Frameworks provide you with a means of formatting your findings  e.g. as a graphical representation of the organisation under investigation  In using a framework you are encouraged to  (re)organise your data  understand what it is that your data represent  present your findings in a format that is understandable to others – the representation can be used as a short-cut to shared understanding
  57. 57. Actor-network theory as a framework - example  Background  Optimism associated with the development of systems to promote knowledge sharing is misguided.  Examples in the literature go back to 1980s.  “Culture” often takes the blame.  Case study organisation wanted explanations as to why the efforts of its knowledge management staff to promote information systems for knowledge sharing were sub-optimal.
  58. 58. Actor-network theory as a tool of analysis  History  Developed in 1980s  Michel Callon and Bruno Latour  Key concepts  Non-humans, as well as humans, are actors  Relationships between actors shift as they compete for organisational resources, from tangible, e.g. office space, to intangible, e.g. corporate attention  Actor-networks grow through successful “translation”  Actor-networks diminish/disintegrate when ties in the network loosen
  59. 59. Relevance of actor-network theory to this case  The organisation was understood as a mesh of competing actor-networks.  The success/failure of corporate initiatives was suspected to be related to the degree to which particular groups enhanced or diminished their organisational power-base.  Service delivery could be examined with reference to historical and social context of the organisation.  The approach provided opportunities to reflect, learn, act.
  60. 60. Actors in the organisation External consultants Senior staff with KM responsibilities (not KM specialists) Knowledge sharing as a concept Intranet Repositories Shared collaboration space Mission statements Specialist KM staff members in centralised unit Specialist KM staff members in business units Senior sponsors of KM (not KM specialists) External systems vendors Intranet usage statistics “Ordinary” staff (not KM specialists) KM strategy KM as a concept
  61. 61. Analysis phase 1 Mission statements KM as a concept Senior sponsor of KM (not a KM specialist) Intranet Specialist IT/KM staff member in centralised unit Senior specialist IM/KM staff member in centralised unit
  62. 62. Analysis phase 2 Mission statements KM as a concept Senior sponsor of KM (not a KM specialist) Intranet Specialist IT/KM staff member in centralised unit Senior specialist IM/KM staff member in centralised unit Senior specialist IM/KM staff member in centralised unit Specialist IM/KM staff members in centralised unit Some specialist IM/KM staff members in business units “Ordinary” staff (not KM specialists)
  63. 63. Analysis phase 3 Mission statements KM as a concept Senior sponsor of KM (not a KM specialist) Intranet Senior specialist IM/KM staff member in centralised unit Senior specialist IM/KM staff member in centralised unit Specialist IM/KM staff members in centralised unit Specialist IM/KM staff members in business units “Ordinary” staff (not KM specialists)
  64. 64. Some findings  Central position of intranet, and its proximity to KM as a concept, account for confusion over what KM represented in the organisation.  Distance between policy documentation and “ordinary” staff explained lack of engagement with KM, and what it implied in terms of behaviours.  Ties between KM staff in business units and “ordinary” staff strengthened over time at the expense of their relationship with the central KM team and the main tool of the KM implementation. As a result their commitment to KM weakened, as did that of their “ordinary” colleagues.
  65. 65. ESRC Scottish Doctoral Training Centre Information Science Pathway Training day 25th June 2014

×