Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Evidence-Based Practice, Evidence From Key Domains: Management


Published on

Calls for both practical and scholarly activities to be grounded more in actual evidence have become louder, especially in the last decade. Four domains in particular have embraced evidence-based thinking, resulting in the respective developments of evidence-based medicine, evidence-based management, evidence-based education and evidence-based policy. Despite the presumed benefits of drawing on different sources of evidence for decision-making in practice, whether in medicine, management, education or policy, this does not seem to prevail. Whilst one likely reason for this slow uptake could simply be down to practitioners not always having much time to consult the evidence-base in their day-to-day work, another reason might be that they are not aware of specific insights applicable to their domain of work or to practice in general.

This is where the workshop contributes:

Representatives from the four key domains engaged with evidence-based practice will share with the audience their latest insights and the consequences thereof for practice. Further, all speakers will discuss questions such as:

What do we have in common?
How can we learn from one another?
How can we combine insights from the four domains?

These will be discussed as part of a concluding panel.

Workshop organiser:

Dr Celine Rojon, University of Edinburgh,

Published in: Education, Health & Medicine
  • Be the first to comment

Evidence-Based Practice, Evidence From Key Domains: Management

  1. 1. Evidence-Based Management Evidence-Based Practice Insights from Key Domains University of Edinburgh, May 2nd 2014 Eric Barends Rob Briner
  2. 2.  A brief history  Current developments  Future developments  Challenges  Implications for practice & research 2 Evidence-Based Management
  3. 3. 3 Evidence based management: A brief history
  4. 4. 4 Manager > ‘Manage’ introduced mid 16th century: from Italian maneggiare, “to handle”, based on Latin manus, “hand”. Management? (we’re all managers)
  5. 5. History of management/business schools  Late 1800s - started as trade schools  1950s - Ford and Carnegie reports criticised business schools for not being sufficiently academic or scientific  1980s onwards – criticised for producing research irrelevant to practice and students without relevant skills or knowledge 5
  6. 6.  On the one hand they fear . . . the scorn of other, more traditional academic subjects. On the other hand, they often stand accused of being less than relevant to business. (Grey, 2001)  2000s onwards – criticized for producing ideas and students that lead to financial crisis and poor ethical behaviour (e.g., Enron, only one US president has MBA, MBA students cheat more than other graduate students) 6 History of management/business schools
  7. 7. Management schools are big business  Nat. Center for Education Statistics (USA, 2011-’12)  20% of all Batchelor’s Degrees  25% of all Master’s Degrees  Higher Education Statistics Agency (UK, 2012-’13)  27% of all Higher Degrees awarded  18% of all Higher Education qualifications obtained  UK Council for International Student Affairs (2011-’12)  36% of all business school students are international  30% of all international students studying business 7
  8. 8. EBP in Management  Many similar ideas in past (e.g., Mode 1-Mode 2, collaborative research, rigour and relevance)  Rousseau (2005) Presidential address  Peffer and Sutton (2006) Hard Facts, Dangerous Half- Truths, and Total Nonsense: Profiting from Evidence- based Management 8
  9. 9. EBP in Management  EBMgt Collaborative (2007-’08) Rousseau/Carnegie Mellon  Wiley Evidence-Based Management Insights (2007-’08) (attempt to create Cochrane database for management) Briner and Denyer  Presentation to AoM Board of Governors (2008) (attempt to persuade to develop systematic reviews in management)  Center for Evidence-Based Management – more later 9
  10. 10. Evidence based management: What is it? 10
  11. 11. 11
  12. 12. Evidence-based practice Central Premise: Decisions should be based on a combination of critical thinking and the ‘best available evidence‘. 12
  13. 13. Evidence? findings from scientific research, organizational facts & figures, benchmarking, best practices, professional experience 13
  14. 14. All managers base their decisions on ‘evidence’ 14
  15. 15. But…many managers pay little or no attention to the quality of the evidence they base their decisions on 15
  16. 16. Trust me, 20 years of management experience 16
  17. 17. SO ... 17
  18. 18. Teach managers how to critically evaluate the validity, and generalizability of the evidence and help them find ‘the best available’ evidence 18
  19. 19. Evidence-based practice is about making decisions through the conscientious, explicit and judicious use of the best available evidence from multiple sources by 1. Asking: translating a practical issue or problem into an answerable question 2. Acquiring: systematically searching for and retrieving the evidence 3. Appraising: critically judging the trustworthiness and relevance of the evidence 4. Aggregating: weighing and pulling together the evidence 5. Applying: incorporating the evidence in the decision-making process 6. Assessing: evaluating the outcome of the decision taken to increase the likelihood of a favorable outcome. Definition 19
  20. 20. Evidence based decision Professional experience and judgment Organizational data, facts and figures Stakeholders’ values and concerns Scientific research findings Ask Acquire Appraise Aggregate Apply Assess 20
  21. 21. Evidence-based practice: Focus on the decision making process Think in terms of probability 21
  22. 22. Differences?  Research: low internal validity
  23. 23. 0 50 100 150 200 80-84 85-89 90-94 95-99 00-04 05-09 uncontrolled controlled Controlled vs uncontrolled studies
  24. 24. Differences?  Research: low internal validity  Lack of evidence summaries  Focus on multiple sources  Organizational (BIG) data  Stakeholders’ concerns
  25. 25. How evidence-based are we (managers)? “I’ve never thought I need more evidence before making a decision; I know what needs to be done, we get on with it and we get results.” 25
  26. 26. How evidence-based are we?  959 (US) + 626 (Dutch) HR professionals  35 statements, based on an extensive body of evidence  true / false / uncertain HR Professionals' beliefs about effective human resource practices: correspondence between research and practice, (Rynes et al, 2002, Sanders et al 2008) 26
  27. 27. 1. Incompetent people benefit more from feedback than highly competent people. 2. Task conflict improves work group performance while relational conflict harms it. 3. Encouraging employees to participate in decision making is more effective for improving organizational performance than setting performance goals. True (likely) or false (not likely)? 27
  28. 28. Outcome: not better than random chance 28
  29. 29. Evidence based management: Current developments 29
  30. 30. Postgraduate Course 31
  31. 31. Postgraduate Course
  32. 32. Postgraduate Course Current developments Education Access to research databases REAs Building a community
  33. 33. Evidence based management: Future developments 34
  34. 34. Postgraduate Course CEBMa Database of Evidence Summaries Online learning modules Accreditational bodies Future developments: practice
  35. 35. Postgraduate Course  Some move towards systematic review methods in management  Limited interest in teaching EBP in management schools  Management schools still a bit trapped: Cash cows for universities; focus on ‘top quality’ academic research; few incentives to be relevant Future developments: academia 36
  36. 36. Postgraduate Course  Though issues around accountability, ethics, corporate social responsibility could in part be resolved by research and teaching EBP  Need for a professional doctorate? Future developments: academia
  37. 37. Evidence based management: Challenges 38
  38. 38. In general incentives to do non-EBP and punishment (or no incentives) for doing EBP 39 Challenges (same for practitioners and academics)
  39. 39. Incentives for managers  Not rewarded for doing what ‘works’ – few evaluations  Speed and action valued more highly than accuracy and analysis  Managing and understanding power and politics to get things done more valued than understanding and using evidence to make decisions  It may be too late to change existing senior managers… 40
  40. 40.  Barriers from academic / buss school context • In publishing and research • In teaching content and teaching style  Barriers from organizational contexts • Managers love fads and quick fixes • Power and politics 41 Other challenges
  41. 41. In general: academics don’t like EBMgt
  42. 42.  Ambivalence about the value and applicability of management research  Few incentives to get involved  Primary research (collecting new data) valued more highly than secondary research (reviewing existing data)  EBMgt not academics‘responsibility – this is about practice not research  Some concern that systematic reviews will expose the limited nature of management research  Some academics are like ‘gurus’ and feel that EBMgt might show their claims to be untrue Why don’t academics like EBMgt?
  43. 43. Espoused and more implicit goals of management school educators ESPOUSED GOALS  To develop critical thinking  To help students understand the full body of knowledge  To discuss gaps and limitations of our knowledge  To challenge thinking and assumptions  To educate  To teach students how to think for themselves  To maintain quality standards IMPLICIT GOALS  To help students feel successful  To select those bits of that are interesting or digestible or ‘cutting edge’ fads  To reassure students that what we’re teaching them is solid  To make sure students are satisfied  To entertain (edutainment)  Teach students what they need to pass assessments  To give ever-higher grades 44
  44. 44. ESPOUSED GOALS  To advance scientific understanding  Using the best research techniques  Publishing all results and replications – unbiased  Focus on what’s important  Being honest about existing evidence  To disseminate all our evidence and make publically available  Collaboration & cooperation IMPLICIT GOALS  To advance career  Use whatever techniques will get you published  Publishing (mostly) only positive results, no replications  Identifying ‘new’ or trendy topics – creating empires  Exaggerating how much we know  Locking up our evidence behind publishers’ pay walls  Competition for resources, slots in journals, between universities 45 Espoused and more implicit goals of management school researchers
  45. 45. Implications for research  Less focus on collecting new primary data  More focus on systematic reviews and understanding what we know and don’t know  Focusing research efforts on what is academically and/or practically important without compromising ‘academic freedom’  Broader-based research training – management researchers highly and narrowly specialized 46
  46. 46. WHAT’S STOPING US?In general: managers don’t like EBMgt
  47. 47.  Undermines formal authority  They feel it constrains freedom to make managerial decisions  Speed valued and rewarded more than accuracy  Feel they cannot use their own experience and judgment (not true)  Managers not necessarily rewarded for doing what works (organizations rarely evaluate)  THEY LOVE FADS & QUICK FIXES Why don’t managers like EBMgt?
  48. 48. Huge (peer) pressure to adopt fads & quick fixes Fads & quick fixes
  49. 49. Postgraduate Course 51
  50. 50. Postgraduate Course How are fads a problem?  “The main problem…is their lack of any solid intellectual foundation. Implicit in each fad is a cause effect statement that is rarely made explicit and never properly supported.” 52(Donaldson & Hilmer, 1998)
  51. 51. 53
  52. 52. 54
  53. 53. 55 George BuckleyJames McNerney
  54. 54. QUICK FIXES Because quick fixes / fads  Can be career-enhancing for managers  Speed is often valued over accuracy  Do we crave quick and easy solutions? So who needs or wants academic research? So why do managers love quick fixes?
  55. 55. Implications for management practice  Focus on accountability  Focus on next generation  Management = profession  EBP > Professional standard >Accreditation 57
  56. 56. Think about this How can a profession / domain be evidence-based if it is not managed in an evidence-based way? 58 “No job is more vital to our society than that of a manager. It is the manager who determines whether our social institutions serve us well or whether they squander our talents and resources.” Henry Mintzberg