Bridging The Research-Practice Gap Through Evidence-Based Management And Systematic Review. David Denyer and Rob Briner Academy of Management Annual Meeting 2014, Philadelphia

1. BRIDGING THE RESEARCH-PRACTICE
GAP THOUGH EVIDENCE-BASED
MANAGEMENT AND SYSTEMATIC
REVIEW
Friday, Aug 1 2014 12:30PM - 2:30PM at Pennsylvania Convention
Center
Room 202 B (Program Session #: 109 | Submission: 15526)
Prof. Rob Briner and Prof. David Denyer
@daviddenyer | @Rob_Briner | @CenterforEBMgt | #EBMgt

2. Welcome!
1. A brief introduction to evidence-based practice (EBM) and the role of
systematic reviews (SR) in other fields
2. The nature, purposes and processes involved in evidence-based
management (EBMgt) and SR
– Plenary discussion: what questions do you have about EBP, EBMgt
and SR?
3. The language of evidence-based management
– Table discussions
4. Teaching EBMgt and developing SR skills
– Table discussions
5. Academic and practitioner collaboration though EBMgt and SR
– Table discussion
6. Resources available to develop EBMgt and SR

3. 1.
A brief introduction to evidence-
based practice (EBM) and the role of
systematic reviews (SR) in other
fields

4. Evidence-based practice
and SRs not unusual in
other fields
• All practitioners use some of types of evidence in their practice
• Main problem is not using enough good quality evidence nor critically
appraising it for its trustworthiness
• Started in medicine with BMJ editorial (1991) stated that: ‘‘only about
15% of medical interventions are supported by solid scientific evidence’’
• Spread to many other areas of practice

5. What is evidence-based
practice?
• “Evidence-based management is about making decisions through the
conscientious, explicit, and judicious use of four sources of information:
practitioner expertise and judgment, evidence from the local context, a
critical evaluation of the best available research evidence, and the
perspectives of those people who might be affected by the decision.”
(Briner et al., 2009, p. 19)

6. What is evidence-based
practice?

17. Traditional literature
reviews
• Do not usually focus on a specific or practice-relevant question
• Have diverse aims and purposes
• Adopt a wide variety of approaches and structure
• Do not use particular methods or do not explicitly state the methods
used to conduct the review
• Are more prone to bias because, for example, authors can select
studies that support their views and ignore those that do not
• Are less likely to identify the best available evidence
• Do not aim to be comprehensive by including all relevant and available
research
• Are not replicable nor easy to update

18. Systematic literature
reviews
• Always focus on specific and usually practice-relevant questions
• Have similar aims and purposes in that they all focus on answering specific
questions
• Adopt similar approaches and a structure based on a set of broad
principles
• Use particular and explicitly stated methods in order to:
– Search for and identify relevant literature
– Make decisions about what research to include or exclude
– Judge the quality and relevance of the research
– Integrate or synthesize findings
• Are less prone to bias, because, for example, decisions about what to
include and exclude are made explicitly
• Are more likely to identify the best available evidence
• Aim to be comprehensive by including all relevant and available research
within explicitly-stated boundaries or constraints
• Are replicable and easy to update

19. Systematic literature
reviews: Medicine
• Cochrane Collaboration founded in 1993 it aims to help people make well-
informed decisions by preparing, maintaining and promoting the
accessibility of systematic reviews of the effects of interventions in all areas
of health care
• Cochrane Database of Systematic Reviews
– 1995 36 reviews
– 1999 500 reviews
– 2001 1000 reviews
– 2004 2000 reviews + 1400 published protocols (plans)
– 2012 5000+ reviews
• Reviews prepared by healthcare professionals who volunteer (10000
people worldwide)
• Cochrane Review Groups
• Application and debate about quality standards

20. Systematic literature
reviews: Social Policy
• Campbell Collaboration founded in 2000
• An independent, non-profit, and largely voluntary collaboration
• Mission is help people make well-informed decisions by preparing,
maintaining and disseminating systematic reviews in education, crime
and justice, and social welfare.

21. 2.
The nature, purposes and processes
involved in evidence-based
management (EBMgt) and SR

22. The steps in Evidence-
Based Practice
1. Asking: Translating a practical issue or problem into an answerable
question
2. Acquiring: Systematically searching for and retrieving the evidence
3. Appraising: Critically judging the trustworthiness and relevance of the
evidence
4. Aggregating: Weighing and and pulling together the evidence
5. Applying: Incorporating the evidence in the decision-making process
6. Assessing: Evaluating the outcome of the decision taken

23. The mains steps in
conducting a Systematic
Review
1. Identify and clearly define the question the review will address.
2. Determine the types of studies and data that will answer the question.
3. Search the literature to locate relevant studies.
4. Sift through all the retrieved studies in order to identify those that meet
the inclusion criteria (and need to be examined further) and those that
do not and should be excluded.
5. Extract the relevant data or information from the studies.
6. Critically appraise the studies by assessing the study quality
determined in relation to the review question.
7. Synthesize the findings from the studies.
8. Consider potential effects of publications or other biases.

24. Plenary discussion
• What questions do you have about EBP,
EMBgt and SR?

25. 6.
Resources available to develop
EBMgt and SR

26. Plenary discussion
• What questions do you have about EBP,
EMBgt and SR?

27. 3.
The language of evidence-based
management

28. Table discussions
• What does 'evidence' mean?
• What different connotations does it have?
• What does 'evidence-based' mean?

29. Misinterpretations and
barriers
Misinterpretations:
• EBMgt is done by scholars not practitioners
• EBMgt is a single rigid formulaic method
• EBMgt is only concerned with effect size, randomised-
controlled trials and meta-analysis etc.
Barriers
• Preoccupation with ‘new knowledge’ and ‘new
questions’
• Evidence aware/informed or based?
• Integration with experience and other forms of evidence

30. 4.
Teaching EBMgt and developing SR
skills

31. Table discussions
• Why are SRs useful and important -
academically and practically?
• How do (or could) you teach students and
others to do this?
• Share examples of curricula, assignments,
etc.

32. How EBMgt and SR skills
can be taught to
different groups
• Cranfield DBA
• Co-producing SRs
Engagement
Research
Dissemination
Exploitation
Evaluation
From
Engagement
Integration?
Integration?

33. How EBMgt and SR skills
can be taught to
different groups
• Cranfield DBA, executive education
• Co-producing SRs
Engagement
To
Dissemination
Exploitation
Research
Evaluation
Integration

34. 5.
Academic and practitioner
collaboration though EBMgt and SR

35. Table discussions
• How can EBMgt be done in practice? How
common is it?
• What are the barriers to managers becoming
more evidence-based?
• How can academics work with organizations
and managers to produce systematic reviews
that address practical questions?

36. HOW organisations ARE
commissioning systematic
reviews from academics
• AWA – Advanced Workplace Associates
• Network for Business Sustainability

37. 6.
Resources available to develop
EBMgt and SR

38. CEBMa Website

39. Other websites and
publications

40. BRIDGING THE RESEARCH-PRACTICE
GAP THOUGH EVIDENCE-BASED
MANAGEMENT AND SYSTEMATIC
REVIEW
@daviddenyer | @Rob_Briner | @CenterforEBMgt | #EBMgt

41. Extra
slides

42. Some people care about
research impact
– Hambrick - What if the Academy actually mattered?
– Huff - Mode1/2 knowledge production
– Van de Ven - Engaged scholarship
– Bartunek - Collaborative research
– Pearce - What do we know and how do we really know it?
– Rousseau - Evidence-based Management
42

43. The situation
The research conducted by most business school
academics does not have a marked influence on the
practice of management
(Currie, Knights and Starkey, 2010)

44. The current evidence-
base
In general, managers
prefer to update their
knowledge and skills by
reading popular
management books,
typically written by
consultants, former
executives or journalists
(Pfeffer and Fong, 2002; Rynes, Bartunek and Daft, 2001)

45. Evidence-based management:
integrate different forms
of knowledge
“Evidence-based management
is about making decisions
through the conscientious,
explicit, and judicious use of
four sources of information:
practitioner expertise and
judgment, evidence from the
local context, a critical
evaluation of the best available
research evidence, and the
perspectives of those people
who might be affected by the
decision.”
(see Briner, Denyer, Rousseau, 2009)

46. Hierarchy of evidence?