Best Evidence Medical Education AMEE 2014 Milan Madalena Patricio António Vaz Carneiro Yvonne Steinert

1. BICC/BREC Orientation Session
Sunday, August 31
Yvonne Steinert Madalena Patricio António Vaz Carneiro

2. Part 1
 BREC and BICC collaboration
 Mutual responsibility
Part 2
Introduction to BICCs
 The McGill BICC – A Case Study
 Reviewing Protocols and
Complete Reviews
 Moving forward

3. BREC / BICC collaboration
Handling process
Facilitating tools
 Checklist to support reviewers
 Master flow sheet
The way forward

4. BREC / BICC collaboration
BEME
Collaboration

5. Board
The Board’s overall
responsibility is for the
strategic development
and co-ordination of
current activities.
BEME
Collaboration
BREC / BICC
collaboration

6. Board
BREC BEME
BREC’s responsibility is to
the BEME Board for
reviewing, monitoring,
and guiding BEME
reviews under the process.
Collaboration
BREC / BICC
collaboration

7. BICC’s responsibility is for
progressing the ongoing
work of the BEME
Collaboration and planning for
its further development.
Board
BREC
BICCs
BEME
Collaboration
BREC / BICC
collaboration

8. The responsibility of the
Congress is for advising the
Board with regard to current
activities, policy, and future
directions for the BEME
collaboration.
Board
BREC
BICCs
BEME
Collaboration
BREC / BICC
collaboration
Congress

9. The responsibility of review
groups is to register and
undertake a review of an
agreed review topic relating to
education in the health
professions.
Board
BREC
BICCs
BEME
Collaboration
BREC / BICC
collaboration
Congress
Review
Groups

10. AMEE provides academic
and administrative support
for BEME and provides
facilities for BEME at its
annual meeting and for
publishing BEME reviews.
Board AMEE
BREC
BICCs
BEME
Collaboration
BREC / BICC
collaboration
Congress
Review
Groups

11. Board AMEE
BREC
BICCs
BEME
Collaboration
Congress
Review
Groups
BREC / BICC collaboration

12. António Vaz Carneiro
University of Lisbon
Madalena Patricio
University of Lisbon
Diana Dolmans
Maastricht University
Trevor Gibbs
AMEE
Morris Gordon
University of Central
Lancashire/Blackpool Victoria
Hospital
Larry Gruppen
University of Michigan
Simon Guild
St Andrews University
Barry Issenberg
University of Miami
Yvonne Steinert
McGill University
Jill Thistlethwaite
Australia
Andrei Achimas
Iuliu Hatieganu University of
Medicine and Pharmacy
Nehad El-Sawi
Alabama College of
Osteopathic Medicine
Chris Green
Consortium of Universities of
Birmingham, Bournemouth,
Essex, and East Anglia
Sara Mortaz Hejri
Tehran University of Medical
Sciences
Mark Newman
University College of London
and Institute of Education
Teresa Pawlikowska
Royal College of Surgeons
Jenny Schafer
University of Queensland
Susan van Schalkwyk
Stellenbosch University
Faculty of Medicine and
Health Sciences
David Taylor
University of Liverpool School
of Medicine
Rochell Tractenberg
Georgetown University
Medical Centre/Samueli
Institute
Aliston Whelan
Washington University School
of Medicine, St Louis

13. Maastricht University, The Netherlands
McGill University, Canada
St Andrews University, UK
University of Michigan, USA
University of Queensland, Australia
UCLAN/Blackpool Victoria Hospital, UK
University of Liverpool, UK
London BICC, UK
Consortium of Universities of Birmingham, Bournemouth, Essex and East Anglia, UK
Royal College of Surgeons, Ireland
Kings College, London/University of Washington, Seattle, USA
Tehran University of Medical Sciences, Iran
Georgetown University, USA
Stellenbosch University, South Africa
Washington University, Saint Louis, USA
Iuliu Hatieganu University of Medicine and Pharmacy, Romania
Alabama College of Osteopathic Medicine, USA
Aberdeen Patient Safety Consortium, UK
orig
new

14. BEME
Dundee,
Scotland, UK
The BEME office, with
Louise Russell as the
BEME Administrator, is
crucial for the good and
easy articulation of
BREC and BICCs
Louise
Russell

15. BEME
Dundee,
Scotland, UK
The BEME website and
database are crucial to
support the mutual
responsibility of the
handling process

16. BREC / BICC collaboration
Handling process
Facilitating tools
The way forward

17. Handling process
Is structured in 10 steps

18. Handling
process
1
1. Selecting a
topic
2. Form a review
group
3. Registering a
topic
4. Protocol
5. Pilot review
6. Progress
reporting
7. Submit a
completed
review
8. Publication
process
9. Update review
10. Discussion
forum
2
3
4
5
6
7
9
8
10

19. How does collaboration occur in
practice between BREC and BICC?
Let us look at the example
of a protocol

20. Protocol
Protocol allocation
BREC

21. Protocol
Protocol allocation Protocol acceptance
BICC

22. Protocol
Protocol allocation Protocol acceptance
Paperwork sent to
BICC BREC

23. Protocol
Protocol allocation Protocol acceptance
Paperwork sent to
BICC
Protocol to be sent
to three independent
reviewers
BICC

24. Protocol
Protocol allocation Protocol acceptance
BICC
Paperwork sent to
BICC
Protocol to be sent
to three independent
reviewers
Comments received
from reviewers

25. Protocol
Protocol allocation Protocol acceptance
Paperwork sent to
BICC
Protocol to be sent
to three independent
reviewers
Comments received
from reviewers
BICC
recommendation
based on comments
by reviewers to be
sent to BREC
BICC

26. Protocol
Protocol allocation Protocol acceptance
Paperwork sent to
BICC
Endorsement by
BREC members
Protocol to be sent
to three independent
reviewers
Comments received
from reviewers
BICC
recommendation
based on comments
by reviewers to be
sent to BREC
BREC

27. Where there is a clear
recommendation from the BICC
and a consensus of the reviewers
BREC
Members of the committee will be invited
to endorse the decision with information
made available to them
Members of the committee may choose
not to look at the documentation, but if
they wish to do so information is
available
If no consensus is achieved, a decision is
made by voting – Chair has a casting vote

28. Where there were problems with a
review or a lack of consensus
about a decision
BREC
Review should be referred to BREC for a
wider discussion and agreement as to
what should be done. BREC forum will be
used for getting consensus and a web
meeting is arranged when needed
It is hoped that only a small number of
reviews will fall into this category

29. Protocol
Protocol allocation Protocol acceptance
Paperwork sent to
BICC
Endorsement by
BREC members
BREC decision
Protocol to be sent
to three independent
reviewers
Comments received
from reviewers
BICC
recommendation
based on comments
by reviewers to be
sent to BREC
BREC

30. Protocol
Protocol allocation Protocol acceptance
Paperwork sent to
BICC
Endorsement by
BREC members
BREC decision
Information to be
sent to authors
Protocol to be sent
to three independent
reviewers
Comments received
from reviewers
BICC
recommendation
based on comments
by reviewers to be
sent to BREC
BREC

31. Protocol
BREC
Information to be
sent to authors
Protocol
decision
YES
Protocol
accepted as
it stands
Protocol
decision
NO
Protocol is
not accepted
Protocol
decision
YES, but
Protocol must
be revised and
resubmitted
Protocol
decision
UNCERTAIN
Further
discussion/
approval is
needed by
BREC members

32. Protocol
Protocol
decision
YES, but
Protocol
must be
revised and
resubmitted
Minor Changes
If minor changes are
requested, BICC can
decide on progress
Major Changes
If major changes are
requested,
BICC/BREC
procedures are
repeated
BICC
BREC BICC

33. How is the BREC’s final
decision made?
Who is responsible for what?
BREC

34. The BREC, as a committee, is
responsible for decisions with
regard to systematic reviews
Registration
of a review
Acceptance
of a protocol
Acceptance
of a final
report /
publication
BREC

35. Responsibility of the handling process
will be delegated to a BICC who is
responsible for the work of the BREC
through the BREC representative
a lead role Following an
agreed review
process
Recommendation
to BREC is made
at protocol and
final review steps
BICC

36. BREC members are requested to
endorse the BREC decision at
three points:
Topic registration
Protocol
Submit a completed review
BICC
BREC

37. BREC / BICC collaboration
Handling process
Facilitating tools
The way forward

39. 12 sections
1. Lead author, team, title, and abstract
2. Background to the topic
3. Review questions/objectives, type of review and
key words
4. Study selection criteria
5. Search sources and strategies
6. Extracting data
7. Appraisal of studies
8. Synthesis of evidence and transfer to research
and practice
9. Time frame
10. Human and material resources
11. Overall assessment
12. Recommendation / Final decision

40. 1. Lead author, team, title, and abstract
2. Background to the topic
3. Review questions/objectives, type of review and
key words
4. Study selection criteria
5. Search sources and strategies
6. Extracting data
7. Appraisal of studies
8. Synthesis of evidence and transfer to research
and practice
9. Time frame
10. Human and material resources
11. Overall assessment
12. Recommendation / Final decision

41. 1. Lead author, team, title, and abstract
2. Background to the topic
3. Review questions/objectives, type of review and
key words
4. Study selection criteria
5. Search sources and strategies
6. Extracting data
7. Appraisal of studies
8. Synthesis of evidence and transfer to research
and practice
9. Time frame
10. Human and material resources
11. Overall assessment
12. Recommendation / Final decision

42. 1. Lead author, team, title, and abstract
2. Background to the topic
3. Review questions/objectives, type of review
and key words
4. Study selection criteria
5. Search sources and strategies
6. Extracting data
7. Appraisal of studies
8. Synthesis of evidence and transfer to research
and practice
9. Time frame
10. Human and material resources
11. Overall assessment
12. Recommendation / Final decision

43. 1. Lead author, team, title, and abstract
2. Background to the topic
3. Review questions/objectives, type of review and
key words
4. Study selection criteria
5. Search sources and strategies
6. Extracting data
7. Appraisal of studies
8. Synthesis of evidence and transfer to research
and practice
9. Time frame
10. Human and material resources
11. Overall assessment
12. Recommendation / Final decision

44. 1. Lead author, team, title, and abstract
2. Background to the topic
3. Review questions/objectives, type of review and
key words
4. Study selection criteria
5. Search sources and strategies
6. Extracting data
7. Appraisal of studies
8. Synthesis of evidence and transfer to research
and practice
9. Time frame
10. Human and material resources
11. Overall assessment
12. Recommendation / Final decision

45. 1. Lead author, team, title, and abstract
2. Background to the topic
3. Review questions/objectives, type of review and
key words
4. Study selection criteria
5. Search sources and strategies
6. Extracting data
7. Appraisal of studies
8. Synthesis of evidence and transfer to research
and practice
9. Time frame
10. Human and material resources
11. Overall assessment
12. Recommendation / Final decision

46. 1. Lead author, team, title, and abstract
2. Background to the topic
3. Review questions/objectives, type of review and
key words
4. Study selection criteria
5. Search sources and strategies
6. Extracting data
7. Appraisal of studies
8. Synthesis of evidence and transfer to research
and practice
9. Time frame
10. Human and material resources
11. Overall assessment
12. Recommendation / Final decision

47. 1. Lead author, team, title, and abstract
2. Background to the topic
3. Review questions/objectives, type of review and
key words
4. Study selection criteria
5. Search sources and strategies
6. Extracting data
7. Appraisal of studies
8. Synthesis of evidence and transfer to
research and practice
9. Time frame
10. Human and material resources
11. Overall assessment
12. Recommendation / Final decision

48. 1. Lead author, team, title, and abstract
2. Background to the topic
3. Review questions/objectives, type of review and
key words
4. Study selection criteria
5. Search sources and strategies
6. Extracting data
7. Appraisal of studies
8. Synthesis of evidence and transfer to research
and practice
9. Time frame
10. Human and material resources
11. Overall assessment
12. Recommendation / Final decision

49. 1. Lead author, team, title, and abstract
2. Background to the topic
3. Review questions/objectives, type of review and
key words
4. Study selection criteria
5. Search sources and strategies
6. Extracting data
7. Appraisal of studies
8. Synthesis of evidence and transfer to research
and practice
9. Time frame
10. Human and material resources
11. Overall assessment
12. Recommendation / Final decision

50. 1. Lead author, team, title, and abstract
2. Background to the topic
3. Review questions/objectives, type of review and
key words
4. Study selection criteria
5. Search sources and strategies
6. Extracting data
7. Appraisal of studies
8. Synthesis of evidence and transfer to research
and practice
9. Time frame
10. Human and material resources
11. Overall assessment
12. Recommendation / Final decision

51. 1. Lead author, team, title, and abstract
2. Background to the topic
3. Review questions/objectives, type of review and
key words
4. Study selection criteria
5. Search sources and strategies
6. Extracting data
7. Appraisal of studies
8. Synthesis of evidence and transfer to research
and practice
9. Time frame
10. Human and material resources
11. Overall assessment
12. Recommendation / Final decision

52. What is expected from
reviewers?
• Confidential comments to the lead
BICC and BREC
• Comments which can be forwarded
as they stand to the systematic review
group
Above all…
• Clear recommendation (acceptance
as it stands/ accepted with
minor/major changes, not accepted)

53. The checklist for reviewers is JUST a support tool
It’s a dynamic document so any suggestions for
improvement are most welcomed
Therefore, feedback is expected from reviewers

56. 10 sections
Each corresponding to a
step in the handling process

59. BICC
• To describe who, what, and when
concerning each stage of a
systematic review
• To set the deadline for each stage
• To indicate appropriate
standardized letter(s) for each stage
(which will be made available to all
BREC members asap)

61. All relevant material
concerning reviews under
process will be available
for BREC members on the
BEME website / database
or through Dropbox.
This system is under
process and will be
communicated to all BREC
members when available.
Only ‘Confidential’
comments sent by
reviewers will have
restricted access.

63. A full set of
standardized letters will
be available for BREC
members to use by
clicking on the SL# on
the master flow sheet.
Each BICC
representative can
adapt the text to his/her
own convenience.

64. BREC / BICC collaboration
Handling process
Facilitating tools
The way forward

65. BREC is responsible to the
BEME Board for reviewing,
monitoring, and guiding
BEME reviews under the
process
BICCs are responsible for
progressing the ongoing
work of the BEME
Collaboration and planning for
its further development
BREC
BICC

66. 26

68. 43

69. • After registration and protocol
are accepted, AMEE is
committed to publish the
review
• Publication depends on the
successful achievement of all
steps of the handling process

70. • The BEME peer review
process is different from peer
review in other journals
• BEME is committed to support
authors throughout the
handling process for a
successful systematic review

71. • The right balance between high
quality/exigency and high support from
BREC and BICCs is difficult to achieve,
but must be pursued
• BEME achieved a new level in the
complex area of secondary research and
we believe it can take a leading role
worldwide
• The collaboration of BREC and BICCs is
a key element for this success.
Therefore, BREC counts on the BICCs
and the BICCs count on us.

73. BICC/BREC Orientation Session
Sunday, August 31
Yvonne Steinert Madalena Patricio António Vaz Carneiro