The document outlines the peer review evaluation system for learning from major public order events, highlighting its goals, methods, and the importance of stakeholder involvement. It emphasizes the need for a structured approach to evaluations, focusing on practical implications and continuous adaptation rather than mere accountability. Various challenges and recommended practices for effective evaluation, particularly in police operations during public events, are also discussed.

1. The peer review evaluation system for Evaluation & Review
learning from major events • Who is responsible?
• When?
• Who evaluates?
• What is the goal of evaluation?
• What are the most important questions?
Otto Adang
• Who needs it? / What’s the use?
Peer Review Centre
• How is evaluation done?
at the Police Academy of the Netherlands
• What criteria are being used?
Why learning is important
Quality of learning public order Incidents (observer data)
management
12
• Fragmentary: isolated from other events and from
10
partners
% of samples
8
• Too often under pressure
• Often not transparant 6
• Confusion with accountability 4
• Lack of analysis 2
• More focused on past than on future 0
• Recommendations often not used in practice NL/ B 2000 Portugal 2004 NRW 2006
A good evaluation Utility focused evaluation
• Fullfills a need
• Has a relevant scope (Patton, 1997)
• Has a justified design
• Analyses reliable data • Focus is on intended use by intended users
• Draws conclusions that fit the data – Involve the users!
• Has impartial conclusions – Clarify use!
• Is clear
• Is about learning, not judging
• Does not stand alone: cyclical process
1

2. Approach to evaluation of The concept: peer review
public order managament
• On a voluntary basis
• Conduct intercollegiate reviews, focused on learning:
– identification of good practices • Involve experienced officers
– exchange of experiences • From several forces/ countries
– development of professional norms • Based on an evaluation plan, developed in relation
• Well-planned, in cooperation with local commander with inviting force, involving issues relevant to
• Using a transparent methodology & criteria inviting force
• Making use of international community of experienced • Observe in real time: focus not after-the-fact on
practicioners, trained in evaluation incidents, but on handling of events
• Development: evaluation teams (Netherlands, Euro • EU-handbook starting point
2000, Euro 2004, pilot in EU) • Goal is continuous adaptation and learning, not
judging: utilisation focused-evaluation
Advantages EU peer reviews
• Police concerned receive informed and constructive • 14 peer reviews in 10 different countries (UK, D, DK,
feedback in a safe manner E, CH, AU, NL, Cyprus, Romania, Bulgaria)
• Officers involved gain a lot of additional experience • 15 experts from 13 different countries (Austria,
• Exchange of perspectives and findings deepens Belgium, Cyprus, Denmark, France, Germany, Italy,
discussion, contributes to development of Netherlands, Portugal, Spain, Sweden, Switzerland
professional norms and helps international and the United Kingdom) + support from 3 different
understanding and cooperation countries (NL, D, UK)
• 4 Champions league matches, 4 national team
matches, 5 local risk matches
Use of the peer reviews
• For hosts AND reviewers: a win-win situation
• The peer review PROCESS itself was useful: For me it was a great experience and
holding up a mirror, catalyst for change it was very valuable for my work in
• Use of reports future in international football
– the report was sent to and discussed with commanders/ matches. Thanks for the possibility to
units involved in the operation have joined this team.
– the report was used for internal reflection within the force
– the report was used in discussions with partners
• Examples: Romania, Cyprus, Denmark, Switzerland
and Euro 2008
2

3. Euro 2008 Switzerland Points of attention
Points of attention
• Implementation of recommendations from EU handbook
• Implementation of the 3D concept
• Involvement of non-EU member states
• Relation between risk assessment and tactical
deployment • A failure of the operation/ officers to implement the
chosen strategy
• Cooperation between host cities and police forces
providing support • A lack of balance between police officer uniform and
behaviour on the one hand and actual risk on the other
• Uniformity in approach between different host cities
• A lack of interaction and communication between police
• Role of and cooperation with stadium security officers and fans
• Communication with fans • Imperfections in the cooperation between police and
partners (either police or non-police partners such as
transport police, club, stewards etc.)
Good practices Good practices
• careful and elaborate preparation of and planning • specific attention for crowd safety and for officer
for the operation, good cooperation with agencies safety, consistent use of well-structured briefings
external to the police and effective management of (with the use of visual aids), half-time briefings and
media for benefit of police and local community. The debriefings, positioning of officers in the operation
teams also witnessed examples of use of dynamic being determined by their experience rather than
risk assessment, existence of a balance between their rank, use of a GPS system to monitor the
police profile and actual situational risk and tactical movements of all units in the operation, and wearing
deployments in line with a friendly and firm, of protective vests underneath rather than above
community oriented policing approach, also in clothing.
situations of increased risk
Good practices: Dynamic risk
Good practices assessment
• positive facilitation of and communication with fans, • The clear distinction between different types of risks,
including language courses for officers and and especially between spontaneous and planned
conveyance of key information to fans via websites disorder
before arrival in a foreign city, police involvement in • The availability of different types of contingency
preventive activities aimed at fans, formulation of a scenarios related to risks (e.g. for evacuation, bomb
clear behavioral profile that was clearly threat, flares, pitch invasion)
communicated to officers • A clear identification of hot spots based on previous
experiences and complaints by the public and flexible
deployment in relation to these hot spots
• The fact that intelligence officers had received
dedicated training
3

4. Good practices: Tactical
deployment
Critical succes factors
• Rapid intervention and degrading when necessary • Peer aspect: Involvement of commanders
• Use of the planning officer as a quality assurance • Open, informal and utilisation focused approach
officer • Learning from the bottom up, not an inspection
• Use of dedicated tactical advisors • Holding up a mirror
• Use of a liaison officer working with units from • Also identifying good practice
visiting/ neighbouring forces providing assistance • Mix between practice and theory
- Explicit use of police officers with specific • focus on specific topics, tailored
experience for specific functions or at specific posts
• consistent systematic methodological approach
Challenge for the future Topics identified
• Exchange and implement good practices identified, • Use of spotters (plainclothes/ uniform)
including 3D/ low-profile strategies • Task of spotters in relation to fans
• Address points of attention with wider relevance: • Use of stewards from away teams
especially dynamic risk assessment • Division of tasks between police and stewards
• Discuss/ resolve topics where there is a lot of • Segregation of fans
variation between forces/ countries
• Use of intelligence
• Treatment of fans
Next steps Suggestions for the future
• EU training course with peer reviews as practical • implement peer reviews at a national level as well
component • cascade information about peer reviews through NFIPs
• involve commanders first as member of a peer review before
• National peer reviews asking them to be host
– Scotland • involve the peer review team in the host debriefing process
– Sweden • link up with other initiatives within Europe (including CEPOL)
– Metropolitan police (UK) so that experiences can be shared more fully in a more
coordinated way
– Netherlands
• encourage and facilitate peer reviews in countries that
– … organise championships
• Involve partners
• Apply to other (non-football) public order policing
4