This journal club discusses the development and validation of the Swedish Tool for Risk/Resource Estimation at EvenTs (STREET). The tool was developed to enable common risk assessment and resource planning for mass gathering events by different organizations. It consists of 35 items across 6 dimensions. The tool was validated through expert review and testing on 3 simulated mass gathering event cases. Results found the tool had acceptable reliability and validity for risk assessment. While it provides a way for organizations to collaborate, further real-world testing is still needed to validate the tool's reliability in practice.

1. Journal club
Upasana Sharma
PhD Scholar
National Institute of Epidemiology, Chennai
SCTIMST , TRIVANDRUM
Risk Assessment of
Mass Gathering Events
Seminar 1 1

2. Selected Paper
STREET : Swedish Tool for Risk/Resource
Estimation at EvenTs. risk assessment - face
validity and inter-rater reliability
Berner A, Alharbi T, Carlström E, Khorram-Manesh A.
Journal of Acute Disease. 2015;4(1):37-43.
Published online 30 January 2015
Study Conducted in 2014
Seminar 1 2

3. OUTLINE
1. Background and Rationale
2. Methodology
3. Description of STREET tool
4. Results
5. Discussion
6. Critique
7. References
Seminar 1 3

4. Background & Rationale 1/5
Mass Gathering EVENTS :
• Variable definitions
– Minimum attendance 1,000 up to 25,000
• Large numbers attending a focused event for finite time
• A mass gathering (MG), as defined by WHO, is “any occasion,
either organized or spontaneous, that attracts sufficient
numbers of people to strain the planning and response
resources of the community, city or nation hosting the event”.
.
Seminar 1 4

5. Background & Rationale 2/5
• People in MGs are exposed to common risks such as
collapsing buildings, fire, trampling, high temperature, storm,
aggressions and terrorism
• Different types of mass gatherings may pose very different
public health threats.
• Cause an increase in the level of existing risks, or they can
pose entirely new risks.
• Knowledge of the kinds of problems that may be seen is
important to planning emergency services and health care
Multiple factors have been identified that influence the demand
for healthcare during such an event (Arbon ,2004):
 Biomedical
 Environmental
 Psychosocial
Seminar 1 5

6. Background & Rationale 3/5
• Continuous monitoring of participants' vulnerability
to various risks at MGs and understanding trends in
risks associated with specific events (religious,
sports, or concerts) can be used to predict what
might happen in the future.
• This is also essential for successful preparedness and
management of risk reduction and strengthening the
response capacity of host and home communities.
Seminar 1 6

7. Background & Rationale 4/5
• A reasonable estimation of risks and needed resources
during an event is prerequisite for creating common
understanding & also for planning
• Organizers as well as high reliable organizations (HRO)
need to define the type and potential risks of the event,
degree of collaboration, organizational settings and
assembling areas. For this to happen quickly, there is
need of common tools and plans.
Seminar 1 7

8. Background & Rationale 5/5
• No collaborative instrument to use in assessing the
risks & needs for resources for each organization and
in total prior to the event
• A tool based on the modified version of British “
Purple guide” adjusted to Swedish context was
proposed.
Seminar 1 8

9. Objective
• To develop a validated and generalized high
reliability organizations collaborative tool in
order to conduct common assessments and
information sharing of potential risks during
mass-gatherings.
9Seminar 1

10. Methodology 1/3
Study conducted in three steps :
1. Face validity
2. Data collection Preparation for testing reliability
3. Statistics
Seminar 1 10

11. Methodology 2/3
Tool Development
Old estimation tool completed including the most
important items related to Swedish context
Khorram-Manesh A, Berner A, Hedelin A, Örtenwall P. Estimation of healthcare
resources at sporting events. Prehosp Disaster Med 2010; 25(5): 449-455.
Berner A, Khorram-Manesh A, Hedelin A, Örtenwall P. Avoiding disasters, a tool for
estimating the needs of healthcare resources at sporting and other public events. ICU
Management 2011; 3: 38-39.
Seminar 1 11

12. Methodology 3/3
Seminar 1 12
1. Expert group 1(n=5), skilled instrument developer reviewed
tool
2. Expert group 2( n=9) , PD, RT & EMS* staff for face validity
3. Expert group 1 & 2 pilot tested the tool on simulated cases
4. Suggested changes made in the tool, Face validity
established
Expert group 3 (n=55) consisting of organizers , PD,RT & EMS
staff performed assessment of three fictive case reports
using the tool
Kappa statistics and accuracy calculated
*Emergency medical services (EMS), rescue teams (RT e.g. firefighters) and
police department (PD), etc.

13. Cases
• Three simulated case-reports of planned events (a concert, a
festival and a public hockey game) were included, inspired by
case studies from the literature used
• Adjusted based on written comments from Expert Groups I
and II and reflected all dimensions of the tool
• Face Validity : Expert Groups I and II reviewed the tool,
resulting in three new dimensions and 12 additional items.
(assessment of logic, relevance, understanding, readability, clarity and usefulness)
• Expert Group II provided further comments after testing the
tool in collaboration.
Seminar 1 13

14. Description of STREET tool
• A predictive tool “ STREET” consisting of 35 items grouped
into 6 dimensions was designed.
• Fit different types of events
• Suit HROs as well as organizers
Consists of two different parts :
• First part :Overview of event ( main focus of the study)
• Second part: Adapted to the HROs (Resource assessment)
HROs : emergency medical services (EMS), rescue teams (RT e.g. firefighters) and police
department (PD), etc.
Seminar 1 14

15. Description contd..
Overview of event (Part -1) : 13 items
Character (5 items), Population (3 items) : information about
planned event
Risks ( 5 items ) : Prediction based on character and population
• Added items result in a total score ( Range 0-142)
• Distributed in low, middle and high risk event
• A high score implied a need for HRO reinforcement
STREET.pdf
Seminar 1 15

16. Results
Seminar 1 16

17. Demographic characteristics of Expert Group III
(N=55)
• 27% women and 73% men
• Divided into organizers (n=22, 40%), PD staff
• (n=10, 18%), RT (firefighters) (n=10, 18%) and EMS
staff (n=13, 24%).
• They ranged in age from 29-64 years (m=44.4,
SD=9.7).
• Had at most 36 years of practical experience in their
profession (m=17.2 ,SD=10.1) in planning and
management of different types of event
Seminar 1 17

18. Assessed risk of event
(Data expressed as Mean±SD, Scale 0: no risk, 142: extremely high)
Seminar 1 18

19. Accuracy of tool
Seminar 1 19

20. Inter-rater reliability of Tool (Unweighted
kappa values )
Seminar 1 20

21. Discussion
• Three different expert groups used to develop such a
tool and analysis of inter-rater reliability through
simulated cases showed acceptable reliability and
validity
• Covered all involved organizations and engaged them
in individual evaluations, as well as, a joint discussion
• A common understanding of the event and its
consequences
• All partners could do a risk assessment based on the
information available from organizers.
Seminar 1 21

22. Discussion contd..
• Low accuracy in the festival case was probably due to
the learning curve as the individuals had no previous
experience of using the tool
• Indicates that the tool should be used widely within
all organizations before it can be used in
collaboration with other partners to achieve the
highest possible validity and reliability.
Seminar 1 22

23. Discussion contd..
Use of Simulated cases
• Although using simulated cases is common, it does
not offer all real facts and information that can be
presented in a real environment.
• Advantage of using simulated cases is comparability,
since all participants receive similar information.
Seminar 1 23

24. What does this study adds to existing
knowledge ?
• An assessment tool (STREET) offers a common
understanding of risks prior to an event
• In turn may help prevent disastrous consequences of
identified risks by mutual planning and resource
estimation.
Seminar 1 24

25. Critique (-)
• Reliability in real life scenario ?
• Public health issues &Environmental issues not
covered in detail
• Sampling method not specified in methods.
• Method used for finalizing the items
• Categorization of assessed risk of the event has not
been explained clearly
Seminar 1 25

26. Critique (+)
• Use of Simulations in the form of case studies
provided uniform conditions in order to aid
interpretation.
• Use of simulation is cost effective method
• Involved multiple partners
• Experts from the respective fields were involved
• Gives insight in each organization’s capacity and
limitations
Seminar 1 26

27. References
1. Mass gathering medicine: a descriptive analysis of a range of mass
gathering event types, Samuel Loloch Danou et al, American Journal of
emergency medicine Volume 31 [2013] pages 843-846
2. WHO. Communicable disease alert and response for mass gatherings.
Technical workshop.
Geneva,Switzerland.http://www.who.int/csr/resources/publications/WHO
_HSE_EPR_2008_8c.pdf; April 29–30, 2008.
3. Arbon P The development of conceptual models for mass-gathering
health. Prehosp Disaster Med. 2004 Jul-Sep;19(3):208-12
4. Khorram-Manesh A, Berner A, Hedelin A, Örtenwall P. Estimation of
healthcare resources at sporting events. Prehosp Disaster Med 2010;
25(5): 449-455.
5. Berner A, Khorram-Manesh A, Hedelin A, Örtenwall P. Avoiding disasters,
a tool for estimating the needs of healthcare resources at sporting and
other public events. ICU Management 2011; 3: 38-39
Seminar 1 27

28. Thank you
28Seminar 1