A step-by-step guide to developing a national Report Card on the physical activity of children and youth

1. Report Card – 101
A step-by-step guide to developing a national
Report Card on the physical activity of
children and youth
Before reviewing this slide deck it is recommended that you read the
background paper: Colley RC, Brownrigg M, Tremblay MS. The Active
Healthy Kids Canada Report Card on Physical Activity for Children and
Youth. Health Promotion Practice 13(3):320-330, 2012.

2. So you want to lead a Report Card in your
country?
The Report Card is a large undertaking. To
achieve the desired quality and impact, it is
important that the Report Card is not left
until the last minute. Before you begin, ask
yourself, Can you commit the resources
(human and financial) needed to see a
Report Card through to completion?

3. What is the Report Card?
• The Report Card is a synthesis of the most
recently available published or unpublished
data related to the physical activity of children
and youth in a nation (new data collection is
not required or expected).
• The Report Card is an evidence-based
communications and advocacy tool.

4. What isn’t the Report Card?
• The Report Card is not a research study.
• It is commonly misreported in the media that the
Report Card is a new national survey. While new
analyses of existing survey data may be
commissioned, the Report Card is a synthesis of
the best available existing research.

5. Purpose of the Report Card
• To advance knowledge about how a country
is being responsible in providing physical
activity opportunities for children and youth.
• To influence issue stakeholders who affect
physical activity opportunities for children and
youth.

6. AHKC Report Card Framework
• The AHKC Report Card
Model assesses the
behaviours that contribute
to overall physical activity
and sedentary behaviour.
• The Model also assesses the
contextual factors (e.g.,
strategies, investments,
settings) that influence
these behaviours.

7. AHKC Report Card Framework
• In this version 9 indicators
are grouped into 3
categories to reflect their
relationships with a child’s
physical activity level.
• Physical activity level affects
outcomes (e.g., happiness
and fitness) that can affect
the child’s overall physical
activity levels.

8. Content Development Process
Through the model we:
• Amass the best possible evidence;
• Identify gaps in the existing knowledge;
• Provide recommendations for what we
can do to improve.

9. How to do a Report Card
(allow 8-12 months for development and production)

10. Annual Work Cycle to Support the Development and Production of a Physical Activity Report Card
Month
Annual Cycle of Activities 1 2 3 4 5 6 7 8 9 10 11 12+
Identify Research Work Group, Project Manager ,
Principal Investigator
Indicator Meeting, form writing team
Comprehensive literature reviews in key topic areas
Unique data analyses submitted to PI by RWG
Summary of evidence document produced
Grade Assignment Meeting
Additional research to address gaps identified in
grading
Identify design team, copyeditor, and translator
Production of draft 1, review, production of draft 2,
review
Coordinate design and layout of RC
Coordinate copy editing, translation, and printing of
RC
RC launch, release, and dissemination
Internal and/or external evaluation strategy
development and execution
Communication and media strategy, issue advocacy

11. Step 1
ASSEMBLE YOUR TEAM

12. The Report Card Team
• Invite leading national researchers and policy
experts to join the Research Work Group
(RWG)
• Identify a Principal Investigator (PI)
• Hire a Project Manager/Lead Writer
• Identify the design team, copy-editor,
translator (if required), web-designer

13. How to Choose a RWG?
• Recognized experts in a relevant field
• Can provide access to survey data
• Can commit to participating in the Indicator
Meeting and the Grade Assignment Meeting
in person or by connecting remotely.
• Also consider, are you the PI or will the RWG
choose a PI as a group?

14. The Development Team
• Will the PI be responsible for project
management and writing, or will someone be
hired for these roles?
– Project Management and writing will require a
large time commitment. You may wish to involve a
graduate student or hire a project manager.
– Inviting students to write sections allows them to
gain experience in knowledge translation.

15. Production Team
• Efforts to identify the production team should
begin as soon as the RWG and Development
Team are in place
– Allows time to brainstorm themes and cover story
ideas.
– Prevents last minute struggle to find the right
people.

16. Communications & Media Specialist (CMS)
• A key purpose of producing the Report Card is
to inform policy makers, practitioners,
educations, and the media.
• Early identification of a CMS is important to
ensure:
– Relevant groups are engaged & informed
– Key findings are effectively disseminated across
the country

17. Step 2
GETTING STARTED

18. Literature Search
• The search for content should begin
immediately upon identifying your Lead
Writer.
– The amount of research available will vary widely
across countries.
– RWG members should forward materials
proactively.

19. Literature Synthesis
• Don’t wait until the last minute
– Countries with a strong research record should
allow greater time and resources for literature
search and writing.
• Consider your audience
– The Report Card should be written in an engaging
and jargon-free manner appropriate for media,
government, and practitioner audiences.

20. Step 3
THE INDICATOR MEETING

21. Objectives
• Finalize the list of indicators to be graded
• The 9 pre-specified indicators were required to
participate in the Global Matrix 1.0, although others of
importance in your country can be included. See past
Report Cards for examples. [Archive link]
• Agree on the data sets that will be used to
inform the grades.
• Agree on the benchmarks/optimal scenarios
indicators will be graded against.

22. Preparation: Summary of Data Sources
• The Project Manager prepares and provides a
‘Summary of Data Sources’ document to the PI
and RWG in advance of the meeting
• The summary should include all recently available
(this may mean within the past 5 years, depending
on data availability) national and regional surveys
related to each indicator.

23. Summary of Data Sources Document (example)
Overall Physical Activity Levels
CHMS (2010 – 2011) PAM (2010 – 2011)
Nationally representative sample of children and
youth
Most provinces and territories included, 8
– 12 year olds
Accelerometry How many days last week did your child
do physical activities for a total of at least
60 minutes per day?
During the past 7 days, on how many days were
you physically active for a total of at least 60
minutes per day? Add up all the time you spent
in any kind of PA that increased your heart rate
and made you breathe hard some of the time.
(0-7 days)
During the past 7 days, did you do any physical
activities? (not including PE or gym class)
Indicator
Measures
Survey name
& year(s)
collected
Important details
about the survey

24. Preparation: Proposed
Benchmarks
• The PI and Project Manager propose benchmarks for
each indicator to be circulated to the RWG before
the meeting. This would be harmonized among
countries in the case of the Global Matrix.
• Benchmarks should be based on evidence-informed
guidelines where possible.
– E.g., “the percentage of children and youth who
meet the Canadian Physical Activity Guidelines”

25. Indicator Meeting
• Indicators
– RWG decides if other indicators will be
included
• Summary of Data Sources
– RWG discusses and assesses completeness
– RWG identifies and provides details for
additional sources if any were missed
– Data gaps are noted
• Benchmarks
– RWG discusses and finalizes benchmarks

26. Decisions You May Face
• How far back in time will you cover?
• How often will you release a Report Card?
• Will you include other indicators?
– If you grade obesity it is likely that media will
only focus on obesity, at the expense of physical
activity
• How to assign benchmarks to indicators without
evidence-based guidelines?
– E.g., active play

27. Stage 4
GRADE ASSIGNMENT MEETING

28. Objectives
• Discuss and agree on grades for each indicator
– Note and record rationale for grade
• Decide if new analyses are needed
– New analyses may be commissioned if there are
existing data available

29. Grade Assignment Methodology
Grades are assigned based on examination of the
current data and literature for each indicator
against a benchmark or optimal scenario:
A (81-100%) = We are succeeding with a large majority of children
B (61 - 80%) = We are succeeding with well over half of children
C (41 - 60%) = We are succeeding with about half of children
D (21 - 40%) = We are succeeding with less than half, but some, children
F (00 - 20%) = We are succeeding with very few children

30. Other Considerations: Time Trends
• Grades may be graded up (+) or down (-)
based on trends over time
– E.g., if less than half of kids are achieving the benchmark,
but longitudinal data show there have been improvements
over time, you may wish to recognize the improvement by
changing a D to a D+

31. Other Considerations: Disparities
• Grades may be graded down (-) based on the
presence of disparities such as age, gender,
disability, ethnicity or SES, or up (+) if there is
evidence past disparities are disappearing
– E.g., if well over half of kids are achieving the benchmark,
but sex-based analyses show that girls are doing worse
than boys, you may wish to change a B to a B-

32. Follow-Up
• Additional research may be gathered to
address gaps identified during Grade
Assignment Meeting.
• Final grades are confirmed by email or by
teleconference if further discussion is needed.

33. Stage 5
PRODUCTION

34. Production Tasks to be Completed
1. Produce long form draft
2. Review long form draft
3. Produce long form draft
2
4. Review long form draft
2
5. Produce short form
6. Coordinate design and
layout of Report Card
7. Coordinate copyediting
8. Translate long and
short form final
versions (if required)
9. Coordinate printing
10. Coordinate website
production, support
tools, social media

35. Cover Story
• The media are instrumental in increasing the
reach and impact of the Report Card.
• A cover story enables you to gain media
attention and to shape the message.
• Rely on leadership from the CMS in selecting
an attention-grabbing cover story.
• See Archive for past examples. [Archive link]

36. Communications and Media Strategy
• Plan the release strategy (event, media release)
• Establish network of provincial/ territorial/ state
partners to support the release.
– Distribute Report Card through their networks
– Provide media spokespeople to respond to
ongoing media requests
• Develop post-Report Card release
communications.

37. Issue Advocacy
• Target government and NGO Officials
whose work in policy and programming
impacts physical activity opportunities for
children and youth.
• Conduct strategic industry meetings.
• Present at local, national, and international
conferences and meetings.

38. Evaluation
• Options for evaluation include:
– Report Card dissemination and media uptake.
– Stakeholder feedback (e.g., Is the Report Card
meeting stakeholder needs?).
– Process evaluation (i.e., Inputs, activities, outputs).
– Evaluation of immediate, intermediate, and long-term
outcomes (e.g., Is the Report Card being used
to increase awareness about physical activity in
children and youth?).

39. Register Your Country
• To initiate the Report Card process in your
country and/or to participate in the Global
Matrix 2.0 visit
http://www.activehealthykids.org/ and
register your country.