Evaluating Media Literacy as Competences: What can we agree on?

Evaluating media literacy as competences: what can we agree on?
Pierre Fastrez
Groupe de Recherche en Médiation des Savoirs
Université catholique de Louvain, Belgium

Where do I stand? (a few oratoricalprecautions)
Pierre Fastrez [GReMS/UCL]
2
What I intend to bring to the table
a social/cognitive/communication scientist’s point of view
a newcomer’s perspective
EuroMeducBellaria – Oct. 22nd, 2009

A simple observation…
EuroMeduc Bellaria – Oct. 22nd, 2009
3
Media Education, a vivid field
Media Literacy evaluation
is still in its infancy
still lacks systematic efforts to make it a coherent endeavour

Assessing ML : what priorities?
4
Agreement:
Regarding theoretical definitions of Media Literacy
Not on definitions and frameworks
But rather on how these definitions and frameworks relate
Regarding methodological principles for assessment

Outline
5
Assessing Media Literacy: what can we agree on?
Theoretical issues
Methodological issues
A few proposals regarding Media Literacy competence assessment

Theoretical issues
6
Assessing ML requires an agreement on what it is
Defining ML is dependent on
Historical and cultural context
Media landscape development
Policies
Epistemological views

Theoretical issues
7
Goal: identify common ground for Media Literacy definitions
Independent of cultural contexts
That can be specified and interpreted in these contexts
Media Literacy lies in the individual’s competences
not the media environment: media literacy is not equivalent to media education
Outcome vs. process
Evaluating the process is evaluating its efficiency with respect to the outcome
Benchmarking outcomes is a sine qua non condition

Theoretical issues
8

Theoretical issues
9
A fundamental hypothesis:
Media Literacy is related to a number of generic skills
E.g. media and communication technologies as mindtools
These skills are generic in that they are not related to
specific media (film, print, internet…)
specific activity domains
E.g. commercial information, product placement, minor protection, civic participation…
A bi-lateral relationship (virtuous circle)
These skills can be developed through specific media use and specific media education practices
The more these skills are developed, the more they facilitate other media appropriation processes

Theoretical issues
10
Consequence of this hypothesis: we need to relate specific skills to their “common ground”
E.g. “digital” skills are not of a different nature than “traditional media” skills
E.g. Film studies vs. media education
appreciating one’s media cultural heritage relies on the same skills as appreciating other cultures’ contribution to the media environment
E.g. Understanding the semiotic specificities of a given media:
Representing time in blogs and twitter (input flow and chronology)
Representing time on the silver screen (editing and rhythm)
E.g. Building typologies: reading the news and managing bookmarks
E.g. Perceiving the producer’s intent: watching a documentary and moderating a forum

Methodological principles
11
Methodological issues regarding ML evaluation
Definition of levels
Competence and performance
Assessment tools validation process

Levels of Media Literacy
12
There are degrees of complexity in competences
Operationalization: Levels (when it makes sense)
Tools for researchers
Not meant for unilateral ranking
Which would assume one can weigh the relative importance of skills
Rather, a multidimensional analysis tool

Competence vs. performance
13
Competences are virtual, and can only be observed when actualized in practical and concrete tasks
Evaluating competence through performance
Consequences:
The production vs. reception distinction should be set (for now) at the performance level (vs. as categories of skills)
Performances are connected to different competences
E. g. moderating a forum: social and cognitive competences

From concept to indicators
14

From concept to indicators
15

2. An attempt at breaking down the Media Literacy concept
Cognitive vs. social(vs. technical) skills
Semio-cognitive vs. socio-cognitive skills
Conceptual distinctions independent of specific activity fields
16

17

18

Semio-cognitive skills
19
Dimensions (inductive grouping) :
Reading (decoding / synthesizing / evaluating)
Writing (coding, creating, producing)
Navigating (searching, exploring, managing uncertainty)
Organizing (keeping, sorting, annotating)
Using responsibly (knowledge of use regulation, stakes and principles)

Socio-cognitive skills
20
Pragmatic socialization: The extent to which an individual poses communicational acts that establish socialrelationships in more or less extended social spaces.
Informational socialization: The extent to which each individual extends and diversifies their relationships as a receiver, a relay, or a producerof information within social space.

21
Social levels logic (criteria, not thresholds)
From contact and participation, to networking and active presence, to cooperative networking
Diversity (from tribe to society)
Creativity and involvement (from follower to autonomous agent)
Decentration (from egocentric to allocentric)
Dimensions:
Reception: media selection and exploration
Expression: communicative acts, idioms and receivers
Pragmatics: relational contexts and interactions
Ethics: perceived rules and lines of conduct

Are informational socialization skills only socio-cognitive?
22

Theoretical questions left open
23
How are Media Literacy skills correlated?
From an analytical to a systemic point of view: connecting after having distinguished
E.g. does critical thinking development facilitate creativity?
Ethics as a socio-cognitive skill and responsible use as a semio-cognitive skill
…
Need to work on the structure of ML as a concept federating a set of competences
Not just conceptually, but empirically

Back to methodology: validation
24
Building relevant ML assessment tools takes time and a lot of research
ML dimensions (skills) can (and must) be translated into multiple indicators related to different media / activity domains
Validation requires
Multiple indicators for the same skill
Internal consistency evaluation

Back to methodology: validation
25
Data / indicators based on (roughly in order of preference)
Observation of specific tasks
Experiments
Participant observation
Analysis of traces
Personal environments / personal information spaces
Including statistics
Interviews and surveys
Self-assessment tools
Triangulation: a requirement for validation

Conclusions and implications
26
Working on competence definitions and their operationalization is a requirement for ML assessment.
Constructing ML assessment tools are a means for setting objectives for media educators
bridging the gap between
asymptotic goals (“be an active citizen and media user”)
and practical know-hows (“know how to frame a close shot”)

Thank you for your attention
Questions? Comments?
pierre.fastrez@uclouvain.be