Slides from JISC Developing Digital

1. 4th October 2011
Maple House, Birmingham
Developing Digital Literacy:
Baselining and evaluation
Helen Beetham, Synthesis Consultant
Jay Dempster, Evaluation Consultant

2. Digital Literacy
Welcome – who we are
Jay Dempster
Evaluation consultant
Helen Beetham
Synthesis consultant

3. Digital Literacy
Session outline
Why baseline?
The programme-level view
The project-level view (1):
How will practice/provision/literacy be developed?
The project-level view (2):
Collecting and making sense of baseline data
Programme tools and resources for baselining

4. Digital Literacy
Why baseline?
At programme level:
Provide snapshot of the sector at start-up
Develop shared understanding
Review what we already know and have achieved
Contextualise programme aims and rationale
At project level:
Provide starting-point for evaluation
Provide clear structure for collecting evidence
Refine objectives and success indicators

5. Digital Literacy
What do we know?
2008 Learners' experiences of e-learning programme
Students' success depends on strategies for integrating ICT into
academic practice; strategies and preferences differ widely
2009 Learning Literacies for a Digital Age study
Digital literacy needs to be integrated across the curriculum: learners
develop through authentic tasks in meaningful situations
2010 Supporting Learners in a Digital Age
Institutions are: addressing skills and expectations at transition; creating
digitally-enabled learning spaces; listening to learners; embedding digital
activities into the curriculum
2011 Digital literacy workshop series
Lessons and recommendations, tools for organisational and curriculum
development; sharing best practice
2011-13 Developing Digital Literacies programme
12 institutions, taking different views across the DL landscape

6. Digital Literacy
Why digital literacy?
Fair access and opportunity
(digital technology reduces some
barriers/inequalities,
can introduce others)
Impacts of digital technology
on practice
(research, teaching, learning,
professionalism, knowledge transfer,
development)
New demands on the sector
(graduate attributes and outcomes,
the learning experience, lifelong
learning)

7. Digital Literacy
Addressing digital literacy...
into higher/further education
(transition, entitlement, access,
student markets, participation)
in
higher/further education
(curriculum development,
learning skills/academic practice)
through higher/further education
(employability, graduate attributes,
the co-curriculum)

8. Digital Literacy
Whose digital literacy?

9. Digital Literacy
How do we conceive of digital literacy?
It will be a big challenge to identify how
digital literacy makes a difference to the
individual in terms of how they think and
what they do.
Joe Nicholls, Digidol
We need to know more about the messy,
day-to-day lived practices of students and
staff.
Lesley Gourlay, Institute of Education

10. Digital Literacy
How do we conceive of digital literacy?
Presentation, Production and
Performance
play, performance, simulation,
appropriation
Collaboration and work skills
multitasking, distributed cognition,
collective intelligence
Participating in information
practices
transmedia navigation, networking,
negotiation
Ryberg and Georgsen (2010), drawing on
Buckingham (2005) and Jenkins (2006)

11. Digital Literacy
At what level?
Discipline/subject
specialist focus
identities 'I am...'
practices 'I do...'
skills 'I can...'
Generic/entitlement
focus
access 'I have...'
(Beetham and Sharpe 2009)

12. Digital Literacy
With what institutional impact?

13. Digital Literacy
With what focus?
(Sharpe et al. 2011)

14. Digital Literacy
Working with which professionals?

15. Digital Literacy
'If we all do it properly, and climb and stay out of our
bunkers, this will be a cross-community project on a
grand and productive scale'
David Baume, SEDA

16. Digital Literacy
Activity in projects
Review some of the different ways of 'developing digital
literacies'
Which have resonance for your project?
What part(s) of the landscape will your project focus on?
Write down three ways in which your project will bring
about development
what will be developed (people, places, practices, processes,
policies, curricula, opportunities...) and how?

17. Digital Literacy
Activity in projects
Review some of the different ways of 'developing digital
literacies'
Which have resonance for your project?
What part of the landscape is your project focusing on?
Write down three ways in which your project will bring
about development, i.e. what will be developed (people,
places, practices, processes, policies, curricula,
opportunities...) and how.
Share your statements with another project, working in
project pairs. Review, compare, critique.

18. Baselining
From the original circular:
“Review current processes and practices to establish a baseline
description of current digital literacy provision to aid in the
evaluation of subsequent innovations.”
This aims to provide the evidence
and ideas to support your project in
planning your interventions.
Baseline report by end Dec/Jan
Many projects are looking to do a more
detailed review by end of year 1.

19. Baselining as...
 stakeholder review for shared/distinct meanings
 contextualising ‘big picture’ evaluation questions
 provide clear structure for collecting evidence
 checking back against the ‘why and how’ of project
achievements

20. ‘Baselining for success’
Combine your 30,000 ft view activity with 'what success looks like‘ ...
 produce a ‘rich picture’ of your project’s intended outcomes
 identify some clear indicators of success for your different stakeholder audiences
 clarify starting points for sources & types of data for your evidence base
 hard metrics e.g. statistics in audit documents, quantitative data collected from
student feedback surveys
 vs. narrative descriptions e.g. example in CDD of a project (T-Sparc) doing an
entire baseline as a series of video clips .... on the Design Studio website at:
http://blogs.test.bcu.ac.uk/tsparc/t-sparc-baseline-review/

21. Baselining data and indicators of success

22. Baselining Digital Literacies
Share your initial ideas/issues in pairs & discuss ...
for instance, you might touch on:
1. Is there a shared sense of what digital literacies mean/are?
2. Based on what/whose evidence?
3. How are you triangulating/cross-checking with stakeholders?
4. What are the assumptions made about the project and its context?
5. Are there existing measures available from prior audits/surveys/initiatives?
6. What descriptors/metrics/rubrics are emerging?
(towards a common focus and common language for how we define and measure
effectiveness & progress in digital literacies)
1. What are the most useful/feasible baseline data/ indicators of success for YOUR
project?

23. Tools & Resources

24. Tools & Resources
Design Studio pages for JISC DLL:
http://jiscdesignstudio.pbworks.com/w/page/46421608
/Developing%20digital%20literacies
http://bit.ly/pHxQnS
Support Wiki pages for Baselining:
https://programmesupport.pbworks.com/w/page/46353821/
Supporting%20tools%20and%20frameworks
http://bit.ly/nfF8Mg

25. Digital Literacy
Support for baselining
Evaluation support
- a starting point for conversations
- DigLit programme wiki
- follow up Elluminate session in November
Jay Dempster
- project support
Synthesis support
- mapping project objectives and success
criteria to programme view
- develop resources on Design Studio
Helen Beetham - work with professional bodies