4th October 2011 Maple House, BirminghamDeveloping Digital Literacy:Baselining and evaluationHelen Beetham, Synthesis ConsultantJay Dempster, Evaluation Consultant
Digital Literacy Welcome – who we are Jay Dempster Evaluation consultant Helen Beetham Synthesis consultant
Digital Literacy Session outlineWhy baseline?The programme-level viewThe project-level view (1):How will practice/provision/literacy be developed?The project-level view (2):Collecting and making sense of baseline dataProgramme tools and resources for baselining
Digital Literacy Why baseline?At programme level:Provide snapshot of the sector at start-upDevelop shared understandingReview what we already know and have achievedContextualise programme aims and rationaleAt project level:Provide starting-point for evaluationProvide clear structure for collecting evidenceRefine objectives and success indicators
Digital Literacy What do we know?2008 Learners experiences of e-learning programmeStudents success depends on strategies for integrating ICT intoacademic practice; strategies and preferences differ widely2009 Learning Literacies for a Digital Age studyDigital literacy needs to be integrated across the curriculum: learnersdevelop through authentic tasks in meaningful situations2010 Supporting Learners in a Digital AgeInstitutions are: addressing skills and expectations at transition; creatingdigitally-enabled learning spaces; listening to learners; embedding digitalactivities into the curriculum2011 Digital literacy workshop seriesLessons and recommendations, tools for organisational and curriculumdevelopment; sharing best practice2011-13 Developing Digital Literacies programme12 institutions, taking different views across the DL landscape
Digital Literacy Why digital literacy?Fair access and opportunity(digital technology reduces some barriers/inequalities, can introduce others) Impacts of digital technology on practice (research, teaching, learning, professionalism, knowledge transfer, development) New demands on the sector (graduate attributes and outcomes, the learning experience, lifelong learning)
Digital Literacy Addressing digital literacy...into higher/further education(transition, entitlement, access, student markets, participation) in higher/further education (curriculum development, learning skills/academic practice) through higher/further education (employability, graduate attributes, the co-curriculum)
Digital Literacy How do we conceive of digital literacy?It will be a big challenge to identify howdigital literacy makes a difference to theindividual in terms of how they think andwhat they do.Joe Nicholls, DigidolWe need to know more about the messy,day-to-day lived practices of students andstaff.Lesley Gourlay, Institute of Education
Digital Literacy How do we conceive of digital literacy? Presentation, Production and Performance play, performance, simulation, appropriation Collaboration and work skills multitasking, distributed cognition, collective intelligence Participating in information practices transmedia navigation, networking, negotiation Ryberg and Georgsen (2010), drawing on Buckingham (2005) and Jenkins (2006)
Digital Literacy At what level?Discipline/subjectspecialist focus identities I am... practices I do... skills I can...Generic/entitlementfocus access I have... (Beetham and Sharpe 2009)
Digital Literacy With what institutional impact?
Digital Literacy With what focus? (Sharpe et al. 2011)
Digital Literacy Working with which professionals?
Digital LiteracyIf we all do it properly, and climb and stay out of ourbunkers, this will be a cross-community project on a grand and productive scale David Baume, SEDA
Digital Literacy Activity in projectsReview some of the different ways of developing digitalliteraciesWhich have resonance for your project?What part(s) of the landscape will your project focus on?Write down three ways in which your project will bringabout developmentwhat will be developed (people, places, practices, processes,policies, curricula, opportunities...) and how?
Digital Literacy Activity in projectsReview some of the different ways of developing digitalliteraciesWhich have resonance for your project?What part of the landscape is your project focusing on?Write down three ways in which your project will bringabout development, i.e. what will be developed (people,places, practices, processes, policies, curricula,opportunities...) and how.Share your statements with another project, working inproject pairs. Review, compare, critique.
BaseliningFrom the original circular:“Review current processes and practices to establish a baselinedescription of current digital literacy provision to aid in theevaluation of subsequent innovations.” This aims to provide the evidence and ideas to support your project in planning your interventions. Baseline report by end Dec/Jan Many projects are looking to do a more detailed review by end of year 1.
Baselining as... stakeholder review for shared/distinct meanings contextualising ‘big picture’ evaluation questions provide clear structure for collecting evidence checking back against the ‘why and how’ of project achievements
‘Baselining for success’Combine your 30,000 ft view activity with what success looks like‘ ... produce a ‘rich picture’ of your project’s intended outcomes identify some clear indicators of success for your different stakeholder audiences clarify starting points for sources & types of data for your evidence base hard metrics e.g. statistics in audit documents, quantitative data collected from student feedback surveys vs. narrative descriptions e.g. example in CDD of a project (T-Sparc) doing an entire baseline as a series of video clips .... on the Design Studio website at: http://blogs.test.bcu.ac.uk/tsparc/t-sparc-baseline-review/
Baselining Digital LiteraciesShare your initial ideas/issues in pairs & discuss ...for instance, you might touch on:1. Is there a shared sense of what digital literacies mean/are?2. Based on what/whose evidence?3. How are you triangulating/cross-checking with stakeholders?4. What are the assumptions made about the project and its context?5. Are there existing measures available from prior audits/surveys/initiatives?6. What descriptors/metrics/rubrics are emerging? (towards a common focus and common language for how we define and measure effectiveness & progress in digital literacies)1. What are the most useful/feasible baseline data/ indicators of success for YOUR project?
Tools & ResourcesDesign Studio pages for JISC DLL:http://jiscdesignstudio.pbworks.com/w/page/46421608/Developing%20digital%20literacies http://bit.ly/pHxQnSSupport Wiki pages for Baselining:https://programmesupport.pbworks.com/w/page/46353821/Supporting%20tools%20and%20frameworks http://bit.ly/nfF8Mg
Digital Literacy Support for baselining Evaluation support - a starting point for conversations - DigLit programme wiki - follow up Elluminate session in NovemberJay Dempster - project support Synthesis support - mapping project objectives and success criteria to programme view - develop resources on Design StudioHelen Beetham - work with professional bodies