Developing Digital Literacies programme Interim report guidelines Helen Beetham & Jay Dempster October 2012
Some general philosophies/goals• Synthesis and evaluation is in itself part of the change process• Accountability is a growing area of need for funded programmes• Clarity, relevance of key lessons/messages with links to your outputs, rather than needless complexity• Illustrate & animate your findings to inspire/inform• Offer useful, meaningful, actionable evaluation.
Gathering, analysing, reporting outcomes collating & evidencing making sense of & verifyingquantitative ‘deliverables’ qualitative ‘lessons’ (accountability >> (knowledge transfer >> institutions/partners/funders) programme/funders, the HE sector) gathering feedback on project processes, practices and outcomes across the Programme (developmental >> programme team/funders)
Writing the report narrative• 1. Be interesting – this is not just a day job, unexpected, exciting or aggravating things happen, communicate the ‘human’ element, use examples, avoid waffle/needless detail, convey the excitement of change rather than just writing to sub-headings.• 2. Be research-like – investigative as well as pragmatic, micro-theories based on observations, hunches and conversations as well as on findings and solid evidence. Record them. This is what makes projects interesting.• 3. Be communicative – the first audience for reports is the rest of the programme, use them as critical friends, feedback from them is evidence of what is interesting & useful.• 4. Be meaningful – think about what project activities, outcomes and lessons might mean for people beyond your organisation, the wider sector (graphic above might help).• 5. Be opportunistic – look for ways of recording what you need to record that dont take up too much time (iterative reporting, blogging, capturing conversations/outputs, routine monitoring/usage, turning the best bits into an update)• 6. Be pragmatic – in terms of rigour (reliable, valid data/methods determine the quality of the evidence produced)• 7. Be ‘big picture’ esq – related to baseline evidence, seek overarching relevance/value
Checklist questions - 1• Is your evidence facilitating discussion or decision making/ action taking? – What kinds of discussion & feedback is your project generating and how are you recording/capturing this? – How useful is it? (to the work of your project, institutional change, partners/associations engagement/contribution, to students, to the wider sector) – Is your synthesis of findings and evaluation evidence delivering relevant, meaningful, ‘actionable’ insights to your institution, to the programme?
Checklist questions - 2• What kind of outputs are you producing? – What ways are you providing ‘snapshots’ & ‘sense making’ on processes & outcomes synthesised across your project? – Are you tagging topics and key audiences of findings for later synthesis/dissemination? – How are you critiquing the data/evidence you are gathering?
Checklist questions - 3• Does the evidence add up to something? – Is it fit-for-purpose and supporting claims you are making about change and impact? – Are you communicating outcomes/benefits or defending a situation or finding in the project? – How are you filtering/tagging what is valuable and relevant to your project/strategic objectives and stakeholders as you go along? – Are you generating an overall picture of the (emerging) impact of the work?