DELIVERABLEProject Acronym:                DigCurVProject Number:                 510023Project Title:                  Di...
D2.1 Report on baseline survey and evaluation framework                               Section 2: Evaluation frameworkConte...
D2.1 Report on baseline survey and evaluation framework                             Section 2: Evaluation framework1.     ...
D2.1 Report on baseline survey and evaluation framework                        Section 2: Evaluation framework1.3.   Imple...
D2.1 Report on baseline survey and evaluation framework                        Section 2: Evaluation frameworkArea 1: Know...
D2.1 Report on baseline survey and evaluation framework                        Section 2: Evaluation framework           o...
D2.1 Report on baseline survey and evaluation framework                        Section 2: Evaluation framework          o ...
D2.1 Report on baseline survey and evaluation framework                        Section 2: Evaluation frameworkArea 2: Skil...
D2.1 Report on baseline survey and evaluation framework                        Section 2: Evaluation framework            ...
D2.1 Report on baseline survey and evaluation framework                        Section 2: Evaluation framework            ...
D2.1 Report on baseline survey and evaluation framework                        Section 2: Evaluation framework            ...
D2.1 Report on baseline survey and evaluation framework                        Section 2: Evaluation frameworkArea 3: Audi...
D2.1 Report on baseline survey and evaluation framework                        Section 2: Evaluation framework            ...
D2.1 Report on baseline survey and evaluation framework                        Section 2: Evaluation frameworkData directo...
D2.1 Report on baseline survey and evaluation framework                           Section 2: Evaluation framework         ...
D2.1 Report on baseline survey and evaluation framework                        Section 2: Evaluation framework            ...
D2.1 Report on baseline survey and evaluation framework                         Section 2: Evaluation frameworkArea 4: Par...
D2.1 Report on baseline survey and evaluation framework                        Section 2: Evaluation frameworkArea 5: Teac...
D2.1 Report on baseline survey and evaluation framework                          Section 2: Evaluation frameworkArea 6: Pr...
Upcoming SlideShare
Loading in …5
×

DigCurV 3 of 4. Evaluation framework: Report on baseline survey into training opportunities and evaluation framework

744 views
607 views

Published on

Other documents included in this report are:
- Executive summary
- Survey of existing training opportunities
- Conclusions and Recommendations

http://www.digcur-education.org/eng/Resources

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
744
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

DigCurV 3 of 4. Evaluation framework: Report on baseline survey into training opportunities and evaluation framework

  1. 1. DELIVERABLEProject Acronym: DigCurVProject Number: 510023Project Title: Digital Curator Vocational Education EuropeDissemination Level: Public D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation frameworkAuthors: Vilma Karvelyte, Nijole Klingaite, Jurate Kupriene (Vilnius University Library) Laura Molloy, Kellie Snow, Ann Gow (HATII)
  2. 2. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation frameworkContents1. DigCurV Evaluation Framework ....................................................................................... 3 1.1. Background ................................................................................................................ 3 1.2. Structure and usage .................................................................................................. 3 1.3. Implementing the evaluation framework when designing a curriculum .................... 4 Area 1: Knowledge and principles .................................................................................... 5 Area 2: Skills and competences ....................................................................................... 8 Area 3: Audience/profile types........................................................................................ 12 Area 4: Part of digital curation lifecycle .......................................................................... 17 Area 5: Teaching methods/training delivery ................................................................... 18 Area 6: Professional context........................................................................................... 19Revision History Revision Date Author Organisation Description 1.0 30/01/12 V Karvelyte VUL Final report N Klingaite L Molloy HATII Statement of originality: This deliverable contains original unpublished work except where clearly indicated otherwise. Acknowledgement of previously published material and of the work of others has been made through appropriate citation, quotation or both.http://www.digcur-education.org Page 2 of 19
  3. 3. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation framework1. DigCurV Evaluation Framework1.1. BackgroundThe DigCurV Evaluation Framework builds on various pieces of previous work in digitalcuration curriculum design and evaluation framework development. The layout is based onthe matrix format of the DigCCurr Matrix of Digital Curation Competencies and Knowledge(Version 13, 17 June 20091). Other models drawn upon include the Digital PreservationOutreach and Education initiative’s training audiences pyramid2, the Research InformationNetwork’s Researcher Development Framework – taxonomy for information literacy, Version33, and the Digital Curation Centre lifecycle model4.The structures of, and approaches to, the Evaluation Framework have been suggested bythese models. The content of the Evaluation Framework has been supplied by the findingsfrom the training opportunities survey (section 4 above) and a companion survey by DigCurVof training needs. Together they show the various aspects that need to be considered foreach part of any digital curation curriculum which is being developed or being assessed.The DigCurV project aims to develop a Curriculum for vocational training in digital curationand this Evaluation Framework provides groundwork for this development. The DigCurVCurriculum will indicate core digital curation competences and pathways of skills progressionthrough these. The Evaluation Framework described here will be of practical use in thisdevelopment work by being applicable to existing curricula, and by acting as a critical toolapplied to progressive iterations of the Core Curriculum, to aid its development.1.2. Structure and usageThe Evaluation Framework is intended to be helpful to those designing, providing orassessing digital curation curricula (or individual pieces of training which may form part of acurriculum). The Evaluation Framework provides a series of different ways to view andevaluate a digital curation curriculum or piece of training. These different ways are describedas Areas and numbered from 1 to 6. Taking a structured approach to consideration of acurriculum or piece of training can help those developing training to assess what is alreadyavailable, and to clarify which potential approaches, audiences and skills may need to beaddressed. For those assessing training, the Evaluation Framework provides a structure towhich training offerings can be mapped. This serves to clarify where provision is ample andwhich approaches, audiences or skills are scarcely served in existing training. Mapping canalso provide a benchmark to allow comparison of different training offerings against eachother.1 http://www.ils.unc.edu/digccurr/digccurr-matrix.html2 http://www.digitalpreservation.gov/education/documents/DPOE_handout.pdf3 http://www.rin.ac.uk/system/files/attachments/RDF_taxonomy_for_IL_-_v3.pdf4 http://www.dcc.ac.uk/resources/curation-lifecycle-modelhttp://www.digcur-education.org Page 3 of 19
  4. 4. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation framework1.3. Implementing the evaluation framework when designing a curriculumTo get the fullest benefit from the Evaluation Framework, those developing or assessing acurriculum should consider each Area of the Evaluation Framework in turn, and check that atleast one category from each Area fits to each part of the curriculum they are examining.This will ensure that each aspect of the curriculum has been carefully considered and can bejustified from a variety of viewpoints.For example, a particular training module might be identified by the training provider orassessor as corresponding to the following:  Area 1: Knowledge and principles: Repositories/management systems  Area 2: Skills and competences: Knowledge base, subject knowledge, practical level  Area 3: Profile types: Library Manager, Data Manager  Area 4: Part of lifecycle: Ingest  Area 5: Teaching method(s): Lecture with one small group exercise  Area 6: Disciplinary context: Digital art conservation; Institutional or organisational context: National library of art.(Additional contextual information may be gathered, where relevant and available, in Area 6,‘Professional context’.)This analysis would allow training providers to group modules or parts of the curriculumbeing examined into clear areas; this allows users of the curriculum to pick and chooseaspects relevant to them based on their requirements from each Area of the Framework.http://www.digcur-education.org Page 4 of 19
  5. 5. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation frameworkArea 1: Knowledge and principlesArea 1: Knowledge and principlesThis area is taken from topics identified in the training opportunities registry and DigCurV survey work, and lists the knowledge and principles requiredfor digital curation, which could be reasonably expected by an employer recruiting to a post in this field. Clearly different job roles will require differentknowledge: the aim of this list is to establish the core knowledge and principles currently used in digital curation.  Access and reuse. Examples include: o Providing controlled access to digital assets o Encouraging reuse of digital assets  Audio/audiovisual material. Examples include: o Specific requirements and techniques for successful curation of digital audio and audiovisual objects  Audit and certification. Examples include: o Trusted digital repositories o National and international standards  Digital archives. Examples include: o Principles and planning for successful set-up and management of a digital archive o Digital collection/content management. Examples include: o Planning o Content management software selection o Preservation workflows o Collection policy o Ingest o Disposal  Digital curation lifecycle. Examples include: o Digital curation centre lifecycle model o Applying a lifecycle model to the institution  Digital images. Examples include: o Specific requirements and techniques for successful curation of digital images  Digital preservation workflows.  Digitisation. Examples include: o Management of digitisation projectshttp://www.digcur-education.org Page 5 of 19
  6. 6. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation framework o Workflow o Significant properties and parameters  Electronic records management. Examples include: o Legal requirements including freedom of information legislation o Providing controlled access to electronic records o Version control  File formats. Examples include: o File format identification, o Characteristics and best use of various file formats, o File format selection for digital preservation o Identifying significant object properties  Legal. Examples include: o Intellectual property rights o Licensing o Copyright o Freedom of Information  Metadata. Examples include: o Standards o Use o OAIS model  Policy. Examples include: o Institutional policy o Funder requirements  Preservation approaches. Examples include: o Migration o Emulation  Preservation planning.  Project management for digital preservation. Examples include: o Fundraising o Planning o Monitoring o Evaluation  Repositories/management systems. Examples include:http://www.digcur-education.org Page 6 of 19
  7. 7. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation framework o Software system characteristics and selection o Best practice o Embedding in institution  Research data management. Examples include: o Good research practice in data gathering and storage o Secure storage o Data sharing o Institutional policy o Funder requirements o Informed consent and rights management  Risk management. Examples include: o Risks to longevity of digital holdings o Legal challenges to ownership / right of access o Disaster recovery planning  Security. Examples include: o Secure storage and transit of digital assets o Controlled access to digital assets  Tools of a project/organisation. Examples include: o DAF o AIDA o PLATO o Planets Testbed o DRAMBORA  Web archiving. Examples include: o Workflow o Scoping o Storage o Legal issueshttp://www.digcur-education.org Page 7 of 19
  8. 8. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation frameworkArea 2: Skills and competencesThis area is based on the RIN Researcher Development Framework Information Literacy Taxonomy v3, published by RIN, May 2011, and populatedwith results of training needs survey and focus group findings. The ‘Skills Area’ and ‘Descriptor’ columns are those in the RIN Taxonomy which areapplicable to digital curation. We are considering how practical, managerial and executive roles in digital curation map to each Descriptor. Theseskills encompass not just technical knowledge and duties as covered in Area 1, but widen out to encompass personal attributes, attitudes andbehaviours, further helping to define the approaches that a curriculum should encourage in individuals to shape them for success in digital curationprofessions.Skills Area Descriptor Practical Managerial Executive(domains, sub- (RDF and RIN (from DPOE audiences pyramid) (from DPOE audiences pyramid) (from DPOE audiencesdomains based taxonomy) pyramid)on RDFarrangement)Knowledge base Subject Delivers firm knowledge of Provides awareness of subjects in Provides broad overview of knowledge subjects that data/information which practitioner team are subjects in which institutional relates to and awareness of new currently working with data from. data curation activity is developments in field. Covers main principles or features currently involved. of these subjects. Raises Raises awareness of main awareness of main sources of sources of further information further information (people, (people, institutions). institutions).Cognitive Evaluating Teaches ability to evaluate if item Delivers ability to prioritise Supports ability to identifyabilities is required for preservation and to collections in terms of their value to institutional priorities for determine most appropriate institution and the current level of funding and select requests for curation actions to be performed. risk facing them, to make case for funding accordingly. funding of preservation activity.Creativity Inquiring mind Encourages engagement with Encourages awareness of, and Engenders interest in new or developments in both appetite for, new or innovative innovative processes or subject/domain and in digital practices in subject/domain and in practices employed elsewhere curation; supports continuously digital curation practice that could in the sector. seeking new developments in improve efficiency of workflows.http://www.digcur-education.org Page 8 of 19
  9. 9. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation framework both.Creativity Innovation Stimulates willingness and ability Stimulates willingness to find out Confirms value of new to use new information to inform more about new developments in developments in sector and curation practice when available. curation practice and consider their encourages deployment when deployment. they can improve institutional performance / profile.Personal Integrity Teaches understanding of good Encourages expectations of high Encourages leadership inqualities research practice and recognition quality performance from quality and transparency of of malpractice e.g. fraud or practitioner team and responsibility work and decision-making. plagiarism in datasets. for effectiveness and quality of Encourages setting and curation/preservation function. application of high standards to Supports transparent decision- own work and attention to detail making. when engaged in digital curation activity.Self- Responsiveness Encourages awareness of new / Encourages finding out more about Provides a sense of newmanagement to change emerging developments in field new developments in curation directions or moods in the field and their practical applications to practice and considering their or sector. digital curation practice; and deployment. Encourages recognition of uptake of new methods where value of new developments in appropriate. sector and willingness to authorise deployment when they can improve institutional performance / profile.Professional Ethics, Delivers understanding of need to Provides understanding of need to Presents ethical andconduct principles and manage, share and curate manage, share and curate sustainable work practices as sustainability information/data ethically; information/ data ethically. central to business of awareness of rights of data Ensures familiarity with ethical institution. Supports ability to creators, depositors and any practice. Delivers ability to initiate/approve institutional named parties in curated data/ implement workflow to manage policy development information. rights of those connected to accordingly. Highlights importance of data/information. Sets expectation to take public familiarisation with relevant Supports contribution to useful and responsibility for failures in institutional policies. applicable institutional policy these areas.http://www.digcur-education.org Page 9 of 19
  10. 10. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation framework development informed by realistic view of practical implementation.Professional Legal Provides understanding of Provides understanding of Clarifies organisationalconduct requirements obligations/ requirements from obligations/ requirements from responsibilities under relevant relevant legislation such as data relevant legislation such as data legislation and role of senior protection and freedom of protection and freedom of staff to ensure application in Information, and the concomitant Information, and the concomitant their areas of responsibility. actions to apply in practice. actions to apply in practice. Supports ability to Highlights importance of Supports useful and applicable initiate/approve sound familiarisation with relevant institutional policy development institutional policy institutional policies. informed by realistic view of development. practical implementation. Sets expectation to take public responsibility for failures in these areas.Professional IPR and Delivers understanding of Delivers understanding of Explains legal principles ofconduct copyright information/data ownership, and information/data ownership, and by information/data ownership, by extension the implications of extension the implications of and by extension the copyright and licensing. Provides copyright and licensing. implications of copyright and good practice in handling Supports ability to contribute to licensing. Supports ability to copyright throughout preservation useful and applicable institutional initiate/approve sound / curation activity to minimise policy development informed by institutional policy. unnecessary infringement. realistic view of practical Highlights importance of implementation. familiarisation with relevant institutional policies.Professional Attribution and Explains relevance Supports leadership of practitioner Supports ability to provideconduct co-authorship of citation and bibliometrics, and team in familiarity with importance leadership in best practice in the importance of ensuring that, of data /information being curated this area. where possible, data / information in such as way as to be findable, Sets expectation to take public is findable, understandable and understandable and citable. responsibility for failures in citable. Supports development of curation / these areas. preservation strategies and workflows to ensure such data remains findable, understandablehttp://www.digcur-education.org Page 10 of 19
  11. 11. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation framework and citable.Research Project planning Teaches planning for start of a Teaches effective project Supports ability to providemanagement and delivery project re. how information/data management through the setting of definition of institutional will be managed. Delivers ability team goals, intermediate objectives, milestones and to identify and communicate data milestones and prioritisation of priorities. management requirements to activities. other stakeholders. Teaches how to provide wider public access to and long-term preservation of data.Research Risk Delivers ability to assess risk level Delivers ability to assess risk level Provides expertise onmanagement management currently faced by item and currently faced by collections and organisational risk capture and identify necessary steps to prioritise curation/preservation analysis, prioritisation and risk mitigate. action across collections. management measures.Communication Delivers ability to articulate extent Engenders responsibility for Supports consistent promotionand of knowledge both to peers and to dissemination of curated/ of excellence in institutionaldissemination other staff groups. preserved information/data and practice across the sector and collections. to other stakeholders.http://www.digcur-education.org Page 11 of 19
  12. 12. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation frameworkArea 3: Audience/profile typesThese are being identified by through the findings of the DigCurV survey of training needs, focus groups and by analysis of job descriptions that arebeing collected by the project and, where appropriate, the findings from the JISC/RIN/DCC Data Management Skills Support Initiative (DaMSSI).This Area lists the key duties of each identified Job Profile, and maps the roles to the DPOE audience levels. This is useful for assessment ofcurricula as well as to help training developers ensure they are meeting the training needs of established and emerging roles in digital curation.Job profile Knowledge, skills and competences DPOE audience levelCEO ExecutiveLibrary Manager ManagerialData Manager Analysis Practical Collaboration Communication Computer literacy Current trends in field Influencing and persuading Organising Planning Project management Relevant legislation, e.g. IPR Selection Self motivation Service user liaison and support Service development Writing data management plansSenior data manager Analysis Practical Collaboration Communicationhttp://www.digcur-education.org Page 12 of 19
  13. 13. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation framework Computer literacy Current trends in field Construction and maintenance of data files Data checking and editing Data linkage checking and validating Provision of data to users Organising Planning Self motivation SPSS and/or STATA or programming experience Writing documentation of procedures Writing reports Relevant legislation, e.g. IPRhttp://www.digcur-education.org Page 13 of 19
  14. 14. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation frameworkData director Advising on data collection, management, analysis and Managerial protection Collaboration Communication Costing Current trends in field Directing management of large and complex datasets Influencing and persuading Organising Planning Project management Quality assurance Relevant legislation, e.g. IPR Resource planning Strategy and policy development Supervising team Writing funding proposalsData librarian Cataloguing and metadata standards Practical, Managerial Collaboration Communication Current trends in field Digital curation knowledge – standards, systems, tools Data discovery and retrieval Metadata Organising Planning Project management Quality assurance Relationship building Service evaluation Service promotion and development Service user liaison and supporthttp://www.digcur-education.org Page 14 of 19
  15. 15. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation framework Supervising team Training others User requirements research Writing data management plans Writing service user guidanceDigital archivist Archival / information management experience Practical Budget monitoring Collaboration Communication Computer literacy Current trends in field Digital curation knowledge – standards, systems, tools Implement standards Organising Planning Project management Quality assurance Self motivation Service promotion and development Supervising team Writing reportsDigital curation officer Collaboration Communication Service user liaison and support Training others User requirements researchDigital resources officer Creativity Collaboration Communication Computer literacy Data checking and editinghttp://www.digcur-education.org Page 15 of 19
  16. 16. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation framework Data discovery and retrieval Data ingest Database construction Librarian, archiving or information management experience Metadata Organising Self motivation Service development and promotion Service user liaison and support Training others User requirements research Webpage construction Writing documentation of data file content Writing service user guidancehttp://www.digcur-education.org Page 16 of 19
  17. 17. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation frameworkArea 4: Part of digital curation lifecycleThe Digital Curation Centre lifecycle model is available athttp://www.dcc.ac.uk/resources/curation-lifecycle-model.Mapping to the digital curation lifecycle model is a useful way for many professionals workingin the digital curation professions to understand and conceptualise how and where differenttraining offerings fit together / intersect as this model is widely used in digital curation trainingboth in the UK and in various other regions including the USA. This is also helpful fortraining developers to establish which areas of the digital curation lifecycle are currentlyabundantly and poorly served by current training offerings.  Conceptualise  Create or receive  Appraise and select  Ingest  Preservation Action (migration, emulation etc.)  Store  Access, use and reuse  Transform  Preservation planning  Description or representation informationhttp://www.digcur-education.org Page 17 of 19
  18. 18. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation frameworkArea 5: Teaching methods/training deliveryThis Area helps training developers to make clear and appropriate decisions about the bestformat or method for their training offering. This builds on research already published byDPOE about effective training methods for different audiences and also draws on findingsfrom the DigCurV training needs survey. The first five rows look at training products whilethe last two rows describe how an entire curriculum could be delivered.Teaching method DPOE audience levelLecture AllSmall group exercise Managerial, PracticalOnline seminar (‘webinar’) Executive, ManagerialOnline self-paced course PracticalCorporate briefing ExecutiveWritten manual Managerial, PracticalSupervised one-to-one training by senior staff member PracticalWorkshop (hands-on training) Managerial, PracticalBlended learning (mixture of face to face and online learning) Managerial, Practicalhttp://www.digcur-education.org Page 18 of 19
  19. 19. D2.1 Report on baseline survey and evaluation framework Section 2: Evaluation frameworkArea 6: Professional contextThis area is based on existing work by DigCurr, particularly their matrix, with descriptionsadded relevant to DigCurV findings. This may help training providers to tailor the trainingthey’re developing to a specific professional context, and become aware of challenges andbehaviours or expectations specific to this specific professional context. For example, adigital art conservator will have different priorities, discipline requirements and legislation toconsider than a curator working in a national library.)This table is available under the Creative Commons license reproduced here.It can be important to understand challenges, opportunities and characteristics of particulartypes of work contexts (e.g. social science data archive in a university, commercial collectionof scanned page images, state archives, serving a population with specific cultural norms).Context categories DescriptionProfessional context History of professional History of activities relevant to digital curation in various activities streams of work activity. (e.g. Care and properties of physical media; Hardware and software interoperability; Long-term management of institutional archives and personal papers; Social science data archives...) Professional development Important elements of and strategies for actively participating in a profession and remaining aware of current state of professional principles and practices (e.g. professional associations, conferences, continuing education)Disciplinary contextInstitutional or organisationalcontext Characteristics of information and record keeping environmentsCultural context "The distinctive ideas, customs, social behaviour, products, or way of life of a particular society, people, or period." [1]From Christopher (Cal) Lee, Professional, Disciplinary, Institutional, Organisational or Cultural Context (Dimension 3 of Matrix of Digital Curation Knowledge and Competencies) School of Information and Library Science, University of North Carolina at Chapel Hill. Draft: June 17, 2009 (Version 18). Reproduced here under Creative Commons AttributionNon-Commercial Share-Alike 3.0 License [http://creativecommons.org/licenses/by-nc-sa/3.0/][1] "Culture." Oxford English Dictionary. Draft Revision. June 2009 .http://www.digcur-education.org Page 19 of 19

×