20140808 glam evaluation_workshop_intro

•Download as PPTX, PDF•

0 likes•1,266 views

This document outlines the goals and agenda for the Bern University of Applied Sciences GLAM-Wiki Evaluation Workshop. The workshop aims to (1) provide an overview of different approaches to evaluating GLAM-Wiki cooperation, (2) discuss the strengths and shortcomings of various evaluation approaches, and (3) tackle pressing challenges. The document discusses why evaluation is important for accountability, decision-making, and organizational learning. It also presents five key questions to characterize different evaluation approaches: what is being evaluated, when the evaluation occurs, what values are being judged, who is making the judgments, and the evaluation's intended uses. The workshop agenda includes group work and discussions.

Bern University of Applied Sciences
GLAM-Wiki Evaluation Workshop
Beat Estermann, 8 August 2014
Wikimania 2014, London
▶ Bern University of Applied Sciences, E-Government Institute
British Museum (London), Wikimedia Commons, M.chohan, public domain
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 License.

Bern University of Applied Sciences
▶ Get an overview of different approaches towards
evaluating GLAM-Wiki cooperation
(input presentations)
▶ Discuss the strengths and shortcomings of the various
approaches
(input presentations + plenary discussion)
▶ Tackle pressing challenges
(workshop)
Goals of the session

Bern University of Applied Sciences
 Why evaluation?
• For accountability purposes
• As a decision making tool
(project/programme design, funding decisions, etc.)
• To promote (inter-)organizational learning
Five questions to ask in order to characterize
evaluative approaches

Bern University of Applied Sciences
 What is being evaluated?
• A project
• A programme
• A thematic aspect cutting across programmes
Five questions…

Bern University of Applied Sciences
 When is the evaluation taking place?
Five questions…
before the start of the
project / programme
after the end of the
project / programme
at a point during the
project / programme
at several points during
the project / programme

Bern University of Applied Sciences
 What is the value judgement about?
Five questions…
Needs
Challenges
Goals Resources Outputs
Context / Background
Project / Programme
Evaluation
Outcomes
Impact
Relevance
Usefulness
Effectiveness
Efficiency
Based on European Commission, DG XVI/G2: Collection MEANS, Evaluer les programmes socio-économiques, 1998.

Bern University of Applied Sciences
 Who is making the value judgement?
• programme managers, as the evaluation is part of programme
management
(focus: value for money)
• a multi-stakeholder evaluation board involving the various
stakeholders concerned by the activity that is to be evaluated
(focus: collective problem solving)
• a board composed of people elected by their constituency/ies
(focus: democratic legitimacy)
Five questions…

Bern University of Applied Sciences
Phase Problem owner(s) Rest of the group
1 Define a clear problem
statement
2 Answer questions Ask «open» questions in order to
reach a better understanding of
the problem
3 (Take notes in view of phase 4) Present their interpretation of the
problem situation. Suggest
solutions (taking turns)
 Documentation on flip chart
4
In the plenary:
Give a short feedback which
proposed interpretations /
solutions they deem most
useful
Group Workshops

Bern University of Applied Sciences
▶ Contact details:
• Beat Estermann
E-mail: beat.estermann@bfh.ch
Phone: +41 31 848 34 38
▶ OpenGLAM:
• http://openglam.org
Thank you for your attention!

The document summarizes the successful implementation of an ePortfolio system at HAN University in the Netherlands which has over 25,000 active users. It describes the 4 step process: 1) Developing a strategy in 2003 to introduce competence-based learning and assessment using ePortfolios. 2) Implementing the initial ePortfolio system (DPF 1.0) in 2004. 3) Incorporating the system across the university through centralized and local management from 2008. 4) Embedding the ePortfolio (DPF 2.0) into the university's virtual learning environment in 2010 to provide an integrated system for all current and future students. Lessons learned included not scheduling assessments on weekends and customizing templates to better align

Strijker, A. (2005 12 06). Piloting Sakai In A Master Course Does It Really...

Saxion

1) Dr. Allard Strijker conducted a pilot of the Sakai course management system in a master's program course at the University of Twente to evaluate if it could effectively support their educational needs. 2) The pilot encountered several usability issues like inconsistent interfaces, unclear relationships between functions, and technical problems accessing course materials. 3) While the basic functions worked, Dr. Strijker was ultimately disappointed as Sakai did not meet expectations for collaboration, educational support, mobile access, or interoperability for their needs. Improved documentation and interface consistency were seen as needed.

Assessment of Higher Education Learning Outcomes

Kolds

- The document discusses the Assessment of Higher Education Learning Outcomes (AHELO) project, an OECD initiative to assess learning outcomes in higher education internationally. - AHELO aims to create valid measures of learning outcomes that can be used across cultures, languages, and types of higher education institutions. This will provide a more balanced assessment than rankings which often rely on input factors. - The feasibility study will develop and test assessment instruments in engineering, economics, and generic skills, and collect contextual data from students and faculty. It will evaluate whether reliable cross-national assessment is possible and implementation is feasible.

Comparative Analysis of Engineering Math Edication in EU and Georgia & Armenia

mathgear

The document summarizes the key activities and findings from Work Package 1 of the MathGear project, which compared engineering mathematics education in the EU, Georgia, and Armenia. Some main points include: - Course comparisons were conducted between EU universities and those in Georgia and Armenia to analyze similarities and differences. - Overall the contents of engineering mathematics courses were found to be very similar, but Georgian and Armenian courses sometimes covered more topics or had more lecture hours. - Recommendations included emphasizing applications over proofs, adding relevant engineering examples, and increasing the use of mathematical tools and e-learning platforms. - Bridging courses and student testing were found to be more established practices in the EU that could help

Kowledge zoom michelle

Roya Hosseini

This document describes KnowledgeZoom, a concept-based exam study tool with zoomable open student modeling to help students efficiently prepare for final exams. It integrates fine-grained concept-based guidance, an open student model that progressively zooms between topics and sub-topics, and adaptive problem sequencing. The tool includes a Knowledge Explorer that visualizes students' knowledge gaps through a zoomable treemap, and a Knowledge Maximizer that sequences practice problems based on students' knowledge levels and impact. An evaluation with 14 students found the tool helped identify knowledge weaknesses and generated quizzes covering multiple concepts, though better integration between its components could further improve the student experience.

Workshop K7: Digitale Media Concepts

NHTVBreda

The document discusses strategies for successfully acquiring projects in professional fields for educational purposes. It addresses identifying potential project types and partners, as well as managing the project acquisition process. Some specific project types and dimensions mentioned include theme/module projects, placement projects, and research projects with varying scopes, natures, durations and resources. The document also provides tips for acquiring projects by networking inside and outside an organization, and suggests that multiple parties like lecturers, students, and placement offices could be involved in the acquisition process. Overall, the goal discussed is to independently organize educational programs through professional field projects.

This document discusses modernizing math courses for STEM education in Armenian universities. It outlines activities under Work Package 2, including analyzing differences between EU and Armenian math curricula and recommendations for structural improvements. Key activities involve studying EU best practices, comparing course durations/credits, and modifying selected courses based on the SEFI framework. Courses will focus more on applied competencies and introduce e-learning tools like Math-Bridge. Outcomes include localized TEL tools, improved math programs, and a planned large-scale evaluation to assess the new curricula's impact on engineering/science education quality.

Evaluation Methodology

mathgear

This document outlines an evaluation methodology for testing reformed math courses. It proposes conducting a longitudinal study comparing student outcomes between a controlled group taught with old courses and an experimental group taught with new reformed courses. Data would be collected through pre-and post-tests to assess knowledge gain, pre-and post-questionnaires to measure motivation and attitudes, and course evaluation forms. The goal is to demonstrate educational benefits like improved grades, knowledge, and motivation or decreased drop-out rates for the reformed courses. Challenges include accounting for uncontrolled differences between groups and ensuring student participation in surveys.

Sevaq_BMerison_EMorin_belfast_2007

Edwige Morin

The document summarizes a European self-evaluation tool called SEVAQ that was developed to improve the quality of e-learning experiences. The tool combines the EFQM excellence model and Kirkpatrick evaluation model into an innovative framework. It identifies various stakeholders in the evaluation process and provides a flexible questionnaire design tool to allow customization of questions for different applications. The tool allows designers to set up questionnaires, publish them to target learners or groups, and analyze the results. The overall goal is to provide a guide for self-evaluation and quality improvement of e-learning.

Evaluation Methodology

mathgear

This document outlines an evaluation plan to compare the effectiveness of reformed mathematics courses to traditional courses. It proposes a longitudinal study with two groups: one taught the old course in 2015 and another taught the new course in 2016. Effectiveness will be measured by comparing student performance on pre- and post-tests, as well as surveys on motivation, knowledge gain, dropout rates, and course satisfaction between the two groups. The plan details what will be measured, the test design, and pre- and post-questionnaires to collect demographic data and assess changes in motivation and expectations.

[SLP webinar week] IMPLEMENTING European SLPs - Recommendations for Stakehold...

EADTU

Intervention Plan Instructitons

Miami-Dade County Public Schools

This document provides instructions for an intervention lesson plan assignment due on April 14, 2011. Students are asked to select three struggling readers and collect background information on each, including school performance factors. They must then write a report summarizing the background information. The intervention lesson plan should include three sessions, each with a goal, supporting activity, observation, and reflection on student learning and needed support. The typed intervention plans and activity artifacts are to be submitted to the instructor.

Promoting Student Access and Success Through Research

Tanya Joosten

The National Research Center for Distance Education and Technological Advancement (DETA) aims to promote student access and success through rigorous research on distance education. Over three years, DETA will develop research models and frameworks, host a national summit, establish research questions, create research toolkits, fund pilot studies through a request for proposals, and evaluate models and technologies. The goal is to understand how different instructional designs and support structures impact student outcomes like learning, satisfaction, and completion for diverse populations. Findings will ensure quality education and inform sustainable teaching practices in online, blended, and competency-based models. DETA is led by co-directors Drs. Tanya Joosten and Diane Reddy.

[SLP webinar week] Challenges and opportunities for the recognition of short ...

EADTU

Incorporating divergent vs convergent thinkers into teaching

Jane Yin-Kim Yau

This document summarizes a course on diversity in teaching. It provides details on the structure and activities of the course from 2020 and 2021. In 2020, the course included introductory lectures to inspire research ideas, having students develop research questions in teams, and allowing freedom in projects and methodologies. Some students flourished while others struggled with workload and time constraints. In 2021, changes were made including pre-and post-tests, more structure in research activities based on convergent or divergent thinking styles, and grouping students through self-analysis to support different needs. The goal was to foster both creativity and logical, analytical skills among students.

E-SLP webinar week: Concept and role of Short Learning Programmes in European...

EADTU

PPT Presentation from Parent Workshop

educationalnetworks

The document discusses the exhibition process at MHS students. It explains that the exhibition requires students to apply skills and demonstrate content knowledge through presenting a completed class project. Students will go through lessons to prepare their presentation to a committee. The presentation grade will count as 15% of the student's grade in the class the project was from. The document also provides the exhibition timeline and ways families can support students.

EMPOWER webinar week on short learning programmes: Design and expanding HE ho...

EADTU

E-SLP webinar week: lessons learned from the pilot digital competent educators

EADTU

Implementation of a Novel Industrial Robotics Course and its Evaluation by St...

Technological Ecosystems for Enhancing Multiculturality

Seminario eMadrid

Israel Gutiérrez

The document discusses open standards and specifications for outcome-based assessment in higher education. It analyzes various assessment formats, finding scarce reuse of assessment resources and acceptance of the IMS QTI specification. It proposes an outcome-based assessment app integrated with an open content repository to support searching, filtering, importing, and annotating assessment resources based on intended learning outcomes. The app aims to gather and share best practices for interoperable, outcome-oriented assessment in higher education.

Blackboard Analytics for Learn @JCU – a proactive approach to the use of data...

Blackboard APAC

1) James Cook University implemented Blackboard Analytics for Learn to take a dual-pronged, data-driven approach to enhancing learning/teaching and the student experience. 2) Initial rollouts involved customizing dashboards and reports to provide insights for support officers, coordinators, and academics. 3) The analytics aim to inform subject redesign, evaluation, planning and monitoring to improve student engagement, retention and blended learning design.

Assessing Experiential Learning Through E-portfolios

Richard Robles

The document discusses the University Honors Program at UC. It provides an overview of the program framework which includes a first-year experience using e-portfolios, honors seminar courses, and experiential learning projects. Students in the top 5% of undergraduates are challenged through honors seminars and projects focused on community engagement, global studies, leadership, research, and creative arts. Experiential learning projects are significant learning experiences outside the classroom related to the thematic areas. E-portfolios are used to showcase and integrate learning through artifacts from coursework and projects.

E-Portfolios for Student Success

Richard Robles

The document discusses e-portfolios for student success at the University of Cincinnati. It provides details on the e-portfolio landscape at UC, including programs in the DAAP college and UC Honors program. The UC Honors program introduced e-portfolios in 2007 to help students develop identity, integrate learning, and showcase experiences. In 2008, the program switched to a new e-portfolio platform, iWebfolio, to allow for customized templates, assessment, and feedback. Lessons learned included the importance of priorities and partnerships, and implications involved developing new assessment methods and building a culture of reflection.

The Truth is Out There: Dutch Nursing realized sustainable adoption of OER

Robert Schuwer

TCI Cluster Evaluation Meeting Oslo

TCI Network

The document outlines the agenda for a two-day Cluster Evaluation Working Group meeting in Oslo, Norway. Day one will focus on program-level evaluation, including a presentation on Norway's program evaluation and small group discussions around guiding questions. Day two will discuss capturing the human element of clusters, with a review of survey pilots and alternative approaches, as well as small group discussions and reporting back. The working group aims to collectively learn about common challenges in cluster evaluation and share learning and new approaches.

Framing Blended learning, teaching, and education

EADTU

What's hot

Student Content Evaluation UC & R April 2009

Kaye Towlson

WP 2: Modernization of Math Courses for STEM Education in GE/AM Universities

mathgear

Evaluation Methodology

mathgear

Sevaq_BMerison_EMorin_belfast_2007

Edwige Morin

Evaluation Methodology

mathgear

[SLP webinar week] IMPLEMENTING European SLPs - Recommendations for Stakehold...

EADTU

Intervention Plan Instructitons

Miami-Dade County Public Schools

Promoting Student Access and Success Through Research

Tanya Joosten

[SLP webinar week] Challenges and opportunities for the recognition of short ...

EADTU

Incorporating divergent vs convergent thinkers into teaching

Jane Yin-Kim Yau

E-SLP webinar week: Concept and role of Short Learning Programmes in European...

EADTU

PPT Presentation from Parent Workshop

educationalnetworks

EMPOWER webinar week on short learning programmes: Design and expanding HE ho...

EADTU

E-SLP webinar week: lessons learned from the pilot digital competent educators

EADTU

Implementation of a Novel Industrial Robotics Course and its Evaluation by St...

Technological Ecosystems for Enhancing Multiculturality

Seminario eMadrid

Israel Gutiérrez

Blackboard Analytics for Learn @JCU – a proactive approach to the use of data...

Blackboard APAC

Assessing Experiential Learning Through E-portfolios

Richard Robles

E-Portfolios for Student Success

Richard Robles

What's hot (19)

Student Content Evaluation UC & R April 2009

WP 2: Modernization of Math Courses for STEM Education in GE/AM Universities

Evaluation Methodology

Sevaq_BMerison_EMorin_belfast_2007

Evaluation Methodology

[SLP webinar week] IMPLEMENTING European SLPs - Recommendations for Stakehold...

Intervention Plan Instructitons

Promoting Student Access and Success Through Research

[SLP webinar week] Challenges and opportunities for the recognition of short ...

Incorporating divergent vs convergent thinkers into teaching

E-SLP webinar week: Concept and role of Short Learning Programmes in European...

PPT Presentation from Parent Workshop

EMPOWER webinar week on short learning programmes: Design and expanding HE ho...

E-SLP webinar week: lessons learned from the pilot digital competent educators

Implementation of a Novel Industrial Robotics Course and its Evaluation by St...

Seminario eMadrid

Blackboard Analytics for Learn @JCU – a proactive approach to the use of data...

Assessing Experiential Learning Through E-portfolios

E-Portfolios for Student Success

Similar to 20140808 glam evaluation_workshop_intro

The Truth is Out There: Dutch Nursing realized sustainable adoption of OER

Robert Schuwer

TCI Cluster Evaluation Meeting Oslo

TCI Network

Framing Blended learning, teaching, and education

EADTU

How to write Proposal for a Project by Ahmadshah Pashaye

Imperial university

The document provides guidance on writing effective project proposals, outlining key sections and content to include. It discusses identifying the problem and project idea, developing goals and objectives, describing implementation methodology and partners, preparing the budget, and planning for monitoring, evaluation and sustainability. The document emphasizes showing thorough planning, fitting the proposal to donor guidelines, and demonstrating a well-thought out solution to a genuine community need or problem.

Xdelia evaluation framework

grainne

The document outlines plans for developing an evaluation framework for a project involving games and sensors for financial decision training. It discusses holding stakeholder workshops to develop shared definitions and an evaluation framework involving research questions, methods, and timelines. The framework will use a participatory, iterative approach drawing on utilization-focused evaluation, participatory design, and learning design principles. Key project milestones are identified relating to workshops, developing the framework, and conducting evaluation studies throughout the project.

Online educa 2011-arnold

Patricia Arnold

The document discusses an online peer-to-peer counseling program introduced as part of the BASA-Online social work degree program in Germany. Students use an online platform to work in autonomous teams on real cases from their own practice, guided through a 10-step counseling process. An evaluation found that students viewed peer counseling positively as a way to integrate academic theory with work experience. Further refinements were suggested, such as introducing assessment tools earlier and increasing transparency of the process.

U koersythesis&evaluation

loumcgill

This document discusses evaluation and synthesis for the UK OER 2 program. It introduces the evaluation and synthesis framework developed in the pilot phase and discusses identifying evaluation questions and gathering evidence to answer those questions. Key focus areas for evaluation are identified such as approaches to OER release, expertise, cultural issues, and pedagogy/end-use issues. The roles of the evaluation team in supporting projects and collating cross-project evidence are also outlined.

Evaluation training for wellcome trust 15th may

Bruce Etherington

This document summarizes a presentation on evaluating engagement activities. The presentation aimed to help participants develop evaluation strategies and make strong cases for engagement. It covered why evaluation is important, how to identify what to evaluate using logic models, who evaluations are for, and making the case for engagement through evaluation. The presentation included activities where participants discussed their experiences with evaluation and worked through examples of logic models and evaluation plans.

Pedagogical and digital tool for the tertiary level

Andrea Ximena Castaño Sanchez

This document discusses the use of ePortfolios at the Rovira i Virgili University in Spain. It provides context about the university's education reality and the goal of finding a viable solution to a previously identified problem. It then describes the ePortfolio initiative undertaken at the university, including how it was used for formative assessment. Key aspects that were considered in the initiative included whether the ePortfolio would be decentralized or centralized, open or closed, public or private, and long or short term. Reflection, interaction, self-assessment, and monitoring learning were also discussed.

Research Plan Development Workshop, Mario Tabucanon and Philip Vaughter, UNU-IAS

Hanna Stahlberg

Masterclass on project building

tamsin.rose

Interim report and main findings, by Lasakova, Bajzikova & Sasso

gaihe

This interim report summarizes the main findings from three research streams of the GAIHE project. The desk research analyzed over 180 sources to classify types of higher education innovation and identify drivers, governance factors, and barriers. A survey of 47 higher education institutions found that module innovations are more common than program or institutional innovations and identified key motivations and challenges. Case studies of 10 institutions found examples of innovative teaching practices and supporting governance structures. The project aims to develop a self-assessment toolkit to measure institutional support for innovation and disseminate results through 2016.

Redesigning assessment and feedback - landscape review and areas for development

Jisc

OABC UG & PG PPT.pptx

bansalbookstore

This document outlines a course on Social Responsibility and Community Engagement. The 2-credit course involves outdoor activities and community engagement. The objectives are to develop an appreciation of rural culture and life, understand rural challenges, and contribute to community development. The course content covers socio-cultural aspects, rural economy, institutions, and government programs. Pedagogy includes lectures, discussions, and assignments. Assessments include presentations, assignments, and a portfolio evaluated based on learning outcomes like understanding rural life and identifying community development opportunities. The course aims to create socially responsible practitioners committed to national development.

OpenER, OER @OUNL

guest2503e3

The OpenER experiment was a Dutch initiative that aimed to lower barriers to participation in formal higher education by offering free online courses. It received over €660,000 in grants and launched 24 courses in 2006 that attracted several hundred thousand unique visitors. While over 5,700 users registered for courses, only 85 applied to take a formal examination for certification and 29 passed. Evaluations found that for many users, certificates were not important and courses were too short or not in their native language. However, 9.3% of those who later paid for regular courses cited taking a free OpenER course as influencing their decision. Going forward, the initiative focused on sustainability and producing higher quality open courses as a regular part of university offerings.

The MOOC Production Fellowship: Reviewing the first German MOOC funding program

oncampus

The document summarizes the results of a survey on the first MOOC funding program in Germany. It provides the following key details: - The program funded 10 MOOCs with €25,000 each, receiving over 260 applications. Funded MOOCs reached 224,446 learners but only 6,921 certificates were issued. - Feedback from funded MOOC creators was mostly positive, though production required significant time investments. Non-funded applicants also created MOOCs by investing time but without funding support. - Ongoing support like crediting MOOCs as teaching and financial funding were seen as important to further developing MOOCs in Germany. While the program was seen as helpful, continued support

The MOOC Production Fellowship: Reviewing the first German MOOC funding program

Anja Lorenz

The document summarizes the results of a survey on the first MOOC funding program in Germany. It provides the following key details: - The program funded 10 MOOCs with €25,000 each, receiving over 260 applications. Funded MOOCs reached 224,446 learners but only 6,921 certificates were issued. - Feedback was generally positive for funded MOOCs but challenges included high production costs, limited interaction from learners, and issues with the platform. - Non-funded applicants also created MOOCs by financing production themselves or finding other funding, but faced difficulties without the initial support. - Ongoing support is needed for MOOC creators, such as credit for

2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke

Christian M. Stracke

Formative Aspects of Summative Assessment, LSBU, 6 May 2010

Colston Sanger

This document summarizes a presentation about a research project on formative aspects of summative assessment. The project investigated what happens when students negotiate their own summative assessment criteria and assess their own and peers' work. Preliminary results found that students felt it was useful to be assessed on criteria they felt were important. Students also found assessing peers to be a valuable learning experience, though it could be tricky. The presentation discussed implications for developing students' critical judgment and self-directed learning skills.

The Next 25 Years: Bonner's Commitment to Access & Engagement

Bonner Foundation

Similar to 20140808 glam evaluation_workshop_intro (20)

The Truth is Out There: Dutch Nursing realized sustainable adoption of OER

TCI Cluster Evaluation Meeting Oslo

Framing Blended learning, teaching, and education

How to write Proposal for a Project by Ahmadshah Pashaye

Xdelia evaluation framework

Online educa 2011-arnold

U koersythesis&evaluation

Evaluation training for wellcome trust 15th may

Pedagogical and digital tool for the tertiary level

Research Plan Development Workshop, Mario Tabucanon and Philip Vaughter, UNU-IAS

Masterclass on project building

Interim report and main findings, by Lasakova, Bajzikova & Sasso

Redesigning assessment and feedback - landscape review and areas for development

OABC UG & PG PPT.pptx

OpenER, OER @OUNL

The MOOC Production Fellowship: Reviewing the first German MOOC funding program

2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke

Formative Aspects of Summative Assessment, LSBU, 6 May 2010

The Next 25 Years: Bonner's Commitment to Access & Engagement

More from Beat Estermann

Using Wikidata for Performing Arts Related Data

20140808 glam evaluation_workshop_intro

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to 20140808 glam evaluation_workshop_intro

Similar to 20140808 glam evaluation_workshop_intro (20)

More from Beat Estermann

More from Beat Estermann (20)

Recently uploaded

Recently uploaded (20)

20140808 glam evaluation_workshop_intro