Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

An Open Research Approach - 2015 Open Education Week TU Delft


Published on

Openness in research and education is central in the TUD mission. It was the first edX partner to issue all its MOOCs under a creative commons license. Here we present our open research approach and toolbox. This toolbox facilitates sharing of data and the process of collaborative research in the context of MOOCs/online education. The presentation explains our rationale for open research, describes the toolbox, and our experiences with it, including research highlights.

Published in: Education
  • Be the first to comment

An Open Research Approach - 2015 Open Education Week TU Delft

  1. 1. 9-3-2015 Challenge the future Delft University of Technology An ‘open research’ approach Thieme Hennis – Pieter de Vries - TU Delft Online Learning
  2. 2. A bit of context
  3. 3. Big ambition… • And questions on different levels..
  4. 4. Level (Example) topics of interest and stakeholders Strategic University role into the future, open en online education, business models, ROI Deans, DelftX, MT, marketing Organizational Support structure, documentation and guidelines, collaboration protocols, data use, teacher satisfaction O&S Course Course design, use of media, pedagogy, methods and tools, student support, experiments, interaction & collaboration Course teams, Media Center Technology/edX Design of tools for learning, teaching, and data analysis/visualization edX consortium, ‘TUD tech’ Students Learner demographics, background, performance, engagement, satisfaction, etc. Students
  5. 5. Many opportunities • … to research MOOCs • … and use MOOCs for research But.. • Limited capacity • No/little history in educational research at university • Potential to connect with researchers worldwide
  6. 6. Open Research approach • What is open research?
  7. 7. Open research… Facilitating research activities 1. …with relevant parties globally and locally 2. …on a broad range of relevant topics 3. …by giving them access to our data and research instruments 4. …and managing the collaboration workflow 5. …and publishing the results together
  8. 8. 1a. Who are these ‘relevant parties’? • MOOC Teachers / course teams • University students / PhD’s • Researchers from partner institutes • Researchers globally with expertise on a specific, relevant topic
  9. 9. 1b. Our current ‘MOOC research network’ • Stanford University - Psychology • UC St. Barbara - Psychology • University of South Australia - Pedagogy, coll. learning • Edinburgh University - Communities of Inquiry • Harvey Mudd College - Gender, diversity • TU München - Gender, diversity • Memphis university - Linguistics • UVA - Methodologies • TU Delft - Privacy/ethics - Course-relevant topics (researching course teams)
  10. 10. 2a. The ‘broad range’ of relevant topics • Who is the DelftX MOOC learner? - UNISA • Does self-affirmation of personal values increase engagement of at- risk students (motivation research) – Stanford, UC St. Barbara • (How) does communication in MOOC forums influence social centrality and performance? – Memphis Univ. • How well does the Community of Practice framework fit learning in a MOOC context? What effects of social and teaching presence on social networks and collaboration? – Edinburgh univ. • Why do female students have lower performance in our MOOCs? How to design socially inclusive courses? – Harvey Mudd College & TU München • Etc..
  11. 11. 2b. Choice of topics Research questions come/emerge from • Evaluation outcomes (DelftX MOOCs) • Literature, current developments in educational research • Teachers, developers, and other support staff • External parties interested in research collaboration
  12. 12. 3. Giving access to data and instruments Data sources Research & evaluation edX subscription data Number of participants, dropouts, location, age, gender, schooling edX student data, learning analytics Progress, tests results, quizzes, exams, etc. Forum participation Video clicks, navigation External data, other media (i.e. Facebook groups, discussions) Social networks, content and discourse analysis Surveys (pre, mid, post) Information about demographics, intention, expectations, satisfaction, media use, etc. Interventions embedded in surveys Interviews / self-assessment: teachers, NMC, DelftX Experiences with workflow and organization Questions and expectations for evaluation
  13. 13. 4a. Facilitating collaboration workflow
  14. 14. 4b. The Open Research Toolbox Collaborating • Google Drive folder with • Documentation for partners • Data files Documentation • Memorandum of Understanding • Ethical clearance documentation • Paper outline • All available data sources, instruments, and courses Communication • Monthly Skype conversations (depending on stage in process) • Email
  15. 15. 5. Selection of publications • 6 Working papers • 5 course reports • 1 comparative analysis paper • Understanding Social Learning Behaviors (SEFI’14 - Skrypnyk, Hennis, De Vries, 2014) • Learners’ Social Centrality and Performance (EDM’15 – Dowell, Joksimovic, Skrypnyk, Hennis, De Vries, 2015) • An exploratory study in the concerns for information privacy (MSc thesis - Hassing, 2015) • Scalability and Flexibility through Open Research (LAK’15) Hennis, De Vries, 2015) • Who is the Learner in the DelftX Engineering MOOCs? (SEFI’15 - Hennis, Skrypnyk, De Vries, 2015) • Diversity in Engineering MOOCs, a first Appraisal (SEFI’15 – Ihsen, Yves, Hennis, De Vries 2015) • Reconsidering Retention in MOOCs: the Relevance of Formal Assessment and Pedagogy (EMOOCS’15 - Skrypnyk, De Vries, Hennis, 2015)
  16. 16. Potential of ‘open research’ • Being highly flexible in what you research • Being able to scale the research efforts • Research network and learn from their expertise • Interdisciplinary research
  17. 17. Next steps • Optimize the work flow & instruments • Research across courses (other edX partners/MOOC providers) • More data cleaning capacity + open data workflow • Extend the research HUB function (also: organizations) • Better integration and involvement course teams to achieve design based research → feedback and forward loops between research and teaching (online) Additional ideas: • Tender for MOOC research • Promote TU Delft research areas • Students as researchers (crowdsource research models)
  18. 18. Some interesting outcomes
  19. 19. Gender gap • Low percentage female students at start • Even lower percentage female students finishing • Stereotype threat? → collaboration Univ. Munich & Harvey Mudd College Ihsen, S, Yves, J, Hennis, T. A. & Vries, P. de. (2015). Diversity in Engineering MOOCs, a first Appraisal. In SEFI 43th Annual Conference. Orleans, FR.
  20. 20. Performance attitude •Among young, male students • ‘Spike’ around grades 6 and 10 • Reason to join MOOC: “To get a certificate..” Young vs Old Male vs female Hennis, T. A., Skrypnyk, O., & Vries, P. de. (2015). Who is the Learner in the DelftX Engineering MOOCs. In SEFI 43th Annual Conference. Orleans, FR.
  21. 21. Dropout rate • Non-starters are biggest dropout group (up to ~ 80%!) • Then: after first Homework Assignment (HWA.01) • GRADE HWA.01 IS IMPORTANT PREDICTOR • Avg. grade HWA.01 of group that does not continue: to HWA.02: 3.58 • Avg. grade of group that does continue: 8.89 • Also: pass rate is > 80% of group who performs better than 20% • Including the ‘explorers’ • “No Time” most important factor Pedagogy and Assessment design → Collaboration with University of South Australia Skrypnyk, O., Vries, P. de. & Hennis, T. A.(2015). Reconsidering Retention in MOOCs: the Relevance of Formal Assessment and Pedagogy. In EMOOCs’15, Mons, BE.
  22. 22. (Professional) background students • Many professionals: 50-70% • “No experience” predictor of early dropout / not starting • Not during course! • Relevance for course team: • Recommended level course ~ student level? • Who are my students? • etc Hennis, T. A., Skrypnyk, O., & Vries, P. de. (2015). Who is the Learner in the DelftX Engineering MOOCs. In SEFI 43th Annual Conference. Orleans, FR.
  23. 23. Forum (1): performance en structuur • Forum activity ~ grade • Facebook groups vs forum • More ‘social’ behavior and peer- support • Not anonymous • Better performance • edX forum social network analysis • Few ‘superposters’ responsible for most posts • Few ‘communities’ Skrypnyk, O., Hennis, T. A., & Vries, P. de. (2014). Understanding social learning behaviours of xMOOC completers Conference Topic : Technology in Learning. In SEFI 42th Annual Conference. Birmingham, UK.
  24. 24. Forum (2): Language ~ social capital + Community of Inquiry • Community of Inquiry • Measuring social and teaching presence • Effects on engagement, performance and forum activity • Linguistic analysis • Aumated textual analysis: how conversations influence development of social capital → with Memphis Univ & Univ of Southern Australia Dowell, N, Skrypnyk, O., Joksimović, S, Graesser, A, Dawson, S, Gasevic, D, Hennis, T. A.,, Vries, P. de., Kovanovic, V. (2015). Modeling Learners’ Social Centrality and Performance through Language and Discourse. In EDM’15. Madrid, Spain.
  25. 25. Forum (3): cultural group & preference to collaborate • EUR / Anglo > alone • Asia / African /Latin > together → edX forum: does not show this cultural preference! (vooral EUR/Anglo)