LAK15 - SCALABILITY AND FLEXIBILITY THROUGH OPEN RESEARCH
•Download as PPTX, PDF•
2 likes•987 views
TU Delft is exploring an open research approach to maximize opportunities to research MOOCs and use MOOCs for research. This involves connecting with relevant researchers globally and locally to study a broad range of topics by providing access to TU Delft MOOC data and research instruments. The collaboration workflow includes defining research ideas, signing agreements, deploying instruments, analyzing data, and publishing results. Current work focuses on streamlining these processes and building research capacity while addressing issues like data standards and privacy.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (8)
Similar to LAK15 - SCALABILITY AND FLEXIBILITY THROUGH OPEN RESEARCH
Similar to LAK15 - SCALABILITY AND FLEXIBILITY THROUGH OPEN RESEARCH (20)
More from Thieme Hennis
Recently uploaded
Recently uploaded (20)
LAK15 - SCALABILITY AND FLEXIBILITY THROUGH OPEN RESEARCH
- 1. An ‘open research’ approach Thieme Hennis – thieme@hennis.nl Pieter de Vries - Pieter.deVries@tudelft.nl TU Delft Online Learning – presentation LAK’15
- 3. • Established: 1842 • 15 bachelor programs • 33 Masters programs • 8 faculties • 18,000 students (3,000 from abroad) • >2000 PhD-students • #66 ranking THIS
- 4. Environment Infrastructures & mobility Health Delft Research Initiatives Energy
- 5. • Moral obligation • growing demand in higher education worldwide 2012-2025: 80 million! • Quality • improve our materials, teaching methods • Reputation • to be there with the other top universities • Innovation • digital and online education inevitable
- 6. In 2013…
- 7. Next Generation Infrastructures 2 Drinking Water Treatment Functional Programming Delft Design Approach Technology for Bioproducts Solving Complex Problems Responsible Innovation Treatment of Urban Sewage Introduction to Water & Climate Introduction to Solar Energy Aeronautical Engineering Credit Risk Management
- 8. Economics of Information Security (ProfEd) Transport Phenomena: mass, heat and momentum Topology for Condensed Matter: Untying Quantum Knots Framing. Communicating your message in the world of politics Take your Data Analysis to the MAX This year 5 more NEW MOOCs..
- 9. “Registered” MOOC students Total ~ 300.000 Nov’14
- 10. ProfEd, Highschool initiative, blended/flipped, online MSc education Pre-University Calculus Economics of Cybersecurity
- 12. TU Delft Online Learning • Open education • Open courseware (since 2007 – still growing) • 15 MOOCs (since 2013) – 6 reruns, blended MOOCs • Online distance education – MSc (since 2013) • Professional education • 2 studios, 1 mobile studio • 6 instructional designers • Marketing and business development people • Media center (simulations, presentation quality, videos) • TU Delft Online Learning research
- 13. Why Open & Online education? • MIT does it.. • Altruistic reasons • New revenue models • Globalization • Scaling up education • Flexible education (profEd / LLL) • Innovation online <> campus capacity building research
- 14. Questions questions questions • Can open education improve campus education? • How to assess and certify people online? • How to create more inclusive courses? • How to facilitate interaction between students? • Is peer-grading a valid approach? And how to do that? • How to create personalized learning experiences? • Etc.
- 15. Level Topics of interest and stakeholders Strategic University role into the future, new students, reputation, business models, ROI Deans, DelftX, MT Organizational Efficient workflows and procedures, support structure, documentation Course team, instructional designers, NMC, DelftX, marketing Course Course design, pedagogy, use of media, methods and tools, etc. Course team Technology/edX Design of tools for learning, teaching, and data analysis/visualization edX consortium, NMC Students Learner demographics, background, performance, engagement, etc. Students
- 16. Level Topics of interest and stakeholders Strategic University role into the future, new students, reputation, business models, ROI Deans, DelftX, MT Organizational Efficient workflows and procedures, support structure, documentation Course team, instructional designers, NMC, DelftX, marketing Course Course design, pedagogy, use of media, methods and tools, etc. Course team Technology/edX Design of tools for learning, teaching, and data analysis/visualization edX consortium, NMC Students Learner demographics, background, performance, engagement, etc. Students
- 17. How to answer these questions?* Don’t: teaching is an art, not a science Hire researchers Machine learning Open up the research process * limits: few resources and little expertise
- 18. How TUD answers these questions: Don’t: teaching is an art, not a science Hire researchers Machine learning Open up the research process
- 19. Hence… Great opportunities • … to research MOOCs • … and use MOOCs for research But.. • Limited capacity • No/little history in educational research at university Our approach → connect with relevant researchers worldwide to answer our questions
- 20. The currency: access to data + research instruments
- 21. Open research…
- 22. Open research… Facilitating research activities 1. …with relevant parties globally and locally 2. …on a broad range of relevant topics 3. …by giving them access to our data and research instruments 4. …and managing the collaboration workflow 5. …and publishing the results together
- 23. 1a. Who are these ‘relevant parties’? • MOOC Teachers / course teams • MSc. students / PhD’s • Researchers from partner institutes • Researchers globally with expertise on a specific, relevant topic
- 24. 1b. Our current ‘MOOC research network’ • Stanford University - Psychology • UC St. Barbara - Psychology • University of South Australia - Pedagogy, coll. learning • Edinburgh University - Communities of Inquiry • Harvey Mudd College - Gender, diversity • TU München - Gender, diversity • Memphis university - Linguistics • UVA - Methodologies • TU Delft - Privacy/ethics - Course-relevant topics (researching course teams)
- 25. 2a. The ‘broad range’ of relevant topics • Learner demographics, pedagogy, MOOC forums - UNISA • Does self-affirmation of personal values increase engagement of at-risk students (motivation research) – Stanford, UC St. Barbara • (How) does communication in MOOC forums influence social centrality and performance? – Simon Fraser Univ. • How well does the Community of Practice framework fit learning in a MOOC context? What effects of social and teaching presence on social networks and collaboration? – Edinburgh univ. • Why do female students have lower performance in our MOOCs? How to design socially inclusive courses? – Harvey Mudd College & TU München • Are concerns over privacy rising? And what implications does it have for online learning? – TU Delft master student • Etc..
- 26. 2b. How to choose research topics? Research topics Literature, developments Partner interest Teachers / support staff Own research
- 27. 3a. Provide access to data & instruments Data sources Research & evaluation edX subscription data Number of participants, dropouts, location, age, gender, schooling edX student data, learning analytics Progress, tests results, quizzes, exams, etc. Forum participation Video clicks, navigation External data, other media (i.e. Facebook groups, discussions) Social networks, content and discourse analysis Surveys (pre, mid, post) Information about demographics, intention, expectations, satisfaction, media use, etc. Interventions embedded in surveys Interviews / self-assessment: teachers, NMC, DelftX Experiences with workflow and organization Questions and expectations for evaluation
- 28. 3b. Would you be willing... We are doing a lot of exciting research at Delft University of Technology. Would you be willing to receive information about participation in one of our future research projects? This could be a survey, interview, or something entirely different (building a weather balloon?). Let us know by clicking yes below, and we may send an email to you when we need you. ★ Yes (1) ★ No (2)
- 30. 4b. Process and involvement Step Me Partner TUD Research idea! (or problem) x x x Intake x x Provide overview of data & instruments x Signing MoU, Eth. clearance, abstract + meth. x Preparing instruments x x x Deploying instruments x x Collecting and sharing data x Analysis x x x Feedback (papers, guidelines, code, data) x x x
- 31. 4c. Integration… some… • Evaluation feedback loop happens, but locally (iteration of MOOC, blended experiments) • First steps into Research to Practice feedback loop • Teacher training • On-boarding days • Self-assessment tool teachers • Talking with course design teams (example: assessment)
- 32. 5a. Selection of publications • 6 Working papers: 5 course reports + 1 comparative analysis paper • Understanding Social Learning Behaviors (SEFI’14 - Skrypnyk, Hennis, De Vries, 2014) • Learners’ Social Centrality and Performance (EDM’15 – Dowell, Joksimovic, Skrypnyk, Hennis, De Vries, 2015) • An exploratory study in the concerns for information privacy (MSc thesis - Hassing, 2015) • Who is the Learner in the DelftX Engineering MOOCs? (SEFI’15 - Hennis, Skrypnyk, De Vries, 2015) • Diversity in Engineering MOOCs, a first Appraisal (SEFI’15 – Ihsen, Yves, Hennis, De Vries 2015) • Reconsidering Retention in MOOCs: the Relevance of Formal Assessment and Pedagogy (EMOOCS’15 - Skrypnyk, De Vries, Hennis, 2015)
- 33. 5b. Open data? • More difficult… • Privacy concerns • Semi-open data? (only edX partners?)
- 34. Potential of ‘open research’ • Being highly flexible in what you research • Being able to scale the research efforts • Research network and learn from their expertise • Interdisciplinary research
- 35. Current work • Streamline the process • Awareness of research potential • Building capacity for cleaning and preparing data • What about data standards? MOOCDb – xAPI – IMS Caliper etc. • Enabling research across courses / MOOC providers? • More focus on design based research: involvement of teachers • Involvement partners: from hub to community Additional ideas: • Tender for MOOC research • Promote TU Delft research areas • Citizen science / crowdsource research models
Editor's Notes
- View of Delft
- ProfEd → professional education Economics of Cybersecurity edX Highschool initiative Pre-University Calculus Blended and online MSc education Aeronautical Engineering, Architecture, Technology Policy and Management, Industrial Design, EWI, Library, …
- Same reasons as before… but now research is becoming an important reason as well.
- And of course, there are so many different questions you can ask. So how can open and online education improve campus education? How to engage students or prevent them from dropping out? What is effective learning design? Etc.
- Not just questions on the course level, but there are questions on many different levels: from organizational level to the student level.
- OF course with the objective of providing sustainable, effective and inclusive education.
- To sum it up: great opportunities to do research, but limited capacity and little background in educational research. Our approach therefore was to open up the research process and invite researchers, globally, to make use of our data and feed back the results into our course design process.
- And how do we connect? By giving access to our data and research instruments. In other words: we invite them to participate in the research process.
- So our strategy was to open up the research process, and we called it open research. So what is open research?
- Some faces still missing. Also working with Srecko, who has like 8 papers in this conference. No just 4.
- Make visualization
- Usually good to start small with a few survey questions, getting your hands dirty on the data, and then expand to a more elaborate research design. NMC: “ Changing background increases paying attention” > let’s investigate that Gender gap in MOOCs > Does it matter if a woman presents? Level of math around the world: using self-tests to explore the different entry levels around the world. A/B testing with didactic formats, for example letting some students visualize a problem before solving it. Does it matter?
- The evaluation feedback loop is going pretty well: teachers are very interested in the evaluation results, and adapt their courses based on the outcomes. They also experiment and test different blended approaches, and analyse the outcomes and compare it with last year’s outcomes. Maybe not perfect, but showing the right attitude. The research to practice feedback loop is of course more difficult, to translate vague and theoretical ideas to practice, but we are doing it. We found out about a possible influence of assessment placement and strategy on retention rates, and the instructional design team is very interested in that. Also, we are developing teacher training that addresses the opportunity to do research with MOOCs.
- This is just a selection of papers we have published so far. It’s quite significant because Pieter and I were only working at most 2 man-days per week on evaluating and researching online learning.