SlideShare a Scribd company logo
ELI 2016 Annual Meeting Leadership Seminar (Day 1)
Promoting Effective Teaching and Learning
Ecosystems via Research Proven Practice
http://www.educause.edu/eli/events/eli-annual-meeting/program/leadership-seminar
• Tanya Joosten, PhD, Director, eLearning Research and Development,
University of Wisconsin-Milwaukee
• Diane Reddy, PhD, Professor, Psychology, University of Wisconsin-
Milwaukee
• Ray Flemming, PhD, Professor, Psychology, University of Wisconsin-
Milwaukee
Tools for today
eli2016.wikispaces.com
What is the question?
Specifically, what do you want to know or hope to
learn about online, blended or flipped, or
competency-based education?
Let’s develop a draft research question.
eli2016.wikispaces.com
Share
Flickr cc niklaswikstrom
How do we ensure all
students have access to a
quality higher education?
Pathway
to
degree
Online
Blended
and
Flipped
CBE
Access through distance education
Ensuring quality
Flickr vaxzine
F2F gold standard
Flickr xenonbpics
Conduct rigorous, interdisciplinary,
and standardized research
Flicker cc astronomyblog
National Research Center for
Distance Education and Technological
Advancements (DETA)
Tanya Joosten, DETA Co-Director and PI, @tjoosten,
tjoosten@uwm.edu
Diane Reddy, DETA Co-Director and PI, reddy@uwm.edu
Area of investigation
guided by
overarching
research questions
•Gather
pertinent
literature
Formulate
hypotheses to be
tested
• Identify
variables and
their direction
Design study
• Develop measures
and
instrumentation to
test hypotheses
Conduct the study
and analyze results
• Collect data and
analyze results
across
institutions
Research models
Year 1 activity: Host national summit
Desired outcomes
Access
Learning Effectiveness
Satisfaction
Instructional Effectiveness
http://uwm.edu/deta/desired-outcomes/
Research questions
In understanding student success in distance
education, we as a field need to have a
consistent understanding of student success
across programs and institutions, including
identifying shared desired outcomes. In
considering this need, what are the research
questions that will assist us in identifying key
factors impacting these shared desired
outcomes to best enhance the understanding
of distance education across institutions and
populations?
Research questions
Defined
What are the different design components (content, interactivity,
assessments) that impact student learning?
What patterns of behaviors lead to increased student learning for different
populations?
What support structures are critical to providing quality access to online
instruction?
Exploratory (Interpretive)
What are the definitions of success from students’ perspective?
How can we define and measure student success beyond traditional
outcomes?
What is the currency of student learning beyond the existing credit hours?
What are the key components that promote a sustainable and an effective
teaching and learning ecosystem?
Questions
Tools for today
eli2016.wikispaces.com
<insert image/s>
Share
Flickr cc niklaswikstrom
Year 1 activity: Host national summit
Framework of inquiry
In consideration of the research questions
proposed in enhancing our understanding
of the key factors, what variables should
be included in the framework of inquiry to
help us answer these questions
surrounding distance education and
technological advancements?
Framework of inquiry
Shared measures –
Student performance is
based on numerical
representation of grade
converted to a 4.0 scale
received in the course on
assessments and as an
overall grade.
RQs - What are the
different design
components (content,
interactivity, assessments)
that impact student
learning?
Goal: Collect data
Questions
Research toolkit deep dive
uwm.edu/deta/toolkits
Year 2 activity: RFP for pilot
grant awards
Flicker cc yourdon
Research toolkit: Research model
Research toolkit: Guides to research
Research toolkit: Data collection
Research toolkit: Data codebooks
InstitutionsIn
Learners
Instructor
s
Course and
Instructional
Characteristic
s
Program
Characteristic
s
Institutions
In Class
Out of Class
Institutions
Interventions
Inputs/Output Throughputs
InstitutionsIn
Learners
Instructors
Course and
Instructional
Characteristics
Program
Characteristics
Institutions
In Class
Out of Class
Institutions
Interventions
Inputs/Output Throughputs
Framework of inquiry
Framework of inquiry
What is your question?
What are the different design
components (content,
interactivity, assessments) that
impact student learning?
How to measure…
How to measure…
How to measure…
How to measure…
How to measure
How to measure
Diane Reddy and Ray Fleming
One Group
Pre-Post Test
Study
Survey Studies
Qualitative
Studies
Randomized
Controlled
Trials (RCT)
RCT
Systematic
Inquiry
Informal
Reflection
Started with Systematic Inquiry
 Same content & textbook
 Objective performance outcomes
Intervention Comparison
Examined Pre-existing Differences between
Intervention and Comparison Students
Variables that could possibly affect
student success & learning
Checked for Differences in Percent of Racial/Ethnic
Students Showing Gaps in Graduation
60%
48.3%
37.4%
37.2%
White
Hispanic
African American
American Indian/Alaskan
Native
Six-Year Graduation Rates
Undergraduates Enrolling in 4-Year Institutions for the First Time
Ginder, S.A. & Kelly-Reid, J.E. (2013). Enrollment in postsecondary institutions, fall 2012; Financial statistics, fiscal year 2012; Graduation rates,
selected cohorts, 2004-09, and employees in postsecondary institutions, fall 2012: First look (provisional data) (NCES 2013-183). Washington, DC:
National Center for Education Statistics. Retrieved from http://nces.ed.gov/pubs2013/2013183.pdf
Checked for Differences in the Percent of
Pell Grant Eligible Students
Family Income
Top Quartile
Family Income
Bottom Quartile
All students had SAT-equivalent scores in top quartile
82% 44%
Carnevale, A. P., & Strohl, J. (2010). How increasing college access is increasing inequality, and what to do about it. In R. Kahlenberg
(Ed.), Rewarding strivers: Helping low-income students succeed in college (pp. 71-190). New York, NY: The Century Foundation.
Checked for Differences in Percent of
Academically Underprepared Students
ACT composite < 19
Cumulative college GPA < 2.0
ACT College Readiness Benchmarks:
https://www.act.org/solutions/college-career-readiness/college-readiness-benchmarks/
Secured Campus Leaders’ Support
 Waived online course fee
 Set section sizes to be equivalent
 Controlled classroom assignment
 Hid study sections in Schedule of Classes
 Enrolled students
 Recruited students with campus email
n=1128
No YesFinancial hold? n=1121 n=7
Flow Diagram of Recruitment
http://www.consort-statement.org
n=1128
No
No Yes
YesFinancial hold? n=1121
Academic hold? n=1117 n=4
n=7
Flow Diagram of Recruitment
http://www.consort-statement.org
n=1128
No
No
No Yes
Yes
YesFinancial hold? n=1121
Academic hold? n=1117
n=1036 n=81
n=4
n=7
Repeating course?
Flow Diagram of Recruitment
http://www.consort-statement.org
n=1128
No
No
No
No Yes
Yes
Yes
YesFinancial hold? n=1121
Academic hold? n=1117
n=1036
n=1034 n=2
n=81
n=4
n=7
Repeating course?
Under 18?
Flow Diagram of Recruitment
http://www.consort-statement.org
n=1128
No
No
No
No
No Yes
Yes
Yes
Yes
YesFinancial hold? n=1121
Academic hold? n=1117
n=1036
n=1034
n=980 n=54
n=2
n=81
n=4
n=7
Repeating course?
Under 18?
Schedule conflict?
Flow Diagram of Recruitment
http://www.consort-statement.org
n=1128
No
No
No
No
No Yes
Yes
Yes
Yes
YesFinancial hold? n=1121
Academic hold? n=1117
n=1036
n=1034
n=980 n=54
n=2
n=81
n=4
n=7
Repeating course?
Under 18?
Schedule conflict?
Yes n=22Non=958Credit overload?
http://www.consort-statement.org
Flow Diagram of Recruitment
http://www.consort-statement.org
Students Randomized to Instructional Condition
with Equal Distribution of At-Risk Students
Chi, M. T. H., & Wylie, R. (2014). The ICAP Framework: Linking cognitive engagement
to active learning outcomes. Educational Psychologist, 49 (4), 219-243.
DOI: 10.1080/00461520.2014.965823
Intervention
Concept Mastery
&
Proactive Support
Online
Intervention
Concept Mastery
Online
Intervention
Proactive Support
Online
Business-as-
Usual
Student-centered
Instruction
Face-to-Face
Checked for Fidelity throughout Study
 Trained Instructors
 Created Manual & Implementation Rubric
 Checked Fidelity
- classes & course sites
- email correspondence (shared folders)
- conversations with instructors
Other ideas in DETA Toolkit
http://www.oup.com.au/titles/academic/medicine/9780195304633
Nezu, A. M., & Nezu, C. M. (Eds.). (2007). Evidence-based Outcome Research: A Practical
Guide to Conducting Randomized Controlled Trials for Psychosocial Interventions.
Oxford University Press.
Maximizing Student Survey Response Rates
Sent repeated reminders
attended to visual design, content, writing style
conveyed reasons for & importance of completing
survey
sent to class sections & then to individuals
See DETA Toolkit for example email reminders
Tools for today
eli2016.wikispaces.com
What is your research question?
What variables will be examined?
Define.
How will you measure the variable?
Who/How will the data be collected?
Who/How will the data be analyzed?
What tools or support do you need to
complete the research?
wcet15.wikispaces.com
Share
Flickr cc niklaswikstrom
Tanya Joosten, @tjoosten,
tjoosten@uwm.edu
Diane Reddy
reddy@uwm.edu
Ray Flemming
mundo@uwm.edu
http://www.uwm.edu/deta
http://www.slideshare.edu/tjoosten
ELI 2016 Annual Meeting Leadership Seminar (Day 2)
Promoting Effective Teaching and Learning
Ecosystems via Research Proven Practice
http://www.educause.edu/eli/events/eli-annual-meeting/program/leadership-seminar
• Tanya Joosten, PhD, Director, eLearning Research and Development, University of
Wisconsin-Milwaukee
• Laura Pedrick, Executive Director, UWM Online, University of Wisconsin-Milwaukee
• Simone Conceicao, PhD, Professor, Administrative Leadership, University of
Wisconsin-Milwaukee
Learning objectives
 Discuss leadership challenges in gathering
evidence of key factors that impact student
outcomes
 Brainstorm ideas for motivating and
incentivizing faculty and staff
 Explore key factors and effective use cases
in faculty development and instructional
design
 Share effective institutional practices for
research and teaching
 Reflect on and revise research plan
Activity 1
 “Challenge Points”
 What are potential challenges of conducting
research on your campus? What are
potential solutions to these challenges?
 http://eli2016.wikispaces.com/Day+2
Activity 2
 Implementing New Practices
 How can we facilitate implementation of
new instructional and institutional practices?
 http://eli2016.wikispaces.com/Day+2
ELI 2016 Annual Meeting Leadership Seminar (Day 3)
Promoting Effective Teaching and Learning
Ecosystems via Research Proven Practice
http://www.educause.edu/eli/events/eli-annual-meeting/program/leadership-seminar
• Tanya Joosten, PhD, Director, eLearning Research and Development,
University of Wisconsin-Milwaukee
• Roger O. Smith, PhD, Professor, Occupational Science and Technology;
Director, R2D2 Center, University of Wisconsin-Milwaukee
Learning objectives
 Describe key disability factors included in
current or planned projects.
 Identify possible effects of including or
excluding disability factors on the overall
research questions.
 List possible actions for managing
unintended disability related factors in
studies.
Roger Smith
Goal 7:
Include Disability in All Aspects of the DETA Center’s Research
Activities
R2D2 Center
The R2D2 Center performs interdisciplinary basic research investigations, applied research and
development, and innovative instruction related to technology and disability.
http://www.r2d2.uwm.edu/
Access-ED
The ACCESS-ed Website presents products and resources for the implementation of universal
design on post-secondary campuses. In addition to products created by R2D2, the website
compiles and indexes a database of universal design resources from other available
sources to incorporate a comprehensive set of information and resources. http://access-
ed.r2d2.uwm.edu/
Goal 7:
Include Disability in All Aspects of the DETA Center’s Research
Activities
DETA Project Resources in Research Toolkit
• Embedded disability variables and variable sets
• Survey questions with branching elicitation templates
3 Survey Tools (example templates)
• 1) Impairments
• 2) COURSE (course features)
• 3) Interventions.
Activity
 How can eLearning research support
access for students with disabilities?
 http://eli2016.wikispaces.com/Day+3

More Related Content

What's hot

A Workshop: Promoting Student Access and Success Through Research
A Workshop: Promoting Student Access and Success Through ResearchA Workshop: Promoting Student Access and Success Through Research
A Workshop: Promoting Student Access and Success Through Research
Tanya Joosten
 
DETA Research Center Update
DETA Research Center UpdateDETA Research Center Update
DETA Research Center Update
Tanya Joosten
 
Promoting Student Access and Success Through Research
Promoting Student Access and Success Through ResearchPromoting Student Access and Success Through Research
Promoting Student Access and Success Through Research
Tanya Joosten
 
Pedagogical change
Pedagogical changePedagogical change
Pedagogical changeCOHERE2012
 
Engagement in a Disengaged Age
Engagement in a Disengaged AgeEngagement in a Disengaged Age
Engagement in a Disengaged Age
Kenneth Ronkowitz
 
Using A Research Lens To Examine Your COVID-19 Pandemic Response
Using A Research Lens To Examine Your COVID-19 Pandemic ResponseUsing A Research Lens To Examine Your COVID-19 Pandemic Response
Using A Research Lens To Examine Your COVID-19 Pandemic Response
Tanya Joosten
 
Scary World of Letting Go, Snow and Akoh
Scary World of Letting Go, Snow and AkohScary World of Letting Go, Snow and Akoh
Scary World of Letting Go, Snow and AkohCOHERE2012
 
Learning Analytics Community Exchange
Learning Analytics Community ExchangeLearning Analytics Community Exchange
Learning Analytics Community Exchange
Doug Clow
 
What Makes Openness Work?
What Makes Openness Work?What Makes Openness Work?
What Makes Openness Work?
Robert Farrow
 
Presentation 2
Presentation 2Presentation 2
Presentation 2CYUT
 
Cohere accessibility ca_sl_oct-22
Cohere accessibility ca_sl_oct-22Cohere accessibility ca_sl_oct-22
Cohere accessibility ca_sl_oct-22COHERE2012
 
The messy realities of learning and participation in open courses and MOOCs
The messy realities of learning and participation in open courses and MOOCsThe messy realities of learning and participation in open courses and MOOCs
The messy realities of learning and participation in open courses and MOOCs
George Veletsianos
 
Innovating pedagogy
Innovating pedagogyInnovating pedagogy
Innovating pedagogy
Rebecca Ferguson
 
Chen & Hagerman
Chen & HagermanChen & Hagerman
Chen & HagermanCOHERE2012
 
Learning Analytics
Learning AnalyticsLearning Analytics
Learning Analytics
Lee Schlenker
 
Learning Analytics: What is it? Why do it? And how?
Learning Analytics: What is it? Why do it?  And how?Learning Analytics: What is it? Why do it?  And how?
Learning Analytics: What is it? Why do it? And how?
Timothy Harfield
 
Dennis Small
Dennis SmallDennis Small
Dennis Small
Glenn E. Malone, EdD
 
SOLAR - learning analytics, the state of the art
SOLAR - learning analytics, the state of the artSOLAR - learning analytics, the state of the art
SOLAR - learning analytics, the state of the art
Rebecca Ferguson
 
Changing Technology Changing Practice: Empowering Staff and Building Capabili...
Changing Technology Changing Practice: Empowering Staff and Building Capabili...Changing Technology Changing Practice: Empowering Staff and Building Capabili...
Changing Technology Changing Practice: Empowering Staff and Building Capabili...
CITE
 
Give them what they want: Participatory approaches to developing anonymous as...
Give them what they want: Participatory approaches to developing anonymous as...Give them what they want: Participatory approaches to developing anonymous as...
Give them what they want: Participatory approaches to developing anonymous as...
Simon Davis
 

What's hot (20)

A Workshop: Promoting Student Access and Success Through Research
A Workshop: Promoting Student Access and Success Through ResearchA Workshop: Promoting Student Access and Success Through Research
A Workshop: Promoting Student Access and Success Through Research
 
DETA Research Center Update
DETA Research Center UpdateDETA Research Center Update
DETA Research Center Update
 
Promoting Student Access and Success Through Research
Promoting Student Access and Success Through ResearchPromoting Student Access and Success Through Research
Promoting Student Access and Success Through Research
 
Pedagogical change
Pedagogical changePedagogical change
Pedagogical change
 
Engagement in a Disengaged Age
Engagement in a Disengaged AgeEngagement in a Disengaged Age
Engagement in a Disengaged Age
 
Using A Research Lens To Examine Your COVID-19 Pandemic Response
Using A Research Lens To Examine Your COVID-19 Pandemic ResponseUsing A Research Lens To Examine Your COVID-19 Pandemic Response
Using A Research Lens To Examine Your COVID-19 Pandemic Response
 
Scary World of Letting Go, Snow and Akoh
Scary World of Letting Go, Snow and AkohScary World of Letting Go, Snow and Akoh
Scary World of Letting Go, Snow and Akoh
 
Learning Analytics Community Exchange
Learning Analytics Community ExchangeLearning Analytics Community Exchange
Learning Analytics Community Exchange
 
What Makes Openness Work?
What Makes Openness Work?What Makes Openness Work?
What Makes Openness Work?
 
Presentation 2
Presentation 2Presentation 2
Presentation 2
 
Cohere accessibility ca_sl_oct-22
Cohere accessibility ca_sl_oct-22Cohere accessibility ca_sl_oct-22
Cohere accessibility ca_sl_oct-22
 
The messy realities of learning and participation in open courses and MOOCs
The messy realities of learning and participation in open courses and MOOCsThe messy realities of learning and participation in open courses and MOOCs
The messy realities of learning and participation in open courses and MOOCs
 
Innovating pedagogy
Innovating pedagogyInnovating pedagogy
Innovating pedagogy
 
Chen & Hagerman
Chen & HagermanChen & Hagerman
Chen & Hagerman
 
Learning Analytics
Learning AnalyticsLearning Analytics
Learning Analytics
 
Learning Analytics: What is it? Why do it? And how?
Learning Analytics: What is it? Why do it?  And how?Learning Analytics: What is it? Why do it?  And how?
Learning Analytics: What is it? Why do it? And how?
 
Dennis Small
Dennis SmallDennis Small
Dennis Small
 
SOLAR - learning analytics, the state of the art
SOLAR - learning analytics, the state of the artSOLAR - learning analytics, the state of the art
SOLAR - learning analytics, the state of the art
 
Changing Technology Changing Practice: Empowering Staff and Building Capabili...
Changing Technology Changing Practice: Empowering Staff and Building Capabili...Changing Technology Changing Practice: Empowering Staff and Building Capabili...
Changing Technology Changing Practice: Empowering Staff and Building Capabili...
 
Give them what they want: Participatory approaches to developing anonymous as...
Give them what they want: Participatory approaches to developing anonymous as...Give them what they want: Participatory approaches to developing anonymous as...
Give them what they want: Participatory approaches to developing anonymous as...
 

Similar to Promoting Effective Teaching and Learning Ecosystems via Research Proven Practice

Team 5 (narrated ppt)
Team 5 (narrated ppt)Team 5 (narrated ppt)
Team 5 (narrated ppt)acefellowspro
 
Online Learning at Illinois Computing Educators' Conference
Online Learning at Illinois Computing Educators' ConferenceOnline Learning at Illinois Computing Educators' Conference
Online Learning at Illinois Computing Educators' Conference
Jeffrey Hunt
 
A Quiet Crisis
A Quiet CrisisA Quiet Crisis
A Quiet Crisis
Diana Laboy-Rush
 
Credit Flexibility Presentation by Sarah Luchs
Credit Flexibility Presentation by Sarah LuchsCredit Flexibility Presentation by Sarah Luchs
Credit Flexibility Presentation by Sarah Luchs
Eric Calvert
 
Mike Mullen update on UK Retention, Spring 2011, University Committee on Acad...
Mike Mullen update on UK Retention, Spring 2011, University Committee on Acad...Mike Mullen update on UK Retention, Spring 2011, University Committee on Acad...
Mike Mullen update on UK Retention, Spring 2011, University Committee on Acad...
University of Kentucky
 
DuPage Online Meeting
DuPage Online MeetingDuPage Online Meeting
DuPage Online MeetingJeffrey Hunt
 
The power of learning analytics to measure learning gains: an OU, Surrey and ...
The power of learning analytics to measure learning gains: an OU, Surrey and ...The power of learning analytics to measure learning gains: an OU, Surrey and ...
The power of learning analytics to measure learning gains: an OU, Surrey and ...
Bart Rienties
 
Virtual Learning Policy Consideration (iNacol)
Virtual Learning Policy Consideration (iNacol)Virtual Learning Policy Consideration (iNacol)
Virtual Learning Policy Consideration (iNacol)
Blacketor Consultants, LLC
 
China delegation 06.12.17 v1
China delegation 06.12.17 v1China delegation 06.12.17 v1
China delegation 06.12.17 v1
Louis Soares
 
ICSN 2010
ICSN 2010ICSN 2010
ICSN 2010
Kerry Rice
 
La Bonte Quality In Dl Apr09
La Bonte Quality In Dl Apr09La Bonte Quality In Dl Apr09
La Bonte Quality In Dl Apr09
LaBonte Randy
 
Student Retention Foothill College Roundtable
Student Retention Foothill College RoundtableStudent Retention Foothill College Roundtable
Student Retention Foothill College Roundtable
Robert Cormia
 
Disrupted Futures 2023 | Wisconsin's career guidance project
Disrupted Futures 2023 | Wisconsin's career guidance projectDisrupted Futures 2023 | Wisconsin's career guidance project
Disrupted Futures 2023 | Wisconsin's career guidance project
EduSkills OECD
 
Standing at the Intersection of College Choice and Competency-based Education...
Standing at the Intersection of College Choice and Competency-based Education...Standing at the Intersection of College Choice and Competency-based Education...
Standing at the Intersection of College Choice and Competency-based Education...
American Public University System
 
Blackboard World 2011 -- Online Learning
Blackboard World 2011 -- Online LearningBlackboard World 2011 -- Online Learning
Blackboard World 2011 -- Online Learning
Jeffrey Hunt
 
Systematic Review And Environmental Scan On Digital Learning At Minority Serv...
Systematic Review And Environmental Scan On Digital Learning At Minority Serv...Systematic Review And Environmental Scan On Digital Learning At Minority Serv...
Systematic Review And Environmental Scan On Digital Learning At Minority Serv...
Tanya Joosten
 
Affective behaviour cognition learning gains project presentation
Affective behaviour cognition learning gains project presentationAffective behaviour cognition learning gains project presentation
Affective behaviour cognition learning gains project presentation
Bart Rienties
 

Similar to Promoting Effective Teaching and Learning Ecosystems via Research Proven Practice (20)

Team 5 (narrated ppt)
Team 5 (narrated ppt)Team 5 (narrated ppt)
Team 5 (narrated ppt)
 
Online Learning at Illinois Computing Educators' Conference
Online Learning at Illinois Computing Educators' ConferenceOnline Learning at Illinois Computing Educators' Conference
Online Learning at Illinois Computing Educators' Conference
 
commrelatrainPP
commrelatrainPPcommrelatrainPP
commrelatrainPP
 
A Quiet Crisis
A Quiet CrisisA Quiet Crisis
A Quiet Crisis
 
Credit Flexibility Presentation by Sarah Luchs
Credit Flexibility Presentation by Sarah LuchsCredit Flexibility Presentation by Sarah Luchs
Credit Flexibility Presentation by Sarah Luchs
 
Mike Mullen update on UK Retention, Spring 2011, University Committee on Acad...
Mike Mullen update on UK Retention, Spring 2011, University Committee on Acad...Mike Mullen update on UK Retention, Spring 2011, University Committee on Acad...
Mike Mullen update on UK Retention, Spring 2011, University Committee on Acad...
 
DuPage Online Meeting
DuPage Online MeetingDuPage Online Meeting
DuPage Online Meeting
 
The power of learning analytics to measure learning gains: an OU, Surrey and ...
The power of learning analytics to measure learning gains: an OU, Surrey and ...The power of learning analytics to measure learning gains: an OU, Surrey and ...
The power of learning analytics to measure learning gains: an OU, Surrey and ...
 
Virtual Learning Policy Consideration (iNacol)
Virtual Learning Policy Consideration (iNacol)Virtual Learning Policy Consideration (iNacol)
Virtual Learning Policy Consideration (iNacol)
 
China delegation 06.12.17 v1
China delegation 06.12.17 v1China delegation 06.12.17 v1
China delegation 06.12.17 v1
 
NASPA AnP 2014
NASPA AnP 2014NASPA AnP 2014
NASPA AnP 2014
 
ICSN 2010
ICSN 2010ICSN 2010
ICSN 2010
 
La Bonte Quality In Dl Apr09
La Bonte Quality In Dl Apr09La Bonte Quality In Dl Apr09
La Bonte Quality In Dl Apr09
 
Student Retention Foothill College Roundtable
Student Retention Foothill College RoundtableStudent Retention Foothill College Roundtable
Student Retention Foothill College Roundtable
 
Disrupted Futures 2023 | Wisconsin's career guidance project
Disrupted Futures 2023 | Wisconsin's career guidance projectDisrupted Futures 2023 | Wisconsin's career guidance project
Disrupted Futures 2023 | Wisconsin's career guidance project
 
Standing at the Intersection of College Choice and Competency-based Education...
Standing at the Intersection of College Choice and Competency-based Education...Standing at the Intersection of College Choice and Competency-based Education...
Standing at the Intersection of College Choice and Competency-based Education...
 
Blackboard World 2011 -- Online Learning
Blackboard World 2011 -- Online LearningBlackboard World 2011 -- Online Learning
Blackboard World 2011 -- Online Learning
 
Sebastian: VCCS eLearning Plan
Sebastian: VCCS eLearning PlanSebastian: VCCS eLearning Plan
Sebastian: VCCS eLearning Plan
 
Systematic Review And Environmental Scan On Digital Learning At Minority Serv...
Systematic Review And Environmental Scan On Digital Learning At Minority Serv...Systematic Review And Environmental Scan On Digital Learning At Minority Serv...
Systematic Review And Environmental Scan On Digital Learning At Minority Serv...
 
Affective behaviour cognition learning gains project presentation
Affective behaviour cognition learning gains project presentationAffective behaviour cognition learning gains project presentation
Affective behaviour cognition learning gains project presentation
 

Recently uploaded

一比一原版(UCSB毕业证)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB毕业证)圣芭芭拉分校毕业证如何办理一比一原版(UCSB毕业证)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB毕业证)圣芭芭拉分校毕业证如何办理
bbeucd
 
US Economic Outlook - Being Decided - M Capital Group August 2021.pdf
US Economic Outlook - Being Decided - M Capital Group August 2021.pdfUS Economic Outlook - Being Decided - M Capital Group August 2021.pdf
US Economic Outlook - Being Decided - M Capital Group August 2021.pdf
pchutichetpong
 
what is a pi whale and how to access one.
what is a pi whale and how to access one.what is a pi whale and how to access one.
what is a pi whale and how to access one.
DOT TECH
 
BYD SWOT Analysis and In-Depth Insights 2024.pptx
BYD SWOT Analysis and In-Depth Insights 2024.pptxBYD SWOT Analysis and In-Depth Insights 2024.pptx
BYD SWOT Analysis and In-Depth Insights 2024.pptx
mikemetalprod
 
Instant Issue Debit Cards - High School Spirit
Instant Issue Debit Cards - High School SpiritInstant Issue Debit Cards - High School Spirit
Instant Issue Debit Cards - High School Spirit
egoetzinger
 
what is the future of Pi Network currency.
what is the future of Pi Network currency.what is the future of Pi Network currency.
what is the future of Pi Network currency.
DOT TECH
 
Instant Issue Debit Cards - School Designs
Instant Issue Debit Cards - School DesignsInstant Issue Debit Cards - School Designs
Instant Issue Debit Cards - School Designs
egoetzinger
 
This assessment plan proposal is to outline a structured approach to evaluati...
This assessment plan proposal is to outline a structured approach to evaluati...This assessment plan proposal is to outline a structured approach to evaluati...
This assessment plan proposal is to outline a structured approach to evaluati...
lamluanvan.net Viết thuê luận văn
 
一比一原版(GWU,GW毕业证)加利福尼亚大学|尔湾分校毕业证如何办理
一比一原版(GWU,GW毕业证)加利福尼亚大学|尔湾分校毕业证如何办理一比一原版(GWU,GW毕业证)加利福尼亚大学|尔湾分校毕业证如何办理
一比一原版(GWU,GW毕业证)加利福尼亚大学|尔湾分校毕业证如何办理
obyzuk
 
What website can I sell pi coins securely.
What website can I sell pi coins securely.What website can I sell pi coins securely.
What website can I sell pi coins securely.
DOT TECH
 
Webinar Exploring DORA for Fintechs - Simont Braun
Webinar Exploring DORA for Fintechs - Simont BraunWebinar Exploring DORA for Fintechs - Simont Braun
Webinar Exploring DORA for Fintechs - Simont Braun
FinTech Belgium
 
Donald Trump Presentation and his life.pptx
Donald Trump Presentation and his life.pptxDonald Trump Presentation and his life.pptx
Donald Trump Presentation and his life.pptx
SerdarHudaykuliyew
 
The secret way to sell pi coins effortlessly.
The secret way to sell pi coins effortlessly.The secret way to sell pi coins effortlessly.
The secret way to sell pi coins effortlessly.
DOT TECH
 
how can i use my minded pi coins I need some funds.
how can i use my minded pi coins I need some funds.how can i use my minded pi coins I need some funds.
how can i use my minded pi coins I need some funds.
DOT TECH
 
Abhay Bhutada Leads Poonawalla Fincorp To Record Low NPA And Unprecedented Gr...
Abhay Bhutada Leads Poonawalla Fincorp To Record Low NPA And Unprecedented Gr...Abhay Bhutada Leads Poonawalla Fincorp To Record Low NPA And Unprecedented Gr...
Abhay Bhutada Leads Poonawalla Fincorp To Record Low NPA And Unprecedented Gr...
Vighnesh Shashtri
 
The Role of Non-Banking Financial Companies (NBFCs)
The Role of Non-Banking Financial Companies (NBFCs)The Role of Non-Banking Financial Companies (NBFCs)
The Role of Non-Banking Financial Companies (NBFCs)
nickysharmasucks
 
Analyzing the instability of equilibrium in thr harrod domar model
Analyzing the instability of equilibrium in thr harrod domar modelAnalyzing the instability of equilibrium in thr harrod domar model
Analyzing the instability of equilibrium in thr harrod domar model
ManthanBhardwaj4
 
what is the best method to sell pi coins in 2024
what is the best method to sell pi coins in 2024what is the best method to sell pi coins in 2024
what is the best method to sell pi coins in 2024
DOT TECH
 
when will pi network coin be available on crypto exchange.
when will pi network coin be available on crypto exchange.when will pi network coin be available on crypto exchange.
when will pi network coin be available on crypto exchange.
DOT TECH
 
how to swap pi coins to foreign currency withdrawable.
how to swap pi coins to foreign currency withdrawable.how to swap pi coins to foreign currency withdrawable.
how to swap pi coins to foreign currency withdrawable.
DOT TECH
 

Recently uploaded (20)

一比一原版(UCSB毕业证)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB毕业证)圣芭芭拉分校毕业证如何办理一比一原版(UCSB毕业证)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB毕业证)圣芭芭拉分校毕业证如何办理
 
US Economic Outlook - Being Decided - M Capital Group August 2021.pdf
US Economic Outlook - Being Decided - M Capital Group August 2021.pdfUS Economic Outlook - Being Decided - M Capital Group August 2021.pdf
US Economic Outlook - Being Decided - M Capital Group August 2021.pdf
 
what is a pi whale and how to access one.
what is a pi whale and how to access one.what is a pi whale and how to access one.
what is a pi whale and how to access one.
 
BYD SWOT Analysis and In-Depth Insights 2024.pptx
BYD SWOT Analysis and In-Depth Insights 2024.pptxBYD SWOT Analysis and In-Depth Insights 2024.pptx
BYD SWOT Analysis and In-Depth Insights 2024.pptx
 
Instant Issue Debit Cards - High School Spirit
Instant Issue Debit Cards - High School SpiritInstant Issue Debit Cards - High School Spirit
Instant Issue Debit Cards - High School Spirit
 
what is the future of Pi Network currency.
what is the future of Pi Network currency.what is the future of Pi Network currency.
what is the future of Pi Network currency.
 
Instant Issue Debit Cards - School Designs
Instant Issue Debit Cards - School DesignsInstant Issue Debit Cards - School Designs
Instant Issue Debit Cards - School Designs
 
This assessment plan proposal is to outline a structured approach to evaluati...
This assessment plan proposal is to outline a structured approach to evaluati...This assessment plan proposal is to outline a structured approach to evaluati...
This assessment plan proposal is to outline a structured approach to evaluati...
 
一比一原版(GWU,GW毕业证)加利福尼亚大学|尔湾分校毕业证如何办理
一比一原版(GWU,GW毕业证)加利福尼亚大学|尔湾分校毕业证如何办理一比一原版(GWU,GW毕业证)加利福尼亚大学|尔湾分校毕业证如何办理
一比一原版(GWU,GW毕业证)加利福尼亚大学|尔湾分校毕业证如何办理
 
What website can I sell pi coins securely.
What website can I sell pi coins securely.What website can I sell pi coins securely.
What website can I sell pi coins securely.
 
Webinar Exploring DORA for Fintechs - Simont Braun
Webinar Exploring DORA for Fintechs - Simont BraunWebinar Exploring DORA for Fintechs - Simont Braun
Webinar Exploring DORA for Fintechs - Simont Braun
 
Donald Trump Presentation and his life.pptx
Donald Trump Presentation and his life.pptxDonald Trump Presentation and his life.pptx
Donald Trump Presentation and his life.pptx
 
The secret way to sell pi coins effortlessly.
The secret way to sell pi coins effortlessly.The secret way to sell pi coins effortlessly.
The secret way to sell pi coins effortlessly.
 
how can i use my minded pi coins I need some funds.
how can i use my minded pi coins I need some funds.how can i use my minded pi coins I need some funds.
how can i use my minded pi coins I need some funds.
 
Abhay Bhutada Leads Poonawalla Fincorp To Record Low NPA And Unprecedented Gr...
Abhay Bhutada Leads Poonawalla Fincorp To Record Low NPA And Unprecedented Gr...Abhay Bhutada Leads Poonawalla Fincorp To Record Low NPA And Unprecedented Gr...
Abhay Bhutada Leads Poonawalla Fincorp To Record Low NPA And Unprecedented Gr...
 
The Role of Non-Banking Financial Companies (NBFCs)
The Role of Non-Banking Financial Companies (NBFCs)The Role of Non-Banking Financial Companies (NBFCs)
The Role of Non-Banking Financial Companies (NBFCs)
 
Analyzing the instability of equilibrium in thr harrod domar model
Analyzing the instability of equilibrium in thr harrod domar modelAnalyzing the instability of equilibrium in thr harrod domar model
Analyzing the instability of equilibrium in thr harrod domar model
 
what is the best method to sell pi coins in 2024
what is the best method to sell pi coins in 2024what is the best method to sell pi coins in 2024
what is the best method to sell pi coins in 2024
 
when will pi network coin be available on crypto exchange.
when will pi network coin be available on crypto exchange.when will pi network coin be available on crypto exchange.
when will pi network coin be available on crypto exchange.
 
how to swap pi coins to foreign currency withdrawable.
how to swap pi coins to foreign currency withdrawable.how to swap pi coins to foreign currency withdrawable.
how to swap pi coins to foreign currency withdrawable.
 

Promoting Effective Teaching and Learning Ecosystems via Research Proven Practice

  • 1. ELI 2016 Annual Meeting Leadership Seminar (Day 1) Promoting Effective Teaching and Learning Ecosystems via Research Proven Practice http://www.educause.edu/eli/events/eli-annual-meeting/program/leadership-seminar • Tanya Joosten, PhD, Director, eLearning Research and Development, University of Wisconsin-Milwaukee • Diane Reddy, PhD, Professor, Psychology, University of Wisconsin- Milwaukee • Ray Flemming, PhD, Professor, Psychology, University of Wisconsin- Milwaukee
  • 3. What is the question? Specifically, what do you want to know or hope to learn about online, blended or flipped, or competency-based education? Let’s develop a draft research question. eli2016.wikispaces.com
  • 4.
  • 6. How do we ensure all students have access to a quality higher education?
  • 10. Conduct rigorous, interdisciplinary, and standardized research Flicker cc astronomyblog
  • 11. National Research Center for Distance Education and Technological Advancements (DETA) Tanya Joosten, DETA Co-Director and PI, @tjoosten, tjoosten@uwm.edu Diane Reddy, DETA Co-Director and PI, reddy@uwm.edu
  • 12. Area of investigation guided by overarching research questions •Gather pertinent literature Formulate hypotheses to be tested • Identify variables and their direction Design study • Develop measures and instrumentation to test hypotheses Conduct the study and analyze results • Collect data and analyze results across institutions Research models
  • 13. Year 1 activity: Host national summit
  • 14. Desired outcomes Access Learning Effectiveness Satisfaction Instructional Effectiveness http://uwm.edu/deta/desired-outcomes/
  • 16. In understanding student success in distance education, we as a field need to have a consistent understanding of student success across programs and institutions, including identifying shared desired outcomes. In considering this need, what are the research questions that will assist us in identifying key factors impacting these shared desired outcomes to best enhance the understanding of distance education across institutions and populations?
  • 17. Research questions Defined What are the different design components (content, interactivity, assessments) that impact student learning? What patterns of behaviors lead to increased student learning for different populations? What support structures are critical to providing quality access to online instruction? Exploratory (Interpretive) What are the definitions of success from students’ perspective? How can we define and measure student success beyond traditional outcomes? What is the currency of student learning beyond the existing credit hours? What are the key components that promote a sustainable and an effective teaching and learning ecosystem?
  • 22. Year 1 activity: Host national summit
  • 24. In consideration of the research questions proposed in enhancing our understanding of the key factors, what variables should be included in the framework of inquiry to help us answer these questions surrounding distance education and technological advancements?
  • 26. Shared measures – Student performance is based on numerical representation of grade converted to a 4.0 scale received in the course on assessments and as an overall grade. RQs - What are the different design components (content, interactivity, assessments) that impact student learning?
  • 29.
  • 30. Research toolkit deep dive uwm.edu/deta/toolkits
  • 31. Year 2 activity: RFP for pilot grant awards Flicker cc yourdon
  • 36.
  • 41. What is your question? What are the different design components (content, interactivity, assessments) that impact student learning?
  • 48. Diane Reddy and Ray Fleming
  • 49. One Group Pre-Post Test Study Survey Studies Qualitative Studies Randomized Controlled Trials (RCT)
  • 51. Started with Systematic Inquiry  Same content & textbook  Objective performance outcomes Intervention Comparison
  • 52. Examined Pre-existing Differences between Intervention and Comparison Students Variables that could possibly affect student success & learning
  • 53. Checked for Differences in Percent of Racial/Ethnic Students Showing Gaps in Graduation 60% 48.3% 37.4% 37.2% White Hispanic African American American Indian/Alaskan Native Six-Year Graduation Rates Undergraduates Enrolling in 4-Year Institutions for the First Time Ginder, S.A. & Kelly-Reid, J.E. (2013). Enrollment in postsecondary institutions, fall 2012; Financial statistics, fiscal year 2012; Graduation rates, selected cohorts, 2004-09, and employees in postsecondary institutions, fall 2012: First look (provisional data) (NCES 2013-183). Washington, DC: National Center for Education Statistics. Retrieved from http://nces.ed.gov/pubs2013/2013183.pdf
  • 54. Checked for Differences in the Percent of Pell Grant Eligible Students Family Income Top Quartile Family Income Bottom Quartile All students had SAT-equivalent scores in top quartile 82% 44% Carnevale, A. P., & Strohl, J. (2010). How increasing college access is increasing inequality, and what to do about it. In R. Kahlenberg (Ed.), Rewarding strivers: Helping low-income students succeed in college (pp. 71-190). New York, NY: The Century Foundation.
  • 55. Checked for Differences in Percent of Academically Underprepared Students ACT composite < 19 Cumulative college GPA < 2.0 ACT College Readiness Benchmarks: https://www.act.org/solutions/college-career-readiness/college-readiness-benchmarks/
  • 56. Secured Campus Leaders’ Support  Waived online course fee  Set section sizes to be equivalent  Controlled classroom assignment  Hid study sections in Schedule of Classes  Enrolled students  Recruited students with campus email
  • 57. n=1128 No YesFinancial hold? n=1121 n=7 Flow Diagram of Recruitment http://www.consort-statement.org
  • 58. n=1128 No No Yes YesFinancial hold? n=1121 Academic hold? n=1117 n=4 n=7 Flow Diagram of Recruitment http://www.consort-statement.org
  • 59. n=1128 No No No Yes Yes YesFinancial hold? n=1121 Academic hold? n=1117 n=1036 n=81 n=4 n=7 Repeating course? Flow Diagram of Recruitment http://www.consort-statement.org
  • 60. n=1128 No No No No Yes Yes Yes YesFinancial hold? n=1121 Academic hold? n=1117 n=1036 n=1034 n=2 n=81 n=4 n=7 Repeating course? Under 18? Flow Diagram of Recruitment http://www.consort-statement.org
  • 61. n=1128 No No No No No Yes Yes Yes Yes YesFinancial hold? n=1121 Academic hold? n=1117 n=1036 n=1034 n=980 n=54 n=2 n=81 n=4 n=7 Repeating course? Under 18? Schedule conflict? Flow Diagram of Recruitment http://www.consort-statement.org
  • 62. n=1128 No No No No No Yes Yes Yes Yes YesFinancial hold? n=1121 Academic hold? n=1117 n=1036 n=1034 n=980 n=54 n=2 n=81 n=4 n=7 Repeating course? Under 18? Schedule conflict? Yes n=22Non=958Credit overload? http://www.consort-statement.org Flow Diagram of Recruitment
  • 64. Students Randomized to Instructional Condition with Equal Distribution of At-Risk Students Chi, M. T. H., & Wylie, R. (2014). The ICAP Framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49 (4), 219-243. DOI: 10.1080/00461520.2014.965823 Intervention Concept Mastery & Proactive Support Online Intervention Concept Mastery Online Intervention Proactive Support Online Business-as- Usual Student-centered Instruction Face-to-Face
  • 65. Checked for Fidelity throughout Study  Trained Instructors  Created Manual & Implementation Rubric  Checked Fidelity - classes & course sites - email correspondence (shared folders) - conversations with instructors Other ideas in DETA Toolkit
  • 66. http://www.oup.com.au/titles/academic/medicine/9780195304633 Nezu, A. M., & Nezu, C. M. (Eds.). (2007). Evidence-based Outcome Research: A Practical Guide to Conducting Randomized Controlled Trials for Psychosocial Interventions. Oxford University Press.
  • 67. Maximizing Student Survey Response Rates Sent repeated reminders attended to visual design, content, writing style conveyed reasons for & importance of completing survey sent to class sections & then to individuals See DETA Toolkit for example email reminders
  • 69. What is your research question? What variables will be examined? Define. How will you measure the variable? Who/How will the data be collected? Who/How will the data be analyzed? What tools or support do you need to complete the research? wcet15.wikispaces.com
  • 71.
  • 72. Tanya Joosten, @tjoosten, tjoosten@uwm.edu Diane Reddy reddy@uwm.edu Ray Flemming mundo@uwm.edu http://www.uwm.edu/deta http://www.slideshare.edu/tjoosten
  • 73. ELI 2016 Annual Meeting Leadership Seminar (Day 2) Promoting Effective Teaching and Learning Ecosystems via Research Proven Practice http://www.educause.edu/eli/events/eli-annual-meeting/program/leadership-seminar • Tanya Joosten, PhD, Director, eLearning Research and Development, University of Wisconsin-Milwaukee • Laura Pedrick, Executive Director, UWM Online, University of Wisconsin-Milwaukee • Simone Conceicao, PhD, Professor, Administrative Leadership, University of Wisconsin-Milwaukee
  • 74. Learning objectives  Discuss leadership challenges in gathering evidence of key factors that impact student outcomes  Brainstorm ideas for motivating and incentivizing faculty and staff  Explore key factors and effective use cases in faculty development and instructional design
  • 75.  Share effective institutional practices for research and teaching  Reflect on and revise research plan
  • 76. Activity 1  “Challenge Points”  What are potential challenges of conducting research on your campus? What are potential solutions to these challenges?  http://eli2016.wikispaces.com/Day+2
  • 77. Activity 2  Implementing New Practices  How can we facilitate implementation of new instructional and institutional practices?  http://eli2016.wikispaces.com/Day+2
  • 78. ELI 2016 Annual Meeting Leadership Seminar (Day 3) Promoting Effective Teaching and Learning Ecosystems via Research Proven Practice http://www.educause.edu/eli/events/eli-annual-meeting/program/leadership-seminar • Tanya Joosten, PhD, Director, eLearning Research and Development, University of Wisconsin-Milwaukee • Roger O. Smith, PhD, Professor, Occupational Science and Technology; Director, R2D2 Center, University of Wisconsin-Milwaukee
  • 79. Learning objectives  Describe key disability factors included in current or planned projects.  Identify possible effects of including or excluding disability factors on the overall research questions.  List possible actions for managing unintended disability related factors in studies.
  • 81. Goal 7: Include Disability in All Aspects of the DETA Center’s Research Activities R2D2 Center The R2D2 Center performs interdisciplinary basic research investigations, applied research and development, and innovative instruction related to technology and disability. http://www.r2d2.uwm.edu/ Access-ED The ACCESS-ed Website presents products and resources for the implementation of universal design on post-secondary campuses. In addition to products created by R2D2, the website compiles and indexes a database of universal design resources from other available sources to incorporate a comprehensive set of information and resources. http://access- ed.r2d2.uwm.edu/
  • 82. Goal 7: Include Disability in All Aspects of the DETA Center’s Research Activities DETA Project Resources in Research Toolkit • Embedded disability variables and variable sets • Survey questions with branching elicitation templates 3 Survey Tools (example templates) • 1) Impairments • 2) COURSE (course features) • 3) Interventions.
  • 83. Activity  How can eLearning research support access for students with disabilities?  http://eli2016.wikispaces.com/Day+3

Editor's Notes

  1. Tanya
  2. Identify the problem, specifically what you hope to learn through researching your online program- develop a draft research question
  3. So, we know one way we can provide access is by offering multiple pathways or options to obtaining a degree. Student are allowed to push time Courses from k-12 and post secondary/higher ed are based on a system of time and credit hours. Yet, as we live in a time that is much more hectic, with greater demands on our time due to changes in our society (two income households, more students, if not all, are working students, more undergraduates coming with families and jobs), we see that technology can help us become more efficient and overcome some of these barriers. Mediated learning opportunities, such as CBE, Online, and Blended, offer new pathways to degree providing students an access to an education that provides them more flexibility to manage their time and their lives. However, these alternate forms of mediated learning have been questions through the decades, which has often raised the question of quality.
  4. F2F to online Mode comparison research indicates that there was little difference in student satisfaction (Allen et al., 2002; Castle & McGuide, 2010; Lim, Morris, & Kupritz, 2006) and learning (Allen et al., 2004; Park & Gemino, 2001). However, as Dziuban and Picciano (2015) discuss the “no significant difference phenomenon” refering to Roberts (2007) where they allude to the idea that research in online learning as “a kind of collective amnesia surrounds changes that happened over a more distant time frame. We tend to trust what we have seen for ourselves and thus dismiss events that occurred in the more distant past” (p. 13). Some researchers in disciplines newer to online learning tend to replicate the same studies with very little new to contribute to the understanding of the phenomenon. Moore and Kearsley (2011) mention “…one of the major threats to good practice as well as to good scholarship in distance education is the common failure of the newcomers to the field to understand what a depth of knowledge there is” (p. xvi). More recently, some practitioners and researchers realized to better understand online learning they need to look more at process variables and build off of the previous decades of research.
  5. Ensuring quality isn’t about comparing f2f and online…ensure quality is more. We know that F2F is not the gold standard either. There is good f2f and bad f2f, good and bad online…now, let’s ensure all education is good, quality education. The online medium brings many new questions to an instructor. It is no longer about just how do I put my lectures and exams online. Well, those probably should have never been the questions. The transformation becomes just as much, if not more, about the pedagogy and social processes in the classroom as it does about the technology.
  6. …to identify outcomes and influences on all students. We want to identify key factors that are impacting our desired outcomes. Specifically, we look to examine the effectiveness of education interventions across courses, programs, and institutions We would like to follow WWC standards by conducting Randomized Controlled Trials and Quasi-Experimental Designs when possible
  7. Thanks for coming….
  8. We need to speak the same language… We will accomplish our objectives through 7 goals and an array of activities by developing a research infrastructure that considers practice, instructional and research, and technological consideration in developing a national collaborative for DE research recognizing, again, the interrelatedness of research and pedagogy. See p. 8 Table 11 Provide Justification for year 1 goal Discuss each activity for year 1 goal 2. Determine desired outcomes 3. Identify research questions to facilitate cross-institutional research 4. Establish framework of inquiry, including common variables Moving forward: 5. Formulate measures 6. Establish research instrumentation and methodology Transition: Now, let’s talk more specifically about our first activity as we are launching this grant.
  9. Once of our first activities was to bring experts from around the country to assist us in developing a research model impacting our research agenda that will be conducted in year 2 of the grant. We want to bring some coherency to the field of distance education research and use that foundation to guide our research as part of this grant. We hope our work today will assist us in developing research toolkits that will drive cross-institutional distance education research in year 2 of our grant. These research toolkits will have priority research questions to be answered, a framework of inquiry including variables and shared measures, guidance in methodology including instrumentation, data mining techniques, and sample syntax, and more.
  10. As part of the grant, we think of success as providing students access to quality learning, while leaving them satisfied with their experiences. Desired Outcomes Access All learners who wish to learn online can access learning in a wide array of programs and courses,1 particularly underrepresented, those with disabilities and minorities.2 An essential component in distance education is a comprehensive infrastructure for learning that provides all individuals with the resources they need when and where they are needed. The underlying principle is that infrastructure includes people, instructional resources, processes, learning resources, policies, broadband, hardware, and software. It brings state-of-the art technology into learning to enable, motivate, and inspire all students, regardless of background, languages, or disabilities, to achieve.4 Data can be collected by examining administrative and technical infrastructure, which provides access to all prospective and enrolled learners. Access quality metrics are used for information dissemination, learning resource delivery, and tutoring services.1 Other possibilities include data gathered from student information systems, from student perception surveys, or objective accessibility ratings of online courses and programs. Learning effectiveness Learning effectiveness indicates a demonstration that learning outcomes were met or exceeded standards.1 This includes areas of study with research outcomes focusing on student success in achieving learning outcomes2 and other potential indicators of achievement (success, failure, achievement gains, academic achievement, improvement).3 Moreover, learning effectiveness could also include topics of retention (of content) and retention in a course (sometimes called attrition) or program (degree completion). Typically data are gathered through direct assessment of student learning (e.g. overall grades, exam grades, or other assessments), faculty perception surveys, faculty interviews comparing learning effectiveness in delivery modes, and student focus groups or interviews measuring learning gains.1 Additionally, requests for new and better ways to measure what matters include concurrent data collection. Here, focusing on diagnosing strengths and weakness during the course of learning provides the opportunity for more immediate improved student performance. Furthermore, these technology-based assessments provide the opportunity to allow data to drive decisions on the basis of what is best for each and every student based on their unique attributes and interactivity in class.4 Other possibilities include data gathered from student information systems or from student perception surveys. Satisfaction Faculty are pleased with teaching online, citing appreciation and happiness. Students are pleased with their experiences in learning online, including interaction with instructors and peers, learning outcomes that match expectations, services, and orientation.1 Faculty and student surveys can indicate equal or growing satisfaction to traditional forms of learning. Other metrics can include repeat teaching of online courses by individual faculty and increase in percentage of faculty teaching online showing growing endorsement. Qualitative methods can include interviews, focus groups, testimonials with faculty, staff (including advisors and tutors), and/or students.1 Instructional effectiveness Instructional effectiveness indicates the quality of education meets program, institutional, and national standards.1 The focus is on what and how we teach to match what people need to know, how they learn, where and when they will learn, and who needs to learn.4 The areas of study might include instructional improvement, program effectiveness, administrator effectiveness, curriculum evaluation, educational quality, outcomes of education programs, and instructional media.3 Additionally, instructional effectiveness is not limited to instruction provided inside the classroom, but extends itself to instructional support or supplemental instruction and guidance provided through institutional services or through staff and individuals outside of the classroom. Traditionally, as in face-to-face delivered courses, student ratings of instructional effectiveness are collected. However, typically these standards in distance education and online learning are communicated in a course or program rubric (e.g., UC Chico, QM) which is administered through an objective rating of a course or program in addition to traditional methods. Recent work looks to gather this data through student perceptions of instructional effectiveness through course and program rubrics converted to student surveys. Other possibilities include objective ratings of online course and program design and instructional delivery. References: 1. Online Learning Consortium, 5 Pillars 2. U.S. Department of Education, Application for Grants 3. What Works Clearinghouse 4. National Ed Tech Plan, U.S. Department of Education
  11. Provide Justification for year 1 goal Discuss each activity for year 1 goal 1. Host national summit 2. Determine desired outcomes 3. Establish framework of inquiry 4. Formulate measures 5. Establish research instrumentation Transition: Now, let’s talk more specifically about our first activity as we are launching this grant.
  12. *** THIS IS THE AGENDA COPY AND PASTED FROM OEB, NOT SURE EXACT TIMELINE FOR ELI *** 8:30-9:15 Idea generation and brainstorming activities 9:15-9:30 Break 9:30-10:15 Idea generation and brainstorming activities 10:15-10:30 Break 10:30-10:45 Vote: Consensus gathering and prioritization through online polling 10:45-11:00 Summary and next steps
  13. Provide Justification for year 1 goal Discuss each activity for year 1 goal 1. Host national summit 2. Determine desired outcomes 3. Establish framework of inquiry 4. Formulate measures 5. Establish research instrumentation Transition: Now, let’s talk more specifically about our first activity as we are launching this grant.
  14. What are your variables? Consider -- How will you measure them?  What data are available for you collect? How are they defined?
  15. Once of our first activities was to bring experts from around the country to assist us in developing a research model impacting our research agenda that will be conducted in year 2 of the grant. We want to bring some coherency to the field of distance education research and use that foundation to guide our research as part of this grant. We hope our work today will assist us in developing research toolkits that will drive cross-institutional distance education research in year 2 of our grant. These research toolkits will have priority research questions to be answered, a framework of inquiry including variables and shared measures, guidance in methodology including instrumentation, data mining techniques, and sample syntax, and more.
  16. Provide Justification for year 1 goal Discuss each activity for year 1 goal 1. Host national summit 2. Determine desired outcomes 3. Establish framework of inquiry 4. Formulate measures 5. Establish research instrumentation Transition: Now, let’s talk more specifically about our first activity as we are launching this grant.
  17. *** THIS IS THE AGENDA FROM OEB COPY AND PASTED, NOT SURE THE TIMELINE FOR ELI *** 9:30-10:15 Idea generation and brainstorming activities 10:15-10:30 Break 10:30-10:45 Vote: Consensus gathering and prioritization through online polling 10:45-11:00 Summary and next steps
  18. Provide Justification for year 1 goal Discuss each activity for year 1 goal 1. Host national summit 2. Determine desired outcomes 3. Establish framework of inquiry 4. Formulate measures 5. Establish research instrumentation Transition: Now, let’s talk more specifically about our first activity as we are launching this grant.
  19. So, we developed a list of RQs and a list of variables, which informed the framework of inquiry. All of these activites led to the research toolkit. Let’s take a deep dive into the toolkit. 1. Host national summit 2. Determine desired outcomes 3. Establish framework of inquiry 4. Formulate measures 5. Establish research instrumentation Transition: Now, let’s talk more specifically about our first activity as we are launching this grant.
  20. Now, as mentioned, the toolkits are a year 2 activity that has already taken place, to assist us in our primary year 2 goal -- Collect data by replicating research models at the course, program, and institutional level Let’s talk in a bit more detail how the contents of the toolkits are going to help make this happen…
  21. 9:15-9:30 Vote and Break 9:30-10:15 Idea generation and brainstorming activities
  22. The research toolkits produced from year 1 activities will be used by our research fellows, institutional partners, CBEN institutions Furthermore, they will provide the guides and tools necessary too facilitate rigorous cross institutional research from experimental and survey designed studies gathering data from in-class assessment, survey packets, and student information system or data warehoused student data. Note: Do not go into detail until slide 22 Now, let’s talk about the potential for you to get funding to collaborate with us….
  23. Request for proposals for DETA subgrant awards is available in the research toolkit in Section 1. You will find background information, grant participation requirements, proposal requirements, and proposal review criteria. The goal with the process is not to build hurdles in obtaining funding and support, but to ensure rigorous cross-institutional research and data collection and address audiences of interest Goal 2 activities 2. Evaluate pilot grant proposal submissions 3. Engage research fellows in research projects 4. Ensure data collection addresses audiences of interest
  24. The first goal of the grant activities is to develop research models for online learning that provide guidance in the practice of distance education research. The models were intended to facilitate the exploration of instructional practices, inform future instructional practices, serve as a model for future research practices across educational institutions, and enhance consistency in the field. In the development process, it became clear that a more general research model was needed to represent the various research models and designs that would be deployed as part of the DETA research efforts rather than several specific research models. More information is available in the toolkit or on our website.
  25. Diane could discuss methodological considerations – how it can help them do research, etc. She could mention WWC standards
  26. Tanya can discuss gathering data through SIS and Surveys Diane can mention how these will compliment experimental studies
  27. Tanya can discuss how codebooks will help with data merging and how these are iterative…they will be improved once partners are identified and as data acquisition and merging
  28. So, we developed a list of RQs and a list of variables, which informed the framework of inquiry. All of these activites led to the research toolkit. Let’s take a deep dive into the toolkit. 1. Host national summit 2. Determine desired outcomes 3. Establish framework of inquiry 4. Formulate measures 5. Establish research instrumentation Transition: Now, let’s talk more specifically about our first activity as we are launching this grant.
  29. Coming from the communication field and thinking of the transactional model of communication, I am thinking through a model of distance education, blended and online education. Or, actually, all learning – as a social process. We make strategic choices about how we want interactions in class and out of class to support desired student and program/institutional outcomes. These strategic choices can be thought of as interventions – instructional or pedagogical and/or programmatic and institutional interventions. The choices are with hopes to positively influence student interactions with content, each other, instructors, and supporting staff to impact student outcomes/success.
  30. Coming from the communication field and thinking of the transactional model of communication, I am thinking through a model of distance education, blended and online education. Or, actually, all learning – as a social process. We make strategic choices about how we want interactions in class and out of class to support desired student and program/institutional outcomes. These strategic choices can be thought of as interventions – instructional or pedagogical and/or programmatic and institutional interventions. The choices are with hopes to positively influence student interactions with content, each other, instructors, and supporting staff to impact student outcomes/success.
  31. Provide Justification for year 1 goal Discuss each activity for year 1 goal 1. Host national summit 2. Determine desired outcomes 3. Establish framework of inquiry 4. Formulate measures 5. Establish research instrumentation Transition: Now, let’s talk more specifically about our first activity as we are launching this grant.
  32. Provide Justification for year 1 goal Discuss each activity for year 1 goal 1. Host national summit 2. Determine desired outcomes 3. Establish framework of inquiry 4. Formulate measures 5. Establish research instrumentation Transition: Now, let’s talk more specifically about our first activity as we are launching this grant.
  33. Thanks to all of you for funding your travel to join the DETA partners here at ELI and share your knowledge and expertise in this area to ensure we have a well informed research model moving forward. Thanks to ELI for hosting us at their annual meeting. We greatly appreciate the efforts of their staff in helping us make this event a success. Enjoy the conference.
  34. Tanya
  35. Tanya