1. LEVERAGING CSCL RESEARCH ANALYZING
ONLINE DISCUSSIONS TO INFORM DCLA
ALYSSA WISE
SIMON FRASER UNIVERSITY
FACULTY OF EDUCATION
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

2. LEARNING ENVIRONMENT – ONLINE DISCUSSIONS

3. ONE CENTRAL CONCERN
CONNECTING THE DEVELOPMENT AND USE OF
DISCOURSE-CENTRIC LEARNING ANALYTICS WITH
THEORETICAL UNDERSTANDINGS OF THE QUALITIES
OF PRODUCTIVE ONLINE DISCOURSE
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

4. ONE CENTRAL CONCERN
THEORETICAL MODELS OF ONLINE DISCOURSE
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

5. TWO KEY POINTS
THERE IS AN ADDITIONAL, NOVEL KNOWLEDGE
BASE NEEDED TO LEVERAGE CSCL MODELS OF
DISCOURSE TO INFORM DCLA
CREATING A CLEAR THREAD BETWEEN DISCOURSE
GOALS, ACTIONS AND ANALYTICS IS CRITICAL FOR
LEARNERS (AND TEACHERS) TO USE DCLA AS A TOOL
FOR SUPPORTING PRODUCTIVE DISCOURSE
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

6. TWO KEY POINTS
DCLA > LAK + CSCL
TRANSPARENCY & INTEGRATION
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

7. DCLA > LAK + CSCL
CSCL HAS MANY WELL-DEVELOPED MODELS FOR THEORIZING
AND ANALYZING THE QUALITIES OF ONLINE DISCOURSE
MOST ARE PROCESS MODELS THEORETICALLY (THOUGH
ANALYSES HAVE OFTEN DISREGARDED THIS)
MOST ARE GROUP, NOT INDIVIDUAL, MODELS
THERE IS ADDITIONAL, NOVEL KNOWLEDGE NEEDED TO
LEVERAGE CSCL MODELS OF DISCOURSE TO INFORM DCLA
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

8. KNOWLEDGE CONSTRUCTION MODEL
FROM GUNAWARDENA ET AL., 1997….. TO WISE & CHIU, 2011
THEORETICAL PATTERN
Agreeing &
5
Applying
Testing & 4
Modifying
Negotiating 3
Meaning
Exploring 2
Dissonance
Sharing 1
Information
1 6 11 16
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

9. KNOWLEDGE CONSTRUCTION MODEL
FROM GUNAWARDENA ET AL., 1997….. TO WISE & CHIU, 2011
ACTUAL PATTERN (PRETTY GOOD)
Agreeing &
5
Applying
Testing & 4
Modifying
Negotiating 3
Meaning
Exploring 2
Dissonance
Sharing 1
Information
1 6 11 16
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

10. KNOWLEDGE CONSTRUCTION MODEL
FROM GUNAWARDENA ET AL., 1997….. TO WISE & CHIU, 2011
ACTUAL PATTERN (FAILURE TO LAUNCH)
Agreeing &
5
Applying
Testing & 4
Modifying
Negotiating 3
Meaning
Exploring 2
Dissonance
Sharing 1
Information
1 6 11 16
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

11. KNOWLEDGE CONSTRUCTION MODEL
FROM GUNAWARDENA ET AL., 1997….. TO WISE & CHIU, 2011
ACTUAL PATTERN (QUICK CONSENSUS)
Agreeing &
5
Applying
Testing & 4
Modifying
Negotiating 3
Meaning
Exploring 2
Dissonance
Sharing 1
Information
1 6 11 16
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

12. ISSUES IN DRAWING ON KC MODEL TO DEVELOP DCLA
AUTOMATING POST CODING, IN REAL TIME
EVALUATING “APPROXIMATIONS” OF GOOD DISCOURSE PATTERNS
(DESIRABLE ENDSTATE VS PATHS FOR CHANGE)
EVALUATING DISCOURSE PATTERNS IN PROGRESS (IS “QUALITY” OF A
POST PATTERN PARTIALLY DETERMINED BY THOSE YET TO COME)
CREATING ACTIONABLE ANALYTICS
(GROUP / INDIVIDUAL LEVELS, INTERPRETABILITY)
ASSESSING INDIVIDUAL’S POSTS IN CONTEXT (SEQUENTIAL ANALYSIS)
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

13. TRANSPARENCY & INTEGRATION
LEARNERS HAVE A VARIETY OF PERCEPTIONS ABOUT THE
PURPOSE OF ONLINE DISCOURSE (THAT OFTEN DIFFER FROM
THE INTENDED PURPOSE) - HOW THEY PARTICIPATE IN ONLINE
DISCUSSIONS RELATE TO THESE PERCEIVED PURPOSES
(KNOWLTON, 2005; WISE ET AL. 2012; 2013)
LEARNERS (AND TEACHERS) NEED TO SEE THE CONNECTIONS
BETWEEN THE GOALS OF DISCOURSE, RELATED ONLINE
ACTIVITY, AND ANALYTIC FEEDBACK IN ORDER TO USE DCLA AS
A TOOL FOR SUPPORTING PRODUCTIVE DIALOGUE
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

14. TRANSPARENCY & INTEGRATION
THEORETICAL MODEL
DISCOURSE GUIDELINES
RELATED ANALYTICS
GOAL SETTING
DATA
REFLECTION
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

15. THEORETICAL MODEL
SOCIAL CONSTRUCTIVIST PERSPECTIVE - LEARNERS BUILD UNDERSTANDING
THROUGH DIALOGING WITH OTHERS
VARYING IMPORTANCE OF SHARING AND SUPPORTING ONE’S OWN
IDEAS, BEING EXPOSED TO MULTIPLE VIEWPOINTS, EXPERIENCING PERSONAL
COGNITIVE CONFLICT, NEGOTIATION OF GROUP UNDERSTANDINGS
IN ALL, TWO BASIC (AND RELATED) UNDERLYING PROCESSES:
SPEAKING
EXTERNALIZING ONE’S IDEAS BY CONTRIBUTING POSTS TO THE DISCUSSION
LISTENING
TAKING IN THE EXTERNALIZATIONS OF OTHERS BY ACCESSING EXISTING POSTS
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

16. THEORETICAL MODEL
IN ONLINE DISCUSSIONS LEARNERS HAVE A HIGH DEGREE OF CONTROL
OVER THE TIMELINE AND PACE OF THEIR ENGAGEMENT
OPPORTUNITIES FOR THOUGHTFUL LISTENING AND REFLECTIVE SPEAKING
CHALLENGES OF TIME MANAGEMENT, ESPECIALLY IN PROLIFIC DISCUSSIONS
HELPING LEARNERS TO ACTIVELY MONITOR AND REGULATE HOW THEY
SPEAK AND LISTEN IN ONLINE DISCUSSIONS IS AN IMPORTANT TOOL FOR
SUPPORTING PRODUCTIVE ENGAGEMENT IN DISCUSSIONS
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

17. SPEAKING LISTENING
MECHANISM FOR SHARING IDEAS WITH ATTENDING TO THE IDEAS OF OTHERS IS A
OTHERS CRITICAL, THOUGH “INVISIBLE” PART OF
LEARNING THROUGH ONLINE DISCUSSIONS
VALUE IN SPEAKING THAT IS VALUE IN LISTENING THAT IS
- RECURRING, RESPONSIVE AND - BROAD (TO CONSIDER MULTIPLE IDEAS)
RATIONALED
- INTEGRATED (SO REPLIES ARE INFORMED BY
- TEMPORALLY DISTRIBUTED READS)
- MODERATE PORTIONED - REFLECTIVE (TO PROVIDE CONTEXT FOR
DISCUSSION FLOW)
WHILE “SPEAKING” IS VISIBLE, NOT ALL
QUALITIES ARE SALIENT IN THE SYSTEM EARLY RESEARCH SUGGESTED UNIVERSALLY
(ESP. AS RELATED TO TIME) POOR BEHAVIORS, BUT OUR RECENT WORK
SHOWS STUDENTS INTERACT WITH PRIOR
POST QUALITY INFO VALUABLE, COMPLEX MESSAGES IN VERY DISTINCT WAYS
TO ASSES
- E.G. COVERAGE, INTERACTIVE, SELF-
FOCUSED, TARGETED
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

18. RELATED ANALYTICS - EMBEDDED
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

19. RELATED ANALYTICS - EXTRACTED
Metric Definition Criteria
Range Span of days a student logged in to the discussion
# of sessions # of times a student logged in to the discussion
Total time a student spent in the discussions divided by his / Temporal
Av session length Distribution
her # of sessions
% of sessions # of sessions in which a student made a post, divided by
with posts his/her total of # sessions
Posts Total # of posts a student contributed to the discussion
Speaking
Total # of words posted by a student divided by the # of Quantity
Av post length
posts he/she made to the discussion
# of unique posts that a student read divided by the total # Listening
% of posts read
of posts made by others to the discussion Breadth
# of reviews # of times a student revisited posts that he/she had made
(own posts) previously in the discussion Listening
# of reviews # of times a student revisited others’ posts that he/she had Reflectivity
(others’ posts) viewed previously in the discussion
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

20. DISCOURSE GUIDELINES ANALYTICS GUIDELINES
CONNECT PURPOSE OF THE LEARNING CONNECT PURPOSE OF LEARNING
ACTIVITY WITH THE INSTRUCTOR’S ACTIVITY + INSTRUCTOR’S EXPECTATIONS
EXPECTATIONS FOR ENGAGEMENT WITH LEARNING ANALYTICS INDICATORS
Discussion Participation Guidelines Learning Analytics Guidelines
Attending to Others Posts Attending to Others’ Posts
Broad Listening: Try to read as many posts % of The proportion of posts you
as possible to consider everyone’s ideas in the posts read (not scanned) at least
discussion. This can help you examine and read once.
support your own ideas more deeply.
However, when time is limited it is better to
It is good to read as many posts as
view a portion in depth, then everything
possible to consider everyone’s ideas in
superficially.
the discussion However, when time is
limited it is better to view a portion in
*The visual interface shows posts that you depth, then everything superficially.
have viewed in blue and new ones in red to
help you track this.
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

21. GOAL SETTING, DATA & REFLECTION
SUPPORTING INTENTIONAL ATTENTION TO DISCOURSE
GUIDELINES PRESENT (DIVERSE) METRICS AS A STARTING POINT FOR
CONSIDERATION, NOT AS ABSOLUTE ARBITERS OF ENGAGEMENT IN THE
ACTIVITY
USE OF CLASS AVERAGE TO PROVIDE CONTEXT FOR #S
STUDENTS SET PERSONAL GOALS FOR PARTICIPATION AND USE THE
ANALYTICS TO HELP MONITOR THESE REAL-TIME AND WEEKLY
REFLECTIVE DIALOGUE BETWEEN EACH STUDENT AND INSTRUCTOR
ABOUT THEIR PARTICIPATION, GROUNDED IN ANALYTICS
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

22. INITIAL FINDINGS
ANALYTICS PILOTED WITH SEMESTER-LONG DOCTORAL COURSE
STRONG STUDENT BUY-IN TO USEFULNESS OF ANALYTICS TO
SUPPORT PRODUCTIVE DISCUSSION, BUT DIFFERENT STUDENTS
FOUND DIFFERENT ANALYTICS USEFUL FOR SETTING GOALS AND
MONITORING PROGRESS TOWARDS THEM
NO “BIG BROTHER” ISSUES
STUDENT RECOGNITION OF WHEN THEY WERE AND WEREN’T
MAKING PROGRESS ON THEIR GOALS
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY

23. ONE CONCERN
THEORETICAL MODELS OF ONLINE DISCOURSE
TWO KEY POINTS
DCLA > LAK + CSCL
TRANSPARENCY & INTEGRATION
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY