"Exploring methodological approaches to
        conduct process analysis
      in fragmented, long-duration
   collaborati...
Pedagogical design..
                                  Course wiki & Blog (announcements; schedules; lecture materials; wo...
led to messy data..
                                                                 RSS Feeds                            ...
Research problem was
 and is uh..challenging

What they did really do with the social
software and mobile devices? A
tempo...
Methodological skills oriented
my interests towards variable
       level approach
• Content Analysis
• Social Network Ana...
Well, what’s the problem
  with variable data?
Variable level

           We werent’ only using
            MULTIPLE tools..



                 Blog      Wiki

but we a...
Actually results lived in data
archipelago, were small (or
     big)islands there
Having a holistic approach into
analysis and design, it was little
    bit too tricky to start..
Event level



                Ishikawa diagrams
              (also called fishbone diagrams or cause-and-effect diagrams)...
Event level

         A Framework for Analyzing Interactional Processes in
                          Online Learning
     ...
What’s next?
              • Didn’t found appropriate methodological
Event level     literature of event level analysis
  ...
Variable level approach
     was chosen..
    Appropriate coding categories?
   Blog                       Wiki


  FA IL ...
Cress & Kimmerle (2008)
  A systemic and cognitive view on collaborative knowledge building with wikis.
 International Jou...
Problem remains
  Blog               Wiki                Other
                L                   L
            FAI      ...
Analytical toolkit for
        time-series data
               MISSING
 Wiki: timetags are essential. No way to import int...
TIME IS RUNNING!
New tryout with variable
    level approach

• How to measure dependencies
 between (all) variables??!
• ...
Bayesian dependency modeling
•   SRL Self-report questionnaire (Pintrich, 1993; Wolters, 2001)
•   Short-answer (paper & p...
19


       Are learners’ actions in the web2.0 tools and
       F2F sessions related to their learning outcomes?
        ...
20


                   WHY IT WAS DOOMED TO BE A
                   FAIL?
    • Bayesian approach provided only surface
 ...
OOPS!
What’s next?
What I had on my
hands after Bayesian?
• Analytical toolkit: Tatiana
• Some coding schemes which didn’t stretch
  over the...
Task level

           Narrative and learning with web2.0
        technologies: towards a research agenda.
             Pa...
Missing piece is ..

Missing piece
Time is precious:Variable- and event-centred
 approaches to process analysis in CSCL research.
          (2009) Computer-S...
Process method should..
       (Reimann, 2009)
• a process method should, in addition to
  efficient cause (typical for eve...
One coding scheme to evaluate
   interpersonal exchange
•   Mercer, N. (2008). The seeds of Time: Why Classroom
    Dialog...
I propose an applied
    approach for Mercer
•   To evaluate interactions that are mediated by
    artifacts or than talk ...
Unit of analysis in my
       research
• Main entities are individuals and groups
 • events are constrained to those incid...
led to messy data..
                                                                 RSS Feeds                            ...
Upcoming SlideShare
Loading in...5
×

What they did really do with the social software and mobile devices? A temporal analysis of long-term technology mediated collaboration.

368

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
368
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

What they did really do with the social software and mobile devices? A temporal analysis of long-term technology mediated collaboration.

  1. 1. "Exploring methodological approaches to conduct process analysis in fragmented, long-duration collaboration mediated by multiple artifacts." Jari Laru Learning & Educational Technology Research Unit University of Oulu 10/2009
  2. 2. Pedagogical design.. Course wiki & Blog (announcements; schedules; lecture materials; workspaces; helpdesk) Media-sharing services Theoretical grounding Blog Wiki Individual virtual work Media Reflection, Constructing Collaborative Lecture Grounding session Meaning-making session virtual work Media Reflection, Constructing Theoretical Groups’ grounding grounding Elaborating Constructing Media Reflection, Constructing Media Reflection, Constructing Syndication service via RSS Monitoring = Face-to-Face sessions
  3. 3. led to messy data.. RSS Feeds RSS Feeds Media Media Media Reflection, Theoretical Theoretical Media Media Constructing grounding grounding Media pre-test Media Media (conceptual Media Reflection, SRI Theoretical Theoretical Elaborating Constructing post test knowledge) Constructing interviews grounding grounding Media Media + questionnaire Media Media Media Reflection, Theoretical Theoretical Constructing grounding grounding Media time Event level media video lecture video media Blog content video Wiki history artefacts recordings topics recordings artefactsmonitored recordings content Variable level statistical Conversation Content Content Content Content Conversation Content statistical conversation methods analysis analysis analysis analysis analysis analysis analysis methods analysis Ouput: a lot of different kind of frequency data a la coding categories, log data, conceptual test etc statistical methods used to calculate dependencies or probabilities between variables
  4. 4. Research problem was and is uh..challenging What they did really do with the social software and mobile devices? A temporal analysis of long-term technology mediated collaboration.
  5. 5. Methodological skills oriented my interests towards variable level approach • Content Analysis • Social Network Analysis • Conversation Analysis Variable level • Mann-Whitney U-test • etc..
  6. 6. Well, what’s the problem with variable data?
  7. 7. Variable level We werent’ only using MULTIPLE tools.. Blog Wiki but we also had different kind of F2F settings in different phases
  8. 8. Actually results lived in data archipelago, were small (or big)islands there
  9. 9. Having a holistic approach into analysis and design, it was little bit too tricky to start..
  10. 10. Event level Ishikawa diagrams (also called fishbone diagrams or cause-and-effect diagrams) FA IL First event-level try-out. No success. No methodological bacground. FAIL
  11. 11. Event level A Framework for Analyzing Interactional Processes in Online Learning Daniel Suthers, Nathan Dwyer, Richard Medina, Ravi Vatrapu (2007). Paper presented at the Annual Meeting of the American Educational Research Association (AERA), Chicago, April 9-13, 2007. FA IL Too instrumental & multi-widget-chat tool orientation (several presentations at CSCL’09)
  12. 12. What’s next? • Didn’t found appropriate methodological Event level literature of event level analysis • wrong terminology used (didn’t know) WEB2.0 / Social WEB • this terminology didn’t help neither • Methodologically exteremely heterogenous area
  13. 13. Variable level approach was chosen.. Appropriate coding categories? Blog Wiki FA IL K O
  14. 14. Cress & Kimmerle (2008) A systemic and cognitive view on collaborative knowledge building with wikis. International Journal of Computer-Supported Collaborative Learning, 3(2), 105–122. Basic categories: “Besides assimilating information the wiki can also accommodate. This happens when new information is not only attached to the existing information, but the information in the wiki is organized in a new way.” O K
  15. 15. Problem remains Blog Wiki Other L L FAI FAI Data Data Data Scheme Scheme Scheme Variables Variables Variables
  16. 16. Analytical toolkit for time-series data MISSING Wiki: timetags are essential. No way to import into NVIVO8 Blogs: cumulative nature and “metadata” essential. No way to import into NVIVO8 WHERE IS A TOOL FOR ANALYSING HETEROGENOUS, MULTITOOL TIME-SERIES DATA? Answer: Tatiana! (CSCL 09)
  17. 17. TIME IS RUNNING! New tryout with variable level approach • How to measure dependencies between (all) variables??! • Petri Nokelainen visited our group: IDEA! Bling!
  18. 18. Bayesian dependency modeling • SRL Self-report questionnaire (Pintrich, 1993; Wolters, 2001) • Short-answer (paper & pencil) conceptual knowledge test – 3 x 6 concepts (about lectures), same in pre/post tests • Basic activity measures as “log data” – # of images/videos (file-sharing services) – # of articles/edits (blog, wiki) – # of wordcount (blog, wiki, face-to-face sessions) – # of items (rss) • Qualitative data collection for other analyses (stimulated recall interviews, group interviews, content produced by learners, wiki history) • Statistical dependencies between the variables were studied with Bayesian Dependency Modeling (BDM), which predicts the most probable statistical dependency structure between the observed variables (Myllymäki, Silander, Tirri & Uronen, 2002). • B-Course: A Web-Based Tool for Bayesian and Causal Data Analysis was used as a analysis toolkit • Bayesian analysis is based on propabilities instead of frequencies (Nokelainen, Silander, Ruohotie, Tirri, 2007) var 1 var 2 B-Course
  19. 19. 19 Are learners’ actions in the web2.0 tools and F2F sessions related to their learning outcomes? wiki Learning outcome “monitor” Syndication service direct dependencies: Learning gain: - RSS_Read (# of read RSS items) FA IL => Monitor - WP_Entries (# of blog entries) => Reflector wiki mediasharing “reflector” Some arcs are pretty strong, so that removing any of them would result in a model with probability less: * than one millionth of that of the original model blog ** than one thousandth of that of the original model LET - Learning and Educational Technology Research Unit *** smaller than one thousandth (weak connection) Jari Laru. 27.8.09 ERLI09, Amsterdam
  20. 20. 20 WHY IT WAS DOOMED TO BE A FAIL? • Bayesian approach provided only surface level analysis: –deep insight into collaborative interaction in F2F groups, wikis & blogs missing –order, duration and quality of events: eg. lecture > F2F > image/video > blog reflection > F2F > Wiki • Interesting question “How learning unfolds over time” wasn’t answered at all.. • Didn’t support design of the study! LET - Learning and Educational Technology Research Unit Jari Laru. 27.8.09 ERLI09, Amsterdam
  21. 21. OOPS!
  22. 22. What’s next?
  23. 23. What I had on my hands after Bayesian? • Analytical toolkit: Tatiana • Some coding schemes which didn’t stretch over the whole design • Some methodological hints about event based approach.. • Intuition that event based approach is “my way”
  24. 24. Task level Narrative and learning with web2.0 technologies: towards a research agenda. Pachler & Daly (2009). Journal of Computer Assisted Learning. 25, 6-18 •Narrative trails (+) •Sense-making (+) •Blog centric (-) •vague understanding of collaboration (-) (commenting in blogs) •Holistic approach?.. m m e ll..h W
  25. 25. Missing piece is .. Missing piece
  26. 26. Time is precious:Variable- and event-centred approaches to process analysis in CSCL research. (2009) Computer-Supported Collaborative Learning (2009). 4:239-257 Methodological link Mercer (2008) et. al Muukkonen et. al Suther et. al (2007) communication & (2008) graphical techniques learning in classroom Time series analysis Central tenet: “methods developed in conversation analysis and ethnomethodology...do not carry over to interactions that are fragmented over time, stretch over longer durations, or are mediated by artifacts rather than talk” pp.241
  27. 27. Process method should.. (Reimann, 2009) • a process method should, in addition to efficient cause (typical for event approach), be able to deal with at least two other kinds of causes: • formal cause, referring to the patterns of which things are made • final cause, the end for which things are made [goal, in our study: wiki]
  28. 28. One coding scheme to evaluate interpersonal exchange • Mercer, N. (2008). The seeds of Time: Why Classroom Dialogue Needs a Temporal Analysis. The Journal of Learning Sciences, 17: 33-59 • Focus of article is on examining long-term classroom talk and how it’s used to: • represent past experiences • carry ideas forward from one occasion to another • approach future activities • achieve learning outcomes
  29. 29. I propose an applied approach for Mercer • To evaluate interactions that are mediated by artifacts or than talk and can be categorized according Mercer: • represent past experiences [for example: in personal blog, or F2F discussions] • carry ideas forward from one occasion to another [photo => blog, or peer blog => own blog etc.] • approach future activities [design groups’ / individual activities] • achieve learning outcomes [troughout the design]
  30. 30. Unit of analysis in my research • Main entities are individuals and groups • events are constrained to those incidents in which either individuals or groups can participate • Focus will be put on the sequences of activities, incidents, crises, or stages that unfold in the groups over the duration of the course
  31. 31. led to messy data.. RSS Feeds RSS Feeds Media Media Media Reflection, Theoretical Theoretical Media Media Constructing grounding grounding Media pre-test Media Media (conceptual Media Reflection, SRI Theoretical Theoretical Elaborating Constructing post test knowledge) Constructing interviews grounding grounding Media Media + questionnaire Media Media Media Reflection, Theoretical Theoretical Constructing grounding grounding Media time Event level media video lecture video media Blog content video Wiki history artefacts recordings topics recordings artefactsmonitored recordings content Variable level statistical Conversation Content Content Content Content Conversation Content statistical conversation methods analysis analysis analysis analysis analysis analysis analysis methods analysis Ouput: a lot of different kind of frequency data a la coding categories, log data, conceptual test etc statistical methods used to calculate dependencies or probabilities between variables
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×