Online Learning Processes –
What Users Do and Why it is Important
Karsten D. Wolf
Didactical Design of Interactive Learnin...
Karsten D. Wolf
         Faculty educational science
               Didactical design of
interactive learning environments...
www.everlearn.eu

User Generated Content
 Didactical Prototyping
 Collaborative Learning




                 cc by Karste...
4
roadmap

1. Why educational research is hard


2. How can we produce evidence in educational research?


3. Multimethod ap...
soft sciences are hard




                    cc by Karsten D. Wolf 2007
                                                 6
cc by Karsten D. Wolf 2007
                             7
btw: this is not a real sim.
rocks do not decelerate before hitting the ground!




                                      ...
what if rocks would behave?




                      cc by Karsten D. Wolf 2007
                                         ...
cc by Karsten D. Wolf 2007
                             10
covariates hell




                  pic by Don Gato onWolf 2007
                            cc by Karsten D. Flickr
    ...
3 strategies

 control them
live with them
r&d approach
Fischer/Waibel/Wecker (2006):
Nutzenorientierte Grundlagenforschun...
experimental research
 design based research
engineering approaches


                    cc by Karsten D. Wolf 2007
     ...
cc by Karsten D. Wolf 2007
                             14
75%
  cc by Karsten D. Wolf 2007
                               15
Educational Researcher, Vol. 32, No. 9, 3–14




Educational Researcher, Vol. 33, No. 2, 3–11
                            ...
Problems with quasi-experiments




                n
              o
            s
          p
       im ox
       Sd
   ...
Problems with quasi-experiments




           Steyer 2007 - www.causal-effects.de   cc by Karsten D. Wolf 2007
          ...
Problems with quasi-experiments




           Steyer 2007 - www.causal-effects.de   cc by Karsten D. Wolf 2007
          ...
Random Assignment & Propensity Score Matching

• Random Assignment


  • is the population large enough?


  • can the ass...
Multimethod approaches: assessment methods

• self-reports (behaviors, psychological contructs, forms of behaviors)


• in...
Multimethod approaches:assessment methods

• quantitative text analysis


• multilevel analysis: physiological & biochemic...
Experiments vs. Assessments

 Why is Explanandum?


 If (A1 and A2) then Explanandum   (general law, explanans)


        ...
Level of nonreactivity
                                                Low                                                ...
Methods for analyzing multimethod data

• Multitrait-Multimethod Matrix (Campbell and Fiske 1959)


• categorical variable...
cc by Karsten D. Wolf 2007
                             26
27
28
29
30
31
32
I feel taken seriously (EG1 TN1 SoLe III)




                                            33
1 2 3 4 5 6 7 8 9 10 11 12 13 14




                                   34
Taken seriously   Cognitive Load    Participation




I feel good       I understand     I am interested




             ...
I am interested for19 students in EG1 SoLe III




                                                 36
I feel taken seriously (EG1 TN1 SoLe III)




                                            37
Trend
I feel taken seriously (EG1 TN1 SoLe III)
               Linear Trend




                                          ...
Trend
I feel taken seriously (EG1 TN1 SoLe III)
               3rd Polynom




                                            39
Trend
I feel taken seriously (EG1 TN1 SoLe III)
             Spline λ=10.000




                                         ...
Trend
I feel taken seriously (EG1 TN1 SoLe III)
             Spline λ=100.000




                                        ...
Decomposition
I feel taken seriously (EG1 TN1 SoLe III)




Trend                      Microprocesses


                  ...
ARMA-Analysis




                43
Model parameters
                           R2               Φ
         AIC      BIC


                                   ...
AR(2)-Model




              45
Multivariate Analyis:
Cross Correlations

           0,120
   I                    I           0,122

                    ...
Multivariate Models:
GMDH

                       I          0,097

                                      FG
          P
 ...
Fuzzy Modelling (Zadeh)

“As the complexity of a system increases, our
ability to make precise and significant
statement ab...
Fuzzification
                                           1




                                           0,5




        ...
Fuzzy Rule Induction
                          IF

                               TakenSeriously (t-1)
NOT_Participation (...
Advantages?

Better communication?
Different interrelationships at different
levels? (Thresholds)
non-linear models?
basis...
disadvantages
time consuming
lot of rules
no selection criteria
no „test of significance“




                           52
Online learning
as a research frontier?




                     cc by Karsten D. Wolf 2007
                              ...
cc by Karsten D. Wolf 2007
                             54
cc by Karsten D. Wolf 2007
                             55
how to measure this?




           what is happening here?




                                     cc by Karsten D. Wolf...
???




      cc by Karsten D. Wolf 2007
                                   57
???
      Learning accompanying actions

      Browse
      Search
      Write
      …




                              c...
???
      creation




                 cc by Karsten D. Wolf 2007
                                              59
???

      construction




                     cc by Karsten D. Wolf 2007
                                              ...
???




      communication




                      cc by Karsten D. Wolf 2007
                                         ...
???




      cooperation


                    cc by Karsten D. Wolf 2007
                                               ...
???




      collaboration



                      cc by Karsten D. Wolf 2007
                                          ...
Some results from a prior online seminar




                    ZfE Cover




                                           ...
Activity


                              # forum       # created
           # chat msg
                              entri...
Activity and Post-Interest


                                # forum       # created
             # chat msg
             ...
Pre-Interest and Activity


                                  # forum       # created
               # chat msg
          ...
Activity and Post-Interest


                                # forum       # created
             # chat msg
             ...
Activity and Social Relatedness


                                 # forum       # created
              # chat msg
      ...
Pre-Knowledge and Activity


                               # forum       # created
            # chat msg
               ...
Activity and Knowledge


                               # forum       # created
            # chat msg
                   ...
Which means: constructivism is right after all!




                                              © Karsten D. Wolf 2007
 ...
© Prof. Dr. Karsten D. Wolf 2005
http://www.karsten-d-wolf.de/


    wolf@uni-bremen.de


                                73
Upcoming SlideShare
Loading in...5
×

Online learning processes - what users do and why it is important

1,683

Published on

Keynote Young Researcher's Workshop at ITK07, Finland
Abstract:
Online settings provide non-reactive ways to observe learners' activities. While the learning itself is not directly visible for example in a logfile or within the history of a Wiki, the growing expressiveness in online learning environments takes us a step nearer to witness and analyse learning processes.

In this presentation four questions will be discussed:

(1) What is the relation between possible user activities in recent E-Learning environments and actual learning processes?

(2) What data is available to observe the users' activities?

(3) What data analysis methods are fitting to model these activities?

(4) What research designs do we need to get more insight into learning processes?

As an example, some ideas, experiences and results from an ongoing research project about online user participation and motivation will be presented and discussed, focussing on a multimethod approach.

Published in: Education, Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,683
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
117
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Online learning processes - what users do and why it is important

  1. 1. Online Learning Processes – What Users Do and Why it is Important Karsten D. Wolf Didactical Design of Interactive Learning Environments interaktiivinen tekniikka koulutuksessa 2007 Wednesday, 18.04.2007 cc by Karsten D. Wolf 2007 1
  2. 2. Karsten D. Wolf Faculty educational science Didactical design of interactive learning environments 2
  3. 3. www.everlearn.eu User Generated Content Didactical Prototyping Collaborative Learning cc by Karsten D. Wolf 2007 3
  4. 4. 4
  5. 5. roadmap 1. Why educational research is hard 2. How can we produce evidence in educational research? 3. Multimethod approaches in educational research 4. Alternative methods: Intra-individual analysis of micro process timeseries 5. Educational technology as a research frontier in learning science 6. Some applied results © Karsten D. Wolf 2007 5
  6. 6. soft sciences are hard cc by Karsten D. Wolf 2007 6
  7. 7. cc by Karsten D. Wolf 2007 7
  8. 8. btw: this is not a real sim. rocks do not decelerate before hitting the ground! cc by Karsten D. Wolf 2007 8
  9. 9. what if rocks would behave? cc by Karsten D. Wolf 2007 9
  10. 10. cc by Karsten D. Wolf 2007 10
  11. 11. covariates hell pic by Don Gato onWolf 2007 cc by Karsten D. Flickr 11
  12. 12. 3 strategies control them live with them r&d approach Fischer/Waibel/Wecker (2006): Nutzenorientierte Grundlagenforschung im Bildungsbereich Argumente einer internationalen Diskussion cc by Karsten D. Wolf 2007 12
  13. 13. experimental research design based research engineering approaches cc by Karsten D. Wolf 2007 13
  14. 14. cc by Karsten D. Wolf 2007 14
  15. 15. 75% cc by Karsten D. Wolf 2007 15
  16. 16. Educational Researcher, Vol. 32, No. 9, 3–14 Educational Researcher, Vol. 33, No. 2, 3–11 cc by Karsten D. Wolf 2007 16
  17. 17. Problems with quasi-experiments n o s p im ox Sd a r a p Steyer 2007 - www.causal-effects.de cc by Karsten D. Wolf 2007 17
  18. 18. Problems with quasi-experiments Steyer 2007 - www.causal-effects.de cc by Karsten D. Wolf 2007 18
  19. 19. Problems with quasi-experiments Steyer 2007 - www.causal-effects.de cc by Karsten D. Wolf 2007 19
  20. 20. Random Assignment & Propensity Score Matching • Random Assignment • is the population large enough? • can the assignment truly random? • doesn‘t it feel like a lottery? • Propensity Score Matching • need to collect covariates • matching at an individual level between treatment and control © Karsten D. Wolf 2007 20
  21. 21. Multimethod approaches: assessment methods • self-reports (behaviors, psychological contructs, forms of behaviors) • informant assessment • ecological momentary assessment • web based / computer based methods • ability tests • implicit methods (performance measures) • sequential observational methods © Karsten D. Wolf 2007 21
  22. 22. Multimethod approaches:assessment methods • quantitative text analysis • multilevel analysis: physiological & biochemical measures • brain imaging • experimental methods • nonreactive methods © Karsten D. Wolf 2007 22
  23. 23. Experiments vs. Assessments Why is Explanandum? If (A1 and A2) then Explanandum (general law, explanans) A1 (antecedent condition 1) A2 (antecedent condition 2) Therefore: E (explanandum) Experiments aim to develop and test general laws Assessments aim to show that the antecedent conditions are met Erdfelder & Musch 2006 © Karsten D. Wolf 2007 23
  24. 24. Level of nonreactivity Low High Type 0 Type 1 Type2 Type 3 Type 4 Type 5 Setting initially designed yes yes yes yes yes no or selected for research Participants are aware likely likely likely likely no no of the research setting Participants are aware of the likely likely likely no no no research question Participants are aware of the likely likely no no no no research hypothesis Participants are aware of the likely no no no no no measures‘ manipulability Participative Bogus pipeline Personality Cover story Lost letter Analysis of Examples expert interview technique questionaire experiments technique archival data Fritsche and Linneweber 2006 © Karsten D. Wolf 2007 24
  25. 25. Methods for analyzing multimethod data • Multitrait-Multimethod Matrix (Campbell and Fiske 1959) • categorical variables analysis for asessing multimethod association • multimethod item response theory • multilevel analysis • structural equation models • longitudinal methods © Karsten D. Wolf 2007 25
  26. 26. cc by Karsten D. Wolf 2007 26
  27. 27. 27
  28. 28. 28
  29. 29. 29
  30. 30. 30
  31. 31. 31
  32. 32. 32
  33. 33. I feel taken seriously (EG1 TN1 SoLe III) 33
  34. 34. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 34
  35. 35. Taken seriously Cognitive Load Participation I feel good I understand I am interested 35
  36. 36. I am interested for19 students in EG1 SoLe III 36
  37. 37. I feel taken seriously (EG1 TN1 SoLe III) 37
  38. 38. Trend I feel taken seriously (EG1 TN1 SoLe III) Linear Trend 38
  39. 39. Trend I feel taken seriously (EG1 TN1 SoLe III) 3rd Polynom 39
  40. 40. Trend I feel taken seriously (EG1 TN1 SoLe III) Spline λ=10.000 40
  41. 41. Trend I feel taken seriously (EG1 TN1 SoLe III) Spline λ=100.000 41
  42. 42. Decomposition I feel taken seriously (EG1 TN1 SoLe III) Trend Microprocesses 42
  43. 43. ARMA-Analysis 43
  44. 44. Model parameters R2 Φ AIC BIC Φ1 = 0,307 AR(1) 1201,8 1208,0 0,094 Φ1 = 0,238 AR(2) 1190,2 1200,9 0,139 Φ2 = 0,223 Φ1 = 0,237 Φ2 = 0,222 AR(3) 1192,1 1206,5 0,139 Φ3 = 0,007 Φ1 = 0,236 Φ2 = 0,241 Φ4 = AR(5) -0,077 1194,2 1215,8 0,145 Φ5 = -0,018 Φ3 = 0,030 44
  45. 45. AR(2)-Model 45
  46. 46. Multivariate Analyis: Cross Correlations 0,120 I I 0,122 FG P -0.143 0.146 CL t-3 t-2 t-1 t 46
  47. 47. Multivariate Models: GMDH I 0,097 FG P -0.15 0.13 CL t-2 t-1 t 47
  48. 48. Fuzzy Modelling (Zadeh) “As the complexity of a system increases, our ability to make precise and significant statement about its behaviour diminishe until a threshold is reached … the closer one look at a real-world problem the fuzzier becomes its solution” (Kosko, 94) 48
  49. 49. Fuzzification 1 0,5 0 Negative_X ZERO_X Positive_X 49
  50. 50. Fuzzy Rule Induction IF TakenSeriously (t-1) NOT_Participation (t-1) & NOT_Participation (t-2) & TakenSeriously (t-2) OR OR NOT_Participation (t-1) NOT_Understand (t-1) & CognitiveLoad (t-2) & CognitiveLoad (t-2) THEN FeelGood TN1 (prewhitened) 50
  51. 51. Advantages? Better communication? Different interrelationships at different levels? (Thresholds) non-linear models? basis for building hypotheses? 51
  52. 52. disadvantages time consuming lot of rules no selection criteria no „test of significance“ 52
  53. 53. Online learning as a research frontier? cc by Karsten D. Wolf 2007 53
  54. 54. cc by Karsten D. Wolf 2007 54
  55. 55. cc by Karsten D. Wolf 2007 55
  56. 56. how to measure this? what is happening here? cc by Karsten D. Wolf 2007 56
  57. 57. ??? cc by Karsten D. Wolf 2007 57
  58. 58. ??? Learning accompanying actions Browse Search Write … cc by Karsten D. Wolf 2007 58
  59. 59. ??? creation cc by Karsten D. Wolf 2007 59
  60. 60. ??? construction cc by Karsten D. Wolf 2007 60
  61. 61. ??? communication cc by Karsten D. Wolf 2007 61
  62. 62. ??? cooperation cc by Karsten D. Wolf 2007 62
  63. 63. ??? collaboration cc by Karsten D. Wolf 2007 63
  64. 64. Some results from a prior online seminar ZfE Cover © Prof. Dr. Karsten D. Wolf 2007 64
  65. 65. Activity # forum # created # chat msg entries words female 46.2 7.0 2301 n=23 (57.92) (7.44) (363.5) male 24.7 3.0 1919 n=13 (23.43) (3.08) (419.0) 1-tail significant test, n=36 © Prof. Dr. Karsten D. Wolf 2007 65
  66. 66. Activity and Post-Interest # forum # created # chat msg entries words Post- .11 .19 .34* interest (.258) (.135) (.022) 1-tail significant test, n=36 © Prof. Dr. Karsten D. Wolf 2007 66
  67. 67. Pre-Interest and Activity # forum # created # chat msg entries words .30* .28 .36* Pre-interest (.043) (.053) (.019) 1-tail significant test, n=36 © Prof. Dr. Karsten D. Wolf 2007 67
  68. 68. Activity and Post-Interest # forum # created # chat msg fact entries words by p ors ou re-in .34* t t(.022) eres Post- .11 .19 t! interest (.258) (.135) 1-tail significant test, n=36 © Prof. Dr. Karsten D. Wolf 2007 68
  69. 69. Activity and Social Relatedness # forum # created # chat msg entries words social -.08 .30* .31* relatedness (.319) (.036) (.035) 1-tail significant test, n=36 © Prof. Dr. Karsten D. Wolf 2007 69
  70. 70. Pre-Knowledge and Activity # forum # created # chat msg entries words pre- .34* .47** .54*** problem (.021) (.002) (<.001) solving 1-tail significant test, n=36 © Prof. Dr. Karsten D. Wolf 2007 70
  71. 71. Activity and Knowledge # forum # created # chat msg entries words post- .03 .28* .43** problem (.425) (.048) (.005) solving 1-tail significant test, n=36 © Prof. Dr. Karsten D. Wolf 2007 71
  72. 72. Which means: constructivism is right after all! © Karsten D. Wolf 2007 72
  73. 73. © Prof. Dr. Karsten D. Wolf 2005 http://www.karsten-d-wolf.de/ wolf@uni-bremen.de 73
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×