TUG-KMI




               Authoring System in TARGET
                   www.reachyourtarget.org


                       ...
TUG-KMI




Outline


    Research environment

    Competence performance assessment

    Experts competence structure mo...
TUG-KMI




Outline


    Research environment

    Competence performance assessment

    Experts competence structure mo...
TUG-KMI




Transformative, Adaptive, Responsive and enGaging
EnvironmenT (TARGET)



          Serious game based learnin...
TUG-KMI




Five key concepts of TARGET




        ¨
  Georg Ottl          April 29, 2010   Page 5/37
TUG-KMI




TARGET Learning Process




        ¨
  Georg Ottl         April 29, 2010   Page 6/37
TUG-KMI




Role of TUG-KMI in TARGET



               TUG-KMI responsible for TARGET learning process
               TUG...
TUG-KMI




Competence performance assessment mockup




        ¨
  Georg Ottl         April 29, 2010        Page 8/37
TUG-KMI




Outline


    Research environment

    Competence performance assessment

    Experts competence structure mo...
TUG-KMI




Competence performance assessment




                                    Competence
               Problems  ...
TUG-KMI




TARGET competence performance assessment
               Interpret observable performance in game experiences i...
TUG-KMI




TARGET competence performance assessment model
authoring



               Interpretation of the game experien...
TUG-KMI




TARGET competence performance assessment
requirements



               Knowledge model/competence state excha...
TUG-KMI




Basic principle of probabilistic assessment of the
competence state


       1. If the learner has the compete...
TUG-KMI




Assessment calculation complexity reduction

               No structure. Possibly 2n states to be updated on ...
TUG-KMI




Mathematical and computational model for competence
assessment
               Nondeterministic assessment4
   ...
TUG-KMI




Outline


    Research environment

    Competence performance assessment

    Experts competence structure mo...
TUG-KMI




Current state Competence Modeler



               Support to create competence assessment models
            ...
TUG-KMI




Hasse diagram visualization




               Figure: Popular knowledge space visualizations

        ¨
  Geo...
TUG-KMI




Current state Competence Modeler
Problems solved and Related Problems




               Computer supported Ha...
TUG-KMI




Hasse diagram visualization




                                →
                  Figure: Version 0.13 and 0...
TUG-KMI




        ¨
  Georg Ottl   April 29, 2010   Page 22/37
TUG-KMI




Current state Competence Modeler
Problems solved and Related Problems

               Creation of a Hasse diag...
TUG-KMI




Current State Competence Modeler
On-Line Demo Afternoon




               https://dev-css.tu-graz.ac.at/




...
TUG-KMI




Outline


    Research environment

    Competence performance assessment

    Experts competence structure mo...
TUG-KMI




Probabilistic Graphical Models
Whatfor?




               Simple way to visualize the structure of a probabil...
TUG-KMI




Graph Terminology




          A graph comprises vertices
          V = (a, b, c, d) connected
          by e...
TUG-KMI




    Definition
    A graph G is called undirected iff

                  ∀A, B ∈ V : (A, B) ∈ E ⇒ (B, A) ∈ E    ...
TUG-KMI




Visual Representation Graph Models




                                                      Figure: Directed ...
TUG-KMI




Probabilistic Graphical Models (1/2)
Whatfor?



               In a probabilistic graph model every vertice r...
TUG-KMI




Graphical probabilistic models


               Question: Can directed probabilistic models such as as
       ...
TUG-KMI




A flexible probabilistic graphical model, the Factor Graph


               Factor Graphs13 as a single represe...
TUG-KMI




Factor graph conversion (1/3)




  Example 1:(A simple probabilistic graph)      S1        S2
  Let f (S1, S2...
TUG-KMI




Factor graph conversion (2/3)


                                                S1        S2

  Example 1:(A f...
TUG-KMI




Factor graph conversion (3/3)



               Efficient algorithms available to calculate probabilities
      ...
TUG-KMI




Thank You!




        ¨
  Georg Ottl   April 29, 2010   Page 36/37
TUG-KMI


      [1]      A. V. Aho, M. R. Garey, and J. D. Ullman. “The Transitive
               Reduction of a Directed ...
TUG-KMI


               British Journal of Mathematical and Statistical Psychology
               41 (1988), pp. 1–23.
  ...
TUG-KMI


      [9]      Frank Kschischang et al. “Factor Graphs and the
               Sum-Product Algorithm”. In: IEEE T...
TUG-KMI




Acronyms




    JUNG Java Universal Network/Graph Framework




        ¨
  Georg Ottl               April 29...
Upcoming SlideShare
Loading in …5
×

Authoring System in TARGET

452 views

Published on

Workshop zu „Authoring Systems"
Organisation: Dietrich Albert (KFU und TU Graz) und Hermann Körndle (TU Dresden)

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
452
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Authoring System in TARGET

  1. 1. TUG-KMI Authoring System in TARGET www.reachyourtarget.org ¨ Georg Ottl Knowledge Management Institute Cognitive Science Section April 29, 2010 ¨ Georg Ottl April 29, 2010 Page 1/37
  2. 2. TUG-KMI Outline Research environment Competence performance assessment Experts competence structure modeler Probabilistic graphical models Factor graphs ¨ Georg Ottl April 29, 2010 Page 2/37
  3. 3. TUG-KMI Outline Research environment Competence performance assessment Experts competence structure modeler Probabilistic graphical models Factor graphs ¨ Georg Ottl April 29, 2010 Page 3/37
  4. 4. TUG-KMI Transformative, Adaptive, Responsive and enGaging EnvironmenT (TARGET) Serious game based learning environment Enterprise Competence Development Improve competences in the project management and innovation domain ¨ Georg Ottl April 29, 2010 Page 4/37
  5. 5. TUG-KMI Five key concepts of TARGET ¨ Georg Ottl April 29, 2010 Page 5/37
  6. 6. TUG-KMI TARGET Learning Process ¨ Georg Ottl April 29, 2010 Page 6/37
  7. 7. TUG-KMI Role of TUG-KMI in TARGET TUG-KMI responsible for TARGET learning process TUG-KMI responsible for workpackage competence development Competence performance assessment component Story adaptation/interventions Integration competence development/TARGET learning process ¨ Georg Ottl April 29, 2010 Page 7/37
  8. 8. TUG-KMI Competence performance assessment mockup ¨ Georg Ottl April 29, 2010 Page 8/37
  9. 9. TUG-KMI Outline Research environment Competence performance assessment Experts competence structure modeler Probabilistic graphical models Factor graphs ¨ Georg Ottl April 29, 2010 Page 9/37
  10. 10. TUG-KMI Competence performance assessment Competence Problems Assessment Competences ¨ Georg Ottl April 29, 2010 Page 10/37
  11. 11. TUG-KMI TARGET competence performance assessment Interpret observable performance in game experiences in regards to a competence state1 . Include motivational state emotional state in interpretation Competence assessment as basis for macro and microadaptive2 interventions and adaptations. Computational model to automatically assess competence state. 1 Klaus Korossy. “Modeling Knowledge as Competence and Performance”. In: Knowledge Spaces: Theories, Empirical Research, Applications. Ed. by Dietrich Albert and Josef Lukas. Mahwah, NJ: Lawrence Erlbaum Associates, 1999, pp. 103–132. 2 Dietrich Albert et al. “Microadaptivity within Complex Learning Situations - a Personalized Approach based on Competence Structures and Problem Spaces”. In: Proceedings of the international Conference on Computers in Education (ICCE 2007). 2007. ¨ Georg Ottl April 29, 2010 Page 11/37
  12. 12. TUG-KMI TARGET competence performance assessment model authoring Interpretation of the game experiences in terms of competences can be done by a social community through inspection. Creation of a model by using the social community observations input (cold start problem) Experts create a model to automatically interpret performance ¨ Georg Ottl April 29, 2010 Page 12/37
  13. 13. TUG-KMI TARGET competence performance assessment requirements Knowledge model/competence state exchange with HRM Systems such as SAP. Assessment in realtime3 to enable microadaptive interventions. 3 O. Conlan et al. Realtime Knowledge Space Skill Assessment for Personalized Digital Educational Games. IEEE, 2009, pp. 538–542. ¨ Georg Ottl April 29, 2010 Page 13/37
  14. 14. TUG-KMI Basic principle of probabilistic assessment of the competence state 1. If the learner has the competence ci , than increase the likelihood of all competence states γci containing ci and decrease the likelihood of all competence states γ ci . 2. If the learner does not have the competence ci , than decrease the likelihood of all competence states γci containing ci and increase the likelihood of all competence states γ ci . ¨ Georg Ottl April 29, 2010 Page 14/37
  15. 15. TUG-KMI Assessment calculation complexity reduction No structure. Possibly 2n states to be updated on every performance observation Definition of a partial order relation on competences exploiting the properties of the “PrerequesiteOf” relation type reduces amount of possible competence states to be taken into consideration. Can experts or the community directly create a competence assessment model?? Authoring tools can help to create a model for competence assessment. ¨ Georg Ottl April 29, 2010 Page 15/37
  16. 16. TUG-KMI Mathematical and computational model for competence assessment Nondeterministic assessment4 Traditional, multiplicative update rule56 Belief propagation networks such as Bayesian Networks7 4 C. Hockemeyer. “A Comparison of non-deterministic procedures for the adaptive assessment of knowledge”. In: Psychologische Beitrage 44.4 (2002), pp. 495–503. 5 Jean-Claude Falmagne and Jean-Paul Doignon. “A class of stochastic procedures for the assessment of knowledge”. In: British Journal of Mathematical and Statistical Psychology 41 (1988), pp. 1–23. 6 Jean-Claude Falmagne and Jean-Paul Doignon. “A markovian procedure for assessing the state of a system”. In: Journal of Mathematical Psychology 32.3 (1988), pp. 232–258. 7 M. Villano. “Probabilistic Student Models: Bayesian Belief Networks and Knowledge Space Theory”. In: Proceedings of the Second International Conference on Intelligent Tutoring Systems. Springer, 1992, 491–498. ¨ Georg Ottl April 29, 2010 Page 16/37
  17. 17. TUG-KMI Outline Research environment Competence performance assessment Experts competence structure modeler Probabilistic graphical models Factor graphs ¨ Georg Ottl April 29, 2010 Page 17/37
  18. 18. TUG-KMI Current state Competence Modeler Support to create competence assessment models Using the well studied PrerequesiteOf relation8 Support experts (psychologists) to create knowledge structures. 8 Dietrich Albert et al. Knowledge Structures. Ed. by Dietrich Albert. New York: Springer Verlag, 1994. ¨ Georg Ottl April 29, 2010 Page 18/37
  19. 19. TUG-KMI Hasse diagram visualization Figure: Popular knowledge space visualizations ¨ Georg Ottl April 29, 2010 Page 19/37
  20. 20. TUG-KMI Current state Competence Modeler Problems solved and Related Problems Computer supported Hasse diagramm creation Visualizations done with the Java Universal Network/Graph Framework (JUNG) framework9 9 J. Madadhain et al. “Analysis and visualization of network data using JUNG”. In: Journal of Statistical Software 10 (2005), pp. 1–35. ¨ Georg Ottl April 29, 2010 Page 20/37
  21. 21. TUG-KMI Hasse diagram visualization → Figure: Version 0.13 and 0.16 ¨ Georg Ottl April 29, 2010 Page 21/37
  22. 22. TUG-KMI ¨ Georg Ottl April 29, 2010 Page 22/37
  23. 23. TUG-KMI Current state Competence Modeler Problems solved and Related Problems Creation of a Hasse diagram reduced to the problem of calculating the minimal transitive reduction of a graph which was shown to have the same complexity as calculation of the transitive closure of a graph10 . Effective calculation and detection of cycles by maintaining the online topological order of the graph11 Visualizations done with the JUNG12 10 A. V. Aho, M. R. Garey, and J. D. Ullman. “The Transitive Reduction of a Directed Graph”. In: SIAM Journal on Computing 1.2 (1972), pp. 131–137. 11 David J. Pearce and Paul H. J. Kelly. “A dynamic topological sort algorithm for directed acyclic graphs”. In: J. Exp. Algorithmics 11 (2006), p. 1.7. 12 J. Madadhain et al. “Analysis and visualization of network data using JUNG”. In: Journal of Statistical Software 10 (2005), pp. 1–35. ¨ Georg Ottl April 29, 2010 Page 23/37
  24. 24. TUG-KMI Current State Competence Modeler On-Line Demo Afternoon https://dev-css.tu-graz.ac.at/ ¨ Georg Ottl April 29, 2010 Page 24/37
  25. 25. TUG-KMI Outline Research environment Competence performance assessment Experts competence structure modeler Probabilistic graphical models Factor graphs ¨ Georg Ottl April 29, 2010 Page 25/37
  26. 26. TUG-KMI Probabilistic Graphical Models Whatfor? Simple way to visualize the structure of a probabilistic model Graphical representation allows insights into the properties of the model Insights into conditional independence properties Complex computations can be expressed in terms of graphical representations; use of graph based inference algorithms that exploit graph properties for calculation. ¨ Georg Ottl April 29, 2010 Page 26/37
  27. 27. TUG-KMI Graph Terminology A graph comprises vertices V = (a, b, c, d) connected by edges Definition A graph G is a pair G = (V , E ), where V is a (finite) set of vertices and E ⊆ V × V is a (finite) set of edges. ¨ Georg Ottl April 29, 2010 Page 27/37
  28. 28. TUG-KMI Definition A graph G is called undirected iff ∀A, B ∈ V : (A, B) ∈ E ⇒ (B, A) ∈ E (1) Two ordered pairs (A, B) and (B, A) are identified and represented by only one undirected edge. Definition A graph G is called directed iff ∀A, B ∈ V : (A, B) ∈ E ⇒ (B, A) ∈ E (2) An edge (A, B) considered to be a directed edge from A towards B ¨ Georg Ottl April 29, 2010 Page 28/37
  29. 29. TUG-KMI Visual Representation Graph Models Figure: Directed Graph Figure: Undirected Graph ¨ Georg Ottl April 29, 2010 Page 29/37
  30. 30. TUG-KMI Probabilistic Graphical Models (1/2) Whatfor? In a probabilistic graph model every vertice represents a random variable The edges express probabilistic relationships between the variables Directed Graphical probabilistic Models Bayesian Networks Undirected Graphical Probabilistic Models Markov Random Fields Loose coupling between statistical variables. ¨ Georg Ottl April 29, 2010 Page 30/37
  31. 31. TUG-KMI Graphical probabilistic models Question: Can directed probabilistic models such as as Bayesian networks be used for assessment. How does believe propagation relate to the classical update rule? Question: How is the relation between directed and undirected probabilistic graphical models? Use of directed and undirected graphical probabilistic models to assess the players state Efficient sum and dot product calculation. ¨ Georg Ottl April 29, 2010 Page 31/37
  32. 32. TUG-KMI A flexible probabilistic graphical model, the Factor Graph Factor Graphs13 as a single representation for directed and undirected graphical probabilistic models Factor Graphs were successfully applied for Bayesian Networks and Markovian Models Multiple applications in artificial intelligence and signal processing based on Factor Graphs 13 Frank Kschischang et al. “Factor Graphs and the Sum-Product Algorithm”. In: IEEE Transactions on Information Theory 47 (2001), pp. 498–519. ¨ Georg Ottl April 29, 2010 Page 32/37
  33. 33. TUG-KMI Factor graph conversion (1/3) Example 1:(A simple probabilistic graph) S1 S2 Let f (S1, S2, S3) be a function of three variables, and suppose that f can be expressed as a product f (S1, S2, S3) = p(S1)p(S2)p(S3|S1, S2) S3 ¨ Georg Ottl April 29, 2010 Page 33/37
  34. 34. TUG-KMI Factor graph conversion (2/3) S1 S2 Example 1:(A factor graph) Let f (S1, S2, S3) be a function of three variables, and suppose that f can be f expressed as a product f (S1, S2, S3) = p(S1)p(S2)p(S3|S1, S2) S3 ¨ Georg Ottl April 29, 2010 Page 34/37
  35. 35. TUG-KMI Factor graph conversion (3/3) Efficient algorithms available to calculate probabilities (Sum-Product algorithm) Makes extensive use of “conditional independent” properties Parallelization possible Approximative algorithms Efficient marginalization ¨ Georg Ottl April 29, 2010 Page 35/37
  36. 36. TUG-KMI Thank You! ¨ Georg Ottl April 29, 2010 Page 36/37
  37. 37. TUG-KMI [1] A. V. Aho, M. R. Garey, and J. D. Ullman. “The Transitive Reduction of a Directed Graph”. In: SIAM Journal on Computing 1.2 (1972), pp. 131–137. [2] Dietrich Albert et al. Knowledge Structures. Ed. by Dietrich Albert. New York: Springer Verlag, 1994. [3] Dietrich Albert et al. “Microadaptivity within Complex Learning Situations - a Personalized Approach based on Competence Structures and Problem Spaces”. In: Proceedings of the international Conference on Computers in Education (ICCE 2007). 2007. [4] O. Conlan et al. Realtime Knowledge Space Skill Assessment for Personalized Digital Educational Games. IEEE, 2009, pp. 538–542. [5] Jean-Claude Falmagne and Jean-Paul Doignon. “A class of stochastic procedures for the assessment of knowledge”. In: ¨ Georg Ottl April 29, 2010 Page 36/37
  38. 38. TUG-KMI British Journal of Mathematical and Statistical Psychology 41 (1988), pp. 1–23. [6] Jean-Claude Falmagne and Jean-Paul Doignon. “A markovian procedure for assessing the state of a system”. In: Journal of Mathematical Psychology 32.3 (1988), pp. 232–258. [7] C. Hockemeyer. “A Comparison of non-deterministic procedures for the adaptive assessment of knowledge”. In: Psychologische Beitrage 44.4 (2002), pp. 495–503. [8] Klaus Korossy. “Modeling Knowledge as Competence and Performance”. In: Knowledge Spaces: Theories, Empirical Research, Applications. Ed. by Dietrich Albert and Josef Lukas. Mahwah, NJ: Lawrence Erlbaum Associates, 1999, pp. 103–132. ¨ Georg Ottl April 29, 2010 Page 36/37
  39. 39. TUG-KMI [9] Frank Kschischang et al. “Factor Graphs and the Sum-Product Algorithm”. In: IEEE Transactions on Information Theory 47 (2001), pp. 498–519. [10] J. Madadhain et al. “Analysis and visualization of network data using JUNG”. In: Journal of Statistical Software 10 (2005), pp. 1–35. [11] David J. Pearce and Paul H. J. Kelly. “A dynamic topological sort algorithm for directed acyclic graphs”. In: J. Exp. Algorithmics 11 (2006), p. 1.7. [12] M. Villano. “Probabilistic Student Models: Bayesian Belief Networks and Knowledge Space Theory”. In: Proceedings of the Second International Conference on Intelligent Tutoring Systems. Springer, 1992, 491–498. ¨ Georg Ottl April 29, 2010 Page 37/37
  40. 40. TUG-KMI Acronyms JUNG Java Universal Network/Graph Framework ¨ Georg Ottl April 29, 2010 Page 37/37

×