Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Valladolid Presentation - Nov, 16, 2011


Published on

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

The Valladolid Presentation - Nov, 16, 2011

  1. 1. Ubiquitous Orchestration Perspectives from the Pedagogical Space and Tools for the CSCL teacher Stavros Demetriadis Assistant Professor [email_address] @sdemetri Aristotle University of Thessaloniki , Department of Informatics Greece
  2. 2. <ul><li>...boiled down to a Special Theory of Orchestration </li></ul>A General Theory of the Universe
  3. 3. A more modest objective... <ul><li>(a) Ubiquitous Orchestration </li></ul><ul><ul><li>A comment on Orchestration linking to the Pedagogical Space </li></ul></ul><ul><ul><li>Implications (linking to EEE...) </li></ul></ul><ul><ul><li>Flexibility: “Adaptation Patterns” </li></ul></ul><ul><li>(b) Orchestrating the CSCL classroom </li></ul><ul><ul><li>Extending IMS-LD capabilities to cater for adaptation </li></ul></ul><ul><ul><li>MentorChat: Orchestrating students’ dialogue </li></ul></ul><ul><ul><li>Augmented Reality: The PROTEAS project </li></ul></ul><ul><ul><li>Scripting: What happens in scripted collaboration when it is not scripted? </li></ul></ul>
  4. 4. The C&B vs. the CSCL classroom I <ul><li>Did the “chalk & blackboard” teacher have orchestration problems? </li></ul><ul><ul><li>No chalk (sends a student to bring some) </li></ul></ul><ul><ul><li>Wet sponge  Wet board (has to wait before writing) </li></ul></ul><ul><ul><li>No curtains (striking sun  writings on the board difficult to read) </li></ul></ul><ul><li>But look at the poor CSCL teacher: </li></ul><ul><ul><li>A student is missing  Three students (instead of two) have to work together </li></ul></ul><ul><ul><li>– Is the script efficient for a group of three also? </li></ul></ul><ul><ul><li>Some students have problems logging in the web system </li></ul></ul><ul><ul><li>– What exactly is the problem? </li></ul></ul><ul><ul><li>Students surfing the Internet / others playing a computer game </li></ul></ul><ul><ul><li>– What do I do? (while trying to help the others with logging in) </li></ul></ul><ul><ul><li>Supervising the computer-supported activity </li></ul></ul><ul><ul><li>– Do they work as instructed/scripted? </li></ul></ul>A comment on orchestration
  5. 5. The C&B vs. the CSCL classroom II <ul><li>... the teacher can easily overcome problems in the C&B classroom </li></ul><ul><ul><li>Interaction in the C&B classroom is low </li></ul></ul><ul><ul><li>Students sit quiet and read or write individually </li></ul></ul><ul><ul><li>No communication ..... </li></ul></ul><ul><li>But need Collaboration and Technology to make it really a mess! </li></ul><ul><ul><li>C & T multiply the interactions </li></ul></ul><ul><ul><li>Teacher - Student // Student – Student // Student – Content </li></ul></ul><ul><ul><li>// Student – Technology </li></ul></ul>A comment on orchestration
  6. 6. Orchestration I <ul><li>Orchestration: Design , Enactment and Adaptation of rich pedagogical scenarios </li></ul><ul><li>Over... </li></ul><ul><ul><li>Multiple Spaces (f2f, Web, 3D Worlds, Augmented reality) </li></ul></ul><ul><ul><li>Multiple Digital Tools (in the above spaces) </li></ul></ul><ul><ul><li>Multiple Learning Activities (using various tools in spaces) </li></ul></ul><ul><ul><li>Multiple Levels (Individual, Small Group, etc.) </li></ul></ul><ul><ul><li>Multiple Agents (teacher, student, technology) </li></ul></ul><ul><ul><li>Multiple Synergies </li></ul></ul><ul><ul><li>.................................................................. </li></ul></ul>A comment on orchestration
  7. 7. Orchestration II <ul><li>Ubiquitous : Continuous and over many dimensions </li></ul><ul><li>Distributed : to many agents (off-loading..) </li></ul>A comment on orchestration Example: Nick misses the Monday lesson because he is ill. The teacher has designed a dyad-based script but now student number is odd (23). The system supporting the activity adapts the activity so that a triad can also work productively.  Orchestration duties are off-loaded to the system. Example: T he teacher asks the students to form 4-member small groups and enact a jigsaw-based scenario. The teachers says “We work in groups of four as we did the other time”. Students experienced with the scenario understand the hint for the Jigsaw activity.  Orchestration duties are off-loaded to the students
  8. 8. Reflections <ul><li>Type I Reflection </li></ul>TEACHER Mapping PS onto TS A comment on orchestration Technology Spaces (TS) Data-based Pedagogical Space (PS) Pattern-based Cool technology! I know a good pattern that I want to implement. How can this technology help me? Type II Reflection Cool technology! I want to use it. But what patterns fit well with these tools? Mapping TS onto PS
  9. 9. An example: Twitter <ul><li>Type II Reflections : Twitter allows students to send short messages. Is this of some use? </li></ul><ul><li>...Well, I put another screen in my classroom and project there students’ Twitter messages. Lets see what happens... </li></ul><ul><li>Type I Reflections : a good learning “pattern” is that my students communicate with domain experts. I can use Twitter to support this expert-novice communication by enabling students to “follow” experts. </li></ul>A comment on orchestration
  10. 10. Reflections (cont’d) <ul><li>Linking Spaces: Opportunities to teachers to reflect on </li></ul><ul><li>PS  (Physical + Multiple TSs) relationship </li></ul><ul><li>in at least three modes: </li></ul><ul><li>(a) “Complementary” Mode  Distribute a pattern over various Spaces </li></ul><ul><li>(b) “Constrain” Mode  Good practices “constrain” the use of novel tools/services/representations in a TS </li></ul><ul><li>(c) “Construct” Mode  Reflect on activities successfully distributed over many Spaces and construct an abstract understanding enriching the Pedagogical Space and possibly the affordances of TSs </li></ul>A comment on orchestration
  11. 11. A Model of Reflections Complement? (Type I) Construct? (Type I & II) Constrain? (Type II) A comment on orchestration Reflections Data-based Continuity Discontinuity Technology-based Space (TS) Pedagogical Space (PS) Pattern-based TEACHER Technology-based Space (TS) Physical Space
  12. 12. Implications I <ul><li>Support Type I Reflection </li></ul><ul><li>Simple – easy to handle – tools to allow intuitive integration into pattern enactment (i.e. support mapping of PS onto TS) </li></ul><ul><ul><li>cf. “Awareness” tools (Dillenbourg in Stellar Report, Dillenbourg et al. 2011) </li></ul></ul>Implications Example : Simple interaction analysis tools can be easily used by teachers for monitoring and adaptation purposes
  13. 13. Implications II <ul><li>Support Type II Reflection </li></ul><ul><li>Provide teachers with good practices to make optimal use of new tools/technologies (i.e. help mapping TS onto PS) </li></ul><ul><ul><li>(cf. Dimitriadis in STELAR Report - Dillenbourg et al. 2011) </li></ul></ul>Implications Example : Provide scenarios (“scripts”) for using educational robots in the classroom to help improve (a) domain learning, (b) students’ collaboration and metacognitive skills development
  14. 14. Implications III An Architecture for Orchestration Tools <ul><li>Easy to use (low complexity) pattern-supported, tools </li></ul><ul><ul><li>“ Pattern-intuitive” level </li></ul></ul>Interface Level Specific to Spaces/Tools Specific to Spaces/Tools Rule Level <ul><li>Advanced users define the Rules for Interface Level tools </li></ul><ul><ul><li>Information-based </li></ul></ul>DataFlow Level <ul><li>Defining DataFlow (protocols, models, etc.) </li></ul><ul><ul><li>Data-based </li></ul></ul>Linking Spaces Modified across Spaces
  15. 15. Implications IV More on the proposed Architecture ... <ul><li>What Representations ? </li></ul><ul><li>What Patterns ? </li></ul>DataFlow Level <ul><li>DataFlow streaming between spaces </li></ul><ul><li>What Data ? </li></ul><ul><li>How to stream data across multiple Spaces ? </li></ul><ul><li>Semantics to describe Data? </li></ul><ul><li>Flexibility : Cater for future extensions of data-relevant protocols? </li></ul>Linking Spaces Modified over Spaces Specific to Spaces/Tools Rule Level <ul><li>What Information ? (from Data) </li></ul><ul><li>What Rule? (from Information) </li></ul><ul><li>Tools to define Rules </li></ul>Interface Level Specific to Spaces/Tools <ul><li>Relevant to EEE </li></ul>
  16. 16. Adaptation patterns <ul><li>Introducing Flexibility </li></ul><ul><li>Exemplifying the reflection-based approach & Tool architecture </li></ul>
  17. 17. Flexibility <ul><li>Pre-defined : I can integrate in my design alternative activity flow to be enacted if conditions occur </li></ul><ul><li>On-the-fly : I (or the system) can trigger alternative activity flow during the activity alerted by specific events </li></ul>
  18. 18. Adaptation Patterns <ul><li>A core idea of pedagogical value on how to adapt the collaborative learning activity when specific conditions occur </li></ul><ul><li>… an AP suggests a valuable alternative (to the whole or part of the initial design) depending on conditions </li></ul><ul><li>IF [ something specific occurs] THEN [ adapt your design] </li></ul>Example: Students work in dyads on a collaborative task. An “advanced” student works in a dyad with a “beginner” student. The task is easy for the advanced and has lost interest. Adaptation : the teacher adjusts the task to address the advanced level of student, i.e. makes the task more demanding and interesting for the advanced student (of course without failure of the beginner). Adaptation Pattern: “Advance the Advanced” Adaptation patterns
  19. 19. Adaptation Pattern Life-Cycle Adaptation patterns
  20. 20. List of Adaptation Patterns Karakostas & Demetriadis, 2010 Adaptation patterns
  21. 21. FlexCoLab: Implementing Adaptation Patterns 1/2 Adaptation patterns <ul><li>Adapt the resources </li></ul><ul><li>Adapt the activity </li></ul><ul><li>Novice </li></ul><ul><li>Intermediate </li></ul><ul><li>Advanced </li></ul>
  22. 22. PPR: Pyramid Peer Review Implementing Adaptation Patterns 2/2
  23. 23. Overall <ul><li>Adaptation patterns “live” in Pedagogical Space </li></ul><ul><ul><li>Identifiable through Teacher activity analysis & Interaction analysis </li></ul></ul><ul><li>Tools like FlexCoLab & PPR aim to help teachers identify favorite adaptations and integrate them in their design (pre-defined) </li></ul><ul><li>The tools support mainly Type I Reflections at Interface & Rule level </li></ul><ul><ul><li>known patterns to implement with technology </li></ul></ul><ul><li>Next steps </li></ul><ul><ul><li>Better interface: Difficulties with complex pattern setting interface </li></ul></ul><ul><ul><li>Evaluation in context </li></ul></ul><ul><ul><li>Clearly define Interface vs. Rule vs. DataFlow levels </li></ul></ul><ul><ul><li>What about On-the-fly adaptations? </li></ul></ul>Adaptation patterns
  24. 24. IMS-LD Enhancing IMS-LD to cater for adaptivity
  25. 25. Adaptation Pattern Specification <ul><li>Input, Rule(-s), Model(-s), Output (IRMO) </li></ul>RULES INPUT Interaction Analysis MODELLED ENTITIES OUTPUT On Screen Representation A D A P T A T I O N P A T T E R N or db manifest <ul><li>During design: </li></ul><ul><ul><li>Define monitored parameters (e.g. from interaction analysis tools) </li></ul></ul><ul><ul><li>Rules (the adaptation model of the pattern) are hard-wired to the pattern </li></ul></ul><ul><ul><li>Define Output (form, content, etc.) </li></ul></ul>IMS-LD
  26. 26. Extending IMS-LD capabilities <ul><li>IMS-LD modeling language has limited capabilities in expressing and implementing complex information processing (necessary when complex adaptive interventions need to be made by the CSCL system) </li></ul><ul><li>How IMS-LD, a de-facto standard in the CSCL area, could facilitate the required adaptive behavior of a CSCL system through communication with external software components? </li></ul><ul><ul><li>such as web services, communication tools or virtual learning environments (VLEs). </li></ul></ul>IMS-LD
  27. 27. MAPIS: A reference architecture for implementing ACLS systems with IMS-LD IMS-LD
  28. 28. MentorChat Orchestrating Students’ Dialogue
  29. 29. MentorChat Design Rationale <ul><li>Dialogues between students and tutors: a prominent component of effective tutoring. </li></ul><ul><li>In CSCL: a supportive conversational agent may model and intelligibly trigger the peer dialogue. </li></ul><ul><li>The MentorChat agent acts as a peer discourse facilitator enabling the teacher to define what type of intervention to implement. </li></ul><ul><li>Not a teacher substitute, but a teacher‘s tool to orchestrate intervention/support strategy during students’ discussions. </li></ul>MentorChat
  30. 30. MentorChat
  31. 31. MentorChat Architecture MentorChat <ul><li>Teacher defined </li></ul>
  32. 32. Teacher Orchestration using MentorChat <ul><li> Define a set of rules to guide the agent’s behavior </li></ul><ul><li>A rule can be divided into three parts: </li></ul><ul><li>(a) an event </li></ul><ul><ul><li>A set of keywords that can be used in conjunction with regular expressions in order to recognize specific phrases and language structures </li></ul></ul><ul><li>(b) a triggering prompt </li></ul><ul><ul><li>When an event occurs the agent posts an appropriate triggering prompt as a participant that joins the dialogue </li></ul></ul><ul><li>(c) a final prompt </li></ul><ul><ul><li>is posted by the agent at the end of the conversation </li></ul></ul><ul><ul><li>to display some important aspects of the problem‘s solution that have not been mentioned in the group discussion </li></ul></ul>MentorChat
  33. 33. Dialogue Sample <ul><li>(Yannis and Nelly discuss online issues of cognitive theory for Multimedia learning) </li></ul><ul><li><Y ANNIS >: ... in the brain some connections are formed with the prior knowledge and stored in long-term memory </li></ul><ul><li><Tutor>: What do you think Nelly about the role of long-term memory? </li></ul><ul><li><N ELLY >: In order to store information in the long-term memory, it should... </li></ul><ul><li><Y ANNIS >: ... we should also mention the concepts of Selecting and Organizing ... </li></ul><ul><li><Tutor>: Do you think Nelly that the concept of Organizing can be part of your answer? Why? </li></ul><ul><li><N ELLY >: Of course, because the better the two models (verbal and pictorial) are organized in memory the better the integration with prior knowledge. Is this a satisfactory answer? </li></ul><ul><li><Y ANNIS >: Wait Nelly , I will write down the answer and then you can check if something is missing </li></ul><ul><li><Tutor>: Before you submit your final answer please consider commenting on the concepts of: verbal and pictorial model. </li></ul><ul><li><N ELLY >: ok, then. Lets discuss these issues and I think also we need to mention the redundancy and contiguity principles </li></ul>MentorChat
  34. 34. <ul><li>Two studies: School and University classes </li></ul><ul><li>Students discussing online in dyads in lab conditions </li></ul><ul><li>MentorChat posts messages whenever a concept modeled in the vocabulary appears in students’ discussion </li></ul>Teacher monitoring students conversation Preliminary Evaluation I MentorChat
  35. 35. Preliminary Evaluation II <ul><li>MentorChat interface has been evaluated as user friendly and easy to use </li></ul><ul><li>Students mention that agent‘s messages helped their group discussion to advance and improve their collaboration </li></ul><ul><li>Also: the prompts before the final answer very helpful to refresh their domain knowledge </li></ul><ul><li>School : The teacher found very useful the ability to monitor the discussion from a simple panel </li></ul><ul><li>She stated that “… a MentorChat activity, ..., can motivate students to actively participate in a collaborative task… ‘ She also said that agent‘s messages can promote the critical thinking of students . </li></ul><ul><li>University : Preliminary statistical analysis indicates learning benefits for the treatment condition (discussion in MentorChat) </li></ul>MentorChat
  36. 36. Implications for Orchestration I <ul><li>MentorChat can help teachers to orchestrate multiple students’ discussions by predefining interventions (rules) </li></ul><ul><li>Beneficial interventions may be offered to multiple groups </li></ul><ul><li>PROBLEMS </li></ul><ul><li>What is the necessary agent intelligence to intervene in a human-like way? </li></ul><ul><li>Students after using MentorChat model the behaviour of the agent and may not pay attention to what is said </li></ul><ul><li>Sometimes students ask the agent for further support </li></ul><ul><ul><li>Unavailable at the moment </li></ul></ul>MentorChat
  37. 37. Implications for Orchestration II <ul><li>Relevant to EEE? </li></ul><ul><li>An agent in a space can become mediator for teacher interventions and orchestration actions </li></ul><ul><li>Once these actions are predefined then multiple students groups could be benefited (or not?) from interacting with this agent </li></ul>MentorChat
  38. 38. The PROTEAS project PRO gramming T angibl E A ctivity S ystem
  39. 39. Tangible (AR) Introductory programming constructs PROTEAS
  40. 40. Virtual (Isomorphic to AR) PROTEAS
  41. 41. Some preliminary research results <ul><li>Participants and Method </li></ul><ul><ul><li>61 children (36 boys, 25 girls) </li></ul></ul><ul><ul><li>3 age groups (5-6, 7-8, 11-12) </li></ul></ul><ul><ul><li>They played with both systems </li></ul></ul><ul><ul><li>Qualities and Quantities recorded </li></ul></ul><ul><ul><li>Pictorial questionnaire </li></ul></ul><ul><li>Likeness (Total) </li></ul><ul><ul><li>Tangible 80,0% </li></ul></ul><ul><ul><li>Virtual 20,0% </li></ul></ul><ul><li>Easiness (Total) </li></ul><ul><ul><li>Tangible 64,4% </li></ul></ul><ul><ul><li>Virtual 35,6% </li></ul></ul><ul><li>Play again With Friends (Total) </li></ul><ul><ul><li>Tangible 75,0 </li></ul></ul><ul><ul><li>Virtual 25,0% </li></ul></ul><ul><li>Tangible easiness between ages </li></ul><ul><ul><li>5-6 years 7-8 years 11-12 years </li></ul></ul><ul><ul><li>94,1% 73,1% 18,8% </li></ul></ul><ul><li>This is significant and confirms that tangible interface should be considered as easier for the younger </li></ul>PROTEAS
  42. 42. Implications for Orchestration <ul><li>Relevant to EEE? </li></ul><ul><li>Students may understand faster and deeper the affordances of a space if they use it in parallel with another isomorhic space (?) </li></ul><ul><li>Restrictions in one space may help students “discover” and apply self-orchestration rules that can be transferred to other spaces (?) </li></ul>PROTEAS
  43. 43. What happens in scripted collaboration when it is not scripted? Orchestrating scripted student groups
  44. 44. A common orchestration problem <ul><li>If a script works well when it guides the collaborative activity of a dyad ... </li></ul><ul><li>...does it work equally well when it guides a triad of students? </li></ul><ul><li>Research Questions: </li></ul><ul><li>1) How is the script efficiency dependent on group size ? </li></ul><ul><li>2) What happens when the script leaves unscripted a part of the activity for the three students? </li></ul>Scripting
  45. 45. Study design <ul><li>Learning domain : Algorithms + Algorithm visualization tools </li></ul><ul><li>Control : Students working in triads to solve an algorithm problem unscripted </li></ul><ul><ul><li>Simple Prompt: “Collaborate” </li></ul></ul><ul><li>Treatment : Students working also in triads on the same problem </li></ul><ul><ul><li>Additionally: a peer-tutoring script assigned successively each student in the triad to become tutor and pose an issue to his/her colleagues relevant to solving the problem </li></ul></ul><ul><ul><li>Simple Prompt: “Collaborate” </li></ul></ul><ul><ul><li>When the Student-tutor posed the issue the two other peers were unscripted on how to collaborate </li></ul></ul>Scripting
  46. 46. Alcolab tool Scripting
  47. 47. “ Issue” level analysis <ul><li>Suppose S1 poses an issue </li></ul><ul><li>Control : while discussing with others </li></ul><ul><li>Treatment : Triggered by the script </li></ul><ul><li>S2 and S3 are in both conditions unscripted as to what to do next - What do they do? </li></ul><ul><li>Issue level analysis : Refers to the analysis of peer interaction (between S2 and S3) after S1 posed an issue </li></ul>Scripting S1 S2 S3 S1 poses an issue
  48. 48. Results <ul><li>Control behavior </li></ul><ul><li>Treatment behavior </li></ul>S1 poses an issue Peer interaction for answering: Directly to S1 Leaving the other peer (S3) in a low-participation condition Peer interaction for answering: Interaction between S2 & S3 Then answering the issue by S1 This behavior resulted to triads improved learning outcomes (port-test) Scripting S1 S2 S3 S1 S2 S3
  49. 49. Why? <ul><li>When a script leaves student interaction unscripted then students’ internal scripts are activated </li></ul><ul><li>In this case internal scripts responding to prompt “collaborate” were activated </li></ul><ul><li>CONTROL : All Students identified themselves as peers </li></ul><ul><ul><li>Therefore directly answering the issue by S1 was considered as collaboration </li></ul></ul><ul><li>TREATMENT : assigning the role of tutor to S1 resulted in the other two students S2 & S3 identifying themselves as peers </li></ul><ul><ul><li>Therefore “collaborate” meant interact with peer before answering to the tutor </li></ul></ul>Scripting
  50. 50. Implications for Orchestration <ul><li>Assigning roles to students may create social hierarchies and activate peer interaction among certain partners </li></ul><ul><li>Assigning such roles can be a teachers’ handy technique to activate peer interaction between peers </li></ul><ul><li>A peer tutoring script effective when implementing with dyads may also be effective when implemented with triads </li></ul><ul><li>Relevant to EEE? </li></ul><ul><li>Social hierarchies established in a space (for example with virtual partners) may direct and trigger collaboration (discource activity) between students (in this or other spaces) </li></ul>Scripting
  51. 51. Final Remarks <ul><li>Orchestration: Ubiquitous & Distributed </li></ul><ul><li>Pedagogical Space: establishes continuity </li></ul><ul><li>Support teachers’ reflections while linking (mapping) various Physical space & Technology-based Spaces on Pedagogical Space </li></ul><ul><li>A proposed architecture for orchestration tool entails three levels: Interface, Rule, DataFlow </li></ul><ul><li>...but will these ideas and guidelines help making orchestration simpler and easier for the average teacher? </li></ul>
  52. 52. Ευχαριστώ! Muchas Gracias!
  53. 53. Invitations <ul><li>(1) IEEE Learning Technology Newsletter </li></ul><ul><li> </li></ul><ul><li>Next issue: Adaptive & Intelligent Systems for Collaborative Learning </li></ul><ul><li>Guest Editor: Stavros Demetriadis </li></ul><ul><li>Critical Dates: Deadline for submission of articles: 15 December 2011 </li></ul><ul><ul><li>1000 words submission </li></ul></ul><ul><li>(2) 4 th International Conference on Intelligent Networking and Collaborative Systems (INCOS-2012) </li></ul><ul><ul><li>IEEE Technical Sponsorship // Proceedings by IEEE CPS </li></ul></ul><ul><ul><li>September 19-21, Bucarest, Romania </li></ul></ul><ul><li>(3) 3rd International Workshop on Adaptive & Intelligent Systems for Collaborative Learning (IWASCL-2012) </li></ul><ul><ul><li>In conjunction with INCoS-2012 </li></ul></ul><ul><ul><li>Previous: IWASCL-2009 in Barcelona, IWASCL-2010 in Thessaloniki </li></ul></ul>