Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Authoring Tools, Complexity,
Epistemic Forms, and Cognitive
Development:
“What works for authors?”
Tom Murray
University o...
# 2
Authoring Tool Projects
• SETS – (1991) Equipment
maintenance training
• Eon - ITS authoring for
domain, student, teac...
Authoring Tools for Advanced Technology
Learning Environments
Murray, Blessing,
Ainsworth (Eds)
(2003)
Recent interests applied to ITS
authoring -- Theory
• User Roles vs Tools: Matching Complexity
• Activity Theory: Tools, T...
ITS Authoring Tool
Design Tradeoffs
Usability
DepthFlexibility
Student
model
Expert
model
Pedagogy
model
Curriculum
model
...
Authoring Tool Users
Roles Benefits Problems
(tool use roles) (of that role) (of that role)
Teachers
PRACTICAL
Domain Expe...
Matching Complexity—
Tools vs. User capacity
Complexity?
Tools Users
1) Know your user (anticipate user needs)
2) Usabilit...
Complex Software:
(investment) Risk Assessment
ITS design == AI-complete…….ITS A-Tool design == ITS-Complete!
Activity Theory
Auth. tool
(artifact)
User
(cognition)
Task
(goals)
Com. of
Practice
Rules
(“forms, games”)
Div. of Labor
...
Complexity Coordination
Auth. tool
(artifact)
User
(cognition)
Task
(goals)
Com. of
Practice
Rules
(“forms, games”)
Div. o...
Authoring Tools for All Users?
(Tiered Authoring/Work Flow)
Roles Benefits Problems
(tool use roles) (of that role) (of th...
Capacity is context-dependent
• User complexity capacity: f(S,I,T)
Background skill (generic—see table)
+ Investment in tr...
Sources of (software) system complexity
• Structural complexity (space)
– Object has many properties
– Many parts
– Many t...
Epistemic Forms & Games
(Mental Models)
(Collins & Furgeson, 1993)
• list
• matrix or table
• molecular model
• periodic t...
Epistemic forms in interfaces
Can we estimate the complexity of
epistemic forms/games in authoring
tools?
• >> use cognitive developmental theory
Cognitive Developmentalists
Kegan; Fisher; Commons; Cook-Greuter...
• Human development and learning can be
described in t...
Development: the concept of Fun
Single Rep.
(unconnected list)
Fun is swinging on a swing. It’s sliding on a slide.
Rep.
M...
• Actions at a higher order of hierarchical
complexity organize and transform the lower
order actions
• Complexity level (...
Fisher’s Skill Theory
For any skill
algebra, reading, piano, parenting, tennis…
Addition > Multiplication > Algebra >
Calculus > …
• Single Set (e.g. a list)
(e.g. addition; subtraction…)
• Mapping (e.g...
• System of Systems (an entire complex
system with feedback loops)
• System (e.g. many interconnected parts)
• Mapping (e....
(concrete operations)
(formal op.)
…(post-formal op.)
SYNTAX
(STRUCTURE)
SEMANTICS
(OBJECTS)
Increasing complexity of the
Mental Model
Complexity Level Epistemic Form
Text information fill-in boxes
Lists, choices, s...
Epistemic Forms Complexity
1. Simple objects: lists, sliders, simple relationships
2. Complex mappings: tables, trees, scr...
(cont.)
NEED TESTING/
DEBUGGING
TOOLS
3. Formal systems: Add variables, equations, static models
4. Dynamic Systems:
Loops...
(cont.)
5. Dynamic Systems/ Architectures
(version control; constant monitoring)
Increasing complexity of the
Mental Model
Complexity Level Mental Model Characteristics
1. Simple objects Facts, isolated ...
Roles Benefits Problems
(tool use roles) (of that role) (of that role)
Teachers
PRACTICAL
Domain Experts &
Content Develop...
Review
• User Roles vs Tools: Matching Complexity
• Activity Theory: Tools, Tasks, Users,
Community
• Epistemic Forms/Game...
Conclusions:
If “we build it will they come”?
• Market & Buy-in: Have ITSs demonstrated cost-benefit
yet? In what situatio...
Thank You
Extra Slides
GIFT Functional Block Diagram
Behavior Recorder
Data Shop
CTAT’s Modular Architecture
Student Interface
(or external problem-solving
environment)
Cool M...
Principles
• Syntax/structure in increasing complexity (first);
– they repeat for each:
• Semantic tier (abstraction level...
About 40 ALE authoring tools
CATEGORY PROJECTS/SYSTEMS
1 Curriculum Sequencing
and Planning
Swift/DOCENT, IDE, ISD Expert,...
Authoring Tools, Complexity, Epistemic Forms, and Cognitive Development
Authoring Tools, Complexity, Epistemic Forms, and Cognitive Development
Authoring Tools, Complexity, Epistemic Forms, and Cognitive Development
Authoring Tools, Complexity, Epistemic Forms, and Cognitive Development
Authoring Tools, Complexity, Epistemic Forms, and Cognitive Development
Upcoming SlideShare
Loading in …5
×

Authoring Tools, Complexity, Epistemic Forms, and Cognitive Development

675 views

Published on

ITS-2014 authoring tool workshop, Tom murray

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Authoring Tools, Complexity, Epistemic Forms, and Cognitive Development

  1. 1. Authoring Tools, Complexity, Epistemic Forms, and Cognitive Development: “What works for authors?” Tom Murray University of Massachusetts ITS 2014 Workshop: Intelligent Tutoring System Authoring Tools
  2. 2. # 2 Authoring Tool Projects • SETS – (1991) Equipment maintenance training • Eon - ITS authoring for domain, student, teaching models, and interface • MetaLinks - hyperbook authoring tool • SimForest-G - Glass box simulation authoring • Rashi - Coached inquiry learning environment w/ authoring tool • Wayang/MathSprings – (2014) teacher tools
  3. 3. Authoring Tools for Advanced Technology Learning Environments Murray, Blessing, Ainsworth (Eds) (2003)
  4. 4. Recent interests applied to ITS authoring -- Theory • User Roles vs Tools: Matching Complexity • Activity Theory: Tools, Tasks, Users, Community • Epistemic Forms/Games • Developmental Theory (of complexity)
  5. 5. ITS Authoring Tool Design Tradeoffs Usability DepthFlexibility Student model Expert model Pedagogy model Curriculum model easy, efficient? realistic, “intelligent? • Interoperability • Multi-user ITS • ... Many domains, and types?
  6. 6. Authoring Tool Users Roles Benefits Problems (tool use roles) (of that role) (of that role) Teachers PRACTICAL Domain Experts & Content Developers PARTIAL Instructional Designers & Learning Theorists THEORETICAL Knowledge Engineers and ITS Developers EXPERIENCED Computer scientists & Software developers (ACTUAL?!) Complexity capacity. Don't have to build to a real user base. "its intuitively obvious to the casual observer…" HIGH Know learning theories & research Rare; not trained in knowledge engineering MED Know the tools; Are sometimes also plugged into user testing May not know what it is like to teach or learn the material MED-HIGH Complexity Design Capacity Practical experience Not good at articulating or abstracting expertise LOW Auth. tool infers the instructional methods A fixed instructional method MED
  7. 7. Matching Complexity— Tools vs. User capacity Complexity? Tools Users 1) Know your user (anticipate user needs) 2) Usability Testing (“early and often”) 3) Theory (of usability, cognition...)
  8. 8. Complex Software: (investment) Risk Assessment ITS design == AI-complete…….ITS A-Tool design == ITS-Complete!
  9. 9. Activity Theory Auth. tool (artifact) User (cognition) Task (goals) Com. of Practice Rules (“forms, games”) Div. of Labor ITS (outcome) Jonassen & Rohrer-Murphy 1999; Engestrom et al. 1999;
  10. 10. Complexity Coordination Auth. tool (artifact) User (cognition) Task (goals) Com. of Practice Rules (“forms, games”) Div. of Labor ITS (outcome) Complexity matching: 1. User <> Tool 2. Task <> User 3. COP (& user) <> Tool • Cognitive Complexity (user) • Task Complexity • Tool Complexity • Socio-cognitive complexity (COP)
  11. 11. Authoring Tools for All Users? (Tiered Authoring/Work Flow) Roles Benefits Problems (tool use roles) (of that role) (of that role) Teachers PRACTICAL Domain Experts & Content Developers PARTIAL Instructional Designers & Learning Theorists THEORETICAL Knowledge Engineers and ITS Developers EXPERIENCED Computer scientists & Software developers (ACTUAL?!) Complexity capacity. Don't have to build to a real user base. "its intuitively obvious to the casual observer…" HIGH Know learning theories & research Rare; not trained in knowledge engineering MED Know the tools; Are sometimes also plugged into user testing May not know what it is like to teach or learn the material MED-HIGH Complexity Design Capacity Practical experience Not good at articulating or abstracting expertise LOW Auth. tool infers the instructional methods A fixed instructional method MED
  12. 12. Capacity is context-dependent • User complexity capacity: f(S,I,T) Background skill (generic—see table) + Investment in training A-Tool + Time available to author this ITS
  13. 13. Sources of (software) system complexity • Structural complexity (space) – Object has many properties – Many parts – Many types of parts – Many relationships – Many types of relationships • Perspectival complexity – alternatives, hypotheticals, variables, decision spaces • Dynamic complexity – Loops, Feedback, recursion relationships (“non-linearity”)
  14. 14. Epistemic Forms & Games (Mental Models) (Collins & Furgeson, 1993) • list • matrix or table • molecular model • periodic table • web page menu • x-y graph • pert chart • binary tree • floor plan • street map • org. chart • musical score • timeline • cause/effect diagram • network • relational database • sentence diagram • term paper outline
  15. 15. Epistemic forms in interfaces
  16. 16. Can we estimate the complexity of epistemic forms/games in authoring tools? • >> use cognitive developmental theory
  17. 17. Cognitive Developmentalists Kegan; Fisher; Commons; Cook-Greuter... • Human development and learning can be described in terms of "qualitative differences in mental complexity." • …that add a hierarchical "structural perspective in analyzing the organization of actions and thought."
  18. 18. Development: the concept of Fun Single Rep. (unconnected list) Fun is swinging on a swing. It’s sliding on a slide. Rep. Mapping (connections) Fun is when Tommy and I put blocks together and then knock them down so that they make a loud noise that makes us laugh. Rep. System (interconnections) Fun is different things. Sometimes I like to climb…that makes me… Single Abstr. (unconnected list) Fun is a way of enjoying yourself. It is a form of pleasure. Abst. Mapping (connections) There are a variety of ways that a person can have fun. Some people enjoy physical activities, like sports or just exercise. Some people…
  19. 19. • Actions at a higher order of hierarchical complexity organize and transform the lower order actions • Complexity level (or “order”) based on: 1. complexity of objects operated upon (vertical complexity; order of abstraction) 2. complexity of object coordination (horizontal complexity; structure of objects)
  20. 20. Fisher’s Skill Theory For any skill algebra, reading, piano, parenting, tennis…
  21. 21. Addition > Multiplication > Algebra > Calculus > … • Single Set (e.g. a list) (e.g. addition; subtraction…) • Mapping (e.g. linear causal link) (coordinating addition & subtraction) • System (e.g. many interconnected parts) (e.g. coordinating +, -, x, /) • [System of Systems (an entire complex system) (moving to Algebra)]
  22. 22. • System of Systems (an entire complex system with feedback loops) • System (e.g. many interconnected parts) • Mapping (e.g. linear causal link) • Single Set (e.g. a list)
  23. 23. (concrete operations) (formal op.) …(post-formal op.) SYNTAX (STRUCTURE) SEMANTICS (OBJECTS)
  24. 24. Increasing complexity of the Mental Model Complexity Level Epistemic Form Text information fill-in boxes Lists, choices, sliders, and check boxes Tables and matrices Hierarchies and trees Simple scripts, Forms, schemas, or templates Procedures with branches Variables/Equations and Boolean logic Structural models: concept networks, boxology diagrams Causal and constraint models, Decision Trees Behavioral/procedural models: If/then and rule- based procedural representations Complex interactions 5. Architectures & Ecosystems Systems of systems, models, or rule-sets 1. Simple objects/info 2. Mappings & Abstractions 3. Formal Systems 4. Dynamic Systems
  25. 25. Epistemic Forms Complexity 1. Simple objects: lists, sliders, simple relationships 2. Complex mappings: tables, trees, scripts, concept nets
  26. 26. (cont.) NEED TESTING/ DEBUGGING TOOLS 3. Formal systems: Add variables, equations, static models 4. Dynamic Systems: Loops, conditionals, dynamic/constrain models, rule systems
  27. 27. (cont.) 5. Dynamic Systems/ Architectures (version control; constant monitoring)
  28. 28. Increasing complexity of the Mental Model Complexity Level Mental Model Characteristics 1. Simple objects Facts, isolated info-bits 2. Mappings & Abstractions Many relatinships, fairly linear, predicatable 3. Formal Systems Abstractions, variables, decisions 4. Dynamic Systems Non-linearities, sub-systems, decision trees 5. Architectures & Ecosystems Complex interactions, whole-systems, evolving, unpredicatable
  29. 29. Roles Benefits Problems (tool use roles) (of that role) (of that role) Teachers PRACTICAL Domain Experts & Content Developers PARTIAL Instructional Designers & Learning Theorists THEORETICAL Knowledge Engineers and ITS Developers EXPERIENCED Computer scientists & Software developers (ACTUAL?!) Complexity capacity. Don't have to build to a real user base. "its intuitively obvious to the casual observer…" HIGH Know learning theories & research Rare; not trained in knowledge engineering MED Know the tools; Are sometimes also plugged into user testing May not know what it is like to teach or learn the material MED-HIGH Complexity Design Capacity Practical experience Not good at articulating or abstracting expertise LOW Auth. tool infers the instructional methods A fixed instructional method MED
  30. 30. Review • User Roles vs Tools: Matching Complexity • Activity Theory: Tools, Tasks, Users, Community • Epistemic Forms/Games • Developmental Theory (of complexity)
  31. 31. Conclusions: If “we build it will they come”? • Market & Buy-in: Have ITSs demonstrated cost-benefit yet? In what situations? • Creating a pipeline for training and trained ITS authors and knowledge engineers • Building communities of practice (examples: CTAT, WISE, Knowledge Forum…) • Expectation management: matching tool and user complexity (and constraining the breadth/depth of the outcome ITSs) • Building an ITS authoring tool is like…?? What known completed project? Using lessons from a parallel domain?
  32. 32. Thank You
  33. 33. Extra Slides
  34. 34. GIFT Functional Block Diagram
  35. 35. Behavior Recorder Data Shop CTAT’s Modular Architecture Student Interface (or external problem-solving environment) Cool Modes CyclePad Java Swing Flash Learner Management System LMS Tutor Shop Cognitive Model Development Tools TDK (Lisp-based) Jess tools (Java- based) Editor Eclipse Tutor Engine Tertle (Lisp-based) + Model Tracer Jess (Java--based) + Model Tracer Example Tracer (Java-based) Example Tracer (Flash-based) GUI Builder IntelliJ Dreamweaver Code Warrior Netbeans Flash MX 2004
  36. 36. Principles • Syntax/structure in increasing complexity (first); – they repeat for each: • Semantic tier (abstraction level; “depth” of the objects) • New semantic tiers (depth) emerge with sufficient syntax (breadth) • You can’t skip levels in “development” • Building intuitions of complexity is more important that a formal theory of complexity.
  37. 37. About 40 ALE authoring tools CATEGORY PROJECTS/SYSTEMS 1 Curriculum Sequencing and Planning Swift/DOCENT, IDE, ISD Expert, Expert CML 2 Tutoring Strategies REDEEM (& COCA), Eon, GTE 3 Device Simulation and Equipment Training SIMQUEST, XAIDA, RIDES, DIAG, Instructional Simulator 4 Domain Expert System Demonstr8, DIAG, D3 Trainer, Training Express 5 Multiple Knowledge Types XAIDA, DNA, Instructional Simulator & IDVisualizer, ID-Expert, IRIS, CREAM- Tools 6 Intelligent/adaptive Hypermedia InterBook, MetaLinks, CALAT, GETMAS, TANGOW, ECSAIWeb 7 Special Purpose IDLE-Tool/Imap/Indie, LAT, BioWorld Case Builder, WEAR 40-60 as of 2002

×