Denis Smolin,       American University in Bosnia and Herzegovina                                     Sergey Butakov,SolBr...
…                                           • First many higher education                                             inst...
   Presentation in a glance     We will be talking about using AI to analyze      syllabus quality as a driving force to...
   Syllabus and its characteristics   Definition of the “good” syllabus   Syllabus quality management   Syllabus metri...
   A syllabus that follows acknowledged patterns (Davis,    1993; McKenney, 2001):   A syllabus that promotes good outco...
Syllabus Quality Depends onQuality of theincluded materials       Instructor                                              ...
If a student has difficulties withmultiple choice tests in Programming I course                                           ...
If majority of students complain about the textbook                                            Correct the    Component   ...
Components         List of                                                             Other TraditionalSchedule         B...
Overall Syllabus Quality ?                                                  or  The quality of its Components ?           ...
Traditional Approach:   Student Surveys / Evaluations   Student Outcomes (e.g. current and final grades)   Evaluations ...
How many errors are in the                                              structure of the syllabus   Calculate Syllabus co...
      COURSE SYLLABUS       FALL, 2011       INSTRUCTOR: JOHN SMITH   A. DESCRIPTIONThis course involves a careful exami...
It takes a Syllabus (created with the special template), represents it     as a Graph, Detects Consistency Errors and calc...
   Consistency report items:       Check existence of required elements       Check links between elements         (fo...
Relation type Typical Problems                              Possible Examples              (Rule Left Side)               ...
Good ?   Enough?                                               If it is Good it should                                    ...
N of   results                      F D        C         B          A      student gradeGrades for the very difficult (non...
This is the Expert                                                           System, described in this                    ...
1.   Syllabus quality can be objectively measured     if we look at it is at multi-dimensional     function based on the s...
3. Accepting the definition a good syllabus as a    mathematical graph with individual    pathways, we state that the qual...
7. The sound theoretical framework, as well as the   promising results of its implementation in an   intelligent system, o...
2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
Upcoming SlideShare
Loading in...5
×

APPLYING ARTIFICIAL INTELLIGENCE TO THE EDUCATIONAL DATA

431

Published on

Presentation @ LAK 2012

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
431
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

APPLYING ARTIFICIAL INTELLIGENCE TO THE EDUCATIONAL DATA

  1. 1. Denis Smolin, American University in Bosnia and Herzegovina Sergey Butakov,SolBridge International School of Business, Daejeon, South Korea2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  2. 2. … • First many higher education institutions are striving to be strategic rather than merely reactive • … • Forth, universities must operate more efficiently overall. • …2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  3. 3.  Presentation in a glance  We will be talking about using AI to analyze syllabus quality as a driving force to improve quality of a course and finally quality of a program  We will be talking about a priori and a posteriori evaluations  We will discuss system prototype that allows us to automate the evaluation process2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  4. 4.  Syllabus and its characteristics Definition of the “good” syllabus Syllabus quality management Syllabus metrics Syllabus consistency Student Outcomes as the Syllabus Quality metrics2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  5. 5.  A syllabus that follows acknowledged patterns (Davis, 1993; McKenney, 2001): A syllabus that promotes good outcomes (Zhao, 2007) Tailored or “negotiated syllabus” that fits course requirements and student learning style (Clarke, 1991)2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  6. 6. Syllabus Quality Depends onQuality of theincluded materials Instructor Other, often Motivation & Abilities Student Unknown & Motivation & Abilities Uncertain Scheduling factors School facilities 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  7. 7. If a student has difficulties withmultiple choice tests in Programming I course Correct the Component Evaluate Component Quality Changes Structure Pros: Could improve the Try to replace some tests with practical understanding coding assignments Cons: Could be more expensive 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  8. 8. If majority of students complain about the textbook Correct the Component Evaluate Component Quality Changes Structure Use another textbook Pros: Increased understanding Cons: Revision is required for the practical component of the course 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  9. 9. Components List of Other TraditionalSchedule Books Tests topics & New Elements: subject specific elements, Web 2.0 apps, etc. 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  10. 10. Overall Syllabus Quality ? or The quality of its Components ? Both!2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  11. 11. Traditional Approach: Student Surveys / Evaluations Student Outcomes (e.g. current and final grades) Evaluations from Accrediting Organizations Some others, mainly based on Expert Opinions 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  12. 12. How many errors are in the structure of the syllabus Calculate Syllabus consistency in the number of It is valid Real results are statistically implementations close to the expected ones Calculate Syllabus validity and reliability lab facilities, supplimentary materails, etc. We called the system Calculate Syllabus Costs “Chopin”The task is Complex– it requires an Intelligent tool 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  13. 13.  COURSE SYLLABUS FALL, 2011 INSTRUCTOR: JOHN SMITH A. DESCRIPTIONThis course involves a careful examination of … B. ORGANIZATION publisher says that this TheThis is a lecture-lab course in which … book is not available this C. COURSE OBJECTIVESTo introduce students to the … semester E. TEXT AND REQUIRED SUPPLIES Required text: Basic Technical Drawing, by Spencer & Dygdon 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  14. 14. It takes a Syllabus (created with the special template), represents it as a Graph, Detects Consistency Errors and calculates Costs 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  15. 15.  Consistency report items:  Check existence of required elements  Check links between elements  (for example, each test should have associated readings)  Required timings and verifies time totals  Distribution of course materials  etc.2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  16. 16. Relation type Typical Problems Possible Examples (Rule Left Side) (Rule Right Side)Refers to dead link if it doesn’t exist in the appropriate section of the syllabus. redundant link if it is never used in the syllabus sections maldistribution of links if One topic of the syllabus is supported with much more (ratio is more than threshold) links than others.Mapping Polysemy if The test for the topic covers more than this topic or allows different sequences of tests non-optimal mapping if Bad correlation between the names of the learning goals and the names of topics. More rules can be added by system user 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  17. 17. Good ? Enough? If it is Good it should give the Required results2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  18. 18. N of results F D C B A student gradeGrades for the very difficult (non-valid), very simple courses (non-valid) and a course of required (valid) complexity 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  19. 19. This is the Expert System, described in this presentation “to-do” list: Expert Database of EDL editor: goals, links System. Syllabi (EDL calculates and tests scripts), Tests Consistency, V Takes EDL script as the (TDL scripts) alidity and Professors, ass input , results, logs, Reliability istants, and etc. administrators Testing A set of data Expert analysis System. programs Takes TDL as its input TDL Test editorStudent 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  20. 20. 1. Syllabus quality can be objectively measured if we look at it is at multi-dimensional function based on the statistical data, which takes into account the quality of syllabus elements and the quality of its structure.2. Defining a good syllabus as a “syllabus that promotes good outcomes” we expand this definition with “always”, i.e. we define the quality as validity and reliability. 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  21. 21. 3. Accepting the definition a good syllabus as a mathematical graph with individual pathways, we state that the quality has to be calculated for each branch of the graph.4. Evaluation of the syllabus quality against validity and reliability shall be done in two stages: pre- processing to exclude uncertainty from the syllabus and post-processing aimed at the analysis of efficiency.5. Practical implementation demonstrates efficiency of the proposed approach and its usability. 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  22. 22. 7. The sound theoretical framework, as well as the promising results of its implementation in an intelligent system, opens new opportunities in the syllabus/course quality management: Creation of Smart Syllabus Bases; Creation of efficient teaching materials for target groups;  Solution is based on data warehouse that takes from LMS and Smart Syllabi DB 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.
  23. 23. 2nd Intl Conference on Learning Analytics and Knowledge (LAK12): Vancouver, April 29 - May 2, 2012.

×