• Save
Computer assisted assessment patterns for different target users: the case of Multimedia Questionnaires
Upcoming SlideShare
Loading in...5
×
 

Computer assisted assessment patterns for different target users: the case of Multimedia Questionnaires

on

  • 3,335 views

In eLearning a large diversity of patterns are proposed for different types of target users (e.g. teachers, developers, etc.), although in some cases these potential users are not clearly defined. Our ...

In eLearning a large diversity of patterns are proposed for different types of target users (e.g. teachers, developers, etc.), although in some cases these potential users are not clearly defined. Our context of study is focused on Computer Assisted Assessment (CAA) processes. Within the large number of methods to assess learning, we study the use of eQuestionnaires that make use of technical assessment specifications such as IMS Question and Test Interoperability (QTI). We analyse how the design, creation and use of an eQuestionnaire in a CAA process is structured in stages where different users are involved. These users have to solve a problem in order to enable the next stage. According with a user and his/her problem, a specific good technique can be applied. This information can be organized in a pattern. We propose a collection of patterns for different target users who work in the integration of new opportunities on CAA using multimedia Web 2.0 services in an eQuestionnaire. This collection is composed by: one pattern for teachers which introduces the benefits of using eQuestionnaires with Web2.0 interactions; a second pattern for CAA designers which presents how to create a model of new types of interaction without extending the assessment specification; and finally a third pattern for developers which solves the problem of how to implement a tool with Multimedia Web 2.0 Interactions using an assessment specification to design eQuestionnaires. Each pattern belongs to different pattern languages, however they are complementary and dependent among them. We have done a survey study to identify if the use of patterns can improve the tasks and relations among users in a CAA process. The evaluation results indicate that users recognize the potential use of patterns useful for facilitating the sharing of good practices.

Statistics

Views

Total Views
3,335
Views on SlideShare
3,329
Embed Views
6

Actions

Likes
2
Downloads
0
Comments
0

1 Embed 6

http://www.slideshare.net 6

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Computer assisted assessment patterns for different target users: the case of Multimedia Questionnaires Computer assisted assessment patterns for different target users: the case of Multimedia Questionnaires Presentation Transcript

  • Computer Assisted Assessment Patterns for Different Target Users: the Case of Multimedia Questionnaires Patricia Santos Rodríguez, Davinia Hernández-Leo, Toni Navarrete and Josep Blat Pompeu Fabra University, Department of Information and Communication Technologies 5 th March,2009
  • Content
    • Motivation
    • Objectives
    • State of the Art
    • Patterns proposal
    • Survey study
    • Conclusions & Future work
    Patricia Santos – eLearning Patterns workshop,
  • Motivation Patricia Santos - eLearning Patterns workshop, New questionItems  New forms of evaluate knowledge = Improvement of students’ skills. Patterns (capture best practices in a CAA context based on eQuestionnaires) Education (Teachers & students, Assessment, Competences, Methodologies, Context, Problems...) Technology (Educational Technology standards, eQuestionnaries,...) Assessment Specification (QTI) Multimedia Web 2.0 services + +
  • Content
    • Motivation
    • Objectives
    • State of the Art
    • Patterns proposal
    • Survey study
    • Conclusions & Future work
    Patricia Santos - eLearning Patterns workshop,
  • Objectives
    • Analyse the stages, context, problems and users involved in our test-based CAA case
    • Analyse if the steps followed in this case can potentially represent good practice solutions for common problems
    • Find solutions to bridge the gap between users with problems in this context
    Patricia Santos – elearning Patterns workshop,
  • Content
    • Motivation
    • Objectives
    • State of the Art
    • Patterns proposal
    • Survey study
    • Conclusions & Future work
    Patricia Santos – eLearning Patterns workshop,
  • CAA based on Multimedia eQuestionnaires
    • CAA use computers for assessing student learning
    • New types of questionItems are necessary for evaluating students’ competences
    • Assessment specifications & multimedia Web 2.0 services
    Patricia Santos – eLearning Patterns workshop,
  • IMS Question & Test Interoperability
    • IMS QTI educational technology specification for assessment
    • To represent an assessment scenario using questions (assessmentItem) and tests (assessmentTest) and their corresponding results reports
    • QTI offers different types of questions:
      • Simple Interactions
      • Text-based Interactions
      • Graphical Interactions
      • Miscellaneous Interactions
    • To interpret the specification we need an engine (e.g. APIS)
    • The interoperability of this specification enables the easy communication with other services.
    Patricia Santos – eLearning Patterns workshop,
  • New questionItems
    • Teachers need to evaluate specific student’s skills
    • If user can interact in new ways  the user could be evaluated in new ways
    • We study the approach of combining GoogleMaps using the specification IMS QTI
    • The student now is directly related to a geographic map, and not just with a verbatim question, the teacher can increase their evaluation possibilities because now he can get results from the interaction made over the map
    Patricia Santos – eLearning Patterns workshop,
  • Multimedia Web 2.0 questionItems
    • A flexible and powerful model for including a wide variety of user interactions of geographic type with maps to perform assessment making use of GoogleMaps
    • Technical solution: Middleware + web-based editor
    Patricia Santos – eLearning Patterns workshop,
  • Modelling eAssessment Process
    • Model the CAA context:
        • the users
        • their roles
        • their tasks
        • different stages of the process
    Patricia Santos – eLearning Patterns workshop,
  • Patricia Santos – eLearning Patterns workshop,
  • Content
    • Motivation
    • Objectives
    • State of the Art
    • Patterns proposal
    • Survey study
    • Conclusions & Future work
    Patricia Santos – eLearning Patterns workshop,
  • Patterns
    • A pattern is designed to capture best practice in a specific domain
    • Provides: organized information that explains a reusable sound solution to a commonly occurring contextualized problem
    • Pattern main elements:
        • Context
        • Problem
        • Solution
    Patricia Santos – eLearning Patterns workshop,
  • Patterns Learning Patterns for educationalists Software Patterns for computer scientists Patricia Santos – eLearning Patterns workshop,
  • Network of connections among patterns from different target users Patricia Santos – eLearning Patterns workshop,
  • Pattern Language: adding new opportunities in CAA based on questionnaires
    • Teachers’ Pattern “ Evaluation of Exploration Skills using eQuestionnaires ”: This pattern explains how to create new questionItems to evaluate the exploration skills of the students automatically.
    • Specification Expert Designers’ Pattern “Modelling new questionItems using a Multimedia Web 2.0 service”: This pattern can be used as a bridge of communication between teachers and developers.
    • Developers’ Pattern “ Implementation of Multimedia Web 2.0 questionItems using an Assessment Specification to design eQuestionnaires ”: It is created to provide a solution for developers who want to communicate an assessment engine with a multimedia Web 2.0 service. This communication will enable the integration of new questionItems in an eQuestionnaire.
    Patricia Santos – eLearning Patterns workshop,
  • Content
    • Motivation
    • Objectives
    • State of the Art
    • Patterns proposal
    • Survey study
    • Conclusions & Future work
    Patricia Santos – eLearning Patterns workshop,
  • Survey Study
    • Survey study: “ Providing a solution to bridge the gap between developers of assessment tools, specification expert designers and teachers ”
    • Conclusions:
    • The majority of the teachers value positively the fact of integrating “help information” in software tools to facilitate the preparation of assessment materials
    • To develop an educational technology tool which uses an external Web 2.0 service, a developer must have knowledge about: eLearning specifications, eLearning tools and Web 2.0 services
    • It is necessary to create well-structured information about best practices in developing, designing and using assessment tools
    • Technicians believe that it is very useful to know teachers good practices
    • Example: A comment of a developer was : “Knowing those best practices would greatly help me to guess how to apply a solution to my own context. Furthermore, I could even discover new ideas on eLearning/eAssessment that I didn’t think of” [Technicians’ Questionnaire].
    Patricia Santos – eLearning Patterns workshop,
  • Content
    • Motivation
    • Objectives
    • State of the Art
    • Patterns proposal
    • Survey study
    • Conclusions & Future work
    Patricia Santos – eLearning Patterns workshop,
  • Conclusions
    • The target users of a PL can be different types of users from different types of communities.
    • We propose a network of connections among patterns from different PLs.
    • The network of connections creates a new PL, which solves the complex problem in the CAA case we are studying.
    • Patterns can be a good solution to capture good practices on CAA and bridge the communication between users of different communities.
    • Our collection of patterns enables the addition of new opportunities on CAA using eQuestionnaires.
    Patricia Santos – eLearning Patterns workshop,
  • Future Work
    • Include patterns in an eAssessment authoring tool
    • Increase the PL “patterns for adding new opportunities on CAA using eQuestionnaires”
    • This is an initial work, then it is necessary to do further evaluations, an evaluation of the proposed PL is planned for demonstrating that the solutions proposed in the set of patterns can be used in similar contexts and with similar problems
    Patricia Santos – eLearning Patterns workshop,
  • Questions? Patricia Santos – eLearning Patterns workshop,
  • Thank you! Patricia Santos – eLearning Patterns workshop,
  • More information: [email_address] http://pilot.tencompetence.org/QTI_GMaps/Demo_Exam_qti_gmaps.html Patricia Santos – eLearning Patterns workshop,