Assessment outcomes from the TENCompetence project

Assessment in TENCompetence: Sharing Assessment Process Information Yongwu Miao Educational Technology Expertise Centre Open University of The Netherlands CETIS Assessment SIG meeting Glasgow, Sep. 26, 2007

Overview ,[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],TENCompetence Project

[object Object],[object Object],[object Object],[object Object],[object Object],TENCompetence Project

A Standard-based and Service-oriented Approach to Support Assessment Process ,[object Object],[object Object],[object Object],[object Object],[object Object]

learning design (or assessment design) unit of learning (or unit of assessment) role activity environment property component method condition expression action play act role-part learning support service learning object staff learner internal change-property notification show/hide assessee assessor assessment- specific service item activity-description resource QTI_type external assessment An extended IMS LD model assessment_item outcome item_body assessment_test interaction response response_processing A simplified IMS QTI model associate choice . . . . . .

[object Object],A Standard-based and Service-oriented Approach to Support Assessment Process Player Player Player internet Dispatcher/Service broker IMS LD engine IMS QTI engine ePortfolio editor Concept mapping Latent Semantic Analysis (LSA) . . . . A A A A A generic assessment service special assessment services UoL management and enactment service client client

[object Object],A Standard-based and Service-oriented Approach to Support Assessment Process SLeD SLeD SLeD internet CopperCore Service Integration (CCSI) CopperCore APIS A A generic assessment service UoL management and enactment service client client

A peer assessment example A Standard-based and Service-oriented Approach to Support Assessment Process design assignment writing article1 reviewing article2 improving article1 writing article2 reviewing article1 improving article2 final assessment2 final assessment1 learner1 learner2 tutor article2 feedback1 improved a2 paper1 article1 feedback2 improved a1 paper2 tutor’s feedback1 tutor’s feedback2 assessment form1 assessment form2

[object Object],A Standard-based and Service-oriented Approach to Support Assessment Process

[object Object],[object Object],[object Object],[object Object],Assessment Process Specification

Conceptual Model role activity assessment design artefact rule certifier designer assessee assessor stage rating responding instructing editing portfolio constructing item performing input tool output assignment portfolio assessment test assessment item evidence assessment result expression action portfolio editor concept mapping QTI editor feedback simulator LSA QTI player giving feedback react on feedback preparation evaluation evi. collection processing teaching-learning add/remove role member trigger activity alloca. activity to role send email change property …… …… evi. provider consultant decision maker …… …… distribute artifacts present information ……

Assessment process model Programmed instruction model Peer assessment model <…> 360 degree feedback model Domain-specific models Assessment process specification Unit of assessment mapping Assessment Process Specification IMS LD + IMS QTI + Assessment-specific services

Assessment Process Modeling Tool ,[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],Assessment Process Modeling Tool Design assessment Give feedback Do assignment React to feedback constructing assi. /designing instruction /designing criteria /setting time viewing /rating feedback /requiring more /improving responding /performing 0,1 1 1 1 0,1 0,1 1 commenting /rating /grading /elaborating 1 preceding preceding preceding 1 1, * 1 1 1, * 1 1, * 1, * designer candidate reviewer candidate assignment 1, * 1, * 1, * 1, * 1, * 1, * 1, * 1, * 0, * 1, * 1, * 1, * assignment outcome 1, * 1, * 1, * 1, * feedback 1, * 1, * 1, * 1, * improved assignment outcome 1, * 0, * 1, * 1, * assessment form 1, * 0, * 0, * 1, * performing performing performing performing producing used_in

[object Object],[object Object],[object Object],Assessment Process Modeling Tool define stages? define activities? define roles? include design stage? default default additional feedback cycles? how many? for each stage, define sub-roles? how many instances for each roles? what are relations between candidates and reviewers? will tutor be involved? for each stage, define activities? completion? assign roles to activities? define artifact-flow? default choose distribution patterns (1-1, 1-m, m-1, m-m, m-n, rotate, mutual , reciprocal , random, ...) define assignment, assessment form, calculation rules Anonymous/ confidential/ public customize default completion? synchronize activities? define services/tools?

File Edit View Tool Template Help Peer assessment Here shows the guidance to make design decisions State bar user’s decisions Specify role structure, artifacts, and services Show and edit peer assessment model artifact role service lane stage

File Edit View Tool Template Help Peer assessment How many stages in the peer assessment process? Please make a decision user’s decisions artifact role service lane stage Default (three stages) With design assessment stage With additional feedback stage More complicated setting

File Edit View Tool Template Help Peer assessment How many roles will be involved? Five stages have been defined artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 Default (two persons) With the involvement of tutor More complicated setting

File Edit View Tool Template Help Peer assessment Would you like to define role structure? Default role setting and with the involvement of a tutor artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 reviewer R designer R candidate R learner1 R tutor R learner2 R learner1 R learner2 R tutor R

File Edit View Tool Template Help Peer assessment Would you like to define a typical activity for each stage? Please make a decision artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 reviewer R designer R candidate R learner1 R tutor R learner2 R learner1 R learner2 R tutor R Default setting More complicated setting

File Edit View Tool Template Help Peer assessment Would you like to define a typical activity for each stage? Nine activities are defined and roles are assigned artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 reviewer R designer R candidate R learner1 R tutor R learner2 R learner1 R learner2 R tutor R learner1 learner1 learner1 learner 2 learner 2 learner 2 learner1 learner2 tutor

File Edit View Tool Template Help Peer assessment Which stages will the tutor be involved? Please make a decision artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 reviewer R designer R candidate R learner1 R tutor R learner2 R learner1 R learner2 R tutor R learner1 learner1 learner1 learner 2 learner 2 learner 2 learner1 learner2 Give feedback 1 Give feedback 2 More complicated setting tutor

File Edit View Tool Template Help Peer assessment Which stages will the tutor be involved? The tutor is assigned to work in the stage of “give feedback 2” artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 reviewer R designer R candidate R learner1 R tutor R learner2 R learner1 R learner2 R tutor R learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor

File Edit View Tool Template Help Peer assessment Would you like to create artifacts ? Please make a decision artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 reviewer R designer R candidate R learner1 R tutor R learner2 R learner1 R learner2 R tutor R Default setting More complicated setting learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor

File Edit View Tool Template Help Peer assessment Would you like to create typical artifacts ? artifacts are created artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1.1 A feedback1.2 A outcome1 A outcome2 A assignment1 A assignment2 A improved outcome1 A improved outcome2 A feedback2.1 A feedback2.2 A

File Edit View Tool Template Help Peer assessment Would you like to change the titles of artifacts? artifacts are created finished artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1.1 A feedback1.2 A outcome1 A outcome2 A assignment1 A assignment2 A improved outcome1 A improved outcome2 A feedback2.1 A feedback2.2 A

File Edit View Tool Template Help Peer assessment Would you like to change the titles of artifacts? The title of an artifact has been changed finished artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1 A feedback2 A article1 A article2 A paper1 A paper2 A improved article1 A improved article2 A tutor’s feedback1 A tutor’s feedback2 A

File Edit View Tool Template Help Peer assessment Would you like to define artifacts? An artifact has been created finished artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1 A feedback2 A article1 A article2 A paper1 A paper2 A improved article1 A improved article2 A tutor’s feedback1 A tutor’s feedback2 A learner 2 learner 2 Review Form How do you think about this article? accept without modification accept with minor/major modification reject Comments: assessment form QTI assessment editor assessment form (QTI item) A

File Edit View Tool Template Help Peer assessment Would you like to define artifacts? An artifact has been defined finished artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1 A feedback2 A article1 A article2 A paper1 A paper2 A improved article1 A improved article2 A tutor’s feedback1 A tutor’s feedback2 A assessment form A

File Edit View Tool Template Help Peer assessment How will the artifacts be distributed? Please make a decision artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1 A feedback2 A article1 A article2 A paper1 A paper2 A improved article1 A improved article2 A tutor’s feedback1 A tutor’s feedback2 A assessment form A Reciprocally Mutually Randomly Default setting

File Edit View Tool Template Help Peer assessment How will the artifacts be distributed? An artifacts flow has been defined artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1 A feedback2 A article1 A article2 A paper1 A paper2 A improved article1 A improved article2 A tutor’s feedback1 A tutor’s feedback2 A assessment form A

File Edit View Tool Template Help Peer assessment How will the artifacts be distributed? An artifact flow has been defined artifact role service lane stage design assessment do assignment give feedback react to feedback give feedback 2 learner1 learner1 learner1 learner 2 learner 2 learner 2 tutor tutor tutor feedback1 A feedback2 A article1 A article2 A paper1 A paper2 A improved article1 A improved article2 A tutor’s feedback1 A tutor’s feedback2 A assessment form A

[object Object],[object Object],[object Object],Conclusions and Future Work ,[object Object],[object Object],[object Object],[object Object]

Thanks for your attention! Questions? ,[object Object],[object Object],[object Object],[object Object]

Assessment outcomes from the TENCompetence project

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (9)

Similar to Assessment outcomes from the TENCompetence project

Similar to Assessment outcomes from the TENCompetence project (20)

More from University of Strathclyde

More from University of Strathclyde (20)

Recently uploaded

Recently uploaded (20)

Assessment outcomes from the TENCompetence project