Need for an e taxonomy

The Need for anE-TaxonomyCAA 201211 July, 2012Bobby ElliottScottish Qualifications Authoritybobby.elliott@sqa.org.uk

“Need for an e-taxonomy to facilitate e-assessment”http://www.scribd.com/doc/99329621/The-Need-for-a-Taxonomy-to-Facilitate-E-Assessment

Outline of presentation Research questions Research design Some background Findings Recommendations

Research questions1. Can a traditional taxonomy be used to analyse online writing?2. Can a taxonomic approach be used for assessment?

Research design M.Sc. In E-Learning at University of Edinburgh Two courses  Introduction to Digital Environments for Learning  Understanding Learning in an Online Environment Two forums 33 post-graduate students 1400 messages September-December 2009

Some background What is online writing? Problems with current assessment practice Anderson & Krathwohl’s Taxonomy Practical Enquiry Model of critical analysis

Problems with assessment Traditional assessment  Low validity  Irrelevant  Old fashioned  Contrived Online assessment  Transfers current practice to online domain  Ignores new affordances  Badly marked

New affordances Large-scale projects Developed co-operatively and collaboratively Improved authenticity Real life value Large audience Richer content More reflective More transparent Peer reviewed/assessed

Problems with marking1. Expression varied considerably from rubric to rubric.  Some were expressed as criteria, some were written as statements, and some were simply examples of contribution at a particular performance point.2. Little consistency in terminology  For example, the quality of academic discourse was expressed in terms of “depth of insight”, “intellectual challenge”, “quality of knowledge”, and “much thought”.3. Subjective language was common  Words such as “good”, “substantive” and “regular” (without further definition) were often used.4. Language used varied in formality  While the majority of rubrics used formal English, some used a conversational style of language including such phrases as “Your messages didn’t leave me jazzed”.

Bloom’s Taxonomy Evaluation Synthesis Analysis Application Comprehension Knowledge

Anderson & Krathwohl’s Taxonomy Cognitive dimension Knowledge dimension 1.Remember 2.Understand 3. Apply 4. Analyse 5. Evaluate 6. CreateA. Factual knowledgeB. ConceptualknowledgeC. ProceduralknowledgeD. Meta knowledge

Practical Enquiry Model Issue or problem is identified; this may be a learning TriggeringPhase 1 challenge initiated by the teacher or an observation event posted by a learner. The social exploration of ideas. In this phase, learners seek to comprehend the problem or issue, movingPhase 2 Exploration between discourse with other learners and critical reflection. Meaning is constructed from the ideas and comments produced in the previous phase. In this phase, ideasPhase 3 Integration are assessed and connections are made between ideas. In this phase, conclusions are reached and somePhase 4 Resolution consensus is created between learners.

Modifications to Taxonomy Knowledge 1. Remember 2. Understand State relevant facts. Explain facts. Provide references/hyperlinks. Check facts. Ask relevant questions. Clarify questions. Describe relevant personal Correct data/factual information.A. Factual experiences. Classify facts. Factual replies to messages. Summarise facts/data. Suggest relevant resources. Exemplify facts. Provide appropriate links. Suggest related resources.

Modifications to Model RESOLVE TRIGGER Summarise discussion. Describe personal experience. Synthesise key points. Summarise a reading/position. Build consensus. Comment on reading. Create a solution. Ask a question. Defend a solution. Identify issue. Implement idea. Suggest reading. Use resource. Point out resource. Add to knowledge. Propose a learning challenge. Test ideas. Recognise a problem. Make conclusions. Take discussion in a new direction. INTEGRATE EXPLORE Identify relevancy of issue/resource. Reply to a question Propose how idea links to other ideas. Develop an existing issue. Suggest use of resource. Clarify nature of issue. Suggest how to apply idea. Exchange information. Suggest how idea/resource "fits in". Suggest related resource/reading. Agree/disagree with substantiation. Identify what is relevant. Suggest criteria for testing. Provide an unsupported opinion. Link to previous learning. Agree/disagree without substantiation. Assess idea. Brain storm. Relate issue/position to learning.

Findings Factual Conceptual Procedural Meta NoneIDEL 21% 22% 23% 2% 32%ULOE 11% 59% 18% 5% 7%

70%60%50%40% ULOE30% IDEL20%10%0% Factual Conceptual Procedural Meta

Findings: by model IDEL ULOETrigger 17% 24%Explore 27% 27%Integrate 20% 23%Resolve 4% 9%

30%25%20%15% IDEL ULOE10%5%0% T E I R

Conclusions It works It produced worthwhile results It was easy to use It standardised language/descriptions We don’t have to invent new words or new systems It can be used for assessment It needs to be refined We need to professionalise (e-)assessment

What’s new? New taxonomy Anderson & Krathwohl’s Taxonomy Bloom’s Taxonomy

Advantages It’s evolutionary – not revolutionary It’s familiar to teachers It’s based on something teachers will trust It wasn’t invented by us

Summary Technology is not changing assessment Practice is pretty much the same  Especially high stakes  Transfer of practice from offline to online  e.g MCQs CAA 2002 would be disappointed in CAA 2012 Why haven’t we changed?  Lack of accepted theory?  Lack of standards?

http://sqacomputing.blogspot.co.uk	402
http://sqacomputing.blogspot.com	83
http://www.sqacomputing.blogspot.co.uk	33
http://sqacomputing.blogspot.pt	4
http://sqacomputing.blogspot.in	3
http://sqacomputing.blogspot.de	2
http://sqacomputing.blogspot.com.es	2
http://sqacomputing.blogspot.dk	1
http://sqacomputing.blogspot.co.il	1
http://sqacomputing.blogspot.fr	1
http://sqacomputing.blogspot.it	1
http://sqacomputing.blogspot.ie	1
http://sqacomputing.blogspot.tw	1
http://sqacomputing.blogspot.com.au	1

Need for an e taxonomy

by bobbyelliott on Jul 12, 2012

Accessibility

Categories

Upload Details

Usage Rights

14 Embeds 536

Statistics