• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
 

Linda Meyer - Moderation of FSAs - best practice (1)

on

  • 516 views

The final integrated summative assessment refers to the process of making judgments about achievement. This is carried out when a learner is ready to be assessed at the end of a learnership or full ...

The final integrated summative assessment refers to the process of making judgments about achievement. This is carried out when a learner is ready to be assessed at the end of a learnership or full qualification. The FSA will be set by a registered assessor appointed by the Certification/Outsourced Partner after notification from the provider/assessor that the candidates are ready.

Statistics

Views

Total Views
516
Views on SlideShare
516
Embed Views
0

Actions

Likes
0
Downloads
3
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Linda Meyer - Moderation of  FSAs - best practice (1) Linda Meyer - Moderation of FSAs - best practice (1) Presentation Transcript

    • Moderation of the FSA – Best Practice Linda Meyer
    • INTEGRATED FSA The FSA for the qualification will focus on the extent to which a learner can demonstrate applied competence. Applied competence, in terms of the NQF is evidenced through learner’s ability to integrate concepts, ideas and actions in authentic, real-life contexts and is expressed as practical, foundational and reflexive competence
    • PURPOSE The final integrated summative assessment refers to the process of making judgments about achievement. This is carried out when a learner is ready to be assessed at the end of a learnership or full qualification. The FSA will be set by a registered assessor appointed by the Certification/Outsourced Partner after notification from the provider/assessor that the candidates are ready.
    • FSA Practical competence - demonstrated ability to perform a set of tasks and actions in authentic contexts Foundational competence - demonstrated understanding of what we are doing and why we are doing it Reflexive competence - demonstrated ability to integrate our performances with our understanding so that we are able to adapt to changed circumstances and explain the reason behind these adaptations.
    • ASSESSMENT METHODS In setting the FSA, SETQAA would like Certification/ Outsourced Partners and/or assessors to take note of the different assessment methods and encourage the use of a wide variety of methods.
    • SETTING OF FINAL INTEGRATEDSUMMATIVE ASSESSMENT (FSA) • Function and role of the Certification Partner/Outsourced Partners  The Certification/Outsourced Partner is responsible for setting of the FSA at NQF Level 4, with a memorandum of model answers and a grid showing its linkages with the assessment criteria of the qualification.
    • FSA All FSA tools and memoranda, even in draft form, are the property of the Services SETA ETQA and may not be revealed to, or discussed with any unauthorized person. The highly confidential nature of these assessment instruments must be safeguarded at all times.
    • FSA If the assessment instrument and memoranda are stored on computer, thorough security measures must be taken to prevent any illegal access to these files. The contents of the assessment instrument must reflect the relevant unit standards and qualification and must not be biased towards any particular textbook or learning material.
    • FSA A second assessment instrument should be set as standby assessment instrument. SETQAA have the right to propose the use of either assessment instruments for the FSA, if necessary.
    • FSA The Certification/Outsourced Partner must ensure that the assessment instruments are not identical. The Certification/Outsourced Partner must also ensure that neither of these assessment instruments are the same as any previous assessment instruments.
    • MODERATION CRITERIA The moderator must ensure that the assessment instrument is aligned with the purpose and rationale of the qualification and that it complies with the applicable SAQA level descriptor. Draft assessment instruments must fall within the ambit and in accordance with the guideline document.
    • Functions and role of the moderator After the Certification/Outsourced Partner has set the FSA instrument, with a memorandum of model answers, an internal moderator, appointed by the Certification/Outsourced Partner, must moderate all draft assessment instruments and memoranda for the assessment instruments under his/her jurisdiction.
    • Functions and role of the moderator During the moderation, the assessment instrument and memoranda are to be objectively assessed by the moderator with regard to the following:
    • Functions and role of the moderator cont. The moderator must verify that papers set for the FSA`s are not identical. The moderator must verify that the Unit Standards are appropriate to the registered qualification.
    • Functions and role of the moderatorThe moderator must ensure that thefinal summative integratedassessment instrument is alignedwith the purpose and rationale of thequalification and that it complies withthe applicable SAQA level descriptor.
    • Functions and role of the moderator The moderator must ensure that the final integrated assessment instruments must fall within the ambit and in accordance with this guideline document. The extent to which the prescribed Unit Standards are covered as well as the assurance that nothing is asked that is outside the scope of the Unit Standard/Qualification must be verified.
    • Functions and role of the moderatorThe spread and number of marksallocated according to the weighting and credits as per the registered learnership/qualification must be assessed.
    • Functions and role of the moderator The effectiveness of the type of assessment methods used, writtenquestions, essays, paragraphs, case studies, must be assessed.
    • Functions and role of the moderator The questions, terminology, numbering, spelling, punctuation, sentence construction, units, and symbols must be clear, unambiguous and grammatically correct so that candidates will know exactly what is expected of them.
    • Functions and role of the moderator The length of the assessment instrument must take into account the time allocated as well as reading time. The draft assessment instrument must be of such a length that a well- prepared learner will be able to answer it comfortably within the time allocated, with reasonable time remaining for revision.
    • Functions and role of the moderator It should be borne in mind that it takes considerable time to read through certain assessment instruments. The memorandum must ensure that model answers give enough detail for the assessor of the FSA to conduct an accurate and fair assessment.
    • Criteria for moderation of the FSA instrument Standard of the paper (validity, reliability and fairness) Questions of various types e.g. Multiple Choice Questions, Case studies, paragraphs, data response, essay, etc Questions from which candidates are to choose – are they of equal difficulty level Correct distribution in terms of cognitive levels (Bloom’s Taxonomy)
    • Bloom’s Taxonomy In 1956, Benjamin Bloom, a professor at the University of Chicago, shared his famous "Taxonomy of Educational Objectives". Bloom had identified six levels of cognitive complexity that have been used over the past 40 years to make sure that instruction stimulates and develops learners higher-order thinking skills. The Levels are: Knowledge : Rote memory skills (facts, terms, procedures, classification systems) Comprehension : The ability to translate, paraphrase, interpret or extrapolate material. Application : The capacity to transfer knowledge from one setting to another. Analysis : The ability to discover and differentiate the component parts of a larger whole. Synthesis : The ability to weave component parts into a coherent whole. Evaluation : The ability to judge the value or use of information using a set of standards.
    • Bloom’s Taxonomy 1. Knowledge: remembering of previously learned material; recall (facts or whole theories); bringing to mind. Terms: defines, describes, identifies, lists, matches, names. 2. Comprehension: grasping the meaning of material; interpreting (explaining or summarizing); predicting outcome and effects (estimating future trends). Terms: convert, defend, distinguish, estimate, explain, generalize, rewrite. 3. Application: ability to use learned material in a new situation; apply rules, laws, methods, theories. Terms: changes, computes, demonstrates, operates, shows, uses, solves. 4. Analysis: breaking down into parts; understanding organization, clarifying, concluding. Identify parts: See Related Order; Relationships; Clarify. Terms: distinguish, diagrams, outlines, relates, breaks down, discriminates, subdivides. 5. Synthesis: ability to put parts together to form a new whole; unique communication; set of abstract relations. Terms: combines, complies, composes, creates, designs, rearranges. 6. Evaluation: ability to judge value for purpose; base on criteria; support judgment with reason. (No guessing). Terms: appraises, criticizes, compares, supports, concludes, discriminates, contrasts, summarizes, explains.
    • Bloom’s Taxonomy .Knowledge (finding out) a. Use - records, films, videos, models, events, media, diagrams, books... b. observed behavior - ask match, discover, locate, observe, listen. 2. Comprehension (understanding) a. Use - trends, consequences, tables, cartoons.... b. observed behavior - chart, associate, contrast, interpret, compare. 3. Application (making use of the knowledge) a. use - collection, diary, photographs, sculpture, illustration. b. observed behavior - list, construct, teach, paint, manipulate, report. 4. Analysis questions (taking apart the known) a. use - graph, survey, diagram, chart, questionnaire, report.... b. observed behavior - classify, categorize, dissect, advertise, survey. 5. Synthesis (putting things together in another way) a. use - article, radio show, video, puppet show, inventions, poetry, short story... b. observed behavior - combine, invent, compose, hypothesis, create, produce, write. 6. Evaluation (judging outcomes) a. use - letters, group with discussion panel, court trial, survey, self- evaluation, value, allusions... b. observed behavior - judge, debate, evaluating, editorialize, recommend.
    • Criteria for moderation of the FSA instrument Overall: how does the standard of the assessment instrument compare in relation to other qualifications/s assessment instruments and previous assessment instruments.
    • Criteria for moderation of the FSA instrument •Intellectually challenging and allowing for creative responses from candidates •Suitability of examples and illustrations  Relationship between mark allocation, level of difficulty and time allocation
    • Criteria for moderation of the FSA instrument FSA do not count more than continuous assessment of providers – the objective for the FSA is to check the standard across providers and across qualifications - STANDARDISATION
    • Technical Criteria•Cover page with all relevant details such as time, unit standards and instructions to candidates•Clarity of instructions to candidates•Lay out: learner friendly•Correct numbering
    • Technical Criteria•Mark/weighting allocation clearly indicated•Quality of illustrations, graphs, tables etc must be print ready•Complete memorandum with model answers with mark/weight allocation and provision for alternatives
    • Registered Unit Standards and Qualifications•Relevance to registered Qualification, i.e. unit standards, specific outcomes, assessment criteria•Levels of questions•Coverage of unit standards•Weighting and spread of contents
    • History of tool Full history of FSA (when it will be conducted, who will be assessed, when and where) with all drafts, internal moderators’ comments etc. must accompany the assessment instrument each time it is submitted to the external moderator (SETQAA)
    • Cognitive skills What conceptual constructs of the unit standards does the assessment instrument deal with? e.g. – reasoning ability • ability to communicate • ability to translate from verbal to symbolic • ability to compare and contrast • ability to see causal relationship • ability to express an argument clearly
    • Cognitive skills Are these constructs representative of the best and latest developments in the training of this knowledge field? Are the questions challenging and allowing for creative responses from candidates? Suggested application of cognitive levels for an NQF Level 4 qualification is: • 10% knowledge • 20% comprehension • 40% application • 30% analysis, synthesis, evaluation
    • Language and bias• Correct terminology• Appropriate language register – for the level of the learner• Avoidance of gender, race, cultural, provincial bias• Clear and unambiguous specification of instructions within questions e.g. list, describe
    • Time• Relationship between difficulty level of questions and time• Relationship between time and mark allocation and answer requirements
    • Internal Moderation Is there evidence that the paper has been internally moderated? Quality, standard and relevance of input from internal moderator.
    • Competency of Assessor• Knowledge of assessor setting the FSA with regard to qualification.• Knowledge, experience, expertise in assessment
    • Marking Memorandum Correctness of memorandum Correspondence with questions. Alternative answers provided Does it facilitate assessment?
    • Alignment Grid Did they use the grid showing the alignment of the instrument with the purpose, rational, assessment criteria and NQF level descriptor of the qualification as issued by the SSETA used?
    • Overall impression of the paper• Fairness of the assessment instrument as whole• Will the assessment instrument as a whole assess the achievement of the purpose of the qualification and the exit level outcomes• Recommendations for improvement or maintenance of qualification/unit standards.• final acceptance/rejection of individual assessment questions and whole assessment instrument needs to be substantiated
    • REFERENCES OR RECORDS ETQA-F 005 Final Integrated Summative Assessment Instrument
    •  Only after written approval from the external moderator, SETQAA will approve and sign off the final assessment instrument and notify the Certification/Outsourced Partner to proceed with the distribution of the assessment instrument and conduct the assessment.
    • CONTINUOUS ASSESSMENT VS FINAL SUMMATIVE ASSESSMENT STAKEHOLDERS INVOLVED IN QA OF ASSESSMENT PROCESSES Provider internal assessors Provider internal moderators Certification/Outsourced Partner assessors Certification/Outsourced Partner moderators SETQAA DOC SETQAA external moderators SETQAA external verifiers PRACTICAL COMPONENT THEORETICAL COMPONENT FSA (WRITTEN COMPONENT) TRAINING PROVIDER ASSESSMENT FSA (PRACTICAL Conducted AT END ofAssessment should happen on a CONTINUOUS OBSERVATION) learnership/Qualificationbasis during learnership/training – formative aswell as summative Should be set and conducted Set and moderated by registeredEvidence gathered in PoE CONTINUOUSLY and/or AT END of assessors and moderators appointedInternally moderated and reports submitted to learnership/ qualification by CPSETQAA at regular specified intervals (See Assessed and moderated by registered Set and assessed by registeredmoderation policy)Externally moderated/verified by SETQAA assessors, moderated by registered assessors and moderators appointedPoE submitted for external moderation by CP at moderators appointed by CP by CPend of learning Externally verified by SETQAA Externally moderated/verified by SETQAA appointed moderators
    • FSA Moderation process APPLICATION FOR MODERATION OF FINAL INTEGRATED SUMMATIVE ASSESSMENT INSTRUMENT RECEIVED BY SSETQA EXTERNAL MODERATOR APPOINTED BY SSETQA NO COMMUNICATION PERMITTED BEWEEN DEVELOPER AND MODERATOR MODERATION OF FINAL INTEGRATED SUMMATIVE ASSESSMENT INSTRUMENT / RPL TOOL
    • FSA Moderation process PRELIMINARY REPORT PREPARED BY MODERATOR AFTER THE FSA/RPL TOOL HAS BEEN MODERATED - REMEDIATION IDENTIFIED TOOL SENT BACK TO DEVELOPER FOR REMEDIATION TO BE ADDRESSED THE DEVELOPER MAY REMEDIATE / REQUEST ADDITIONAL INFORMATION OR APPEAL THE MODERATION DECISION THROUGH THE SSETA APPEALS PROCEDURE
    • FSA Moderation process REMEDIATION COMPLETED SENT TO EXTERAL MODERATOR TO CONFIRM REMEDIATION & FINAL REPORT SUBMITTED ELECTRONIC VERSIONS SENT FSA/RPL TOOL REGISTERED ON DATABASE
    •  The Certification/Outsourced Partner assures the rectification is done and sends the final draft together with the Moderators report to SETQAA for external moderation.
    •  The Certification/Outsourced Partner assures the rectification is done and sends the final draft together with the Moderators report to SETQAA for external moderation.
    •  Only after written approval from the external moderator, SETQAA will approve and sign off the final assessment instrument and notify the Certification Partner to proceed with the distribution of the assessment instrument and conduct the assessment.
    • Thank you