On Screen Marking
Get it Right, by Design…and Not by Chance
NAGENDRAN SUNDARARAJAN
EXECUTIVE VICE PRESIDENT, MeritTrac Ser...
Evaluations are High Stakes
Errors Lead to Huge Reputational Risk
There is a Need for Transformation
Scanned paper
script images
stored in a server
Electronic copies
distributed and
marked...
Key Contributors to Enhance Adoption
Strong
Management
Commitment
Evaluating
best-fit solution
Progressive
Transformation
...
“
”
What Determines Quality of
On Screen Marking
Quality of Marking
 Quality of marking
 Well defined mark scheme
 Qualified and trained markers
 Quality assurance pro...
Quality of Delivery
 Quality of Delivery
 Error Free Scanning
 Indexing
 Barcoding
 Exception management
 Security o...
Essential Features of On Screen Marking
Practice, Standardization and
Seeding
Marking Tools Marking Flexibility
Automated ...
Flexibility: Key to Faster Adoption
• Marking flexibility
• Mark from home
• Most nations that have adopted On Screen
• Ma...
Response to RTI Queries to be Made Convenient
• Storage
• Maintain pdf copies of evaluated answer scripts
• Annotations
• ...
India’s Experience of
On Screen Marking
This implementation is an
example of leveraging best
practices:
• Multiple vendors during
pilot for evaluation,
• Progress...
Evaluators defy
constraints of
space & time Volume
• >30 lakh answer scripts in one year
• Marking infrastructure set up a...
India Evaluates
international
students Volume
• 2 lakh answer scripts in the first year
• Marking infrastructure set up at...
Thank You
NAGENDRAN SUNDARARAJAN
nagendran_s@merittrac.com
Upcoming SlideShare
Loading in …5
×

WES 2014 - Role of Assessment, Accreditation, Evaluation & Quality Standard in Higher Education - Nagendran Sundarajan, Executive Vice President, MeriTrac Services

266
-1

Published on

WES 2014 - Role of Assessment, Accreditation, Evaluation & Quality Standard in Higher Education - Nagendran Sundarajan, Executive Vice President, MeriTrac Services

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
266
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

WES 2014 - Role of Assessment, Accreditation, Evaluation & Quality Standard in Higher Education - Nagendran Sundarajan, Executive Vice President, MeriTrac Services

  1. 1. On Screen Marking Get it Right, by Design…and Not by Chance NAGENDRAN SUNDARARAJAN EXECUTIVE VICE PRESIDENT, MeritTrac Services
  2. 2. Evaluations are High Stakes Errors Lead to Huge Reputational Risk
  3. 3. There is a Need for Transformation Scanned paper script images stored in a server Electronic copies distributed and marked on Screen by examiners Question-level marks and examiners’ annotations captured Scores tabulated automatically. Digital output provided to the exam body Challenges in the Conventional Evaluation of Descriptive Answer Scripts • Time of evaluators spent on administration/operations (posting in cover page) • Errors in evaluation due to wrong totaling and tabulation of scores • Responding to RTI queries in time • Challenges in logistics and paper movement (Security, cost and effort) • Lack of availability of evaluators to meet large volume of scripts • Multiple evaluations of answer scripts, the need to share evaluated scripts to students Solution: – Onscreen Marking OSM is NOT New to India The largest central Board in the country adopts OSM for 10th and 12th Standards A Large Technical University leverage OSM to streamline evaluation process Doctors use OSM to evaluate answer scripts for a Medical University from campuses across various countries One of the Largest Distance Education University uses On Screen Marking to evaluate Internal assessments and term-end exams AND A FEW MORE…
  4. 4. Key Contributors to Enhance Adoption Strong Management Commitment Evaluating best-fit solution Progressive Transformation Experienced Service Provider Knowledge of examination Secure Processes Ability to scale Proven solution Tried and tested Intuitive Scalable
  5. 5. “ ” What Determines Quality of On Screen Marking
  6. 6. Quality of Marking  Quality of marking  Well defined mark scheme  Qualified and trained markers  Quality assurance process  Marking tools  Quality Assurance Process  Continuous monitoring: Dashboard, reporting and analysis
  7. 7. Quality of Delivery  Quality of Delivery  Error Free Scanning  Indexing  Barcoding  Exception management  Security of script  Storage  Transmission  Management
  8. 8. Essential Features of On Screen Marking Practice, Standardization and Seeding Marking Tools Marking Flexibility Automated Script allocation and de-allocation Forced annotation
  9. 9. Flexibility: Key to Faster Adoption • Marking flexibility • Mark from home • Most nations that have adopted On Screen • Marking has enabled markers mark scripts from home • Multiple Marking • Concurrent multiple marking • No previous marker bias • Give flexibility to allocate questions to markers and not the full script • Ability to allocate specific questions to be marked only by experts
  10. 10. Response to RTI Queries to be Made Convenient • Storage • Maintain pdf copies of evaluated answer scripts • Annotations • Marked answer scripts to be stored with annotations and comments • Retrieval • Retrieval of answer scripts a matter of seconds • Self-service • Can also offer self-service to students where they can make necessary payments and download scripts online from the university / board portal
  11. 11. India’s Experience of On Screen Marking
  12. 12. This implementation is an example of leveraging best practices: • Multiple vendors during pilot for evaluation, • Progressively expanding scope • Made scripts ready for access by students Adoption of technology for evaluations Volume • ~1 million scripts as marked within 45 days • 25 Cities, 90 centers • Over 6000 markers; 2000 concurrent Fast tracking adoption • 1 hour training provided for teachers • Training on use of computers and marking solution Acceptance • Ease of use and avoidance of clerical tasks led to markers being able to mark up to 50 scripts a day • Board was able to generate end of the reports with ease from across the country • Recommended as the way to go for descriptive evaluations • One of the largest school board in the country completed a successful implementation of On Screen Marking; the largest OSM project in the country • Scripts of 10th and 12th grade students marked from across the country
  13. 13. Evaluators defy constraints of space & time Volume • >30 lakh answer scripts in one year • Marking infrastructure set up at markers home / preferred location Controlling marking quality • Centralized control of marking quality and speed using marking processes like standardization and seeding • Process for training and qualification of markers • Centralized helpdesk to manage queries Acceptance • Constant increase in pool of evaluators • University decides to get their Internal Assessments also marked On Screen through the pool of expert evaluators • Students allowed to upload internal assessments that are written in MS Word • One of the Largest Distance Education University leveraged expert markers from across the country for marking scripts in 2800 subjects, across 800 different programs • 800 markers are enabled marking environment at their home / preferred location The University offered convenience to markers by allowing them to mark from home, yet maintaining quality of marking and speed of marking
  14. 14. India Evaluates international students Volume • 2 lakh answer scripts in the first year • Marking infrastructure set up at marking hubs or their home / preferred location • 3 functional skills evaluated from India – Mathematics, ICT, English Controlling marking quality • Answer scripts hosted in UK, marking done in India • High degree of quality assurance through standardization and seeding Acceptance • High quality evaluations prompting the board to increase marking volume out of India • Evaluator team strength to increase from ~80 to ~150 • Biggest name in Vocational Education leverage knowledge base in India to evaluate answer scripts of British students • Native English speakers are evaluated on English • Wide coverage and debate in UK and India over evaluation The awarding body overcame traditional constraints of script movement and shortage of good quality evaluators by reaching out to large talent pool
  15. 15. Thank You NAGENDRAN SUNDARARAJAN nagendran_s@merittrac.com

×