Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Streamlining assessment, feedback, and archival with auto-multiple-choice


Published on

Auto-multiple-choice (AMC) is an open-source optical mark recognition software package built with Perl, LaTeX, XML, and sqlite. I use it for all my in-class quizzes and exams. Unique papers are created for each student, fixed-response items are scored automatically, and free-response problems, after manual scoring, have marks recorded in the same process. In the first part of the talk I will discuss AMC’s many features and why I feel it’s ideal for a mathematics course. My contributions to the AMC workflow include some scripts designed to automate the process of returning scored papers
back to students electronically. AMC provides an email gateway, but I have written programs to return graded papers via the DAV protocol to student’s dropboxes on our (Sakai) learning management systems. I will also show how graded papers can be archived, with appropriate metadata tags, into an Evernote notebook.

Published in: Education
  • Be the first to comment

Streamlining assessment, feedback, and archival with auto-multiple-choice

  1. 1. Streamlining assessment, feedback, and archival with auto-multiple-choice WEB SIGMAA meeting January 8, 2016 Matthew Leingang Courant Institute of Mathematical Sciences New York University
  2. 2. My issues • I like weekly quizzes on paper • I hate data entry • I commute by bus •I am a digital packrat •I have bad handwriting
  3. 3. Things I have tried Home rolled scripts with PDF forms
  4. 4. Last year Papers LMS
  5. 5. My Dream Scantron ScanSnap
  6. 6. Dream Workflow #1 Papers LMS
  7. 7. Dream Workflow #2 Papers LMS
  8. 8. OMR Software • SDAPS (python) • queXF (PHP) • G’n’T Eval (Ruby) • AMC (perl)
  9. 9. Why AMC • FOSS (perl, sqlite, LaTeX) • Good LaTeX package for creating docs • features MCQ, FRQ, ID encoding, numerical answers • GUI with many features • Unix Xwindows; Macports package
  10. 10. Examples
  11. 11. Simple Example documentclass{article} usepackage[insidebox]{automultiplechoice} begin{document} begin{question}{taxicab} What is the smallest integer that can be represented as the sum of two cubes in two different ways? begin{choices} correctchoice{1729} wrongchoice{1728} wrongchoice{1001} wrongchoice{4104} wrongchoice{13832} end{choices} end{question} end{document}
  12. 12. Output
  13. 13. Supported Variants • Check all that apply (questionmult) • Horizontal vs. vertical layout (choiceshoriz) • compatible with the multicols environment • smart “None of the above” option
  14. 14. begin{questionmultx}{SEssCalcET2 13-1-012} scoring{b=2} Let $R = [0,1]times[1,3]$.par Evaluate $iint_R ( x^3 - 9x^2 y^2),dA$. AMCnumericChoices{-7.5}{ digits=3,decimals=2,sign=true, borderwidth=0pt, backgroundcol=lightgray, approx=5} end{questionmultx} Numerical Responses
  15. 15. Output
  16. 16. begin{question}{SEssCalcET2 12-3-016} Let $E$ be the solid that is bounded on the outside by the sphere $x^2 + y^2 + z^2 = 16$ and inside by the cylinder $x^2 + y^2 = 4$. Find the volume of $E$. AMCOpen{lines=3,dots=false}{% wrongchoice[0]{0}scoring{0}% wrongchoice[1]{1}scoring{1}% wrongchoice[2]{2}scoring{2}% wrongchoice[3]{3}scoring{3}% correctchoice[4]{4}scoring{4}} end{question} Free Response Qs
  17. 17. Output
  18. 18. begin{minipage}[b]{0.3textwidth} AMCcode{NNumber}{8}% end{minipage} hspace*{1em} begin{minipage}[b]{0.5textwidth}raggedright MATH-UA 123.007 Calculus III [...] Code the eight digits of your N number to the left, and write your name below. vspace{3ex} hfillnamefield{fbox{ begin{minipage}{.9linewidth} Name: vspace*{1cm} end{minipage} }}hfillvspace{0.1ex} end{minipage} end{document} ID Blocks
  19. 19. Output
  20. 20. Problem Groups • Select k from a group of n • shuffle items in a group • pipe dream: keep questions in a flat file or database
  21. 21. Workflows
  22. 22. Method #1 (quizzes) Prof AMCStudents LMS LaTeX PDF give quiz take quiz PDF Scores Scan write Roster CSV Gradebook CSV Gradebook CSV Annotated PDF Grade
  23. 23. AMC Workflow #1 Papers LMS
  24. 24. Dream Workflow #1 Papers LMS
  25. 25. Method #1 (quizzes) Prof AMCStudents LMS LaTeX PDF give quiz take quiz PDF Scores Scan write Roster CSV Gradebook CSV Gradebook CSV Annotated PDF Grade
  26. 26. Method #2 (exams) ( (exams)quizzes)Prof AMCStudents LMS take quiz PDF Scores Scan write Problem PDFs Grade on iPad Problem PDFs
  27. 27. PDFExpert Grading • PDF annotator • Syncs with cloud storage apps • $10 but totally worth it � ���� ��
  28. 28. Emoji
  29. 29. Stamps
  30. 30. More stamps
  31. 31. Image Stamps
  32. 32. AMC Annotation
  33. 33. Report
  34. 34. Annotation
  35. 35. Annotation
  36. 36. Annotation
  37. 37. Annotation
  38. 38. Annotation
  39. 39. Cool trick: returning graded “papers” • AMC annotation method marks MCQs and creates a new PDF • Email feature sends them back • Sakai “dropbox” (not Dropbox) has a DAV interface • Script to batch PUT PDFs into Sakai!
  40. 40. What is DAV? • Distributed Authoring and Versioning • Treat website “directory” like a bona fide filesystem share. • Extension of HTTP
  41. 41. • Command line, interfaces with AMC data model and app config. • Attach files (e.g., solutions, summary stats) • Encrypt so they can't alter the file • Dry run feature • Upload via curl library
  42. 42. $ curl --netrc -T file.pdf<si Money Line
  43. 43. Another cool trick: saving files to Evernote
  44. 44. What is Evernote? • Instead of folders and files, notebooks and notes (and tags) • notes can contain rich text and file attachments • a note is only in one notebook but can have arbitrarily many tags • Very searchable
  45. 45. Archiving Script • Same command line interface • Create note with PDF, tagged by course, term, student, etc. • Upload via Evernote SDK
  46. 46. Advantages • Paper when you need it, not when you don't • No more office/book bag clutter • Quick turnaround • Reduces cheating by good eyesight • Eliminates cheating by altering the graded sheet • Regrades and post-exam conferences are easy
  47. 47. Disadvantages • Command line is not for everyone • Printing individual exams can take time • Scanning multipage exams on a ScanSnap takes time • Files can be big • Students who write in the margins!
  48. 48. Future directions
  49. 49. Next steps • Scale up • Tighter integration with AMC GUI • Item analysis
  50. 50. Goodnight Seattle
  51. 51. Credits Francois Roche Mark Atwood Torben Hansen
  52. 52. Credits Paul Van der Werf Andi Narvaez unknown
  53. 53. Credits Wiki episteme Jody Cloutier Wonderlane
  54. 54. Credits Kevin Kerrick Dan McKay Adelle Charles
  55. 55. Another cool trick: saving files to Evernote Seattle Municipal Archives Credits Ethan Kan Clipart from