Auto-multiple-choice (AMC) is an open-source optical mark recognition software package built with Perl, LaTeX, XML, and sqlite. I use it for all my in-class quizzes and exams. Unique papers are created for each student, fixed-response items are scored automatically, and free-response problems, after manual scoring, have marks recorded in the same process. In the first part of the talk I will discuss AMC’s many features and why I feel it’s ideal for a mathematics course. My contributions to the AMC workflow include some scripts designed to automate the process of returning scored papers back to students electronically. AMC provides an email gateway, but I have written programs to return graded papers via the DAV protocol to student’s dropboxes on our (Sakai) learning management systems. I will also show how graded papers can be archived, with appropriate metadata tags, into an Evernote notebook.

1. Streamlining assessment,
feedback, and archival with
auto-multiple-choice
WEB SIGMAA meeting
January 8, 2016
Matthew Leingang
Courant Institute of Mathematical Sciences
New York University

2. My issues
• I like weekly quizzes on paper
• I hate data entry
• I commute by bus
•I am a digital packrat
•I have bad handwriting

3. Things I have tried
Home rolled
scripts with PDF
forms

4. Last year
Papers
LMS

5. My Dream
Scantron ScanSnap

6. Dream Workflow #1
Papers LMS

7. Dream Workflow #2
Papers LMS

8. OMR Software
• SDAPS (python)
• queXF (PHP)
• G’n’T Eval (Ruby)
• AMC (perl)

9. Why AMC
• FOSS (perl, sqlite, LaTeX)
• Good LaTeX package for creating docs
• features MCQ, FRQ, ID encoding,
numerical answers
• GUI with many features
• Unix Xwindows; Macports package

10. Examples

11. Simple Example
documentclass{article}
usepackage[insidebox]{automultiplechoice}
begin{document}
begin{question}{taxicab}
What is the smallest integer that can be represented as the
sum of two cubes in two different ways?
begin{choices}
correctchoice{1729}
wrongchoice{1728}
wrongchoice{1001}
wrongchoice{4104}
wrongchoice{13832}
end{choices}
end{question}
end{document}

12. Output

13. Supported Variants
• Check all that apply (questionmult)
• Horizontal vs. vertical layout
(choiceshoriz)
• compatible with the multicols
environment
• smart “None of the above” option

14. begin{questionmultx}{SEssCalcET2 13-1-012}
scoring{b=2}
Let $R = [0,1]times[1,3]$.par
Evaluate $iint_R ( x^3 - 9x^2
y^2),dA$.
AMCnumericChoices{-7.5}{
digits=3,decimals=2,sign=true,
borderwidth=0pt,
backgroundcol=lightgray,
approx=5}
end{questionmultx}
Numerical Responses

15. Output

16. begin{question}{SEssCalcET2 12-3-016}
Let $E$ be the solid that is bounded on the
outside by the sphere $x^2 + y^2 + z^2 = 16$
and inside by the cylinder $x^2 + y^2 = 4$.
Find the volume of $E$.
AMCOpen{lines=3,dots=false}{%
wrongchoice[0]{0}scoring{0}%
wrongchoice[1]{1}scoring{1}%
wrongchoice[2]{2}scoring{2}%
wrongchoice[3]{3}scoring{3}%
correctchoice[4]{4}scoring{4}}
end{question}
Free Response Qs

17. Output

18. begin{minipage}[b]{0.3textwidth}
AMCcode{NNumber}{8}%
end{minipage}
hspace*{1em}
begin{minipage}[b]{0.5textwidth}raggedright
MATH-UA 123.007 Calculus III
[...]
Code the eight digits of your N number to the left,
and write your name below.
vspace{3ex}
hfillnamefield{fbox{
begin{minipage}{.9linewidth}
Name:
vspace*{1cm}
end{minipage}
}}hfillvspace{0.1ex}
end{minipage}
end{document}
ID Blocks

19. Output

20. Problem Groups
• Select k from a group
of n
• shuffle items in a
group
• pipe dream: keep
questions in a flat file
or database

21. Workflows

22. Method #1 (quizzes)
Prof AMCStudents LMS
LaTeX
PDF
give quiz
take quiz
PDF
Scores
Scan
write
Roster CSV
Gradebook CSV Gradebook CSV
Annotated PDF
Grade

28. AMC Workflow #1
Papers LMS

29. Dream Workflow #1
Papers LMS

30. Method #1 (quizzes)
Prof AMCStudents LMS
LaTeX
PDF
give quiz
take quiz
PDF
Scores
Scan
write
Roster CSV
Gradebook CSV Gradebook CSV
Annotated PDF
Grade

31. Method #2 (exams)
( (exams)quizzes)Prof AMCStudents LMS
take quiz
PDF
Scores
Scan
write
Problem PDFs
Grade
on iPad
Problem PDFs

32. PDFExpert Grading
• PDF annotator
• Syncs with cloud
storage apps
• $10 but totally
worth it
� ���� ��

34. Emoji

35. Stamps

36. More stamps

37. Image Stamps

38. AMC Annotation

39. Report

40. Annotation

41. Annotation

42. Annotation

43. Annotation

44. Annotation

45. Cool trick: returning
graded “papers”
• AMC annotation method marks MCQs and
creates a new PDF
• Email feature sends them back
• Sakai “dropbox” (not Dropbox) has a DAV
interface
• Script to batch PUT PDFs into Sakai!

46. What is DAV?
• Distributed Authoring
and Versioning
• Treat website “directory”
like a bona fide
filesystem share.
• Extension of HTTP

47. AMC-sakai-return.pl
• Command line, interfaces with AMC data model and
app config.
• Attach files (e.g., solutions, summary stats)
• Encrypt so they can't alter the file
• Dry run feature
• Upload via curl library

48. $ curl --netrc -T file.pdf
https://newclasses.nyu.edu/dav/group-user/<si
Money Line

49. Another cool trick: saving
files to Evernote

50. What is Evernote?
• Instead of folders and files, notebooks and
notes (and tags)
• notes can contain rich text and file
attachments
• a note is only in one notebook but can have
arbitrarily many tags
• Very searchable

51. Archiving Script
• Same command line interface
• Create note with PDF, tagged by course,
term, student, etc.
• Upload via Evernote SDK

52. Advantages
• Paper when you need it, not when you don't
• No more office/book bag clutter
• Quick turnaround
• Reduces cheating by good eyesight
• Eliminates cheating by altering the graded sheet
• Regrades and post-exam conferences are easy

53. Disadvantages
• Command line is not for everyone
• Printing individual exams can take time
• Scanning multipage exams on a ScanSnap
takes time
• Files can be big
• Students who write in the margins!

55. Future directions

56. Next steps
• Scale up
• Tighter integration with AMC GUI
• Item analysis

57. Goodnight Seattle

58. Credits
Francois Roche
Mark Atwood
Torben Hansen

59. Credits
Paul Van der Werf
Andi Narvaez
simasima.info unknown

60. Credits
Wiki episteme
Jody Cloutier
Wonderlane

61. Credits
Kevin Kerrick
Dan McKay
Adelle Charles

62. Another cool trick: saving
files to Evernote
Seattle Municipal Archives
Credits
Ethan Kan
Clipart from OpenClipArt.org