Calculus	Courses’	
Assessment	Data
Matti	Pauna
University	of	Helsinki
Department	of	Mathematics	and	Statistics
2001:	Helsinki	Learning	System
Calculus	Courses’	Assessment	Data
• Online	courses
• Curriculum	development
• Assessment	and	data
Single	Variable Calculus 2004
Modules	
Course	
Navigator
Online	Calculus
Each	module	contains
• YouTube	videos
• Reading	materials
• Quizzes	(Automatic	
assessment)
• Workshops	(Peer	
assessment)
• Discussion	forum,	
calendar,	gradebook,	
etc.
Learning	Paths
Theory	and	worked	out	
examples	explained	in	
YouTube	videos
- Viewable	also	with	
mobile	devices
Course Design
Supports individual study paths
Online	Calculus	Courses
• Single	Variable	Calculus		2004	
• Calculus	I	&	II
• Advanced	Calculus			2013		(Real	analysis)
Completeness	property	of	real	numbers	with			ℇ - 𝛿
proof	method
• Option	to	take	UH	undergraduate	Calculus	
curriculum	completely	online	and	in	English			2014
Online
Course
Content Credits Traditional
Course
Content Credits
Calculus I Limits,	Differentiation,	
Antiderivatives,	
Riemann	Sums
8 Limits Completeness,	
Supremum,	Theory	of	
Limits
5
Calculus II Advanced	Integration,	
Improper	Integrals,	
Convergence	of	
Series,	Taylor	Series
8 Differentiation Continuity,	Bolzano’s	
Theorem,	
Differentiation
5
Advanced
Calculus
Completeness,	
Continuity,	Bolzano-
Weirstrass	Theorem
6 Integration Riemann	Sums,	
Antiderivatives,	
Improper	Integrals
5
Series Convergence	of	Series,	
Taylor	Series
5
Total ECTC
credits
22 20
Homework	and	Assessment
• Diagnostic,	Formative,	and	Summative
• Automatic	with	immediate	feedback	implemented	
with	STACK,	WEPS	also	supports	WeBWorK
• Peer	to	peer
Mathematical	Competencies
(Pointon &	Sangwin 2003,	Rämö,	Oinonen,	Wikberg 2015)
1.	Factual	recall
2.	Carry	out	a	routine	calculation	or	algorithm	
3.	Classify	some	mathematical	object
4.	Interpret	situation	or	answer
5.	Proof,	show,	justify	(general	argument)
6.	Extend	a	concept
7.	Construct	example/instance
8.	Criticize	a	fallacy
9.	Information	transfer
STACK	assists	
with	the	
correct	input	
of	formulas
Automatic	&	Immediate	Feedback
• Systems	such	as	STACK	and	WeBWorK take	
student’s	answer	as	a	mathematical	object
• Evaluate	whether	the	answer	satisfies	the	
mathematical	properties	required
• Give	feedback	also	about	common	misconceptions
• Assessment	is	objective:	The	same	criteria	are	
consistently	applied	to	all	answers
Peer	Feedback
Workshop	module	in	Moodle:
1. Students	are	given	homework	problems	asking	a	complete	and	
clear	presentation	of	a	solution.	
2. After	the	submission	deadline,	an	example	solution	is	provided
3. According	to	the	model	solution	and	assessment	instructions,	
students	grade	and	give	constructive	feedback	to	other	students	
even	to	yourself
Pedagogical	goals:
• Students	study	example	and	other	studens’	solutions
• Learning	from	other	ways	to	solve,	common	mistakes	
and	provide	corrective	feedback	
• Creates	interaction	between	students	in	an	online	
course	
• Scalable	to	MOOCs
Student	Feedback	on	Peer	Review
Assessment	Data
• Each	action	is	logged:	Clickstream	data	about	
behavior	in	an	online	course
• Attempts,	answers,	scores	in	quizzes
• Workshop	submissions,	grades	&	feedback	given	
and	received
• Questionnaires	about	learning	styles,	self-efficacy,	
motivation
Quiz	Log	Data
Achievement	and	homework
UH	Calculus	Courses	since	2012	
390	registered	students
110	took	1st exam
82	either	took	both	exams	or	passed	the	course	with	grade	1
Conclusions
• E-Assessment	is	becoming	more	popular	in	Finland
• Digitalizing	teaching	not	in	the	core	of	the	
institution	or	because	of	lack	of	resources
• Imaginative	ways	to	assess	higher	competencies	
automatically
• More	data	becoming	available	
(Spring	2016:	statistics	courses	900	cases)

Matti Pauna SOED 2016