A maxi-edition of my guide to Learning Management Environment optimisation, whereby we hack the workload model to our favour, make marking easier (and more fun) for the lecturer, and put the students second in the pursuit of "How can we use this teaching technology to deliver something superior to our own work days?". Because quite often, the better systems for improving academic workplace effectiveness never get sold on the basis of "Want to do less work with more resources for better personal outcomes?"

1. Learning Environment Online
Doing less with more
Dr Stephen Dann
Senior Lecturer, Research School of Management
Australian National University

2. Index
• About the author
• A Teaching Assessment Journey
• Reframing Marking
• Decisions to make to improve your time invested in teaching
• The Workload Time Model of Assessment
• Giving better feedback
• Turnitin: Feedback Machine
• Contact

3. About Dr Stephen Dann
• PhD from 1998
• Been in the business since 1997
• Business School Students
• Compulsory first year courses
• Compulsory second year courses
• Angry engineering students doing a “Soft Bludge Subject In Commerce”
• Really angry quantitative students doing “This Stupid Subject With Words”
• Very upset and confused students doing this really hard and scary subject that uses
words, and “how do I sentence not-equation answer?” fears
• “Small” classes of 60 to 130 students
Index

4. Background
• ANU Vice Chancellor’s Citation for Outstanding Contribution to Student Learning
• ANU Achievement award: Candidate for Citation for Outstanding Contribution to Learning
• ANU Commendation for Outstanding Contribution to Student Learning
• Pearson ANZMAC Emerging Educator of the Year
• ANU College of Business and Economics Award for Teaching Excellence
• Senior Fellow, Higher Education Academy (UK)

5. A journey of assessment,
technology, and much improved
outcomes (for me)
Because when it’s no fun, it’s no fun for anyone
Index

6. Where I came from…
• Exams were scary
• Undergrad sitting them, academic marking them
• Assessment was a dreaded chore
• Stories told slightly differently, slightly wrong,
• Grades were on the ordinary end of extra ordinary
• Marking was postponed, avoided or generally delayed
• It wasn’t a lot of fun for anyone.
• Student complaints about feedback, guidance and assessment load
• I was burning time out of my research quota to perform badly in teaching

7. If it could be done wrong…
• Spending more time on marking than allocated
• Getting frustrated by marking
• A multiple choice exam of doom
• Highest score was 40%.
• The bit where I had to scale… upwards

8. Where was it all going
wrong?
Index

9. Problems and Solutions!
• Problem 1:PEBKAC
• Solution: Hack the Workload Model
• Problem 2: Why did I set that task?
• Solution: Chekov’s Assessment Task
• Problem 3: Out of Alignment
• Solution: Curriculum Design makes the work happier
• Problem 4: Assessment was a stick
• Solution: Assessment is a better carrot
• Problem 5: Student Insultation
• Solution: Self solving student

10. Problem 1:PEBKAC
• Problem exists between keyboard and chair
• I wasn’t investing time, I was spending time
• No upfront time investment
• All of the late semester time sinks in marking
• Mechanical and slow marking process
• Teaching was chewing up more than the 40% of time, for less than 40% of
the success
• So many student queries about things that I thought I’d covered…

11. Solution: Hack the Workload Model
• All hail the workload model, giver of quantification, bringer of
approximations
• 1 hour per student for marking
• 100 students,
• 35 hour week,
• 2 weeks and 4 days of full time marking to budget
• When do I want to spend my marking?
• Oh, it’s how much, right, I’ll be needing this day and that day
• Motive to invest 4 days up front to clarify my own marking tasks

12. Problem 2: Why did I set that task?
• Here’s the scenario
• Weekly lecture (2 hours)
• Tutorial activity involved discussion questions
• Assessable presentation
• Short answer questions culminating in a “Tutorial Kit” of submitted answers
• An extended essay
• So let’s finish a semester with an exam with multiple choice questions which I
hadn’t prepared the students to face.
• It went badly

13. Chekhov's assessment task
• "If you say in the first chapter that there is a rifle hanging on the wall,
in the second or third chapter it absolutely must go off. If it's not
going to be fired, it shouldn't be hanging there."
• Anton Chekhov (From S. Shchukin, Memoirs. 1911.)
• “If you’re going to put a style of question in the final exam, teach to
the style of question throughout the semester”

14. Problem 3: Out of Alignment
• When somebody said “Constructive alignment”, I heard “blah, blah,
blah”
• I should have heard “Would you like your job to be easier?”

15. Solution: Constructive Alignment
• Biggs
• 1) What outcomes do we intend
the students to learn?
• 2) How do we teach them to do
that?
• 3) Does the assessment enable the
students to do what we’ve taught
them to do?
• Assessment should then do what the
subject says on the tin
• Your new best friend
• Graduate Attributes
• Course Level Outcomes
• Learning Outcomes
• National Standards
• Anything that says “This is what
you can do, or should be able to
do, or think, or perform”

16. Problem 4: Assessment was a stick
• Assignments were hoops
• “Achieve this. Answer that. Do that. Retell these factoids”
• Exams were punishment
• Exams were written to punish students for deviating from the curve

17. Solution: Align assessment to the course
• CA to GA to LO to “hello, this looks fun”!
• Graduate Attribute:
• Enthusiasm to search for further knowledge and understanding.
• Learning Outcome:
• Demonstrate an ability to search for further knowledge through the use of academic
literature in written essays
• Essay Task:
• “Using existing papers from the library database, Google Scholar, draft an updated version of
this Wikipedia page”
• Marks Criteria: Use of academic sources + Breadth of research
• No more boring (to you) assignments on dull (to you) topics
• Support
• LEO links to the Academic Skills
• Explain how the literature is the easy path, not the hardest path

18. Problem 5: Consultation versus Insultation
• Students would come to the office to see me about issues I’d thought
I’d covered in the class
• Clearly I hadn’t covered those issues
• I thought I had, and I thought the students were being a bit… unhelpful

19. Solution 5: The Self Service Student
• Aligning assessment to the course expectation reduced
• “I don’t know what I’m doing” when they meant “I don’t know why I’m
doing”
• “Why am I doing this? / “What’s the point? / This is pointless”
• “Maybe if we all fail the course, he’ll get into trouble”
Self solving students who can ask “How do I meet this goal you’ve set?” rather
than “Why am I do this at all?”

20. Digging upwards
Getting out of the teaching is burning all my time hole
Index

21. Reset your headspace
The job is fun
The job resides within a system
Systems are designed to maximise input:output ratios
Tune up the input

22. People can enjoy complexity
within clear structure…
Realisation 1

23. People do maths for fun.
• Video games are hard
• Tough Mudder exists
• And charges $165
• Sudoko. Crosswords. IKEA
• People don’t fear challenges when
they understand the requirements
of the task.
• Clarify the why and how of the
assessment

24. I should be enjoying assignment marking.
I’m not, so something isn’t going right.
Realisation 2

25. How to make assessment fun to mark?
• Connecting what you expect
someone to do with what you
teach/train them to do…
• Ask questions that you want to
discuss.
• Pursue research in the classroom
as a “What if?” and workshop
the ideas
• Assessment as exchange of ideas
• You get ideas
• They get feedback
• You both win.
• Teach from your research,
research from your teaching

26. Works nicely as well

27. Why should I enjoy assessment and marking?
• I’m going to be doing a lot of it, for a long career
• Death and Taxes are covering “inevitable not fun” shifts
• Students are a lens to see problems in a new light
• What if I ask questions for which I don’t know the answer?
• What about questions that engage technique change?
• Where else do I get this sort of focus group opportunity?

28. Changing the Question
• Ask questions where you don’t have a fixed answer in mind
• Or if you do, there’s several of them
• Criteria should govern technique and method
• Assess for technique, and set up so that technique can be showcased
through content
• Skills sets are graduate outcomes
• Give Feedforward and Feedback to the students
• Explain success as much, if not more, than you explain failure

29. Workload and Marking
Index

30. Upfront Investment
• Assessment Criteria
• Know if it can be marked before you set it out to the students
• Something I was routinely bad at doing, and it cost days at the marking end
• Set expectations well in advance
• Reward the expected.
• Do not begrudge success
• Assessment Support Before The First Assignment
• You’ve been there before, and they haven’t

31. Assessment Videos are your friend
• Talking through the assignment to camera
• Many takes make for a good solid think through before writing up the task
• Be interviewed by a colleague
• Work in pairs to extract the key components
• Explain expectations
• Talk about adequate, good and great performance
• Convey enthusiasm for the task
• Be genuinely keen to see the assessment
• Students worry about technicalities in class, and engage depth in video
• Formatting is a fear founded in modern education

32. Solving the assessment problem: LEO training
• http://www.acu.edu.au/staff/our_university/training_and_development/academic_s
taff/leo_learning_materials
• Ballarat , Brisbane , Canberra , Melbourne , North Sydney , Strathfield
• Learning and Teaching Centre 1st Feb 2016 – 20th Feb 2016
• Guided by local eLearning Advisors from the Learning and Teaching Centre
(LTC), you will learn how to create and organise learning materials in your
LEO unit/s; create and style LEO Page and Book activities; upload and
access materials in LEO content; and use Kaltura for your video resources.
• Intended Outcomes
• At the end of this workshop it is expected that participants will be able to:
• create an item in EQUELLA and add a document, or documents to it;
• add documents from your EQUELLA items to your LEO units.
• style Page and Book activities in LEO
• create and upload video materials using Kaltura

33. The Pointy End of Assessment
Marking
Index

34. 1 hour per student per semester
• That’s 6 minutes per 10%
• Six minutes is not enough time to have any fun with assessment
• Minimum cut-off: 20% and 12 minutes.
• In 12 minutes, you can
• Read the paper (5 minutes)
• Select marks from the criteria (1 minute)
• Add QuickMarks for feedback (3 minutes)
• Leave a 3 minute voice feedback of personalised instruction (3 minutes)
• 40% Assignments – 24 minutes
• 3 minutes of voice feedback (21 minutes left)
• Read the paper (10 minutes) (11 minutes left)
• Select marks from the criteria (1 minute) (10 minutes left)
• Add QuickMarks for feedback (5 minutes) (Change on the clock)

35. Mandatory Business Unit Exams
Unit Exam Weighting Exam Structure Total Subcomponents Minutes per component
1A 50% 2 x 4 questions 17 1 min 42 sec
1B 50% 3 from 4 questions 4-5 6 – 7 mins 30 sec
1C 40% 6 compulsory 24 1 minute
1D 50%
A 12.5%
B 37.5%
A – 25 mcq
B – 3 compulsory
25 + 11 MCQ – 7.5 minutes for processing,
calculation, delivery and input
B – 22.5 total = 2 minutes
1E 45% A – 20 mcq
B – 3
C - 2
20 + 9 MCQ – 6 minutes for processing,
calculation, delivery and input
B – 1 min 26 sec
C – 4 mins
1F 50% 5 questions 10 3 minutes

36. Assessment Maths: 25 students
• 30% Exam. 1 x 20 mark, 2 x 5% from a choice of 3
• 75 moving parts
• 40: Exam with 6 questions, choice of 5, marked from 100, 20 marks
each, 5 sub components
• 625 moving parts

37. Feedback
The single, universal, consistent point of failure in education (according to student
satisfaction surveys)
Index

38. Feedback
• the most powerful single moderator that enhances achievement
• https://teaching.unsw.edu.au/assessment-feedback
• It’s also really fun (and sometime emotional)
• You got to get a little teared up when they do so well

39. Doing Feedback Well
• constructive.
• highlighting the strengths and weaknesses of a given piece of work,
• set out ways in which the student can improve the work.
• Tell the students what they did right, and why it was right

40. Doing Feedback Well
• Timely
• Sooner rather than later
• Students will have always moved on.
• Aim the feedback as feedforward (talk to the next task)

41. Doing Feedback Well
• Meaningful.
• Talk to the assessment task
• Talk to the subject, and the broader scheme of the assessment
• Linked to specific assessment criteria
• Show a connection between the actions in the assessment and the learning
outcomes
• Move quickly

42. Why is feedback not done well?
• Timely Feedback is hard work
• 100 assessment tasks x 10 comments of 5 to 10 words in hand writing
= a very long day at the office
• Repetition happens on the marking side, not the student side
• “Didn’t I just correct this?”
• Repetition creates Resentment
• Resentment is never fun
• Fatigue

43. Fatal 4 Marking
• Fatigue
• Tired markers make mistakes
• Frustration
• Anger does not help the process either
• Inconsistency and Shifting Expectations
• Is this good? Bad? What even was my question or even my answer?
• Mmrmrmrmrmm
• Handwriting approaches catastrophic failure

44. Turnitin: Feedback Machine
For when feedback needs to be timely, industrial strength, and fast
http://www.acu.edu.au/staff/our_university/training_and_development
/academic_staff/leo_assessments
Index

45. Turnitin and Textmatch

46. Textmatch is an opportunity
• Establish expectations
• Explain that a text match is cliché,
and clichéd writing is tacky
• Encourage the deeper learning of
using the idea, not the phrase
• Tag and highlight matches with
advice on how to write from the
ACU Academic Skills Unit
• http://students.acu.edu.au/office_of
_student_success/academic_skills_u
nit_asu

47. Turnitin buys time back
Quickmarks are quick.

48. QuickMarks are drag and drop

49. Quickmarks can give depth

50. Quickmarks can praise

51. Quickmarks can inform

52. Quickmarks are more than you’re even going
to consider writing by hand

53. But what about?

54. I like to mark on the couch/beach/park
bench/backpacking in the Andes/McDonalds

55. But I like the physical paper…
• Turnitin provides
• Reduced liability for lost assignments
• Backups of comments
• Security and confidentiality
• Can enable blind marking
• “Dog ate my homework” proof to p=0.01 level
• Reduced physical strain from handwriting
• Improved portability with increased security for work-from-home
• Much lighter to carry than 100 20 page essays

56. What does it do for you?
• Improves student outcomes
• Feedback has impact
• Individual personalised voice feedback
• Creates cut through
• Really reaches the student
• Adds nuance to the conversation
• Reduces assessment follow-up meetings
• Buys time back from the workload

57. Why do it?
Because it buys time back in your work-life

58. Questions?
Stephen.dann@anu.edu.au