Authentic assessment and social media

Authentic assessmentin the era of social media:
Ideas from Internet Communications
Matthew Allen ALTC Teaching Fellow Curtin University of Technology

Overview
Background:The cultural affordance of social media
Context:Curtin’s BA (Internet Communications)
Examples:Assessment tasks / structure for net units
Conclusions:Social media, assessment and authenticity

Cultural affordance of social media
My argument can be understood by thinking about the way we name the developments in networked, online culture and technology in recent years – a labelling that marks out a ‘change’ from something we had before, a change from when first universities systematically built online learning.
Three common names for how the contemporary web is different:
Web 2.0
Read/write web
Social media

Setting aside, for a moment, technological matter, what is seen from different perspectives?
Web 2.0 (within e-commerce)value from and through data (inc. human), understood as a massively underexploited resource (amazon.com)
Read/write web (evokes education)websites as spaces of interaction between user and the web itself, re-inscribing the web with literacy (wikipedia.org)
Social media (from/against ‘media’)the end of convergence; web allows tacit collaboration for social programming, distribution, and making media (youtube.com)

How do these labels invoke different kinds of techno-human relationships through the way they emphasise varying meanings of technology?
Web 2.0APIs, fast code, sites ‘act’, data traffic that is largely invisible – a site is an application now; humans work for the database
Read/write webusers write to (and from) web, not just reading; immediacy of response on screen; web calls for change, but humans enact it
Social mediadisguises the technology: speaks of a new social process which sets aside the tech, to spotlight human communication

Social media Society collectively and variously usesinteractivitythrough read/write and programmable web Humans return technology to ‘normal’ object status
society ‘uses’ human-technology symbiosis?
Read/write webIndividuals using programmable web to interactTechnology and humans act together
Web 2.0 Data-based, programmable webTechnology as actor on humans
Interrelating our ‘names’ for the change from 1990s to contemporary web

So why social media?
Our attention slides away from technologies and skills in using technology towards a more open domain:
The life-world of studentsit is an insistently, intensively, media-driven world (whether mass, online, personal, or simply mediated): technology is media
Knowledge work is granular, specific online tasksanswers, inputs, outputs are - in themselves - quick and small; the whole is always a networked accretion; knowledge is tasks
Media makes us think audiences media is always going to somewhere, someone else; media is thinking information as communication

BA (Internet Communications)
First critical context is, of course, the subject matter:
Aim is to teach people both socio-cultural knowledge and analytical skills in a way that might be traditionally associated with a humanities degree
Internet Communications is a major, roughly 40% of the whole degree, within an arts degree
But, it is also aimed at quasi-professional qualification for “Internet communicators” (across many areas of employment); can include 2nd major in more technical / professional areas (e.g. Digital design), but not required
“Technology” (the Internet) is taken seriously as both the object of study, the means of study, and the mechanism by which we can do the work of ‘studying’ the Internet

Second context is mode of delivery:
Each unit in the course is taught on-campus, to an increasingly ‘campus’ oriented population of students in a traditional 2-semester / annum
Taught also for external students enrolled through Curtin, at same time as on-campus students, 2 semesters / annum; some external students are ‘time/space shifting’ campus students
And taught, solely online, at distance for Open Universities Australia (more of a broker than an institution), using a 4 study periods / annum model; study periods do not map to semesters (except, partially, semester 1).

This mode of study also helps to reveal student context:
Internet Communications course, on campus, has small cohort – perhaps 10-15 per year; yet units have larger cohorts (Mass Communication, Librarianship, and other courses) – up to 100+ in one unit
On-campus – diverse mix of enthusiasts, compelled students, Australians, various Asian overseas students; students all ‘qualified’ for uni in academic terms but often are motivationally unqualified; language skills issue for some students
Through OUA, we can have any student studying – open access, but a major market for us – continuous growth over several years, most units have 50+ students; first-level units 150+
OUA students can lack knowledge of, connection with ‘university’; suffer unrealistic expectations or lack of confidence; poorly advised; motivations are good but diverse

History of the program – necessary context also
Commenced in 2000 (2001 OUA); dot.edu start up; highs and lows on campus, but steady growth - ‘Founded’ a department
Was an early innovator – fully online, utilising theories of communities of learning sustained via online discussion groups in LMS; classic ‘Web 1.0’ online learning: didactics (lectures) replaced by additional readings; extra scaffolding; increasingly sophisticated facilitation of discussion
Assessment largely traditional with strong emphasis on discussion (20-30%)
Some tinkering with new approaches (e.g. Wikis; delicious) but insufficient resources and institutional context (‘waiting for the great leap forward’)

The great leap forward –context of recent change at Curtin
Whole-of-university change program to manage curriculum and enforce control via learning outcomes; plus structural reform of faculties and degrees
This external environment enabled rapid change – classic unfreezing, even some funding, yet still required manoeuvring past the ‘teacherly’ learning outcomes approach (and staffing / money remain scarce for continuation)
Contrasting approaches: assessment as driver in both cases – for them, assessment as ‘guarantee’ of learning outcomes demonstration; for us, assessment as starting point to create engaging / challenging learning experience

So how does context matter?
There is no generalised application of technology in education:
Diverse and complex student cohortmany variations from the ‘imagined’ student of learning design; conflicting responses from students to what we offer
Circumstances determine ‘innovation’what we think of a technology, how we conceive of both the problem and the solution, determined by history and structure
Content and discipline matterwhat we do with social media and the Internet is bound up in the fact we are teaching / learning social media and the Internet

the Internetfor learning
Context as prism
learning theories
Localised, contextual outcomes of ‘educational technology’

ExamplesAssessment tasks / structure for net units
Web Communications 101Introductory unit focusing on learning the mechanics of social media, online presence, and identity through communication
Internet and Everyday Life 102Introductory unit with a more ‘academic’ focus on ideas, and learning to do knowledge work in online context
Internet Communities and Social Networks 204Main focus is on concepts, histories, ideas but exemplifies ‘total assessment’ approach based on online activity
PlusBrief summary of various approaches and assessment tasks

Web PRESENCE :: “Networked self”
Web Communications 101

Assessment summary
Short-answer test(5 questions, 1 paragraph each) –Week 4 – 15%
Essay(1500 words, set question – choice of focus) –Week 8 – 25%
Web Presence(a website – e.g. ‘blog’ - theme and identity) –Week 13 – 35%
Learning Portfolio(collated weekly entries re tasks / conversations + reflection) –Week 13 – 25%

Web Presence (extract of instructions)
The aim of the Web Presence Creation is for you to develop an initial online presence for yourself. Your web presence has some required elements: a central node (e.g. the main website that at the heart of your web presence, which might be a blog, a wiki, or another form of website) , which includes an ‘About’ page and an ‘Exegesis’; and at least three contributing nodes, which are different web 2.0 tools or services which are linked to from your central node, link back to your central node, and are part of your overall web presence.Your web presence must combine your identity and a purposeful theme, which means it can’t just be about ‘Jane Smith the Curtin Student’ but must focus on an interest of some sort , for example, ‘Jane Smith the Future Public Relations Guru’...
.

What students thought:
Students learned ‘how to’, when building their presence – a site and linking to it other “web 2.0” services such as Flickr, Twitter and Delicious. As one student puts it:
The web presence assignment for the Net11 unit has provided me with many hours of research and a greater understanding of how social media can assist in communicating a message to an audience. http://photographycomp.wordpress.com/
It also made them think:
Overall, the creation of my web presence has been enlightening and useful. ... I have developed a new form of self-censorship as I am now mindful of what it is I am placing online. My view of the online tools available to me has been broadened with the understanding of what it takes to bring these different platforms together. http://siimonrampe.wordpress.com/

Plus / Minus / Interesting
The question of online publication was not contentious; identity and self online were part of the assignment, not taken for granted
Students were more enthusiastic and did more work than for the ‘academic’ tasks; they interpreted instructions better and explored
But, many students had to be led into the task and did not see it as a natural part of ‘who they were’, either as students or net users; some formulaic responses
Students often shared publically what they were working on (usually via twitter) for peer feedback (unprompted)
We had to impose ‘artificial’ constraints to make it assessable

SELF-expressive tasks :: Granularity
Internet and Everyday Life 102

Assessment summary
Learning Tasks Portfolio(5 ½ page responses to set tasks; plus reflective questions) –Week 6 – 20%
Essay(1500 words, choose 1 task area from above, focus on it) –Week 7 – 20%
Learning Conversations(collated examples, plus reflection, of student interaction) –Week 11 –30%
Essay(1500 words, integrate material via conceptual themes) –Week 13 – 30%

Tasks Portfolio (extract of instructions)
Choose the five best responses to the tasks set out in topics 1.1 to 1.6. Your response to each task should generally be about one page as set out in the relevant module. In addition to this, from the topics you have chosen answer the following 3 questions.
How is the Internet part of your everyday life?
How is the Internet part of the everyday life of other people in society?
How is your experience of the Internet different from people around you?
Example of tasks:
Select a particular health topic that interests you for any reason (it could be a friend or relative's medical condition, for instance). Find out more about the topic using the Internet. While you are doing this, note down the main ways you'll begin looking and your process of selecting useful sites. How would you say your knowledge of the topic has changed in the course of this research? What are the criteria by which you'd select sites? Finally, describe 2-3 of the most useful sites and Internet applications and the ways in which they'd be useful to people visiting them (no more than a page)

Surveys at end of unit of students’ attitudes revealed a significant challenge:
Yes, specifying tasks associated with readings and ideas linking them to assessment, motivated students to work more consistently
and
Yes, students overall liked this approachbutNo, students did not find easy the process of doing small tasks, and collecting them into a portfolio with reflection
And
No, students did not actually do as much work as they thought they should

Shorter, more regular tasks fits with the life-cycle of students’ attention to their work
Grades do motivate, when otherwise not (but note distinct minority ca. 20% say no)
Assessment is not bounded and discrete in itself; therefore smaller tasks were like grains of rice, not grains of sand
‘Collation’ (because we can’t have more than 4 assessments) is arbitrary: does not integrate the work into coherent whole
Overly authentic to learning theory; will the problems disappear if there are more public, shareable activities? (we will see)

ONLINE CONFERENCE :: KNOWLEDGE work
Internet Communities & Social Networks 204

Assessment summary
Conference Paper(2000 words; submitted – feedback – re-submitted) –Week 6 – 50%
Conference Participation(peer and tutor assessment of commenting / responding) –Weeks 9-11 – 30%
Review of learning(focus via Wikipedia entry; reflect on study, assess it) –Week 13 –20%

Participation (extract of instructions)
You need to spend around 30 hours on this stage, completing this work by the end of Week 11 of the study period. Your aim is to learn about communities and networks by being a successful participant in the online conference. Your tasks are to read some of the other papers, comment on them constructively, reply to comments on your own paper, and build a dialogue through these interactions.
[...]
It is good to develop a strategy for how you plan to engage with the conference. Remember when you are posting comments and engaging in debate about other people's papers to try to link in to your own paper as part of your discussions where appropriate.

A recent comment:
I have to say this is one of the most enjoyable assignments I’ve had to do across all the units I’ve done (all 20 to be more specific). Great idea. ... I have seen some amazing papers and I’m very excited to be a part of this. Well done everyone and good luck. Hope you all get a good mark.http://networkconference.netstudies.org
And, in terms of intensity and debate:
Over 95 papers, with 1298 comments; and every paper has at least one comment;

From the outside…
And, from outside:
...the papers submitted all show a level of passion and writing skill that speaks for itself. From a university perspective, this conference is certainly a way to showcase student work but also a way to demonstrate the dedication and support lecturers and tutors have provided to their students. It is an excellent way to attract new students and foster networking within the student community.http://britbohlinger.wordpress.com

Students took ownership of the promotion and networking of the conference (e.g. Youtube videos; links from their own sites)
Appears that student conversations, focused on papers, are more effective, thoughtful and productive than in ‘discussion forums’
Students need significant assistance in writing / self-perception not aligned to tutors’ judgments; ‘public’ is not uniformly motivating
Conference demonstrates that we will always have variable engagement with learning – but now it is just more visible
Approach would appear, in large part, to bankrupt attempts to impose normative, instrumentally rational educational models

Some brief examples of other activities

Various uses of social media
Diigo as ‘individuals working as network’ tool for bibliographic work (in process – not sure how it is going – will compare with other possible approaches)
Wiki as tool for individuals to write assignments which all can observe (not even anticipated – students learned through parallel play)
Online teams working to produce large report for publication: the publication choice (by us) failed; the teamwork tool choices (by students) worked better
Slideshare for simple publication: challenges students to condense, utilise only the slides, and not the voice

Follow up
Web Communications 101
http://www.slideshare.net/Tama
http://tamaleaver.net
Internet Communities and Social Networks 204
http://networkconference.netstudies.org/
More generally...
http://netstudies.curtin.edu.au
maybe, http://netstudies.org

ConclusionsDoes it work?
In large part, yes it does: once already working in an online mode, through a learning management system, creating and managing these activities imposes no greater burden than would be the case without them
Data collection and reviews still being finalised, but the impression is that students are - on the whole – at first uncertain, then slowly accepting, and eventually enthused by approaches which treat the ‘network screen’ as the locus of their assessment, rather than what they expect (exams, tests, essays)
But it works because the social media used are used within the frame of reference afforded by detailed attention to assessment design

External audience
TheScreen
Teacher
Student
Other students
The Internet
Other students
Central to social media is the interactive screen (the return of the read/write web) which changes practical means of production, distribution, and reception

ConclusionsAssessment as the key design strategy
Assessment is the interface between ‘teaching’ and ‘learning’ - the most threatening but also productive moment when students and teachers engage in a joint enterprise of learning/teaching
In the units described, each individual assignment contributes to all the others (more or less successfully) such that ‘assessment’ is a single central reference point: lectures, materials, discussions, feedback, tasks all make sense as part of ‘assessment’
But, when understood in this way, assessment foregrounds the tensions between students as ‘learners’ and students as ‘disciplined subjects’ of educational process: these tensions influence the experience of authentic assessment more than theory might suggest

Didactics and discussionProvide ‘content’ in a way that allows choice of topic, then supports paper-writing
Write paper for conference; submit 3 weeks before start; feedback 1 week before; revise and finalise and post live
Conference
Participationand discussionComment on othersReply to comments on ownUse comments to link papers
Plan and discuss how to do conference; how to recruit; ‘student-led’ and done while tutors are marking papers in 2 weeks after submission
Review and extend learninglinks original learning withpaper written withwith discussion
Conference narrative, not weeks or topics, provides the time stream for unit of studystages, each of which has a purpose relating to conference; then mapped to dates

Conclusions‘Authentic’ assessment as negotiating meaning
There is no such thing as truly authentic assessment. If it were ‘authentic’ then it would not be assessed; it would not be education.
Authenticity has several meanings in educational terms: none excludes the others. Assessment is authentic if it is consistent with
learning outcomes / curriculum
real-world practice for which students are learning
knowledge work as experienced in society
expected norms of behaviour for students and teachers
pragmatics of the transformation of learning into education
Therefore, the ‘real’ authenticity of assessment is not a priori, but a negotiated compromise in which students must suspend their disbelief in being ‘students’ and teachers must stop ‘teaching’, and both must trust each other at the moment when the act of judgment by one over the other re-enters the field of play

Formal requirement makes learning “education” (e.g. credentials); some students expect / want assessment to meet this norm
Particular kinds of work or practice for which the course of study is preparing students; some students expect / want assessment to be ‘real’
No single factor makes assessment inevitably ‘authentic’
Authentic to the life-world of the students as they relate institutional education to their own views on and habits of learning; students vary markedly in their self-judgments
Authentic to the practices of knowledge work as understood by academics as knowledge workers themselves (hence internet scholars enthusiasm for networks, social media)
SocialMedia
Assessment is the focus because of the assumption that students learn more, and more effectively, when they ‘do’ something and assessment is what prompts and rewards doing

ConclusionsSocial media, authentic assessment and tech
One key aspect of ‘authentic assessment’ here is the desire to tap into the students’ own world –this is not a world of ‘technology’, it’s a world in which students behave because of the consequences for society of technological change
So, social media is a mindset: it is about the public exchange and networking of ideas and knowledge work; it is not about technology per se. Students often don’t see the technology
The challenge is not the skilled / not skilled so much as students not seeing:
‘their’ social media is also ‘our’ social media
the depth of knowledge work that can be involved in social media (‘shared space’ - not like TV / books)
that the technology is not a ‘black box’

Return to “why social media?”What to look for in ‘social media’
Collaboration can be teamwork, but more likely to be:
parallel play (individuals working side by side, but looking on)
aggregation (individuals working alone, but ends up together)
networking (individuals deliberately link ‘nodes’)
Conversations can be the primary form, but more likely that individuals hold a conversation:
with themselves (talking to the represented self on screen)
through individual ‘statements’ which they link together
Participation creates the content effectively through:
low thresholds for participation (start small and learn as go)
ownership of process and equipotential for contribution
granular tasks that contribute, satisfy, and complete discretely
(Social Media: State of the Art, Bruns &Bahnisch, 2009)

Thank you
Matthew Allen (@netcrit)m.allen@curtin.edu.auhttp://netcrit.net/

http://www.netcrit.net	162
http://www.knowledgenetworklearning.net	43
http://www.slideshare.net	8
http://webcache.googleusercontent.com	1
http://translate.googleusercontent.com	1

Authentic assessment and social media

by Matthew Allen on May 10, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

5 Embeds 215

Statistics

Authentic assessment and social media — Presentation Transcript