The document describes a study that investigated the effectiveness of a wiki-based learning activity on student performance in a human-computer interaction course. The study found that over half of students improved their test scores by at least 50% after participating in the wiki activity. The activity was most beneficial for students with lower initial scores. Additionally, the role students took on in their wiki teams did not affect their learning gains. The wiki activity was designed based on critical factors from prior research and included student collaboration on usability evaluation topics in their wiki groups.

1. Nikolaos Tselios* | nitse@ece.upatras.gr
Panagiota Altanopoulou* | galtanopoulou@gmail.com
Christos Katsanos# | ckatsanos@eap.gr
Effectiveness of a framed wiki-based learning
activity in the context of HCI education
PCI 2011 | Kastoria, Greece | 30 Sep -2 Oct 2011
*Educational Science and Early Childhood Education, University of Patras
# Internal Assessment and Education Unit, Hellenic Open University

2. Are the wikis suitable to
design learning activities?
 Web 2 has brought much greater attention to collaboration
on the Web, and this has begun to flow into e-learning
 Wikis are open-nature and can provide strong opportunities
for collaboration, critical thinking and reflection
 Central tenets of sociocultural views of learning, and in particular of
project based learning (Duffy & Kirkley, 2004)
 Wikis are also easy to learn and usable
 Can be used both for distance and blended learning
2
However, without planning, design, and effective facilitation,
a wiki is no more than an empty webpage (West & West, 2009)

3. Research Goal & Questions
 Investigate if (and to what extent) a properly designed
wiki based activity can enhance the learning process
 Research Questions
 Does students’ learning performance improve through a
properly designed wiki-based activity?
 Are such activities more beneficial to students with lower initial
performance?
 Does the student’s role in the team affect her/his learning
performance?
3

4. Using wikis in education
( West & West, 2009)
4
•Class encyclopedia
•Glossary
•Electronic book
Knowledge
construction
Critical
thought
•Case studies
•Structured debate
•Annotated bibliography
Contextual
application
•Organize design activities
•Story creation
•Organize real-world activities

5. Critical factors in the design of
wiki-based learning (West & West, 2009)
1) Establish a purpose for the wiki project
2) Define & classify the learning goals
3) Design a rich context & problem that support the
achievement of the purpose & goals
4) Prepare students for work in the new environment,
5) Promote a collaborative process through which active,
social learning can take place
5

6. Study Methodology
 Participants
 36 students
 Dept. of Education & Early Childhood Education, Upatras
 Compulsory project, Course: “Introduction to Web Science”
 Within-subjects, Pretest-Posttest design
 Demographic questionnaire (pre-)
 5 questions, demographic data (e.g. age, ICT skills, wiki usage)
 Knowledge assessment questionnaire (pre- & post-)
 35 multiple-choice questions with 4 answer options
 Questions on usability evaluation & heuristic evaluation
 Students were not informed about the assessment
6

7. Activity design (1/4)
 Framework: West & West (2009)
 Goal: Study usability evaluation techniques in general and
heuristic evaluation in particular
 Learning objectives
 Understand impact and importance of usability
 Understand usage of wikis (both in general and for education)
 Study usability evaluation techniques
 Be able to conduct a heuristic evaluation
 Be able to form collaboration teams with specific roles assigned
 Be able to organize and report the results of such a process
 Be able to use a Wiki
7

8. Activity design (2/4)
 Each group had to create a wiki (WikiSpaces service)
 Assignment presented to the students through a wiki
initiated by the researchers (http://web-usability.wikispaces.com)
 The wiki included 3 parts (Usability Evaluation, Heuristic
Evaluation, Actual Heuristic Evaluation of the Dept. Site)
 Concise instructions
 Indicative outline
 Arguments to be developed
 Supporting material (mainly in the form of hyperlinks)
8
Students were encouraged to use additional material
& arguments (discrete evaluation criterion)

9. Activity design (3/4)
 Procedure
1. Instruction on the wiki’s basic functionality
2. Presentation of the compulsory project in the form of an already
initiated wiki
3. Students worked in freely-formed groups of 4
4. Each team member had a specific role in the group
 Collector: obtain appropriate material relevant to the subject
 Coordinator: organize collected material, check its consistency,
coordinate discussion on website heuristic evaluation
 Editor: compose the outline of the topics according to the
objectives of the scenario
 Verifier: evaluate content completeness, structure and
compliance with the objectives of the project.
9

10. Activity design (4/4)
 Evaluation criteria (included in the wiki)
 Coherence
 Content and text originality
 Text clarity, argumentation and reasoning
 Compliance to the provided structure & format guidelines
 Use of additional material & arguments
 Material appropriateness and richness
 Degree of collaboration
 Appropriate use of references
10

11. Wiki Screenshots
11

12. Results (1/3)
12
 Does students learning performance improve through a
properly designed wiki based activity?
 More than half (56%)
improved their performance
by at least 50%
 Highest improvement in
score 65.7 (14.3 => 80)
 Some (22%) did not
improve or even did worse

13. Results (2/3)
13
 Are such activities more beneficial to students with lower
initial performance?
Correlation
r=-0.586, p<0.001

14. Results (3/3)
14
 Does the student’s role in the team affect her/his learning
performance?
ANOVA: F(3,32)=0.732, p=0.541
Students were benefited
regardless of the roles they
selected themselves

15. Summary & Questions
 Through a properly-designed wiki-based activity (West & West, 2009)
 h
 Future work:
 Compare learning gain with a control group (traditional teaching)
 Design more wiki-based activities in a various educational settings
 Investigate learners’ behavioral intention to use wiki technology using
technology acceptance models
 Investigate to what extent students improve in other non-cognitive
aspects (e.g. self-organization, collaboration)
15

16. Extra Slides
16

17. What about final scores in the
project?
 We were interested in investigating the learning gain,
not the final outcome
 Knowledge Assessment Questionnaire
 Learning gain = Posttest_Score – Pretest_Score

18. References
 Cress, U., & Kimmerle, J. (2008). A systemic and cognitive view on
collaborative knowledge building with wikis. Computer-Supported
Collaborative Learning, 3(2), 105-122.
 Duffy, T., & Kirkley, J. (2004). Learner-Centred theory and practice
in distance education cases from higher education. Mahwah, N.J.:
Lawrence Erlbaum Associates.
 Mindel, J., & Verma, S. (2006). Wikis for teaching and leading.
Communications of the Association for Information Systems, 18(1),
1-23.
 Raman, M., Ryan, T., & Olfman, L. (2005). Designing knowledge
management systems for teaching and learning with wild
technology. Journal of Information Systems Education, 16(3), 311-
320.
 West, J.A. & West, M.L. (2009). Using Wikis for Online
Collaboration. San Francisco:Jossey-Bass.