M-learning brings the possibility of creating situated learning activities that take place in physical spaces (Jeng et al. 2010). Benefits derived from this type of activities are related to developing exploration skills and cooperation (Hwang et al. 2008). In this line, previous research work on QuesTInSitu, an m-learning app to support assessment in situ , shows that students put explorative and spatial skills into practice and fosters their motivation, and personal observation. However, further research work involves supporting the students ’ reflexion when performing this type of learning activities (Santos et al. 2011).
Games is an option to adress learning and motivation. Empirical evidence supports the positive effects of computer games as learning tools. The evidence indicate that games implementing pedagogical designs can strengthen and support school achievement, cognitive abilities, motivation towards learning, reflection, attention and concentration. In order to engage students to reflect on their decisions while performing learning activities in situ, we mainly focus on the following challenge: How can we support more fruitful learning activities in situ that can be designed by teachers? With the aim of allowing teachers the design this type of learning activities, we adopt an approach that consists in using a metaphor based on puzzle games as an innovative way to design gamified in situ learning activities. In general, we take as a basis the jigsaw puzzles, meaning the arrangement of a set of given pieces into a single, well-fitting structure that interrelates the pieces. Also: Puzzle-based games can engage students in the subject topics, foster students ’ problem solving, analytical and memory skills (Huang 2007; Bottino 2008). A puzzle should be interesting because its result is not immediately intuitive. The nature of puzzle-based games seems relevant to consider as potential educational strategy to feasibly involve teachers as game designers (Huang 2007; Crawford 1982).
To tackle our research question: First we propose a conceptual model, and its associated binding, for the design and computational representation of puzzle-based games. We aim at support the teachers the creation of in situ learning activities following the conceptual model. The resulting designed activities we ain at enactment with students.
The proposed conceptual model considers: Previous research studies about game design elements that have to take into account when designing educational games (Fisch, 2005; Jones, 1998; Kirriemuir et al., 2004; Malone, 1981; Sandford et al., 2005; Squire et al., 2003) A 4-dimension framework, considering the role of teachers, intended to evaluate the potential of using games- and simulation-based learning (de Freitas et al., 2006). As a result, the approach defines the learning flow of the whole game, the context where the game takes places as well as the puzzles associated to each activity.
In general terms, a puzzle-based game design considers a person playing a role in the gaming process. The outcomes of the games, stated as gaming objectives, have to be achieved by performing the learning activities associated to the levels of the game. Also, each activity can have associated a hint to scaffold the learning process. Besides, the story of the game consists in defining which player role has to perform which activity and at what moment in the gaming process. Each activity has associated a puzzle, composed by a set of virtual or physical pieces that can also include scaffolding mechanisms. Pieces can be part of a board in which players should relate each piece with its corresponding slot. Depending on the puzzle game design (e.g.: game to be played indoors or outdoors), either pieces or slots could have associated virtual, tagged or geo-located positions.
Considering the proposed conceptual model, we explore how can we applied in the creation of question item-based activities. Focusing on the role of the students, we claim that the puzzle-based game metaphor can be applied for creating gamified situated learning activities to enhance the students ’ learning experience: The aim of applying this strategy is to engage students to reflect on the correct solutions for each question proposed by the teacher. Similarly to jigsaw puzzles, players could try to solve the different questions as many times as needed until reaching a correct solution. We escape from just having one attempt to solve the questions and providing immediate feedback when solutions are incorrect. Instead, students have more chances to solve the questions until finding the right solution. Students have the possibility of finding the correct solutions either by reflecting on their wrong past choices or taking benefit of the resources provided not only by the gamified application itself but also by the information that can be found in the real place, discussing with other students, etc.
For teachers, we believe that this is a flexible metaphor for the design of gamified situated learning activities (independent of the subject matter). The puzzle-based game metaphor considers the following elements to design the (assessment) in situ learning activities: The “ board ” is the set of questions that is associated to geographical zones or buildings maps. The “ slots ” are the different questions designed for the gamified learning activity. The “ puzzle pieces ” are the different options for each designed question. Just one puzzle piece can fit in a concrete slot, meaning that there is only a correct option for each question. The “ puzzle ” . The puzzle is formed by a group of questions (i.e. slots). The “ level ” . A level contains just one puzzle. It can be defined as many levels as the designer wants. For instance, the designer can create a level per each museum ’ s room or geographical zone. The “ points ” are defined to reflect the students ’ performance: a) correct answers add points to the overall player ’ s punctuation, b) incorrect answers subtract points, c) consulting hints subtract points. The “ bonus ” . When all the questions for a given level have been correctly answer, students will obtain a bonus that will add extra points to the overall punctuation. The extra bonus is a reward to engage and encourage students to correctly complete the different puzzles. The “ feedback ” . This means textual information associated to specific range of points in order to show the students how is their performance. The “ hints ” . Hints are provided to scaffold the learning process in order to avoid frustrations and advance forward the whole gamified in situ learning activity. Besides, when all questions, except one, have been correctly answered, the student will obtain a “ free hint ” . This means no points are subtracted when consulting the hint of the last question to be solved of a given puzzle.
Since this research introduces a new approach to different subjects (teachers and students), and we aim at analysing the usefulness, comprehension and understanding, we follow a design-based research methodology (Barab et al., 2004; Collins et al. 1992; DBRC, 2003). This methodology involves continuous cycles of design, enactment, analysis, and redesign. One of the key points of following this methodology is that subjects became co-participants in the design and analysis, addressing complex problems in real contexts. In this sense, the development of design-based research occurs in authentic settings, involves multiple dependent variables, focuses on characterizing the situation in all its complexity, involves flexible design revision conducting rigorous and reflective inquiry to test and refine innovative learning environments. Thus, this methodology involves complex social interactions with participants sharing ideas, involves looking at multiple aspects of the design and developing a profile that characterizes the design in practice, involves different participants in the design so as to bring their differing expertise.
Context : The puzzle-based game metaphor as an approach to create gamified in situ learning activities Participants : Teachers (designers) and students (end users) of secondary education Design : Game design task filling the templates containing the different key elements of the conceptual model to design the gamified situated learning activities Enactment : QuesTInSitu: The Game: http://youtu.be/BTSsXa_e-6M Analysis : Mixed evaluation method considering: Observations, tests, questionnaires, and log files (with students) Interviews and questionnaires (with teachers)
Gamified in situ learning experience: The MNAC Activity associated to a subject, as a learning activity. Learning about different contemporary pictures of the MNAC. One teacher involved in the game design task. 72 students divided in two big groups: one using the gamified application, the other not. Game application: 4 levels (each level corresponds with each museum ’ s room) consisting of 20 questions (with 3 possible answers per question). The correct answers added 50 points to the overall student ’ s punctuation. The incorrect answers subtracted 10 points to the overall student ’ s punctuation. Accessing to the hints subtracted 50 points to the overall student ’ s punctuation. The extra bonus, obtained when correctly answering all the questions for a given level, added 50 points to the overall student ’ s punctuation. 19 hints were designed. These were descriptions about the subject matter related to the question. The level ’ s information was a text containing information about museum ’ s room. Feedbacks were written in an informal way, encouraging students either when making good or bad decisions. Learning experience 2: Discovering Vic Activity associated to a subject (part of its formative assessment). Learning about the city of Vic and its art history (unfamiliar city for most of the students). One teacher involved in the game design task. 72 students using the gamified application. Game application: 4 levels (corresponding to zone geographical zones of the city) consisting of 75 questions (with 4 possible answers per question). The correct answers added 1 point to the overall student ’ s punctuation. Incorrect answers subtracted 0.3 points at the first attempt, 0.5 points at the second attempt and 1 point at the third attempt (each wrong attempt accumulated the previous extracted points). Accessing to the hints subtracted 0.2 points to the overall student ’ s punctuation. The extra bonus was independent of the number of questions per level. Thus, 1.5 extra points were obtained if all the questions were correctly answered at first attempt. Otherwise, 0.75 points were obtained if all the questions were correctly answered, but not at the first attempt. 25 hints were designed. Here, hints were small texts giving clues to the students. Each level ’ s information was just a short sentence of the geographical zone where the questions were located. Feedbacks were written in a formal way, just indicating if actions, or proposed solutions, were correct or not. Learning experience 3: Discovering l ’ Hospitalet Transversal activity in the school. Discovering and learning about the heritage of the city of l ’ Hospitalet (their own city). 7 teacher involved in the game design task. 64 students using the gamified application. Game application: 10 levels (corresponding to geographical zones of the city) consisting of 55 questions (with 4 possible answers per question). The correct answers added 250 points to the overall student ’ s punctuation. The incorrect answers subtracted 100 points to the overall student ’ s punctuation. Accessing to the hints subtracted 50 points to the overall student ’ s punctuation. The amount of extra bonus that could be obtained for each level depends on the number of questions contained in the level. This means, each question was equivalent to 100 points. So, if one level contains 5 questions, the students obtained 500 points more to the overall punctuation when completing the level. 52 hints were designed. These were suggestion to the students rather than clues (e.g. look at the bottom of the statue, ask the people nearby, etc.). The level ’ s information was a text containing information of the overall questions for the particular level (i.e. geographical zone) and the questions in particular. Feedbacks were written in an informal way, encouraging students either when making good or bad decisions.
The puzzle-based game metaphor is perceived as an approach that can have educational benefits, being an entertaining approach that can motivate students. The proposed puzzle-based game metaphor has been found stimulating and a good approach to motivate students. Paying attention to the possibility of answering several times the questions, the teacher had concerns about knowing how many times the students have failed each question. Despite this methodology is better to help students reflect upon their mistakes; it is difficult to check what questions have been the most troubled
In all the learning experiences, the majority of groups answered all the questions until reaching the solution. In the MNAC, students couldn ’ t decide; they were forced to answer all the questions. In Vic and l ’ Hospitalet, the students could decide what to do. While students from the MNAC did not like the approach, and they believed to not think better the solution, the students from Vic and l ’ Hospitalet, preferred this approach compared to having only one chance to solve the questions, and think and pay more attention when answering the questions.
Regarding the punctuation, in Vic, the obtained punctuation was taken into account as a part of the history art subject. In l ’ Hospitalet, the punctuation was not taken into account as a part of any evaluation, but this not influenced in letting the questions without answered or incorrectly answered. Regarding the punctuation, the students from l ’ Hospitalet found this information as the most important, while the students from Vic experiment do not. This makes sense since, l ’ Hospitalet was an isolated learning activity that did not have any implications in the students ’ curricula, and the punctuation was the only outcome that students could obtain. However, the students participating in Vic, this experiment was as a part of other activities proposed by the teacher, and as a consequence, their outcome depended not only in the in situ learning activity but also in previous and post activities that the students had to perform. Results regarding punctuation mechanisms (adding and subtracting points when correctly or incorrectly answer questions) suggest that should be design following a more game-oriented approach based on obtaining higher amount of points. Students from Vic and l ’ Hospitalet agreed on the subtraction mechanisms. However, students from Vic experiment were discouraged each time they failed a question. On the other hand, students from Hospitalet experiment were not discouraged when failing the questions. Considering the learning context of both experiments it is logical these reactions since, the overall punctuation obtained by the students from Vic experiment had a repercussion in their subject about history arts, while the other students had not. Students from Vic and l ’ Hospitalet experiments agreed that the extra bonus and the points obtained when correctly answering the different questions were appropriate and motivating. Students for MNAC the extra bonus did not make sense to the students, since everyone obtained it because we forced to complete correctly all the questions of the different levels. In this case, no bonus extra will be obtained and, as a consequence, the students ’ efforts are not rewarded.
Hints have been the most controversial mechanism. Overall, hits have been used very poorly in the three experiments, and the gathered data regarding this mechanism has been quite low. The main reason has been because the subtraction of points. Also students preferred to asking people or searching by the Internet. However, some tendencies have been detected. There is no a clear agreement from the students from l ’ Hospitalet experiment when asking about whether hints are useful or help to guess the correct solution. Despite they found logical to subtract points when accessing to the hints, they did not agree on the amount of points subtracted neither on the usefulness of the hints. This is reasonable since hints were designed in a way that were more like suggestions rather than clues, and students got upset about the obtained information and for losing points for this kind of information. On the other hand, the hints designed for the Vic experiment consisted in small descriptions about contextualised information of the questions. This information contained clues for answering the questions. In this case, hints have been perceived as useful to achieve the correct solutions. Also, students agree on subtracting points when accessing to the hints. However, there is a positive tendency of considering hints unnecessary. The main reason for this, it is because students could complete the whole game without consulting most of the designed hints. Regarding the hints, further research is still needed about how properly design this mechanism. Students found hints unnecessary because the content of the hints was considered unclear and misleading, and disagreeing in the subtraction points. Results suggest that in order to encourage the students the access to the hints, these should subtract less points than failing a wrong answer.
Regarding levels ’ information, while students from l ’ Hospitalet experiment agreed on the helpfulness and the necessity of having the levels ’ information, the students from Vic, as well as from the MNAC experiment had doubts about whether it is needed or useful. This makes sense since the levels ’ information designed for Vic or MNAC experiment did not contain useful information to help answering the questions; this was only a introductory sentence about the zone where the questions were placed. In Hospitalet experiment, however, the levels ’ information was a descriptive paragraph not only about the geographical zone but also containing information useful to answers some of the questions. Students from all experiments agreed that feedback is necessary and motivating.
The students using the puzzle-based game approach obtained slightly better outcomes in the post-test than the students using the test-based approach. In fact, the mean of the students using the puzzle game approach was one point higher (i.e. 7) than the others (i.e. 6). Also, the range of marks was more concentrate in one group than the other. While students using the puzzle-based game approach obtained between 4 and 8 points, the other students obtain between 2 and 8 points. Paying attention to the results, this indicates that all the students of the first group would pass the post-test, while the students of the second group won't. More specifically, four students using the test-based approach obtained less than 4 points in the post-test. Thus, considering these facts we can claim that it seems that using the puzzle game metaphor students can obtain better learning outcomes.
Summarizing, in regards to the teachers ’ opinions, the conclusion are: The puzzle-based game approach can allow the design of fruitful gamified learning activities in situ Some concerns related to having the possibility of answering several times the questions arose. These are mainly about knowing the most troubled questions. In this regards, further research lines suggest to support somehow the teachers with visualizations about the students ’ performance. Problems understanding the “ level ” element. Problems designing hints suggesting that not all questions should have associated hints. Extra bonus valued as a good approach to motivate students.
On the other hand, considering the students ’ opinions we conclude that: We do not have to force students to correctly answer all the questions for a given level to advance forward the game. Like in jigsaw puzzles, one could leave it without finishing it Punctuations, when the activity is contained in other activities, is not considered the most important information The approach is flexible in a way that allows the teachers to design their own punctuation depending on the activity ’ s purpose. When the application is presented to the students as a game, results show that teachers should follow a design based on higher punctuations. Otherwise, if the application is presented has a formal approach to evaluate students ’ knowledge, it seems that there is not a problem to design punctuations based on traditional tests. Does not make sense having bonus when forcing the students to answer all the questions. In this line, results suggest not to force the approach and obtain the bonus only when correctly answering all the questions of the different levels. Otherwise, no extra bonus will be obtained. Hints are use as textual information or clues helping the students to find the information. Hints are not consulted because subtract points, and students preferred to ask people or search by the internet. A careful design in relation to the points related to the hints should be crucial to promote the access to the hints. Levels ’ information design depends on the purpose. Some teachers prefer to describe concrete content while other just contextualizes the questions. Feedbacks are necessary and motivating, no matter in which way are designed. These results seem to be related to the purpose of the activity, contextual characteristics (teachers motivating their students, students age, socioeconomics factors, controlled environment vs. freedom, etc.), and the teachers ’ decisions for the game design.
Gamified Learning Activities In Situ: Lessons Learnt with Teachers and Students
GAMIFIED LEARNING ACTIVITIES IN SITU:Lessons Learnt with Teachers and StudentsJavier Melero, Davinia Hernández-Leo, Josep BlatInteractive Technologies GroupUniversitat Pompeu FabraEEE Meeting – Leganés, 27-28 May 2013
OUTLINEEEE Meeting – Leganés, 27-28 May 20131. Introduction2. A conceptual model for puzzle-based games design3. Methodology4. Evaluation5. Results on teachers and students6. Conclusions and future work
INTRODUCTION3• M-learning: Situated learning activities in physical spaces• Benefits: Exploration skills and cooperation (Jeng et al. 2010)• Previous Work: “QuesTInSitu” (Santos et al. 2011)• Explorative and Spatial Skills• Foster students’ motivation and self-assessmentFurther research work involve improving students’ reflexion whenperforming situated learning activitiesEEE Meeting – Leganés, 27-28 May 2013
INTRODUCTION4How can we support more fruitful learning activities in situ thatcan be designed by teachers?• Adopted approach: A metaphor based on puzzle games• Educational Games: strengthen and support school achievement, cognitiveabilities, motivation towards learning, reflection, attention and concentration.• Puzzles: arrangement of a set of pieces into a single, well-fitting structure thatinterrelates them• Benefits:• Puzzle-based games can engage students in the subject topics, fosterstudents’ problem solving, analytical and memory skills (Huang 2007; Bottino2008).• The nature of puzzle-based games seems relevant to consider as potentialeducational strategy to feasibly involve teachers as game designers (Huang2007; Crawford 1982)Research QuestionEEE Meeting – Leganés, 27-28 May 2013
INTRODUCTION5Main objectives:• To propose a conceptual model, and its associated binding, for the design andcomputational representation of puzzle-based games• To support the teachers the creation of in situ learning activities following theconceptual model• Enactment with studentsObjectivesEEE Meeting – Leganés, 27-28 May 2013
PUZZLE-BASED GAMES6• Previous research studies aboutgame design elements that have totake into account when designingeducational games (Fisch, 2005;Jones, 1998; Kirriemuir et al., 2004;Malone, 1981; Sandford et al., 2005;Squire et al., 2003)• A 4-dimension framework,considering the role of teachers,intended to evaluate the potential ofusing games- and simulation-basedlearning (de Freitas et al., 2006)A Conceptual Model (1/2)The proposed conceptual model considers:EEE Meeting – Leganés, 27-28 May 2013
PUZZLE-BASED GAMES7A Conceptual Model (2/2)Graphical representation:EEE Meeting – Leganés, 27-28 May 2013
8We explore how can we apply the puzzle-based game metaphor in thecreation of question item-based activitiesConsidering students:• Applying the puzzle-based game metaphor in situated learning activitiesenhance the students’ learning experience• Aim: to engage students in reflecting on the correct solutions• Similar to jigsaw puzzles: players could try to solve the different questions asmany times as needed until reaching a correct solution• Ways to find the correct solutions: reflecting on wrong past choices,consulting resources provided by the gamified application, discussing withother students, asking people, searching by the Internet, etc.PUZZLE-BASED GAMESThe metaphor to design situated activities (1/2)EEE Meeting – Leganés, 27-28 May 2013
9Considering teachers:• Flexible metaphor for the design of gamified situated learning activities(independent of the subject matter)• Elements considered in the metaphor to design the (assessment) in situ learningactivities:PUZZLE-BASED GAMESThe metaphor to design situated activities (2/2)Board Geographical zones or museums roomsSlots Questions designed for the gamified learning activityPieces Options associated to each questionPuzzle Groups of slotsLevel Contains a puzzle. Levels asPoints Correct/Incorrect answers, consulting hints.Bonus Extra points when all questions from a level have been correctly answeredFeedback Information associated to ranges of pointsHints Information of help to guide students to find the correct answerEEE Meeting – Leganés, 27-28 May 2013
10• The teachers became co-participants in the design andanalysis• Involves social interactions withthe teachers:• Sharing ideas• Looking at multiple aspects ofthe design and developing• Involves different participantsin the design (researchers andteachersMETHODOLOGYDesign-based Research Methodology (1/2)DesignEnactmentAnalysisRe-design• The methodology (Barab et al., 2004; Collins et al. 1992; DBRC, 2003)involves continuous cycles of:EEE Meeting – Leganés, 27-28 May 2013
11Applying the design-based research methodology:• Context: The puzzle-based game metaphor as an approach to create gamified insitu learning activities• Participants: Teachers (designers) and students (end users) of secondaryeducation• Design: filling the templates containing the different key elements of theconceptual model• Enactment: QuesTInSitu: The Game• Analysis: Mixed evaluation method considering:• Observations, tests, questionnaires, and log files (with students)• Interviews and questionnaires (with teachers)METHODOLOGYDesign-based Research Methodology (2/2)EEE Meeting – Leganés, 27-28 May 2013
12EVALUATIONThree gamified situated learning activities (1/2)MNAC Discovering Vic Discovering l’HTeachersInvolved1 teacher 1 teacher 7 teachersPurpose Activity associated to asubject, as a learningactivityActivity associated to asubject, as part of itsformative assessmentTransversal activity in theschoolContext Learning about differentcontemporary picturesof the MNACLearning about the cityof Vic and its art history(unfamiliar city for mostof the students)Discovering and learningabout the heritage of thecity of l’Hospitalet (theirown city)Secondary teachers of different schools were involved in the design of their owngamified situated learning activities considering the puzzle-based gamemetaphor:EEE Meeting – Leganés, 27-28 May 2013
13EVALUATIONThree gamified situated learning activities (2/2)MNAC Discovering Vic Discovering l’HospitaletNumber Levels 4 levels 4 levels 10 levelsNumber Questions 20 questions 75 questions 55 questionsPoints CorrectAnswers50 points more 1 point more 250 points morePoints IncorrectAnswers10 points less 0.3 points less the first attempt, 0.5point the second one, 1 point the thirdone.100 points lessNumber Hints 19 hints 25 hints 52 hintsPoints Hints 50 points less 0.2 points less 100 points lessExtra Bonus 50 points more 1.5 points more when all the questionscorrectly answered at the first attempt,0.75 points otherwise.Proportional to the number of questionsHints Content Short text about the contextrelated to the questionShort text about the context related tothe questionSuggestions rather than clues (askpeople, read the information that appearsnext to the statue, etc.)Levels Information Short sentence of themuseums roomShort sentence of the geographical zone General information about the zone andparticular information about the questionsFeedbackMessagesInformal Formal InformalSummary of the game design task for creating situated learning activities:EEE Meeting – Leganés, 27-28 May 2013
14RESULTSMNAC Discovering Vic Discovering l’HospitaletEducationalbenefits(metaphor)Metaphor perceived asmotivating, entertaining,educationalMetaphor allowsconsolidatingknowledge and learningfrom mistakesMetaphor is stimulating,encouraging, andmotivatingApproach(answering asmany times asneeded)Useful to help studentsto reflect but difficult forthe teachers to knowthe students’ difficultiesThe teacher totallyagrees on theimportance of theapproachDifferent ratings whenasking about theimportance of theapproach*Data gathering techniques: questionnaires (once the teachers finished their gamedesign task) and interviewsSummary of results (according to the teachers’ opinions) [1/2]:* A teacher pointed out that “I find the bonus, hints and punctuation more motivating and interesting than trying and trying to reach thecorrect answer or having a free hint. It is not bad, but I find these elements dispensable”EEE Meeting – Leganés, 27-28 May 2013Teachers’ opinions (1/2)
15RESULTSTeachers’ opinions (2/2)MNAC Discovering Vic Discovering l’HospitaletElementsunderstandingNo problemsunderstanding what thedifferent elements meanNo problems understandingwhat the different elementsmeanThe “Level” element was themost problematic*. “Slot” and“pieces”, just at the beginning**Hints Hints are useful to guidestudents, but not eachquestion should haveHints are important to guidestudents in case that theyare lost or stuckHints are a good mechanism toadvance in the game, but notall the questions should haveBonus Important to keep studentsmotivatedImportant to keep studentsmotivatedImportant to keep studentsmotivatedPunctuation Important since it maystimulate students toreflect on their decisionswhen selecting an answerHighlights the importance ofdesigning adaptedpunctuation depending onthe number of wrongattemptsGood approach to allowstudents self-reflecting on theirperformance. 5 out of the 7teachers found very importantthe adapted punctuationdepending on the number ofwrong attempts.Feedback Important to students toreflect on their actionsThe teacher did not find veryimportant the feedbackFeedback are perceived asmotivating and necessarySummary of results (according to the teachers’ opinions) [2/2]:* 6 out of the 7 teachers quite or totally agreed that they had difficulties understanding what a level means** During the discussion of a meeting, some teachers argued that at first the metaphor is quite abstract, and it is needed to recall and interpretthe meanings of each element [Observer-1]; but once the elements were understood, it was easy [Observer-2]EEE Meeting – Leganés, 27-28 May 2013
16RESULTSMNAC Discovering Vic Discovering l’HospitaletFollowing themetaphor16/36 students don’t like tobe forced to answer thequestions until reaching thecorrect solution52/63 students followed themetaphor25/56 students fully followedthe metaphor18/36 students preferredhaving only one chance tosolve the questions45/63 students preferredthis approach to only havingone chance32/56 students preferred thisapproach to only having onechance28/36 students did not try todo it better in next attempts53/63 students think moreabout the possible solutionsand pay more attention36/56 students totally agreethat think better the solutionsand pay more attentionData gathering techniques: questionnaires (once the students finished the activity),observations (from researchers during the activity), log files (from the app.)Summary of results (according to the students) [1/4]:EEE Meeting – Leganés, 27-28 May 2013Students Opinions (1/4)
17RESULTSMNAC Discovering Vic Discovering l’HospitaletPunctuation 29/36 students didn’t findappropriate and (30/36)motivating the amount ofpoints when solving correctlythe questions47/63 students foundappropriate and (47/63)motivating the amount ofpoints when solving correctlythe questions45/56 students found (30/56)appropriate and motivating theamount of points when solvingcorrectly the questions14/36 students didn’t findappropriate and (14/36)discourage the amount ofpoints when solving correctlythe questions30/63 students didn’t findappropriate and (16/63)discourage the amount ofpoints when solving correctlythe questions20/56 students didn’t findappropriate and (4/56) discouragethe amount of points whensolving correctly the questionsExtra bonus 19/36 students didn’t findappropriate the amount ofpoints37/63 students foundappropriate the amount ofpoints42/56 students found appropriatethe amount of points17/36 students didn’t find itmotivating46/63 students found itmotivating31/56 students found it motivatingSummary of results (according to the students) [2/4]:EEE Meeting – Leganés, 27-28 May 2013Students Opinions (2/4)
18RESULTSMNAC Discovering Vic Discovering l’HospitaletHints No data gathered 9/14 groups ofstudents accessed tothe hints (max. 2 hints)7/14 groups of students accessedto the hints (max. 5 hints)0/18 Students foundhints useful because(0/15) help them not tobe stuck13/28 Students foundhints useful because(10/25) help them notto be stuck4/20 Students found hints usefulbecause (9/19) help them not tobe stuck3/16 students foundappropriate the amountof points subtracted14/27 students foundappropriate theamount of pointssubtracted4/19 students found appropriatethe amount of points subtracted2/20 students carefullyaccessed because of thepoints*24/28 studentscarefully accessedbecause of the points*11/20 students carefully accessedbecause of the points** Strategies followed by the students were: looking the information of the surroundings, asking people, searching by the Internet, all themembers agreeing on the answers, and divide the questions among the members of the group [data from questionnaires and observations]Summary of results (according to the students) [3/4]:EEE Meeting – Leganés, 27-28 May 2013Students Opinions (3/4)
19RESULTSStudents Opinions (4/4)MNAC Discovering Vic Discovering l’HospitaletInformation’slevels4/36 studentsfound it helpful31/63 students found ithelpful26/56 students found ithelpful17/36 students itfound unnecessary17/63 students found itunnecessary5/56 students found itunnecessaryFeedback 7/36 students itfound motivating31/63 students found itmotivating21/56 students found itmotivating16/36 students itfound unnecessary12/63 students found itunnecessary9/56 students found itunnecessarySummary of results (according to the students) [4/4]:EEE Meeting – Leganés, 27-28 May 2013
19RESULTSComparing Students’ PerformanceEEE Meeting – Leganés, 27-28 May 2013Particular case: students from the MNAC experiment using the gamified applicationvs. not using the gamified application:• Post-test containing 8 random questions already answered in the app.• Comparison of correct answer: gamified application vs. not gamified application
20The conclusions, considering the teachers’ opinions, are:• The metaphor allows the design of gamified learning activities in situ depending onthe teacher’s purpose• Concerns about having the possibility of answering several times the questions aremainly about knowing what the most problematic questions are to the students• Problems understanding the “level” element• The teachers can design their own punctuation depending on the activity’s purpose• Different approaches followed to design hints: textual information vs. clues• Results suggest that not all the questions should have associated hints• Extra bonus valued as a good approach to motivate students• Levels’ information design depends on the purpose: concrete vs. generalizeCONCLUSIONS AND FUTURE WORKConclusions (1/2)EEE Meeting – Leganés, 27-28 May 2013
21CONCLUSIONS AND FUTURE WORKConclusions (2/2)The conclusions, considering the students, are:• Not to force students to answer all the questions for a given level. Like in jigsawpuzzles, one could leave it without finishing it• Punctuation is not the most important information when situated activities are as apart of the students formative assessment• Does not make sense having bonus when forcing the students to answer all thequestions.• Hints are not consulted because students follow other strategies to find theanswers• Feedbacks are necessary and motivating• When interpreting these results, we should also consider contextual aspects(relation teachers-students, students age, socioeconomics factors, sense offreedom, etc.), and the teachers’ decisions for the game designEEE Meeting – Leganés, 27-28 May 2013
22Future work includes:• Evaluating the students’ performance on “Discovering l’Hospitalet”• Evaluating a gamified design for “Discovering Sant Sadurni” in which 7 teachershave been involved• Evaluating an authoring tool with teachers (excepted end of June)CONCLUSIONS AND FUTURE WORKFuture WorkEEE Meeting – Leganés, 27-28 May 2013