Download to read offline
Uploaded byClay Ewing
Hypothesis Tested: Designing Games with Theoretical Frameworks
The document discusses the development and evaluation of various social impact games, including tabletop games that address health insurance markets and skin cancer prevention. It highlights both positive and negative outcomes of these games, emphasizing the complexities of measuring their effectiveness in changing attitudes and behaviors. The author suggests that the integration of behavioral models and character-driven narratives may enhance future interventions in public health messaging.
Technologyā¦
More Related Content
Similar to Hypothesis Tested: Designing Games with Theoretical Frameworks
![Getting Started with Apache Spark: Big Data Made Simple [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/apachesparkgettingstarted-260203175547-8361bcc3-thumbnail.jpg?width=640&height=640&fit=bounds)
Hypothesis Tested: Designing Games with Theoretical Frameworks
- 1. HYPOTHESIS TESTED Designing SocialImpact Games with Theoretical Frameworks Clay Ewing @claytical c.ewing@miami.edu Associate Professor University of Miami
- 3. HUSTLING HEALTH CARE (2010) Atabletop game that simulated health insurance markets before the Affordable Care Act.
- 4. TANZANIAN SOCIAL ACTION FUND(2012) Working with the Tanzanian government and the World Bank, we developed a facilitated tabletop game intended to teach rural farmers about a new conditional cash transfer program.
- 5. ā ā IF YOU CANMEASURE IT, THEN ITāS NOT THE CHANGE I WANT TO SEEā¦ IF YOUR GAME OR TECHNOLOGY REALLY āWORKSā IT FREAKS ME OUT Paolo Pedercini, Making Games in a F****d Up World , Games for Change 2014 Pedercini, P. (2014) Making Games in a F****d Up World, Games for Change 2014. Retrieved from https://www.youtube.com/watch?v=MflkwKt7tl4
- 6. VANITY A tabletop gamewhere players take on the role of aspiring actors competing at auditions. The game aims to teach teenagers about the dangers of using tanning beds. Preliminary results are inconclusive but promising. The game may be an acceptable preventative tool to keep people from engaging in tanning behavior that could lead to skin cancer. Tran, Carcioppolo, Colantonio, Ewing, Lang, Beyea, Beecker. āThe Empty Comfort of Vanity: Assessing the Effectiveness of an Interactive Game to Increase Skin Cancer Prevention Outcomesā (GLS10)
- 7. UNSAVORY (2015) You playas a minimum wage worker in a fast casual restaurant preparing burritos. Your goal is to survive a month on the job. Eventually, your work is interrupted by an oncoming sneeze. If you sneeze, your boss knows youāre sick and sends you home. If you miss too much work, youāre fired and wonāt be able to pay your bills.
- 8. ā ā AFTER I ANALYZEDTHE RESULTS FROM THIS STUDY, I WAS DISMAYED TO FIND THAT PLAYING THE GAME HAD NO EFFECT ON POSITIVE FEELINGS TOWARD THE POOR. IN FACT, THE GAME HAD A NEGATIVE EFFECT ON ATTITUDES AMONG CERTAIN PARTICIPANTSāINCLUDING SOME PEOPLE WHO WERE SYMPATHETIC TO THE POOR TO BEGIN WITH. Gina Roussos, When Good Intentions Go Awry, Psychology Today Roussos, G., & Dovidio, J. F. (2016). Playing below the poverty line: Investigating an online game as a way to reduce prejudice toward the poor. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 10(2). Roussos, G. (2015). When Good Intentions Go Awry: The Counterintuitive effects of a prosocial online game. Psychology Today. Retrieved from https://www.psychologytoday.com/us/blog/sound-science-sound- policy/201512/when-good-intentions-go-awry
- 9. ā ā WHEN IMPACT ISDEFINED TOO NARROWLY, SOME GAMES ARE DISMISSED FOR THE WRONG REASONS AND THEIR IMPACT IS OVERLOOKED. Impact with Games: A Fragmented Field Stokes, B., OāShea, G., Walden, N., Nasso, F., Mariutto, G., Hill, A., & Burak, A. (2016). IMPACT WITH GAMES: A FRAGMENTED FIELD. Pittsburgh, PA: ETC Press. Retrieved from http://gameimpact.net/reports/fragmented- field/
- 10. Integrated Model ofBehavioral Prediction: Fishbein, 2000, 2007; Fishbein & Yzer, 2003 Health Belief Model: Rosenstock, 1974; Rosenstock, Strecher & Becker, 1988
- 11. POTENTIAL SUITORS Anti TanningNeutral Pro Tanning For each tanning belief, we created 3 personality types that would cover different combinations of tanning beliefs and life values.
- 12.
- 13. IN GAME TANNINGOFFERS Tanning Salon Dermatologist
- 14. THE EPILOGUE The finalpiece of the game has an anti- tanning health message implicitly presented. It gets shown when the character went to a tanning salon or they chose to date someone with a history of using tanning beds.
- 15. STUDY RESULTS The gamewas the most effective intervention at reducing intent to tan with women and people who havenāt indoor tanned previously. However, they did not enjoy the game as much as the other interventions. Kim, S., Zhou, C., Carcioppolo, N., Ewing, C. (2018). Rethinking the Role of Engagement and Enjoyment: The Effect of a Digital Health Game on Indoor Tanning Attitude and Intention. Kentucky Conference on Health Communication. Retrieved from https://comm.uky.edu/kchc/abstract/768.
- 16. STUDY RESULTS There wasa boomerang effect! Men and people who have indoor tanned previously enjoyed the game and are more likely to tan after playing the game. Kim, S., Zhou, C., Carcioppolo, N., Ewing, C. (2018). Rethinking the Role of Engagement and Enjoyment: The Effect of a Digital Health Game on Indoor Tanning Attitude and Intention. Kentucky Conference on Health Communication. Retrieved from https://comm.uky.edu/kchc/abstract/768.
- 17. STUDY RESULTS We assumedincreasing engagement would have a positive effect but the increase in variables can decrease the persuasive effects of the game. Kim, S., Zhou, C., Carcioppolo, N., Ewing, C. (2018). Rethinking the Role of Engagement and Enjoyment: The Effect of a Digital Health Game on Indoor Tanning Attitude and Intention. Kentucky Conference on Health Communication. Retrieved from https://comm.uky.edu/kchc/abstract/768.
- 18. STUDY RESULTS In traditionalinterventions, increased transportation leads to better outcomes. In the game condition, the players were transported but it had a negative effect. We canāt make a conclusion on this yet. Kim, S., Zhou, C., Carcioppolo, N., Ewing, C. (2018). Rethinking the Role of Engagement and Enjoyment: The Effect of a Digital Health Game on Indoor Tanning Attitude and Intention. Kentucky Conference on Health Communication. Retrieved from https://comm.uky.edu/kchc/abstract/768.
- 19. It would beirresponsible to publish the game knowing what we found during the first study
- 20. When we figureout how to eliminate the boomerang effect, the game will be a solid health intervention tool.
- 21. THE ROAD AHEAD ā¢Limited character versions that focus on one category (tanning vs. anti-tanning) ā¢ Deep dive into specific character messages and player behaviors
- 22. DREAMY ā NONTANNING EDITION The idea of interacting with a series of characters with the pretext of getting a date is a pathway for other public health interventions.
- 23. THANK YOU! Clay Ewing(@claytical) c.ewing@miami.edu http://www.claytical.com Interactive Media Program http://interactive.miami.edu NERDLab http://nerdlab.miami
Editor's Notes
- #2Ā CONTACTS: Clay Ewing c.ewing@miami.edu Dr. Soyoon Kim soyoon.kim@miami.edu This project was funded by the School of Communication at the University of Miami (http://com.Miami.edu) and the Center for Communication, Culture, and Change (http://com.miami.edu/center-for-ccc) Integrated Belief Model: https://www.sibson.com/media/1536/using-theory-to-design-effective-health-behavior-interventions.pdf Health Belief Model: http://foresttwpseniorcenter.com/5695732-the-health-belief-model-and-personal-health-behavior.pdf
- #3Ā Hello! Iām Clay Ewing and Iām an associate professor at the University of Miami. Iām the Co-Director of NERDLab, where students and faculty work together on social impact projects that are primarily game based. Most of the student members of NERDLab come from our MFA program in Interactive Media. At the School of Communication, we also offer a PhD in Communication. Our PhD program has a strong health communication influence, so it was only a matter of time before we started working together on evidence based games for public health. Today Iām going to talk about a game I designed and developed with my colleague Dr. Soyoon Kim. Dr. Kim is a health communication researcher and together we worked to integrate theoretical frameworks into gameplay in order to change tanning intentions. But before I get to that, I want to step back in time to provide a bit of context for my relationship with measuring impact and how my experience evaluating games has evolved.
- #4Ā In 2010, I was here in New York at Parsons finishing up my MFA in Design and Technology. My thesis project was Hustling Health Care, a board game that spoke to the debate about health insurance before the Affordable Care Act had been passed. My feeling at the time was fairly simple. If you presented players with an argument through a system, they would understand it better. If you used real world data, as I was doing, a player would come away with hard facts as well. And if the game was engaging enough, it could perhaps spark a deeper conversation. The game was playtested thoroughly. It was definitely engaging. I can remember Nick Fortugno coming by after this picture was taken and saying something to the effect of āIf you can keep 4 high school students playing your game for 2 hours in the middle of Playtech, you have succeeded.ā If youāre not familiar with Playtech, itās an annual event held at Parsons where a bunch of kids are brought in to playtest a wide variety of games and experiences in development. Itās basically a big free arcade.
- #5Ā Hustling Health Care pushed me towards social impact game design as a career. When my partner Lien Tran and I were asked to design a game for the World Bank and the Tanzanian government, it felt like we were moving towards something. The Tanzanian government had created the Tanzanian Social Action Fund with the objective of enabling poor households to increase incomes and opportunities while improving consumption. The game we designed set out to teach the TASAF staff about the program. It was well received and the staff suggested that the game could be modified to teach the community about the program as well. When it was piloted, the game had a debrief that reinforced ideas and concepts as well as asking about engagement and enjoyment. In certain contexts, this would be a sufficient evaluation, but the for study Iām focusing on today, the scope is much more specific.
- #6Ā The thing that drives me to make social impact games is that they have the possibility of making a difference. When Paolo Pedercini commented in 2014 at G4C āIf you can measure it, then itās not the change I want to seeā¦ if your game or technology really works, it freaks me outā, I felt a bit of āYeah, butā¦ā Weaponizing games as a propaganda is scary, but just as film and radio continue to be used effectively, itās inevitable that games will be used as well. Understanding the way in which they can persuade us is important otherwise weāre bound to be manipulated. Also, we use film and TV to deliver persuasive arguments for prosocial purposes. A good example would be East Los High on Hulu. Embedded in East Los are educational narratives that promote sexual and reproductive health. It makes sense to do something like this with games as well.
- #7Ā At the same time we were developing the game for TASAF, we started working on a tabletop game funded by the Canadian Dermatology Association called Vanity. The game sought to prevent use of indoor tanning beds, particularly at an early age. We designed the game with a dermatologist as the content expert. There was no evaluation plan but as designers we felt that working with content experts would ensure the game was accurate and provide good information. When we started working with researchers at UM, Vanity was selected to evaluate from a health communication perspective. Our preliminary results were promising, but collecting data has been incredibly slow as the game requires a minimum of 3 players and more than an hour of play.
- #8Ā Thinking about positive change and previous projects, I wanted to make something with a much greater reach. I had been working an app with ROC United on a Diners guide for ethical eating and one of the things that struck me was that most restaurant workers donāt receive paid sick days. Following my feeling with Hustling Health Care, I sought to create a game that advocated for restaurant workers by allowing the player to walk in their shoes for a month, working a restaurant making minimum wage and getting sick. But thenā¦
- #9Ā I was here at Games for Change and during her presentation, Colleen Macklin mentioned this article in Psychology Today. [READ QUOTE] The article is about a game called Spent. Is anyone familiar with Spent? Iām a big fan of Spent, I love it using it as an example in my design classes. So hearing that a well designed game had no effect on positive feelings toward the poor was interesting. But the fact that some participants who were sympathetic to the poor beforehand had a negative effect on attitude was troublesome. It freaked me out a bit. Since that Psychology Today article was published, the full study has been published in the journal Cyberpsychology. As an FYI, one of the key takeaways was that playing the game wasnāt effective at positive change, but watching someone else play it was.
- #10Ā I think that we often have trouble with evaluating games because we debate what is considered impact and what it means to properly evaluate a game. Educational games are probably the most straight forward as itās pretty easy for the designers and researchers understand the test results before and after. Increases in test scores means that the game is effective at teaching a subject. With games for public health and communication, youāre dealing with intentions and attitudes. Through work with our colleagues on the research side, we were finding that games with a narrative focus seemed to align well with theoretical frameworks in communication as where games focused more on play were a different beast.
- #11Ā The games research group in our school came to the conclusion that a tanning intervention game was a good idea but as a tabletop requiring multiple players, it was requiring too much time to collect data. So now that you have a bit of context from my previous work, letās dive into Dreamy. Dreamy is a single player dating simulator. Itās also digital. As the first game at the University of Miamiās School of Communication to be developed with a researcher from day one, we thought it was a great opportunity to match existing theoretical frameworks to the game mechanics. We chose two behavior prediction models: Fishbeinās integrated model of behavioral prediction and Rosenstockās health belief model. The game mechanics target salient beliefs and attitudes to determine behavioral intention to engage in tanning behavior and discourage indoor tanning.
- #12Ā The game has 9 conversation trees based on a 3x3 matrix of character combinations: outgoing, health conscious, and average paired with their attitude on tanning: anti, neutral and pro. In case youāre wondering, the way the each potential suitor looks is randomly generated. At the beginning of the game, a friendly guide asks about their gender preference to find matches. They can choose men, women or say that love has no boundaries. Based on this preference, the game generates the suitor avatars. Once the player starts a conversation with one of these characters, they wonāt be able to talk to any other characters with tanning beliefs until they fail with the current conversation tree.
- #13Ā You might also be wondering how all the characters end up talking about tanning. As the different characters have specific reasons for tanning or not tanning, we attempted to build conversations that would naturally end up mentioning tanning. There are dead end characters that never talk about tanning but the player can never get a date with them. In early playtests, we allowed the player to talk to multiple tanning suitors and while players didnāt think āThis must be a tanning interventionā they did think the designers had some hangups about tanning. In this example, the player is talking to a character that is pro-tanning. You can see with the potential replies that the player is able to challenge the characterās beliefs with āI donāt think tanning beds are supposed to be used like that. Or at all.ā Or they can agree with the character choosing āIāve never thought of a tanning bed as being so corrective! Thatās smart!ā
- #14Ā In the game, the player is invited to visit a tanning salon to improve their physical attractiveness. The playerās avatar can burn and even get moles. If they choose to go to the tanning salon, a dermatologist sends them a message offering a free check up. The dermatologist gives accurate information regarding tanning beds and skin cancer.
- #15Ā To win the game, the player has to navigate successfully through a characterās conversation tree to get a date. If they date someone with a tanning history or engage in tanning behavior, their epilogue changes. There are a series of outcomes for both the player and their love interest that include having a brush with death due to skin cancer and in some cases, dying from skin cancer.
- #16Ā So what did we find out? Our target demographic was young women, so in that aspect, the game can be said to have a positive impact because it reduced their intent to tan. They really didnāt enjoy the game though.
- #17Ā Men and people who have indoor tanned enjoyed the game. However, they were more likely to tan after playing the game. Thatās the exact opposite of what we wanted.
- #18Ā We went really deep on data collection and intervention design. The results Iām sharing today are from one study but we will have quite a bit more as the research team analyzes all of the data. Itās also a problem figuring out what specifically caused the boomerang effect because there are many variables: how many characters they interacted with and which characters they interacted with, along with marketing offers, the tanning salon and the dermatologist. There are 117 health specific messages alone from just the suitors in the game.
- #19Ā Transportation refers the feeling of being so absorbed in a story that the connection to the real world is lost for some time. One assumption that weāve had for games is that increased transportation is a good thing. Weāre not sure about this and we need more data, specifically limiting the amount of health messages the player receives. During the development process, Chun Zhou, one of our doctoral students, decided to use the game for her dissertation. She created a modified version to study interactivity. In the game, youāre able to construct your avatar, so she created a version that removed the avatar creation process as well as streamlined the conversations so that players choices were predetermined. Her initial findings have shown that story based interactivity is more effective than character based interactivity.
- #20Ā After working on Dreamy, Iāve been thinking a lot about impact and how we talk about games for change. Dreamy was definitely impactful. By playing the game, it will change a personās intent to tan. But not always in the right direction. The most important takeaway for me is thinking deeply about failure. We tend to celebrate and talk about games that either feel like theyāre creating impact or are proven to have impact, but we shy away from telling people about the ones that didnāt come out the way we expected and might have unintended consequences and effects. We need to discuss our failures more.
- #21Ā The exciting part is that we can iterate and figure out what works. We can also analyze the things that didnāt work as expected and try to figure out what happened so we donāt do it again.
- #22Ā Our first step in figuring out the boomerang effect is to simplify the game and to do this weāre going to create multiple versions that focus on specific categories. For example, would our results change if we removed all pro-tanning characters? What if the game pushed every player to tan but then had a dermatologist intervene?
- #23Ā For the designers in the room, you probably know what Iām talking about when I say that working with researchers can setup some hurdles. One of the bigger ones is finding common language. Iāve found that sometimes, envisioning a game through a design brief isnāt enough for a researcher to wrap their head around an idea. But prototyping a game just to explain an idea is a big ask of the developers. Itās much easier now to talk about using the skeleton of Dreamy where players talk to suitors with embedded health messages to think about other possible interventions.
- #24Ā Iām a designer and Iām always happy to talk shop. Although if you are interested in the research side, Iām happy to connect you to my colleagues for further information regarding the nitty gritty of the studies. And just in case youāre interested in more information on our MFA program and our lab, here are the URLs. If there are any questions/comments and we have time, fire away. Thank you!