Katelyn Procci                                           Daniel Brown USS Rivet City:                           Clint Bowe...
02 | USS Rivet City            Modding – An Introduction               Modding: Adding new content to existing games, some...
03 | USS Rivet City            Modding – Two Uses               1) Participatory Learning Experiences                   o ...
04 | USS Rivet City              USS Rivet City: The Problem        Interpreting a Bullseye          o    Line 1: Door Acc...
05 | USS Rivet City“Rivet City is an aircraft carrier. Let’s just mod that.”            Fallout 3 - Rivet City Concept Art...
06 | USS Rivet City “USS Rivet City”                     USS Oriskany                                   Comparing Battlesh...
07 | USS Rivet CityStep 1: Plan1) Have a plan with defined goals     o Clearly state your objectives2) Determine what tool...
08 | USS Rivet CityStep 1: Plan (continued)3) Explore your options     o   Creation Kit (Skyrim – Bethesda Softworks)     ...
09 | USS Rivet CityStep 2: Develop
10 | USS Rivet CityStep 2: Develop (continued)1) Set a timeline, and then give yourself double that time    o These tools ...
11 | USS Rivet CityStep 3: Test1) User testing, user testing, user testing!    o Will help you catch problems early-on!2) ...
12 | USS Rivet CityUSS Rivet City: Lessons Learned1) Define your goals2) Determine your needs3) Pick the most suitable gam...
13 | USS Rivet City       -[ UCF RETRO Lab | 407-823-5335 | www.ucfretrolab.org ]-Institute for Simulation and Training / ...
Upcoming SlideShare
Loading in …5
×

#G4C12: USS Rivet City: Lessons Learning from Modding Fallout 3 for Serious Games Research

861 views

Published on

With a little creativity, a mod is a cost-effective way to develop both training and testing materials for serious games research. In this short talk, the development process of modding Fallout 3 to create a testbed for shipboard navigation skills is discussed. Lessons learned from our successes and failures will be highlighted.

PARTICIPANTS:
Katelyn Procci, Research Coordinator and Project Manager, UCF RETRO Lab

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
861
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Hello everyone, my name is Katelyn Procci and I am the research coordinator and project manager of the RETRO Lab, a serious games research and development organization with the University of Central Florida’s Department of Psychology and the Institute for Simulation and Training in Orlando. It is most certainly an honor to speak to you today. I am here to discuss our project where we attempted to mod Fallout 3 to create a training session and testbed for shipboard navigation skills. I'm not here to talk about the project itself, but instead the process. The ability to mod games is a powerful tool for anyone who would like to make a game, but doesn't have the resources to do so. It's something we can all do. I would like to share with you what I learned about modding games so that you may learn from my experience and apply it to your own development projects.
  • But first, what are mods? Mods are definitely my most favorite part of PC games. When I am talking about modding here, I mean making use of software tools provided by the developer to change a game to add in original content – quests, maps, skins, you name it. Modding games certainly prolongs the shelf-life of a game, increasing replay and making them more commercially viable. User communities generate new missions and content for games, and mods themselves can become games. Half-Life mods gave us Team Fortress Classic, Day of Defeat, and Counter-Strike, a franchise which continues to be a commercial success. Mods encourage innovation, and provide useful feedback to the developers themselves. Nevermind that really great modders can be hired, so they can turn their passion into a career. The utility of modding transcends benefits to the gaming industry. Modding tools provide invaluable resources for those interested in creating game-based experiences, but don't have a lot of money or personnel to do so.
  • It seems that modding can serve us two purposes. One is providing a participatory learning experience. By learning how to mod a game, students will have to take content, understand it, and build with it. You are taking a problem, discovering the underlying knowledge needed to solve it, and applying it correctly in context. There are several successful case studies, and think of all of the secondary benefits - learning to work in teams, learning better communication skills, getting those who otherwise wouldn't be interested in computer science engaged from an early age. The other purpose is providing inexpensive, but incredibly powerful tools to build games that we can use to inform, educate, and make change. Going way back, creating simulations from scratch has always been prohibitively expensive. One very, very early example from the late 1980s of how this was first addressed was the Intelligent Maintenance Training System (IMTS, Towne & Munro 1987; 1991). As a basic simulation, it is an intelligent tutoring system that focuses on maintenance training. It is important because it was one of the first to include rudimentary authoring tools. The authoring tools allowed individuals to create libraries of objects, link those objects together, and to create runable simulations, rather than spending an enormous amount of time and money to build these simulations from scratch every time. Games, as we know, are also expensive to develop from scratch, which is exactly why game engines exist! Beyond that, modding tools are more accessible to those with less programming experience and for teams that maybe don’t even have their own artists. So, there is a great potential in modding existing games, especially for small teams, such as my own lab, that operate under very tiny budgets, to create new and effective gaming experiences.
  • And so, my story begins. Over the past few years, our lab was been funded by the Office of Naval Research. Working under Raytheon BBN Technologies, we helped to validate the VESSEL Damage Control Trainer, or DCT, which was developed to train US Navy recruits damage control procedures – essentially, what do to when you’re on a sinking ship in order to save your own life and the lives of others. BBN managed game development while we focused on research.  Our most difficult problem was how to best train shipboard navigation skills using virtual environments. US Navy ships are all laid out on a grid-system made up of numbers that corresponds to location in terms of deck, frame, and location from the center line. Spaces, compartments, hallways, you name it, conform to this system and are labeled with placards with this numerical address, or bullseyes. It is a very confusing system to new recruits. They just don’t get it. We tried our best to figure out how to train it. We tried a point-and-click flash game, but our studies revealed that no one learned anything from it (Procci, Blair, & Bowers, 2010). So, one of my labmates and I had to brief our lab on everything we could find about training navigation skills and figure out how we could better train the recruits using a game. Everything we found had to do with wayfinding by specific markers in the environment. While this system was the same across ships, the ship interiors themselves would change. This didn't help us much. We started to brainstorm – how could you create a game that forces you to make use of your knowledge of how a system works. How can you learn to apply a system to navigation consistently, even when the physical environment changes? Further, we needed something that could serve as both training and as a evaluation tool -- something important for those interested in the study of serious games. After 30 minutes of beating our heads against the wall, one of our researchers blurted out, “Rivet City is an aircraft carrier. Let’s just mod that.”
  • So, Bethesda gave us Fallout 3. One of the major in-game locations, Rivet City, is a large US Navy ship modeled in-part after a World War II aircraft carrier, the USS Oriskany.
  • Fallout 3 comes with an extensive editor, the Garden of Eden Creation Kit, or GECK. It was perfect. Rivet City is a big ol’ confusing ship and we could mod it by putting the placards over every door in accordance to the navigation system and then build quests into the game that required players to go to those doors. With our very small amount of funds, we hired Dan Brown for three months, who was then an undergraduate at UCF. He had a programming background, knew how to photoshop, and was very interested in modding and game design.  Our team was made up of the two of us. Our project, codenamed USS Rivet City, was broken up into three major phases: planning, development, and testing.
  • We knew that we could change the interior, we knew that we could make unique quests. Together, we planned something simple, that taught the individual how to move around and then gave them commands through intercoms to go to room to room, similar to a roving security task that could take place on a real US Navy ship. So far, so good. But, importantly, at this stage we had a whole lot of ideas, but we had to be sure: First, have a plan with a defined goalSet down your objectives first and design your mod accordingly Secondly, determine the tools and capabilities you really need to meet this goal. You must know what you are working with so you don’t design too big. We knew we wanted to change the interiors and we wanted quests and, eventually NPC interactions. All of this could be achieved with the GECK. So, my team and I brainstormed what we thought would be important elements that different mods may need to better serve those interested in making their own games: Dialogue - Are you allowed to interact with NPCs and hold conversations?Levels/Quests - Are there quests / can you add new ones to create missions?Assets - Are you able to edit and create new objects, textures, skins?Library - How extensive is the available library? Does it fit your needs?Environment - Can you change the physical environment? What about interiors and exteriors?Support - Is there community that you can use for help? For each of these, you need to think about whether you want to be able to change what exists, or if it already aligns with your specific needs.
  • Many games come with level editors and construction kits. For you, we've selected a few newer games (newer because they’re more likely to have advanced editing options and active community support) that may be fairly promising for creating new serious games: Creation Kit (Skyrim – Bethesda) - http://www.creationkit.com/Hammer SDK (Source engine games - Valve) - https://developer.valvesoftware.com/wiki/SDK_DocsElectron toolset (Neverwinter Nights 2 – Obsidian Entertainment) - http://www.nwn2toolset.dayjo.org/WC3 World Editor – (Warcraft 3 – Blizzard) - http://world-editor-tutorials.thehelper.net/Dragon Age Toolset – (Dragon Age - Bioware) - http://social.bioware.com/page/da-toolset For example, the Creation Kit for Skyrim allows for dialogue editing options, quest design, access to all of their enormous asset library, dungeon design, and a scripting language, Papyrus, which allows you to create your own events. There is a ton of community support.  So maybe you’d like to make a mod to raise awareness of environmental change would require a way to trigger changes in the physical environment so the player can see the results of their actions. You could probably do something really creative using the WC3 World Editor.  With such detailed editors and incredible support from both developers and the modding community, what you can create pretty much whatever you want. My advice? If you are low on people, time, and money, really think about the kinds of assets in these games. We didn't have an artist, so modding a ship that already existed was perfect for us. But, if you can hire an artist, then you can create whatever objects you need and then focus on capabilities (quests, dialogue, etc.) rather than art constraints.
  • We ended up stripping everything out the entire ship, replaced particularly damaged set pieces, doubled the number of lights, added an extra level, placed placards above doors using existing sign models within Rivet City itself, and embedded intercoms into the environment as a part of the quest we built. The GECK comes with a quest viewer that breaks down each objective and dialogue option. All Dan had to do was create a new quest option and link it to a trigger area.  During development, we faced some surprising challenges. These tools are all somewhat complicated, because what you can do with them is complex. Things crashed, things took a while to figure out.  
  • So, I asked Dan to give you all some advice on modding, both in general and using the GECK: Plan a timeline, and then double it. These tools have a learning curve, and things will go wrong. This iteration of the software was unstable, so it crashed a lot (but we’ve heard things have been fixed since)Also, when I asked him what he would’ve done differently, he stated less time shaping the ship and more time on getting the quest to workWhen I asked Dan what was awesome about the GECK, his answer was the community behind it. You are NOT alone. Trial and error is a great way to learn, but chances are, someone else has had the same problem that you are having and now knows how to fix it. He got stuck for a week trying to figure out how to get different placards to show up on the door. He took to the forums and got great responses.
  • The Fallout 3 mod was used in a pilot study conducted last summer. The purpose of the study was to examine how well shipboard navigation skills transferred from one virtual learning environment to another. Essentially, was our mod a good test bed? So what did we find? Neither conditions that underwent training (one trained using the DCT, the other played TF2) differed on quiz about bullseyes and there were no differences in the times it took the participants to navigate to each of the locations presented in the mod. So it was a bust. It wasn't a great test-bed, even though it replicated what a real environment would be like. Why? Pulling back from the data, I went to my research assistants. Participants simply were stuck in the mod. Even though they just had 30 minutes of practice using WASD+mouse controls before playing our mod, most participants spent their time running into corners, staring at ceilings, and walking awkwardly into walls. It seems we had made the most rookie mistake possible -- we failed at user-centered design in our mod.  Make use of user testing at some point. Iterative user testing, beginning early in the design process, would have alerted us to this problem earlier and we may have been able to fix it. This is a critical methodological issue for serious games research at universities -- we often don't have access to our target population and have to make the best of participant pools, usually made up of first-year undergraduate students taking a psychology class as a part of their gen-ed requirement. This is a problem for both design and development in that both usability and validation testing samples what is potentially the wrong population. I have no idea if our mod would've worked with recruits because didn't have access to them. Build it into your contract, plan on collecting, beg, do what you need to get real user input, even for a mod. Which leads me to my second point... Even if you do have access to members of your target population: Your players will not always be gamers, but the game you're modding was likely made for a very different population that is already familiar with games. The interface may be confusing, the controls may be difficult to master to someone who is, essentially, a noob.Provide more training - build a better tutorial into your mod, make it a part of gameplay, ease the player into the action.Simplify gameplay and the interface, if possible.Provide alternative control inputs. Maybe, because the corridors were so tight in our mod, those not great at using the keyboard for moving the player-character would have fared better using an X-Box 360 controller  So, as an warning, when you are building mods to be used as a serious game, consider the level of gaming knowledge your players have and do your best to develop the mod accordingly. If this means taking extra time to build tutorials, it will be worth the extra time in the end. Had we done this, we may have gotten different results.
  • So, if you are a small dev team, with not a lot of people, time, or money, and you want to make a game, to either encourage change or to contribute to the science of serious games, please, consider undertaking a modding project. It seems daunting, but it’s not that scary, as long as you keep the following in mind:Define your goalsDetermine your needsPick the most suitable game/toolsGive yourself more time than you think you needAsk the community for helpMake use of early and iterative user testingDesign with your player's level of gaming familiarity in mind 
  • Thank you!
  • #G4C12: USS Rivet City: Lessons Learning from Modding Fallout 3 for Serious Games Research

    1. 1. Katelyn Procci Daniel Brown USS Rivet City: Clint Bowers Lessons Learned UCF RETRO Lab from Modding Fallout 3 for Serious Games Recent and Emerging Technology Research Organization Research Dr. Janis Cannon-Bowers, Co-Founder Dr. Clint Bowers, Co-Founder www.ucfretrolab.orgGames for Change | June 20, 2012
    2. 2. 02 | USS Rivet City Modding – An Introduction Modding: Adding new content to existing games, sometimes through developer- provided tools. Modding will*: o Prolong shelf-life o Increase replay value o Become a commercial success! o Promote innovation o Provide feedback o Help gamers break into the game industry o http://www.bit-tech.net/news/gaming/2008/07/17/valve-hires-minevra-mod-maker/1 o http://www.insidegamingdaily.com/2012/02/29/mojang-hires-bukkit-modders-to-create- new-minecraft-api/ Modding tools are an invaluable resource for those who want to create a serious game, but do not have an experienced dev team.* Kücklich, J. (2005). Precarious playbour: Modders and the digital games industry. The Fibreculter Journal, 5. http://five.fibreculturejournal.org/fcj-025-precarious-playbour-modders-and-the-digital-games-industry/
    3. 3. 03 | USS Rivet City Modding – Two Uses 1) Participatory Learning Experiences o Take content, build a mod o Learn about the content, learn about programming o Effective! 2) Tools to Create Entirely New Experiences o Games are prohibitively expensive for small teams to develop from scratch o Authoring tools provide opportunities for those with less experience to create effective gaming experiences Developer-provided modding tools make serious game development accessible to everyone!e.g. El-Nasr, M. S., & Smith, B. K. (2006). Learning through game modding. ACM Computers in Entertainment, 4(1), article 3B.; Learn with Portals; Towne, D. M., &Munro, A. (1987). The intelligent maintenance training system. SCS Simulators Conference.
    4. 4. 04 | USS Rivet City USS Rivet City: The Problem Interpreting a Bullseye o Line 1: Door Access Number o Line 2: Compartment Name o Line 3: Compartment Number 1-58-2 Repair Locker 1 1-50-2-Q Deck – Frame – Centerline – Classification This repair locker is on the main deck at the 50th frame and is the first compartment on the port side of the ship.Procci, K., Blair, L., & Bowers, C. (2010, October). Cognitive load and the role fidelity in training games: Lessons learned from Bullseye. Poster presented atMeaningful Play 2010, East Lansing, MI.
    5. 5. 05 | USS Rivet City“Rivet City is an aircraft carrier. Let’s just mod that.” Fallout 3 - Rivet City Concept Art Craig Mullins, Bethesda Softworks
    6. 6. 06 | USS Rivet City “USS Rivet City” USS Oriskany Comparing Battleship NorthU.S. Navy National Museum of Naval Aviation (1996.488.196.049) Carolina and Rivet City
    7. 7. 07 | USS Rivet CityStep 1: Plan1) Have a plan with defined goals o Clearly state your objectives2) Determine what tools and capabilities you need o Pick a game and tool that best aligns with your goals o Know the limitations of the tool you choose o Dialogue - Are you allowed to interact with NPCs and hold conversations? o Levels/Quests - Are there quests you can edit, can you add new ones to create unique missions? o Assets - Are you able to edit and create new objects, textures, skins? How extensive is the library and what is already in it? o Environment - Can you change the physical environment? o Support - Is there community that you can draw upon for help?
    8. 8. 08 | USS Rivet CityStep 1: Plan (continued)3) Explore your options o Creation Kit (Skyrim – Bethesda Softworks) o http://www.creationkit.com/ o Hammer SDK (Source engine games - Valve) o https://developer.valvesoftware.com/wiki/SDK_Docs o Electron Toolset (Neverwinter Nights 2 – Obsidian Entertainment) o http://www.nwn2toolset.dayjo.org/ o WC3 World Editor – (Warcraft 3 – Blizzard Entertainment) o http://world-editor-tutorials.thehelper.net/ o Dragon Age Toolset – (Dragon Age - BioWare) o http://social.bioware.com/page/da-toolset
    9. 9. 09 | USS Rivet CityStep 2: Develop
    10. 10. 10 | USS Rivet CityStep 2: Develop (continued)1) Set a timeline, and then give yourself double that time o These tools have a learning curve o Things will go wrong! o You will want to tweak and refine2) Ask the community for help o What was awesome about the GECK?: “The community behind it. Theyre incredibly helpful and knowledgeable.”
    11. 11. 11 | USS Rivet CityStep 3: Test1) User testing, user testing, user testing! o Will help you catch problems early-on!2) Understand that your player may not be a gamer, but thegame you’re modding was probably meant to be played byone o Provide training o Tutorials, practice, in-game hand-holding, etc. o Simplify o Provide alternate inputs o Maybe a controller would’ve helped…
    12. 12. 12 | USS Rivet CityUSS Rivet City: Lessons Learned1) Define your goals2) Determine your needs3) Pick the most suitable game/tools4) Give yourself more time than you think you need5) Ask the community for help6) Make use of early and iterative user testing, even for “just a mod”7) Design with your players level of gaming familiarity in mind
    13. 13. 13 | USS Rivet City -[ UCF RETRO Lab | 407-823-5335 | www.ucfretrolab.org ]-Institute for Simulation and Training / UCF Department of Psychology

    ×