Hi everyone. So my name is Jason Lipshin and I’m a graduate student here in Comparative Media Studies at MIT. And today, I’m going to be presenting on something related, but also a bit different from what is written in the official conference agenda, mostly because I found my old topic on free labor, data mining, and Farmville to be heavy-handed and obvious. My new topic is still on the tracking of player data (and user interaction with game texts), but it’s more from a digital humanities perspective and how we can visualize that tracking and basically learn and make use of it. Hopefully you’ll like this new topic better (as I do), and I’m really looking forward to getting your feedback, as I’m not a games person – my background is more in data visualization and digital humanities.
So like all good academic conference presentations, I thought I would start off by sort of deliminating my project, defining it by saying what its not.Typically, in digital humanities (and specifically with regards to projects having to do with data visualization), there’s this tendency to fetishize data’s “bigness”, it’s scale. Lots of examples of this: READ LIST. And while I think that there is something genuinely novel and innovative about these large-scale data crunching projects that really take advantage of the affordances of computation (what Stephen Ramsay has called “machine reading”) – allowing us to ask new kinds of research questions, data visualization at a smaller, more human scale I believe gets overlooked. Also, I believe that traditionally within the roots of digital humanities (in what was once called humanities computing) there’s for one, a lot of focus on textual media and furthermore, analyzing components that are within the text itself. For example, using Google N-Grams to visualizing hundreds of years of books and analyzing word frequencies within them would be the classic example.
While I find all of this very interesting, my approach for this presentation is much more inspired by Wolfgang Iser and other’s reader response theory: this idea that each reader approaches the text in a different way, and that the reader actively co-constructs meaning with an author. Interested in using data visualization in order to understand not the text itself, but how readers/viewers/ and players interact with a media object. - Interested in filtering it through these questions: READ QUESTIONS.Finally, taking this idea of medium-specificity very seriously, I’m interested in how each data viz approach needs to shift and change given the medium. I look at three case studies tools (two of which, I worked on): Annotation Studio (text)Movie Tagger (video)Various projects of the Software Studies Initiative (games)
So the first case study is Annotation Studio, and ANS is a collaborative, web-based annotation tool currently under development at the research lab that I work at, MIT HyperStudio. Within ANS, you have a number of affordances: Collaborative Notetaking mechanism – where students can annotate any portion of a text with comments, and begin to build a discussion forum with fellow classmates in the margins. Color-coding tags – grouping annotations by themes. Visualizing interaction hotspots and even different trajectories through it.
So just to give you a little bit of a demo, here is a very typical page in Annotation Studio, which displays the text at the center and annotations on the side.
On the left, we’ve mocked up this idea of visualizing hotspots of interaction. Darker coloring signifies a greater density of annotations in this spot.
Very traditional pathway through the text – started at the beginning, plow way through it, but actually didn’t finish (which is interesting).
This reader, had a more interesting trajectory, sort of cheated and looked at the end, and then made way back to the beginning, before making her way through the text.
Reiterate the fact that no two readers are totally alike in their reading habits. And while I find this tool and the hotspot tool interesting in that they almost materializes reader response theory, they also have very practical applications.With the hotspot visualization - a teacher could flag passages with lots of interaction and know to focus on it during classtime. Data Viz folds back into curriculum, teaching methods. Generate greater dialogue between teachers and students, making the reading process more transparent for teachers. (Although, does have the potential to be abused as a kind of panopticon – did you ACTUALLY read the assignment?)
Like Annotation Studio, the MovieTagger project which I worked on at USC,sees a reader’s response to a text as extremely internally complex. And it does this through a method of time-based video tagging.So what I mean by that is, when you think of Youtube, you tag an entire video with a series of tags; but with MovieTagger, you can tag any portion of an entire film with any tag that you like, also with any in- and out-point that you so choose. And with MovieTagger, we wanted to embark on this sort of experiment in how film scholars watch film, so we had two very different film scholars, one taking a formalist approach and another informed by cultural studies and critical race theory, and see how they would tag the same film differently, and what we could sort of see in that mashup.We did a lot of experiments involving high-profile scholars…..
But the most interesting, in my opinion emerged from a “comparative reading” of the 1961 musical West Side Story.Here you can see the tags visualized for the first hour and a half of the film.
But if we look solely at the first 30 minutes….
Two very different scholars’ work overlapping – one taking a formalist approach and another informed by cultural studies and critical race theory. First scholar was interested in dance and movement of bodies, and mapping there intensity throughout the film. The second scholar was also interested in movement of bodies and intensity of movement, but more in relation to mobs, interracial conflict, and the presence of cops. Here, when you put the two together, we found that high intensity movement (dancing, mobbing) is very highly correlated to the presence of cops and what she tagged interracial conflict within the film.Interestingly, we also performed the same experiment with the film Strange Days – which is this sci-fi film which is also a comment on the Rodney King riots of 1991, and found very similar patterns there. So the take away is that even though the humans doing the tagging – messy, but nonetheless looking at and comparing the interaction patterns of two different scholars produced really interesting and telling results.
So, how does this apply to games. Taking what I’ve learned from visualizing reader and viewer interaction with texts and with films, I’m wondering about how we can apply this model to games. In fact, to a certain extent, on the one hand, data visualization of user interaction seems to be a perfect fit with games. Data viz of user interaction, also exists natively within them, as maps, various kinds of heads-up displays, inventories, etc. Here, within this game context, data viz has a various specific use – a means to self-knowledge, a kind of epistiphilia, the activity of pattern-recognition in interpreting and making sense of data viz as a way of sort of navigating complex systems. While at the same time, as Alex Galloway points out in his book Gaming: Essays on Algorithmic Culture, there’s this tenuous relationship between narrative and ludic (tired debate – not my intention to revive it here.)And finally, of course, the industry already employs many of these techniques in making use of player traces in order to understand how to make their games better. AND users often create these system to better understand their own gameplay. (maps in WoW).However, very few have approached this topic from a humanistic framework (and I’ll talk about the exceptions in a minute). Methodological difficulty: how do we track emergent behavior?
One such group that I think has done a fantastic job is the Software Studies Initiative led by LevManovich, but I want to focus more specifically on the work of William Huber, who is a PhD candidate in his lab at UCSD. Here his method: capture videos of gameplay, translate this footage into sequential series of still frames (at a rate of 2 frames per second). Specifically worked with the game, Fatal Frame II – member of the survival horror genre.)HIS GOAL: “Modes of play corresponding to clusters of operations and representations reflected in visual record of game navigation.” “Author identifies patterns of repetition and suspense.” Capturing the rhythm and tempos of gameplay.
The different modes can be seen from a macro-scale perspective. In the same spirit as MovieTagger, comparing four different players’ run-throughs (can be thought of as an INSTANCE of reception). One of which was an expert playthrough – trying to go through as quickly possible, see this in the data. White frames indicate that an enemy has been defeated – the defeat of an enemy is accompanied by a bright, white flash which fills the screen completely. What does the white cluster mean? Different Modes of Play: Cinematic Mode Navigational Mode Camera Mode Combat Mode
Saving breaks up long cut-sequences.
Similar experiment from Noah Wardrip-Fruin which is less aesthetically pleasing, more legible.
Speculating:Heatmap of Manhattan – tourists vs. locals. Polarization in the use of the space, how it is appropriated matters based on your relationship with the city (your identity). Political implications.
Map of who is going where and why.Mapping the spataliFound such maps created by users in World of Warcraft. What would it mean for game studies and/or digital humanities scholars to create such tools for analysis?
How might each of these interact differently with a game space?
And here are links to both HyperStudio’s website and the CFRP, if you are interested in learning more. Thank you so much. :)
