Make It So: What Interaction Designers can Learn from ScienceFiction InterfacesPresentation Notes, Nathan Shedroff and Chris Noessel4 September 2009, dConstruct 09 Conference, Brighton, UKThis is the first presentation of only a portion of the material weve found in our analysis ofScience Fiction films and television series. We!re also looking a industry future films (like ApplesKnowledge Navigator) as well as existing products and research projects. Our analysis includesproperties (films and TV), themes (different issues in interface design), as well as the historicalcontext of the work (such as the current technology of the time of the property!s release). Inaddition, we!re interviewing developers (including production designers from films) but thismaterial isn!t presented in this talk. For this presentation, we!ve focused on the major issues, partacademic and theoretical, and part lessons (more practical) we!ve uncovered.How design influences SciFi and how SciFi influences design:Weve chosen to focus on interface and interaction design (and not technology or engineering).Some visual design issues relate but, mostly, in this talk, we!re not approaching issues of styling.We!ve chosen the media of SciFi (TV and films) because a thorough analysis of interactiondesign in SciFi requires that the example be visual so interfaces are completely and concretelyrepresented, include motion that describe the interaction, and (sometimes) has been seen by awide audience.Scientifically determining “influence” in any context (whether from Design on SciFi or visa versa)is difficult, and much of what we illustrate is inference on the part of the authors.
Design influences SciFi: Design (reality) sets the paradigm that scifi builds uponIn turn, Sci Fi influences design in one of four ways: Inspiration, Expectation, Social Context, andProposed Paradigm.Science Fiction relies on the context set by the design and development of existing products. Thisallows SciFi to communicate effectively with its audience. For example, during the Industrial Age,when the first SciFi film was created, Le Voyage Dane La Lune, the metaphors used wereindustrial and had no interfaces. To open the door, you pushed the door open. While thisprincipally reflected the whimsical and theatrical nature of the film, it also reflected a lack of“interfaces” in the world at the time.
By the time Fritz Lang directed Metropolis, we see an attempt to realistically present a vision ofthe future that builds upon the audiences awareness of the telegraph, tickertape, radio, andtelephone technologiesAs depicted in the scene, to control the screen, the user “checks messages” via a tickertape,“tunes” in the video like a radio, and talks via “telephone” to his employee.However, by the time Buck Rogers appears on screen, the audience is now familiar withelectronic television, allowing SciFi interfaces to resemble the screens, dials, and interaction ofthis medium. Interestingly, the characters, must leave the room to talk to the ship through a radiointerface, though, since “talking” through your television wasn!t seen as a possibility.By the 1980s, the personal computer is now common, and though the director of Jurassic Parkmust make some references to specific interactions (such as the mouse and file system),audiences are now able to understand and accept this medium, allowing the young girl to savethe day.
Inspiration: Viewers like what they see and seek to replicate it in real worldIn the 1990s, when Douglas Caldwell, with the U.S. Army Topographic Engineers, saw a 3D mapsystem in the film, X-Men, he realized it was a novel solution to an age-old challenge: how torepresent any relevant topography in 3D in the battlefield.His department!s request for proposals for development of something similar yielded the XenotranMark II Dynamic Sand Table, including the improvement of a smooth surface and overheadprojection.Lesson: Sci Fi is a powerful cultural influence. It affects designers ideas as well as those of ourclients and audiencesExpectation: Viewers see things in SciFi they begin to expect. Expectations are set whentechnologies are shown as desirable for their form or their function.Wheres my jetpack?Function: Robots are a SciFi staple and have influenced the spending of millions, if not billions ofdollars on human-like machines that, effectively, create an ethics-free slave class. This, despitethe fact that robots with clear industrial use don!t require any human reference points inrepresentation, behavior, or movement.
GORT from The Day the Earth Stood Still Delivery robot from I, Robot Honda!s ASIMOForm: What audiences experience influences the desired forms and expectations of new devices(they!ve seen before in SciFi). It also influences engineers and developers who are, often, SciFifans themselves. Consider the juxtaposition of Star Trek!s communicator with the MotorolaStarTac exactly 30 years later:
Lesson: Users may already be predisposed to certain interface solutions solely based on whatthey!ve seen in the media. These solutions may be more comfortable for them than otheralternatives.Social Context: Reminders of limitations and constraints of who we are as people, who werelate to each other and how we relate to technologyAnthropomorphism influences expectations of human behavior and sentience. This plays-out inseveral ways.Microsoft!s “Clippy,” “BOB,” and Ms. Dewey (msdewey.com)Most of the time, when designers flirt with anthropomorphized elements in the interface, it isn!tsuccessful. Either the technology or feature doesn!t live-up to the expectations suggested by therepresentation or the “character!s” behavior is just annoying. The mechanisms that allowanthropomorphism to work are well defined and researched (see the work of Nass and Reeves,
and their book The Media Equation) and legitimate. However, simply adding a human (or animalor alien) character is a surface treatment that doesn!t relate to the system!s behavior.There are examples of where this is done, effectively. For example, Apple!s future scenario,Knowledge Navigator, uses an anthropomorphized character in the computer to provide arealistic level of function with a minimum of annoying behavior.Lesson: Though its been clearly shown that socially appropriate interfaces can aid learning,speed use, and make more comfortable interfaces (Nass and Reeves), when this is merely as avisual adornment, it often fails.Lesson: Wading into the social context is more tricky than it looksA fully human-like representation isn!t necessary to see the anthropomorphistic effect. Sound,alone, can often suggest the expectations of human-ness or sentience. In the case of K.I.T.T., thetalking car in the TV series Knight Rider, almost the entirety of this effect is accomplished throughK.I.T.T.!s voice.Likewise, audiences infer greater capabilities to the Enterprise!s computer system in the TVseries Star Trek because of the quality of human-like representation in the voice.
In the Star Wars films, one of the most endearing characters in the franchise, R2-D2, is not ableto speak at all and his behaviors and movements are limited. Instead, his sounds are enough foraudiences to relate to and assume human-like sentience and feeling.Lesson: Sound is enough to trigger anthropomorphism.Human-like sound and visual appearance isn!t even necessary for anthropomorphism to takeplace. Behavior is often the mechanism for expectations of human-comparable function andbehavior to exist. For example, Amazon!s OneCLick™ is devoid of any animate references yetfunctions much like a favorite shop keeper or bar tender would (welcome you back, getting youyour “usual,” trusting your choices, and handling the “details” of your transaction). Amazon!s OneClick™Lesson: Anthropomorphism works through behavior as well as form.The degree of representation often affects the degree of expectations of human-likeunderstanding and behavior in the system. The director of Until the End of the World could have
simply represented a computer!s search function as a standard computer element. But, by givingit a character and animate representation, it inferred greater capability than a standard service. Bounty Bear from Until the End of the WorldLikewise, in the film Matrix, system functions (programs) are represented as fully humancharacters, imparting greater impact, depth, and danger to the audience than a standard programrepresentation would. Therefore, the hunt and destroy program, represented as Agent Smith, isfelt as more dangerous and capable than one would expect form a program.Likewise, the prediction program, The Oracle, seems more powerful and accurate represented asa person than we would expect from a collection of “code.”Lesson: The more "human" the representation, the higher the expectations of behavior. Whenwe cast technologies or processes in anthropomorphic ways, it raises expectations about theextent of their behavior. In SciFi this doesnt, necessarily, have to only be human (animals, aliens,etc.) but the effect is the same.
Proposed Paradigm: We can use SciFi as a kind of scenario to evaluate theinterface/interaction and find potential solutions and lessons.SciFI often serves as a simple reminder of constraints and affordances (and their role) that wealready understand. For example, in The Fifth Element, we find affordances taking the place of amanual for a device that was last used thousands of years ago.Likewise, we know the frustration and confusion caused by interfaces without clear affordances orconstraints, as in the humorous “interface play” Lifted.Lesson: Designing constraints and affordances into the interface helps users quickly understandan interface and get by without the manual.We!ve also found examples of bad solutions, such as when input devices should recognize theaffective state of their users and adjust their behavior and reaction accordingly. In The FifthElement, a choking character mashes at his technological desks keyboard, triggering all sorts ofinappropriate actions.Instead, an example of the system ignoring identical inappropriate or ambiguous input can befound in 2001, where a child!s bashing of the teleconferencing system!s buttons doesn!t interferewith the call.
Lesson: Input devices should recognize the affective state of their users and adjust accordingly.This is especially important for some users (such as children and seniors), some situations (suchas emergencies or critical/dangerous functions), and some contexts (like mobile or portabledevices vs. desktop or ubiquitous, stationary ones).We!ve found surprising cases where paradigms suggest solutions to interface problems. Theseexamples allow us to see what solutions may work in improving situations—and even test them.In Star Wars, there is almost always a correlation between social hierarchy and the size ofrepresented users in holography. For example, the social hierarchy of the “Empire” is nearlyalways represented during holographic conversations by scaling the projections to representsocial status. In this example, though he is one of the highest leaders in the Empire, Darth Vaderis still dwarfed by the projection of the Emperor, his superior. In other scenes, when Darth Vaderaddresses his subordinates, he is almost always projected bigger or higher than them.In contrast, the Jedi council!s social hierarchy is one of equality. So, when they meet with mixedpresence, even the older and more revered characters, are all portrayed at the same scale.Lesson: Whenever there is a social context in the interface (which is often these days), whetherit is a social network, video chat, community site, buddy list, etc.), social hierarchy is engaged(and might be appropriately represented). Our systems can make this hierarchy apparent whenthis is useful through scale.
"Sometimes, the lessons are new and particular to the given technology. This is when Science Fiction most acts as a prototype for the designers to learn new things about it. In the case of Minority Report, the famous gestural interface is mostly presented as a powerful, deeply engaging tool for manipulating and investigating a huge amount of video and related data. The film bypasses the problems inherent in such systems such as fatigue, but does make a nod to another problem. When Anderton is introduced to Witwer during a scrubbing session, he turns to shake hands, he inadvertently wipes all of the content from his screen. This small bit of interface humor points out that users of future systems will need some explicit way to engage and disengage the system, enabling them to do other things with their bodies, such as attend to social or biological needs."Lesson: What works for audiences often works for users as well.Last Lesson: Watching Science Fiction can be beneficial to your career!
ConclusionsThis is only the beginning. Theres a lot more lessons were learning and a lot more properties toexplore.Look for a book on this material sometime next year and check back at the website for moreupdates: scifiinterfaces.comSend questions and inquiries to Nathan Shedroff (Nathan@nathan.com) and Chris Noessel(firstname.lastname@example.org).