Seven Master of Arts students from Constance at the University of Applied Sciences Communication Design faculty will be working on design research concerning multi-touch interfaces during summer term 2008. Faces and history.

1. Touch Research
A project for Communication Design M1
HTWG Constance
Summer Term 2008
Part 1: Introduction and History

2. Introduction

3. Me, Myself, and I

4. You!

5. Participants
Dani Costa Filipe
Felix Seyfarth
Fan Zhang
Florian Roebig
Joel Lück
Laura Pabst
Liesa Maier
Stefan Herzet

6. Time and Place

7. Spaces

8. Phase 1: Inspiration

9. A little bit of history …

10. Student Presentations

11. 1. Pre-history: How did humans interact with computers between
1940 and 1970? Were there computers anyway?
2. CLI: How could one interact via a command line interface?
Disadvantages? Advantages?

12. 3. Mouse: Who invented the mouse? How did ancient mice look like?
4. Alternatives: Are there alternatives to the mouse?

13. 5. iPhone: Present the iPhone. What is special concerning HCI?
6. Bill Buxton: Present Bill Buxton’s overview of multi-touch.

14. Links

15. Konrad Zuse http://en.wikipedia.org/wiki/Konrad_Zuse
ENIAC http://en.wikipedia.org/wiki/ENIAC
CLI http://en.wikipedia.org/wiki/Command_line_interface
Douglas Engelbart http://en.wikipedia.org/wiki/Douglas_Engelbart
Finger gestures, mouse alternatives discussion on Fog Creek
Software http://discuss.fogcreek.com/joelonsoftware/default.asp?
cmd=show&ixPost=56366

16. iPhone ad http://www.apple.com/iphone/ads/ad5/
Edward Tufte’s iPhone video http://www.edwardtufte.com/bboard/
iphone-video.adp
Bill Buxton on multi-touch http://www.billbuxton.com/
multitouchOverview.html
Four multi-touch videos
http://www.youtube.com/watch?v=UcKqyn-gUbY
http://www.youtube.com/watch?v=Id43juZ3_o0
http://www.youtube.com/watch?v=v4Kl3_aU1bY
http://www.youtube.com/watch?v=ttQOQDu-IQs

17. A collection of faces/ context on a timeline …

18. 1934 – 1968
1934 Paul Otlet
1945 Vannevar Bush
1960 J.C.R. Licklider
1965 Ted Nelson
1968 Doug Engelbart

19. Paul Otlet
Information science, 1934
Wrote numerous essays on how to collect and organize the world’s
knowledge.
Founding father of documentation, the field of study now more
commonly referred to as information science.
His vision of a great network of knowledge was centered on
documents and included the notions of hyperlinks, search engines,
remote access, and social networks.

20. Vannevar Bush
Links and web, 1945
Bush proposed the notion of blocks of text joined by links and he
also introduced the terms links, linkages, trails and web to
describe his conception of textuality.
A single author connects documents that are associated by some
common theme, annotated with commentary and available for others
to read long after the original associations are made.
Bush goes on to describe the sharing of trails between people and
the creation of a new profession of trail blazers, those who find
delight in the task of establishing useful trails through the
enormous mass of common record.

21. Vannevar Bush
[Differential Analyzer, 1931
The differential analyser was a mechanical analog computer
designed to solve differential equations by integration, using
wheel-and-disc mechanisms to perform the integration. It was one
of the first advanced computing devices to be used operationally.
The analyser was invented in 1876 by James Thomson, brother of
Lord Kelvin. Lord Kelvin advised Arthur Pollen to use an analyser
for Pollen's fire-control system developed to control naval
gunnery. This electrically driven analogue computer was ready by
about 1912. Mechanical integrators for differential equations
were also designed by the Italian mathematician Ernesto Pascal in
1913.
A practical version of Thomson's differential analyzer was first
constructed by H. W. Nieman and Vannevar Bush starting in 1927 at
MIT. They published a report on the device during 1931.]

22. Vannevar Bush
Memex Machine, 1930s
Bush introduced the concept of what he called the memex (possibly
derived from "memory extension") during the 1930s, which is a
microfilm-based "device in which an individual stores all his
books, records, and communications, and which is mechanized so
that it may be consulted with exceeding speed and flexibility. It
is an enlarged intimate supplement to his memory."
He wanted the memex to behave like the "intricate web of trails
carried by the cells of the brain"; essentially, causing the
proposed device to be similar to the functions of a human brain.
It was also important that it could be easily accessible '"a
future device for individual use... a sort of mechanized private
file and library" in the shape of a desk'.
The important feature of the memex is that it ties two pieces
together. Any item can lead to another immediately.

23. Vannevar Bush
As We May Think, 1945
In the article, Bush predicted that "Wholly new forms of
encyclopedias will appear, ready made with a mesh of associative
trails running through them, ready to be dropped into the memex
and there amplified."
Bush explains how the human mind works differently that
traditional storage paradigms. For example oftentimes data is
stored alphabetically, and to retrieve it one must trace it down,
from subclass to subclass.
The brain rather, Bush explains, works by association, rather
than index, and with the brain being one of the "awe-inspiring"
phenomenon in nature one should learn from it.

24. J.C.R. Licklider
Networked computers with easy user interfaces, GUIs, 1960
Computers should be developed with the goal „to enable men and
computers to cooperate in making decisions and controlling
complex situations without inflexible dependence on predetermined
programs.“
He foresaw the need for networked computers with easy user
interfaces.
His ideas foretold of graphical computing, point-and -click
interfaces, digital libraries, e-commerce, online banking, and
software that would exist on a network and migrate to wherever it
was needed.

25. Ted Nelson
Project Xanadu, 1965
He coined the terms "hypertext" and "hypermedia" in 1963 and
published it in 1965.
However, Nelson says he dislikes the World Wide Web, XML and all
embedded markup, and regards Berners-Lee's work as a gross over-
simplification of his own work:
“HTML is precisely what we were trying to prevent — ever-breaking
links, links going outward only, quotes you can't follow to their
origins, no version management, no rights management.”

26. Douglas C. Engelbart
First hypertext system, 1960s
NLS = oN-Line System
A revolutionary computer collaboration system designed by Douglas
Engelbart and the researchers at the Augmentation Research Center
(ARC) at the Stanford Research Institute (SRI).
The first system to employ the practical use of hypertext links,
the mouse (co-invented by Engelbart and colleague Bill English),
raster-scan video monitors, information organized by relevance,
screen windowing, presentation programs, and other modern
computing concepts.
Features: The mouse, 2-dimensional display editing, in-file
object addressing, linking, hypermedia, outline processing,
flexible view control, multiple windows, cross-file editing,
integrated hypermedia email, hypermedia publishing, document
version control, shared-screen teleconferencing, computer-aided
meetings, formatting directives, context-sensitive help,
distributed client-server architecture, uniform command syntax,
universal "user interface" front-end module, multi-tool
integration, grammar-driven command language interpreter,
protocols for virtual terminals, remote procedure call protocols,
compilable "Command Meta Language"

27. Douglas C. Engelbart
The mother of all demos at the Fall Joint Computer Conference in
San Francisco, December 9, 1968
The demo featured the first computer mouse the public had ever
seen, as well as introducing interactive text, video
conferencing, teleconferencing, email, hypertext and a
collaborative real-time editor.
Engelbart, with the help of his geographically distributed team,
demonstrated the workings of the NLS (which stood for oNLine
System) to the 1,000 computer professionals in attendance.

28. Douglas C. Engelbart
Magnification of the first computer mouse, 1963, co-invented with
Bill English
Engelbart never received any royalties for it, as his patent ran
out before it became widely used in personal computers.

29. State of the Internet
The ARPANET, 1969 – 1971
Initially, on October 29, 1969, the ARPANET consisted of four
nodes (IMPs, interface message processors)
1. SDS Sigma 7 at UCLA
2. SDS 940 running NLS at Stanford
3. IBM 360/75 running OS/MVT at UCSB
4. DEC PDP-10 running TENEX at Utah
The first message transmitted over the ARPANET was sent by UCLA
student programmer Charley Kline, at 10:30 p.m, on October 29,
1969. The message text was the word "login"; the "l" and the "o"
letters were transmitted, but the system then crashed. Hence, the
literal first message over the ARPANET was "lo". About an hour
later, having recovered from the crash, the SDS Sigma 7 computer
effected a full "login".
The contents of the first e-mail transmission in 1971 have been
forgotten; in the Frequently Asked Questions section of his Web
site, the sender, Ray Tomlinson, who sent the message between two
computers sitting side-by-side, claims that the contents were
"entirely forgettable, and I have, therefore, forgotten them",
and speculates that the message likely was "QWERTYUIOP" or some
such.

30. 1976 – 1991
1976 Richard Saul Wurman
1981 Don Norman
1987 Bill Atkinson
1991 Tim Berners-Lee

31. Richard Saul Wurman
Information architecture, 1976
“I thought the explosion of data needed an architecture, needed a
series of systems, needed systemic design, a series of
performance criteria to measure it.”
[TED Talks, 1984
“I did the TED conferences out of my own boredom, and the only
ones that seemed interested were those in the technology
business, entertainment industry and design profession. What
astonished me is that they didn't realize they were all in one
business.”]

32. State of the Internet
The ARPANET, 1977

33. Don Norman
The trouble with UNIX, 1981
He is an expert of cognitive science and is widely considered to
be the first to apply advanced human factors to design via
cognitive design.

34. Don Norman
User centered design, 1986
The Psychology (Design) of Everyday Things.
User-centered design can be characterized as a multi-stage
problem solving process that not only requires designers to
analyze and foresee how users are likely to use an interface, but
also to test the validity of their assumptions with regards to
user behavior in real world tests with actual users. Such testing
is necessary as it is often very difficult for the designers of
an interface to understand intuitively what a first-time user of
their design experiences, and what each user's learning curve may
look like.
The chief difference from other interface design philosophies is
that user-centered design tries to optimize the user interface
around how people can, want, or need to work, rather than forcing
the users to change how they work to accommodate the software
developers' approach.

35. Bill Atkinson
HyperCard, 1987
One the first successful hypermedia systems before the World Wide
Web.
HyperCard is one of the first products that made use of and
popularized the hypertext concept to a large popular base of
users.
Atkinson later lamented that if he had only realized the power of
network-oriented stacks, instead of focusing on local stacks on a
single machine, HyperCard could have become the first Web
browser.
HyperCard saw a loss in popularity with the growth of the World
Wide Web, since the Web could handle and deliver data in much the
same way as HyperCard without being limited to files on one's own
hard disk. HyperCard had a significant impact on the web as it
inspired the creation of both HTTP itself (through its influence
on Tim Berners-Lee's colleague Robert Cailliau, and JavaScript
(whose creator, Brendan Eich, was inspired by HyperTalk).
The pointing-finger cursor used for navigating stacks later found
its way into the first web browsers, as the hyperlink cursor.

36. Tim Berners-Lee
World Wide Web/ HTTP/ Web browser
While an independent contractor at CERN from June to December
1980, Berners-Lee proposed a project based on the concept of
hypertext, to facilitate sharing and updating information among
researchers. While there, he built a prototype system named
ENQUIRE.
In 1989, CERN was the largest Internet node in Europe, and
Berners-Lee saw an opportunity to join hypertext with the
Internet: "I just had to take the hypertext idea and connect it
to the Transmission Control Protocol and domain name system ideas
and — ta-da! — the World Wide Web.” He wrote his initial proposal
in March 1989, and in 1990, with the help of Robert Cailliau,
produced a revision which was accepted by his manager, Mike
Sendall.
He used similar ideas to those underlying the Enquire system to
create the World Wide Web, for which he designed and built the
first Web browser, which also functioned as an editor
(WorldWideWeb, running on the NeXTSTEP operating system), and the
first Web server, CERN HTTPd (short for HyperText Transfer
Protocol daemon).
The first Web site built was at CERN, and was first put on line
on 6 August 1991: http://info.cern.ch/hypertext/WWW/
TheProject.html.

37. Tim Berners-Lee
The first Web pages and Web browser WorldWideWeb

38. Tim Berners-Lee
The first Web server, running on Steve Jobs’ NeXTSTEP operating
system.

39. 1994 – 2005
1994 Jakob Nielsen
1994 Marc Andreessen, Eric J. Bina, Jim Clark

40. Jakob Nielsen
Web usability, 1994
From 1994 to 1998 he was a Sun Microsystems Distinguished
Engineer. He was hired to make heavy-duty enterprise software
easier to use, since large-scale applications had been the focus
of most of his projects at the phone company and IBM.
But luckily the job definition of a Distinguished Engineer is
"you're supposed to be the world's leading expert in your field,
so you figure out what would be most important for the company
for you to work on." Therefore, Dr. Nielsen ended up spending
most of his time at Sun defining the emerging field of Web
usability. He was the usability lead for several design rounds of
Sun's website and intranet (SunWeb), including the original
SunWeb design in 1994.
Principles:
Clear orientation
Comprehensible navigation
Need oriented entry points
Consistent user guidance

41. Marc Andreessen
The first user-friendly Web browser Mosaic with Eric J. Bina at
the NCSA, 1994
After his graduation from the university in 1993, Andreessen
moved to California to work at Enterprise Integration
Technologies. Andreessen then met with Jim Clark, the recently-
departed founder of Silicon Graphics. Clark believed that the
Mosaic browser had great commercial possibilities and suggested
starting an Internet software company.

42. Eric J. Bina
The first user-friendly Web browser Mosaic at the NCSA with Marc
Andreessen, 1994

43. Jim Clark
The first commercial Web browser with Marc Andreessen (as
Netscape Communications), the Netscape Navigator, based on
Mosaic, 1994
Soon Mosaic Communications Corporation was in business in
Mountain View, California, with Andreessen as cofounder and vice
president of technology. The University of Illinois was unhappy
with the company's use of the Mosaic name, so Mosaic
Communications changed its name to Netscape Communications, and
its flagship web browser was the Netscape Navigator.

44. 2008
The mobile Web is coming.

45. 2007
The 1st generation iPhone.
Multi-touch …

46. Phase 2: Human Computer Interaction (HCI)

47. Overview

48. Curricula for Human-Computer Interaction: http://sigchi.org/cdg/
cdg2.html
by the ACM SIGCHI, 1992
Human-computer interaction is a discipline concerned with the
design, evaluation and implementation of interactive computing
systems for human use and with the study of major phenomena
surrounding them

49. HCI Area H1: Psychology?
Psychology is the analytic and scientific study of mental
processes and behavior.
Psychology -> cognitive psychology -> information processing

50. Context/ Environment Versus User
Situation: Software interface or hypertext information system
Action: Behavior or interaction
Cognitive psychology is a discipline within psychology that
investigates the internal mental processes of thought such as
visual processing, memory, thinking, learning, feeling, problem
solving, and language.
Within the field of cognitive psychology, information processing
is an approach to the goal of understanding human thinking. It
arose in the 1940s and 1950s. The essence of the approach is to
see cognition as being essentially computational in nature, with
mind being the software and the brain being the hardware.

51. Human Software and Hardware
H1 Human information processing (human thinking as a software
running in the human brain)
H2 Language, communication and interaction (encoding and
decoding)
Computer Hardware and Software
C1 Input and output devices
C2 - C5 Software running on underlying computer hardware

52. The Human User
Interaction is a response to a situation:
Situation -> action

53. Three Levels of Interaction
According to Don Norman:
Visceral/ affective level: Situation -> action
Behavioral level: Situation -> black box -> action
Cognitive level: Situation -> cognitive system -> action

54. In a cognitive system, we distinguish …
Perception
Spoken language
Text
Images
Cerebration
Terms
Reasoning
Decision making
Problem solving
Learning
Acquisition
Memory
Behavior
Motor activity
Speech

55. Situation Action Circle
The action/ behavior feeds back on the perception of the
situation …

56. … and is dependant on a background of
Motivation
“Man, I’m not willing to understand that cryptic ticket form.”
Emotion
“I love those colors!”
Social context
“I an new to this whole Internet stuff.”
Physiology
“I should have had a breakfast.”

57. HCI Areas C2-C5: Computer Software!

58. Computer Software
C2 Dialogue Techniques
C3 Dialogue Genre
C4 Computer Graphics
C5 Dialogue Architecture
Details will be provided in the next session.

59. Software interface vs hypertext system
by Jesse James Garret, 2000
Task-oriented vs information-oriented
1. User needs
2. Site objectives
3. Functional specification vs content requirements
4. Interaction design vs information architecture
5. Information design
6. Interface design vs navigation design
7. Visual design

60. Hypertext system – information oriented – from conception to
completion
1. User needs: Externally derived goals for the site; identified
through user research, ethno/ techno/ psychographics, etc.
2. Site objectives: Business, creative, or other internally
derived goals for the site
3. Content requirements: Definition of content elements required
in the site in order to meet user needs
4. Information architecture: Structural design of the information
space to facilitate intuitive access to content
5. Information design: In the Tuftean sense – designing the
presentation of information to facilitate understanding
6. Navigation design: Design of interface elements to facilitate
the user’s movement through the information architecture
7. Visual design: Visual treatment of text, graphic page elements
and navigational components

61. Software interface – task oriented – from abstract to concrete
1. User needs: Externally derived goals for the site; identified
through user research, ethno/ techno/ psychographics, etc.
2. Site objectives: Business, creative, or other internally
derived goals for the site
3. Functional specifications: Feature set – detailed descriptions
of functionality the site must include in order to meet user
needs
4. Interaction design: Development of application flows to
facilitate user tasks; defining how the user interacts with
site functionality
5. Information design: In the Tuftean sense – designing the
presentation of information to facilitate understanding
6. Interface design: As in traditional HCI – design of interface
elements to facilitate user interaction with functionality
7. Visual design: Graphic treatment of interface elements; the
“look” in “look-and-feel”

62. Goal for the Next Session

63. Touch Tisch -> Touch Research

64. Credits

65. Via Creative Commons on flickr:
Libraryman www.flickr.com/libraryman/718450202/
Photocapy www.flickr.com/photocapy/250355140/
Philippe Tarbouriech www.flickr.com/phitar/7201140/
Dezz www.flickr.com/residae/2560241604/
Joe www.flickr.com/joesflickr/711358450/
Mike Bitzenhofer www.flickr.com/bitzcelt/1450900070/
Jim Crossley www.flickr.com/raindog/532177285/
Matt Locke www.flickr.com/matlocktest/37349112/
Javier Martínez www.flickr.com/hyoga/1165367241/
mac steve/ Michael N www.flickr.com/mac/18590268/
Jim Grisanzio www.flickr.com/jimgris/65769319/
Robert Scoble www.flickr.com/scobleizer/2256358640/
Kit Cowan www.flickr.com/kitcowan/712113879/
Thomas Lieser www.flickr.com/onkel_wart/2377883376/
Cheon Fong Liew www.flickr.com/liewcf/894035077/
CSSA @UCSD www.flickr.com/cssa_ucsd/150160784/
Jordan Fischer www.flickr.com/jordanfischer/61429449/
sparktography/ Sparky www.flickr.com/sparktography/374064022/

66. Via Creative Commons on flickr:
Mathieu Thouvenin www.flickr.com/mathoov/2429735842/
Maurizio D'Arrigo www.flickr.com/keylosa/184606430/

67. ACM SIGCHI Curricula for Human-Computer Interaction:
http://sigchi.org/cdg/cdg2.html
Original Print Media: Copyright © 1992 by the Association for
Computing Machinery, Inc.
Web Version: Copyright © 1996 by the Association for Computing
Machinery, Inc.
All the rest:
http://wikpedia.org
University of Applied Sciences Constance, Faculty for
Communication Design, Project “Touch Research”:
http://www.htwg-konstanz.de
http://www.kd.fh-konstanz.de/dina8/daten_e.php?wodenn=will
http://www.felgner.ch/2008/04/touch_research.html