Nico Ludwig, profile picture
Uploaded byNico Ludwig
469 views

(1) gui history_of_interactivity

The document explores the history of interactivity in computing, detailing the evolution from early data processing using punch cards and light bulbs to more advanced user interfaces involving keyboards, screens, and graphical displays. It highlights key developments such as the transition from batch processing to real-time interactivity, the introduction of command-line interfaces, and significant applications like VisiCalc that changed user perceptions of computers. The discussion also covers technological advancements in display types, moving from vector to raster displays, which enabled richer graphical interactions.

Embed presentation

Download to read offline
1 Nico Ludwig (@ersatzteilchen) (1) GUI – History of Interactivity
2 TOC ● (1) GUI – History of Interactivity – Wires and Bulbs – Batch Processing – Terminal and Mainframe – From the Command-Line to Killer Applications – Vector Displays and Raster Displays – Color Displays – The Mouse and the Takeoff of Interactivity – The Desktop Metaphor ● Sources: – Wikipedia: ● Bubba73, ASR 33; Shieldforyoureyes/Dave Fisher, BM-3279; Oliver.obj, Panel 370-145; Boffy b, IBM PC 5150; Rama & Musée Bolo, Apple II-IMG 4217; SRI International, SRI Computer Mouse – Video series "Triumph of the Nerds" – Matthew MacDonald, Pro .NET 2.0 Windows Forms and Custom Controls in C#, Appendix A – Charles Petzold, Code – The Hidden Language of Computer Hardware and Software, Chapter 25 – other sources are public domain
3 The Beginning – Data Processing ● Basically we'll discuss the user interface (ui), which is the place where computer and human meet. ● In the early days of computing, computing was a job for white collar workers. – In the 1940ies computers were used by switching switches and plugging cables. – The state of the switches and the wiring represented data and program "code". – The results were displayed with an array of lightbulbs. – Using and programming was usually the same thing. ● In the 1950ies, 1960ies and early 1970ies batch processing was applied. – Program code and data were punched into cards or tapes ("punched cards/tapes"), which were interpreted by the computer. – The results have been printed on paper. – The whole input was processed as one batch of punched cards. Input Processing Output Process input in batch Batch of punched cards (data and program code) Print the output to paper Directionofthe processing ● The graphic shows an ENIAC (Electronic Numerical Integrator And Computer) (1947 – 1955). ● Punched cards and tapes have also been used for organs and looms.
4 Interactivity raises – Having a Session with a Computer ● Somewhen in the 60ies computers learned to interact with humans. (interact, from lat.: inter = "between", agere = "to act") – Er, or, … somewhen in the 60ies humans learned to interact with computers. ● Interaction means that human input and computer response happen multiply within the samesession. – The computer and the means of input/output were separated devices connected by a network. – The user inputs data via a teletypewriter (tty) and the computer responded via the tty as well. The tty was the so called "terminal". – So user input can directly follow the last computer output, but not in parallel. This makes the interaction. ● The terminal prompts for input, the user sends his input, after it is processed by the computer it is resend to the terminal and another prompt appears. – The communication is based on transmission of streams of serial data following the American Standard Code for Information Interchange (ASCII) protocol. – The whole session goes only into one direction, it can be traced along the tty's paper roll. Input Processing Output Input Processing Output Input ... computer session Directionofthesession Input & output/ tty computer terminal network computer ● I.e. ttys were used as peripheral device and terminal. ● Input and output of ttys was just text, so it is also called "text terminal" (German: "Schreibmaschinenterminal"). ● Because a paper roll was used to trace the procedure, it was also called "hard-copy terminal". ● A tty is a printer and a keyboard. ● Prompt from lat. promptare: "to publish something". ● There survived some terms from this era of computing: ● ASCII, control characters: LF, CR ● The "TTY", as oldest serial interface, is still available as terminal interface (shell) and to program industry plants.
5 Interactivity raises – Keyboards and Screens ● In the 70ies, separated peripherals for input (the keyboard) and output (cathode ray tube (CRT) screens) were introduced. – Keyboard and CRT acted as terminal together, other terms: "glass ttys", Video Display Units (VDU), Video Display Terminals (VDT). ● In opposite to the term "hard-copy terminal" for the tty, which basically was a printer more than an input device. ● At this time in history two aspects are relevant to understand: – Still the "tty-idea" is used here! VDUs just represented rolls of paper, it was just a "line by line" communication. – VDUs were connected to a remote computer via a network. As being just terminals, VDUs had no intelligence at all! – => This arrangement consists of a "dumb terminal" (has little logic) and an "intelligent computer", the mainframe (has a CPU). Output/ CRT Input/keyboard computer dump terminal network remote, intelligent computer (mainframe) ● "Line-by-line" communication means that we have no cursor control, but following control codes are mandatory: CR, LF and Bell. ● OS' like MS DOS and C/PM uses/used monitors only that ways. ● By a matter of fact today we still use this metaphor in diverse command line interfaces/shells. ● Monitors have the potential to interact with the user by applying the whole two dimensional space of the screen. ASCII was already equipped to be extended with control codes handling two dimensional character devices (clean the whole screen or move the cursor to the left upper corner). ● Before ASCII there was the Extended Binary Coded Decimals Interchange Code (EBCDIC). EBCDIC was used by IBM's mainframes in the 1950ies and early 1960ies. ● EBCDIC's character codes were not arranged sequentially in alphabetic order (i.e. there were gaps), but optimized for punched card processing. For modern applications ASCII was much more useful, because of its alphabetic structure. ● Mainframe, German: Großrechner ● The shown terminal is an IBM-3279 (~1972) a green screen terminal ● The shown mainframe is an IBM System/370-145 (~1970) (the system console). ● We don't talk about applications running on the mainframe, which were basically databases and financial processing software. The large units to store all the data are not shown (the compatible storage unit is the IBM 3850 with magnetic tape cartridges). ● The terms "dumb terminal" or "thin client" have other meanings today.
6 Terminals and Mainframes? – What the heck...? ● One moment! Why was this kind of arrangement with ttys/VDUs and mainframes and a network in between necessary? – Because of the problems, which were solved with computers that time! Read on: ● In the beginning, computers were basically used for financial and military purposes. – Starting in the 1960ies, people understood how computers are able to process masses of data, esp. in business. – This problem domain required holding and managing huge amounts of data! The problem domain drove two developments: ● (1) Mass storage hardware, which holds data. ● (2) Database software, which manages data. ● In the end there are three (four) individual devices, connected by network/wire: – ((0) User.) (1) Terminal. (2) Mainframe. (3) Mass storage device. – It was required, because of technical possibilities that time and because folks just clung to the idea of a "big computer room". Output/ CRT Input/keyboard computer terminal network mainframe database's data mass storage device ● (To be frank in the late 60ies also more intelligent terminals with some microprocessor power and some memory to handle editing operations (insert a character), without sending/re-sending the whole screen to the computer were available)
7 The Evolution of the Command-Line ● With terminals and mainframes the command-line was the major means of interactivity. There were some historical steps: ● The bare command-line age: – Full control, the user could freely use all tools in any order. – No guidance, no structure, no metaphor, commands needed to be remembered by the user. ● The command-line prompt age: – No control, the user needs to follow an application's usage paradigms. A program could only do a single task. – Full guidance, each step is prompted, we had an interview-metaphor (question/answer). ● The menu age: (Let's call a program "application".) – Compromise of control, the user was able to do a set of tasks. An application was able to do multiple tasks. – Ergonomic guidance, menus are a compromise of prompt and free selection of the next action. We have a document-metaphor.
8 Two dimensional Interactivity – Killer Applications ● Its time to give up working with line-by-line communication. – For terminals cursor control was not mandatory, only these control codes were required: CR, LF and Bell – But: screens have the potential to break free line-by-line and use their full two dimensional space. ● ASCII was already ready to be extended with control codes for two dimensional character devices (clean screen, move cursor to left upper corner, etc.). ● In 1979 VisiCorp presented an application that used the screen in a two dimensional manner:VisiCalc. – VisiCalc was a spreadsheet application using the screen as a table for numerical calculation. It brought following new aspects: ● The cursor could now be moved among columns and rows! ● The screen followed a metaphor. The "spreadsheet"-metaphor mimics a chalkboard or pencil/paper on a computer screen. ● The application had menus to guide the user and applied another metaphor: the "document"-metaphor. ● VisiCalc was the first killer application (app) for home computers! – A killer app is an app that significantly changes the attitude of users on using a technology. A killer app is a "game changer". – This killer app proved that one can do useful things with a computer (here: "number crunching"). ● VisiCalc was the breakthrough for the Apple II. – It showed that the Apple II really had a business use! ● The Apple II, the first PC, needed an important application (a "killer app") to be a success: VisiCalc (designed by Dan Bricklin at the "Harvard Business School" in 1979). ● Another way to characterize a killer app: vendors and developers use it as reference. "Let's do it like it is done in VisiCalc." ● VisiCalc was also the killer app on Apple II's competitor "the IBM PC" (IBM model 5150 in 1981). ● With VisiCalc the transition of the computer from a hobby object to a business object started. ● The spreadsheet concept was, however, not patented by VisiCorp and it was copied by Microsoft (Multiplan, Excel), Lotus Development Corporation ((belongs to IBM today) Lotus 1-2-3)). ● Also text processors which were then partially replaced by word processors featuring WYSIWYG.
9 Two dimensional Interactivity – Hey Terminal, good Riddance! ● Apple II? Where did the mainframe go? ● There is a very important point: an app like VisiCalc can't be brought to the dumb terminal/mainframe arrangement: – Fast screen updates as reaction on user input are required, and no extra communication via a slow network. – Memory went short, because usually terminals (the better ones) could only store a few lines of text up to page. ● From terminals to Personal Computers (PC). PCs are standalone computers. – Basically, the intelligence of computer and terminal were integrated into one unit. This unit was called Personal Computer (PC). – Apple (!) was the first manufacturer of PCs for consumers: the Apple II reached the market in 1977 (at 1298$). ● Apple made computers white goods (like vacuum cleaners or microwaves are). ● Speed is important to make interactivity work acceptably! – (1) PCs directly accessed the video memory without the OS or the network. ● Dumb terminals had only means for character generation in the hardware. In opposite to some PCs even having a graphics adapter. – (2) PCs had their own memory (4KB and more). ● In parallel to the development of two dimensional interactivity thedevelopment of video display evolved... ● Virtually IBM "slept" until the early 1980ies, then they understand the meaning of PCs. They released the IBM-PC (IBM 5150) in 1981, four years after Apple's Apple II. ● PC today: IBM-compatible (IBM compatible – read: esp. the interface to video memory is standardized) This "openness" was the key to make software independent from hardware. Apple II and IBM PC were expandable "in the box" by attaching new adapters (also from 3rd party vendors) to the so called mainboard. ● Homecomputers could be connected to the TV, but early IBM-PCs could also be connected to the TV. A better way to categorize homecomputers is that they are closed in opposite to the openness of IBM compatible computers. - In the world of a homecomputer everything is proprietary (connectors and internal adapters, one could buy extensions or replacement parts only from the manufacturer of the homecomputer).
10 Video Display – Vector Displays ● To understand the next steps on the way to interactivity, the further development of video displays has to be understood. – Up to now we only discussed systems using the terminal screen, often only with a two-color palette (e.g. a "green screen"). – The most important step was from character displays to graphical displays. ● The first graphical displays for computing were so called vector displays. – Vector displays are CRT displays, in which an electron ray (emitted by an electron gun) directly draws objects composed of primitives on the screen. ● This is similar to a human drawing on a canvas. Also characters are composed from primitives. ● Examples: radar screens, oszilloscopes and the first primitive computer games. ● Vector displays brought an interesting idea of interaction: thelight pen. – Light pens also brought a new killer app: "Sketchpad" (Ph.D. thesis Ivan Sutherland, 1962/1963). – Sketchpad was the first interactive graphical app, it introduced Computer Aided Design (CAD). ● But these vector displays needed to update the drawing in short intervals. – The time needed to redraw the display limited the amount of content (and complexity) that could be displayed. canvas electron gun vector display screen CRT electron beam ● The Apple II had up to 15 colors. ● Lightpen: ● A lightpen detects the beam emitted by the electron gun. Lightpens are also usable for CRTs. ● Afterwards, lightpens were used to fill out forms on the screens (e.g. ticking check boxes or for simple multiple choice tests). Today we have the stylus for cellular phones.
11 Video Display – Raster Displays ● The next step in the evolution of CRT-based displays were the so called raster displays. – A raster display composes the content of the display by sending a beam of electrons to the canvas in a row wise manner. – Each row consists of column elements, the so called picture elements, or pixels. Raster displays are known from TV screens. ● A raster, or an array of pixels makes up the displayed image. ● And in software a rastered image corresponded to a block of memory in the computer (as an array). ● Technically, each pixel on the canvas is a spot of phosphor, being lighted by the electron beam. ● The benefit of raster displays is that they can also display complex graphics (in opposite to vector displays). – But "feeding" raster displays with data needs much more memory than feeding vector displays with data. electron gunraster with lighted and dark pixels raster display screen CRT canvas electron beam
12 Monochrome Video Display ● From the perspective of raster displays a screen has a resolution, e.g. a resolution of 640 x 480 pixels. – (The resolution is sometimes called spatial dimension.) – This makes a total of 307200 pixels. – If we want to represent each pixel by one bit, we need 307200b or 38400B to store the content of the screen. – With this representation we can only have pixels in either the state "on" (white/green) or "off" (black), like the states of one bit. – The representation of one pixel with one bit makes a raster display a monochrome display. ● In order to be able to represent more color states per pixel, we need more bits to store the state of a pixel. – We could use 1B to store a pixel. This would of course increase the required memory to store an image. ● For a resolution of 640 x 480 at 1B 307200B of memory are required. – If we use 1B per pixel, we can have one pixel presenting 28 (256) colors. ● E.g. we could have 256 different nuances of grey between the pixel values 0 (black) and 256 (FF16 ) (white). ● Technically, the CRT of a monochrome display sends its electron beam in different intensities to have different nuances of grey. – We could have 256 different nuances of grey between a pixel being dark (value: 0, color: black) and lighted (value: 256, color: white).
13 Video Display in Colors ● The step from a monochrome display to a colorful display is simple: – Use three electron guns/beams for each primary color. – Each beam can then have a different energetic intensity. ● The three primary colors of the additive color mix are red, green and blue (RGB). – Additive color mix works with spots of color emitting light. – The additive color mix allows the representation of any color by having combinations of different intensities of each primary color. – Technically, each pixel on the canvas is a combination of three spots of phosphor, colorized in red, green and blue, the subpixels. – Each subpixel can be lighted individually by the electron beam responsible for the color of that subpixel. additive color mix RGB electron guns three electron beams pixel of a color raster display green subpixel (lighted) raster with lighted and dark pixels/subpixels ● Subtractive color mix works with spots of color reflecting light, i.e. a part of the color information is absorbed. The subtractive color mix its relevant for printers.
14 Video Display in Colors – Color Resolution ● To drive this point home we have to represent a pixel in software with a combination of the three primary colors: – A color pixel is represented by a combination of different intensities of each primary color. – The count of intensities of each primary color defines how many different colors a pixel can represent. This is called color resolution or color depth. ● Examples of color resolution: – Use 2B/16b per pixel. ● We have 5b per color (1b left over or 6b for green). ● Then each color can have one of 32 intensities. ● Then a pixel can represent one of 32,768 colors. ● The 2B/16b color resolution is often called high color. – Use 3B/24b per pixel. ● We have 8b per color. ● Then each color can have one of 256 intensities. ● Then a pixel can represent one of 16,777,216 colors. ● The 3B/24b color resolution is often called full color. ● It should be clear that the required memory increases with the color depth. – With a resolution of 640 x 480 we have a total of 307200 pixels. – With the color depth of 3B/24b almost 1MB (307200 x 3B) is required to store an image. General formular: number of representable colors = 2b per pixel Lucy in high color Lucy in full color ● The color resolution of the shown picture was manipulated (in this case downgraded of course) with the program GraphicsGale.
15 The Takeoff of Interactivity – The Mouse ● In the late 1960ies Doug Engelbart showed the fruits of his research concerning computer input devices. – He presented the concept of the computer mouse in the first real demo in computer industry. ● His demo, held on 1968-12-09, is sometimes called "The mother of all demos" (Steven Levy). – => Doug Engelbart invented the mouse in 1968, but it was forgotten afterwards. ● The usage concept of the computer mouse is "point and click". – The user controls a virtual input pointer on the screen by rolling the real mouse over the writing desk. ● In the beginning of the 1970ies, some folks of Engelbart's team left and joined theXerox corporation. – More exactly, they joined the Xerox PARC (Palo Alto Research Center, sometimes just called "PARC"). – Xerox PARC drove Xerox' initiative to enter computer business. ● Originally Xerox produced devices for office and document management (e.g. photocopiers and printers). – Many more early interested engineers joined PARC that time. ● At PARC the problem of the computer-human interface was recognized soon! – It led the engineers at PARC to revolutionary ideas... ● The Xerox corporation (founded as "Haloid Company") is the inventor of the photocopier (1949). Nowadays the term "Xerox Machine" is a synonym for "photocopier".
16 The Takeoff of Interactivity – Graphical User Interfaces ● Affordable raster CRTs became available. The result was the Xerox Alto (1972/73), a computer with revolutionary features. ● Xerox' revolutionary ideas in user interfaces – Part I: – A high resolution CRT 606 x 808 in portrait orientation (i.e. shaped like a sheet of paper). – The content of the main screen was laid out like a real writing desk. Hence, the main screen was just called "desktop". – Applications on the desktop ran in so called windows and could be controlled via graphical elements. ● These graphical elements were called controls: textfields, buttons, checkboxes, icons and menus. ● The user interface goes beyond bare text! – I.e. multiple applications could run simultaneously on the same screen! This was a significant improvement! – The user used a mouse to interact with the controls. Xerox remembered, reused and improved Engelbart's ideas here. – Such a rich interface was then called Graphical User Interface (GUI) in opposite to Console User Interface (CUI). – => The Xerox Alto was the first computer with a GUI. ● See "Xerox" video "Triumph of the Nerds" (3/3, beginning at 5:50). ● The word "control" is a general term, in Java those are called "components", in many unioxid environments those are called "widgets".
17 The Takeoff of Interactivity – Metaphors and Object-Orientation ● Xerox' revolutionary ideas in user interfaces – Part II: – GUIs enforced the definition and application of interface metaphors (i.e. simulations of the reality): – Desktop metaphor: a region, on which items with specific functions can be placed. ● While the mouse was rolling across the writing desk, the virtual mouse pointer moved across the virtual desktop. – Window metaphor: sheets of papers, on which different things are handled, windows can be stacked like real sheets. – Form metaphor: arrangement of controls on a window, the controls could be filled with data (textfields, checkboxes). – What You See Is What You Get (WSIWYG): design documents on the screen exactly as they will be reproduced on paper. ● WYSIWG text editors: "Bravo" (1974, Butler Wright Lampson and Charles Simonyi) and its successor "Gypsy" (1975, Larry Tesler and Timothy Mott) – Metaphors allow to use a computer based on guessing how things work, not knowledge. ● This is a key to being user friendly, sometimes it is what we call an ergonomic user interface. ● Xerox' revolutionary ideas in programming: – The more complex GUI required a mighty programming paradigm to program interface metaphors: object oriented (oo) programming. ● The controls of Alto's GUI were represented as genuine programming objects. ● Control objects could be combined on window objects while programming at design time and were represented as graphical elements at run time. ● (Introduction and application of the Model-View-Controller (MVC) software architecture.) – For the Alto, Alan Kay invented of the first oo programming language "Smalltalk", incl. an IDE, in the 1970ies. – Smalltalk basically influenced all oo languages we have today! ● Ergonomics means that things can be used intuitively. I.e. users can even guess, where needed functions do reside, because the usage pattern is always the same and follows metaphors. After this definition of ergonomics, ergonomics was somewhat broken by Windows 8's so called "Modern UI" (codename "Metro").
18 The Takeoff of Interactivity – The Desktop Metaphor ● Think back: esp. on the command line interface it is difficult to remember all the commands! With the desktop metaphor we have icons with symbols, which are self explanatory. ● On touch devices we have no desktop but a home screen (called springboard on iOS).
19 The Takeoff of Interactivity – Well, the Xerox Alto... ● Isn't that great? Wait! Have you ever heard about the Xerox Alto? Hm... no! ● Xerox didn't revolutionize with its ideas, because: – The management of Xerox did not recognize the potential of the things their engineers created. – The Alto was too expensive: 30,000$ in 1973. (Later on the real commercial product derived from the Alto was the Xerox Star.) – In other words: The Alto was ahead its time, or it wasn't yet the time for this revolution. – The Alto was not yet a PC, esp. because it was not affordable! ● In 1979, Steve Jobs and some fellows from Apple visited PARC and attended a demo of the Alto. – Jobs and his team were impressed and instantly understood the outstanding meaning of having a GUI. – In took until 1983, but Apple made the computer a consumer product: the Apple Lisa. – But the Apple Lisa was too expensive (10,000$). (Em, Steve's fault!) ● Only about 1000 pieces of Xerox Alto were sold. ● The Xerox Star was sold at ~16500$ ● Only about 11,000 – 50,000 pieces of the Lisa and about 50,000 pieces of the Lisa 2 were sold.
20 The Takeoff of Interactivity – Breakthrough ● The real breakthrough of interactivity on consumer computers was theApple Macintosh (Mac) in 1984. – A revolutionary TV commercial during a break of the telecast of the Super Bowl (1984-01-22) started to attract potential users. – Affordable at ~2,500$. – All in one computer (Motorola 68000 CPU), keyboard, mouse and screen (512×342 monochrome). – The revolutionary system software Mac OS (the ancestor of OS X), which provided the user interactivity. ● The Mac introduced a new class of applications based on the desktop metaphor (applications beyond "number crunching"). – Desktop Publishing (DTP) – What You See Is What You Get, WYSIWYG ("wizzy wig") – (Before that, CUI applications were basically applications using the file system.) ● After the desktop metaphor was established it was mostly only copied by other companies for more than four decades. – VisiCorp VisiOn (1983) – Digital Research GEM (Graphical Environment Manager) (1984) – Microsoft Windows 1.0 (1985); the real takeoff was in 1990 with Windows 3.0 – X Window System (1984) ● The keyboard of the Macintosh in 1984 didn't even have arrow keys! - Obviously Apple's engineers thought that all users will prefer the mouse. And it took until the 1990ies, when Apple applied the "inverted-T" layout of the arrow keys. ● Windows 2.0 was introduced 1987 and it allowed overlapping windows.
21 Not discussed Topics ● X Window System and multi user support ● Voice recognition ● Touch
22 History – Part I ● Users were white collar workers. Input: plugging cables; output: lightbulbs. – Users were programmers! ● Users applied batch processing. – Users needn't to be programmers any longer! – Input: a batch of punched cards/tapes (code and data); output on a printer (continuous printing). ● Users sat in front of a terminal. A terminal was one device for input/output (tty), later separate devices (glass ttys). – Users had a session with mainframe, which was controlled from the terminal via a network! – One dimensional interactivity: line-by-line communication, one program for one task. – Bare command-line: full control, less comfort. Command-line prompt: less control, some comfort. ● Two dimensional applications allowed moving the cursor in two dimensions and using menus. – Users worked with a standalone PC. – Metaphors (simulations of the reality) emerged in applications: document and spreadsheet. Multipurpose applications. – Compromise of control and comfort. ● It should be said that there is a tendency to reduce the freedom of using multipurpose applications on the desktop somewhat. The idea is to guide users through using certain applications at a certain time for a certain task, along a and as part of a so called business process. ● This tendency is still ongoing and evolving, because the technical infrastructure grew in the meantime: cheaper network throughput, clouds, applications able to access cloud services etc. ● This tendency merges up with web-applications, esp. with three-tiered applications, on which much processing power and handling multiple user sessions reside on a server. - In a sense it means a step "back" to the terminal/mainframe idea!
23 History – Part II ● The desktop metaphor. Input: keyboard, mouse; output: screen, printer/WYSIWYG. – Users controlled applications with the mouse via point and click. Ultimate two dimensional usage with windows and controls/forms. – Control and comfort were mixed up completely, because applications could be used by "guessing", i.e. ergonomics. ● Emerging UI aspects (read: not only graphical aspects): – Touch-gestures (killer app: Apple iOS) – Haptic feedback (Apple's Force Touch Trackpad) – Physical-gestures (killer app: Nintendo EAD Wii Sports) – Voice recognition (killer app: Apple's Siri) – Motion detection – Ambient detection ● Nintendo EAD, EAD for Entertainment Analysis & Development. ● Meanwhile we are confronted with further usage paradigms and emerging metaphors because of new applications: mobile devices, smart devices (e.g. TVs) support of disabled persons and the internet of things.
24 Thank you!

More Related Content

PPTX
CST 20363-Session 1.2-A Brief History of Computing
byoudesign
 
PDF
01 - Introduction to Computer
byProgramming Techniques and Algorithms
 
PPT
Generation of computers
byMainul Morshed
 
PDF
Basic concepts of a computer
byRaja Adapa
 
PDF
Input and output devices of computer
byRaja Adapa
 
PPT
Unit 1 one part introduction to computers
byNeha Kurale
 
PPT
What is a computer
byLGNIBOARIA
 
PDF
H.R Notes
byHamid Raza
 
CST 20363-Session 1.2-A Brief History of Computing
byoudesign
 
01 - Introduction to Computer
byProgramming Techniques and Algorithms
 
Generation of computers
byMainul Morshed
 
Basic concepts of a computer
byRaja Adapa
 
Input and output devices of computer
byRaja Adapa
 
Unit 1 one part introduction to computers
byNeha Kurale
 
What is a computer
byLGNIBOARIA
 
H.R Notes
byHamid Raza
 

What's hot

PPT
ICT Systems
bydhafinnaviansyah
 
PPTX
In the name of allah
byjahansaz
 
PPT
1. computers introduction
byDr.N.Poorna Chelvan
 
PPTX
Computer generations
byUdara Sandaruwan
 
PPTX
MR. RAMESH KUMAR KANNAUJIA
byrameshkumar1646
 
PPTX
Computer
byNEERAJKUMAR1930
 
PPT
Unit 1 computer programming
bykirthika jeyenth
 
DOC
Module 1 intro evolution computer
byMusa Muhamad
 
PPTX
computer application in hospitality Industry, periyar university unit 1
byadmin information
 
PPT
Bsc cs 1 fit u-1 computer hardware system
byRai University
 
PDF
Basics of computer & internet .pptx
bySatish k
 
DOCX
Installation of drivers
bysyed mehdi raza
 
PDF
Hardware basics: peripherals
byMiguel Rebollo
 
PPTX
Evolution and Types of the Computer
byNur Azlina
 
PPTX
What is Coomputer What are the parts of computer
byTarandeepSingh143
 
PPTX
Generation of computer
byvanitha caroline
 
ICT Systems
bydhafinnaviansyah
 
In the name of allah
byjahansaz
 
1. computers introduction
byDr.N.Poorna Chelvan
 
Computer generations
byUdara Sandaruwan
 
MR. RAMESH KUMAR KANNAUJIA
byrameshkumar1646
 
Computer
byNEERAJKUMAR1930
 
Unit 1 computer programming
bykirthika jeyenth
 
Module 1 intro evolution computer
byMusa Muhamad
 
computer application in hospitality Industry, periyar university unit 1
byadmin information
 
Bsc cs 1 fit u-1 computer hardware system
byRai University
 
Basics of computer & internet .pptx
bySatish k
 
Installation of drivers
bysyed mehdi raza
 
Hardware basics: peripherals
byMiguel Rebollo
 
Evolution and Types of the Computer
byNur Azlina
 
What is Coomputer What are the parts of computer
byTarandeepSingh143
 
Generation of computer
byvanitha caroline
 

Viewers also liked

PDF
Atom ことはじめ
byTomohiro Nishimura
 
PDF
Hackers manual 2015 revised edition
byMD. Monzurul Karim Shanchay
 
PDF
Github github-github
byfusion2011
 
PDF
Csebook for cse students
byVaibhav Singla
 
PDF
Develop Desktop Apps with Electron
byEueung Mulyana
 
PDF
Spring Boot and JHipster
byEueung Mulyana
 
PDF
Learning chef
byJonathan Carrillo
 
Atom ことはじめ
byTomohiro Nishimura
 
Hackers manual 2015 revised edition
byMD. Monzurul Karim Shanchay
 
Github github-github
byfusion2011
 
Csebook for cse students
byVaibhav Singla
 
Develop Desktop Apps with Electron
byEueung Mulyana
 
Spring Boot and JHipster
byEueung Mulyana
 
Learning chef
byJonathan Carrillo
 

Similar to (1) gui history_of_interactivity

ODP
(1) gui history_of_interactivity
byNico Ludwig
 
PPT
1.lecture_07_paradigms.ppt
byUsamajawad3
 
PPT
Interaction Design History
byMarc Rettig
 
PPTX
Operating Systems
byLeon Lei
 
PDF
Rettig.Interaction Design History.V1.5
bypemi hua
 
PDF
A Brief History Of Human-Computer Interaction Technology
byAndrew Molina
 
PPT
Chapter 3 paradigm
byMLG College of Learning, Inc
 
PPT
HCI - Chapter 4
byAlan Dix
 
PPT
HCI 3e - Ch 4: Paradigms
byAlan Dix
 
PPSX
Types and components of computers
byCik Na Shohaili
 
PPTX
Operating System Unit 1
bySanthiNivas
 
PPTX
Unit i
byScenka Manguiat
 
PPT
Paradigms course in Human computer interaction power point
byramlaabdikarim001
 
PPT
IMD 203 - Ch01
byALBAKRI MOHAMMAD
 
PPTX
Human computerinterface
byKumar Aryan
 
PPTX
History of Computer Graphics
byKamal Acharya
 
PDF
[I3 d]04 interactivity
byjylee_kgit
 
PPTX
1.1 Introduction to Operating System .pptx
bySudarshanSharma43
 
PPT
E3 chap-04
byLukmanulhakim Almamalik
 
PDF
Video Terminal Evolution and The Future of Browsers
byThomas Walker Lynch
 
(1) gui history_of_interactivity
byNico Ludwig
 
1.lecture_07_paradigms.ppt
byUsamajawad3
 
Interaction Design History
byMarc Rettig
 
Operating Systems
byLeon Lei
 
Rettig.Interaction Design History.V1.5
bypemi hua
 
A Brief History Of Human-Computer Interaction Technology
byAndrew Molina
 
Chapter 3 paradigm
byMLG College of Learning, Inc
 
HCI - Chapter 4
byAlan Dix
 
HCI 3e - Ch 4: Paradigms
byAlan Dix
 
Types and components of computers
byCik Na Shohaili
 
Operating System Unit 1
bySanthiNivas
 
Unit i
byScenka Manguiat
 
Paradigms course in Human computer interaction power point
byramlaabdikarim001
 
IMD 203 - Ch01
byALBAKRI MOHAMMAD
 
Human computerinterface
byKumar Aryan
 
History of Computer Graphics
byKamal Acharya
 
[I3 d]04 interactivity
byjylee_kgit
 
1.1 Introduction to Operating System .pptx
bySudarshanSharma43
 
E3 chap-04
byLukmanulhakim Almamalik
 
Video Terminal Evolution and The Future of Browsers
byThomas Walker Lynch
 

More from Nico Ludwig

PPTX
Grundkurs fuer excel_part_v
byNico Ludwig
 
PPTX
Grundkurs fuer excel_part_iv
byNico Ludwig
 
PPTX
Grundkurs fuer excel_part_iii
byNico Ludwig
 
PPTX
Grundkurs fuer excel_part_ii
byNico Ludwig
 
PPTX
Grundkurs fuer excel_part_i
byNico Ludwig
 
PDF
(2) gui drawing
byNico Ludwig
 
ODP
(2) gui drawing
byNico Ludwig
 
PDF
New c sharp4_features_part_vi
byNico Ludwig
 
PDF
New c sharp4_features_part_v
byNico Ludwig
 
PDF
New c sharp4_features_part_iv
byNico Ludwig
 
ODP
New c sharp4_features_part_iii
byNico Ludwig
 
PDF
New c sharp4_features_part_ii
byNico Ludwig
 
PDF
New c sharp4_features_part_i
byNico Ludwig
 
PDF
New c sharp3_features_(linq)_part_v
byNico Ludwig
 
PDF
New c sharp3_features_(linq)_part_iv
byNico Ludwig
 
ODP
New c sharp3_features_(linq)_part_iv
byNico Ludwig
 
PDF
New c sharp3_features_(linq)_part_iii
byNico Ludwig
 
PDF
New c sharp3_features_(linq)_part_ii
byNico Ludwig
 
PDF
New c sharp3_features_(linq)_part_i
byNico Ludwig
 
ODP
New c sharp3_features_(linq)_part_i
byNico Ludwig
 
Grundkurs fuer excel_part_v
byNico Ludwig
 
Grundkurs fuer excel_part_iv
byNico Ludwig
 
Grundkurs fuer excel_part_iii
byNico Ludwig
 
Grundkurs fuer excel_part_ii
byNico Ludwig
 
Grundkurs fuer excel_part_i
byNico Ludwig
 
(2) gui drawing
byNico Ludwig
 
(2) gui drawing
byNico Ludwig
 
New c sharp4_features_part_vi
byNico Ludwig
 
New c sharp4_features_part_v
byNico Ludwig
 
New c sharp4_features_part_iv
byNico Ludwig
 
New c sharp4_features_part_iii
byNico Ludwig
 
New c sharp4_features_part_ii
byNico Ludwig
 
New c sharp4_features_part_i
byNico Ludwig
 
New c sharp3_features_(linq)_part_v
byNico Ludwig
 
New c sharp3_features_(linq)_part_iv
byNico Ludwig
 
New c sharp3_features_(linq)_part_iv
byNico Ludwig
 
New c sharp3_features_(linq)_part_iii
byNico Ludwig
 
New c sharp3_features_(linq)_part_ii
byNico Ludwig
 
New c sharp3_features_(linq)_part_i
byNico Ludwig
 
New c sharp3_features_(linq)_part_i
byNico Ludwig
 

Recently uploaded

PDF
Dev Dives: Build and deploy agentic automations - the unified way
byUiPathCommunity
 
PDF
UiPath Automation Developer Associate Training Series 2026 - Session 1
byDianaGray10
 
PDF
Website Redesign Vs Website Refresh, What’s Right for You in 2026?
byAnuj Kumar Singh
 
PDF
Post-Hackathon-Learnings-Maximizing-Impact-Beyond-the-Event.pdf
byishantyadav1111
 
PPTX
TechSprint Inauguration at SJB Institute of Technology held on 18 December 2025
bysuhasspgdg
 
PDF
OpenCharacter AI Reviews: Features, Plans, and Best Alternative
byOpenCharacter AI
 
PPTX
TechSprint WinterHack — Top 10 Teams Pitching Session we are excited for pit...
bybajpaitusharoffon678
 
PDF
Real-Time AI Masterclass: Vector Search at Scale
byScyllaDB
 
PDF
The Enterprise Web3 Landscape - a view from Kaleido based on the past 10 year...
byPeter Broadhurst
 
PDF
AI and Zero Trust: What it takes to do it right
byMark Simos
 
PDF
How to build hackthon projects from ZERO!
byishantyadav1111
 
PPTX
Introduction to Artificial Intelligence (AI).pptx
byammar414783
 
PDF
Chapter 4 Network Security in computer security
byGetnet Tigabie Askale -(GM)
 
PDF
Agentic-Document-Extraction-2026-Presentation.pdf
byTamanna
 
PDF
Supercharge Your Copilot-Driven Collaboration with Microsoft 365 Agents SDK
byAntti Koskela
 
PDF
Designing a Blog Using Wordpress
bymarkzubi50
 
PDF
"Painless Major Upgrade: A Strategy for Updating Large Codebases", Andrii Yat...
byFwdays
 
PDF
January 2026 OpenMetadata Community Spotlight - OpenMetadata @ Wix.pdf
byOpenMetadata
 
PDF
Transcript: What Thema can do: Leveraging metadata to support the discoverabi...
byBookNet Canada
 
PDF
What Thema can do: Leveraging metadata to support the discoverability of Firs...
byBookNet Canada
 
Dev Dives: Build and deploy agentic automations - the unified way
byUiPathCommunity
 
UiPath Automation Developer Associate Training Series 2026 - Session 1
byDianaGray10
 
Website Redesign Vs Website Refresh, What’s Right for You in 2026?
byAnuj Kumar Singh
 
Post-Hackathon-Learnings-Maximizing-Impact-Beyond-the-Event.pdf
byishantyadav1111
 
TechSprint Inauguration at SJB Institute of Technology held on 18 December 2025
bysuhasspgdg
 
OpenCharacter AI Reviews: Features, Plans, and Best Alternative
byOpenCharacter AI
 
TechSprint WinterHack — Top 10 Teams Pitching Session we are excited for pit...
bybajpaitusharoffon678
 
Real-Time AI Masterclass: Vector Search at Scale
byScyllaDB
 
The Enterprise Web3 Landscape - a view from Kaleido based on the past 10 year...
byPeter Broadhurst
 
AI and Zero Trust: What it takes to do it right
byMark Simos
 
How to build hackthon projects from ZERO!
byishantyadav1111
 
Introduction to Artificial Intelligence (AI).pptx
byammar414783
 
Chapter 4 Network Security in computer security
byGetnet Tigabie Askale -(GM)
 
Agentic-Document-Extraction-2026-Presentation.pdf
byTamanna
 
Supercharge Your Copilot-Driven Collaboration with Microsoft 365 Agents SDK
byAntti Koskela
 
Designing a Blog Using Wordpress
bymarkzubi50
 
"Painless Major Upgrade: A Strategy for Updating Large Codebases", Andrii Yat...
byFwdays
 
January 2026 OpenMetadata Community Spotlight - OpenMetadata @ Wix.pdf
byOpenMetadata
 
Transcript: What Thema can do: Leveraging metadata to support the discoverabi...
byBookNet Canada
 
What Thema can do: Leveraging metadata to support the discoverability of Firs...
byBookNet Canada
 

(1) gui history_of_interactivity

  • 1.
    1 Nico Ludwig (@ersatzteilchen) (1)GUI – History of Interactivity
  • 2.
    2 TOC ● (1) GUI– History of Interactivity – Wires and Bulbs – Batch Processing – Terminal and Mainframe – From the Command-Line to Killer Applications – Vector Displays and Raster Displays – Color Displays – The Mouse and the Takeoff of Interactivity – The Desktop Metaphor ● Sources: – Wikipedia: ● Bubba73, ASR 33; Shieldforyoureyes/Dave Fisher, BM-3279; Oliver.obj, Panel 370-145; Boffy b, IBM PC 5150; Rama & Musée Bolo, Apple II-IMG 4217; SRI International, SRI Computer Mouse – Video series "Triumph of the Nerds" – Matthew MacDonald, Pro .NET 2.0 Windows Forms and Custom Controls in C#, Appendix A – Charles Petzold, Code – The Hidden Language of Computer Hardware and Software, Chapter 25 – other sources are public domain
  • 3.
    3 The Beginning –Data Processing ● Basically we'll discuss the user interface (ui), which is the place where computer and human meet. ● In the early days of computing, computing was a job for white collar workers. – In the 1940ies computers were used by switching switches and plugging cables. – The state of the switches and the wiring represented data and program "code". – The results were displayed with an array of lightbulbs. – Using and programming was usually the same thing. ● In the 1950ies, 1960ies and early 1970ies batch processing was applied. – Program code and data were punched into cards or tapes ("punched cards/tapes"), which were interpreted by the computer. – The results have been printed on paper. – The whole input was processed as one batch of punched cards. Input Processing Output Process input in batch Batch of punched cards (data and program code) Print the output to paper Directionofthe processing ● The graphic shows an ENIAC (Electronic Numerical Integrator And Computer) (1947 – 1955). ● Punched cards and tapes have also been used for organs and looms.
  • 4.
    4 Interactivity raises –Having a Session with a Computer ● Somewhen in the 60ies computers learned to interact with humans. (interact, from lat.: inter = "between", agere = "to act") – Er, or, … somewhen in the 60ies humans learned to interact with computers. ● Interaction means that human input and computer response happen multiply within the samesession. – The computer and the means of input/output were separated devices connected by a network. – The user inputs data via a teletypewriter (tty) and the computer responded via the tty as well. The tty was the so called "terminal". – So user input can directly follow the last computer output, but not in parallel. This makes the interaction. ● The terminal prompts for input, the user sends his input, after it is processed by the computer it is resend to the terminal and another prompt appears. – The communication is based on transmission of streams of serial data following the American Standard Code for Information Interchange (ASCII) protocol. – The whole session goes only into one direction, it can be traced along the tty's paper roll. Input Processing Output Input Processing Output Input ... computer session Directionofthesession Input & output/ tty computer terminal network computer ● I.e. ttys were used as peripheral device and terminal. ● Input and output of ttys was just text, so it is also called "text terminal" (German: "Schreibmaschinenterminal"). ● Because a paper roll was used to trace the procedure, it was also called "hard-copy terminal". ● A tty is a printer and a keyboard. ● Prompt from lat. promptare: "to publish something". ● There survived some terms from this era of computing: ● ASCII, control characters: LF, CR ● The "TTY", as oldest serial interface, is still available as terminal interface (shell) and to program industry plants.
  • 5.
    5 Interactivity raises –Keyboards and Screens ● In the 70ies, separated peripherals for input (the keyboard) and output (cathode ray tube (CRT) screens) were introduced. – Keyboard and CRT acted as terminal together, other terms: "glass ttys", Video Display Units (VDU), Video Display Terminals (VDT). ● In opposite to the term "hard-copy terminal" for the tty, which basically was a printer more than an input device. ● At this time in history two aspects are relevant to understand: – Still the "tty-idea" is used here! VDUs just represented rolls of paper, it was just a "line by line" communication. – VDUs were connected to a remote computer via a network. As being just terminals, VDUs had no intelligence at all! – => This arrangement consists of a "dumb terminal" (has little logic) and an "intelligent computer", the mainframe (has a CPU). Output/ CRT Input/keyboard computer dump terminal network remote, intelligent computer (mainframe) ● "Line-by-line" communication means that we have no cursor control, but following control codes are mandatory: CR, LF and Bell. ● OS' like MS DOS and C/PM uses/used monitors only that ways. ● By a matter of fact today we still use this metaphor in diverse command line interfaces/shells. ● Monitors have the potential to interact with the user by applying the whole two dimensional space of the screen. ASCII was already equipped to be extended with control codes handling two dimensional character devices (clean the whole screen or move the cursor to the left upper corner). ● Before ASCII there was the Extended Binary Coded Decimals Interchange Code (EBCDIC). EBCDIC was used by IBM's mainframes in the 1950ies and early 1960ies. ● EBCDIC's character codes were not arranged sequentially in alphabetic order (i.e. there were gaps), but optimized for punched card processing. For modern applications ASCII was much more useful, because of its alphabetic structure. ● Mainframe, German: Großrechner ● The shown terminal is an IBM-3279 (~1972) a green screen terminal ● The shown mainframe is an IBM System/370-145 (~1970) (the system console). ● We don't talk about applications running on the mainframe, which were basically databases and financial processing software. The large units to store all the data are not shown (the compatible storage unit is the IBM 3850 with magnetic tape cartridges). ● The terms "dumb terminal" or "thin client" have other meanings today.
  • 6.
    6 Terminals and Mainframes?– What the heck...? ● One moment! Why was this kind of arrangement with ttys/VDUs and mainframes and a network in between necessary? – Because of the problems, which were solved with computers that time! Read on: ● In the beginning, computers were basically used for financial and military purposes. – Starting in the 1960ies, people understood how computers are able to process masses of data, esp. in business. – This problem domain required holding and managing huge amounts of data! The problem domain drove two developments: ● (1) Mass storage hardware, which holds data. ● (2) Database software, which manages data. ● In the end there are three (four) individual devices, connected by network/wire: – ((0) User.) (1) Terminal. (2) Mainframe. (3) Mass storage device. – It was required, because of technical possibilities that time and because folks just clung to the idea of a "big computer room". Output/ CRT Input/keyboard computer terminal network mainframe database's data mass storage device ● (To be frank in the late 60ies also more intelligent terminals with some microprocessor power and some memory to handle editing operations (insert a character), without sending/re-sending the whole screen to the computer were available)
  • 7.
    7 The Evolution ofthe Command-Line ● With terminals and mainframes the command-line was the major means of interactivity. There were some historical steps: ● The bare command-line age: – Full control, the user could freely use all tools in any order. – No guidance, no structure, no metaphor, commands needed to be remembered by the user. ● The command-line prompt age: – No control, the user needs to follow an application's usage paradigms. A program could only do a single task. – Full guidance, each step is prompted, we had an interview-metaphor (question/answer). ● The menu age: (Let's call a program "application".) – Compromise of control, the user was able to do a set of tasks. An application was able to do multiple tasks. – Ergonomic guidance, menus are a compromise of prompt and free selection of the next action. We have a document-metaphor.
  • 8.
    8 Two dimensional Interactivity– Killer Applications ● Its time to give up working with line-by-line communication. – For terminals cursor control was not mandatory, only these control codes were required: CR, LF and Bell – But: screens have the potential to break free line-by-line and use their full two dimensional space. ● ASCII was already ready to be extended with control codes for two dimensional character devices (clean screen, move cursor to left upper corner, etc.). ● In 1979 VisiCorp presented an application that used the screen in a two dimensional manner:VisiCalc. – VisiCalc was a spreadsheet application using the screen as a table for numerical calculation. It brought following new aspects: ● The cursor could now be moved among columns and rows! ● The screen followed a metaphor. The "spreadsheet"-metaphor mimics a chalkboard or pencil/paper on a computer screen. ● The application had menus to guide the user and applied another metaphor: the "document"-metaphor. ● VisiCalc was the first killer application (app) for home computers! – A killer app is an app that significantly changes the attitude of users on using a technology. A killer app is a "game changer". – This killer app proved that one can do useful things with a computer (here: "number crunching"). ● VisiCalc was the breakthrough for the Apple II. – It showed that the Apple II really had a business use! ● The Apple II, the first PC, needed an important application (a "killer app") to be a success: VisiCalc (designed by Dan Bricklin at the "Harvard Business School" in 1979). ● Another way to characterize a killer app: vendors and developers use it as reference. "Let's do it like it is done in VisiCalc." ● VisiCalc was also the killer app on Apple II's competitor "the IBM PC" (IBM model 5150 in 1981). ● With VisiCalc the transition of the computer from a hobby object to a business object started. ● The spreadsheet concept was, however, not patented by VisiCorp and it was copied by Microsoft (Multiplan, Excel), Lotus Development Corporation ((belongs to IBM today) Lotus 1-2-3)). ● Also text processors which were then partially replaced by word processors featuring WYSIWYG.
  • 9.
    9 Two dimensional Interactivity– Hey Terminal, good Riddance! ● Apple II? Where did the mainframe go? ● There is a very important point: an app like VisiCalc can't be brought to the dumb terminal/mainframe arrangement: – Fast screen updates as reaction on user input are required, and no extra communication via a slow network. – Memory went short, because usually terminals (the better ones) could only store a few lines of text up to page. ● From terminals to Personal Computers (PC). PCs are standalone computers. – Basically, the intelligence of computer and terminal were integrated into one unit. This unit was called Personal Computer (PC). – Apple (!) was the first manufacturer of PCs for consumers: the Apple II reached the market in 1977 (at 1298$). ● Apple made computers white goods (like vacuum cleaners or microwaves are). ● Speed is important to make interactivity work acceptably! – (1) PCs directly accessed the video memory without the OS or the network. ● Dumb terminals had only means for character generation in the hardware. In opposite to some PCs even having a graphics adapter. – (2) PCs had their own memory (4KB and more). ● In parallel to the development of two dimensional interactivity thedevelopment of video display evolved... ● Virtually IBM "slept" until the early 1980ies, then they understand the meaning of PCs. They released the IBM-PC (IBM 5150) in 1981, four years after Apple's Apple II. ● PC today: IBM-compatible (IBM compatible – read: esp. the interface to video memory is standardized) This "openness" was the key to make software independent from hardware. Apple II and IBM PC were expandable "in the box" by attaching new adapters (also from 3rd party vendors) to the so called mainboard. ● Homecomputers could be connected to the TV, but early IBM-PCs could also be connected to the TV. A better way to categorize homecomputers is that they are closed in opposite to the openness of IBM compatible computers. - In the world of a homecomputer everything is proprietary (connectors and internal adapters, one could buy extensions or replacement parts only from the manufacturer of the homecomputer).
  • 10.
    10 Video Display –Vector Displays ● To understand the next steps on the way to interactivity, the further development of video displays has to be understood. – Up to now we only discussed systems using the terminal screen, often only with a two-color palette (e.g. a "green screen"). – The most important step was from character displays to graphical displays. ● The first graphical displays for computing were so called vector displays. – Vector displays are CRT displays, in which an electron ray (emitted by an electron gun) directly draws objects composed of primitives on the screen. ● This is similar to a human drawing on a canvas. Also characters are composed from primitives. ● Examples: radar screens, oszilloscopes and the first primitive computer games. ● Vector displays brought an interesting idea of interaction: thelight pen. – Light pens also brought a new killer app: "Sketchpad" (Ph.D. thesis Ivan Sutherland, 1962/1963). – Sketchpad was the first interactive graphical app, it introduced Computer Aided Design (CAD). ● But these vector displays needed to update the drawing in short intervals. – The time needed to redraw the display limited the amount of content (and complexity) that could be displayed. canvas electron gun vector display screen CRT electron beam ● The Apple II had up to 15 colors. ● Lightpen: ● A lightpen detects the beam emitted by the electron gun. Lightpens are also usable for CRTs. ● Afterwards, lightpens were used to fill out forms on the screens (e.g. ticking check boxes or for simple multiple choice tests). Today we have the stylus for cellular phones.
  • 11.
    11 Video Display –Raster Displays ● The next step in the evolution of CRT-based displays were the so called raster displays. – A raster display composes the content of the display by sending a beam of electrons to the canvas in a row wise manner. – Each row consists of column elements, the so called picture elements, or pixels. Raster displays are known from TV screens. ● A raster, or an array of pixels makes up the displayed image. ● And in software a rastered image corresponded to a block of memory in the computer (as an array). ● Technically, each pixel on the canvas is a spot of phosphor, being lighted by the electron beam. ● The benefit of raster displays is that they can also display complex graphics (in opposite to vector displays). – But "feeding" raster displays with data needs much more memory than feeding vector displays with data. electron gunraster with lighted and dark pixels raster display screen CRT canvas electron beam
  • 12.
    12 Monochrome Video Display ●From the perspective of raster displays a screen has a resolution, e.g. a resolution of 640 x 480 pixels. – (The resolution is sometimes called spatial dimension.) – This makes a total of 307200 pixels. – If we want to represent each pixel by one bit, we need 307200b or 38400B to store the content of the screen. – With this representation we can only have pixels in either the state "on" (white/green) or "off" (black), like the states of one bit. – The representation of one pixel with one bit makes a raster display a monochrome display. ● In order to be able to represent more color states per pixel, we need more bits to store the state of a pixel. – We could use 1B to store a pixel. This would of course increase the required memory to store an image. ● For a resolution of 640 x 480 at 1B 307200B of memory are required. – If we use 1B per pixel, we can have one pixel presenting 28 (256) colors. ● E.g. we could have 256 different nuances of grey between the pixel values 0 (black) and 256 (FF16 ) (white). ● Technically, the CRT of a monochrome display sends its electron beam in different intensities to have different nuances of grey. – We could have 256 different nuances of grey between a pixel being dark (value: 0, color: black) and lighted (value: 256, color: white).
  • 13.
    13 Video Display inColors ● The step from a monochrome display to a colorful display is simple: – Use three electron guns/beams for each primary color. – Each beam can then have a different energetic intensity. ● The three primary colors of the additive color mix are red, green and blue (RGB). – Additive color mix works with spots of color emitting light. – The additive color mix allows the representation of any color by having combinations of different intensities of each primary color. – Technically, each pixel on the canvas is a combination of three spots of phosphor, colorized in red, green and blue, the subpixels. – Each subpixel can be lighted individually by the electron beam responsible for the color of that subpixel. additive color mix RGB electron guns three electron beams pixel of a color raster display green subpixel (lighted) raster with lighted and dark pixels/subpixels ● Subtractive color mix works with spots of color reflecting light, i.e. a part of the color information is absorbed. The subtractive color mix its relevant for printers.
  • 14.
    14 Video Display inColors – Color Resolution ● To drive this point home we have to represent a pixel in software with a combination of the three primary colors: – A color pixel is represented by a combination of different intensities of each primary color. – The count of intensities of each primary color defines how many different colors a pixel can represent. This is called color resolution or color depth. ● Examples of color resolution: – Use 2B/16b per pixel. ● We have 5b per color (1b left over or 6b for green). ● Then each color can have one of 32 intensities. ● Then a pixel can represent one of 32,768 colors. ● The 2B/16b color resolution is often called high color. – Use 3B/24b per pixel. ● We have 8b per color. ● Then each color can have one of 256 intensities. ● Then a pixel can represent one of 16,777,216 colors. ● The 3B/24b color resolution is often called full color. ● It should be clear that the required memory increases with the color depth. – With a resolution of 640 x 480 we have a total of 307200 pixels. – With the color depth of 3B/24b almost 1MB (307200 x 3B) is required to store an image. General formular: number of representable colors = 2b per pixel Lucy in high color Lucy in full color ● The color resolution of the shown picture was manipulated (in this case downgraded of course) with the program GraphicsGale.
  • 15.
    15 The Takeoff ofInteractivity – The Mouse ● In the late 1960ies Doug Engelbart showed the fruits of his research concerning computer input devices. – He presented the concept of the computer mouse in the first real demo in computer industry. ● His demo, held on 1968-12-09, is sometimes called "The mother of all demos" (Steven Levy). – => Doug Engelbart invented the mouse in 1968, but it was forgotten afterwards. ● The usage concept of the computer mouse is "point and click". – The user controls a virtual input pointer on the screen by rolling the real mouse over the writing desk. ● In the beginning of the 1970ies, some folks of Engelbart's team left and joined theXerox corporation. – More exactly, they joined the Xerox PARC (Palo Alto Research Center, sometimes just called "PARC"). – Xerox PARC drove Xerox' initiative to enter computer business. ● Originally Xerox produced devices for office and document management (e.g. photocopiers and printers). – Many more early interested engineers joined PARC that time. ● At PARC the problem of the computer-human interface was recognized soon! – It led the engineers at PARC to revolutionary ideas... ● The Xerox corporation (founded as "Haloid Company") is the inventor of the photocopier (1949). Nowadays the term "Xerox Machine" is a synonym for "photocopier".
  • 16.
    16 The Takeoff ofInteractivity – Graphical User Interfaces ● Affordable raster CRTs became available. The result was the Xerox Alto (1972/73), a computer with revolutionary features. ● Xerox' revolutionary ideas in user interfaces – Part I: – A high resolution CRT 606 x 808 in portrait orientation (i.e. shaped like a sheet of paper). – The content of the main screen was laid out like a real writing desk. Hence, the main screen was just called "desktop". – Applications on the desktop ran in so called windows and could be controlled via graphical elements. ● These graphical elements were called controls: textfields, buttons, checkboxes, icons and menus. ● The user interface goes beyond bare text! – I.e. multiple applications could run simultaneously on the same screen! This was a significant improvement! – The user used a mouse to interact with the controls. Xerox remembered, reused and improved Engelbart's ideas here. – Such a rich interface was then called Graphical User Interface (GUI) in opposite to Console User Interface (CUI). – => The Xerox Alto was the first computer with a GUI. ● See "Xerox" video "Triumph of the Nerds" (3/3, beginning at 5:50). ● The word "control" is a general term, in Java those are called "components", in many unioxid environments those are called "widgets".
  • 17.
    17 The Takeoff ofInteractivity – Metaphors and Object-Orientation ● Xerox' revolutionary ideas in user interfaces – Part II: – GUIs enforced the definition and application of interface metaphors (i.e. simulations of the reality): – Desktop metaphor: a region, on which items with specific functions can be placed. ● While the mouse was rolling across the writing desk, the virtual mouse pointer moved across the virtual desktop. – Window metaphor: sheets of papers, on which different things are handled, windows can be stacked like real sheets. – Form metaphor: arrangement of controls on a window, the controls could be filled with data (textfields, checkboxes). – What You See Is What You Get (WSIWYG): design documents on the screen exactly as they will be reproduced on paper. ● WYSIWG text editors: "Bravo" (1974, Butler Wright Lampson and Charles Simonyi) and its successor "Gypsy" (1975, Larry Tesler and Timothy Mott) – Metaphors allow to use a computer based on guessing how things work, not knowledge. ● This is a key to being user friendly, sometimes it is what we call an ergonomic user interface. ● Xerox' revolutionary ideas in programming: – The more complex GUI required a mighty programming paradigm to program interface metaphors: object oriented (oo) programming. ● The controls of Alto's GUI were represented as genuine programming objects. ● Control objects could be combined on window objects while programming at design time and were represented as graphical elements at run time. ● (Introduction and application of the Model-View-Controller (MVC) software architecture.) – For the Alto, Alan Kay invented of the first oo programming language "Smalltalk", incl. an IDE, in the 1970ies. – Smalltalk basically influenced all oo languages we have today! ● Ergonomics means that things can be used intuitively. I.e. users can even guess, where needed functions do reside, because the usage pattern is always the same and follows metaphors. After this definition of ergonomics, ergonomics was somewhat broken by Windows 8's so called "Modern UI" (codename "Metro").
  • 18.
    18 The Takeoff ofInteractivity – The Desktop Metaphor ● Think back: esp. on the command line interface it is difficult to remember all the commands! With the desktop metaphor we have icons with symbols, which are self explanatory. ● On touch devices we have no desktop but a home screen (called springboard on iOS).
  • 19.
    19 The Takeoff ofInteractivity – Well, the Xerox Alto... ● Isn't that great? Wait! Have you ever heard about the Xerox Alto? Hm... no! ● Xerox didn't revolutionize with its ideas, because: – The management of Xerox did not recognize the potential of the things their engineers created. – The Alto was too expensive: 30,000$ in 1973. (Later on the real commercial product derived from the Alto was the Xerox Star.) – In other words: The Alto was ahead its time, or it wasn't yet the time for this revolution. – The Alto was not yet a PC, esp. because it was not affordable! ● In 1979, Steve Jobs and some fellows from Apple visited PARC and attended a demo of the Alto. – Jobs and his team were impressed and instantly understood the outstanding meaning of having a GUI. – In took until 1983, but Apple made the computer a consumer product: the Apple Lisa. – But the Apple Lisa was too expensive (10,000$). (Em, Steve's fault!) ● Only about 1000 pieces of Xerox Alto were sold. ● The Xerox Star was sold at ~16500$ ● Only about 11,000 – 50,000 pieces of the Lisa and about 50,000 pieces of the Lisa 2 were sold.
  • 20.
    20 The Takeoff ofInteractivity – Breakthrough ● The real breakthrough of interactivity on consumer computers was theApple Macintosh (Mac) in 1984. – A revolutionary TV commercial during a break of the telecast of the Super Bowl (1984-01-22) started to attract potential users. – Affordable at ~2,500$. – All in one computer (Motorola 68000 CPU), keyboard, mouse and screen (512×342 monochrome). – The revolutionary system software Mac OS (the ancestor of OS X), which provided the user interactivity. ● The Mac introduced a new class of applications based on the desktop metaphor (applications beyond "number crunching"). – Desktop Publishing (DTP) – What You See Is What You Get, WYSIWYG ("wizzy wig") – (Before that, CUI applications were basically applications using the file system.) ● After the desktop metaphor was established it was mostly only copied by other companies for more than four decades. – VisiCorp VisiOn (1983) – Digital Research GEM (Graphical Environment Manager) (1984) – Microsoft Windows 1.0 (1985); the real takeoff was in 1990 with Windows 3.0 – X Window System (1984) ● The keyboard of the Macintosh in 1984 didn't even have arrow keys! - Obviously Apple's engineers thought that all users will prefer the mouse. And it took until the 1990ies, when Apple applied the "inverted-T" layout of the arrow keys. ● Windows 2.0 was introduced 1987 and it allowed overlapping windows.
  • 21.
    21 Not discussed Topics ●X Window System and multi user support ● Voice recognition ● Touch
  • 22.
    22 History – PartI ● Users were white collar workers. Input: plugging cables; output: lightbulbs. – Users were programmers! ● Users applied batch processing. – Users needn't to be programmers any longer! – Input: a batch of punched cards/tapes (code and data); output on a printer (continuous printing). ● Users sat in front of a terminal. A terminal was one device for input/output (tty), later separate devices (glass ttys). – Users had a session with mainframe, which was controlled from the terminal via a network! – One dimensional interactivity: line-by-line communication, one program for one task. – Bare command-line: full control, less comfort. Command-line prompt: less control, some comfort. ● Two dimensional applications allowed moving the cursor in two dimensions and using menus. – Users worked with a standalone PC. – Metaphors (simulations of the reality) emerged in applications: document and spreadsheet. Multipurpose applications. – Compromise of control and comfort. ● It should be said that there is a tendency to reduce the freedom of using multipurpose applications on the desktop somewhat. The idea is to guide users through using certain applications at a certain time for a certain task, along a and as part of a so called business process. ● This tendency is still ongoing and evolving, because the technical infrastructure grew in the meantime: cheaper network throughput, clouds, applications able to access cloud services etc. ● This tendency merges up with web-applications, esp. with three-tiered applications, on which much processing power and handling multiple user sessions reside on a server. - In a sense it means a step "back" to the terminal/mainframe idea!
  • 23.
    23 History – PartII ● The desktop metaphor. Input: keyboard, mouse; output: screen, printer/WYSIWYG. – Users controlled applications with the mouse via point and click. Ultimate two dimensional usage with windows and controls/forms. – Control and comfort were mixed up completely, because applications could be used by "guessing", i.e. ergonomics. ● Emerging UI aspects (read: not only graphical aspects): – Touch-gestures (killer app: Apple iOS) – Haptic feedback (Apple's Force Touch Trackpad) – Physical-gestures (killer app: Nintendo EAD Wii Sports) – Voice recognition (killer app: Apple's Siri) – Motion detection – Ambient detection ● Nintendo EAD, EAD for Entertainment Analysis & Development. ● Meanwhile we are confronted with further usage paradigms and emerging metaphors because of new applications: mobile devices, smart devices (e.g. TVs) support of disabled persons and the internet of things.
  • 24.