The normal interaction with computers is with keyboard and a mouse. For display a rectangular somewhat small screen is used with 2D windowing systems. The mouse was invented more the 40 years ago and has been for 20 years dominant input. Now we are seeing new types of input devices. Multi-touch adds new dimensions and new applications. Natural user interfaces or gesture interfaces where people point to drag objects. Computers are also beginning to recognise facial expressions of people, so it knows if you are smiling. Voice and natural language understanding is getting to a usable stage. All this calls all types of new applications. Displays are getting bigger. What if any surface was a screen? If you could spray the wall with screen? Or have you phone project images to the wall. This lectures explores some of these new types of interactions with computers and software. It makes the old mouse look old.

1. LECTURE L15
AUGMENTED AND VIRTUAL REALITY

2. Evolution of Computers

3. Mainframes Mini
Computers PC
Time-sharing
1960 1970 1980 1990 2000 2010 2020 2030

4. 1960 1970 1980 1990 2000 2010 2020 2030
Mainframes Mini
Computers PC
Clients
Time-sharing
Servers
Laptops
Server
rooms
Smartphones
Tablets
Music
players
Consumer Products
Data
centers
Mobile
phones
Convergence
Divergence
DecentralisationCentralisation
Mobility
eReaders
Wearables
IoT
New Business Models
Nearables
Internet of Me
Flyables
Utilities
AI

5. 1960 1970 1980 1990 2000 2010 2020 2030
Mainframes Mini
Computers PC
Clients
Time-sharing
Servers
Laptops
Server
rooms
Smartphones
Tablets
Music
players
Consumer Products
Data
centers
Mobile
phones
Convergence
Divergence
DecentralisationCentralisation
Mobility
eReaders
Wearables
IoT
New Business Models
Nearables
Internet of Me
Flyables
Utilities
What next?
AI

6. World fastest computer?

7. Tianhe-2 boasts a speed of 33.86 petaflops
one petaflop is one quadrillion floating-
point calculations per second

8. The Network is the Computer
The Internet cloud
More programs and data is stored on network
servers
The Personal Computer becomes one of the form
factors to access the network
Examples
Amazon API
Google Apps
Facework Platform API

9. Tom Watson was wrong – there
is not room for five computers.
It’s only one

10. What about Moore’s Law?
Shift to multicore started in
2005 – new dimension

11. Move to cloud-core
What about Moore’s Law?
Amazon has over 2.24 million machines in
some 87 data centers (2014 data)

12. USDbillions
$241
billion in
2020
33,000+ Data
Centers in the
world

13. Data Centers are
now a Software Problem

14. $1,279-per-hour, 30,000-core
cluster built on Amazon EC2 cloud
What about Moore’s Law?

15. Google used 16,000 computers to find cats
in YouTube videos
What about Moore’s Law?

16. Miniature Computers
Small devices that have computer power
Wireless capabilities
Dedicated devices
Enough computer power for limited functionality
Examples
RFID
UAV – Unmanned Arial Vehicles
Internet of things
Medical - nano robots

17. The Future of the PC
How long will the Hard Disk Drive last?
Solid state memory is getting bigger
Terabit Flash Memory
Computer architecture will change
More and more devices are
using Flash memory
Driving prices down

18. Tablets

19. Wearables, flyables, drivables, scannables…

20. What is the future of Personal Computers?

21. PC Evolution
1975 1980 1985 1990 1995 2000 2005
Hardware era
PC, Mac
Software OS era
Windows, Office, MacOS
Internet
Hardware Connects
IBM
Microsoft
Apple
2010
Software web era
Web 2.0, Social
2015
Internet of things
2020
Apple
Google
Smartphone
Wearable
Startups
Facebook
AmazonMicrosoft
Twitter

22. Consumer Devices
The computer and the
Internet
disappear

23. Consumer Devices
Any significantly
important technology
will disappear

24. Future of PC
Cloud Computing and home Serves
Everything will be connected: computer, TV, Phone, etc.
Simpler specialised operating systems
Utility computing
Wireless
Ubiquitous network access
Computers will be smaller and integrated into things
New input methods
Better interfaces
Big Brian, Small Brain
Better friction of devices

25. Future of the Desktop Metaphor

26. Early Computers
Goal: Data processing
Read data, process,
write or print

27. Early Computers
Result:
Dramatic Increase in
productivity
Automation

28. Xerox Early GUIs
Goal: Emulate the
office environment
in a computer

29. Macintosh 1984
First commercial
GUI system

30. Macintosh 1984
Result:
More User Friendly
Graphics, fonts, WYSIWYG
Laserprinter

31. Windows 3.0
Business software
migrates to GUIs

32. Windows 95
First consumer targeted
OS

33. Windows 95
Result:
More people, connection
to the Internet

34. Ubuntu
Mac OS X
Windows
More choices, less important
Android
Operating Systems Today

35. 35
Is the desktop metaphor dead?

36. 36

37. Basically file systems
Concept of files, loading and saving
Then with the Internet, the concepts
of uploading and downloading
These concepts are disappearing…
Multistep install process (parts made
by lawyers)
Traditional Operating Systems

38. 38
Forgot to save
your file?

39. Mac OS X Lion, 2011

41. What do people do with computes -
desktop, laptop, tablet, smartphone - today?
Communication - talk, messages
Reading
Take/View Photos
Watch Movies
Shopping
Browse the Internet
Read news
Write documents, blogs
Financial transactions
Learning

42. Microsoft Bill Buxton on
Natural User Interfaces
2010
42

44. “My specialty is people.”
– Bill Buxton
44

45. The Next Device

46. IBM 5150 - 36 years ago

47. Macintosh - 33 years ago

48. Desktop PC - Today

49. Laptops - Today

50. Smartphone Netbooks Laptop Desktop
The world in 2010

51. Bluetooth Devices
Lots of devices - communicating
Less friction - just works
Simple and easy to use with
bad security

52. Bluedouche
If it look ugly
on this guy,
it is ugly on you

54. Smartphone Netbooks Laptop Desktop
The world in 2010

55. iPad 2010 - “The iPad Effect”

56. The iPad Effect
Multi-touch input
Software Platform – apps
Real OS
The computer just became
a consumer device
Death of the File System,
Install process

57. The iPad Effect
Tablets are disrupting the netbook/laptop/PC market
Replacing eReaders
Killing papers and magazines - or saving them?
Change in life style
Is this new market disruption also?
What happens when you can access the internet with a $100 device?

58. SOFTWARE
WORK
FOCUS
APPS
VIEW/BROWSE
ENTERTAIN
APPS
CHECK
NOW!
VIDEO
WATCHING
RELAXED
DIGITAL WORLD

59. DIGITAL WORLD
http://wersm.com/facebook-wants-to-be-the-leading-second-screen-app/
87% TV viewers are using a device when watching TV
The traditional TV model
of scheduled TV does not
work anymore
The only thing that works in
TV is live TV and premieres
of major shows - due to Twitter

60. From Mary Meeker’s slide deck.

62. What’s the next device?

64. Device on your arm

65. Compared to Smartphone, the Smartwatch market is tiny

66. What’s the next device?
Answer:
Many objects around us will become
“smartobject”
Smartwatch is just one of them

67. Fitness trackers

69. Smart Leggings
Uses electrodes to
monitor the electricity
in the muscles
Measure muscle
fatigue

71. 71
Glasses

72. What happened to Google Glasses?

73. Product was primitive, low quality, very expensive
And not enough people wanted it, social issues
Marketing failure

74. Adjacent Possible
It was not the time - technology not ready

75. Will it come back?
Yes, in some form

77. Microsoft Hololens

78. 78
How can Augmented Reality improve
our life and work?

80. Augmented Reality

82. Virtual Reality Headset

83. Facebook owns Oculus
What is the vision for Facebook of the Future?

85. Adjacent Possible
Really - are phones ready for VR?

87. Future of Game consoles?

89. Microsoft Kinect

93. Gesture-control system
Leap Motion

95. Gesture Control bracelet
Myo

97. Voice Recognition

99. Family robots

101. Amazon Echo

102. Desktop, laptops Mobile, tablets, smartphones
BEFORE Now

103. 67
Desktop metaphor, mouse
keyboard
Gesture, audio, touch
BEFORE Now

104. NEXT:
BIG DATA AND ANALYTICS