Augmented Reality is one of the most exciting technologies of the future. This was also my area of interest for the Final Year Seminar at Nitte.

1. Bridging the Real
World and the Virtual
World
A look at Augmented Reality and
Ubiquitous Computing
Date 15th February, 2010 By Karthik Kastury
Monday 5 July 2010

2. The Past 20 Years of
Computing
Monday 5 July 2010

3. For How Long have
we been computing
this way?...
Paradigm Shift in Computing is here and
its Augmented Reality
Monday 5 July 2010

4. Augmented Reality
Whats that?
Monday 5 July 2010

5. Watch this clip...
From Iron Man the Movie
Monday 5 July 2010

6. Monday 5 July 2010

7. Yes, that’s
Augmented Reality
And don’t worry its not only in movies,
but real life as well...
Monday 5 July 2010

8. Augmented
reality
(AR)
is
a
term
for
a
live
direct
or
indirect
view
of
a
physical
real-­‐world
environment
whose
elements
are
merged
with
(or
augmented
by)
virtual
computer-­‐generated
imagery
-­‐
crea<ng
a
mixed
reality.
Monday 5 July 2010

9. Augmented Reality
Techniques
• Heads Mounted Display
• Handheld Devices
• Spatial Displays
• Modern Software Based AR
Monday 5 July 2010

10. Head Mounted Display
• User can simultaneously view both
the real world and the virtual world.
• Optical See through or Video See
Through.
• Can be a bit bulky in its current form.
Monday 5 July 2010

11. Handheld Devices
• Most commonly used AR Devices
• Uses a variety of Sensors
• ARToolkit is usually used to map the
real world.
Monday 5 July 2010

12. Spatial Devices
• User does not carry any device with
him.
• The data is projected onto real world
entities.
• Multiple Users can interact with the
system.
• Can support large areas of displays.
Monday 5 July 2010

13. Software Based AR
• Use multiple sensors and tracking
devices, like GPS, RFID, Compasses
etc...
• Are leading the innovation in AR now.
• Two Stages, Detection of Interest
Points, Mapping the two worlds.
Monday 5 July 2010

14. Working Principle
• First interest points, or fiduciary
markers, or optical flow detected in
the camera images [Corner
Detection].
• Second, a real world coordinate
system is restored from the data
obtained in the first stage[SLAM].
Monday 5 July 2010

15. Corner Detection
• Used to identify the interest points.
• Can use Correlation Algorithms, but
they tend to be very intensive
computationally.
• So we use Harris and Stephens Corner
Detection algorithm
Monday 5 July 2010

16. Harris and Stephens
Corner Detection
Algorithm
Assume a grayscale 2-dimensional image is used. Let this image be given by
I. Consider taking an image patch over the area (u,v) and shifting it by
(x,y). The weighted sum of squared differences (SSD) between these two
patches, denoted S, is given by:
I(u + x,v + y) can be approximated by a Taylor expansion. Let Ix and Iy be
the partial derivatives of I, such that
This produces the approximation
Monday 5 July 2010

17. contd...
which can be written in matrix form
where A is the structure tensor,
This matrix is a Harris matrix, and angle brackets denote averaging (i.e.
summation over (u,v)).
By analyzing the eigenvalues of A, this characterization can be expressed in
the following way:
If and then this pixel (x,y) has no features of interest.
If and λ2 has some large positive value, then an edge is found.
If λ1 and λ2 have large positive values, then a corner is found.
Monday 5 July 2010

18. Simultaneous Localization
and Mapping
• SLAM is a technique to build up a map within an
unknown environment (without a priori knowledge)
or to update a map within a known environment
(with a priori knowledge from a given map) while at
the same time keeping track of the current location.
• Maps are used to determine a location in an
environment and to depict this environment for an
overview.
• Simultaneous localization and mapping (SLAM) is a
concept to bind the process of locating and mapping
in a loop and therefore supports the continuity of
both aspects in separated processes.
Monday 5 July 2010

19. So this how AR
Systems work...
Now let us have a look at a Software
Based AR Tool for the iPhone/Android
Monday 5 July 2010

20. Monday 5 July 2010

21. Other Applications of AR
• Support with complex tasks
• Navigation device
• Military and emergency services
• Sightseeing
• and
many
more...
Monday 5 July 2010

22. Ubiquitous Computing
• Ubiquitous computing (ubicomp) is a post desktop
model of human-computer interaction in which
information processing has been thoroughly
integrated into everyday objects and activities.
• This paradigm is also described as pervasive
computing, ambient intelligence. When primarily
concerning the objects involved, it is also physical
computing, the Internet of Things, haptic computing,
and things that think.
Monday 5 July 2010

23. C9*.% 1":"6*#0% *.% "8.&% 6$./:*)$6% ".% 1$:=".*=$% /&01,-*2#A% "0)*$2-% *2-$88*#$2/$?%
Challenges of Ubiquitous
D9$2%1:*0":*8<%/&2/$:2*2#%-9$%&)>$/-.%*2=&8=$6A%*-%*.%"8.&%19<.*/"8%/&01,-*2#A%-9$%
@2-$:2$-%&'%C9*2#.A%9"1-*/%/&01,-*2#A%"26%-9*2#.%-9"-%-9*27?%E"-9$:%-9"2%1:&1&.$%
"% .*2#8$% 6$'*2*-*&2% '&:% ,)*+,*-&,.% /&01,-*2#% "26% '&:% -9$.$% :$8"-$6% -$:0.A%
Computing
-"F&2&0<% &'% 1:&1$:-*$.% '&:% ,)*+,*-&,.% /&01,-*2#% 9".% )$$2% 1:&1&.$6A% ':&0%
;9*/9% 6*''$:$2-% 7*26.% &:% '8"=&:.% &'% ,)*+,*-&,.% .<.-$0.% "26% "118*/"-*&2.% /"2% )$%
6$./:*)$6?%
%
%
%
G-% -9$*:% /&:$A% "88% 0&6$8.% &'% ,)*+,*-&,.% /&01,-*2#% 3"8.&% /"88$6% 1$:=".*=$%
/&01,-*2#4% .9":$% "% =*.*&2% &'% .0"88A% *2$F1$2.*=$A% :&),.-% 2$-;&:7$6% 1:&/$..*2#%
Monday 5 July 2010 6$=*/$.A%6*.-:*),-$6%"-%"88%./"8$.%-9:&,#9&,-%$=$:<6"<%8*'$%"26%#$2$:"88<%-,:2$6%-&%

24. "//%+,$%9"7%+&%+,$%+)"26%0"35%:&-2$%)$2&#.-+-&.5%".3%-.+$)'"2$4%3$4-#.$3%+&%*"6$%-+%
%&'()*'&+(,-./01-12)*3'4-
$"4-$)%'&)%+,$%3-4"8/$3%+&%14$%2&*01+$)45%2&*01+-.#%-.+$)'"2$4%,":$%0)&#)$44$3% %&'()'*+,-&&,
Redefining the Vision of
)"0-3/7%9-+,-.%+,$%/"4+%'$9%3$2"3$4;%
%
<1+%+,-4%-4%.&+%+,$%$.3%&'%+,$%)&"35%+,$%(1+1)$%&'%=&*01+-.#%/-$4%-.%+,$%*$)#$)%&'%
+$2,.&/&#-$4%/-6$%>1/+-?&12,5%@$4+1)$45%A&-2$%=&.+)&/5%B1#*$.+$3%C$"/-+75%<)"-.%
Future Computing
=&*01+-.#%".3%,&4+%&'%&+,$)%)$4$")2,%")$"4%-.%.$:$)%8$'&)$%4$$.%9"74;%
%
B+% +,$% =DE% FGHG% I=&.41*$)% D/$2+)&.-24% E,&95% J"4% A$#"4K% 2&*0".-$4% 1.:$-/$3% "%
,&4+% &'% +$2,.&/&#-$4% +,"+% 9-//% '&)$:$)% 2,".#$% +,$% /".342"0$% &'% 2&*01+-.#;% L$)$%
")$%4&*$%&'%+,$%+$2,.&/&#-$4%+,"+%9-//%8$%2&**&.0/"2$%-.%&1)%/-:$4%M14+%/-6$%+,$%
+)"3-+-&."/%3$46+&0%-.+$)'"2$4%,":$%8$$.%'&)%+,$%0"4+%FG%7$")4%&)%4&;% %
%
%&'()'*+,-&&, %
%&'()*'&+(,-./01-12)*3'4- 5678+(,+9-:+'32,4-2(-'-;<(,'=,-/+()-
,
./+0,12+3)4(,%5(56,
% %
Transparent OLED
%
5678+(,+9-:+'32,4-2(-'-;<(,'=,-/+()-
AR In a Contact Lens
!"#$%!"%&'%#$%
,
, % 7&42+8+9(,:;2954),<+=>;(&'?,
./+0,12+3)4(,%5(56,
%,
XBox Project Natal
7&42+8+9(,:;2954),<+=>;(&'?,
!"#$%!"%&'%#$% ,
% Microsoft Surface Computing
!"#$%!"%&'%#$%
%
Monday 5 July 2010

25. Office Labs Vision 2019
Monday 5 July 2010

26. Final Words...
• The most profound technologies are those
that disappear. They weave themselves into
the fabric of everyday life until they are
indistinguishable from it.
• Augmented Reality and various principles of
Ubiquitous Computing will drive innovation
in the future and take computing to an
extent that our future generations will no
longer be able to relate themselves to the
initial forms of computing.
Monday 5 July 2010

27. References
• Azuma, R. A Survey of Augmented Reality, Presence, 6, 4, pp.
355-385, 1997
• "If You’re Not Seeing Data, You’re Not Seeing". Wired Magazine
• Hand of God - Wearable Computer Lab, South Australia
• Mark Weiser - Ubiquitous Computing, during his term at XEROX-
SPARC
• Pervasive Computing – M Satyanarayan, School of Computer
Science, Carnegie Mellon University
• Collaborative Augmented Reality - Mark Billinghurst, Hirokazu
Kato
Monday 5 July 2010

28. Thank You
Do you have any questions?
Monday 5 July 2010