1. Seman&cally
Linked
Photos
for
building
Visual
Web
Ramesh
Jain
Dept.
of
Computer
Science
University
of
California,
Irvine
jain@ics.uci.edu

2. Experiences
Life =
Events
+

4. Literal
Visual
Oral
Progress: Knowledge Creation and
Propagation

5. Oral

6. Literal

7. Visual

8.
Vannevar Bush 1945
Memex:As we may think.
Scaling up Knowledge tools for humans
The human mind works
by association.
The process of
tying two items
together is the
important thing.

9. Berners-­‐Lee:
Suppose
all
the
informa0on
stored
on
computers
everywhere
were
linked.
Suppose
I
could
program
my
computer
to
create
a
space
in
which
anything
could
be
linked
to
anything.

10. Web: Human experiences, knowledge,
and understanding captured using
associative links.

11. And
the
World
changed
forever.

12. Volume determines
organization.
Time
Number
Increasing volume of
documents brought us
to the World Wide Web
and Google

13. • Top 8 Apps in 2013 were Visual.
• Selfies are becoming a popular
communication tool.
• Significantly more time spent on
visual media.
• Photo Uploads continue
exponential growth.
Source:
h*p://digiday.com/pla5orms/5-­‐charts-­‐growth-­‐visual-­‐web/
;
h*ps://www.metabunk.org/threads/the-­‐growth-­‐of-­‐photography-­‐photos-­‐per-­‐year-­‐and-­‐
the-­‐illusion-­‐of-­‐frequency.2491/
21st Century Made WWW Visual.

14. Text
based
documents
are
not
NATURAL.
Language
and
Literacy
come
in
the
way.

15. Disruption: Capturing and sharing is
easier for photos than text.

16. Imagine if every photo and video
captured were connected to every
other.

17. Going
beyond
Gutenberg
Legacy:
Photos
and
Videos
are
New
Documents.

18. What is a camera?

19. Is
a
Smartphone
camera
s&ll
a
camera?
EXIF
data
is
all
metadata
related
to
the
Event.
• Exposure Time
• Aperture Diameter
• Flash
• Metering Mode
• ISO Ratings
• Focal Length
• Time
• Location
• Face
Smartphone
camera
captures
events.

20. Events
are
‘Connectors’
Events
create
‘Context’
Causal
Experiential
Structural
People
Things
Places
Time
Experiences
Events

21. EMPT: Extractable Mobile Photo Tags
Exif
Content
Analysis
EMPT
People
Place
Objects
Events
…

22. EMPT
Provides
Implicit
Seman&c
Links
for
a
Photo.

23. Next Generation Photos.
Mee&ng
at
Dagstuhl.
Gardens
by
the
Marina
Bay.

24. Photo is an Experiential component of a
Moment.
Photos
and
videos
are
experien&al
data
from
an
Event.
People
Place
Events
Objects
Each
Photos
has
different
types
of
implicit
links.

25. How
much
is
a
Picture
worth?
Mee&ng
at
Dagstuhl.
A
SEMANTICALLY
LINKED
PICTURE
IS
WORTH
A
THOUSAND
SEMANTICALLY
LINKED
PICTURES.

26. Time
1-­‐
dimensional
Space
Events
Happen

27. Time
1-­‐
dimensional
Space
Events
Are
Connected

28. Event Base
Event
Presentation/
Navigation
Control
Event Grouping, Linking, Assimilation
Atomic
Event
Entry
EXIF
Features
Tags/
Context
Photo
stream
Segment.Event
Ontology
User
Annot-
ations
Story
Telling
Search
Control
Photos
and
Events

29. Time: 9AM on Feb 6, 2015
Location: Anaheim Marriott
EXIF: Indoor Scene; Face
Keynote Talk
at IEEE-ICSC
2015
by Charles Elkan
on
Theory versus
Practice in Data
Science
Annotations
Calendars
Environment
Conditions
Application
Knowledge
Geographic
Landmarks
Camera
Parameters
Event:
Keynote Talk at ICSC,
Speaker: Dr. Charles Elkan

30. Snapviz Demo: Your Personal Web
www.snapviz.com
Android
App

31. iOS
Android.
Krumbs:
A*aches
EMPT
and
creates
a
message.

32. Too
Many
connec&ons
for
each
photo
–
how
to
select
the
‘RIGHT’
ones.
• Ranking
Photos
– Quality
– Diversity
– Coverage
– Other
Contextual
Factors

33. • Addressing (URI)
• Transfer Protocol (HTTP)
• Authoring and Presentation (HTML)
• Ranking
• Contextual Processing
• Content Analysis
• Privacy and Security
• Information Vs Experience
Challenges

34. Sky
is
the
Limit.
Applica&ons:
Lifestyle
Health
Commerce
Surveillance
and
Monitoring
Agriculture
Research
and
development
Construc&on
…