Presentation given at "Data Analytics Challenges" workshop in School of Mathematics, University of Leeds.

1. School of something
FACULTY OF OTHER
Computing
ENGINEERING
Video Analysis in Autonomous Systems:
Data Analytics Challenges
Krishna Dubba
Institute for Artificial Intelligence and
Biological Systems

2. School of Computing
FACULTY OF ENGINEERING
Leeds Activity Analysis Group
Computer Vision (Prof. David Hogg)
Knowledge Representation and Reasoning (Prof. Tony Cohn)

3. School of Computing
FACULTY OF ENGINEERING
Motivation:
“We are drowning in data yet starving for knowledge”
~ John Naisbitt

4. School of Computing
FACULTY OF ENGINEERING
Motivation:
● Are computers drowning in (video) data?
○ CCTV cameras
○ Personal digital video cameras
○ Video content on TV and Internet
○ In future: Google glass, autonomous cars, personal
robots

5. School of Computing
FACULTY OF ENGINEERING
Trixi
University of Hamburg
LUCIE
Leeds University Cognitive Intelligent Entity

6. School of Computing
FACULTY OF ENGINEERING
Motivation:
● Are computers starving for knowledge?

7. School of Computing
FACULTY OF ENGINEERING
Motivation:
● Applications:
○ Security and Surveillance
○ Intelligent autonomous systems (robots, cars etc.)
○ Content based video retrieval (instead of text tags)
○ Automatic script and commentary generation for videos

8. School of Computing
FACULTY OF ENGINEERING
Nature of Data:
● Images
● Each pixel in image is a tuple (R,G,B)

9. School of Computing
FACULTY OF ENGINEERING
Nature of Data:
● Videos (series of images)

10. School of Computing
FACULTY OF ENGINEERING
Nature of Data:
● Videos (series of images)
Third Person View

11. School of Computing
FACULTY OF ENGINEERING
Nature of Data:
● Videos (series of images)
Third Person View Ego-Centric View

12. School of Computing
FACULTY OF ENGINEERING
Nature of Data
● Sensor data such as laser, depth data etc (Kinect).

13. School of Computing
FACULTY OF ENGINEERING
Nature of Data
● Sensor data such as laser, depth data etc (Kinect).

14. School of Computing
FACULTY OF ENGINEERING
Nature of Data:
● Text (annotations, additional information from web)
● Verbal instructions

15. School of Computing
FACULTY OF ENGINEERING
Challenges:
● Supervised, unsupervised and semi-supervised learning

16. School of Computing
FACULTY OF ENGINEERING
Challenges:
● Supervised, unsupervised and semi-supervised learning
● Data comes from multiple sources and mainly aimed at
humans - Multidisciplinary approach

17. School of Computing
FACULTY OF ENGINEERING
Challenges:
● Supervised, unsupervised and semi-supervised learning
● Data comes from multiple sources and mainly aimed at
humans - Multidisciplinary approach
● Real time analysis: GPU processing
○ LUCIE has three kinects attached and needs a
separate computer for each kinect.

18. School of Computing
FACULTY OF ENGINEERING
Challenges:
● Supervised, unsupervised and semi-supervised learning
● Data comes from multiple sources and mainly aimed at
humans - Multidisciplinary approach
● Real time analysis: GPU processing
○ LUCIE has three kinects attached and needs a
separate computer for each kinect.
● Integrating low-level representation and high level
reasoning: Statistical Relational Models like Markov Logic
Networks

19. School of Computing
FACULTY OF ENGINEERING
Challenges:
● Supervised, unsupervised and semi-supervised learning
● Data comes from multiple sources and mainly aimed at
humans - Multidisciplinary approach
● Real time analysis: GPU processing
○ LUCIE has three kinects attached and needs a
separate computer for each kinect.
● Integrating low-level representation and high level
reasoning: Statistical Relational Models like Markov Logic
Networks
● Online learning and how learning affects the state of the
system.

20. School of Computing
FACULTY OF ENGINEERING
Thank You