Smart vision systems embed intelligence through advanced image processing to enhance visual experiences, improve interactions, and facilitate decisions. They require increasingly complex algorithms to perform tasks like 3D reconstruction, gesture recognition, and event detection. These algorithms must be matched to diverse platform architectures with different performance and power constraints. Successful system design requires matching applications, algorithms, and architectures. The document discusses examples including 3D video interpolation, eye-gaze corrected video chatting, and a 3D camera prototype for elderly monitoring. It concludes that IBBT brings together competences in applications, algorithms, and architectures to enable new smart vision systems.

1. Smart Vision System:
Design of Application
Algorithm-Architecture
Eric Delfosse
IBBT-NES-IMEC

2. What are Smart Vision Systems?
Systems that embed intelligence through advanced
image processing to:
Enhance visual user experience
Improve natural interaction
Facilitate decision making for complex events
…

3. Enhance visual user experience
Multi-camera 3D image reconstruction for advanced
surveillance
3D GPS
(IBBT GBO ISYSS)
(IBBT GBO URBAN)

4. Increase natural interaction
Natural human-machine interface
through gesture recognition
3D (immersive) video (IBBT GBO Hi-Masquerade)
conferencing

5. Facilitate decision making for complex events
Event detection for
surveillance applications
(IBBT GBO ISYSS)
Traffic sign recognition for
driver assistance
(IBBT GBO URBAN)

6. Different applications, different requirements
Throughput
High resolution
High framerate
Functionality
Accuracy
Reduce false negatives
and false positives
…
Size Low Power
Solar energy
Avoid active cooling
...
Cost

7. These applications require increasingly complex
algorithms
Exponential algorithmic complexity increase
> 9000
3000
2500
Complexity (Ops/pixel)
2000
1500
1000
500
0
enhancement
extraction
Depth extraction
Edge detection
AVC encoding
Object detection
understanding
Feature
Image
Image
Algorithm

8. Diversity of platform architectures with different
characteristics
PC + GPU
High performance
High power
Limited portability (laptop)
…
Server rack
Embedded systems
Very high performance
Very high Power Medium performance
Non-portable Low power
… High portability
…

9. Smart Vision Systems design =
Matching Application – Algorithm - Architecture
Application
(Requirements)
Algorithm Architecture
(Complexity) (Constraints)

10. Smart Vision Systems design =
Matching Application – Algorithm - Architecture
Application
(Requirements)
Algorithm Architecture
(Complexity) (Constraints)

11. Smart Vision Systems design =
Matching Application – Algorithm - Architecture
Application
(Requirements)
Algorithm Architecture
(Complexity) Introduce parallelism
(Constraints)
Processor optimizations

12. Complexity - Quality trade-off: reduce complexity with
limited (visual) quality loss

13. Matching Algorithm and Architecture: the DCT example
f0 F0
f1 F1
f2 F2
f3 F3
f4 F4
f5 F5
f6 F6
f7 F7
Classical DCT Butterfly DCT
64 multiplications 20 multiplications
64 additions 26 additions
High regularity Low regularity
Ideal for: Ideal for:

14. Smart Vision Systems design =
Matching Application – Algorithm - Architecture
Companies
Application
(Requirements)
IBBT
Algorithm Architecture
(Complexity) (Constraints)

15. Conclusions
Smart Vision Systems:
Enable new applications
Require new complex algorithms
Use diverse platform architectures
Successful design requires competences on these 3
aspects
IBBT brings these competences together

16. Demo’s

17. Demo 1: 3D video through real-time viewpoint
interpolation
Viewpoint interpolation:
a) stereo capturing
b) depth extraction
c) interpolation
Autostereoscopic displays:
require multiple
(interpolated) views

18. Demo 2: Eye-gaze corrected video chatting
Webcam
Computer
Display

19. Demo 3: Novel 3D Camera prototype and monitoring
application in elderly environment