The document describes research being conducted at the German Research Center for Artificial Intelligence (DFKI) on computer vision technologies for 3D reconstruction, virtual try-on, hypervideos, and gesture recognition to enable applications in fashion, e-commerce, and natural interfaces. DFKI has over 400 researchers working across various areas including augmented reality, robotics, language processing, and more.

1. Technologies for fashion
Constantin Thiopoulos

2. DFKI – German Center of Artificial Intelligence
 423 Researchers
 Kaiserslautern -
Saarbrücken,
Germany
 Research areas
 Augmented Vision
 Intelligent User Interfaces
 Agents and Simulated Reality
 Knowledge Management
 Language Technology
 Robotics Innovation Center
 Safe and Secure Cognitive Systems
 Innovative Retail Laboratory
 Institute for Information Systems
 Innovative Factory Systems

3. Department Augmented Vision @ DFKI
• Founded Juli 2008
• 30 fulltime researchers
• 3 strongly connected research Areas
Computer
Vision
HCI and Sensor
Visualization Fusion

4. Future Media GmbH
www.futuremedia-gmbh.com
A spin-off of the Department of Augmented Vision
of the German Research Center for Artificial
Intelligence

5. Realistic Digital Reconstruction
3D DIGITALIZATION

6. 3D digitalization: full pipeline
Geometry and
Digitalization Visualization
appearence modeling

7. Computer vision: 3D Reconstruction
1. Large scale modeling
• Large buildings (inside)
• Buildings / streets / cities (outside)
2. Object modeling with images
• Heavy objects
• Fixed position objects
3. Object modeling with structured light
• Extremely precise
• Movable objects
4. Object modeling with depth camera
• Low-cost, real-time scanning
• For normal users

8. 1. Large scale modeling
Bridge (Forbidden city)
8

11. 2. Object modeling with images
Input images Structure From Motion
3D model
Multiple View Stereo

12. Hardware

13. 3D Reconstruction Dome
• Outer sphere diameter:
2400mm
• Height: 2800mm
• Object weight: 50kg
• Object edge length:
800mm

14. 3D Reconstruction Dome
• 11 high-resolution
cameras
• 3 high quality projectors

15. 3D Reconstruction Dome
• Sandwich hull with 300
integrated LED ledges
inner hull: light strewing
fleece
outer hull: opaque
material
• Funnels for cameras and
projectors

16. 3D Reconstruction Dome
• Anti-glare glass turntable,
Degree of reflection < 1°
• Stepping- /multiphase
motor,
angular resolution > 1
million steps per turn

17. Current work
W. Lehmbruck
Female Torso (1918)
Material : Bronze
Sizes : 670x400x250 mm
Inv-Nr. : PFG 74/13
Photographed using artistic lighting conditions
with multiple light sources
As a loan by:
Museum Pfalzgalerie Kaiserslautern

18. Current work
W. Lehmbruck
Female Torso (1918)
Reconstruction
6
No. points initial : 630·10
No. points final : 22520
No. triangles : 37801
Map-Resolution : 4000 x 4000
Rendered using neutral lighting conditions and a
single light source with Firefox WebGL extension

19. Current work
W. Lehmbruck
Female Torso (1918)
Reconstruction
No. points : 22520
No. triangles : 37801
Map-Resolution : 4000 x 4000
Rendering by A. Kusch

20. Current work
H. Scherer
Little Girl (1924/25)
Material : Stone pine (painted)
Sizes : 760x380x405 mm
Inv-Nr. : P 04/98
Photographed using artistic lighting conditions
As a loan by:
Museum Pfalzgalerie Kaiserslautern

21. Comparison
Photograph WebGL Firefox 3ds Max Cycles photon tracer

22. 4. Object modeling with Kinect camera
• A 150$ scanning system
– One device
– Free motions
Yan Cui, Didier Stricker, 3D shape scanning with a Kinect
In: ACM Transactions on Graphics (Proc. of SIGGRAPH) 2011. 22

23. 3D reconstruction results with Kinect Camera
23

24. Next Generation e-Commerce
BODY SCAN AND VIRTUAL TRY ON

25. 3D body scanning @ home
• Low cost 3D scanning solution for
small shops and even at home!
 Full 3D body-scanning with a single Time
of Flight (ToF) or Kinect camera
 Automatic extraction of body measures
out of the 3D model
 3D shape and appearance (colored 3D
model) of the customer!
• 3D scanning with a single
camera is a unique technology

26. Online configurator - virtual clothes from designs
• Cooperation of designer and
customer
 Size adjustment
 Design, color and pattern
 2D and 3D viewer for interactive design

27. Virtual try-on
• Shopping experience and style selection
 Real-time visualization of clothes onto the
body
 Natural interaction with the mirror
 Saving of snapshots of the prefered clothes –
link to social networks, or send the image per
mail
 Online purchase with the correct (measured!)
size and personal design

28. Watch click and buy
EMBEDDED ADVERTISEMENT

29. Hypervideos
● Adding informations to objects in videos
● Text, images, hyperlinks etc.
29

30. Challenge: Fast & automatic creation
through robust object tracking
30

31. Technology: Video editor for object tracking
– Load the video
– Select the object(s) to be
tracked
– View the results and eventually
select new objects or do
corrections
– Add the content for the
interactive tags
– Export the results as interactive
videos

32. Technology: Video editor for object tracking
 Competitive advantage of the tracker based
on research results at DFKI
• High robustness to appearance
change, occlusion and strong
motions
• The object does not need to
be rigid, i.e. people tracking

33. Examples of tracking results

34. Deployment: final video
Watch Click Buy Product Web-site
• Tags and meta-data are encoded as video for standard Flash-
players
• Reporting statistics about clicks

35. Natural Interface
GESTURE TRACKING

36. Gesture Recognition

37. Architecture
Image Analysis Hand detection Gesture
and tracking recognition
Camera Gesture control Display