DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
Watch click and buy
1. Watch Click and Buy
Future Media GmbH
www.futuremedia-gmbh.com
A spin-off of the Department of Augmented Vision of the
German Research Center for Artificial Intelligence
(DFKI Deutsches Forschungszentrum für Künstliche Intelligenz)
http://av.dfki.de/
2. Watch Click and buy
• Watch: product placement
• Click: select product of interest in digital video by
clicking on it and get multimedia information
(embedded advertisement). Requires automatic
generation of hotspots in every frame through
object tracking.
• Buy: order product through a web shop
3. Hypervideos
● Adding informations to objects in videos
● Text, images, hyperlinks etc.
3
4. Challenge: Fast & automatic creation
through robust object tracking
4
5. Technology: Video editor for object tracking
– Load the video
– Select the object(s) to be
tracked
– View the results and eventually
select new objects or do
corrections
– Add the content for the
interactive tags
– Export the results as interactive
videos
6. Technology: Video editor for object tracking
Competitive advantage of the tracker based
on research results at DFKI
• High robustness to appearance
change, occlusion and strong
motions
• The object does not need to
be rigid, i.e. people tracking
8. Deployment: final video
Watch Click Buy Product Web-site
• Tags and meta-data are encoded as video for standard Flash-
players
• Reporting statistics about clicks
9. System Overview
• Front end
video editor for creation of hyper-videos
through object tracking
• Back end
Integration into video and ad-server with
reporting statistics (implementation using
VPAID – Video Player-Ad Interface)
10. Business Model
Product Content Aggregator Viewer
provider
Watch click
and buy
Technology enabler
Pay per click
Revenue sharing
11. Benefits for all
Content provider
new revenue stream
competitive pricing based on pay per click performance
placement of non branded products
Aggregator
Ads free surface
Viewer
inspects products on demand
12. Market Segments
TV shows
Video ads
Telenovelas
Trailers
Longer Videos
Video platforms
Social media
13. Google award for object
tracking technology
• The department Augmented Vision of DFKI
won a Google Award for the work on object
tracking for creating interactive hypervideos
• „TechTalk“ Presentation at Google Zürich with
participation of Youtube
