Axa Assurance Maroc - Insurer Innovation Award 2024
Computer Assisted Manufacturing
1. PISA
Production, Indexing and Search
of Audio-visual Material
Computer Assisted Manufacturing
Dieter Van Rijsselbergen, IBBT – MMLab – UGent
2. PISA - Overview
Computer Assisted Design Search Engine
Concept
Indexing
Script Editing
Retrieval
• Parse scenario Script Editing • Timecode based indexing
• Shooting script editor
• Geo-temporal reference
• Storyboard
• Taxonomy based indexing and search
Abstract • Facetted search
Information
Virtual Intelligent Analysis and
Model Driven Development: Modelling Analysis Quantization
• Setting (Stage properties, light) Interpretation
• Character • Character identification
• Synthetic Speech • Background categorisation
• Sound effects Virtual and identification
• Character animation Model • Topic and eventdetection
• Virtual camera
Quantization
Computer Assisted Manufacturing
Automated Reverse Engineering
Realisation Production • Shot segmentation
• Ingest Footage • Video footprint and reuse detection
• Editing • Biometric face detection
• Mastering • Background analysis
• Reproduction to alternative distribution channels • Speech-to-text
35
3. Computer Assisted Manufacturing
! Computer Assisted Manufacturing is about automation
of media production processes
! Based on extensive production metadata
! high level Product Engineering for narratives
! low level Manufacturing Engineering for actual
manufacturing specifications
! Using clever algorithms to fill in the gaps
! supported by Trouvaille
36
7. Rough Cut Editing
! Inclusion of 3-D previsualization into the
drama manufacturing process
! Use previsualization as shooting script
! Integration into the MAMS and Craft Editing systems
! Building evolving products
! mixture of previsualized and shot footage
! evolve from all-previsualized to all-shot composition
40
10. Computer Aided Repurposing (CAR)
! Automated Repurposing of Drama
! Different output channels
! HDTV
! SDTV
! iPod/Cellular
! True Universal Media Experience
! requires semantically sensible scaling/cropping
43
11. Repurposing Concept
! Automated Repurposing of Drama
! Spatial adaptation using semantic Objects of Interest
! as in “focus on Capt. Kirk”
! decided by creative staff in function of the narrative
! is a form of Manufacturing Engineering
! can be defined in early 3-D Previsualization
! preparation not only of shooting script
but also of adaptations
! parameters refined in Quality Assurance
44
12. Repurposing: How?
! 1) Determine where the semantic elements are
! Semantic Objects of Interest
are linked with regions of interest
! manual annotation
! automated
algorithms
! aided by
Product
Engineering
metadata!
Jenny Waldek
45
13. Repurposing: How?
! 2) Provide reframing instructions, e.g.:
! Object Orientation and Range
! “The Bad must be shown on the right…”
0.75
! Object Size
! “… and I want it to be a full shot of The Bad.”
Frame Size
Object of Interest
Size: (30%, 90%)
46
14. Repurposing: Channels
! HDTV
! Originally acquired
high definition material
! SDTV ! Mobile (iPod)
! Legacy applications ! Semantically
! Common pan/scan repurposed
! Editing
(AVID)
47