IMF mechanisms can carry metadata in several ways: 1) In the Composition Playlist (CPL) using elements like <Annotation>, <ContentOriginator>, and <ExtensionProperties>. 2) Through external references in the CPL like <ContentVersionList> which links to metadata in other files/systems. 3) As intrinsic or extrinsic timeline metadata using virtual tracks and references to encode data like markers, technical metadata, or captions. Output Profile Lists (OPLs) allow processing CPLs and accessing metadata via references. Overall, IMF provides flexibility in storing and linking metadata both in and out of CPLs/MXFs depending on the data type and application.

1. EBU Production Technology Seminar, January 2017
Dan Tatut, VP Business Strategy & Development
IMF Mechanisms for carrying
Metadata
Current Practices

2. IMF mechanisms for carrying metadata
• CPL structure
• Intrinsic & Extrinsic metadata
• Metadata in track files
• Metadata processing via OPLs (Output Profile Lists)
• About timecode
• Conclusion / Topics for debate
OVERVIEW

3. IMF mechanisms for carrying metadata

4. IMF mechanisms for carrying metadata
• <Annotation> user-defined text (comments about/description of the composition)
• <Creator> software/device that created the composition (usually not user-editable)
• <ContentOriginator> who created the composition (e.g. studio name)
• <ContentTitle> title of the composition, often using a “naming convention”
• <ContentKind> defined in ST2067-3 or custom value (scope attribute should be used)
• <ContentVersionList> a list of ID/label pairs, references to external metadata
• <LocaleList> a list of <Locale> structures for language(s), region(s) and content maturity
rating(s)
What is available in the “Prologue section” ?

5. • <ContentVersionList> is a list of <ContentVersion> elements
• Each <ContentVersion> is a pair of <Id> and <LabelText> elements
• Each Id/LabelText pair can be anything, e.g.:
IMF mechanisms for carrying metadata
External metadata through “<ContentVersionList>”
OWNER FORMAT
Disney <Id>tag:disney.com,2015:wmls-id:7000001</Id>
<LabelText>In & Out (US English)</LabelText>
Warner Bros <Id>tag:warnerbros.com/series/mpmNumber,2004582</Id>
<LabelText>series(GILLIGAN'S ISLAND - YR02 65/66)</LabelText>
EIDR <Id>urn:eidr:10.5240:4FEC-7003-F150-AD59-9D70-S</Id>
<LabelText>This is an EIDR reference</LabelText>
ISAN <Id>urn:isan:aef0-1000-6721-022-x-0000-9034-1</Id>
<LabelText>This is an ISAN reference</LabelText>

6. • <EssenceDescriptor> elemets are part of <EssenceDescriptorList>
• Each <EssenceDescriptor> exposes in the CPL asset metadata using SMPTE RegXML
• Some assets can carry user defined metadata, e.g. MCA in the audio tracks
IMF mechanisms for carrying metadata
A word on “<EssenceDescriptor>” elements
MCA Metadata
...
<MCATitle>The Great Escape</MCATitle>
<MCATitleVersion>this is version 2</MCATitleVersion>
<MCATitleSubVersion>actually version 2.1b</MCATitleSubVersion>
<MCAEpisode>of the first episode<MCAEpisode>
...

7. • <ExtensionProperties> is the weapon of choice for adding static metadata to the CPL
• One unique instance of any custom-defined element
• Must be identified by their specific name spaces
• Can contain any sub-structure (as defined by their XML schema)
• Can be ignored by QC systems (unless they understand the schema)
• Can be stored internally or externally to the CPL (linking is extension specific)
• <ExtensionProperties> outside of the timeline, not well suited for time-based metadata
IMF mechanisms for carrying metadata
Adding metadata through“<ExtensionProperties>”

8. • CPL-embedded EBU extension for linking EBUCore documents as presented at PTS 2016
IMF mechanisms for carrying metadata
Examples using “<ExtensionProperties>”
<ExtensionProperties>
<ebuext:EBUExtension xmlns:ebuext="http://tech.ebu.ch/schemas/XXXX">
<ebuext:EBUCoreList>
<ebuext:EBUCore>
<Id>urn:uuid:7a5656b3-64b0-4dc7-960b-5a819c4669e5</Id>
<LabelText>France TV</LabelText>
</ebuext:EBUCore>
<ebuext:EBUCore>
<Id>urn:uuid:2945c709-8f76-491e-8aaf-bccb97bf99d8</Id>
<LabelText>UK DPP</LabelText>
</ebuext:EBUCore>
</ebuext:EBUCoreList>
</ebuext:EBUExtension>
</ExtensionProperties>

9. IMF mechanisms for carrying metadata
Examples using “<ExtensionProperties>”
<ExtensionProperties>
<mt:ExtensionProperties xmlns:mt="http://www.marquise-tech.com/imf/2016/cpl-extension-properties">
<mt:DM_AS_11_UKDPP_Framework>
<mt:UKDPP_Audio_Loudness_Standard>Loudness_None</mt:UKDPP_Audio_Loudness_Standard>
<mt:UKDPP_Textless_Elements_Exist>False</mt:UKDPP_Textless_Elements_Exist>
<mt:UKDPP_Picture_Ratio>16/9</mt:UKDPP_Picture_Ratio>
<mt:UKDPP_Tertiary_Audio_Language>zxx</mt:UKDPP_Tertiary_Audio_Language>
<mt:UKDPP_Audio_Comments>Mixed to PPM 6</mt:UKDPP_Audio_Comments>
<mt:UKDPP_PSE_Pass>PSE_Not_tested</mt:UKDPP_PSE_Pass>
...
<mt:UKDPP_Genre>Rock/Pop</mt:UKDPP_Genre>
...
<mt:UKDPP_Originator>BBC S&amp;PP DMS</mt:UKDPP_Originator>
<mt:UKDPP_Product_Placement>False</mt:UKDPP_Product_Placement>
...
</mt:DM_AS_11_UKDPP_Framework>
</mtextprop:ExtensionProperties>
</ExtensionProperties>

10. IMF mechanisms for carrying metadata
Examples using “<ExtensionProperties>”
<ExtensionProperties>
<imf:ExtensionProperties xmlns:imf="http://dalet.com/api/imf/2015-12/imp-metadata-extension">
<WB:WBExtensionProperties xmlns:WB="http://warnerbros.com/schemas/imf/2016/extensionProperties">
<Series xmlns="http://warnerbros.com/schemas/imf/2016/extensionProperties">
<Name>GILLIGAN'S ISLAND - YR02 65/66</Name>
<MpmNumber>2004582</MpmNumber>
...
</Series>
<Title xmlns="http://warnerbros.com/schemas/imf/2016/extensionProperties">
<MpmNumber>4034033</MpmNumber>
...
<EpisodeNumber>001056</EpisodeNumber>
<EpisodeSequenceNumber/>
...
</Title>
...
</WB:WBExtensionProperties>
</imf:ExtensionProperties>
</ExtensionProperties>

11. • IMF allows metadata extension at the timeline level. This is done through virtual tracks
• Can be intrinsic or extrinsic to the CPL
• Can be time varying or constant, contiuous or discrete.
• Extensions are identified by their XML namespace/schema
• Can be ignored by QC systems that do not understand them
• Must follow the timeline rules (tracks, segments, sequence, etc.)
IMF mechanisms for carrying metadata
Adding timeline metadata

12. M1 & M2
A1
Soundfield
5.1
Metadata
IMF mechanisms for carrying metadata
SEGMENT 1 SEGMENT 2 SEGMENT 3
V1
Sequence
Sequence
Sequence

13. IMF mechanisms for carrying metadata
Examples of intrinsic timeline metadata: markers
<MarkerSequence>
...
<ResourceList>
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" si:type="cpl:MarkerResourceType">
<Id>urn:uuid:ce398822-cd01-4eee-b725-965bc78ea3e1</cpl:Id>
<Annotation/>
<EditRate>24000 1001</EditRate>
<IntrinsicDuration>0</IntrinsicDuration>
<EntryPoint>0</EntryPoint>
<SourceDuration>0</SourceDuration>
<Marker>
<Annotation>SMPTE_BARS</Annotation>
<Label scope="http://vubiquity.com/dete/imf/2016/markerLabel">SMPTE_BARS</Label>
<Offset>0</Offset>
</Marker>
</Resource>
</ResourceList>
</MarkerSequence>

14. IMF mechanisms for carrying metadata
Examples of intrinsic timeline metadata: RDD6
<rdd6:RDD6Sequence xmlns:rdd6="http://bbc.co.uk/imf/rdd6/2017">
...
<ResourceList>
<Resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" si:type="rdd6:RDD6ResourceType">
<Id>urn:uuid:ce366622-cd01-4e2e-1115-7622c48ea3e1</Id>
<EditRate>25 1</EditRate>
<IntrinsicDuration>0</IntrinsicDuration>
<EntryPoint>0</EntryPoint>
<SourceDuration>0</SourceDuration>
<RDD6>
<FirstSubFrame>... </FirstSubFrame>
<SecondSubFrame>... </SecondSubFrame>
</RDD6>
</Resource>
</ResourceList>
</rdd6:RDD6Sequence>

15. IMF mechanisms for carrying metadata
Examples of extrinsic timeline metadata: TT commentary
<cc:CommentarySequence xmlns:cc="http://www.smpte-ra.org/schemas/2067-2/2016">
<Id>urn:uuid:0ea040c7-418f-4b3d-b27e-8b4b3da9bdfa</Id>
<TrackId>urn:uuid:b13a9521-df5b-403a-8634-cacf14094835</TrackId>
<ResourceList>
<Resource xsi:type="TrackFileResourceType">
<Id>urn:uuid:53c5dd91-9b31-4a89-a63b-b2d8d7112627</Id>
<EditRate>24000 1001</EditRate>
<IntrinsicDuration>4208</IntrinsicDuration>
<SourceEncoding>urn:uuid:3ef4dbcf-d4e1-a432-b4b0-38da2d96442f</SourceEncoding>
<TrackFileId>urn:uuid:3ef4dbcf-d4e1-a432-b4b0-38da2d96442f</TrackFileId>
<Hash>T8pvcuXA+GVQM/fc0DMlgoMwOSE=</Hash>
</Resource>
</ResourceList>
</cc:CommentarySequence>
• <SourceEncoding> references an <EssenceDescriptor> defined in the <EssenceDescriptorList>
• The actual “external metadata” is referenced by the <TrackFileId> element

16. IMF mechanisms for carrying metadata
Examples of extrinsic timeline metadata: ancillary data
<cc:AncillaryDataSequence xmlns:cc="http://www.smpte-ra.org/schemas/2067-2/2016">
<Id>urn:uuid:cc64ab4c-d7bf-4453-85a6-40f372bc4f6c</Id>
<TrackId>urn:uuid:cf4a166f-3110-4512-a3d7-8193102c0458</TrackId>
<ResourceList>
<Resource xsi:type="TrackFileResourceType">
<Id>urn:uuid:0739ffce-918b-496c-b2c7-f3d2ef06f3f2</Id>
<EditRate>24000 1001</EditRate>
<IntrinsicDuration>719</IntrinsicDuration>
<SourceEncoding>urn:uuid:0d28fd36-8485-490e-a06d-b2bdcebc8fbf</SourceEncoding>
<TrackFileId>urn:uuid:c1e8dfc3-495b-44ba-a436-35027d0db0a3</TrackFileId>
</Resource>
</ResourceList>
</cc:AncillaryDataSequence>
• <SourceEncoding> references an <EssenceDescriptor> defined in the <EssenceDescriptorList>
• The actual “external metadata” is referenced by the <TrackFileId> element

17. • Intrinsic timeline metadata is stored in plain XML in the CPL
• Extrinsic timeline metadata is stored using “any” encoding method as long as there is a MXF
wrapping defined
• Extrinsic timeline metadata can be transmitted as supplemental packages
• Extrinsic timeline metadata can have any size
• Extrinsic timeline metadata can be reused multiple times (multiple references)
• Dynamics are defined by the encoding. No “generic animation mechanism”
IMF mechanisms for carrying metadata
Adding timeline metadata

18. • Product placement
• Per-frame technical metadata (e.g. camera metadata, geo-localisation)
• Creative changes (pan & scan), colorimetry (ST2094), etc.
• Support for legacy essence formats (EBU STL, CEA 608/708, etc.)
• New accessibility tracks
• Live & post-live capture metadata
• Etc.
IMF mechanisms for carrying metadata
Examples of timeline metadata

19. • Instructions on how to “process” CPLs
• List of standard (SMPTE) & custom macros
• Each OPL references a CPL
• Can have <ExtensionProperties> too!
• Metadata access in the CPL via CPL UUID
• Metadata access in the CPL’s virtual tracks via UUID
IMF mechanisms for carrying metadata
Metadata processing via OPLs

20. IMF mechanisms for carrying metadata
Metadata processing via OPLs
...
<MacroList>
<Macro type="mtio:FileOutMacroType">
<Name>macro_9313321f-7f2b-4ced-939d-efbf54ca8dff</Name>
<mtio:InputImageSequence xmlns:mtio="http://www.marquise-tech.com/schemas/imf/2016/fileout-macro">
<Handle>cpl/virtual-tracks/a8bb2208-3915-43b1-ab51-1b8ea48ae8ff</Handle>
</mtio:InputImageSequence>
...
<mtio:RDD6Sequence xmlns:mtio="http://www.marquise-tech.com/schemas/imf/2016/fileout-macro">
<Handle>cpl/virtual-tracks/cff02228-faac-7543-dd23-cba6695aa3e2</Handle>
</mtio:RDD6Sequence>
...
<mtio:OutputFormat xmlns:mtfileout="http://www.marquise-tech.com/schemas/imf/2016/fileout-macro">
<mtio:Container>mxfas11ukdpp</mtfileout:Container>
</mtio:OutputFormat>
</Macro>
</MacroList>
...

21. IMF mechanisms for carrying metadata
How to choose a strategy?
PLACE ROLE GLOBAL TIMELINE
CPL-related Related to specific moments in
the timeline
INTRINSIC Essential to CPL interpretation, XML based,
smaller than the CPL
Use a child element in
<ExtensionProperties>
Store as a resource in a virtual
track (could be custom)
EXTRINSIC Opposite of intrinsic Stored as external asset (e.g. QC
report, EBUCore XML documents,
etc.
Stored as Track file, referenced
by Virtual Track in CPL (e.g.
ST2094)

22. • IMF does not make use of TIMECODE
• CPLs have an optional timecode metadata
• MXF supports timecode
• IMF MXFs can carry the timecode you want/like (e.g. original/source timecode)
• However timecode is asset-bound, not composition-bound
• Stop thinking tape
• Start thinking files, structured edit/playlists & assets
IMF mechanisms for carrying metadata
Last but not least: Timecode!

23. • Current practice: UNREADABLE & INCOMPETENT NAMING CONVENTIONS
• SMPTE 35PM50 working on a specification to “ease the attachement” of sidecar files
• EBU IMF-TV working on this subject too!
• Send comments/feedback, participate!
IMF mechanisms for carrying metadata
Conclusion

24. contact information
www.marquise-tech.com
dtatut@marquise-tech.com
THANK YOU