Metadata & Interoperability: Free Tools

the future in hand | createasphere.com
Metadata & Interoperability: Free Tools

The Instructors
Roger Howard (rogerhoward@mac.com)
• The J. Paul Getty Trust, eVox, Playboy, Blizzard, North Plains Systems
• Digital media technologist, photographer, cataloger
• Software developer, workflow automator
• Always looking for better tools to make life easier for my users (and to make me
look better to my boss and, now, clients).

The Instructors
Mike Jennings (tymcode@gmail.com)
• Technicolor/PRN, Adobe Systems
• Quality Program Manager, Workflow Analyst, Metadata Maven
• Asset pack-rat

The Problem
Your DAM needs to evolve
• Business needs change
• Users demand refinements
• Measurable gains come from workflow improvements
• Out-of-the-box features/services sometimes disappointing
…but your budget can’t keep up

Your DAM is only as good as your metadata
Realizing the value of a DAM always comes down to
the metadata.
• Asset retrieval
• Business processes
• Reporting and analysis
• Integration with other business systems

…But it’s difficult to ensure metadata quality
• People don’t want to do it
• People aren’t very good at it
• People aren’t very consistent about it
• Integrated systems don’t always interoperate well

The Strategies
Add services to improve metadata quality and
workflow efficiency
• Validation – Make sure the metadata is correct, or at least in the correct format
• Automation – The less people have to do, the fewer mistakes they make
• Deduplication – efficient use of resources, metadata version control
• Simplification – minimize user interaction = happier users = buy-in

The Answer
DIY integration of free tools
• Lots of free/cheap/open-source tools to fill where the DAM leaves off
• Most DAMs support integrating external tools and services
• It doesn’t always require an integrator or professional coder
• No procurement = agility and simplicity
• We’re not lawyers, but using open source does not mean giving away your farm

The Tools
Community built is community supported
• Field tested, IT approved
• Typically cross-platform
• Often found “rolled into” trusted commercial products
• Robust communities and accessible developers for support
• Professional training or integrators usually available if you’re stuck

The Methods
Scripting
• Our favorite content creation tools support it with native scripting APIs
• Power tools with a command line interface are inherently scriptable
• Many good free tools support it
• Nearly all DAMs support scripting external tools in one way or another

The Scripts
Shell Scripts
• Usually little more than the name of the tool followed by the particulars
(“arguments”)
• Can get as elaborate as you need

The Scripts
Embedded JavaScript
• JavaScript and variants (e.g. Adobe’s ExtendScript and Flash ActionScript) are
similar – you learn one, you learn them all
• Can get as elaborate as you need
• Loads of great learning resources out there, and you likely have people on staff
already with the baseline skills

Intra-Application Automation
Content creation tools often include workflow
automation features
• Can be trivially simple to create and use
• Can extend to significant levels of sophistication
• Distributable among workgroups
• Can address all four strategies

Intra-Application Automation
Example: Adobe Creative Suite
• Photoshop actions: Easy and fun!
• Photoshop Event Scripting: JavaScript automation for processes and metadata
• Bridge scripting for automated workflows between Adobe applications
• Custom File Info panels, metadata schemas
• Flash-based custom palettes highly tuned to business needs, appear as bespoke
design

DAM Tooling
DAMs provide many paths to integrate external tools
• Direct calling of external tools, often used in ingest and delivery processes
• Native, Web Services and other APIs
• Product-specific integration, such as InDesign plugins
• Direct access to the underlying database

DAM Tooling
Example: Final Cut Server
• Metadata mapping pane for EXIF, IPTC, QuickTime or XMP metadata on ingest
• Executes scripts and executables post-archive, pre-restore, from poll watchers,
or subscription events (asset, job or production) – with parameters
• It is itself scriptable from the command-line

DAM Tooling
Example: Alfresco ECM and Adobe Drive
• Adobe Drive treats a version-controlled content repository as a mounted volume
usable within content creation apps
• Alfresco provides a “CMIS Connector” to interact with it in a standardized way
• Content creators can therefore use version control/asset management as easily
as not using it

Metadata Toolkit
Exiftool
• Cross-platform, free, extremely mature and frequently updated
• Supports 100+ file formats, from images to video to PDF
• Focused primarily on explicit metadata
• Extract, read, write, and sanitize embedded metadata
• Obscures many of the ugly details of embedded metadata

Metadata Toolkit
ffmpeg
• Primarily focused on being the universal video and audio format converter
• Can be used purely to analyze and report on file format characteristics

Metadata Toolkit
ImageMagick
• Primarily focused on being the universal image file format converter and
programmatic editor
• Also has extensive reporting capabilities for analyzing supported file
formats via the “describe” command

Metadata Toolkit
MediaInfo
• GUI for casual use, command-line for scripting and automation
• Features customizable reporting and metadata extraction capabilities
• XML output can be piped to other processes and DAMs

Metadata Toolkit
Adobe Creative Suite metadata tools
• Custom File Info panels
• Custom metadata schemas
• Custom Flash palettes
• Script Events Manager and Photoshop scripting
• BridgeTalk inter-application scripting

Adobe CS – File Info Panels
Adobe Creative Suite – File Info Panels
• Many creatives live and die by the Adobe product stack
• Did you know there’s a data entry interface built in to every CS
product?
• More importantly, did you know it’s insanely customizable?

Adobe CS – File Info Panels
Out of the box, they are one-size-fits-all, but…
•Easily customized for simple panels; tweaking an XML
file is enough to get started
•The real power is in completely custom panels
• Flash based
• XMP enabled
• Network connected
• The UI sky is the limit
• Build and maintain centrally for easy updates
• Modal

Adobe CS – Flash Palettes
• Build your own custom UI within Photoshop
• Uses existing Flash/ActionScript skills
• Modeless – floats in a palette, like other Photoshop elements
• Integrates XMP, Photoshop automation, Web services

Adobe CS – Script Events Manager
• Hooks scripts and actions to Photoshop events
• Start, New, Open, Save, Close, Print, and Export
• Enables workflow policy enforcement at the creative application level
• Validate or apply metadata on open or save
• Boilerplate metadata from templates; dynamic metadata (eg., time stamps for reporting)
• Apply watermarks during printing
• Apply scripts or Photoshop actions based on embedded metadata
• Extensive JavaScript and native scripting APIs
• The unsung hero of Photoshop workflow automation

Great Tools, So Now What?
Solving metadata problems
• Collecting (better) metadata within a creative workflow
• Improving embedded metadata extraction in your DAM
• Enforcing embedded metadata policies at distribution
• Generating better (or any) proxies/previews*
Leveraging metadata to streamline workflow
• Preflighting assets during ingest for formatting and naming compliance
• Integrating your DAM with desktop tools
* Yes, proxies are metadata

Metadata in Creative Workflow
• Critical metadata is often best known by those creating it
• Adding metadata after the fact can be costly and inconsistent
• Metadata capture is often seen as an onerous chore, best avoided
• Your job is to provide tools that…
• Make it easier to capture and contribute controlled metadata so that
• ...you can enforce policies about what data is captured, without making
enemies.

Metadata in Creative Workflow
• A Flash palette or file info panel as a view into your DAM
• Presenting key information about an asset, such as workflow status, job info, or editorial
comments, directly within Photoshop
• Leveraging the controlled vocabularies in your DAM to quickly tag assets from within
Photoshop
• Automatically populating metadata in an asset through a lookup
• Script Events Manager scripts as a compliance tool
• Check metadata, filenaming, and other criteria during opening, saving

Improving Metadata Extraction
DAMs often don’t extract the metadata you need. No one wants to enter it.
Most of the time they shouldn’t have to.
• Much of the needed metadata can be gleaned
• Tools can extract hidden metadata
• Scripts can provide context and workflow data (username, timestamp, approval status etc.)
• The more metadata you can extract for the content and context, the happier
everyone is

Improving Metadata Extraction
exiftool, ImageMagick, MediaInfo and ffmpeg can all provide extremely
robust metadata extraction capabilities, depending on your file formats and
metadata needs
• Run as part of an ingest process from within the DAM
• Run on a schedule, batch extracting metadata and inserting into your DB
• Run within the creative workflow, extracting and summarizing key bits

Enforcing Embedded Metadata Policies
• Embedded metadata is everywhere, in every file
• Embedded production metadata may bloat your distribution file sizes
• …And most of it has no value in distribution/publishing anyway
• Embedded metadata can be very revealing – too revealing
• Do you know what information is being leaked out of your DAM when
you distribute assets?

The embedded metadata whitelisting process…
•For each of your major file types and distribution processes, determine
what embedded fields should be preserved, e.g.:
• Copyright metadata
• Asset identifiers
• Captions, transcriptions, keywords, other blessed descriptors

• Add a step into your distribution processes – whether it’s within the
DAM, or simply a Folder Action attached to an FTP directory
• Using exiftool, delete all metadata *except* the fields you’ve
consciously chosen to preserve
• This way, you don’t have to keep up with what might be in there, selectively deleting; just
preserve what you know you want. It’s as simple as this:
exiftool -all= --Credit --XMP:Description

Generating (Better) Proxies*
• File formats change constantly, your DAM may not
• Your DAM can store any file, but may not generate useful proxies
• Your DAM is customizable, one way or another
• There are tools out there that can help your DAM create better proxies
*proxies are metadata, if you squint just right**
** I guarantee we’re not the only people here today who will abuse the term “metadata”, just the first

Digital Photography Proxy Issues
•Everyone shoots RAW, but no two vendors interpret that RAW file the same way
•Your creatives spend time adjusting their RAW files (in Lightroom, or PhotoMechanic, etc) to get
them looking just right
•When those RAW files are ingested into your DAM, they look nothing like the edits your
creatives made – if they can be displayed at all
•So you’ve resorted to workarounds, like generating a companion TIFF, negating much of the
benefit of the RAW workflow (or at least adding complexity)

Digital Photography Proxy Solutions
•The DNG workflow
• DNG repackages the RAW data into a universal container
• DNG still has all the flexibility of a RAW file, but just one file format to support
• DNG editors can maintain a JPEG preview, embedded in the DNG
• The JPEG preview reflects the exact look that was created in the editor – just like creating that companion TIFF,
but without the extra file
• JPEG previews can be full size, for high-resolution, edit-correct previews
• JPEG previews are easy to extract from a DNG, and insert into your DAM

Digital Photography Proxy Solutions
•The DNG workflow
• Use the Adobe DNG Converter tool to automate DNG conversion processes, or…
• Have the RAW editor convert to DNG while downloading files from the camera/cards
• If your DAM doesn’t already extract previews from DNGs, you can still..
• Use exiftool or dcraw to extract the JPEG preview and insert into your DAM
•Don’t want to go DNG, but at least want to generate viewable previews of your obscure camera file formats?
• dcraw can generate previews of nearly any camera file format you can find
• It may not match the exact tone and color the creative is looking for, but it’s something!

Video Proxy Challenges
•You’ve got a DAM with basic video format support, but it can’t properly preview some formats
•You don’t have a commercial video conversion product like FlipFactory
•You’re increasingly getting video assets in a huge range of formats, from Canon 5dMk2 to
iPhone files, to that obscure format your security cameras use

Video Proxy Solutions
•ffmpeg
• Supports reading and writing dozens of audio and video container formats, codecs, and
importantly the exponentially large mix of the two
• Can provide extensive format reporting
• Can generate streamable proxies in the most popular formats, such as flv and mp4

Ingest Preflighting with Metadata
• You have specific file format requirements in certain workflows, but your DAM
can’t see enough about the file to tell a TIFF from a TIFF.
• You want to check your ingested PDFs for compliance – are fonts
embedded? What PDF profile(s) are the files compatible with?
• Your DAM is customizable (have we said that before?)

Ingest Preflighting with Metadata
• ImageMagick’s “describe” tool in verbose mode can tell you everything you
want to know about your image file formats, including PDFs
• Jhove can provide validation of certain formats, including validating against
many of the most common PDF and TIFF profiles
• ffmpeg can provide detailed analysis of your audio and video file formats

Conclusion
• Format Support: Free tools can support new formats your DAM can’t
• Workflow automation: Free tools can make your DAM less “visible” (intrusive)
• Metadata Extraction: Free tools can improve your DAM’s searchability
• Metadata Processing: Free tools can prevent you from releasing private info
• Preflighting: Free tools can ensure compliance and quality standards
• Format Conversion: Free tools can fix problems in your asset library
Bottom Line: Free Tools Can Add Value to Your Existing DAM

Metadata & Interoperability: Free Tools

More Related Content

What's hot

Viewers also liked

Similar to Metadata & Interoperability: Free Tools

Recently uploaded

Metadata & Interoperability: Free Tools

Editor's Notes