1. Python for Visual Effects
and Animation Pipelines
A Case Study of Tippett Studio's
JET
Russell Darling
russell@tippett.com
2. Tippett Studio
Tippett Studio is an Academy
Award®-winning visual effects
company specializing in
character animation and
computer-generated imagery for
movies and television
commercials. Founded in 1984
by animation pioneer Phil Tippett,
it now employs over 200 talented
artists and technicians in
Berkeley, California.
3. Recent & Upcoming Films
Tippett Studio’s work was
most recently featured in
Charlotte’s Web, The Santa
Clause 3: The Escape
Clause and Pirates of the
Caribbean: Dead Man’s
Chest. It is currently in
production on two feature
films: Disney’s Enchanted
and Paramount’s The
Spiderwick Chronicles.
6. Visual Effects & Animation Pipeline
• A pipeline is an "assembly line" of software
used to organize, automate and facilitate
studio production workflow to enable the
creation of computer-generated imagery.
In simple terms: step-by-step process by which
CG images are created.
• Most pipeline work is typically done within a
distributed computing environment (Render
farm)
8. Hellboy – Skin & Muscles
New proprietary skin & muscle system.
1. Solve any skin dynamics within a
given scene file.
2. Solve any cloth dynamics within a
given scene file.
3. Check-in solved files and scene into
the database.
4. Convert the solved scene file to
multiple RIB files for the length of
the animation.
5. Render each RIB file.
6. Composite each rendered file with
other possible images and/or plates.
7. Create and send dailies of
composited images.
9. Animator’s Ideal Pipeline Tool
•Need to provide a tool which has a simple user interface.
•Automate as many steps as possible.
•Insert new processes in overall studio pipeline.
10. Legacy Pipeline
• Outdated/limited pipeline.
• Custom tools and scripts for each
category of artist.
• Difficult to maintain, enhance and upgrade.
• Poor user interfaces.
• Error prone.
• Not cross-platform.
• Written primarily in Perl.
11. New Pipeline Requirements
• Simple and easy user interface for artists while also
supporting a more advanced interface for technical
users.
• Ability to easily adapt to continually changing
production requirements.
• Cross-platform.
• Rapid implementation.
• Small software development group cannot support
unique and diverse set of tools that basically do the
same thing.
• Robust interface with commercial VFX software
(Maya, Renderman, Shake, etc.)
12. Solution - JET
JET is a system comprised of software
tools and scripts used to implement a
pipeline in the distributed computing
environment at Tippett Studio. The entire
JET system, including artist tools and the
underlying engine, is implemented in
Python.
14. Why Python?
• Rapid development.
• Cross-platform.
• Can be used both for software tools and “glue”
scripts in the pipeline.
• Ease of customization.
• Automated code generation.
• Python objects.
• Plus all the other good stuff we all like about
Python.
15. Dynamic User Interface
• Felxible & highly customizable -- Dynamic
XML-based user interface templates that can
be modified to suit specific types of artists
(animators, painters, technical directors, etc.)
or production needs without modifying the
core application software.
• Simple & clean -- Users only have access to
those options which are relevant to their
department and production.
16. Pipeline Chunks
• Modular "chunks" perform the each of the tasks in the pipeline
(rendering, compositing, etc.). A chunk is a subset of the overall
pipeline.
• Designed to be plug-n-play. Artist can select any combination and
they automatically configure themselves to work with each other by
forming appropriate pipeline connections.
• New chunks can be added at any time to support production
requirements.
• Implemented as centrally located Python objects.
17. Batch Jobs
• A “job” is a unique, individual packaging of chunks.
• JET automatically generates custom Python scripts
which form unique pipelines for each CG job.
• Can be submitted in a batch to the Render farm or
executed locally.
18. Design
The core JET user application
can be divided into two primary
modules:
– User Interface Builder
– Script Reader/Writer
19. UI Builder Module
• Primary purpose is to gather information about the job
configuration and to display this information to the user
through an editable GUI.
• User may view this information and edit the job
configuration through the GUI as he/she needs.
• Queries database for configuration value choices to offer
the user.
• GUI Implemented with PyQt
20. Script Reader/Writer Module
• Primary purpose is to write job configuration received from
the UI Builder into a set of Python scripts.
• Scripts generated by JET can then be submited to the
Render farm for processing.
• Reads in pre-existing script files, parsing the job
configuration saved within these files.
21. Chunk Templates
For every defined chunk, the there is corresponding
template file. It defines chunk objects that operate
only on data passed to the object. This data given to
the chunk objects originate from the data stored in
the job configuration and is written to the batch
scripts output from JET.
Each chunk template file will contain a header
describing the data fields needed by the particular
chunk. If multiple chunks share job configuration
data, the field names and types must be identical in
the batch template files as well as the scope being
defined as shared.
22. Chunk Template Format
#HDR: type | default_name | section
#REV: cvs revision (automatically populated)
#ATH: author name
#DSC: description
#REQ: requirements
#VAR: variable | type | scope | default value | description
# type: bool, int, enum, float, string, list, dict
# scope: local, shared, parent, group, all
class myChunk(jetChunk):
def __init__(self, vars, name):
jetChunk.__init__(self, vars, name)
def main(self):
pass
23. Example Chunk Template
#HDR: makeMayaRib | hero | makerib | MakeRIB
#REV: $Revision: 1.27 $
#ATH: Russell Darling
#DSC: Creates RIB from Maya scene by executing "maya2rib".
#REQ: maya
#VAR: SHOT | string | global | None
#VAR: mayaScene | string | parent | "" | Complete filepath of Maya scene.
#VAR: prmanVersion | string | shared | /usr/local/prman
#VAR: shadows | bool | local | True
class makeMayaRib(jetChunk):
def __init__(self, vars, name):
jetChunk.__init__(self, vars, name)
# run the actual makeRib command
def makeRib(self):
uh.checkFile(self.mayaScene)
# MAIN
def main(self):
self.makeRib()
if self.lyrDoRibSort:
self.sortMe()
24. User Interface Templates
User Interface Layout File
The user interface layout file describes the layout of the
widgets within a window. Each chunk will have its one or
more layout files which contains information coinciding with
the particular chunk.
User Interface Description Language
The user interface layout file is written using a custom JET
user interface description language. This language uses
XML syntax.
34. Python Job
• Dynamically generated.
• Only contain variables and settings designated by
the user.
• Creates an instance of the object defined by the
template class.
• Passes variables to this new object.
• Executes the main function of the object to do the
real work.
• Support for inheriting object to override or
supplement the standard chunk template class (for
advanced user's like TD's)
35. Example Job Script
# Created: Tue Mar 30 15:11:08 2006 by: JET v1.9
vars = {"background":"/show/cw/pr/pr40/plates/2k/cin/pr40.fg1.r1.2k.101-530.cin",
"backgroundType":2,
"compFilePath":"/image/russell/r1/2k/jet2testrussell_r1.comp.2k.338-425.cin",
"defaultBackground":"/plates/2k/cin/jet2test.bg.r6.2k.101-105.cin",
"doHeroElem":1,
"doRoto":1,
"doScaleDownTv":0,
"SHOT":"jet2test",
"SHOTSTART":338,
"SHOTEND":425}
class _comp(comp.comp):
def __init__(self, vars, name):
comp.comp.__init__(self, vars, name)
def main(self):
print "NEW MAIN"
comp.comp.main(self)
if __name__ == '__main__':
_comp(vars, name).main()
37. JET Summary
Lines of Python code:
• JET application and related modules: 39202
• Chunk templates: 22397
Lines of XML:
• User Interface: 29693
Development Stats:
• 1 programmer / ~3 months for design and prototype.
• Developed/maintained by 1 programmer until rev 2.0 in 2006.
Platforms:
• Linux, OSX, SGI
Usage
• In production since mid-2003. First used on Hellboy.
• Rolled out department-by-department, now used by entire studio.
GUI Library:
• PyQt / Qt
Database:
• MySQL
38. Studio Computers
• Primary OS is Linux (Red Hat / Fedora)
• Small base of OSX systems.
• Even smaller base of Windows systems.
• 750+ processors in Render farm.
• 32+ terabyte fileserver.
