Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

(MED304) The Future of Rendering: A Complete VFX Studio in the AWS Cloud | AWS re:Invent 2014


Published on

Today's studios and visual effects companies require massive computing power and large amounts of storage to produce high-end digital scenes and videos. Maintaining the infrastructure required for these jobs is expensive and operationally difficult, plus demand fluctuates day to day. Geographically diverse workforces adds additional complexity to data and content transfer. The low-cost, utility computing model as well as unique virtualization capabilities offered by AWS are well-suited to addressing these challenges. In this session you will learn how to build and deploy a studio-quality, scalable Arnold render farm on AWS with reusable templates. We'll also demonstrate how to run Maya and Deadline remotely with AWS AppStream, and use them to edit scenes and coordinate render jobs entirely in the cloud.

Published in: Technology

(MED304) The Future of Rendering: A Complete VFX Studio in the AWS Cloud | AWS re:Invent 2014

  1. 1. November 13th, 2014 –Las Vegas, NV Usman Shakeel, Gerald Tiu, and Matt Yanchyshyn
  2. 2. STARRING
  4. 4. EPISODEI What about the Cloud?
  5. 5. •Infinite •Infinite •Infinite •Globally distributed studio in a template •Flexibilityturnaround time •Secure
  6. 6. –Lustre, Avere, iSCSI, … –Arnold, Mental Ray, V-Ray, Mantra, Renderman… –Deadline, Tractor, Backburner, … –Maya, 3ds Max, Houdini, Nuke, Cinema 4D… –FlexLM, Pixar License Manager …
  7. 7. Amazon AppStream AWS Direct Connect AMI + AWS CloudFormation Template (Automated Deployment) Amazon S3 Lustre Render Nodes Storage Servers (Cache) On-Premises Storage Modeling Dumb Client GPU Based Modeling Client Nodes Render Pipeline Amazon Glacier
  8. 8. Amazon S3 bucket with content Parallel Reads using Range Gets Render Farm Scale Instance NIC Instance NIC Instance NIC Parallel Writes using Multi-part Uploads
  9. 9. Periodically write back to Amazon S3 bucket Automated deployment and Scale Amazon Virtual Private Cloud Lustre Storage Nodes LustreMetadata Nodes Scale horizontally to add more performance Scale horizontally to add more storage Rendering/Modeling Nodes Amazon S3 bucket with content Hydrate metadata nodes at launch Lustre Client on each processing node AWS CloudFormation Amazon S3 bucket with content
  10. 10. The Resting Place FS On Demand Terminate Amazon Glacier archive Amazon S3 bucket Amazon EBS snapshot AMI AWS CloudFormation template Scale Create a Render Project Storage Nodes Metadata Nodes Select Amazon S3 bucket path to hydrate the LustreFile System Pre-hydrate from Amazon S3 and sync back on an on-going basis Add or remove additional Storage or metadata nodes by calling Stack Updates Delete all the resources attached to the Lustre FS by calling delete stack at the end of the job
  11. 11. AWS Cloud On-Premises NAS A fleet of virtual Storage Servers running on the Cloud Clients 10 G AWS DirectConnect
  12. 12. –Automated –Scale •Backedquickly
  13. 13. Disposable Storage Disposable Render farm On Demand Terminate Amazon S3 bucket Create a Render Project Compositing Scalable Lustre Select AWS CloudFormation templates to generate an on-demand Render Farm Scale the Render Farm appropriately based on producer’s wallet Delete all the resources attached to the Render Farm by calling delete stack at the end of the render queue Virtual Storage Servers (Cache) Effects Rendering License server Render Farm manager
  14. 14. create_role_and_profilecreate_autoscalegroup instance_type security_groups min_size max_size
  15. 15. <s3 bucket>/Deadline/qt.repo mount -v -t lustre-o rw10.1.150.219:/scratch /mnt<s3 bucket>/Deadline/ ./ --licenseserver@<location> <s3 bucket>/Autodesk_Maya_2015_SP5_English_Linux.tgz rpm-i Maya2015_64-2015.0-733.x86_64.rpm<s3 bucket>/Arnold/arnoldRenderer.xml cparnoldRenderer.xml/usr/maya2015-x64/bin/rendererDesc/arnoldRenderer.xml
  16. 16. create_autoscalegroup instance_type security_groups min_size max_size use_spot
  17. 17. DirectXOpenGLYUV444GPU
  18. 18. Disposable Storage Disposable Graphics Workstation On Demand Terminate Amazon S3 bucket Create a Render project Compositing Scalable Lustre Launch Amazon AppStream with modeling software installed Connect to geographically diverse graphic artists Delete all the Amazon AppStream resources after the graphic artists are finished Virtual Storage Servers (Cache) Effects Rendering Render farm manager Licensing server Disposable Render Farm
  19. 19. Storage Nodes Metadata Nodes Virtual Storage Servers(Cache) AWS Direct Connect Amazon S3 Amazon EBS (GP2) 3000 256K IOPS Render Nodes <No. Of Storage Servers> X NIC throughput <No. Of Lustrenodes> X NIC throughput Each node can saturate its link to EBS S3 scales up to saturating each nodes N X 10 Gbps Up to 5GB/sec per Storage node –64 PB Up to 35K/s create, 100K/s stat metadata operations Each Virtual Storage Server can technically be able to push up to the NIC capacity of the instance it is running on Render Nodes
  20. 20. SEASONFINALE On to a bigger project with a fancy VFX studio in the Cloud …
  21. 21.