Your SlideShare is downloading. ×
Docker + GCE + etcd + ray tracing
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Docker + GCE + etcd + ray tracing

1,646
views

Published on

Docker + GCE + etcd + ray tracing

Docker + GCE + etcd + ray tracing

Published in: Engineering, Business, Technology

0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,646
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
20
Comments
0
Likes
5
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Docker + GCE + etcd + ray tracing @syoyo Sunday, April 27, 14
  • 2. ray tracing Sunday, April 27, 14
  • 3. Problem • A lot of computing resource required • Imagine 60fps photorealistic ray tracing for OculusVR • ~ 10K nodes • Efficient deploy, operation for the renderer(ray tracer) Sunday, April 27, 14
  • 4. Why docker • Hoping... • Fine resource control • CPU limit, memory limit, etc • Easy versioning management of the renderer • Application sandboxing Sunday, April 27, 14
  • 5. My contribution (1/2) Sunday, April 27, 14
  • 6. Why etcd? • Hoping... • scalable management of cluster information • node IP addr, redis addr, etc. • up to 10K nodes possible(?) Sunday, April 27, 14
  • 7. Why GCE • CPU-intensive instance available • fastVM boot, minute-level charge • CoreOS image avaiable • 282.0.0(Docker 0.10) Sunday, April 27, 14
  • 8. Render A Render B Render C Render D Render E etcd Architecture Sunday, April 27, 14
  • 9. Container • Busybox(8MB) • Our renderer(single binary. 72MB) • node.js(11 MB) • ~ 90 MBBusybox Renderer(Ray tracer) node.js (application frontend) Sunday, April 27, 14
  • 10. Dockerfile FROM syoyo/aobench MAINTAINER Syoyo Fujita(syoyo@lighttransport.com) ADD libdl.so.2 /lib64/ ADD librt.so.1 /lib64/ ADD lte /bin/lte ADD worker.js /home/default/worker.js ADD node /bin/node ADD node_modules /home/default/node_modules Sunday, April 27, 14
  • 11. Render A Render B Render C etcd etcd: 172.17.42.1:4001 webfrontend redis Sunday, April 27, 14
  • 12. Get redis info from etcd http.get(etcdHost + '/v2/keys/redis-server', function(res) { if (res.statusCode != 200) { console.log('failed to get redis infor from etcd.'); process.exit(-1); } res.on('data', function(chunk) { var j = JSON.parse(chunk); var redisURL = url.parse(j['node']['value']); var redisServerAddr = redisURL['hostname'] var redisPort = redisURL['port'] var redisClient = redis.createClient(redisPort, redisServerAddr); Sunday, April 27, 14
  • 13. Task processing function loop() { redisClient.brpop('render-q', timeout, function(err, reply) { // kick the ray tracer exec(...); // resubmit event loop setTimeout(loop, 100); }); } Sunday, April 27, 14
  • 14. • http://lighttransportengine.com/shader/ Sunday, April 27, 14
  • 15. Future work • ~10K nodes • Leverage PCI-ex accelerator(GPU, Xeon Phi) in Docker container. • Confirmed InfiniBand working. • Automation using Docker API Sunday, April 27, 14