Docker + GCE + etcd + ray tracing

2,047
-1

Published on

Docker + GCE + etcd + ray tracing

Published in: Engineering, Business, Technology

Docker + GCE + etcd + ray tracing

  1. 1. Docker + GCE + etcd + ray tracing @syoyo Sunday, April 27, 14
  2. 2. ray tracing Sunday, April 27, 14
  3. 3. Problem • A lot of computing resource required • Imagine 60fps photorealistic ray tracing for OculusVR • ~ 10K nodes • Efficient deploy, operation for the renderer(ray tracer) Sunday, April 27, 14
  4. 4. Why docker • Hoping... • Fine resource control • CPU limit, memory limit, etc • Easy versioning management of the renderer • Application sandboxing Sunday, April 27, 14
  5. 5. My contribution (1/2) Sunday, April 27, 14
  6. 6. Why etcd? • Hoping... • scalable management of cluster information • node IP addr, redis addr, etc. • up to 10K nodes possible(?) Sunday, April 27, 14
  7. 7. Why GCE • CPU-intensive instance available • fastVM boot, minute-level charge • CoreOS image avaiable • 282.0.0(Docker 0.10) Sunday, April 27, 14
  8. 8. Render A Render B Render C Render D Render E etcd Architecture Sunday, April 27, 14
  9. 9. Container • Busybox(8MB) • Our renderer(single binary. 72MB) • node.js(11 MB) • ~ 90 MBBusybox Renderer(Ray tracer) node.js (application frontend) Sunday, April 27, 14
  10. 10. Dockerfile FROM syoyo/aobench MAINTAINER Syoyo Fujita(syoyo@lighttransport.com) ADD libdl.so.2 /lib64/ ADD librt.so.1 /lib64/ ADD lte /bin/lte ADD worker.js /home/default/worker.js ADD node /bin/node ADD node_modules /home/default/node_modules Sunday, April 27, 14
  11. 11. Render A Render B Render C etcd etcd: 172.17.42.1:4001 webfrontend redis Sunday, April 27, 14
  12. 12. Get redis info from etcd http.get(etcdHost + '/v2/keys/redis-server', function(res) { if (res.statusCode != 200) { console.log('failed to get redis infor from etcd.'); process.exit(-1); } res.on('data', function(chunk) { var j = JSON.parse(chunk); var redisURL = url.parse(j['node']['value']); var redisServerAddr = redisURL['hostname'] var redisPort = redisURL['port'] var redisClient = redis.createClient(redisPort, redisServerAddr); Sunday, April 27, 14
  13. 13. Task processing function loop() { redisClient.brpop('render-q', timeout, function(err, reply) { // kick the ray tracer exec(...); // resubmit event loop setTimeout(loop, 100); }); } Sunday, April 27, 14
  14. 14. • http://lighttransportengine.com/shader/ Sunday, April 27, 14
  15. 15. Future work • ~10K nodes • Leverage PCI-ex accelerator(GPU, Xeon Phi) in Docker container. • Confirmed InfiniBand working. • Automation using Docker API Sunday, April 27, 14

×