Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

OpenIO : How to save 400k$ by leaving S3

310 views

Published on

How to save 400k$ by going from S3 to a full self-hosted OpenIO S3 cluster and a caching reverse proxy for lazy migration

Published in: Internet
  • Be the first to comment

  • Be the first to like this

OpenIO : How to save 400k$ by leaving S3

  1. 1. HOW TO SAVE 400K$ STORAGE WARS * (PER YEAR) *
  2. 2. A long time ago in a galaxy far, far away…
  3. 3. PRODUCTION METRICS
  4. 4. AWS BILLING IN K€ 0 12,5 25 37,5 50 FEB. MARCH APRIL MAY JUNE JULY AUG. SEPT. S3 Cloudfront $$$ METRICS
  5. 5. OPENIO DEPLOYMENT
  6. 6. ▸ Online.net ▸ 40x ST18 for 720TB raw data and 540TB net data ▸ Xeon E5-1510v2 ; 64GB DDR3 ; 3x 6TB SATA
 1Gbit/s unmetered ▸ 79,99 € / month ▸ Total: 3’199,6 € / month SYSTEM ARCHITECTURE
  7. 7. SYSTEM ARCHITECTURE
  8. 8. SYSTEM ARCHITECTURE ▸ Let’s do our own in Europe and use an external for Asia/ USA ▸ 4x servers for redundancy ▸ Xeon E3-1240v6 ; 64GB DDR4 ; 3x 500GB for local cache ▸ 54,99 € / month ▸ Total: 219,96 € / month for 330TB of traffic / month
  9. 9. Online CloudFront CDNify Traffic 330TB 330TB 330TB Price 0,2k$ 15k$ 13k$ (330TB = 1 Gbit/s flat during one month) CDN
  10. 10. Hosting 3420 € OpenIO subscription 1666 € * Total 5086 € * 20’000€ / 12 months CDN
  11. 11. REVERSE PROXY 1 http { 2 3 proxy_cache_path /data/nginx/ levels=1:2 keys_zone=s3cache:1000m max_size=400g inactive=30d; 4 5 server { 6 listen 80; 7 server_name cdn.foo.bar; 8 9 access_log /var/log/nginx/cdn.foo.bar.access.log; 10 error_log /var/log/nginx/cdn.foo.bar.error.log; 11 12 location = /favicon.ico { 13 access_log off; 14 error_log off; 15 } 16 17 location / { 18 proxy_cache_valid 200 30d; 19 proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504; 20 add_header X-Cache-Status $upstream_cache_status always; 21 proxy_cache s3cache; 22 proxy_pass http://127.0.0.1:5000; 23 } 24 } 25 }
  12. 12. 1 from mimetypes import MimeTypes 2 from flask import Flask, Response, abort 3 import tasks 4 5 @app.route('/<string:bucket_name>/<path:path>', methods = ['GET']) 6 def static_proxy(bucket_name, path): 7 ''' 8 First, we try to connect to the OpenIO S3 9 1. if the file is available, we serve it 10 2. if not, we serve the AWS's one and trigger inter-bucket copy 11 ''' 12 if bucket_name not in ALLOWED_BUCKETS: 13 abort(400) 14 15 try: 16 key, headers = get_object(path, bucket_name=bucket_name, mode='openio') 17 mime = MimeTypes() 18 mime_type = mime.guess_type(path) 19 headers['content-type'] = mime_type[0] 20 headers['tzy-type'] = 'openio' 21 return Response(key, headers=headers) 22 except Exception as err: 23 pass 24 25 try: 26 # trigger migration, then serve the s3 content 27 tasks.run(path) 28 key, headers = get_object(path, bucket_name=bucket_name, mode='S3') 29 headers['tzy-type'] = 's3' 30 return Response(key, headers=headers) 31 except Exception as e: 32 return Response(e.body, status=e.status) SERVING THE CONTENT, BEING TRANSPARENT

×