Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

OpenIO - A Practical Approach to Serverless Computing and Object Storage for AI Applications

130 views

Published on

OpenIO presentation about serverless computing and object storage for AI applications during the Paris Open Source Summit 2017

Published in: Technology
  • Be the first to comment

  • Be the first to like this

OpenIO - A Practical Approach to Serverless Computing and Object Storage for AI Applications

  1. 1. Next-Gen Object Storage and Serverless Computing
  2. 2. OpenIO A practical approach to Serverless Computing and Next-gen Object Storage for AI applications Enrico Signoretti – Head of Product Strategy OpenIO
  3. 3. OpenIO 1PB of Flash per Rack Unit!
  4. 4. OpenIO IoT and Edge Computing
  5. 5. OpenIO Machine Learning and Artificial Intelligence OpenIO
  6. 6. OpenIO Topics 1 2 3 Evolution of Computing Models Serverless Computing OpenIO Grid for Apps
  7. 7. OpenIO General purpose Traditional Workloads VMs, DBs, … Specialized tasks AI, ML, VR… Faster Response Hardware evolution X86 ARM Coprocessors Addressing changing needs and workloads Power efficiency Edge, IoT, High Density Massive Parallelization 
 and Workload distribution
  8. 8. OpenIO Computing models evolution Virtualization Containers Serverless From compute to datacentric workloads Stateless ComputingLegacy Consolidation Event-driven processing Amazon EC2 Google Container Engine Amazon Lambda over S3 VMware Kubernetes / Docker OpenIO Time >
  9. 9. “ ”OpenIO It is a cloud computing model for which server management and capacity planning decisions are completely hidden from the developer or the operator What is Serverless Computing?
  10. 10. “ ”OpenIO Automate tasks and offload them 
 to the infrastructure Why?
  11. 11. OpenIO More $/DataBetter $/GB Next-Generation Object Storage Serverless Computing
  12. 12. OpenIO Traditional computing Orchestration layer Application Application Containers Operating System(s) Hypervisor Data storage Application Data processing triggered by events Data storage Serverless computing OpenIO
  13. 13. “ ”OpenIO It simplifies and speeds up 
 many workloads Why?
  14. 14. OpenIO Why? It’s for everyone (Developers and SysAdmins)
  15. 15. OpenIO Use cases? You name it!
  16. 16. OpenIO OpenIO Grid for Apps Let’s start with a demo
  17. 17. OpenIO Grid for Apps in practice Category tag = Volcano Queue manager Everything happens into SDS (no external resources) OpenIO SDS Metadata enrichment
 Add Category metadata tag to original object Processing Indexing
  18. 18. OpenIO Grid for Apps in practice Faces tag = 10 Queue manager Everything happens into SDS (no external resources) OpenIO SDS Metadata enrichment
 Add Faces metadata tag to original object Processing
  19. 19. OpenIO #!/usr/bin/env python import cv2 import json import numpy as np from oio.api import object_storage from oio.event.beanstalk import Beanstalk, ResponseError def faceclassifier(image): # Specify the trained cascade classifier face_cascade_name = "/etc/oio/sds/OPENIO/haarcascade_frontalface_alt.xml" # Create a cascade classifier face_cascade = cv2.CascadeClassifier() # Load the specified classifier face_cascade.load(face_cascade_name) # Run the classifiers faces = face_cascade.detectMultiScale(image, 1.1, 5, 0|cv2.cv.CV_HAAR_SCALE_IMAGE, (30, 30)) return faces b = Beanstalk.from_url("beanstalk://127.0.0.1:6014") b.watch("oio-process") while True: try: event_id, data = b.reserve() except ResponseError: continue # Retrieve the information from the event (namespace, bucket, object name ...) meta = json.loads(data) url = meta["url"] s = object_storage.ObjectStorageAPI(url["ns"], "127.0.0.1:6006") meta, stream = s.object_fetch(url["account"], url["user"], url["path"]) # OpenCV PROCESS image = cv2.imdecode(np.frombuffer("".join(stream), np.uint8), 1) faces = faceclassifier(image) # Update the object with new metadatas, number of faces + position s.object_update(url["account"], url["user"], url["path"], {"face_number" : str(len(faces)), "position" : json.dumps(faces.tolist())}) b.delete(event_id)
  20. 20. OpenIO Open source and proprietary code OpenIO SDS core
 (AGPLv3) S3 and Swift Gateway File Gateway WebUI Deployment
 Tools Open source Proprietary Grid For Apps Pro G4A core (AGPLv3)
  21. 21. OpenIO Thank You!

×