Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

OpenIO Summit'17 - Grid for Apps


Published on

Grid for Apps is an innovative event-driven compute framework that enables users to run applications directly on the storage infrastructure without any other components to manage. Just data and applications, next to each other, simplifying the entire infrastructure and o loading several compute tasks to the storage layer while increasing overall efficiency.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

OpenIO Summit'17 - Grid for Apps

  1. 1. OpenIO Summit ‘17 Grid for Apps
  2. 2. OpenIO Summit’17 What do you ask to your storage system? $/GB-
  3. 3. OpenIO Summit’17 What do you ask to your storage system? $/Data $/GB + - OpenIO Summit’17
  4. 4. “ OpenIO Summit’17 Serverless Computing is a cloud computing model for which server management and capacity planning decisions are completely hidden from the developer or operator
  5. 5. OpenIO Hypervisor Orchestration layer Application Application Containers Operating System(s) Data storage OpenIO Summit’17 Traditional computing Grid for Apps 
 Serverless computing Application Data storage: OpenIO SDS Processing triggered on data by events: Grid for Apps
  6. 6. Why
  7. 7. OpenIO Summit’17 Why serverless computing Offloads tasks 
 to Storage OpenIO Summit’17
  8. 8. OpenIO Summit’17 Why serverless computing It simplifies and speeds up 
 many workloads OpenIO Summit’17
  9. 9. OpenIO Summit’17 Why serverless computing Event driven vs Batch OpenIO Summit’17 (but we will do both)
  10. 10. OpenIO Summit’17 Why serverless computing It’s for everyone OpenIO Summit’17 (several programming and scripting languages supported)
  11. 11. OpenIO Summit’17 Use Cases You name it! OpenIO Summit’17
  12. 12. How
  13. 13. OpenIO Summit ‘17 Topics 1 2 3 Concepts Architecture API 4 Roadmap
  14. 14. OpenIO Summit’17 Nodes • Can be physical machine orVM • Has the grid runtime installed • Runs and manages the user tasks Tasks • Executes the user defined workloads • Runs as container in Docker Users interact only with the cluster via the API resources 1. Concepts
  15. 15. OpenIO Summit’17 Control Plane • Exposes the API • Control the state of the cluster • Runs on some nodes Runtime • Runs and manages the workload • Runs on every nodes 2. Architecture APIController Client Node DockerAgent Node DockerAgent
  16. 16. OpenIO Summit’17 Control Plane API server • Exposes the grid API • Create/Update/List/Remove the API resources Controller • Runs the controllers • Examples: • Node controller: tracks nodes status • Job controller: tracks jobs status and their lifecycle 2. Architecture
  17. 17. OpenIO Summit’17 Runtime Agent • Receives the tasks assigned to the node from the API server • Runs and manages the tasks with docker • Updates the task status Docker • Runs the containers 2. Architecture
  18. 18. OpenIO Summit’17 • Available via HTTP • Resource based • Versioned 3. API
  19. 19. OpenIO Summit’17 API Resources • Describes a desired state for the cluster • Create/Update/List/Remove • Persisted in a durable store 3. API
  20. 20. OpenIO Summit’17 API Resources Spec • Describes the desired state for the given resource • User defined Status • Describes the actual state of the resource • Updated by the system The system is constantly trying to match the actual state with the desired state 3. API
  21. 21. OpenIO Summit’17 Resource example: Job • A job is a resource that represents a distributed workload • The spec defines how many tasks instances should be running in the cluster • The system follows the job spec and starts the configured number of tasks • The job status is updated by the system • If any of those tasks should fail, the system detects the mismatch between the spec and the actual state and starts a replacement task 3. API
  22. 22. OpenIO Summit’17 Task • Basic resource in the API • Represents a running process in the cluster • Runs as containers in docker • Are usually created automatically via a Job by the system • Scheduled to run on a specific Node 3. API
  23. 23. OpenIO Summit’17 Task example: Job Kind: Task API: core/v1 
 Meta: • Namespace: default • Name: foo 
 Spec: • Containers: • Command: bin/sh • Args: sleep 3000 • Image: busybox • Name: “1” • Restart: OnFailure 3. API Resource kind Metadata Task description
  24. 24. OpenIO Summit’17 Task example: Job 3. API Node Task Foo Input A Parallel Processing Node2 Task Foo Input B Task Foo Input C Job Foo: Process $input Foo Input A Foo Input B Foo Input C 3 Jobs
  25. 25. OpenIO Summit’17 Task example: Job 3. API Node Parallel Processing
 with work queue Node2 Job Foo: Process work queue item Foo Input A 1 Job: 3 task instances Queue Task Foo Instance 1 Task Foo Instance 2 Work items
  26. 26. OpenIO Summit’17 Alpha available next week • Launch tasks with the API • Easy to deploy test environment Beta available in october • Jobs • Automatic scheduling • View task logs Command Line Interface Advanced scheduling 4. Roadmap
  27. 27. OpenIO Summit ‘17 What’s next Grid for Apps & TensorFlow Maxime Thomas (OpenIO, Presales Engineer) José Corral (Skapánê) After lunch