Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Container Federation Use Cases

54 views

Published on

Presentation by Ricardo Rocha, at HNSciCloud procurer hosted event in Geneva, 14 June 2018

Published in: Software
  • Be the first to comment

  • Be the first to like this

Container Federation Use Cases

  1. 1. Container Federation Use Cases Ricardo Rocha <ricardo.rocha@cern.ch> CERN IT-CM-RPS
  2. 2. Motivation for Federation Periodic Load Spikes International Conferences, Reconstruction Campaigns Simplification Monitoring, Lifecycle, Alarms Deployment Uniform API, Replication, Load Balancing
  3. 3. Sched Collector Negotiator StartD AcctGroup = "ATLAS" JobPrio = 0 RequestCpus = 2 RequestMemory = 4260 ... CERNEnvironment = “production” Datacenter = “meyrin” HasMPI = true TotalCpus = 8 TotalMemory = 22500 ... Matchmaking with ClassAds Fair Share Preemption Running Virtualized Extensive Experience in HEP External Storage and Networking
  4. 4. Sched Collector Negotiator StartD AcctGroup = "ATLAS" JobPrio = 0 RequestCpus = 2 RequestMemory = 4260 ... CERNEnvironment = “production” Datacenter = “meyrin” HasMPI = true TotalCpus = 8 TotalMemory = 22500 ... Matchmaking with ClassAds Fair Share Preemption Running Virtualized Extensive Experience in HEP External Storage and Networking
  5. 5. Sched Negotiator Collector Host kubefed init fed --host-cluster-context=condor-host ...
  6. 6. kind: DaemonSet ... hostNetwork: true containers: - name: condor-startd image: .../cloud/condor-startd command: ["/usr/sbin/condor_startd", "-f"] securityContext: privileged: true livenessProbe: exec: command: - condor_who Sched Negotiator Collector Host StartD ... StartD ... StartD ... kubefed init fed --host-cluster-context=condor-host ... kubefed join --context fed tsystems --host-cluster-context condor-host --cluster-context tsystems
  7. 7. REANA / RECAST Reusable Analysis Platform Workflow Engine (Yadage) Each step a Kubernetes Job Integrated Monitoring & Logging Centralized Log Collection https://github.com/reanahubhttps://github.com/recast-hep https://github.com/diana-hep/yadage
  8. 8. Summary • Federation support in Kubernetes is ready • Ongoing development for the v2 API, with significant changes • Work with the community to have our use case taken into account • Initial use cases • Started with a legacy application, limited integration • Expanded to a cloud native implementation, with great results • Ongoing work to expand its usage • Use built-in “Kubernetes As a Service” capabilities • Investigations on how to best handle data distribution
  9. 9. https://www.youtube.com/watch?v=jNyd97LiTXk

×