Tim Bell
tim.bell@cern.ch
24/09/2015 2Tim Bell - RDA
CERN Tool Chain
24/09/2015 Tim Bell - RDA 3
OpenStack Status
• 4 OpenStack clouds at CERN
• Largest is ~120,000 cores in ~4,000 servers in
two data centres
• 3 other instances with 45,000 cores total
• Currently running Juno release of
OpenStack
• Migrating to Kilo in next two months
24/09/2015 4Tim Bell - RDA
24/09/2015 5
Microsoft Active
Directory
Database
Services
CERN Network
Database
Account mgmt
system
Horizon
Keystone
Glance
Network
Compute
Scheduler
Cinder
Nova
Block Storage
Ceph & NetApp
CERN
Accounting
Ceilometer
Tim Bell - RDA
IN2P3
INFN
…
Onwards the Federated Clouds
Public Cloud such
as Rackspace
CERN Private
Cloud
120K cores
ATLAS Trigger
28K cores
CMS Trigger
12K cores
Brookhaven
National Labs
NecTAR
Australia
Many Others on
Their Way
24/09/2015 Tim Bell - RDA 6
ALICE Trigger
12K cores
Open Design Process
24/09/2015 Tim Bell - RDA 7
• Started at OpenStack Hong Kong design summit
• Iterative design using open blueprints
• Source code under Apache 2 license
• Continuous integration to ensure maintainability
• Diverse team
Implementation
24/09/2015 Tim Bell - RDA 8
Keystone authentication options
• Password
• Active Directory
• OpenID Connect
• X.509
• Kerberos
• Tivoli Federated Identity Manager
• … plug in architecture for extensions
24/09/2015 Tim Bell - RDA 9
Usage Modes
• OpenStack with Web GUI handled by
Federated Single Sign On
• OpenStack with Keystone authentication
service validating against a SAML IdP
• OpenStack with Keystone authentication
service validating against another Keystone
24/09/2015 Tim Bell - RDA 10
Policy
24/09/2015 Tim Bell - RDA 11
LOGIN: madenis
LANGUAGE: EN
DEPARTMENT: IT/OIS
FULLNAME: Marek Denis
Assertion Keystone
credentials
{
name:
madenis
groups: [
“devs”,
“openlab”
]
}
[
{ "local":
[ { "user": { "name": "{0}" } } ],
"remote":
[ { "type": "ADFS_LOGIN" } ]
},
{
"local":
[ { "group": { "id": “devs" } } ,
{“group”: {ïd”:”openlab”} } ],
"remote":
[ { "type":"DEPARTMENT",
"any_one_of": ["IT/OIS"] } ]
}
]
OpenStack Identity Federation in 2015
24/09/2015 Tim Bell - RDA 12
Examples of potential use #1
• Federation with a cloud provider such as Rackspace
• Scenario
• Project with quota on an external cloud
• Define role mapping in external cloud using attributes
• User authenticates against private cloud IdP
• Accesses public cloud project
• Demo’d at the OpenStack summit in Paris in Autumn 2014
• http://cern.ch/go/h98B
24/09/2015 Tim Bell - RDA 13
Examples of potential use #2
• Indigo dataclouds project
• H2020 funded
• Needs build and test resources
• CERN defines an OpenStack project
• Maps INFN role to project members
• Web SSO
• Federates with EduGain
• API/CLI
• Federates with INFN Keystone using Keystone-to-Keystone
24/09/2015 Tim Bell - RDA 14
Experiences
• Watch out for non-federated services
• Who owns the resources at the site ?
• How to ssh into a VM behind a firewall when no
account on the central login services ?
• Traceability for ephemeral accounts
• CADF logs need to be kept to map user UUID to
originator
24/09/2015 Tim Bell - RDA 15
Summary
• OpenStack now includes Federated Identity as standard
• Web SSO
• CLI
• Pluggable for authentication methods
• SAML and OpenID connect most popular
• Significant commercial interest and investment
• Partner networks such as Cisco and HP
• Easy to miss non-federated services when deploying production
uses
24/09/2015 16Tim Bell - RDA
Questions ?
24/09/2015 17
• OpenStack FIM
links at
http://clouddocs.web.cern.ch/c
louddocs/additional/README.
html
• CERN OpenStack
technical details at
http://openstack-in-
production.blogspot.fr
Tim Bell - RDA
24/09/2015 18Tim Bell - RDA
The Worldwide LHC Computing Grid
Tier-1:
permanent storage, re-
processing,
analysis
Tier-0 (CERN): data
recording,
reconstruction and
distribution
Tier-2:
Simulation,
end-user analysis
> 2 million jobs/day
~350’000 cores
500 PB of storage
nearly 170 sites,
40 countries
10-100 Gb links
1924/09/2015 Tim Bell - RDA
24/09/2015 20Tim Bell - RDA

20150924 rda federation_v1

  • 2.
  • 3.
  • 4.
    OpenStack Status • 4OpenStack clouds at CERN • Largest is ~120,000 cores in ~4,000 servers in two data centres • 3 other instances with 45,000 cores total • Currently running Juno release of OpenStack • Migrating to Kilo in next two months 24/09/2015 4Tim Bell - RDA
  • 5.
    24/09/2015 5 Microsoft Active Directory Database Services CERNNetwork Database Account mgmt system Horizon Keystone Glance Network Compute Scheduler Cinder Nova Block Storage Ceph & NetApp CERN Accounting Ceilometer Tim Bell - RDA
  • 6.
    IN2P3 INFN … Onwards the FederatedClouds Public Cloud such as Rackspace CERN Private Cloud 120K cores ATLAS Trigger 28K cores CMS Trigger 12K cores Brookhaven National Labs NecTAR Australia Many Others on Their Way 24/09/2015 Tim Bell - RDA 6 ALICE Trigger 12K cores
  • 7.
    Open Design Process 24/09/2015Tim Bell - RDA 7 • Started at OpenStack Hong Kong design summit • Iterative design using open blueprints • Source code under Apache 2 license • Continuous integration to ensure maintainability • Diverse team
  • 8.
  • 9.
    Keystone authentication options •Password • Active Directory • OpenID Connect • X.509 • Kerberos • Tivoli Federated Identity Manager • … plug in architecture for extensions 24/09/2015 Tim Bell - RDA 9
  • 10.
    Usage Modes • OpenStackwith Web GUI handled by Federated Single Sign On • OpenStack with Keystone authentication service validating against a SAML IdP • OpenStack with Keystone authentication service validating against another Keystone 24/09/2015 Tim Bell - RDA 10
  • 11.
    Policy 24/09/2015 Tim Bell- RDA 11 LOGIN: madenis LANGUAGE: EN DEPARTMENT: IT/OIS FULLNAME: Marek Denis Assertion Keystone credentials { name: madenis groups: [ “devs”, “openlab” ] } [ { "local": [ { "user": { "name": "{0}" } } ], "remote": [ { "type": "ADFS_LOGIN" } ] }, { "local": [ { "group": { "id": “devs" } } , {“group”: {ïd”:”openlab”} } ], "remote": [ { "type":"DEPARTMENT", "any_one_of": ["IT/OIS"] } ] } ]
  • 12.
    OpenStack Identity Federationin 2015 24/09/2015 Tim Bell - RDA 12
  • 13.
    Examples of potentialuse #1 • Federation with a cloud provider such as Rackspace • Scenario • Project with quota on an external cloud • Define role mapping in external cloud using attributes • User authenticates against private cloud IdP • Accesses public cloud project • Demo’d at the OpenStack summit in Paris in Autumn 2014 • http://cern.ch/go/h98B 24/09/2015 Tim Bell - RDA 13
  • 14.
    Examples of potentialuse #2 • Indigo dataclouds project • H2020 funded • Needs build and test resources • CERN defines an OpenStack project • Maps INFN role to project members • Web SSO • Federates with EduGain • API/CLI • Federates with INFN Keystone using Keystone-to-Keystone 24/09/2015 Tim Bell - RDA 14
  • 15.
    Experiences • Watch outfor non-federated services • Who owns the resources at the site ? • How to ssh into a VM behind a firewall when no account on the central login services ? • Traceability for ephemeral accounts • CADF logs need to be kept to map user UUID to originator 24/09/2015 Tim Bell - RDA 15
  • 16.
    Summary • OpenStack nowincludes Federated Identity as standard • Web SSO • CLI • Pluggable for authentication methods • SAML and OpenID connect most popular • Significant commercial interest and investment • Partner networks such as Cisco and HP • Easy to miss non-federated services when deploying production uses 24/09/2015 16Tim Bell - RDA
  • 17.
    Questions ? 24/09/2015 17 •OpenStack FIM links at http://clouddocs.web.cern.ch/c louddocs/additional/README. html • CERN OpenStack technical details at http://openstack-in- production.blogspot.fr Tim Bell - RDA
  • 18.
  • 19.
    The Worldwide LHCComputing Grid Tier-1: permanent storage, re- processing, analysis Tier-0 (CERN): data recording, reconstruction and distribution Tier-2: Simulation, end-user analysis > 2 million jobs/day ~350’000 cores 500 PB of storage nearly 170 sites, 40 countries 10-100 Gb links 1924/09/2015 Tim Bell - RDA
  • 20.

Editor's Notes

  • #5 Already 4 independent clouds – federation is now being studied Rackspace inside CERN openlab Cells is a key technology to scale
  • #6 Account Management Automation CERN legacy network database No Neutron yet
  • #7 The trigger farms are those servers nearest the accelerator which are not needed while the accelerator is shut down till 2015 Public clouds are interesting for burst load (such as coming up to a conference) or when price drops such as spot market Private clouds allow universities and other research labs to collaborate in processing the LHC data