xrootd Proxies
Andrew Hanushevsky (SLAC)
Middleware Security Group Meeting
5-6 June 2006
http://xrootd.slac.stanford.edu
xrootd is largely funded by the US Department of Energy
Contract DE-AC02-76SF00515 with Stanford University
Outline
xrootd Architecture Overview
Terms and Concepts
Clustering

Proxies
Single and double firewalls
Proxy clusters for scalability

Security transformations
Conclusions & Acknowledgements
MWSG 5-6 June 2006

2: http://xrootd.slac.stanford.edu
xrootd Plugin Architecture
authentication
(gsi, krb5, etc)

Protocol Driver
(XRD)

Protocol (1 of n)
(xrootd)

lfn2pfn
prefix encoding

authorization
(name based)

File System

Storage System

(ofs, sfs, alice, etc)

(oss, drm/srm, etc)

Clustering
(olbd)
MWSG 5-6 June 2006

3: http://xrootd.slac.stanford.edu
Acronyms, Entities & Relationships
xrootd

olbd
Control Network
Managers, Supervisors & Servers
(resource info, file location)

Redirectors
olbd
M

Data Network
(redirectors steer clients to data
Data servers provide data)

ctl
xrootd
Data Clients

MWSG 5-6 June 2006

olbd
S
data
xrootd
Data Servers

4: http://xrootd.slac.stanford.edu
Cluster Architecture
A manager is an optionally
replicated xrootd/olbd pair
functioning as a root node

Up to 64 servers
or cells can connect
to a manager

Server
A server is an xrootd/olbd
pair leaf node that
delivers data

A cell is 1-to-64 entities (servers or cells)
clustered around a cell manager
called a supervisor
MWSG 5-6 June 2006

5: http://xrootd.slac.stanford.edu
Single Level Switch
A
open file X
Redirectors
Cache file
location

2nd open X go to C
open f
ile X

Client

Who has file X?

go to C

Redirector
(Head Node)

I ha

B

ve

C
Data Servers
Cluster

Client sees all servers as xrootd data servers
MWSG 5-6 June 2006

6: http://xrootd.slac.stanford.edu
Two Level Switch
Client
oh
Wh

open file X
open
file X

go to C

X?

(Head Node)

go to F

A
Data Servers

I ha

Redirector
open file X

ile
as f

B
ve

?
ile X
sf
o ha
ve
Wh
I ha

C
Supervisor

I ha

(sub-redirector)

D
E

ve

F
Cluster

Client sees all servers as xrootd data servers
MWSG 5-6 June 2006

7: http://xrootd.slac.stanford.edu
SLAC Configuration
kan01

kan02

kan03

bbr-olb03

kan04

bbr-olb04

client machines
MWSG 5-6 June 2006

8: http://xrootd.slac.stanford.edu

kanxx

kanolb-a
Extending Access
Easy clustered local access
Everyone sees everyone
Simple configuration
Low human overhead to maintain

Remote access
Difficult because of connection constraints
Want to make it humanly administrable
Critical to minimize cross-domain knowledge

Utilize the peer-to-peer nature of xrootd
MWSG 5-6 June 2006

9: http://xrootd.slac.stanford.edu
Proxies I (single firewall)
data01

data02

data03

data04

IN2P3
olbd
2

Firewall

3
proxy xrootd

INDRA
1

client machines

MWSG 5-6 June 2006

10: http://xrootd.slac.stanford.edu
Scaling Proxies
Need to provide more than one proxy
Selection criteria for proxies?

Utilize natural rooted clustering
Create proxy clusters
Automatically load balance
No practical limit on number

MWSG 5-6 June 2006

11: http://xrootd.slac.stanford.edu
Proxy Clusters (single firewall)
data01

data02

data03

data04

olbd
5
4
proxy server
xrootd

olbd
2
proxy manager
xrootd
1

Firewall

3

client machines

MWSG 5-6 June 2006

12: http://xrootd.slac.stanford.edu
Dealing With Lockdowns
Double Firewalls
Reality sets in.
Incoming and outgoing traffic limited

Utilize peer-to-peer nature of rooted
Maintains practical simplicity

Alternative not particularly appealing
Application controlled firewall
LBL and ANL models for gridFTP.

Could use xrootd’s for this as well, though.
MWSG 5-6 June 2006

13: http://xrootd.slac.stanford.edu
Proxies II (double firewall, simplified)
data01

data02

olbd

data03

data04

4
3
remote proxy xrootd
2

Firewalls

local proxy xrootd
1

client machines

MWSG 5-6 June 2006

14: http://xrootd.slac.stanford.edu
N-to-M Authentication issues
Clusters of proxies on each side
Random server-server connections
Authentication key management issues
Complex because of size and interactions
Would like to simplify key distribution

Use a security transformation
GSI to global session key

MWSG 5-6 June 2006

15: http://xrootd.slac.stanford.edu
Scalable Proxy Security
SLAC PROXY

1

RAL PROXY

2

2
Data Servers

Data Servers

3
1 Authenticate & develop session key
2 Distribute session key to authenticated subscribers
3 Servers can log into each other using session key
MWSG 5-6 June 2006

16: http://xrootd.slac.stanford.edu
Extending Security Transforms
xrootd protocol allows security transforms
Redirect can pass along a CGI string
Anyone can redirect!
No practical redirect limit.

Allows security framework substitutions
Minimizes GSI intra-cluster overhead

MWSG 5-6 June 2006

17: http://xrootd.slac.stanford.edu
Security Transforms
data01

data02

data03

olbd
3
GSI “proxy”
xrootd

data04

4
x-auth
xrootd

1

2

client machines

MWSG 5-6 June 2006

18: http://xrootd.slac.stanford.edu
Conclusion
xrootd has a security enabling architecture
Protocol was designed with security in mind
Accommodates security transforms
Server-to-server
Client-server

Very easy to administer
Critical for maintaining security

MWSG 5-6 June 2006

19: http://xrootd.slac.stanford.edu
Acknowledgements
Software collaborators
INFN/Padova: Fabrizio Furano, Alvise Dorigao
Root: Fons Rademakers, Gerri Ganis
Alice: Derek Feichtinger, Guenter Kickinger, Andreas Peters
Cornell: Gregory Sharp
SLAC: Jacek Becla, Tofigh Azemoon, Wilko Kroeger, Bill Weeks
Princeton: Pete Elmer

Operational collaborators
BNL, CNAF, FZK, INFN, IN2P3, RAL, SLAC

MWSG 5-6 June 2006

20: http://xrootd.slac.stanford.edu

Xrootd proxies Andrew Hanushevsky

  • 1.
    xrootd Proxies Andrew Hanushevsky(SLAC) Middleware Security Group Meeting 5-6 June 2006 http://xrootd.slac.stanford.edu xrootd is largely funded by the US Department of Energy Contract DE-AC02-76SF00515 with Stanford University
  • 2.
    Outline xrootd Architecture Overview Termsand Concepts Clustering Proxies Single and double firewalls Proxy clusters for scalability Security transformations Conclusions & Acknowledgements MWSG 5-6 June 2006 2: http://xrootd.slac.stanford.edu
  • 3.
    xrootd Plugin Architecture authentication (gsi,krb5, etc) Protocol Driver (XRD) Protocol (1 of n) (xrootd) lfn2pfn prefix encoding authorization (name based) File System Storage System (ofs, sfs, alice, etc) (oss, drm/srm, etc) Clustering (olbd) MWSG 5-6 June 2006 3: http://xrootd.slac.stanford.edu
  • 4.
    Acronyms, Entities &Relationships xrootd olbd Control Network Managers, Supervisors & Servers (resource info, file location) Redirectors olbd M Data Network (redirectors steer clients to data Data servers provide data) ctl xrootd Data Clients MWSG 5-6 June 2006 olbd S data xrootd Data Servers 4: http://xrootd.slac.stanford.edu
  • 5.
    Cluster Architecture A manageris an optionally replicated xrootd/olbd pair functioning as a root node Up to 64 servers or cells can connect to a manager Server A server is an xrootd/olbd pair leaf node that delivers data A cell is 1-to-64 entities (servers or cells) clustered around a cell manager called a supervisor MWSG 5-6 June 2006 5: http://xrootd.slac.stanford.edu
  • 6.
    Single Level Switch A openfile X Redirectors Cache file location 2nd open X go to C open f ile X Client Who has file X? go to C Redirector (Head Node) I ha B ve C Data Servers Cluster Client sees all servers as xrootd data servers MWSG 5-6 June 2006 6: http://xrootd.slac.stanford.edu
  • 7.
    Two Level Switch Client oh Wh openfile X open file X go to C X? (Head Node) go to F A Data Servers I ha Redirector open file X ile as f B ve ? ile X sf o ha ve Wh I ha C Supervisor I ha (sub-redirector) D E ve F Cluster Client sees all servers as xrootd data servers MWSG 5-6 June 2006 7: http://xrootd.slac.stanford.edu
  • 8.
    SLAC Configuration kan01 kan02 kan03 bbr-olb03 kan04 bbr-olb04 client machines MWSG5-6 June 2006 8: http://xrootd.slac.stanford.edu kanxx kanolb-a
  • 9.
    Extending Access Easy clusteredlocal access Everyone sees everyone Simple configuration Low human overhead to maintain Remote access Difficult because of connection constraints Want to make it humanly administrable Critical to minimize cross-domain knowledge Utilize the peer-to-peer nature of xrootd MWSG 5-6 June 2006 9: http://xrootd.slac.stanford.edu
  • 10.
    Proxies I (singlefirewall) data01 data02 data03 data04 IN2P3 olbd 2 Firewall 3 proxy xrootd INDRA 1 client machines MWSG 5-6 June 2006 10: http://xrootd.slac.stanford.edu
  • 11.
    Scaling Proxies Need toprovide more than one proxy Selection criteria for proxies? Utilize natural rooted clustering Create proxy clusters Automatically load balance No practical limit on number MWSG 5-6 June 2006 11: http://xrootd.slac.stanford.edu
  • 12.
    Proxy Clusters (singlefirewall) data01 data02 data03 data04 olbd 5 4 proxy server xrootd olbd 2 proxy manager xrootd 1 Firewall 3 client machines MWSG 5-6 June 2006 12: http://xrootd.slac.stanford.edu
  • 13.
    Dealing With Lockdowns DoubleFirewalls Reality sets in. Incoming and outgoing traffic limited Utilize peer-to-peer nature of rooted Maintains practical simplicity Alternative not particularly appealing Application controlled firewall LBL and ANL models for gridFTP. Could use xrootd’s for this as well, though. MWSG 5-6 June 2006 13: http://xrootd.slac.stanford.edu
  • 14.
    Proxies II (doublefirewall, simplified) data01 data02 olbd data03 data04 4 3 remote proxy xrootd 2 Firewalls local proxy xrootd 1 client machines MWSG 5-6 June 2006 14: http://xrootd.slac.stanford.edu
  • 15.
    N-to-M Authentication issues Clustersof proxies on each side Random server-server connections Authentication key management issues Complex because of size and interactions Would like to simplify key distribution Use a security transformation GSI to global session key MWSG 5-6 June 2006 15: http://xrootd.slac.stanford.edu
  • 16.
    Scalable Proxy Security SLACPROXY 1 RAL PROXY 2 2 Data Servers Data Servers 3 1 Authenticate & develop session key 2 Distribute session key to authenticated subscribers 3 Servers can log into each other using session key MWSG 5-6 June 2006 16: http://xrootd.slac.stanford.edu
  • 17.
    Extending Security Transforms xrootdprotocol allows security transforms Redirect can pass along a CGI string Anyone can redirect! No practical redirect limit. Allows security framework substitutions Minimizes GSI intra-cluster overhead MWSG 5-6 June 2006 17: http://xrootd.slac.stanford.edu
  • 18.
  • 19.
    Conclusion xrootd has asecurity enabling architecture Protocol was designed with security in mind Accommodates security transforms Server-to-server Client-server Very easy to administer Critical for maintaining security MWSG 5-6 June 2006 19: http://xrootd.slac.stanford.edu
  • 20.
    Acknowledgements Software collaborators INFN/Padova: FabrizioFurano, Alvise Dorigao Root: Fons Rademakers, Gerri Ganis Alice: Derek Feichtinger, Guenter Kickinger, Andreas Peters Cornell: Gregory Sharp SLAC: Jacek Becla, Tofigh Azemoon, Wilko Kroeger, Bill Weeks Princeton: Pete Elmer Operational collaborators BNL, CNAF, FZK, INFN, IN2P3, RAL, SLAC MWSG 5-6 June 2006 20: http://xrootd.slac.stanford.edu