SlideShare a Scribd company logo
1 of 28
Download to read offline
Setup Min.io and Open Policy Agent for
a multi-purpose scientific platform
D. Ciangottini, INFN
stackconf-2021 $ > whoami
● IT Researcher at Istituto Nazionale di Fisica
Nucleare (INFN)
○ Translated: National Institute of Nuclear Physics
● Involved on R&D activities to deploy cloud-native
solutions for the next-gen of data analysis
infrastructure for the INFN/LHC users
2
INFN community
3
The 5
research lines
and the National
Scientific
Committee
4
One of the main scientific challenges
understand the Universe in the first moments after the Big Ban
5
A community of
over 6,000
people
~ 25% of them have PhD grants,
post-doc scholarships and research
grants
6
INFN
facilities
Computing @INFN
Long tradition supporting experiments
For last 10 years, that was meant for supporting LHC
communities.
Quickly widening to many other use cases recently.
7
On-demand computing resources
for the INFN communities
● Easy access to on-demand solutions for scientific data analysis
● Composable services to extend and customize the environment
● Provide INFN users with a set of core tools centrally managed
○ E.g. JupyterHub-aaS, object storage, sync&share ....
● Federating the resources from several centers at national level
● Becoming the hub of reference for most of the activities and projects @ INFN
8
The INFN-Cloud initiative
The INFN-Cloud infrastructure
A backbone composed by the main
computing centers for central services
+ a federation of smaller sites
providing resources for user deployments
9
Computing challenges
Data storage for multiple communities
Providing a cloud storage hosted on the backbone infrastructure means:
● Geo-distributed storage federation
● Heterogeneous set of requirements
○ Object size (few MBs, to 10s GBs)
○ Workflow (imaging, columnar analysis...) and data access (posix, webdav, s3 etc..)
but it also means providing the tools:
● “F.A.I.R.” data
○ Findable, Accessible, Interoperable, Reusable
○ Make it intuitive or eventually transparent for the end user
● Focus on the “R.”! Allow sustainable reuse of data
10
Wrapping up...
Requirements
● Dynamic user registration/acls integrated with Indigo-IAM/OIDC
● Fine grain authz (ro, rw, per file/per user group )
● Easy and robust ops
○ gitOps eventually
● Accessible via posix
● WebUI access
● Vendor neutral
● Open source
11
Quick look to the solution
The components
● Minio has been chosen as the cloud
storage solution
○ S3 compliance
○ Powerful WebUI
○ Proven scalability
● Native integration with AWS STS
credentials
○ External OIDC IdP’s (e.g. Indigo IAM)
● Support for customizable authZ
policies with OpenPolicyAgent
12
User management
Indigo-IAM
● Authentication via SAML IdPs or identity
federations, OpenID Connect providers
and X.509 certificates
● Enrollment and registration
functionalities
○ so that users can join groups/collaborations
according to well-defined flows
○ provides services to manage group
membership
○ attributes assignment and account linking
functionality
● Integrable as IdP with any OIDC
compliant service
13
Cloud storage AuthN:
AWS STS credentials
● Endpoint service that enables clients to request temporary credentials for MinIO
resources
● AWS AssumeRoleWithWebIdentity flow is supported out of the box
○ Allowing the integration with any OpenID Connect-compatible identity provider ⇒ our IAM service
14
Cloud storage AuthZ:
OpenPolicyAgent integration
● A lightweight general-purpose policy
engine that can be co-located with
Minio server
● OPA HTTP API used to authorize Minio
STS credentials
○ Fine grain ACLs
■ Every token claim from authN can be
selected for policy checking
○ Dynamic config
○ Decoupled from the storage configuration
Example of an e2e AuthZ flow
16
OPA server checks custom
policies for the input
API
Policy example
# Allow users to manage their own data.
allow {
username := split(lower(input.claims.email),"@")[0]
input.bucket == username
input.claims.aud == "minio-auth"
permissions := rl_permissions["user"]
p := permissions[_]
p == {"action": input.action}
}
Authorized
JWT
List of operation permissions defined on OPA
"claims": {
"accessKey": "VP43M6DO1N53U2LUBTZ3",
"aud": "https://wlcg.cern.ch/jwt/v1/any",
"client_id":
"5c38c020-b753-4115-a5f4-3f48595e4c1b",
"exp": "1621714730",
"iat": 1621713801,
"iss": "https://login.cloud.infn.it",
"scope": "openid profile email",
"email": “ciangottini@infn.it”
}
Minio STS auth data
Managing policies with OPA bundles
the gitOps way
● OPA can periodically download bundles
of policy and data from remote HTTP
servers
○ Allowing for a gitOps based policy
management
● The policies and data are loaded on the
fly without requiring a restart of OPA
○ Policies and data are then applied immediately
17
So far so good…
Let’s put the hands on some user tools now!
18
Managing temporary credentials:
OIDC-agent
● A set of tools to manage OpenID Connect access tokens and make them
easily usable from the command line
○ ssh-agent design, so users can handle OIDC tokens in a similar way as they do with ssh keys
● Secure sensible information (long living credentials) while exposing short lived
ones (e.g. access token)
● Integrable via API libraries for: python, go and c++
19
POSIX access:
RClone + OIDC-Agent integration
To provide posix access we make use of RClone mount capability
A small patch has been applied to add a dedicated S3 provider integrated with
OIDC-Agent
Users, once oidc-agent is configured on its VM, can then mount its own bucket as a
folder with no further actions/authentication steps.
Backups via Restic are enabled through the use of this patched version of RClone
20
Make it easier:
STS-wire
21
For cases where the user does not/cannot run oidc-agent:
● a tool has been created to manage both the credential renewal and the rclone
mount in a guided/integrated/opinionated way
We found that to be the preferred solution to mount a bucket content on a laptop
for instance.
What about python?
boto3+STS+OIDC-Agent = boto3STS
Access Minio bucket through the
integration of boto library with
temporary credentials
- AWS STS token via IAM
- IAM access token get via
oidc-agent API
Instantiate an S3 session with a simple
line of code
22
Nice stuff! So, how did you do that?
23
Deployment models
● Generic centrally maintained service for each INFN user
○ HA K8s cluster on infrastructure backbone
○ FluxCD for gitOps operations
○ Central repo for OPA bundles
● On-demand cloud storage
○ Deploy the solution for a dedicated experiment/group of people
○ On prem or public cloud k8s instance as the ONLY requirement
○ Helm chart configurable via WebUI thanks to Kubeapps
24
Central service
FluxCD cluster
management
+
OPA bundles
___________
Full gitOps control
25
Self-managed k8s
- Ansible to bring up Kubeapps pointing to the supported INFN Helm charts
- Catalogue of pre configured apps already included
- Minio-Operator to deploy a Minio Tenant with STS credentials and OPA server
- Specifying custom OPA bundles endpoints and other similar configurations
26
● In production supporting physics and not only
○ (e.g. pandemic related research P.L.A.N.E.T.)
R&D continues toward:
● testing/scaling multi-cloud
● improving tools dedicated to data access and reuse
● try out Minio gateway cache instances to reduce latency
27
Wrapping up:
Summary and plans
Quick demo
28

More Related Content

What's hot

라이브드론맵 (Live Drone Map) - 실시간 드론 매핑 솔루션
라이브드론맵 (Live Drone Map) - 실시간 드론 매핑 솔루션라이브드론맵 (Live Drone Map) - 실시간 드론 매핑 솔루션
라이브드론맵 (Live Drone Map) - 실시간 드론 매핑 솔루션
Impyeong Lee
 

What's hot (20)

Implementation & Comparison Of Rdma Over Ethernet
Implementation & Comparison Of Rdma Over EthernetImplementation & Comparison Of Rdma Over Ethernet
Implementation & Comparison Of Rdma Over Ethernet
 
빅데이터 분석 시스템 도입과 AI 적용
빅데이터 분석 시스템 도입과 AI 적용빅데이터 분석 시스템 도입과 AI 적용
빅데이터 분석 시스템 도입과 AI 적용
 
네이버 클라우드 플랫폼의 컨테이너 기술 로드맵 (NBP 박기은 CTO) - NAVER CLOUD PLATFORM in [2018 All A...
네이버 클라우드 플랫폼의 컨테이너 기술 로드맵 (NBP 박기은 CTO) - NAVER CLOUD PLATFORM in [2018 All A...네이버 클라우드 플랫폼의 컨테이너 기술 로드맵 (NBP 박기은 CTO) - NAVER CLOUD PLATFORM in [2018 All A...
네이버 클라우드 플랫폼의 컨테이너 기술 로드맵 (NBP 박기은 CTO) - NAVER CLOUD PLATFORM in [2018 All A...
 
The Juniper SDN Landscape
The Juniper SDN LandscapeThe Juniper SDN Landscape
The Juniper SDN Landscape
 
Cloud Migration 과 Modernization 을 위한 30가지 아이디어-박기흥, AWS Migrations Specialist...
Cloud Migration 과 Modernization 을 위한 30가지 아이디어-박기흥, AWS Migrations Specialist...Cloud Migration 과 Modernization 을 위한 30가지 아이디어-박기흥, AWS Migrations Specialist...
Cloud Migration 과 Modernization 을 위한 30가지 아이디어-박기흥, AWS Migrations Specialist...
 
오픈소스의 이해(교육자료)
오픈소스의 이해(교육자료) 오픈소스의 이해(교육자료)
오픈소스의 이해(교육자료)
 
Apache Flink, AWS Kinesis, Analytics
Apache Flink, AWS Kinesis, Analytics Apache Flink, AWS Kinesis, Analytics
Apache Flink, AWS Kinesis, Analytics
 
AWS DevOps - Terraform, Docker, HashiCorp Vault
AWS DevOps - Terraform, Docker, HashiCorp VaultAWS DevOps - Terraform, Docker, HashiCorp Vault
AWS DevOps - Terraform, Docker, HashiCorp Vault
 
라이브드론맵 (Live Drone Map) - 실시간 드론 매핑 솔루션
라이브드론맵 (Live Drone Map) - 실시간 드론 매핑 솔루션라이브드론맵 (Live Drone Map) - 실시간 드론 매핑 솔루션
라이브드론맵 (Live Drone Map) - 실시간 드론 매핑 솔루션
 
WinFE: The (Almost) Perfect Triage Tool
WinFE: The (Almost) Perfect Triage ToolWinFE: The (Almost) Perfect Triage Tool
WinFE: The (Almost) Perfect Triage Tool
 
Operating Systems: Linux in Detail
Operating Systems: Linux in DetailOperating Systems: Linux in Detail
Operating Systems: Linux in Detail
 
What Linux can learn from Solaris performance and vice-versa
What Linux can learn from Solaris performance and vice-versaWhat Linux can learn from Solaris performance and vice-versa
What Linux can learn from Solaris performance and vice-versa
 
oSC22ww4.pdf
oSC22ww4.pdfoSC22ww4.pdf
oSC22ww4.pdf
 
Windows 11
Windows 11Windows 11
Windows 11
 
디지털트윈 기술 및 스마트시티 적용 사례
디지털트윈 기술 및  스마트시티 적용 사례 디지털트윈 기술 및  스마트시티 적용 사례
디지털트윈 기술 및 스마트시티 적용 사례
 
Lifecycle of a pod
Lifecycle of a podLifecycle of a pod
Lifecycle of a pod
 
FAT vs NTFS
FAT vs NTFSFAT vs NTFS
FAT vs NTFS
 
Oracle نوشته مهندس اسمعیل دخت
Oracle  نوشته مهندس اسمعیل دختOracle  نوشته مهندس اسمعیل دخت
Oracle نوشته مهندس اسمعیل دخت
 
eBPF maps 101
eBPF maps 101eBPF maps 101
eBPF maps 101
 
[WhaTap DevOps Day] 세션 1 : Observability Practice on AWS
[WhaTap DevOps Day] 세션 1 : Observability Practice on AWS[WhaTap DevOps Day] 세션 1 : Observability Practice on AWS
[WhaTap DevOps Day] 세션 1 : Observability Practice on AWS
 

Similar to stackconf 2021 | Setup Min.io and Open Policy Agent for a multi purpose scientific platform

Catania Science Gateway Framework
Catania Science Gateway FrameworkCatania Science Gateway Framework
Catania Science Gateway Framework
riround
 

Similar to stackconf 2021 | Setup Min.io and Open Policy Agent for a multi purpose scientific platform (20)

DDDP 2019 - Brown to Green
DDDP 2019  - Brown to GreenDDDP 2019  - Brown to Green
DDDP 2019 - Brown to Green
 
Cloud Services On UI and Ideas for Federated Cloud on idREN
Cloud Services On UI and Ideas for Federated Cloud on idRENCloud Services On UI and Ideas for Federated Cloud on idREN
Cloud Services On UI and Ideas for Federated Cloud on idREN
 
CTE Phase III
CTE Phase IIICTE Phase III
CTE Phase III
 
Catania Science Gateway Framework
Catania Science Gateway FrameworkCatania Science Gateway Framework
Catania Science Gateway Framework
 
All Things Open SDN, NFV and Open Daylight
All Things Open SDN, NFV and Open Daylight All Things Open SDN, NFV and Open Daylight
All Things Open SDN, NFV and Open Daylight
 
The Enterprise Guide to Building a Data Mesh - Introducing SpecMesh
The Enterprise Guide to Building a Data Mesh - Introducing SpecMeshThe Enterprise Guide to Building a Data Mesh - Introducing SpecMesh
The Enterprise Guide to Building a Data Mesh - Introducing SpecMesh
 
Finalpresentation
FinalpresentationFinalpresentation
Finalpresentation
 
Framework for IoT Interoperability
Framework for IoT InteroperabilityFramework for IoT Interoperability
Framework for IoT Interoperability
 
Using IO Visor to Secure Microservices Running on CloudFoundry [OpenStack Sum...
Using IO Visor to Secure Microservices Running on CloudFoundry [OpenStack Sum...Using IO Visor to Secure Microservices Running on CloudFoundry [OpenStack Sum...
Using IO Visor to Secure Microservices Running on CloudFoundry [OpenStack Sum...
 
Building Open Source Identity Infrastructures
Building Open Source Identity InfrastructuresBuilding Open Source Identity Infrastructures
Building Open Source Identity Infrastructures
 
Google's Infrastructure and Specific IoT Services
Google's Infrastructure and Specific IoT ServicesGoogle's Infrastructure and Specific IoT Services
Google's Infrastructure and Specific IoT Services
 
OpenDaylight nluug_november
OpenDaylight nluug_novemberOpenDaylight nluug_november
OpenDaylight nluug_november
 
Cisco project ideas
Cisco   project ideasCisco   project ideas
Cisco project ideas
 
Day 13 - Creating Data Processing Services | Train the Trainers Program
Day 13 - Creating Data Processing Services | Train the Trainers ProgramDay 13 - Creating Data Processing Services | Train the Trainers Program
Day 13 - Creating Data Processing Services | Train the Trainers Program
 
Federated Cloud Computing
Federated Cloud ComputingFederated Cloud Computing
Federated Cloud Computing
 
Building IAM for OpenStack
Building IAM for OpenStackBuilding IAM for OpenStack
Building IAM for OpenStack
 
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...
Red Hat Summit 2017 - LT107508 - Better Managing your Red Hat footprint with ...
 
Cloud as a GIFT: Exploiting Personal Cloud Free Accounts via Rest APIs
Cloud as a GIFT: Exploiting Personal Cloud Free Accounts via Rest APIsCloud as a GIFT: Exploiting Personal Cloud Free Accounts via Rest APIs
Cloud as a GIFT: Exploiting Personal Cloud Free Accounts via Rest APIs
 
Opensource tools for OpenStack IAAS
Opensource tools for OpenStack IAASOpensource tools for OpenStack IAAS
Opensource tools for OpenStack IAAS
 
DSD-INT 2016 Calibration and scenario generation of hydrodynamics and water -...
DSD-INT 2016 Calibration and scenario generation of hydrodynamics and water -...DSD-INT 2016 Calibration and scenario generation of hydrodynamics and water -...
DSD-INT 2016 Calibration and scenario generation of hydrodynamics and water -...
 

Recently uploaded

Mastering Windows 7 A Comprehensive Guide for Power Users .pdf
Mastering Windows 7 A Comprehensive Guide for Power Users .pdfMastering Windows 7 A Comprehensive Guide for Power Users .pdf
Mastering Windows 7 A Comprehensive Guide for Power Users .pdf
mbmh111980
 

Recently uploaded (20)

architecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdfarchitecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdf
 
AI Hackathon.pptx
AI                        Hackathon.pptxAI                        Hackathon.pptx
AI Hackathon.pptx
 
A Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data MigrationA Guideline to Zendesk to Re:amaze Data Migration
A Guideline to Zendesk to Re:amaze Data Migration
 
The Impact of PLM Software on Fashion Production
The Impact of PLM Software on Fashion ProductionThe Impact of PLM Software on Fashion Production
The Impact of PLM Software on Fashion Production
 
Malaysia E-Invoice digital signature docpptx
Malaysia E-Invoice digital signature docpptxMalaysia E-Invoice digital signature docpptx
Malaysia E-Invoice digital signature docpptx
 
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
Tree in the Forest - Managing Details in BDD Scenarios (live2test 2024)
 
SQL Injection Introduction and Prevention
SQL Injection Introduction and PreventionSQL Injection Introduction and Prevention
SQL Injection Introduction and Prevention
 
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdfThe Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
 
How to install and activate eGrabber JobGrabber
How to install and activate eGrabber JobGrabberHow to install and activate eGrabber JobGrabber
How to install and activate eGrabber JobGrabber
 
Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024
 
Mastering Windows 7 A Comprehensive Guide for Power Users .pdf
Mastering Windows 7 A Comprehensive Guide for Power Users .pdfMastering Windows 7 A Comprehensive Guide for Power Users .pdf
Mastering Windows 7 A Comprehensive Guide for Power Users .pdf
 
Crafting the Perfect Measurement Sheet with PLM Integration
Crafting the Perfect Measurement Sheet with PLM IntegrationCrafting the Perfect Measurement Sheet with PLM Integration
Crafting the Perfect Measurement Sheet with PLM Integration
 
Naer Toolbar Redesign - Usability Research Synthesis
Naer Toolbar Redesign - Usability Research SynthesisNaer Toolbar Redesign - Usability Research Synthesis
Naer Toolbar Redesign - Usability Research Synthesis
 
Microsoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMicrosoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdf
 
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product UpdatesGraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
GraphSummit Stockholm - Neo4j - Knowledge Graphs and Product Updates
 
Microsoft 365 Copilot; An AI tool changing the world of work _PDF.pdf
Microsoft 365 Copilot; An AI tool changing the world of work _PDF.pdfMicrosoft 365 Copilot; An AI tool changing the world of work _PDF.pdf
Microsoft 365 Copilot; An AI tool changing the world of work _PDF.pdf
 
What need to be mastered as AI-Powered Java Developers
What need to be mastered as AI-Powered Java DevelopersWhat need to be mastered as AI-Powered Java Developers
What need to be mastered as AI-Powered Java Developers
 
AI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning Framework
 
AI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in MichelangeloAI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in Michelangelo
 
AI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAG
AI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAGAI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAG
AI/ML Infra Meetup | Reducing Prefill for LLM Serving in RAG
 

stackconf 2021 | Setup Min.io and Open Policy Agent for a multi purpose scientific platform

  • 1. Setup Min.io and Open Policy Agent for a multi-purpose scientific platform D. Ciangottini, INFN
  • 2. stackconf-2021 $ > whoami ● IT Researcher at Istituto Nazionale di Fisica Nucleare (INFN) ○ Translated: National Institute of Nuclear Physics ● Involved on R&D activities to deploy cloud-native solutions for the next-gen of data analysis infrastructure for the INFN/LHC users 2
  • 3. INFN community 3 The 5 research lines and the National Scientific Committee
  • 4. 4 One of the main scientific challenges understand the Universe in the first moments after the Big Ban
  • 5. 5 A community of over 6,000 people ~ 25% of them have PhD grants, post-doc scholarships and research grants
  • 7. Computing @INFN Long tradition supporting experiments For last 10 years, that was meant for supporting LHC communities. Quickly widening to many other use cases recently. 7
  • 8. On-demand computing resources for the INFN communities ● Easy access to on-demand solutions for scientific data analysis ● Composable services to extend and customize the environment ● Provide INFN users with a set of core tools centrally managed ○ E.g. JupyterHub-aaS, object storage, sync&share .... ● Federating the resources from several centers at national level ● Becoming the hub of reference for most of the activities and projects @ INFN 8 The INFN-Cloud initiative
  • 9. The INFN-Cloud infrastructure A backbone composed by the main computing centers for central services + a federation of smaller sites providing resources for user deployments 9
  • 10. Computing challenges Data storage for multiple communities Providing a cloud storage hosted on the backbone infrastructure means: ● Geo-distributed storage federation ● Heterogeneous set of requirements ○ Object size (few MBs, to 10s GBs) ○ Workflow (imaging, columnar analysis...) and data access (posix, webdav, s3 etc..) but it also means providing the tools: ● “F.A.I.R.” data ○ Findable, Accessible, Interoperable, Reusable ○ Make it intuitive or eventually transparent for the end user ● Focus on the “R.”! Allow sustainable reuse of data 10
  • 11. Wrapping up... Requirements ● Dynamic user registration/acls integrated with Indigo-IAM/OIDC ● Fine grain authz (ro, rw, per file/per user group ) ● Easy and robust ops ○ gitOps eventually ● Accessible via posix ● WebUI access ● Vendor neutral ● Open source 11
  • 12. Quick look to the solution The components ● Minio has been chosen as the cloud storage solution ○ S3 compliance ○ Powerful WebUI ○ Proven scalability ● Native integration with AWS STS credentials ○ External OIDC IdP’s (e.g. Indigo IAM) ● Support for customizable authZ policies with OpenPolicyAgent 12
  • 13. User management Indigo-IAM ● Authentication via SAML IdPs or identity federations, OpenID Connect providers and X.509 certificates ● Enrollment and registration functionalities ○ so that users can join groups/collaborations according to well-defined flows ○ provides services to manage group membership ○ attributes assignment and account linking functionality ● Integrable as IdP with any OIDC compliant service 13
  • 14. Cloud storage AuthN: AWS STS credentials ● Endpoint service that enables clients to request temporary credentials for MinIO resources ● AWS AssumeRoleWithWebIdentity flow is supported out of the box ○ Allowing the integration with any OpenID Connect-compatible identity provider ⇒ our IAM service 14
  • 15. Cloud storage AuthZ: OpenPolicyAgent integration ● A lightweight general-purpose policy engine that can be co-located with Minio server ● OPA HTTP API used to authorize Minio STS credentials ○ Fine grain ACLs ■ Every token claim from authN can be selected for policy checking ○ Dynamic config ○ Decoupled from the storage configuration
  • 16. Example of an e2e AuthZ flow 16 OPA server checks custom policies for the input API Policy example # Allow users to manage their own data. allow { username := split(lower(input.claims.email),"@")[0] input.bucket == username input.claims.aud == "minio-auth" permissions := rl_permissions["user"] p := permissions[_] p == {"action": input.action} } Authorized JWT List of operation permissions defined on OPA "claims": { "accessKey": "VP43M6DO1N53U2LUBTZ3", "aud": "https://wlcg.cern.ch/jwt/v1/any", "client_id": "5c38c020-b753-4115-a5f4-3f48595e4c1b", "exp": "1621714730", "iat": 1621713801, "iss": "https://login.cloud.infn.it", "scope": "openid profile email", "email": “ciangottini@infn.it” } Minio STS auth data
  • 17. Managing policies with OPA bundles the gitOps way ● OPA can periodically download bundles of policy and data from remote HTTP servers ○ Allowing for a gitOps based policy management ● The policies and data are loaded on the fly without requiring a restart of OPA ○ Policies and data are then applied immediately 17
  • 18. So far so good… Let’s put the hands on some user tools now! 18
  • 19. Managing temporary credentials: OIDC-agent ● A set of tools to manage OpenID Connect access tokens and make them easily usable from the command line ○ ssh-agent design, so users can handle OIDC tokens in a similar way as they do with ssh keys ● Secure sensible information (long living credentials) while exposing short lived ones (e.g. access token) ● Integrable via API libraries for: python, go and c++ 19
  • 20. POSIX access: RClone + OIDC-Agent integration To provide posix access we make use of RClone mount capability A small patch has been applied to add a dedicated S3 provider integrated with OIDC-Agent Users, once oidc-agent is configured on its VM, can then mount its own bucket as a folder with no further actions/authentication steps. Backups via Restic are enabled through the use of this patched version of RClone 20
  • 21. Make it easier: STS-wire 21 For cases where the user does not/cannot run oidc-agent: ● a tool has been created to manage both the credential renewal and the rclone mount in a guided/integrated/opinionated way We found that to be the preferred solution to mount a bucket content on a laptop for instance.
  • 22. What about python? boto3+STS+OIDC-Agent = boto3STS Access Minio bucket through the integration of boto library with temporary credentials - AWS STS token via IAM - IAM access token get via oidc-agent API Instantiate an S3 session with a simple line of code 22
  • 23. Nice stuff! So, how did you do that? 23
  • 24. Deployment models ● Generic centrally maintained service for each INFN user ○ HA K8s cluster on infrastructure backbone ○ FluxCD for gitOps operations ○ Central repo for OPA bundles ● On-demand cloud storage ○ Deploy the solution for a dedicated experiment/group of people ○ On prem or public cloud k8s instance as the ONLY requirement ○ Helm chart configurable via WebUI thanks to Kubeapps 24
  • 25. Central service FluxCD cluster management + OPA bundles ___________ Full gitOps control 25
  • 26. Self-managed k8s - Ansible to bring up Kubeapps pointing to the supported INFN Helm charts - Catalogue of pre configured apps already included - Minio-Operator to deploy a Minio Tenant with STS credentials and OPA server - Specifying custom OPA bundles endpoints and other similar configurations 26
  • 27. ● In production supporting physics and not only ○ (e.g. pandemic related research P.L.A.N.E.T.) R&D continues toward: ● testing/scaling multi-cloud ● improving tools dedicated to data access and reuse ● try out Minio gateway cache instances to reduce latency 27 Wrapping up: Summary and plans