SlideShare a Scribd company logo
SWAN and its analysis
ecosystem
D. Castro, J. Moscicki, M. Lamanna, E. Bocchi,
E. Tejedor, D. Piparo, P. Mato, P. Kothuri
Jan 29th, 2019
CS3 2019 - Cloud Storage Synchronization and Sharing Services
https://cern.ch/swan
Introduction
2
SWAN in a Nutshell
﹥Analysis only with a web browser
 No local installation needed
 Based on Jupyter Notebooks
 Calculations, input data and results “in the Cloud”
﹥Support for multiple analysis ecosystems
 ROOT, Python, R, Octave…
﹥Easy sharing of scientific results: plots, data,
code
﹥Integration with CERN resources
 Sofware, storage, mass processing power
3
Integrating services
Software Storage
Infrastructure
4
Storage
﹥Uses EOS disk storage system
 All experiment data potentially available
﹥CERNBox is SWAN's home directory
 Storage for your notebooks and data
﹥Sync&Share
 Files synced across devices and the
Cloud
 Collaborative analysis
5
Software
﹥Software distributed through CVMFS
 ”LCG Releases” - pack a series of compatible packages
 Reduced Docker Images size
 Lazy fetching of software
﹥Possibility to install libraries in user cloud storage
 Good way to use custom/not mainstream packages
 Configurable environment
6
LCG Release
CERN
Software
User
Software
Jupyter
modules
Previously on last CS3 conference…
7
New User Interface
8
New User Interface
9
Sharing made easy
﹥Sharing from inside
SWAN interface
 Integration with CERNBox
﹥Users can share
“Projects”
 Special kind of folder that
contains notebooks and
other files, like input data
 Self contained
10
The Share tab
﹥Users can list which projects...
 they have shared
 others have shared with them
﹥Projects can be cloned to the
receiver's CERNBox
 The receiver will work on his own copy
﹥Concurrent editing not supported by
Jupyter
 Safer to clone
11
Spark Cluster
Integration with Spark
﹥Connection to CERN
Spark Clusters
﹥Same environment
across platforms
 User data - EOS
 Software - CVMFS
﹥Graphical Jupyter
extensions developed
 Spark Connector
 Spark Monitor
Spark Master
Spark Worker
Python task Python task Python task
User Notebook
12
Spark Connector/Monitor
13
The result
14
Stats
﹥~200 user sessions a day on
average
 Users doubled last year with new SWAN
interface
﹥~1300 unique users in 6 months
﹥Spark cluster connection: 15 – 20 %
of users
 SWAN as entry point for accessing
computational resources
 Used for monitoring LHC accelerator
hardware devices (NXCals)
15
Courses
New developments
16
Inspecting a Project
﹥Users can inspect shared
project contents
 Browsing of the files
 Static rendering of
notebooks
﹥Useful to decide whether
to accept or not the
shared project
17
Spark improvements
18
1919Worldwide LHC Computing Grid (WLCG)
Connecting More Resources
﹥Ongoing effort: submit
batch jobs from the
notebook
 Monitoring display
 Jobs tab
20
Outreach, Education
21
Science Box: SWAN on Premises
﹥Packaged deployment of SWAN
 Includes all SWAN components: CERNBox/EOS, CVMFS, JupyterHub
﹥Deployable through Kubernetes or docker-compose
﹥Some successful community installations
 AARNet
 PSNC
 Open Telekom Cloud (Helix Nebula)
22
Science Box: SWAN on Premises
﹥UP2University European Project
 Bridge the gap between secondary schools, higher education and the
research domain
 Partner universities (OU, UROMA, NTUA, …), pilot schools
 http://up2university.eu
﹥SWAN used by students to learn physics and other sciences
 Let them use the very same tools & services used by scientists at CERN
 Pilot with University of Geneva (Physiscope)
﹥Establishing collaboration with Callysto project
23
Looking ahead
24
Future work/challenges
﹥Move to Jupyterlab
 Porting the current extensions
 Concurrent editing
﹥New architecture
 Based on Kubernetes
﹥Exploitation of GPUs
 HEP is looking to ML
 Speed up computation of GPU-ready libraries (e.g. TensorFlow)
25
Where to find us
26
Where to find us
﹥Contacts
 swan-talk@cern.ch
 http://cern.ch/swan
﹥Repository
 https://github.com/swan-cern/
﹥Science Box
 https://cern.ch/sciencebox
27
Conclusion
28
Conclusion
﹥Changes introduced since last year improved user experience
 Which translated on more users using the service
﹥SWAN became a fundamental Interface for Mass Processing Resources (Spark)
 Not only for Physics analysis but also for monitoring the LHC hardware
﹥The new Jupyterlab interface will bring new possibilities for collaborative analysis
 With the introduction of concurrent editing of notebooks
 Which can help reach more users
﹥Successfully deployed outside CERN premises
 Including on education related projects
29
SWAN and its analysis ecosystem
Thank you
Diogo Castro
diogo.castro@cern.ch
30

More Related Content

What's hot

Virtual Clusters for (RDF) Stream Processing
Virtual Clusters for (RDF) Stream ProcessingVirtual Clusters for (RDF) Stream Processing
Virtual Clusters for (RDF) Stream Processing
Alejandro Llaves
 
Project Matsu: Elastic Clouds for Disaster Relief
Project Matsu: Elastic Clouds for Disaster ReliefProject Matsu: Elastic Clouds for Disaster Relief
Project Matsu: Elastic Clouds for Disaster Relief
Robert Grossman
 
My Other Computer is a Data Center: The Sector Perspective on Big Data
My Other Computer is a Data Center: The Sector Perspective on Big DataMy Other Computer is a Data Center: The Sector Perspective on Big Data
My Other Computer is a Data Center: The Sector Perspective on Big Data
Robert Grossman
 
Bionimbus - An Overview (2010-v6)
Bionimbus - An Overview (2010-v6)Bionimbus - An Overview (2010-v6)
Bionimbus - An Overview (2010-v6)
Robert Grossman
 
An Experiment-Driven Performance Model of Stream Processing Operators in Fog ...
An Experiment-Driven Performance Model of Stream Processing Operators in Fog ...An Experiment-Driven Performance Model of Stream Processing Operators in Fog ...
An Experiment-Driven Performance Model of Stream Processing Operators in Fog ...
FogGuru MSCA Project
 
Stream Processing
Stream Processing Stream Processing
Stream Processing
FogGuru MSCA Project
 
Applications of PARALLEL PROCESSING
Applications of PARALLEL PROCESSING Applications of PARALLEL PROCESSING
Applications of PARALLEL PROCESSING
Praveen Kumar
 
Fog Computing for Dummies
Fog Computing for Dummies Fog Computing for Dummies
Fog Computing for Dummies
FogGuru MSCA Project
 
Panel at Internet2 Spring Meeting, April 2010
Panel at Internet2 Spring Meeting,  April 2010Panel at Internet2 Spring Meeting,  April 2010
Panel at Internet2 Spring Meeting, April 2010
University of Illinois at Urbana-Champaign
 
From Cloud to Fog: the Tao of IT Infrastructure Decentralization
From Cloud to Fog: the Tao of IT Infrastructure DecentralizationFrom Cloud to Fog: the Tao of IT Infrastructure Decentralization
From Cloud to Fog: the Tao of IT Infrastructure Decentralization
FogGuru MSCA Project
 
Data Stream Algorithms in Storm and R
Data Stream Algorithms in Storm and RData Stream Algorithms in Storm and R
Data Stream Algorithms in Storm and R
Radek Maciaszek
 
Novel Techniques & Connections Between High-Pressure Mineral Physics, Microto...
Novel Techniques & Connections Between High-Pressure Mineral Physics, Microto...Novel Techniques & Connections Between High-Pressure Mineral Physics, Microto...
Novel Techniques & Connections Between High-Pressure Mineral Physics, Microto...
EarthCube
 
The DuraMat Data Hub and Analytics Capability: A Resource for Solar PV Data
The DuraMat Data Hub and Analytics Capability: A Resource for Solar PV DataThe DuraMat Data Hub and Analytics Capability: A Resource for Solar PV Data
The DuraMat Data Hub and Analytics Capability: A Resource for Solar PV Data
Anubhav Jain
 
Nasa HPC in the Cloud
Nasa HPC in the CloudNasa HPC in the Cloud
Nasa HPC in the Cloud
Adianto Wibisono
 
Atomate: a high-level interface to generate, execute, and analyze computation...
Atomate: a high-level interface to generate, execute, and analyze computation...Atomate: a high-level interface to generate, execute, and analyze computation...
Atomate: a high-level interface to generate, execute, and analyze computation...
Anubhav Jain
 
Overview of DuraMat software tool development
Overview of DuraMat software tool developmentOverview of DuraMat software tool development
Overview of DuraMat software tool development
Anubhav Jain
 
Taming Big Data!
Taming Big Data!Taming Big Data!
Taming Big Data!
Ian Foster
 
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Otávio Carvalho
 
io-Chem-BD, una solució per gestionar el Big Data en Química Computacional
io-Chem-BD, una solució per gestionar el Big Data en Química Computacionalio-Chem-BD, una solució per gestionar el Big Data en Química Computacional
io-Chem-BD, una solució per gestionar el Big Data en Química Computacional
CSUC - Consorci de Serveis Universitaris de Catalunya
 
Materials Project computation and database infrastructure
Materials Project computation and database infrastructureMaterials Project computation and database infrastructure
Materials Project computation and database infrastructure
Anubhav Jain
 

What's hot (20)

Virtual Clusters for (RDF) Stream Processing
Virtual Clusters for (RDF) Stream ProcessingVirtual Clusters for (RDF) Stream Processing
Virtual Clusters for (RDF) Stream Processing
 
Project Matsu: Elastic Clouds for Disaster Relief
Project Matsu: Elastic Clouds for Disaster ReliefProject Matsu: Elastic Clouds for Disaster Relief
Project Matsu: Elastic Clouds for Disaster Relief
 
My Other Computer is a Data Center: The Sector Perspective on Big Data
My Other Computer is a Data Center: The Sector Perspective on Big DataMy Other Computer is a Data Center: The Sector Perspective on Big Data
My Other Computer is a Data Center: The Sector Perspective on Big Data
 
Bionimbus - An Overview (2010-v6)
Bionimbus - An Overview (2010-v6)Bionimbus - An Overview (2010-v6)
Bionimbus - An Overview (2010-v6)
 
An Experiment-Driven Performance Model of Stream Processing Operators in Fog ...
An Experiment-Driven Performance Model of Stream Processing Operators in Fog ...An Experiment-Driven Performance Model of Stream Processing Operators in Fog ...
An Experiment-Driven Performance Model of Stream Processing Operators in Fog ...
 
Stream Processing
Stream Processing Stream Processing
Stream Processing
 
Applications of PARALLEL PROCESSING
Applications of PARALLEL PROCESSING Applications of PARALLEL PROCESSING
Applications of PARALLEL PROCESSING
 
Fog Computing for Dummies
Fog Computing for Dummies Fog Computing for Dummies
Fog Computing for Dummies
 
Panel at Internet2 Spring Meeting, April 2010
Panel at Internet2 Spring Meeting,  April 2010Panel at Internet2 Spring Meeting,  April 2010
Panel at Internet2 Spring Meeting, April 2010
 
From Cloud to Fog: the Tao of IT Infrastructure Decentralization
From Cloud to Fog: the Tao of IT Infrastructure DecentralizationFrom Cloud to Fog: the Tao of IT Infrastructure Decentralization
From Cloud to Fog: the Tao of IT Infrastructure Decentralization
 
Data Stream Algorithms in Storm and R
Data Stream Algorithms in Storm and RData Stream Algorithms in Storm and R
Data Stream Algorithms in Storm and R
 
Novel Techniques & Connections Between High-Pressure Mineral Physics, Microto...
Novel Techniques & Connections Between High-Pressure Mineral Physics, Microto...Novel Techniques & Connections Between High-Pressure Mineral Physics, Microto...
Novel Techniques & Connections Between High-Pressure Mineral Physics, Microto...
 
The DuraMat Data Hub and Analytics Capability: A Resource for Solar PV Data
The DuraMat Data Hub and Analytics Capability: A Resource for Solar PV DataThe DuraMat Data Hub and Analytics Capability: A Resource for Solar PV Data
The DuraMat Data Hub and Analytics Capability: A Resource for Solar PV Data
 
Nasa HPC in the Cloud
Nasa HPC in the CloudNasa HPC in the Cloud
Nasa HPC in the Cloud
 
Atomate: a high-level interface to generate, execute, and analyze computation...
Atomate: a high-level interface to generate, execute, and analyze computation...Atomate: a high-level interface to generate, execute, and analyze computation...
Atomate: a high-level interface to generate, execute, and analyze computation...
 
Overview of DuraMat software tool development
Overview of DuraMat software tool developmentOverview of DuraMat software tool development
Overview of DuraMat software tool development
 
Taming Big Data!
Taming Big Data!Taming Big Data!
Taming Big Data!
 
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
 
io-Chem-BD, una solució per gestionar el Big Data en Química Computacional
io-Chem-BD, una solució per gestionar el Big Data en Química Computacionalio-Chem-BD, una solució per gestionar el Big Data en Química Computacional
io-Chem-BD, una solució per gestionar el Big Data en Química Computacional
 
Materials Project computation and database infrastructure
Materials Project computation and database infrastructureMaterials Project computation and database infrastructure
Materials Project computation and database infrastructure
 

Similar to 2019 swan-cs3

Thesies_Cheng_Guo_2015_fina_signed
Thesies_Cheng_Guo_2015_fina_signedThesies_Cheng_Guo_2015_fina_signed
Thesies_Cheng_Guo_2015_fina_signed
Cheng Guo
 
Data-intensive applications on cloud computing resources: Applications in lif...
Data-intensive applications on cloud computing resources: Applications in lif...Data-intensive applications on cloud computing resources: Applications in lif...
Data-intensive applications on cloud computing resources: Applications in lif...
Ola Spjuth
 
[3.6] Beyond Data Sharing - Pieter van Gorp [3TU.Datacentrum Symposium 2014, ...
[3.6] Beyond Data Sharing - Pieter van Gorp [3TU.Datacentrum Symposium 2014, ...[3.6] Beyond Data Sharing - Pieter van Gorp [3TU.Datacentrum Symposium 2014, ...
[3.6] Beyond Data Sharing - Pieter van Gorp [3TU.Datacentrum Symposium 2014, ...
3TU.Datacentrum
 
Scientific
Scientific Scientific
Scientific
marpierc
 
OpenACC Monthly Highlights Summer 2019
OpenACC Monthly Highlights Summer 2019OpenACC Monthly Highlights Summer 2019
OpenACC Monthly Highlights Summer 2019
OpenACC
 
R.E.M.O.T.E. SACNAS Poster
R.E.M.O.T.E. SACNAS PosterR.E.M.O.T.E. SACNAS Poster
R.E.M.O.T.E. SACNAS Poster
Olmo F. Maldonado
 
Open Science and GEOSS: the Cloud Sandbox enablers
Open Science and GEOSS: the Cloud Sandbox enablersOpen Science and GEOSS: the Cloud Sandbox enablers
Open Science and GEOSS: the Cloud Sandbox enablers
terradue
 
Larry Smarr - NRP Application Drivers
Larry Smarr - NRP Application DriversLarry Smarr - NRP Application Drivers
Larry Smarr - NRP Application Drivers
Larry Smarr
 
Cloud Services for Education - HNSciCloud applied to the UP2U project
Cloud Services for Education - HNSciCloud applied to the UP2U projectCloud Services for Education - HNSciCloud applied to the UP2U project
Cloud Services for Education - HNSciCloud applied to the UP2U project
Helix Nebula The Science Cloud
 
GeoChronos
GeoChronosGeoChronos
GeoChronos
curryr
 
Data-intensive bioinformatics on HPC and Cloud
Data-intensive bioinformatics on HPC and CloudData-intensive bioinformatics on HPC and Cloud
Data-intensive bioinformatics on HPC and Cloud
Ola Spjuth
 
Virtual research environments for implementing long tail open science
Virtual research environments for implementing long tail open scienceVirtual research environments for implementing long tail open science
Virtual research environments for implementing long tail open science
Blue BRIDGE
 
OpenACC Monthly Highlights: January 2024
OpenACC Monthly Highlights: January 2024OpenACC Monthly Highlights: January 2024
OpenACC Monthly Highlights: January 2024
OpenACC
 
2016 nov-ieee-sdn-wiki
2016 nov-ieee-sdn-wiki2016 nov-ieee-sdn-wiki
2016 nov-ieee-sdn-wiki
Christian Esteve Rothenberg
 
grid computing
grid computinggrid computing
grid computing
elliando dias
 
Deep Learning on Apache Spark at CERN’s Large Hadron Collider with Intel Tech...
Deep Learning on Apache Spark at CERN’s Large Hadron Collider with Intel Tech...Deep Learning on Apache Spark at CERN’s Large Hadron Collider with Intel Tech...
Deep Learning on Apache Spark at CERN’s Large Hadron Collider with Intel Tech...
Databricks
 
DAWN and Scientific Workflows
DAWN and Scientific WorkflowsDAWN and Scientific Workflows
DAWN and Scientific Workflows
Matthew Gerring
 
Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...
Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...
Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...
Ian Foster
 
Scaling People, Not Just Systems, to Take On Big Data Challenges
Scaling People, Not Just Systems, to Take On Big Data ChallengesScaling People, Not Just Systems, to Take On Big Data Challenges
Scaling People, Not Just Systems, to Take On Big Data Challenges
Matthew Vaughn
 
Openflow wp-latest
Openflow wp-latestOpenflow wp-latest
Openflow wp-latest
KellyCheah
 

Similar to 2019 swan-cs3 (20)

Thesies_Cheng_Guo_2015_fina_signed
Thesies_Cheng_Guo_2015_fina_signedThesies_Cheng_Guo_2015_fina_signed
Thesies_Cheng_Guo_2015_fina_signed
 
Data-intensive applications on cloud computing resources: Applications in lif...
Data-intensive applications on cloud computing resources: Applications in lif...Data-intensive applications on cloud computing resources: Applications in lif...
Data-intensive applications on cloud computing resources: Applications in lif...
 
[3.6] Beyond Data Sharing - Pieter van Gorp [3TU.Datacentrum Symposium 2014, ...
[3.6] Beyond Data Sharing - Pieter van Gorp [3TU.Datacentrum Symposium 2014, ...[3.6] Beyond Data Sharing - Pieter van Gorp [3TU.Datacentrum Symposium 2014, ...
[3.6] Beyond Data Sharing - Pieter van Gorp [3TU.Datacentrum Symposium 2014, ...
 
Scientific
Scientific Scientific
Scientific
 
OpenACC Monthly Highlights Summer 2019
OpenACC Monthly Highlights Summer 2019OpenACC Monthly Highlights Summer 2019
OpenACC Monthly Highlights Summer 2019
 
R.E.M.O.T.E. SACNAS Poster
R.E.M.O.T.E. SACNAS PosterR.E.M.O.T.E. SACNAS Poster
R.E.M.O.T.E. SACNAS Poster
 
Open Science and GEOSS: the Cloud Sandbox enablers
Open Science and GEOSS: the Cloud Sandbox enablersOpen Science and GEOSS: the Cloud Sandbox enablers
Open Science and GEOSS: the Cloud Sandbox enablers
 
Larry Smarr - NRP Application Drivers
Larry Smarr - NRP Application DriversLarry Smarr - NRP Application Drivers
Larry Smarr - NRP Application Drivers
 
Cloud Services for Education - HNSciCloud applied to the UP2U project
Cloud Services for Education - HNSciCloud applied to the UP2U projectCloud Services for Education - HNSciCloud applied to the UP2U project
Cloud Services for Education - HNSciCloud applied to the UP2U project
 
GeoChronos
GeoChronosGeoChronos
GeoChronos
 
Data-intensive bioinformatics on HPC and Cloud
Data-intensive bioinformatics on HPC and CloudData-intensive bioinformatics on HPC and Cloud
Data-intensive bioinformatics on HPC and Cloud
 
Virtual research environments for implementing long tail open science
Virtual research environments for implementing long tail open scienceVirtual research environments for implementing long tail open science
Virtual research environments for implementing long tail open science
 
OpenACC Monthly Highlights: January 2024
OpenACC Monthly Highlights: January 2024OpenACC Monthly Highlights: January 2024
OpenACC Monthly Highlights: January 2024
 
2016 nov-ieee-sdn-wiki
2016 nov-ieee-sdn-wiki2016 nov-ieee-sdn-wiki
2016 nov-ieee-sdn-wiki
 
grid computing
grid computinggrid computing
grid computing
 
Deep Learning on Apache Spark at CERN’s Large Hadron Collider with Intel Tech...
Deep Learning on Apache Spark at CERN’s Large Hadron Collider with Intel Tech...Deep Learning on Apache Spark at CERN’s Large Hadron Collider with Intel Tech...
Deep Learning on Apache Spark at CERN’s Large Hadron Collider with Intel Tech...
 
DAWN and Scientific Workflows
DAWN and Scientific WorkflowsDAWN and Scientific Workflows
DAWN and Scientific Workflows
 
Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...
Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...
Science Services and Science Platforms: Using the Cloud to Accelerate and Dem...
 
Scaling People, Not Just Systems, to Take On Big Data Challenges
Scaling People, Not Just Systems, to Take On Big Data ChallengesScaling People, Not Just Systems, to Take On Big Data Challenges
Scaling People, Not Just Systems, to Take On Big Data Challenges
 
Openflow wp-latest
Openflow wp-latestOpenflow wp-latest
Openflow wp-latest
 

More from Up2Universe

Up2U Pedagogical evaluation
Up2U Pedagogical evaluationUp2U Pedagogical evaluation
Up2U Pedagogical evaluation
Up2Universe
 
Continuous professional development for secondary education teachers to adopt...
Continuous professional development for secondary education teachers to adopt...Continuous professional development for secondary education teachers to adopt...
Continuous professional development for secondary education teachers to adopt...
Up2Universe
 
Up2U brand manual
Up2U brand manualUp2U brand manual
Up2U brand manual
Up2Universe
 
openUp2U booklet
openUp2U bookletopenUp2U booklet
openUp2U booklet
Up2Universe
 
Why choose Up2U?
Why choose Up2U?Why choose Up2U?
Why choose Up2U?
Up2Universe
 
Up2U step by step guides for NRENs
Up2U step by step guides for NRENsUp2U step by step guides for NRENs
Up2U step by step guides for NRENs
Up2Universe
 
Up2U for schools booklet
Up2U for schools bookletUp2U for schools booklet
Up2U for schools booklet
Up2Universe
 
Open Educational Resources for Bridging High School – University Gaps in Acad...
Open Educational Resources for Bridging High School – University Gaps in Acad...Open Educational Resources for Bridging High School – University Gaps in Acad...
Open Educational Resources for Bridging High School – University Gaps in Acad...
Up2Universe
 
Greek IT security flyer
Greek IT security flyerGreek IT security flyer
Greek IT security flyer
Up2Universe
 
Edulearn2019_Up2U_Presentation_G.Cibulskis_A.Urbaityte
Edulearn2019_Up2U_Presentation_G.Cibulskis_A.UrbaityteEdulearn2019_Up2U_Presentation_G.Cibulskis_A.Urbaityte
Edulearn2019_Up2U_Presentation_G.Cibulskis_A.Urbaityte
Up2Universe
 
Pilots results- lessons learned Up2University project
Pilots results- lessons learned Up2University projectPilots results- lessons learned Up2University project
Pilots results- lessons learned Up2University project
Up2Universe
 
Praktyczny przewodnik po bezpieczeństwie teleinformatycznym Up2U
Praktyczny przewodnik po bezpieczeństwie teleinformatycznym Up2UPraktyczny przewodnik po bezpieczeństwie teleinformatycznym Up2U
Praktyczny przewodnik po bezpieczeństwie teleinformatycznym Up2U
Up2Universe
 
IT biztonsági kisokos
IT biztonsági kisokosIT biztonsági kisokos
IT biztonsági kisokos
Up2Universe
 
Guida pratica alla sicurezza ICT per il progetto Up2U
Guida pratica alla sicurezza ICT per il progetto Up2UGuida pratica alla sicurezza ICT per il progetto Up2U
Guida pratica alla sicurezza ICT per il progetto Up2U
Up2Universe
 
Una guía práctica para la seguridad TIC-Up2U
Una guía práctica para la seguridad TIC-Up2UUna guía práctica para la seguridad TIC-Up2U
Una guía práctica para la seguridad TIC-Up2U
Up2Universe
 
A practical guide to IT security-Up to University project
A practical guide to IT security-Up to University projectA practical guide to IT security-Up to University project
A practical guide to IT security-Up to University project
Up2Universe
 
Facilitating curation of open educational resources through the use of an app...
Facilitating curation of open educational resources through the use of an app...Facilitating curation of open educational resources through the use of an app...
Facilitating curation of open educational resources through the use of an app...
Up2Universe
 
Up2U Learning Community interactions
Up2U Learning Community interactionsUp2U Learning Community interactions
Up2U Learning Community interactions
Up2Universe
 
Up to University
Up to UniversityUp to University
Up to University
Up2Universe
 
Up2U webinar for NRENs
Up2U webinar for NRENsUp2U webinar for NRENs
Up2U webinar for NRENs
Up2Universe
 

More from Up2Universe (20)

Up2U Pedagogical evaluation
Up2U Pedagogical evaluationUp2U Pedagogical evaluation
Up2U Pedagogical evaluation
 
Continuous professional development for secondary education teachers to adopt...
Continuous professional development for secondary education teachers to adopt...Continuous professional development for secondary education teachers to adopt...
Continuous professional development for secondary education teachers to adopt...
 
Up2U brand manual
Up2U brand manualUp2U brand manual
Up2U brand manual
 
openUp2U booklet
openUp2U bookletopenUp2U booklet
openUp2U booklet
 
Why choose Up2U?
Why choose Up2U?Why choose Up2U?
Why choose Up2U?
 
Up2U step by step guides for NRENs
Up2U step by step guides for NRENsUp2U step by step guides for NRENs
Up2U step by step guides for NRENs
 
Up2U for schools booklet
Up2U for schools bookletUp2U for schools booklet
Up2U for schools booklet
 
Open Educational Resources for Bridging High School – University Gaps in Acad...
Open Educational Resources for Bridging High School – University Gaps in Acad...Open Educational Resources for Bridging High School – University Gaps in Acad...
Open Educational Resources for Bridging High School – University Gaps in Acad...
 
Greek IT security flyer
Greek IT security flyerGreek IT security flyer
Greek IT security flyer
 
Edulearn2019_Up2U_Presentation_G.Cibulskis_A.Urbaityte
Edulearn2019_Up2U_Presentation_G.Cibulskis_A.UrbaityteEdulearn2019_Up2U_Presentation_G.Cibulskis_A.Urbaityte
Edulearn2019_Up2U_Presentation_G.Cibulskis_A.Urbaityte
 
Pilots results- lessons learned Up2University project
Pilots results- lessons learned Up2University projectPilots results- lessons learned Up2University project
Pilots results- lessons learned Up2University project
 
Praktyczny przewodnik po bezpieczeństwie teleinformatycznym Up2U
Praktyczny przewodnik po bezpieczeństwie teleinformatycznym Up2UPraktyczny przewodnik po bezpieczeństwie teleinformatycznym Up2U
Praktyczny przewodnik po bezpieczeństwie teleinformatycznym Up2U
 
IT biztonsági kisokos
IT biztonsági kisokosIT biztonsági kisokos
IT biztonsági kisokos
 
Guida pratica alla sicurezza ICT per il progetto Up2U
Guida pratica alla sicurezza ICT per il progetto Up2UGuida pratica alla sicurezza ICT per il progetto Up2U
Guida pratica alla sicurezza ICT per il progetto Up2U
 
Una guía práctica para la seguridad TIC-Up2U
Una guía práctica para la seguridad TIC-Up2UUna guía práctica para la seguridad TIC-Up2U
Una guía práctica para la seguridad TIC-Up2U
 
A practical guide to IT security-Up to University project
A practical guide to IT security-Up to University projectA practical guide to IT security-Up to University project
A practical guide to IT security-Up to University project
 
Facilitating curation of open educational resources through the use of an app...
Facilitating curation of open educational resources through the use of an app...Facilitating curation of open educational resources through the use of an app...
Facilitating curation of open educational resources through the use of an app...
 
Up2U Learning Community interactions
Up2U Learning Community interactionsUp2U Learning Community interactions
Up2U Learning Community interactions
 
Up to University
Up to UniversityUp to University
Up to University
 
Up2U webinar for NRENs
Up2U webinar for NRENsUp2U webinar for NRENs
Up2U webinar for NRENs
 

Recently uploaded

Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Torry Harris
 
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
Priyanka Aash
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
DianaGray10
 
Acumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptxAcumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptx
BrainSell Technologies
 
Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10
ankush9927
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and DisadvantagesBLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
SAI KAILASH R
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
Ivanti
 
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and CitiesThe Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
Arpan Buwa
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
sunilverma7884
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
Safe Software
 
Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
ssuser1915fe1
 
Zaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdfZaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdf
AmandaCheung15
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 
(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf
(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf
(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf
Priyanka Aash
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
shyamraj55
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
alexjohnson7307
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
bhumivarma35300
 
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
Priyanka Aash
 

Recently uploaded (20)

Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
 
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
 
How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...How UiPath Discovery Suite supports identification of Agentic Process Automat...
How UiPath Discovery Suite supports identification of Agentic Process Automat...
 
Acumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptxAcumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptx
 
Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and DisadvantagesBLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
 
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and CitiesThe Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
 
Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
 
Zaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdfZaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdf
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 
(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf
(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf
(CISOPlatform Summit & SACON 2024) Cyber Insurance & Risk Quantification.pdf
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
 
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
 

2019 swan-cs3

  • 1. SWAN and its analysis ecosystem D. Castro, J. Moscicki, M. Lamanna, E. Bocchi, E. Tejedor, D. Piparo, P. Mato, P. Kothuri Jan 29th, 2019 CS3 2019 - Cloud Storage Synchronization and Sharing Services https://cern.ch/swan
  • 3. SWAN in a Nutshell ﹥Analysis only with a web browser  No local installation needed  Based on Jupyter Notebooks  Calculations, input data and results “in the Cloud” ﹥Support for multiple analysis ecosystems  ROOT, Python, R, Octave… ﹥Easy sharing of scientific results: plots, data, code ﹥Integration with CERN resources  Sofware, storage, mass processing power 3
  • 5. Storage ﹥Uses EOS disk storage system  All experiment data potentially available ﹥CERNBox is SWAN's home directory  Storage for your notebooks and data ﹥Sync&Share  Files synced across devices and the Cloud  Collaborative analysis 5
  • 6. Software ﹥Software distributed through CVMFS  ”LCG Releases” - pack a series of compatible packages  Reduced Docker Images size  Lazy fetching of software ﹥Possibility to install libraries in user cloud storage  Good way to use custom/not mainstream packages  Configurable environment 6 LCG Release CERN Software User Software Jupyter modules
  • 7. Previously on last CS3 conference… 7
  • 10. Sharing made easy ﹥Sharing from inside SWAN interface  Integration with CERNBox ﹥Users can share “Projects”  Special kind of folder that contains notebooks and other files, like input data  Self contained 10
  • 11. The Share tab ﹥Users can list which projects...  they have shared  others have shared with them ﹥Projects can be cloned to the receiver's CERNBox  The receiver will work on his own copy ﹥Concurrent editing not supported by Jupyter  Safer to clone 11
  • 12. Spark Cluster Integration with Spark ﹥Connection to CERN Spark Clusters ﹥Same environment across platforms  User data - EOS  Software - CVMFS ﹥Graphical Jupyter extensions developed  Spark Connector  Spark Monitor Spark Master Spark Worker Python task Python task Python task User Notebook 12
  • 15. Stats ﹥~200 user sessions a day on average  Users doubled last year with new SWAN interface ﹥~1300 unique users in 6 months ﹥Spark cluster connection: 15 – 20 % of users  SWAN as entry point for accessing computational resources  Used for monitoring LHC accelerator hardware devices (NXCals) 15 Courses
  • 17. Inspecting a Project ﹥Users can inspect shared project contents  Browsing of the files  Static rendering of notebooks ﹥Useful to decide whether to accept or not the shared project 17
  • 20. Connecting More Resources ﹥Ongoing effort: submit batch jobs from the notebook  Monitoring display  Jobs tab 20
  • 22. Science Box: SWAN on Premises ﹥Packaged deployment of SWAN  Includes all SWAN components: CERNBox/EOS, CVMFS, JupyterHub ﹥Deployable through Kubernetes or docker-compose ﹥Some successful community installations  AARNet  PSNC  Open Telekom Cloud (Helix Nebula) 22
  • 23. Science Box: SWAN on Premises ﹥UP2University European Project  Bridge the gap between secondary schools, higher education and the research domain  Partner universities (OU, UROMA, NTUA, …), pilot schools  http://up2university.eu ﹥SWAN used by students to learn physics and other sciences  Let them use the very same tools & services used by scientists at CERN  Pilot with University of Geneva (Physiscope) ﹥Establishing collaboration with Callysto project 23
  • 25. Future work/challenges ﹥Move to Jupyterlab  Porting the current extensions  Concurrent editing ﹥New architecture  Based on Kubernetes ﹥Exploitation of GPUs  HEP is looking to ML  Speed up computation of GPU-ready libraries (e.g. TensorFlow) 25
  • 26. Where to find us 26
  • 27. Where to find us ﹥Contacts  swan-talk@cern.ch  http://cern.ch/swan ﹥Repository  https://github.com/swan-cern/ ﹥Science Box  https://cern.ch/sciencebox 27
  • 29. Conclusion ﹥Changes introduced since last year improved user experience  Which translated on more users using the service ﹥SWAN became a fundamental Interface for Mass Processing Resources (Spark)  Not only for Physics analysis but also for monitoring the LHC hardware ﹥The new Jupyterlab interface will bring new possibilities for collaborative analysis  With the introduction of concurrent editing of notebooks  Which can help reach more users ﹥Successfully deployed outside CERN premises  Including on education related projects 29
  • 30. SWAN and its analysis ecosystem Thank you Diogo Castro diogo.castro@cern.ch 30