"The Open Source effect in the Storage world" by George Mitropoulos @ eLibera...eLiberatica
This is a presentation held at eLiberatica 2009.
http://www.eliberatica.ro/2009/
One of the biggest events of its kind in Eastern Europe, eLiberatica brings community leaders from around the world to discuss about the hottest topics in FLOSS movement, demonstrating the advantages of adopting, using and developing Open Source and Free Software solutions.
The eLiberatica organizational committee together with our speakers and guests, have graciously allowed media representatives and all attendees to photograph, videotape and otherwise record their sessions, on the condition that the photos, videos and recordings are licensed under the Creative Commons Share-Alike 3.0 License.
"The Open Source effect in the Storage world" by George Mitropoulos @ eLibera...eLiberatica
This is a presentation held at eLiberatica 2009.
http://www.eliberatica.ro/2009/
One of the biggest events of its kind in Eastern Europe, eLiberatica brings community leaders from around the world to discuss about the hottest topics in FLOSS movement, demonstrating the advantages of adopting, using and developing Open Source and Free Software solutions.
The eLiberatica organizational committee together with our speakers and guests, have graciously allowed media representatives and all attendees to photograph, videotape and otherwise record their sessions, on the condition that the photos, videos and recordings are licensed under the Creative Commons Share-Alike 3.0 License.
Scheduled Scientific Data Releases Using .backup Volumesretsamedoc
How the Mars Space Flight Facility uses (and abuses) the .backup snapshot feature of OpenAFS.
This presentation was given by Chris Kurtz and myself at the 2008 AFS and Kerberos Best Practices Workshop in Newark, NJ.
OpenStack is the prevailing open source cloud software. It includes numerous API services for programmatic management of all sorts of IaaS and SaaS services. VMs, Containers, Bare Metal, Multi-tenancy. Use this platform to strike the right balance between developer self-service to your infrastructure and a well defined platform for next generation containerized microservice applications that your IT department feels happy to support and your CFO would be happy to pay for.
Learning to Scale Openstack: A Case Study in Rackspace's Open Cloud Deployment was presented at OpenStack Design Summit in Portland, OR on April 17, 2013. Watch the recording of the presentation on youtube at the following link: http://www.youtube.com/watch?v=3x8X6f5mnzc
OpenStack Summit Tokyo - Know-how of Challlenging Deploy/Operation NTT DOCOMO...Masaaki Nakagawa
DOCOMO MAIL is 24/7 cloud mail system which has accesses from over 20 million people. This mail system stores user's mail archive in OpenStack Swift with Peta Byte scale capacity deployed by NTT DATA.
We have been successfully operating this service since Sep 2014 without any downtime. In this session, we'll present the actual issues and challenges we have faced and conquered.
Here're some specific points we'd like to highlight.
* No service degrade, no downtime.
* Massive scale and still growing.
* Hundreds of servers operated by few people.
CERN, the European Organization for Nuclear Research, is one of the world’s largest centres for scientific research. Its business is fundamental physics, finding out what the universe is made of and how it works. At CERN, accelerators such as the 27km Large Hadron Collider, are used to study the basic constituents of matter. This talk reviews the challenges to record and analyse the 25 Petabytes/year produced by the experiments and the investigations into how OpenStack could help to deliver a more agile computing infrastructure.
Scheduled Scientific Data Releases Using .backup Volumesretsamedoc
How the Mars Space Flight Facility uses (and abuses) the .backup snapshot feature of OpenAFS.
This presentation was given by Chris Kurtz and myself at the 2008 AFS and Kerberos Best Practices Workshop in Newark, NJ.
OpenStack is the prevailing open source cloud software. It includes numerous API services for programmatic management of all sorts of IaaS and SaaS services. VMs, Containers, Bare Metal, Multi-tenancy. Use this platform to strike the right balance between developer self-service to your infrastructure and a well defined platform for next generation containerized microservice applications that your IT department feels happy to support and your CFO would be happy to pay for.
Learning to Scale Openstack: A Case Study in Rackspace's Open Cloud Deployment was presented at OpenStack Design Summit in Portland, OR on April 17, 2013. Watch the recording of the presentation on youtube at the following link: http://www.youtube.com/watch?v=3x8X6f5mnzc
OpenStack Summit Tokyo - Know-how of Challlenging Deploy/Operation NTT DOCOMO...Masaaki Nakagawa
DOCOMO MAIL is 24/7 cloud mail system which has accesses from over 20 million people. This mail system stores user's mail archive in OpenStack Swift with Peta Byte scale capacity deployed by NTT DATA.
We have been successfully operating this service since Sep 2014 without any downtime. In this session, we'll present the actual issues and challenges we have faced and conquered.
Here're some specific points we'd like to highlight.
* No service degrade, no downtime.
* Massive scale and still growing.
* Hundreds of servers operated by few people.
CERN, the European Organization for Nuclear Research, is one of the world’s largest centres for scientific research. Its business is fundamental physics, finding out what the universe is made of and how it works. At CERN, accelerators such as the 27km Large Hadron Collider, are used to study the basic constituents of matter. This talk reviews the challenges to record and analyse the 25 Petabytes/year produced by the experiments and the investigations into how OpenStack could help to deliver a more agile computing infrastructure.
CERN and Huawei, one of the companies that is working with T-Systems to develop a prototype for HNSciCloud, in the context of CERN openlab, will jointly work on improvements to OpenStack for running large scale scientific workloads.
This collaboration was announced at the CERN openlab open day on 21st September 2017.
Over 90% of CERN’s compute resources are delivered using OpenStack. OpenStack provides software for private and public clouds and CERN and Huawei are among the major contributors to the open source project (Huawei is a platinum member of the OpenStack foundation).
With the needs of LHC computing in future years, efficient and flexible delivery of compute resources will be key. That's why CERN and Huawei have joined forces to jointly work on improvements to OpenStack for running large scale scientific workloads.
The developments will be done within the OpenStack community following the standard open source processes.
Focus areas will be:
Flexible resource management
Quotas
Bare metal allocation
Compute cells
Changes, resulting from this activity, will then be included into the CERN private cloud and Huawei’s private and public cloud offerings.
CERN is the European Centre for Particle Physics based in Geneva. The home of the Large Hadron Collider and the birth place of the world wide web is expanding its computing resources with a second data centre to process over 35PB/year from one of the largest scientific experiments ever constructed.
Within the constraints of fixed budget and manpower, agile computing techniques and common open source tools are being adopted to support over 11,000 physicists in their search for how the universe works and what is it made of.
By challenging special requirements and understanding how other large computing infrastructures are built, we have deployed a 50,000 core cloud based infrastructure building on tools such as Puppet, OpenStack and Kibana.
In moving to a cloud model, this has also required close examination of the IT processes and culture. Finding the right approach between Enterprise and DevOps techniques has been one of the greatest challenges of this transformation.
This talk will cover the requirements, tools selected, results achieved so far and the outlook for the future.
Slides from our introduction to Ceph and OpenStack webinar. You can watch the webinar on demand also here http://www.inktank.com/news-events/webinars/.
Multi-Cell OpenStack: How to Evolve Your Cloud to Scale - November, 2014Belmiro Moreira
Multi-Cell OpenStack: How to Evolve Your Cloud to Scale
OpenStack Design Summit, Paris - November, 2014
Belmiro Moreira - CERN
Matt Van Winkle - Rackspace
Sam Morrison - NeCTAR, University of Melbourne
[Presented at All Things Open 2015 in Raleigh, NC, USA]
OpenStack is one of the fastest-growing and exciting open source projects of our time. OpenStack has drawn together technologists from all over the world to create a cloud operating system and a huge, diverse community behind it. This talk will provide an introduction to OpenStack for newcomers to the project of those who just want to know more. We’ll take a brief look at OpenStack’s history, get a technical overview of the project, learn how to contribute, and check out a few emerging trends and hot topics in the OpenStack world.
OpenStack Ecosystem – Xen Cloud Platform and Integration into OpenStack - in...IndicThreads
Session presented at the 2nd IndicThreads.com Conference on Cloud Computing held in Pune, India on 3-4 June 2011.
http://CloudComputing.IndicThreads.com
Abstract: OpenStack is an Initiative by RackSpace and NASA that aims for building an Open cloud platform supported by a vibrant Ecosystem to encourage broad adoption in the market.This is currently a hot favorite of enterprises looking to build an Open cloud.
This talk will provide a brief overview of the different OpenStack Modules (Compute and Storage) and explain how to utilize these to build a cloud. We will also explore the newly released Xen Cloud Platform (XCP) and its integration with OpenStack Platform. There will be a hands-on demo (time permitting) where we will show how the integration between the OpenStack Platform and XCP works.
Key Takeaways for the audience:
1) Understanding of OpenStack platform.
2) How to get started with OpenStack for building your own cloud.
3) Understanding of XCP
3) How the integration (OpenStack-XCP) is supposed to work
4) What are the opportunities for building different products that add value in the OpenStack Ecosystem
Speaker: Amit Naik is an Architect at BMC Software and has 15 years of experience in the IT field with experience in delivering multiple end-to-end projects and Products. Multiple speaking engagements at different venues both in India and Abroad. Experience with blogging, evangelizing etc. Excellent communication and interpersonal skills.
Joint Speaker: Prasad Nirantar is a Staff Product Developer at BMC Software. He holds a B.E in Polymer Engineering from the University of Pune and an MS from University of Akron, US. He also holds a diploma in business management from Symbiosis University.
OpenStack Juno The Complete Lowdown and Tales from the SummitNati Shalom
This presentation covers the main points from the summit and the OpenStack Juno release
It also covers how users use OpenStack based on the recent survey
Dell openstack cloud with inktank ceph – large scale customer deploymentKamesh Pemmaraju
This was my presentation at the OpenStack Summit in Hong Kong, November 2013. Learn detail around a unique deployment of the Dell OpenStack-Powered Cloud Solution with Inktank Ceph installed at a large nationally recognized American University that specializes in cancer and genomic research. The University had a need to provide a scalable, secure, centralized data repository to support approximately 900 researchers and an ever-expanding number of research projects and rapidly expanding universe of data. The Dell and Inktank cloud storage solution addresses these storage challenges with an open source solution that leverages the Dell Crowbar Framework and Reference Architecture. After assessing a number of traditional storage scenarios, the University partnered with Dell and Inktank to architect a centralized cloud storage platform that is capable of scaling seamlessly and rapidly, is cost-effective, and that can leverage a single hardware infrastructure, with Dell Power Edge R-720XD servers and the Dell Reference Architecture for their OpenStack compute and storage environment.
Similar to 20121115 open stack_ch_user_group_v1.2 (20)
Review of CERN's objectives and how the computing infrastructure is evolving to address the challenges at scale using community supported software such as Puppet and OpenStack.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Accelerate your Kubernetes clusters with Varnish Caching
20121115 open stack_ch_user_group_v1.2
1. OpenStack and CERN
Tim Bell
Tim.Bell@cern.ch
@noggin143
First OpenStack Switzerland User Group Meeting
15th November 2012
2. What is OpenStack ?
• OpenStack is a cloud operating system that controls large
pools of compute, storage, and networking resources
throughout a datacenter, all managed through a dashboard
that gives administrators control while empowering their users
to provision resources through a web interface
OpenStack Swiss User Group Tim Bell, CERN 2
3. Principles
• Open Source
– Apache 2.0 license, NO ‘enterprise’ version
• Open Design
– Open Design Summit, anyone is able to define core architecture
• Open Development
– Anyone can involve development process via Launchpad & Github
• Open Community
– OpenStack Foundation in 2012, Now 190+ companies, 3000+
developers, 6000+ members
OpenStack Swiss User Group Tim Bell, CERN 3
5. OpenStack Cloud Components
• Nova – Compute Layer
• Swift – Object Store
• Quantum – Networking
• Horizon – Dashboard
• Cinder – Block Storage
• Keystone – Identity
• Glance – Image management
• Each component has an API and is pluggable
• Other non-core projects interact with these components
OpenStack Swiss User Group Tim Bell, CERN 5
7. October Summit in San Diego
• 1,400 people attended with 320 talks and up to 8 parallel
streams
– Double the spring 2012 summit in San Francisco
• International involvement is increasing
– Asia especially China
– Europe
• Companies and Open Source community generally working
well
– Transparency and shared goals
OpenStack Swiss User Group Tim Bell, CERN 7
9. What’s new from the summit ?
Component Folsom Grizzly (plan for Q2/2014)
Nova (Compute) Host Aggregates Cells
Hyper-V/PowerVM support Bare metal support
Cinder (Block Storage) Now in core Multiple backends
NFS as a block device QoS for backends
Resize
Glance Replicated images Incremental images
New client
Keystone PKI support Domains
LDAP support for Active Directory User Groups
Horizon Support for volumes Realtime notifications
Cross project signalling
Swift Versioned objects Multi-range get
SSD acceleration
Quantum Now in core Security groups
Pluggable backends High Availability
OpenStack Swiss User Group Tim Bell, CERN 9
10. Upcoming Projects
• Ceilometer
– Metering and billing
• Heat
– Cloud orchestration
• Red Dwarf
– Database as a Service – being refactored to be on top of Nova
• Equilibrium
– Load Balancing as a Service – 3 proposals being merged to 1
• DNS as a Service
– May become part of Quantum
• High Availability Nova Controller
– Cisco, Hastexo proposals
OpenStack Swiss User Group Tim Bell, CERN 10
11. OpenStack Foundation
• The OpenStack Foundation is an independent body providing
shared resources to help achieve the OpenStack Mission by
Protecting, Empowering, and Promoting OpenStack software
and the community around it, including users, developers and
the entire ecosystem.
• Provides
– Promotion of the OpenStack brand
– Event management such as Summits, User Groups, …
– Legal coverage of trademark, contributions
– Continuous integration and Testing infrastructure
– Developer tools to ease contribution
OpenStack Swiss User Group Tim Bell, CERN 11
12. OpenStack Board Structure
• ”Platinum Members” are companies which make a significant
strategic commitment to OpenStack in funding and resources.
Platinum Members each appoint a representative to the Board
of Directors
• ”Gold Members” are companies which provide funding and
resources, but at a lower level than Platinum Members.
Associate Members as a class elect representatives to the
Board of Directors.
• "Individual Members" who participate on their own or as part
of their paid employment. It’s free to join as an Individual
Member and Individual Members have the right to run for, and
vote for, a number of leadership positions.
OpenStack Swiss User Group Tim Bell, CERN 12
14. Individual Members
• Elected by the membership of the foundation
• Rob Hirschfeld (DELL)
• Monty Taylor (HP)
• Hui Cheng (SINA)
• Yujie Du (99 Cloud)
• Troy Toman (Rackspace)
• Tim Bell (CERN)
• Tristan Goode (Aptira)
• Jesse Andreas (Nebula)
• No company can have more than 2 directors
• Rules for individual member voting are currently under
discussion with the community to find best ways of representing
all the members
OpenStack Swiss User Group Tim Bell, CERN 14
16. What is CERN ?
• Conseil Européen pour la
Recherche Nucléaire – aka
European Laboratory for
Particle Physics
• Between Geneva and the
Jura mountains, straddling
the Swiss-French border
• Founded in 1954 with an
international treaty
• Our business is fundamental
physics , what is the
universe made of and how
does it work
OpenStack Swiss User Group Tim Bell, CERN 16
17. Answering fundamental questions…
• How to explain particles have mass?
We have theories and accumulating experimental evidence.. Getting close…
• What is 96% of the universe made of ?
We can only see 4% of its estimated mass!
• Why isn’t there anti-matter
in the universe?
Nature should be symmetric…
• What was the state of matter just
after the « Big Bang » ?
Travelling back to the earliest instants of
the universe would help…
OpenStack Swiss User Group Tim Bell, CERN 17
18. The Large Hadron Collider
OpenStack Swiss User Group Tim Bell, CERN 18
19. The Large Hadron Collider (LHC) tunnel
OpenStack Swiss User Group Tim Bell, CERN
19
25. Tier-0 (CERN):
•Data recording
•Initial data reconstruction
•Data distribution
Tier-1 (11 centres):
•Permanent storage
•Re-processing
•Analysis
Tier-2 (~200 centres):
• Simulation
• End-user analysis
• Data is recorded at CERN and Tier-1s and analysed in the Worldwide LHC Computing Grid
• In a normal day, the grid provides 100,000 CPU days executing over 2 million jobs
OpenStack Swiss User Group Tim Bell, CERN 25
27. 45,000 tapes holding 73PB of physics data
OpenStack Swiss User Group Tim Bell, CERN 27
28. New data centre to expand capacity
• Data centre in Geneva
at the limit of
electrical capacity at
3.5MW
• New centre chosen in
Budapest, Hungary
• Additional 2.7MW of
usable power
• Hands off facility
• Deploying from 2013
with 200Gbit/s
OpenStack Swiss User Group Tim Bell, CERN
network to CERN 28
29. Time to change strategy
• Rationale
– Need to manage twice the servers as today
– No increase in staff numbers
– Tools becoming increasingly brittle and will not scale as-is
• Approach
– CERN is no longer a special case for compute
– Adopt an open source tool chain model
– Our engineers rapidly iterate
• Evaluate solutions in the problem domain
• Identify functional gaps and challenge them
• Select first choice but be prepared to change in future
– Contribute new function back to the community
OpenStack Swiss User Group Tim Bell, CERN 29
30. Building Blocks
mcollective, yum Bamboo
Puppet
AIMS/PXE
Foreman JIRA
OpenStack
Nova
git
Koji, Mock
Yum repo
Active Directory / Pulp
LDAP
Lemon /
Hardware
Hadoop
database
Puppet-DB
OpenStack Swiss User Group Tim Bell, CERN 30
31. Prepare the move to the clouds
• Improve operational efficiency
– Machine ordering, reception and testing
– Hardware interventions with long running programs
– Multiple operating system demand
• Improve resource efficiency
– Exploit idle resources, especially waiting for disk and tape I/O
– Highly variable load such as interactive or build machines
• Enable cloud architectures
– Gradual migration to cloud interfaces and workflows
• Improve responsiveness
– Self-Service with coffee break response time
OpenStack Swiss User Group Tim Bell, CERN 31
32. Service Model
• Pets are given names like
pussinboots.cern.ch
• They are unique, lovingly hand raised
and cared for
• When they get ill, you nurse them back
to health
• Cattle are given numbers like
vm0042.cern.ch
• They are almost identical to other cattle
• When they get ill, you get another one
• Future application architectures should use Cattle but Pets with
strong configuration management are viable and still needed
OpenStack Swiss User Group Tim Bell, CERN 32
33. Current Status of OpenStack at CERN
• Focus on the Cattle to start with
• Working on an Essex code base from the EPEL repository
– Excellent experience with the Fedora cloud-sig team
– Cloud-init for contextualisation, oz for images with RHEL/Fedora
• Components
– Current focus is on Nova with KVM and Hyper-V
– Keystone running with Active Directory and Glance for Linux and
Windows images
• Pre-production facility with around 200 Hypervisors, with
2000 VMs integrated with CERN infrastructure, Puppet
deployed and used for simulation of magnet placement using
LHC@Home and batch physics programs
OpenStack Swiss User Group Tim Bell, CERN 33
35. Next Steps
• Deploy into production at the start of 2013 with Folsom running the Grid
software on top of OpenStack IaaS
• Support multi-site operations with 2nd data centre in Hungary
• Exploit new functionality
– Ceilometer for metering
– Bare metal for non-virtualised use cases such as high I/O servers
– X.509 user certificate authentication
– Load balancing as a service
• Support the pets with live migration and external block storage such as
Gluster
Ramping to 15K hypervisors with 100K to
300K VMs by 2015
OpenStack Swiss User Group Tim Bell, CERN 35
36. Conclusions
• Production at CERN in next few months on Folsom
– Our emphasis will shift to focus on stability
– Work together with others on scaling improvements
• Community is key to shared success
– Foundation gives a structure for collaboration
– Our problems are often resolved before we raise them
– Packaging teams are producing reliable builds promptly
• CERN contributes and benefits
– Thanks to everyone for their efforts and enthusiasm
– Not just code but documentation, tests, blogs, …
OpenStack Swiss User Group Tim Bell, CERN 36
38. References
CERN http://public.web.cern.ch/public/
Scientific Linux http://www.scientificlinux.org/
Worldwide LHC Computing Grid http://lcg.web.cern.ch/lcg/
http://rtm.hep.ph.ic.ac.uk/
Jobs http://cern.ch/jobs
Detailed Report on Agile Infrastructure http://cern.ch/go/N8wp
Talk on OpenStack at San Diego Summit http://www.openstack.org/summit/san-
diego-2012/openstack-summit-
sessions/presentation/accelerating-
science-with-openstack
OpenStack Swiss User Group Tim Bell, CERN 38
41. Our Challenges - Data storage
• >20 years retention
• 6GB/s average
• 25GB/s peaks
• 30PB/year to record
OpenStack Swiss User Group Tim Bell, CERN 41
42. • Data Centre by Numbers
– Hardware installation & retirement
• ~7,000 hardware movements/year; ~1,800 disk failures/year
Racks 828 Disks 64,109 Tape Drives 160
Servers 11,728 Raw disk capacity (TiB) 63,289 Tape Cartridges 45,000
Processors 15,694 Memory modules 56,014 Tape slots 56,000
Cores 64,238 Memory capacity (TiB) 158 Tape Capacity (TiB) 73,000
HEPSpec06 482,507 RAID controllers 3,749
High Speed Routers
24
Xeon Xeon Xeon Other Fujitsu (640 Mbps → 2.4 Tbps)
3GHz 5150 5160 Xeon 0% 3%
Xeon 4% 2% 10% E5335
Ethernet Switches 350
L5520 7% Xeon Hitachi
33% 23% 10 Gbps ports 2,000
E5345
14% HP Switching Capacity 4.8 Tbps
Seagate
0%
15%
1 Gbps ports 16,939
Maxtor
Western 0% 10 Gbps ports 558
Xeon
Xeon Digital
E5405
Xeon 59%
L5420 6% IT Power Consumption 2,456 KW
8% E5410
16%
Total Power Consumption 3,890 KW
OpenStack Swiss User Group Tim Bell, CERN 42
43. Public Procurement Purchase Model
Step Time (Days) Elapsed (Days)
User expresses requirement 0
Market Survey prepared 15 15
Market Survey for possible vendors 30 45
Specifications prepared 15 60
Vendor responses 30 90
Test systems evaluated 30 120
Offers adjudicated 10 130
Finance committee 30 160
Hardware delivered 90 250
Burn in and acceptance 30 days typical 280
380 worst case
Total 280+ Days
OpenStack Swiss User Group Tim Bell, CERN 43
44. Training and Support
• Buy the book rather than guru mentoring
• Follow the mailing lists to learn
• Newcomers are rapidly productive (and often know more than us)
• Community and Enterprise support means we’re not on our own
OpenStack Swiss User Group Tim Bell, CERN 44
45. Staff Motivation
• Skills valuable outside of CERN when an engineer’s contracts
end
OpenStack Swiss User Group Tim Bell, CERN 45
46. When communities combine…
• OpenStack’s many components and options make
configuration complex out of the box
• Puppet forge module from PuppetLabs does our configuration
• The Foreman adds OpenStack provisioning for user kiosk to a
configured machine in 15 minutes
OpenStack Swiss User Group Tim Bell, CERN 46
47. Foreman to manage Puppetized VM
OpenStack Swiss User Group Tim Bell, CERN 47
48. Supporting the Pets with OpenStack
• Network
– Interfacing with legacy site DNS and IP management
– Ensuring Kerberos identity before VM start
• Puppet
– Ease use of configuration management tools with our users
– Exploit mcollective for orchestration/delegation
• External Block Storage
– Currently using nova-volume with Gluster backing store
• Live migration to maximise availability
– KVM live migration using Gluster
– KVM and Hyper-V block migration
OpenStack Swiss User Group Tim Bell, CERN 48
49. Active Directory Integration
• CERN’s Active Directory
– Unified identity management across the site
– 44,000 users
– 29,000 groups
– 200 arrivals/departures per month
• Full integration with Active Directory via LDAP
– Uses the OpenLDAP backend with some particular configuration
settings
– Aim for minimal changes to Active Directory
– 7 patches submitted around hard coded values and additional filtering
• Now in use in our pre-production instance
– Map project roles (admins, members) to groups
– Documentation in the OpenStack wiki
OpenStack Swiss User Group Tim Bell, CERN 49
50. Welcome Back Hyper-V!
• We currently use Hyper-V/System Centre for our server
consolidation activities
– But need to scale to 100x current installation size
• Choice of hypervisors should be tactical
– Performance
– Compatibility/Support with integration components
– Image migration from legacy environments
• CERN is working closely with the Hyper-V OpenStack team
– Puppet to configure hypervisors on Windows
– Most functions work well but further work on Console, Ceilometer, …
OpenStack Swiss User Group Tim Bell, CERN 50
51. What are we missing (or haven’t found yet) ?
• Best practice for
– Monitoring and KPIs as part of core functionality
– Guest disaster recovery
– Migration between versions of OpenStack
• Roles within multi-user projects
– VM owner allowed to manage their own resources (start/stop/delete)
– Project admins allowed to manage all resources
– Other members should not have high rights over other members VMs
• Global quota management for non-elastic private cloud
– Manage resource prioritisation and allocation centrally
– Capacity management / utilisation for planning
OpenStack Swiss User Group Tim Bell, CERN 51
52. Opportunistic Clouds in online experiment farms
• The CERN experiments have farms of 1000s of Linux servers
close to the detectors to filter the 1PByte/s down to 6GByte/s
to be recorded to tape
• When the accelerator is not running, these machines are
currently idle
– Accelerator has regular maintenance slots of several days
– Long Shutdown due from March 2013-November 2014
• One of the experiments are deploying OpenStack on their farm
– Simulation (low I/O, high CPU)
– Analysis (high I/O, high CPU, high network)
OpenStack Swiss User Group Tim Bell, CERN 52
53. Federated European Clouds
• Two significant European projects around Federated Clouds
– European Grid Initiative Federated Cloud as a federation of grid sites providing
IaaS
– HELiX Nebula European Union funded project to create a scientific cloud
based on commercial providers
EGI Federated Cloud Sites
CESGA CESNET INFN SARA
Cyfronet FZ Jülich SZTAKI IPHC
GRIF GRNET KTH Oxford
GWDG IGI TCD IN2P3
STFC
OpenStack Swiss User Group Tim Bell, CERN 53
54. Federated Cloud Commonalities
• Basic building blocks
– Each site gives an IaaS endpoint with an API and common security
policy
• OCCI? CDMI ? Libcloud ? Jclouds ?
– Image stores available across the sites
– Federated identity management based on X.509 certificates
– Consolidation of accounting information to validate pledges and usage
• Multiple cloud technologies
– OpenStack
– OpenNebula
– Proprietary
OpenStack Swiss User Group Tim Bell, CERN 54
56. CERN’s tools
• The world’s most powerful accelerator: LHC
– A 27 km long tunnel filled with high-tech instruments
– Equipped with thousands of superconducting magnets
– Accelerates particles to energies never before obtained
– Produces particle collisions creating microscopic “big bangs”
• Very large sophisticated detectors
– Four experiments each the size of a cathedral
– Hundred million measurement channels each
– Data acquisition systems treating Petabytes per second
• Top level computing to distribute and analyse the data
– A Computing Grid linking ~200 computer centres around the globe
– Sufficient computing power and storage to handle 25 Petabytes per
year, making them available to thousands of physicists for analysis
OpenStack Swiss User Group Tim Bell, CERN 56
57. Our Infrastructure
• Hardware is generally based on commodity, white-box servers
– Open tendering process based on SpecInt/CHF, CHF/Watt and GB/CHF
– Compute nodes typically dual processor, 2GB per core
– Bulk storage on 24x2TB disk storage-in-a-box with a RAID card
• Vast majority of servers run Scientific Linux, developed by
Fermilab and CERN, based on Redhat Enterprise
– Focus is on stability in view of the number of centres on the WLCG
OpenStack Swiss User Group Tim Bell, CERN 57
59. 500
1500
2000
2500
3000
3500
1000
0
Mar-10
Apr-10
May-10
Jun-10
Jul-10
Aug-10
Sep-10
OpenStack Swiss User Group
Oct-10
Nov-10
Dec-10
Jan-11
Feb-11
Mar-11
Apr-11
May-11
Jun-11
Jul-11
Tim Bell, CERN
Aug-11
Sep-11
Oct-11
Nov-11
Dec-11
Jan-12
Feb-12
Mar-12
Apr-12
May-12
Virtualisation on SCVMM/Hyper-V
Jun-12
Jul-12
Aug-12
Sep-12
59
Linux
Oct-12
Windows
60. Scaling up with Puppet and OpenStack
• Use LHC@Home based on BOINC for simulating magnetics
guiding particles around the LHC
• Naturally, there is a puppet module puppet-boinc
• 1000 VMs spun up to stress test the hypervisors with Puppet,
Foreman and OpenStack
OpenStack Swiss User Group Tim Bell, CERN 60
Editor's Notes
Established by an international treaty at the end of 2nd world war as a place where scientists could work together for fundamental researchNuclear is part of the name but our world is particle physics
Our current understanding of the universe is incomplete. A theory, called the Standard Model, proposes particles and forces, many of which have been experimentally observed. However, there are open questions- Why do some particles have mass and others not ? The Higgs Boson is a theory but we need experimental evidence.Our theory of forces does not explain how Gravity worksCosmologists can only find 4% of the matter in the universe, we have lost the other 96%We should have 50% matter, 50% anti-matter… why is there an asymmetry (although it is a good thing that there is since the two anhialiate each other) ?When we go back through time 13 billion years towards the big bang, we move back through planets, stars, atoms, protons/electrons towards a soup like quark gluon plasma. What were the properties of this?
The LHC is CERN’s largest accelerator. A 17 mile ring 100 meters underground where two beams of particles are sent in opposite directions and collided at the 4 experiments, Atlas, CMS, LHCb and ALICE. Lake Geneva and the airport are visible in the top to give a scale.
The ring consists of two beam pipes, with a vacuum pressure 10 times lower than on the moon which contain the beams of protons accelerated to just below the speed of light. These go round 11,000 times per second being bent by the superconducting magnets cooled to 2K by liquid helium (-450F), colder than outer space. The beams themselves have a total energy similar to a high speed train so care needs to be taken to make sure they turn the corners correctly and don’t bump into the walls of the pipe.
- At 4 points around the ring, the beams are made to cross at points where detectors, the size of cathedrals and weighing up to 12,500 tonnes surround the pipe. These are like digital camera, but they take 100 mega pixel photos 40 million times a second. This produces up to 1 petabyte/s.
- Collisions can be visualised by the tracks left in the various parts of the detectors. With many collisions, the statistics allows particle identification such as mass and charge. This is a simple one…
To improve the statistics, we send round beams of multiple bunches, as they cross there are multiple collisions as 100 billion protons per bunch pass through each otherSoftware close by the detector and later offline in the computer centre then has to examine the tracks to understand the particles involved
To get Quark Gluon plasma, the material closest to the big bang, we also collide lead ions which is much more intensive… the temperatures reach 100,000 times that in the sun.
- We cannot record 1PB/s so there are hardware filters to remove uninteresting collisions such as those whose physics we understand already. The data is then sent to the CERN computer centre for recording via 10Gbit optical connections.
The Worldwide LHC Computing grid is used to record and analyse this data. The grid currently runs over 2 million jobs/day, less than 10% of the work is done at CERN. There is an agreed set of protocols for running jobs, data distribution and accounting between all the sites which co-operate in order to support the physicists across the globe.
Upstairs in the computer centre, a high roof was the fashion in the 1980s for mainframes but now is very difficult to get cooled efficiently
Tape robots from IBM and OracleAround 60,000 tape mounts / week so the robots are kept busyData copied every two years to keep up with the latest media densities
Asked member states for offers200Gbit/s links connecting the centresExpect to double computing capacity compared to today by 2015
Double the capacity, same manpowerNeed to rethink how to solve the problem… look at how others approach itWe had our own tools in 2002 and as they become more sophisticated, it was not possible to take advantage of other developments elsewhere without a major break.Doing this while doing their ‘day’ jobs so it re-enforces the approach of taking what we can from the community
Model based on Google Toolchain, Puppet is key for many operations. We’ve only had to write one new significant custom CERN software component which is in the certificate authority. Other parts such as Lemon for monitoring are from our previous implementation as we did not want to change all at once and they scale.
Standardise hardware … buy in bulk and pile it up then work out what to use it forMemory, motherboards, cables or disks interventionsUsers waiting for I/O means wasted cycles. Build machines at night unused during the day. Interactive machines mainly during the dayMove to cloud APIs … need to support them but also maintain our existing applicationsDetails later on reception and testing
Puppet applies well to the cattle model but we’re also using it to handle the pet cases that can’t yet move over due to software limitations. So, they get cloud provisioning but flexible configuration management.
Biggest international scientific collaboration in the world, over 10,000 scientistsfrom 100 countriesAnnual Budget around 1.1 billion USDFunding for CERN, the laboratory, itselfcomesfrom the 20 member states, in ratio to the grossdomesticproduct… other countries contribute to experimentsincludingsubstantial US contribution towards the LHC experiments
Our data storage system has to record and preserve 30PB/year with an expected lifetime of 20 years. Keeping the old data is required to get the maximum statistics for discoveries. At times, physicists will want to skim this data looking for new physics. Data rates are around 6GB/s average, with peaks of 25GB/s.
So, to the Tier-0 computer centre at CERN… we are unusual in that we are public with our environment as there is no competitive advantage for us. We have thousands of visitors a year coming for tours and education and the computer center is a popular visit.The data centre has around 2.9MW of usable power looking after 12,000 servers.. In comparison, the accelerator uses 120MW, like a small town.With 64,000 disks, we have around 1,800 failing each year… this is much higher than the manufacturers’ MTBFs which is consistent with results from Google.Servers are mainly Intel processors, some AMD with dual core Xeon being the most common configuration.
We’ve been very pleased with our choices. Along with the obvious benefits of the functionality, there are soft benefits from the community model.
Many staff at CERN are short term contracts… good benefits for those staff to leave with skills in need.
Communities integrating … when a new option is being used at CERN in OpenStack, we contribute the changes back to the puppet forge such as certificate handling. Even looking at Hyper-V/Windows openstack configuration…
CERN is more than just the LHCCNGS neutrinos to Gran SassoCLOUD demonstrating impacts of cosmic rays on weather patternsAnti-hydrogen atoms contained for minutes in a magnetic vesselHowever, for those of you who have read Dan Brown’s Angels and Demons or seen the film, there are no maniacal monks with pounds of anti-matter running around the campus
We purchase on an annuak cycle, replacing around ¼ of the servers. This purchasing is based on performance metrics such as cost per SpecInt or cost/GBGenerally, we are seeing dual core computer servers with Intel or AMD processors and bulk storage servers with 24 or 36 2TB disksThe operating system is Redhatlinux based distributon called Scientific Linux. We share the development and maintenance with Fermilab in Chicago. The choice of a Redhat based distribution comes from the need for stability across the grid, where keeping the 200 centres running compatible Linux distributions.
LHC@Home is not an instruction on how to build your own accelerator but a magnet simulation tool to test multiple passes around the ring. We wanted to use it as a stress test tool and in ½ day, it was running on 1000 VMs.