Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Frontiers of Science and Clouds - Tim Bell - Openstack Day Israel 2016

387 views

Published on

CERN has been running a production OpenStack cloud since 2013 for the 11,000 physicists throughout the world to understand how the universe works and what it is made of.

The physics program for the LHC and other experiments at CERN needs both an innovative approach and worldwide contribution to continue pushing the frontiers of science, such as the discovery of the Higgs Boson in 2012 and future research areas in anti-matter/matter asymmetry and dark energy.

Tim will cover CERN’s scientific challenges, how the computing services have been met with major technical and cultural transformations along with the outlook for the next decade.

Published in: Technology
  • DOWNLOAD FULL MOVIE, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... ,DOWNLOAD FULL. MOVIE 4K,FHD,HD,480P here { https://tinyurl.com/yybdfxwh }
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Frontiers of Science and Clouds - Tim Bell - Openstack Day Israel 2016

  1. 1. 02/06/2016 OpenStack Israel 2016 2 Frontiers of Science and Clouds Tim Bell – tim.bell@cern.ch - @noggin143
  2. 2. About Tim 02/06/2016 OpenStack Israel 2016 3
  3. 3. 4 CERN: founded in 1954: 12 European States “Science for Peace” Today: 21 Member States Member States: Austria, Belgium, Bulgaria, Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Israel, Italy, Netherlands, Norway, Poland, Portugal, Slovak Republic, Spain, Sweden, Switzerland and United Kingdom Associate Member States: Pakistan, Turkey States in accession to Membership: Romania, Serbia Applications for Membership or Associate Membership: Azerbaijan, Brazil, Croatia, Cyprus, India, Russia, Slovenia, Ukraine Observers to Council: India, Japan, Russia, United States of America; European Union, JINR and UNESCO ~ 2300 staff ~ 1400 other paid personnel ~ 12500 scientific users Budget (2016) ~1000 MCHF 02/06/2016 OpenStack Israel 2016 4
  4. 4. CMS ALICE ATLAS LHCb Largest Machine on Earth 27 Km circumference, 100m underground 02/06/2016 OpenStack Israel 2016 5
  5. 5. Most Powerful Magnets 9,600 superconducting magnets direct the beam 02/06/2016 OpenStack Israel 2016 6
  6. 6. Highest Vacuum Pressure is 10 times less than on the moon 02/06/2016 OpenStack Israel 2016 7
  7. 7. Coldest Temperature 120 tonnes of liquid helium for cooling to -271°C 02/06/2016 OpenStack Israel 2016 8
  8. 8. 30M USD/year Electricity Bill Up to 200MW at peak utilisation 02/06/2016 OpenStack Israel 2016 9
  9. 9. ATLAS Experiment 02/06/2016 OpenStack Israel 2016 10
  10. 10. ALICE Experiment 02/06/2016 OpenStack Israel 2016 11
  11. 11. CMS Experiment 02/06/2016 OpenStack Israel 2016 12
  12. 12. LHCb Experiment 02/06/2016 OpenStack Israel 2016 13
  13. 13. COLLISIONS 02/06/2016 OpenStack Israel 2016 14 Collisions Produce 1PB/s
  14. 14. 02/06/2016 OpenStack Israel 2016 15 Innovations
  15. 15. TIER-1: permanent storage, re-processing, analysis TIER-0 (CERN): data recording, reconstruction and distribution TIER-2: Simulation, end-user analysis > 2 million jobs/day ~350’000 cores 500 PB of storage nearly 170 sites, 40 countries 10-100 Gb links Worldwide LHC Computing Grid 02/06/2016 OpenStack Israel 2016 16
  16. 16. 02/06/2016 Compute Growth Outlook 0" 20" 40" 60" 80" 100" 120" 140" 160" Run"1" Run"2" Run"3" Run"4" GRID" ATLAS" CMS" LHCb" ALICE" Compute: Growth > x50 Moore’s law only x16 What we can afford … and 400PB/year by 2023 OpenStack Israel 2016 17
  17. 17. 02/06/2016 OpenStack Israel 2016 18
  18. 18. 02/06/2016 OpenStack Israel 2016 19 Expanded Facilities in Hungary
  19. 19. O’Reilly Consideration 02/06/2016 OpenStack Israel 2016 20
  20. 20. Job Trends Consideration 02/06/2016 OpenStack Israel 2016 21 From Indeed.com
  21. 21. CERN Tool Chain 02/06/2016 OpenStack Israel 2016 22
  22. 22. CERN OpenStack Project (*) Pilot ESSEX Nova (*) Swift Glance (*) Horizon (*) Keystone (*) FOLSOM Nova (*) Swift Glance (*) Horizon (*) Keystone (*) Quantum Cinder GRIZZLY Nova Swift Glance Horizon Keystone Quantum Cinder Ceilometer (*) HAVANA Nova Swift Glance Horizon Keystone Neutron Cinder Ceilometer (*) Heat ICEHOUSE Nova Swift Glance Horizon Keystone Neutron Cinder Ceilometer Heat Ironic Trove JUNO Nova Swift Glance Horizon Keystone Neutron Cinder Ceilometer Heat (*) Rally (*) 5 April 2012 27 September 2012 4 April 2013 17 October 2013 17 April 2014 16 October 2014 July 2013 CERN OpenStack Production Service February 2014 CERN OpenStack Havana Release October 2014 CERN OpenStack Icehouse Release 30 April 2015 March2015 CERN OpenStack Juno Release LIBERTY Nova Swift Glance Horizon Keystone Neutron (*) Cinder Ceilometer Heat Rally Magnum (*) Barbican (*) Ironic 15 October 2015 September 2015 CERN OpenStack Kilo Release KILO Nova Swift Glance Horizon Keystone Neutron Cinder Ceilometer Heat Rally Manila May 2016 CERN OpenStack ongoing Liberty 02/06/2016 OpenStack Israel 2016 23
  23. 23. Upstream OpenStack on its own does not give you a cloud service e.g. • 200 people arrive and leave CERN / month • User skill levels vary widely • Application range from server consolidation, dev/test to production compute Cloud is a service! Monitoring and alerting Metering and chargeback Autoscaling Remediation Resource Lifecycle Capacity planning Upgrades SLA Customer supportUser experience Incident resolution Alerting Cloud monitoring Metrics Log processing High availability Config management Infra onboarding CI Builds Net/info sec Network design OpenStack APIs 02/06/2016 OpenStack Israel 2016 24 Subbu Allamaraju @ eBay Not Just The Software
  24. 24. In production: • 4 clouds • >200K cores • >8,000 hypervisors ~60,000 additional cores being installed in Q2 2016 90% of CERN’s compute resources are now delivered on top of OpenStack OpenStack@CERN Status 02/06/2016 OpenStack Israel 2016 25
  25. 25. 02/06/2016 Cultural Change Impact OpenStack Israel 2016 26 • Move to Agile technologies brings great benefits • Reduced deployment time • Continuous integration validation • Flexibility • Don’t forget Hooke’s Law (adapted) • Under load, an organisation can extend proportional to external force • Too much load leads to permanent deformation • Ensure the tail is moving fast as well as the head • Application support • Cultural challenges • Process change
  26. 26. • Open source collaboration sets model for in-house teams • External recognition by the community is highly rewarding for contributors • Reviews and being reviewed is a constant learning experience • Operator sharing is much more than just OpenStack 02/06/2016 Communities Encourage Change Keystone 2013 Paris 2014 OpenStack Israel 2016 27
  27. 27. Deployment Models 02/06/2016 OpenStack Israel 2016 28 • Skills • Culture • Need for Speed • Risk Appetite
  28. 28. Keeping Up with Releases 02/06/2016 OpenStack Israel 2016 29
  29. 29. Users and Technology 02/06/2016 OpenStack Israel 2016 30
  30. 30. When it’s not on the menu? 02/06/2016 OpenStack Israel 2016 31
  31. 31. Top level cell • Runs API service • Top cell scheduler Child cells run • Compute nodes • Nova network • Scheduler • Conductor Version 2 coming • Default for all Nova Cells 02/06/2016 OpenStack Israel 2016 32
  32. 32. Onwards Federated Clouds Public Cloud such as Rackspace or IBM CERN Private Cloud 160K cores ATLAS Trigger 28K cores ALICE Trigger 9K cores CMS Trigger 13K cores INFN Italy Brookhaven National Labs NecTAR Australia Many Others on Their Way 02/06/2016 OpenStack Israel 2016 33 Available in standard OpenStack since Kilo
  33. 33. Containers on Clouds For the user • Interactive • Dynamic • Choice For IT • Timely! • Secure • Managed • Integrated 02/06/2016 OpenStack Israel 2016 34 CERN now runs Magnum on the production cloud
  34. 34. Outlook for next 12 months • Scale out • Another ~100K cores end 2016 • Further significant retirements and migrations • Consolidate • EC2 API project • Nova network to Neutron • Enhance • Kubernetes,Mesos,Swarm containers for all • Bare metal (Ironic) and Workflow (Mistral) to pilot • Accounting / Fleet Management • Investigate further capacity options • Public clouds 02/06/2016 OpenStack Israel 2016 35
  35. 35. Summary • OpenStack at CERN has been in production for 3 years • Major cultural and technology changes have been successfully addressed • Contributing back upstream has led to sustainable tools and effective technology transfer This transformation would not have been possible without community contribution 02/06/2016 OpenStack Israel 2016 36
  36. 36. Technical details at http://openstack-in-production.blogspot.fr 02/06/2016 OpenStack Israel 2016 37 Thanks to all of you for contributing! Thanks to the CERN IT team and Francois Briard from CERN communications
  37. 37. Some history of scale… Date Collaboration sizes Data volume, archive technology Late 1950’s 2-3 Kilobits, notebooks 1960’s 10-15 kB, punchcards 1970’s ~35 MB, tape 1980’s ~100 GB, tape, disk 1990’s ~750 TB, tape, disk 2010’s ~3000 PB, tape, disk For comparison: 1990’s: Total LEP data set ~few TB Would fit on 1 tape today Today: 1 year of LHC data ~27 PB 02/06/2016 OpenStack Israel 2016 38
  38. 38. CERN Archive >100 PB CERN New data 15 PB 23 PB 27 PB 02/06/2016 OpenStack Israel 2016 39
  39. 39. 4002/06/2016 OpenStack Israel 2016
  40. 40. THE CERN MEYRIN DATA CENTRE http://goo.gl/maps/K5SoG 02/06/2016 OpenStack Israel 2016 41
  41. 41. Public Procurement Cycle Step Time (Days) Elapsed (Days) User expresses requirement 0 Market Survey prepared 15 15 Market Survey for possible vendors 30 45 Specifications prepared 15 60 Vendor responses 30 90 Test systems evaluated 30 120 Offers adjudicated 10 130 Finance committee 30 160 Hardware delivered 90 250 Burn in and acceptance 30 days typical with 380 worst case 280 Total 280+ Days 02/06/2016 OpenStack Israel 2016 42
  42. 42. Good News, Bad News • Additional data centre in Budapest now online • Increasing use of facilities as data rates increase But… • Staff numbers are fixed, no more people • Materials budget decreasing, no more money • Legacy tools are high maintenance and brittle • User expectations are for fast self-service 02/06/2016 OpenStack Israel 2016 43
  43. 43. Innovation Dilemma • How can we avoid the sustainability trap ? • Define requirements • No solution available that meets those requirements • Develop our own new solution • Accumulate technical debt • How can we learn from others and share ? • Find compatible open source communities • Contribute back where there is missing functionality • Stay mainstream Are CERN computing needs really special ? 02/06/2016 OpenStack Israel 2016 44
  44. 44. 02/06/2016 45OpenStack Israel 2016
  45. 45. The largest detectors 02/06/2016 OpenStack Israel 2016 46
  46. 46. 02/06/2016 OpenStack Israel 2016 47
  47. 47. Strategic Plan „ Establish multi- tenant, multi-provider cloud infrastructure „ Identify and adopt policies for trust, security and privacy „ Create governance structure „ Define funding schemes To support the computing capacity needs for the ATLAS experiment Setting up a new service to simplify analysis of large genomes, for a deeper insight into evolution and biodiversity To create an Earth Observation platform, focusing on earthquake and volcano research To improve the speed and quality of research for finding surrogate biomarkers based on brain images Adopters Suppliers Additional Users: 02/06/2016 OpenStack Israel 2016 48
  48. 48. 49 * EC co-funded joint Pre-Commercial Procurement (PCP) project: https://indico.cern.ch/event/319753 ** Other work has been conductedoutsideCERN, such as the Amazon Pilot projectat BNL for ATLAS HN - Helix Nebula • Partnership between research organization and European commercial cloud providers Support CERN’s scientific computing programme Investigations
  49. 49. Tier 0 LHCC;25 May 2016 Ian Bird; CERN 50
  50. 50. Run 2 has only just started • Hint of an excess with diphoton mass of 750 GeV • Seen by ATLAS and CMS – coincidence or a new signal? OpenStack Israel 2016 51 200 400 600 800 1000 1200 1400 1600 Events/20GeV 1− 10 1 10 2 10 3 10 4 10 ATLAS Preliminary Spin-0 Selection -1 = 13 TeV, 3.2 fbs Data Background-only fit [GeV]γγm 200 400 600 800 1000 1200 1400 1600 Data-fittedbackground 10− 5− 0 5 10 15 Events/(20GeV) 1 10 2 10 Data Fit model σ1± σ2± EBEB (GeV)γγm 400 600 800 1000 1200 1400 1600 statσ(data-fit)/ -2 0 2 (13 TeV, 3.8T)-1 2.7 fbCMS Preliminary 02/06/2016

×