Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

LHC@home and CernVM – a new approach to porting large-scale applications to BOINC

1,193 views

Published on

In this talk, I will review a new application in the volunteer computing project LHC@home, called Test4Theory, which is currently being alpha tested at CERN. This application uses virtual machine technology and a range of other software tools to handle large, complex software environments that are rapidly evolving, so that they can be distributed using BOINC.

Published in: Education
  • Even though I am not one of the young members with the T4T I have enjoyed working with this project and have been with the LHC since 2004

    This is my type of project and as far as computers that can be *trusted* well mine can and always will be *trusted* and I have been a Boinc member since 2000 running 24/7 with as many computers as I have at any time.

    And it has also been nice talking to Ben Segal during the tests and all three of you have done a fine job with the team.

    Thanks,

    -Samson Ben Yoseph-

    (Magic)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

LHC@home and CernVM – a new approach to porting large-scale applications to BOINC

  1. 1. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsLHC@home and CernVM – a new approach to porting large-scale applications to BOINC ˜ ´ Daniel Lombrana Gonzalez, Ben Segal and Artem Harutyunyan Citizen Cyberscience Centre CERN March 20, 2011 ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  2. 2. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsOutline 1 Introduction 2 Technical challenges 3 Employed solutions 4 CernVM + Co-Pilot 5 Test4theory project 6 Conclusions ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  3. 3. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsThe beginning The challenge Why don’t you run real LHC physics on BOINC? ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  4. 4. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsThe challenge Allow “any” PC to run a full LHC physics application. Make those commodity computers look like a “standard” CERN Data Center. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  5. 5. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsWe did it! ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  6. 6. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsOutline 1 Introduction 2 Technical challenges 3 Employed solutions 4 CernVM + Co-Pilot 5 Test4theory project 6 Conclusions ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  7. 7. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsChallenge 1: Porting source code ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  8. 8. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsChallenge 1: Porting source code ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  9. 9. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsChallenge 1: Porting source code ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  10. 10. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsChallenge 2: Job management systems Jobs must be fed into BOINC PCs, but CERN physics experiments have their own and don’t want to use the BOINC distributions system. Volunteer computing resources are not “managed” and thus cannot be “trusted”. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  11. 11. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsOutline 1 Introduction 2 Technical challenges 3 Employed solutions 4 CernVM + Co-Pilot 5 Test4theory project 6 Conclusions ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  12. 12. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsCernVM + Co-Pilot Using Virtualization and CernVM it is possible to solve the challenges (1) for application deployment on volunteers’ machines. With CernVM’s Co-Pilot it is possible to connect existing Grid infrastructures of LHC experiments with the BOINC volunteer resources, solving challenges (2). ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  13. 13. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsCernVM solution CernVM is a baseline Virtual Software Appliance for the participants of CERN LHC experiments. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  14. 14. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsCernVM solution CernVM is a baseline Virtual Software Appliance for the participants of CERN LHC experiments. The goal is to remove a need for the installation of the experiment software and to minimize the number of platforms. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  15. 15. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsCernVM Co-Pilot solution A framework for the delivery and execution of the workload on the remote virtual machines. Consists of components developed to ease the integration of cloud resources into existing Grid infrastructures. Components communicate using Jabber/XMPP instant messaging protocol. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  16. 16. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsOutline 1 Introduction 2 Technical challenges 3 Employed solutions 4 CernVM + Co-Pilot 5 Test4theory project 6 Conclusions ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  17. 17. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsCernVM Using virtualization and CernVM solve the problem of porting the source code and deploying over PCs. CernVM Co-Pilot can connect existing Grid infrastructure of LHC experiments with BOINC resources. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  18. 18. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsBOINC + CernVM infrastructure ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  19. 19. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsCo-Pilot Adapters Each LHC experiment that wants to connect to a set of CernVM machines via Co-Pilot, needs an adapter. Each adapter fetches jobs from the experiment’s preferred job scheduler, submits them to the CernVM machines, and returns the results to the scheduler. Co-Pilot has a built-in security barrier, preventing untrusted CernVM machines to access Grid resources. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  20. 20. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsOutline 1 Introduction 2 Technical challenges 3 Employed solutions 4 CernVM + Co-Pilot 5 Test4theory project 6 Conclusions ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  21. 21. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsMonte Carlo events generation ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  22. 22. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsAlpha testing ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  23. 23. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsActive forums More than 500 posts in the forums so far. Very active volunteers, reporting all the problems they find. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  24. 24. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsHigh schools Three young students were invited to test the project. One of them became very helpful in the first stages of the testing phase. The obtained feedback allowed us to improve the project, knowing that young students could collaborate with complex projects. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  25. 25. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsNumbers Since the beginning of the project 25000 jobs have been successfully executed. 80% of the jobs were set to generate 300000 events (the rest were smaller ones that generated 50000 events). Totaling 6250 millions of events simulated since December 2010. There are on average 20 on-line users at any given point in time (peaks up to 30 users). We are running 1200-1300 jobs daily. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  26. 26. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsOutline 1 Introduction 2 Technical challenges 3 Employed solutions 4 CernVM + Co-Pilot 5 Test4theory project 6 Conclusions ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  27. 27. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsConclusions A custom virtualized execution environment has been added to BOINC. Very complex and unmodified applications can be run within the system. Adapting Grid services to Co-Pilot is now understood. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  28. 28. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsConclusions ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  29. 29. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsQuestions teleyinex@gmail.com Icons from Tango and Gnome Desktop projects (Creative Commons & GPL License) ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home

×