Your SlideShare is downloading. ×
LHC@home and CernVM – a new approach to porting large-scale applications to BOINC
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

LHC@home and CernVM – a new approach to porting large-scale applications to BOINC

925
views

Published on

In this talk, I will review a new application in the volunteer computing project LHC@home, called Test4Theory, which is currently being alpha tested at CERN. This application uses virtual machine …

In this talk, I will review a new application in the volunteer computing project LHC@home, called Test4Theory, which is currently being alpha tested at CERN. This application uses virtual machine technology and a range of other software tools to handle large, complex software environments that are rapidly evolving, so that they can be distributed using BOINC.

Published in: Education

1 Comment
0 Likes
Statistics
Notes
  • Even though I am not one of the young members with the T4T I have enjoyed working with this project and have been with the LHC since 2004

    This is my type of project and as far as computers that can be *trusted* well mine can and always will be *trusted* and I have been a Boinc member since 2000 running 24/7 with as many computers as I have at any time.

    And it has also been nice talking to Ben Segal during the tests and all three of you have done a fine job with the team.

    Thanks,

    -Samson Ben Yoseph-

    (Magic)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total Views
925
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
1
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsLHC@home and CernVM – a new approach to porting large-scale applications to BOINC ˜ ´ Daniel Lombrana Gonzalez, Ben Segal and Artem Harutyunyan Citizen Cyberscience Centre CERN March 20, 2011 ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 2. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsOutline 1 Introduction 2 Technical challenges 3 Employed solutions 4 CernVM + Co-Pilot 5 Test4theory project 6 Conclusions ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 3. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsThe beginning The challenge Why don’t you run real LHC physics on BOINC? ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 4. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsThe challenge Allow “any” PC to run a full LHC physics application. Make those commodity computers look like a “standard” CERN Data Center. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 5. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsWe did it! ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 6. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsOutline 1 Introduction 2 Technical challenges 3 Employed solutions 4 CernVM + Co-Pilot 5 Test4theory project 6 Conclusions ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 7. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsChallenge 1: Porting source code ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 8. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsChallenge 1: Porting source code ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 9. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsChallenge 1: Porting source code ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 10. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsChallenge 2: Job management systems Jobs must be fed into BOINC PCs, but CERN physics experiments have their own and don’t want to use the BOINC distributions system. Volunteer computing resources are not “managed” and thus cannot be “trusted”. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 11. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsOutline 1 Introduction 2 Technical challenges 3 Employed solutions 4 CernVM + Co-Pilot 5 Test4theory project 6 Conclusions ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 12. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsCernVM + Co-Pilot Using Virtualization and CernVM it is possible to solve the challenges (1) for application deployment on volunteers’ machines. With CernVM’s Co-Pilot it is possible to connect existing Grid infrastructures of LHC experiments with the BOINC volunteer resources, solving challenges (2). ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 13. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsCernVM solution CernVM is a baseline Virtual Software Appliance for the participants of CERN LHC experiments. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 14. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsCernVM solution CernVM is a baseline Virtual Software Appliance for the participants of CERN LHC experiments. The goal is to remove a need for the installation of the experiment software and to minimize the number of platforms. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 15. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsCernVM Co-Pilot solution A framework for the delivery and execution of the workload on the remote virtual machines. Consists of components developed to ease the integration of cloud resources into existing Grid infrastructures. Components communicate using Jabber/XMPP instant messaging protocol. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 16. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsOutline 1 Introduction 2 Technical challenges 3 Employed solutions 4 CernVM + Co-Pilot 5 Test4theory project 6 Conclusions ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 17. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsCernVM Using virtualization and CernVM solve the problem of porting the source code and deploying over PCs. CernVM Co-Pilot can connect existing Grid infrastructure of LHC experiments with BOINC resources. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 18. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsBOINC + CernVM infrastructure ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 19. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsCo-Pilot Adapters Each LHC experiment that wants to connect to a set of CernVM machines via Co-Pilot, needs an adapter. Each adapter fetches jobs from the experiment’s preferred job scheduler, submits them to the CernVM machines, and returns the results to the scheduler. Co-Pilot has a built-in security barrier, preventing untrusted CernVM machines to access Grid resources. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 20. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsOutline 1 Introduction 2 Technical challenges 3 Employed solutions 4 CernVM + Co-Pilot 5 Test4theory project 6 Conclusions ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 21. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsMonte Carlo events generation ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 22. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsAlpha testing ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 23. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsActive forums More than 500 posts in the forums so far. Very active volunteers, reporting all the problems they find. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 24. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsHigh schools Three young students were invited to test the project. One of them became very helpful in the first stages of the testing phase. The obtained feedback allowed us to improve the project, knowing that young students could collaborate with complex projects. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 25. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsNumbers Since the beginning of the project 25000 jobs have been successfully executed. 80% of the jobs were set to generate 300000 events (the rest were smaller ones that generated 50000 events). Totaling 6250 millions of events simulated since December 2010. There are on average 20 on-line users at any given point in time (peaks up to 30 users). We are running 1200-1300 jobs daily. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 26. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsOutline 1 Introduction 2 Technical challenges 3 Employed solutions 4 CernVM + Co-Pilot 5 Test4theory project 6 Conclusions ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 27. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsConclusions A custom virtualized execution environment has been added to BOINC. Very complex and unmodified applications can be run within the system. Adapting Grid services to Co-Pilot is now understood. ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 28. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsConclusions ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home
  • 29. Introduction Technical challenges Employed solutions CernVM + Co-Pilot Test4theory project ConclusionsQuestions teleyinex@gmail.com Icons from Tango and Gnome Desktop projects (Creative Commons & GPL License) ˜ D. Lombrana, B. Segal & A. Harutyunyan LHC@Home