Your SlideShare is downloading. ×
Presentation at CHEP 2006
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Presentation at CHEP 2006


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. The LCG SPI project in LCG Phase II CHEP’06, Mumbai, India Feb. 14, 2006 Andreas Pfeiffer -- for the SPI team [email_address]
  • 2. Overview
    • Introduction and Motivation
    • Services of SPI
      • Savannah service
      • External software and distribution services
      • Configuration and Build Infrastructure
      • Software testing and QA services
      • Documentation service
    • Outlook and future plans
    • Summary
  • 3. The SPI project in the LCG Applications Area
    • Goal of the project
      • Provide to the development projects of the LCG Application Area
        • Basic environment for physics SW development
        • General scientific libraries and class libraries
        • Software development tools
        • Documentation tools and document templates
        • Compiler expertise
      • Have consistency and homogeneity in the development of the different packages of the LCG Application Area
        • Configuration management
        • Central software repositories
        • Common policies (naming, building, etc.)
  • 4. Software Process and Infrastructure
    • Work guided and prioritized by Architects Forum
      • Providing services to the projects (and experiments)
      • Channelling the requests from projects (and experiments)
    • Providing the following services:
      • Savannah service
        • issue and task tracker
      • Software services
        • central installations of external software, software distribution
      • Software development service
        • configuration mgt, pre- and post-build service, tools for testing and QA
      • Web and Documentation
  • 5. Savannah Service
    • Web portal for LCG software projects
      • Single entry point for all projects
        • Uniform “look and feel” for all projects
      • Totally web based
    • Used for various functionalities
      • Bug/issue tracking
      • Task management
      • Download area
    • Customized from GNU savannah project
      • In collaboration with main developer
  • 6. Adoption by projects 1575 users 176 projects 2770 tasks 13500 bugs
  • 7. Functional enhancements
    • Project specific 'cookbooks'
      • based on the tracker engine
      • Inherits the tracker functionality while providing a friendly book-like item display.
    • Export facility
      • Helps extracting data to insert into reports (new releases, statistics, project monitoring, etc)
    • Voting system
      • Gives registered users the possibility to prioritize the way items are handled
    • Additional browsing criteria
      • Selection of items modified/unmodified since a certain date
    • Extended transition mechanism
      • Automatic update of multiple fields on specific value transitions
  • 8. External Software Services
    • Install software needed by LCG projects and experiments
      • Open source and public domain software
        • Libraries and tools
    • Standard structure of directories (central AFS repository)
      • <packageName>/<version>/<platform>/(lib, src, bin, …)
    • Build and dependency information now stored in XML files with scripts to build (semi-)automatically
      • Very easy to build a new version or new platform
      • Easy to convert to formats for other tools (e.g. pacman)
    • Experiments and projects decide what is installed
      • Through “Architects Forum”
  • 9. External software service -- example
  • 10. Software Distribution Service
    • Existing solution since 2002
      • Script to download and install tar-balls of binaries for software of LCG AA projects with all dependencies
      • Limited to binaries of platforms build at CERN
      • Dependency information from configuration management and build tool (scram)
    • Effort started to download source tarballs and (re-) build as needed
      • Simplifies porting to new platforms (or development work)
      • Re-using build and dependency information from XML files
      • Will be extended to download/install binaries
    • Work ongoing to fully automate this service
  • 11. Configuration and Build Infrastructure
    • LCG Librarian
      • Maintaining configuration information centrally for build systems in use
        • XML files + converters to scram and cmt
      • Coordinating releases and pre-releases
    • Librarians integration group
      • Discuss with experiments’ librarians/integrators additional needs on external packages
      • Bi-weekly meeting (LIM)
    • Developed simple tools to automate builds of LCG AA software
      • Three phases: pre-build, build (on all platforms in parallel), post-build
    • Work ongoing to improve automation
      • Provide tools for release managers in projects to do this
  • 12. Software Testing Service
    • Providing Test frameworks, support, policies and documentation
      • CppUnit, Oval
      • Qmtest
      • valgrind
      • On different compilers/platforms
    • Running the tests is an integral part of the development in the projects
      • Unit tests, integration tests and regression tests
    • Also available for experiments
      • Initial setup
      • User support
  • 13. Quality Assurance Service
    • Central QA checklist on each release of LCG AA projects
      • Code coverage
      • Providing statistics on tests and from savannah as a coherent view
    • Automated procedure to generate reports
      • At every release
      • Published on the SPI web site
    • Evaluation and usage of external tools
      • E.g. RuleChecker, Logiscope
  • 14. QA example (ROOT project)
  • 15. QA example (POOL project)
  • 16. Documentation Service
    • Document LCG Policies
      • Homogenize the projects “look and feel”
      • CVS and release directory structures
      • Software testing policies
    • Code documentation
      • Through SPI web site and wiki pages
      • Tools used: doxygen and LXR
      • Run for every release of LCG AA projects plus some external packages (CLHEP, ROOT, HepMC)
        • Procedure is now semi-automated
    • Internal procedures documented
      • “ explicit knowledge” rather than “reading scripts”
  • 17. Web and Documentation
    • Maintain and improve existing web pages
      • Update internal and external documentation (wiki pages)
      • SPI and LCG Workbooks
    • Automate content wherever possible
      • Create more “active/dynamic” web pages
        • E.g.: status of external tools, configuration info
        • New: pages to do pre- and post- build procedures
    • Update and clarify existing policies
      • To be reviewed by AF
    • Update web pages for external packages
      • Responsibles, status, … for each package
  • 18. Challenges for LCG phase II
    • Servicing LHC experiments more than AA projects
      • Discussion with experiments to (re-)define scope
        • What is needed, what can be improved
    • Participation from people in experiments
      • “ On-the-spot” help with requests from experiments
      • SPI provides a “frame” to do complex work
        • Guidelines, AFS space, web space for docs, …
    • Coping with strong (50%) reduction of resources
      • Optimization opportunities
      • Further optimization and automatization of procedures
  • 19. Summary
    • SPI provides a large set of stable services that fully use the current resources
      • Savannah, External Software, Testing Frameworks, Software Distribution, Build and Release, QA Activities
    • SPI work is guided by the Architects Forum
      • AF represents the users and steers our priorities
    • In Phase II we will concentrate more on the experiments’ needs
      • Who see LCG AA project software as “external”
    • Ongoing discussion with AF and experiments
      • To redefine the scope of the project
        • Resources are significantly reduced
  • 20. “ Thank you!” to the SPI team(s)
    • 2005
      • Alberto Aimar, Johanne Benard, Ian McLaren, Yannick Patois, Yves Perrin, Andreas Pfeiffer, Eric Poinsignon
    • 2006
      • Full time: Ian McLaren, Yves Perrin, Andreas Pfeiffer
      • Part time: Bertrand Bellenot, Manuel Gallas, Stefan Roiser
  • 21. Additional Information
    • SPI project web:
    • SPI project wiki pages:
    • LCG Applications Area web:
    • LCG web: