• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Project-Builder.org presentation

Project-Builder.org presentation



Project-Builder.org is a tool to help upstream projects build their own packages for various Operating Systems directly from their VCS/CMS on a regular basis. It provides a continuous packaging ...

Project-Builder.org is a tool to help upstream projects build their own packages for various Operating Systems directly from their VCS/CMS on a regular basis. It provides a continuous packaging approch



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Project-Builder.org presentation Project-Builder.org presentation Presentation Transcript

    • Project Builder:A GPL continuous packaging solutionBruno Cornec HP EMEA Open Source Profession LeadV2.0 April 2013
    • Introducing Myself● Software engineering and Unices since 1988 ● Mostly Configuration Management Systems (CMS), Build systems, quality tools, on multiple commercial Unix systems ● Discover Open Source & Linux (OSL) & first contributions in 1993 ● Full time on OSL since 1995, first as HP reseller then @HP● Currently: ● Master Technology Architect on OSL for the HP/Intel Solution Center, Grenoble ● OSL HP Advocate ● EMEA OSL HP Profession Lead ● Solutions Linux Conference and OWF board member ● MondoRescue, Dploy.org, Project-Builder.org project lead ● LinuxCOE, mrepo, tellico, rinse, fossology, collectl contributor ● FOSSBazaar and OSL Governance enthusiast ● Mandriva, Mageia, Fedora packager 2 HP / OSSI / Bruno Cornec
    • What is the problem ? Users / Sysadmins environment: >= 90% < 10%Users: easyness (GUI, CLI)Admins: distribution compliance, < 1%smooth integration – reduce admin load 3 HP / OSSI / Bruno Cornec
    • What is the problem ? Developers environment: Sourceforge Flat Files ftp, web Project CMS/VCS autotools ? CVS Install proc ? Soft. Eng. ? SVN Fear of distro ? GIT, ...4 HP / OSSI / Bruno Cornec
    • What is the problem ? Packagers environment: Distro ftp, web Flat Files Package CMS/VCSDoes s/he only exists ? CVSOn time ? Lag behind ? rpmlint ?Distro coverage (ver, #) SVN lintian ?Beta avail ? GIT, ... 5 HP / OSSI / Bruno Cornec
    • My answer with my various hats Im a sysadmin so I want to deal with packages only (deployment server integration) Im a developer so I want to ease my tests without screwing up my system and testing as a std user/admin Im a software engineer and want no duplication of code nor information (metadata) Im a a packager so I want to streamline that work for new projects Im interested in packaging Im the marketing department of my own project and use packages as a “competitive advantage” Founding no tool existing to help me doing all that (2005), so I started writing one to scratch my own itch6 HP / OSSI / Bruno Cornec
    • What is the solution ? Packaging should be a project concern as well as coding, testing, installing, .... especially for smaller projects New mantra: “Package early, package always” Use packaging as your only way of delivery (not a dream) Overhead minimal (not much longer than providing a tgz), maximum benefit (consistancy, reproduceability, team work improvement, distribution integration, improved deployment) Packaging is also a marketing activity based on a technical content. Its an easy way to extend your user base, and improve your community. Makes tests easy. THE SOLUTION IS CONTINUOUS PACKAGING7 HP / OSSI / Bruno Cornec
    • Cloud based Continuous Packaging Local Build Server (may host VMs and/or VEs)Packagers VM or VE Build Repository Build + Server metadata Local build Project RM Build Farm (may host VMs) Remote buildDevelopers 9 HP / OSSI / Bruno Cornec
    • Provisioning the Continuous Packaging Cloud Local Build Server (may host VMs and/or VEs) VM or VE Build Build + metadata Local build Repository Server Project RM Build Farm (may host VMs) Remote build10 HP / OSSI / Bruno Cornec
    • Goals Project-builder main goal is to help you package continuously being agnostic: CMS agnostic: no CMS but guys its 21st century now, SVN, CVS, Mercurial, GIT, SVK.... OS agnostic: Linux: RPM, deb, ebuild, slack based, ... 120+ distro tuples at the moment – repositories for yum, urpmi, apt. Solaris pkg. HP-UX sd in progress Build environment agnostic: local, VM (QEMU, KVM...), VE (rpmbootstrap, rinse, mock, debootstrap...), RM (external build farm) No project impact: preserves the md5sum of the delivered upstream sources. Can be completely external to the upstream project.11 HP / OSSI / Bruno Cornec
    • Goodies Project-builder provides additional goodies: Easy VMs/VEs/RMs management Macro system with perl variables to avoid duplication Skeleton generation to help starting Manages delivery up to your repository (ssh based) Manages announces on mailing lists Integration of tests in the process Manages patches/additional sources when not upstream Checks validity of packages built (lintian, rpmlint) Easy creation of new versions for upstream management13 HP / OSSI / Bruno Cornec
    • VM, VE & RM Management VE Creation: debootstrap for .deb based ones. rpmbootstrap (modeled after debootstrap): create chroot for multiple RPM based distributions (Mageia, Fedora, Mandriva, CentOS, OpenSuSE) VM|RM Creation: KVM or QEMU iso install possibilities. LinuxCOE/OpenStack as provisioning tool for your VM|RM VM|VE|RM setup: adds pb tools and build account VM|VE Snapshot feature Building in VM|VE|RM == building locally (build2vm|ve|rm vs build2pkg). ssh communication for VM/RM14 HP / OSSI / Bruno Cornec
    • Metadata Management Packaging process independent of the project and external Metadata isolated - No duplication15 HP / OSSI / Bruno Cornec
    • Configuration file usage Macro example:./rpmbootstrap/pbfilter/all.pbf:filter PBSUMMARY = rpmbootstrap is a tool similarto debootstrap for RPM based distributions./rpmbootstrap/rpm/rpmbootstrap.spec:Summary: PBSUMMARY./rpmbootstrap/deb/control:Description: PBSUMMARY./rpmbootstrap/pkg/pkginfo:NAME="PBSUMMARY" Instantiation possible from distro-ver-arch, distro-ver, distro, distro- family, distro-type, os Similar approach for all other parameters (Cf: man pb.conf)16 HP / OSSI / Bruno Cornec
    • Commands[cms|sbx]2build: Create tar files for the project under your CMS. Parameters arepackages to build if not using default listbuild2pkg: Create packages for your running distributionbuild2ssh: Send the tar files to a SSH hostpkg2ssh: Send the packages built to a SSH hostbuild2vm|ve|rm: Create packages in VMs/VEs/RMs, launching them if needed andsend those packages to a SSH host once builtlaunchvm: Launch one virtual machinescript2vm|ve|rm: Launch a VM/VE/RM if needed and executes a script on ittest2vm|ve|rm: Test inside a VM/VE/RMnewvm|ve: Create a new VM/VE/RMsetupvm|ve|rm: Setup a VM/VE/RM for pb usagenewver: Create a new version of the project derived from the current onenewproj: Create a new project and a template set of configuration files underpbconf17 HP / OSSI / Bruno Cornec
    • The Big picturePackagers newver / newproj newvm|ve / setupvm|ve|rm Packaging build2vm|ve|rm cms|sbx2build Project Building Packages build2pkg pkg2ssh build2ssh Developers 18 HP / OSSI / Bruno Cornec
    • Multi platform life cycleUser Requirements Validation Functional Integration tests Specification Design Unit tests Specification 19 Coding HP / OSSI / Bruno Cornec
    • HP+Intel Solution Brief21 HP / OSSI / Bruno Cornec
    • Multi platform packaging cycle VMs/VEs/RMsDevelopers UUWL Packages VM/VE/RM 23 Packagers HP / OSSI / Bruno Cornec
    • Q & A up to now ?24 HP / OSSI / Bruno Cornec
    • DEMONSTRATION25 HP / OSSI / Bruno Cornec
    • To be done LSB chroot support. (or not) LXC support Support for libvirt, virsh, openstack API Other CMS (Bazaar, …) only when/if needed Other VMs (VMWare, Xen, ...) only when/if needed Multiple delivery means Look at interactions with Buildbot Add non-interactive signature support for .deb Config-Model for configuration file management REST API and Web Interface26 HP / OSSI / Bruno Cornec
    • Learn Project-Builder.org Start with the Lab (63 pages) Use man (pb, pb.conf + 8 ProjectBuilder::* man pages) Use the mailing-list pb-announce and pb-devel at http://www.mondorescue.org/sympa Use examples from http://trac.project-builder.org/browser/projects/27 HP / OSSI / Bruno Cornec
    • Web ResourcesProject-Builder Web site / Trac / Wiki:● http://www.project-builder.org● http://trac.project-builder.orgProjects using project-builder.org: ● http://www.project-builder.org (of course :-) ● http://www.mondorescue.org ● Http://www.linuxcoe.org ● http://www.fossology.org ● http://kde-apps.org/content/show.php/Meganizer?content=129907 28 HP / OSSI / Bruno Cornec
    • Related toolsProject-Builder is mostly suited for upstream projects wanting to package theirapplicationsDistributions provide each their build tools ● SuSE: Open Build Service (Multi distro, Web based, BaaS) ● Fedora (Koji) ● Mandriva/Mageia (Youri, mdvsys, mgarepo...)Other complementary tools: ● Buildbot ● KVM/QEMU ● rpmbootstrap, rinse, mock ● Parallel::ForkManager, DBI, DB::SQLite, File::MimeInfo, Mail::SendmailProject-Builder.org is one of the tool of the vcs-pkg.org initiative. 29 HP / OSSI / Bruno Cornec
    • Bruno_Cornec@hp.com Contact (Linux Solution Consultant in the HP/Intel Solution Center) http://www.hp.com/linux ”Changes are never easy to make. There is comfort and safety in tradition, but change must come, no matter how painful or expensive it may be.” Bill Hewlett Linus Torvalds, Richard Stallman, Eric Raymond, Nat Makarevitch, René Cougnenc, Eric Dumas, Rémy Card, Bdale Thanks Garbee, Bryan Gartner, Gallig Renaud, Joachim Langenbach among others, for their work and devotion to the Open Source Software cause... and my family Open Source at HP for his patience :-) http://opensource.hp.com30 HP / OSSI / Bruno Cornec