XPDS14 - osstest, Xen's Automatic Testing Facility - Ian Jackson, Citrix


Published on

osstest, Xen's automatic testing facility - In the last year, osstest has gained contributors and many new test cases. A wider range of guests is being tested in a wider range of configurations; osstest's scope has increased to a wider set of the software components used in a Xen system. In this presentation, Ian Jackson will highlight some of the most interesting of these contributions (and their contributors). The presentation will include some figures showing osstests's community development. We'll then look forward to changes in the pipeline, including planned new features and infrastructure changes.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

XPDS14 - osstest, Xen's Automatic Testing Facility - Ian Jackson, Citrix

  1. 1. osstest Xen Project automatic test system Community participation Recent developments Future plans Ian Campbell <ian.campbell@citrix.com> Ian Jackson <ian.jackson@eu.citrix.com> Xen Developer Summit Chicago 2014
  2. 2. To: <xen−devel@xxxxxxxxxxxxxxxxxxx> Subject: [libvirt test] 29207: regressions − FAIL flight 29207 libvirt real [real] http://www.chiark.greenend.org.uk/~xensrcts/logs/29207/ Regressions :−( Tests which did not succeed and are blocking, including tests which could not be run: build−amd64−libvirt 5 libvirt−build fail REGR. vs. 29190 build−i386−libvirt 5 libvirt−build fail REGR. vs. 29190 build−armhf−libvirt 5 libvirt−build fail REGR. vs. 29190 Tests which did not succeed, but are not blocking: test−amd64−i386−libvirt 1 build−check(1) blocked n/a test−amd64−amd64−libvirt 1 build−check(1) blocked n/a test−armhf−armhf−libvirt 1 build−check(1) blocked n/a version targeted for testing: libvirt ddf6b139cfebee4193bf6f0b13c995a182d7bc35 baseline version: libvirt 72a8453948691b75a1802524837b6543353bf829 −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− People who touched revisions under test: ....
  3. 3. To: <xen−devel@lists.xensource.com> Subject: [xen−unstable bisection] complete build−armhf−libvirt branch xen−unstable xen branch xen−unstable job build−armhf−libvirt test libvirt−build Tree: gnulib_libvirt git://drall.uk.xensource.com:9419/git://git.sv.gnu.org/gnulib.git%20[fetch=try] Tree: libvirt git://xenbits.xen.org/libvirt.git Tree: qemuu git://xenbits.xen.org/staging/qemu−upstream−unstable.git Tree: xen git://xenbits.xen.org/xen.git *** Found and reproduced problem changeset *** Bug is in tree: xen git://xenbits.xen.org/xen.git Bug introduced: 871b43a309d80ac99458c13c2c3da8d15c482d30 Bug not present: 6cc89d3101d8874e01a69a89a65736a2adfbd199 commit 871b43a309d80ac99458c13c2c3da8d15c482d30 Author: Senatus Populus <senatus.populus@example.com> Date: Fri Jun 20 18:19:12 2014 +0200 libxl: get and set soft affinity Make space a new cpumap in vcpu_info, called cpumap_soft, for retrieving soft affinity, and amend the relevant API accordingly. libxl_set_vcpuaffinity() now takes two cpumaps, one for hard and one for soft affinity (LIBXL_API_VERSION is exploited to retain source level backword compatibility). Either of the two cpumap can be NULL, in which case, only the affinity corresponding to the non−NULL cpumap will be affected. ... For bisection revision−tuple graph see: http://www.chiark.greenend.org.uk/~xensrcts/results/bisect.xen−unstable.build−armhf−libvirt.libvirt−build.html Revision IDs in each graph node refer, respectively, to the Trees above. −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− Searching for failure / basis pass: 27871 fail [host=army] / 27481 ok. Failure / basis pass flights: 27871 / 27481 Tree: gnulib_libvirt git://drall.uk.xensource.com:9419/git://git.sv.gnu.org/gnulib.git%20[fetch=try] Tree: libvirt git://xenbits.xen.org/libvirt.git Tree: qemuu git://xenbits.xen.org/staging/qemu−upstream−unstable.git Tree: xen git://xenbits.xen.org/xen.git
  4. 4. branches jobs libvirt linux−3.10 linux−3.4 linux−arm−xen linux−linus linux−mingo−tip−master linux−next osstest qemu−mainline qemu−upstream−4.2−testing qemu−upstream−4.3−testing qemu−upstream−4.4−testing qemu−upstream−unstable rumpuserxen seabios xen−4.0−testing xen−4.1−testing xen−4.2−testing xen−4.3−testing xen−4.4−testing xen−unstable build−amd64 build−amd64−libvirt build−amd64−oldkern build−amd64−pvops build−amd64−rumpuserxen build−amd64−xend build−armhf build−armhf−libvirt build−armhf−pvops build−i386 build−i386−libvirt build−i386−oldkern build−i386−pvops build−i386−rumpuserxen build−i386−xend test−amd64−amd64−libvirt test−amd64−amd64−pair test−amd64−amd64−pv test−amd64−amd64−rumpuserxen−amd64 test−amd64−amd64−rumpuserxen−i386 test−amd64−amd64−xl test−amd64−amd64−xl−pcipt−intel test−amd64−amd64−xl−qemut−debianhvm−amd64 test−amd64−amd64−xl−qemut−win7−amd64 test−amd64−amd64−xl−qemut−winxpsp3 test−amd64−amd64−xl−qemuu−debianhvm−amd64 test−amd64−amd64−xl−qemuu−ovmf−amd64 test−amd64−amd64−xl−qemuu−win7−amd64 test−amd64−amd64−xl−qemuu−winxpsp3 test−amd64−amd64−xl−sedf test−amd64−amd64−xl−sedf−pin test−amd64−amd64−xl−win7−amd64 test−amd64−amd64−xl−winxpsp3 test−amd64−i386−freebsd10−amd64 test−amd64−i386−freebsd10−i386 test−amd64−i386−libvirt test−amd64−i386−pair test−amd64−i386−pv test−amd64−i386−qemut−rhel6hvm−amd test−amd64−i386−qemut−rhel6hvm−intel test−amd64−i386−qemuu−freebsd10−amd64 test−amd64−i386−qemuu−freebsd10−i386 test−amd64−i386−qemuu−rhel6hvm−amd test−amd64−i386−qemuu−rhel6hvm−intel test−amd64−i386−rhel6hvm−amd test−amd64−i386−rhel6hvm−intel test−amd64−i386−rumpuserxen−i386 test−amd64−i386−xend−qemut−winxpsp3 test−amd64−i386−xend−qemuu−winxpsp3 test−amd64−i386−xend−winxpsp3 test−amd64−i386−xl test−amd64−i386−xl−credit2 test−amd64−i386−xl−multivcpu test−amd64−i386−xl−qemut−debianhvm−amd64 test−amd64−i386−xl−qemut−win7−amd64 test−amd64−i386−xl−qemut−winxpsp3 test−amd64−i386−xl−qemut−winxpsp3−vcpus1 test−amd64−i386−xl−qemuu−debianhvm−amd64 test−amd64−i386−xl−qemuu−ovmf−amd64 test−amd64−i386−xl−qemuu−win7−amd64 test−amd64−i386−xl−qemuu−winxpsp3 test−amd64−i386−xl−qemuu−winxpsp3−vcpus1 test−amd64−i386−xl−win7−amd64 test−amd64−i386−xl−winxpsp3 test−amd64−i386−xl−winxpsp3−vcpus1 test−armhf−armhf−libvirt test−armhf−armhf−xl test−i386−i386−libvirt test−i386−i386−pair test−i386−i386−pv test−i386−i386−rumpuserxen−i386 test−i386−i386−xl test−i386−i386−xl−qemut−winxpsp3 test−i386−i386−xl−qemuu−winxpsp3 test−i386−i386−xl−winxpsp3
  5. 5. Introduction ============ OSStest is the Xen Project automated test infrastructure. ... Terminology =========== "flight": Each run of osstest is referred to as a "flight". Each flight is given a unique ID (a number or name). "job": Each flight consists of one or more "jobs". These are a sequence of test steps run in order and correspond to a column in the test report grid. They have names like "build−amd64" or "test−amd64−amd64−pv". A job can depend on the output of another job in the flight −− e.g. most test−* jobs depend on one or more build−* jobs. ... Standalone Mode =============== To run osstest in standalone mode: − You need to install sqlite3 tcl8.4 tclx8.4 libsqlite3−tcl libdbi−perl libdbd−sqlite3−perl pax rsync curl netcat − Optional: ipmitool − Write a config file ~/.xen−osstest/config See below. − Select the "branch" and job to reproduce. By default the system gives you the "branch" consisting of tests run for the xen−unstable push gate. You need to select a job. The list of available jobs is that shown in the publicly emailed test reports on xen−devel, eg http://lists.xen.org/archives/html/xen−devel/2014−01/msg01614.html
  6. 6. | army earwig spider potato−beetle fire−frog itch−mite rice−weevil | bedbug insider leaf−beetle field−cricket marilith−n4 | hydrazine lace−bug lake−frog marilith−n5 | woodlouse moss−bug chicken−mite grain−weevil | bush−cricket gall−mite −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−+−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− arch−amd64 | x x x x x x x x x x x x x x x x x x arch−armhf | x x x arch−i386 | x x x x x x x x x x x x x x x x x arch−xen−amd64 | x x x x x x x x x x x x x x x x x x arch−xen−armhf | x x arch−xen−i386 | x x x x x x x x x x x x x x x x x blessed−adhoc | x x x x x x x x x x x x x x x x blessed−play | x x x x x x x x x x x x x x x x blessed−real | x x x x x x x x x x x x x x x x equiv−dell−r310 | x x equiv−marilith | x x equiv−rackservers−q21011 | x x equiv−rackservers−s40663 | x x equiv−rackservers−s40670 | x x equiv−rackservers−s40679 | x x equiv−rackservers−s40680 | x x hvm | x x x x x x x x x x x x x x x x x x hvm−amd | x x x x x x x hvm−intel | x x x x x x x x x x x need−firmware−deb−firmware−bnx2 | x x need−kernel−deb−armmp | x x need−uboot−bootscr | x x no−di−kernel | x x no−reinstall | x pcipassthrough−nic | x x purpose−build | x x x x x x x x x x x x x x x x purpose−test | x x x x x x x x x x x x x x x x x x x spong | x suite−squeeze | x x x x x x x x x x x x x x x x x x x x x suite−wheezy | x x x x x x x x x x x x x x x x x x x x x x
  7. 7. 800 700 600 500 400 300 200 100 0 osstest loc by author date 2014−06−09 Anthony Perard 2013−11−21 Roger Pau Monné 2013−11−06 Dario Faggioli 2013−09−16 Wei Liu 2013−09−03 Ian Campbell 2009−11−23 Ian Jackson 01/01/12 01/07/12 01/01/13 01/07/13 01/01/14 01/07/14
  8. 8. 60 50 40 30 20 10 0 16 14 12 10 8 6 4 2 0 jobs in xen-unstable branches ("[BRANCH test]") 01/06/13 01/08/13 01/10/13 01/12/13 01/02/14 01/04/14 01/06/14 01/08/14
  9. 9. References Source repository http://xenbits.xen.org/gitweb/?p=osstest.git;a=summary README http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=HEAD Output example http://xenbits.xen.org/docs/osstest−output−example/24438/ Contact xen−devel@lists.xenproject.org Ian.Jackson@eu.citrix.com Ian.Campbell@citrix.com