1. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
STREAMLINING XEN PROJECT CONTRIBUTIONS
THROUGH CI
DOUG GOLDSTEIN
RACKSPACE
CARDOE@CARDOE.COM // CARDOE @FREENODE
JULY 2019

2. PROBABLY NOT.
“Back to the pile!”

3. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
ABOUT ME
▸ Site Reliability Engineer @ Rackspace
▸ Keeping VM hosts patched and happy
▸ Xen Project Contributor and Maintainer
▸ Maintainer for the GitLab and GitHub integration with
Wei Liu.
“Cloud gonna cloud.”

4. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
OVERVIEW
▸ Current workflow for patches
▸ The issues
▸ How some bots work
▸ Expanding on the vision
▸ Discussing how we get there

5. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
EXISTING WORKFLOW
▸ Follows git-flow https://nvie.com/posts/a-successful-git-
branching-model/
▸ committers commit to staging
▸ OSSTEST builds staging and commits to master
▸ freezes are established to let OSSTEST catch up
▸ release is cut

6. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
THE ISSUES
▸ Common theme from Nanjing: Freeze windows are too long.
▸ Merge skew / Schematic conflicts
▸ [17:00] Alice pulls staging & applies patch series which uses foo()
▸ [17:01] Alice begins local tests
▸ [17:05] Bob pull staging & renames foo() to bar()
▸ [17:06] Bob begins compile only tests
▸ [17:10] Bob’s code compiles, merge and push.
▸ [17:20] Alice creates a merge commit and pushes.

7. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
THE ISSUES CONT.
▸ Committers have different standards by which a commit is
mergeable.
▸ Committers could have the same standards but different
environments in which they verify changes.
▸ Time spent doing highly mechanical work instead of
technical work.

8. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
SOME BOT BASED APPROACHES
▸ Rust - bors. Example: https://github.com/rust-lang/rust/pull/59800

9. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
BORS CONT

10. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
BORS CONT

11. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
BORS CONT

12. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
BORS CONT.
▸ Bors merges to master
▸ Bors has priority values (security issues, scheduled
features)
▸ GitHub does not have this natively integrated so bots must
consume the API
▸ https://bors.tech

13. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
SOME BOT BASED APPROACHES CONT.
▸ GitLab - Merge Trains
MASTER
PR
1
PR
2
PR
3
MASTERPR
1
PR
2
PR
3
MASTER
+PR 1
PR
2
PR
3
MASTER
+PR 1
PR
3
PR 2

14. Xen-devel@
:patchew
or patchwork
Gitlab CI
submit only review
if tests pass
Fetches series/patches
create tmp
git tree
Build (distros, configs, compilers)
& smoke tests (x86 only for now)
Tested-by: bot
or NACKed-by: bot
commit as now
From Lars Kurth’s Xen Project Weather Report
2.0

15. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
STUMBLING BLOCKS
▸ Getting the patches
▸ patchew.org - no public API to get list of available
patches but mbox available by Message-ID
▸ lore.kernel.org - Atom feed to get data but HTML
escaped patches. No Message-ID.
▸ patchwork.kernel.org - Proposed changed by Wei to
support our use case haven’t had much traction
upstream

16. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
STUMBLING BLOCKS
▸ Tracking new versions of a series and updating the branch
▸ The amount of emails on the list
▸ How many will the bot add?
▸ How many repos get patches sent to the xen-devel@ list.

17. WILL ROBOTS AUTOMATE YOUR JOB AWAY?
PROJECT GANIS
▸ Named after the kingdom that King Bors rules from
Arthurian legends.
▸ Aims to start running along side the existing workflow for a
time period until the Xen Project is comfortable with the
results
▸ Design Session: Xen Project CI Infrastructure v2

18. QUESTIONS?