LCE13: Linaro Infrastructure Update


Published on

Resource: LCE13
Name: Linaro Infrastructure Update
Date: 12-07-2013

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

LCE13: Linaro Infrastructure Update

  1. 1. LAVA Infrastructure Services Update Paul Sokolovsky, Milo Casagrande, James Tunnicliffe, Philip Colmer LCE13 - Dublin, July 2013 Present & Future Infrastructure Review
  2. 2. ● There was was a separate Infrastructure Team before 2013-03 ● As Linaro infrastructure matures and stabilizes, Infrastructure team was merged with LAVA team, as account for "A" in its name (Linaro Automation and Validation). ● As an automation team, LAVA owns few codebases and tools (mostly related to CI) and co-maintains services based on them together with ITS team. Intro and History
  3. 3. ● Non-exhaustive list of tools, projects and services (co)maintained by LAVA team (not counting LAVA server itself): ○ linaro-image-tools ○ Jenkins setup on & android-build.linaro. org ○ Frontend app on ○ AOSP/upstream mirror system ○ Gerrit on ○ & ○ Intro
  4. 4. ● Servers migration from Canonical Datacenter to Linaro Cloud (driven by ITS) ● Switching to Linaro Login as SSO for all Linaro services (driven by ITS) ● Gerrit upgrade to 2.5 ○ Contemporary Gerrit version with improved permission granularity ○ Allowed to fully automate AOSP mirror without compromising security Recent Activity
  5. 5. ● Dynamic publishing ● Jenkins setup for complete public vs private build separation ● Improved Git service with Rhodecode frontend ● ○ Patchwork codebase forked to add nice metrics. Original project: ■ More or less popular among big FOSS projects ■ Undermaintained (Google for it) ○ plan to migrate to support Crowd auth Upcoming Projects
  6. 6. System management software to standardize on? ● We'd like to pass maintenance of existing infra to ITS, but codebases are not in ideal shape for production or devel deployment. ● Multitude of heterogeneous adhoc solutions. ● Existing system config management software: ○ Puppet, Chef - Legacy ○ SaltStack - Already used for LAVA lab deployment, requires adhoc daemons ○ Ansible - Requires only ssh Background tasks and questions
  7. 7. ● Scalability ○ git:// protocol is greedy - consumes RAM/resources ● Stability & Reliability ○ OOM errors on server ● Easy to use interface ○ No SSH to create a repository ● Integrated ACL system ○ With authentication ○ That can use Linaro authentication system ● Private repositories ○ Preferably on the same instance We do not want to lock anybody out or blindly throw hardware at the problem Rhodecode - Problems it solves
  8. 8. ● Open source project ○ Written in Python ● Integrated LDAP support ○ Crowd support available in beta version of RhodeCode ● Easily scalable ○ We run 4 instances on the same machine already ● Web UI ○ Easy to create repositories ○ Customizable ● Admin interface, groups support, ... Bonus: ● Has integrated code review (with inline comments) Rhodecode - Why
  9. 9. ● Provides dumb HTTP/HTTPS clone support ○ git processes were spawned, could take up to 1.5GB RAM/process ○ Now done via Apache X-SendFile ■ Slightly slower than git protocol ○ Authorization always happens in RhodeCode ● Authorization caching for HTTP/HTTPS clone operations ○ Files are served one at the time - auth happened on each file ○ Needed to speed up clones (from ~35s to ~5s for a small repo) ● User groups and users matched to system ones ○ Needed to provide git+ssh access to Linaro engineers ○ Needed for push operation to happen with SSH keys ○ Access is granted via file system level ACLs ● Small UI tweaks ○ Added git+ssh clone URLs on the repository page Rhodecode - Linaro changes
  10. 10. ● Android ○ Git & Gerrit ● LAVA ○ BZR (planning to move to Git & Gerrit) ■ need to review Rhodecode/Gerrit Integration ● Git general ○ Rhodecode (previously gitweb) ■ Migration Timeline Overall Git Architecture
  11. 11. ● System management software to standardize on? ● Further optimizations to android-build performance? ● ● LAVA ○ transitioning to Git / Gerrit Future Plans