LCE13: Linaro Infrastructure Update

Uploaded on

Resource: LCE13 …

Resource: LCE13
Name: Linaro Infrastructure Update
Date: 12-07-2013

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. LAVA Infrastructure Services Update Paul Sokolovsky, Milo Casagrande, James Tunnicliffe, Philip Colmer LCE13 - Dublin, July 2013 Present & Future Infrastructure Review
  • 2. ● There was was a separate Infrastructure Team before 2013-03 ● As Linaro infrastructure matures and stabilizes, Infrastructure team was merged with LAVA team, as account for "A" in its name (Linaro Automation and Validation). ● As an automation team, LAVA owns few codebases and tools (mostly related to CI) and co-maintains services based on them together with ITS team. Intro and History
  • 3. ● Non-exhaustive list of tools, projects and services (co)maintained by LAVA team (not counting LAVA server itself): ○ linaro-image-tools ○ Jenkins setup on & android-build.linaro. org ○ Frontend app on ○ AOSP/upstream mirror system ○ Gerrit on ○ & ○ Intro
  • 4. ● Servers migration from Canonical Datacenter to Linaro Cloud (driven by ITS) ● Switching to Linaro Login as SSO for all Linaro services (driven by ITS) ● Gerrit upgrade to 2.5 ○ Contemporary Gerrit version with improved permission granularity ○ Allowed to fully automate AOSP mirror without compromising security Recent Activity
  • 5. ● Dynamic publishing ● Jenkins setup for complete public vs private build separation ● Improved Git service with Rhodecode frontend ● ○ Patchwork codebase forked to add nice metrics. Original project: ■ More or less popular among big FOSS projects ■ Undermaintained (Google for it) ○ plan to migrate to support Crowd auth Upcoming Projects
  • 6. System management software to standardize on? ● We'd like to pass maintenance of existing infra to ITS, but codebases are not in ideal shape for production or devel deployment. ● Multitude of heterogeneous adhoc solutions. ● Existing system config management software: ○ Puppet, Chef - Legacy ○ SaltStack - Already used for LAVA lab deployment, requires adhoc daemons ○ Ansible - Requires only ssh Background tasks and questions
  • 7. ● Scalability ○ git:// protocol is greedy - consumes RAM/resources ● Stability & Reliability ○ OOM errors on server ● Easy to use interface ○ No SSH to create a repository ● Integrated ACL system ○ With authentication ○ That can use Linaro authentication system ● Private repositories ○ Preferably on the same instance We do not want to lock anybody out or blindly throw hardware at the problem Rhodecode - Problems it solves
  • 8. ● Open source project ○ Written in Python ● Integrated LDAP support ○ Crowd support available in beta version of RhodeCode ● Easily scalable ○ We run 4 instances on the same machine already ● Web UI ○ Easy to create repositories ○ Customizable ● Admin interface, groups support, ... Bonus: ● Has integrated code review (with inline comments) Rhodecode - Why
  • 9. ● Provides dumb HTTP/HTTPS clone support ○ git processes were spawned, could take up to 1.5GB RAM/process ○ Now done via Apache X-SendFile ■ Slightly slower than git protocol ○ Authorization always happens in RhodeCode ● Authorization caching for HTTP/HTTPS clone operations ○ Files are served one at the time - auth happened on each file ○ Needed to speed up clones (from ~35s to ~5s for a small repo) ● User groups and users matched to system ones ○ Needed to provide git+ssh access to Linaro engineers ○ Needed for push operation to happen with SSH keys ○ Access is granted via file system level ACLs ● Small UI tweaks ○ Added git+ssh clone URLs on the repository page Rhodecode - Linaro changes
  • 10. ● Android ○ Git & Gerrit ● LAVA ○ BZR (planning to move to Git & Gerrit) ■ need to review Rhodecode/Gerrit Integration ● Git general ○ Rhodecode (previously gitweb) ■ Migration Timeline Overall Git Architecture
  • 11. ● System management software to standardize on? ● Further optimizations to android-build performance? ● ● LAVA ○ transitioning to Git / Gerrit Future Plans