The Geni Experiment Engine
•Download as PPTX, PDF•
0 likes•422 views
The Geni Experiment Engine
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to The Geni Experiment Engine
Similar to The Geni Experiment Engine (20)
More from US-Ignite
More from US-Ignite (20)
Recently uploaded
Recently uploaded (20)
The Geni Experiment Engine
- 1. Andy Bavier Jim Chen Yvonne Coady Matt Hemmings Joe Mambretti Rick McGeer Sean McGeer Jude Nelson Patrick O’Connell Glenn Ricart Ulrike Stege Stephen Tredger Ignite Kickoff, January 26, 2016
- 2. GEE Goal • Rapid Deployment of Distributed Applications on a Continent-Wide and Global Basis – 5 minutes to “Hello, World” across the infrastructure – 15 minutes for deployment of a substantial application • Efficient use of distributed, shared infrastructure • Provision of foundational tools – Orchestration, Messaging, Wide-Area Storage • Easy extension across underlying infrastructures – designed for rapid expansion – First Deployment on GENI – Embed in other infrastructures/serve as cross- infrastructure platform
- 3. GEE Now And In Future Now Future (like, Tomorrow) x Gee Node Gee Node Gee Node Gee Node Gee Node GENI x x x x GENI SAVI Chameleon Gee Node Gee Node Gee Node Gee Node CloudLab Gee Node Gee Node Gee Node Gee Node
- 4. GEE Fundamental Concepts • Containers as fundamental execution environment – Very lightweight units of isolation – Extremely rapid deployment – Highly Efficient use of resources – Easy for sites to provision and allow • Leverage Cloud ecosystem and tools – FOSS orchestration, messaging system – Users benefit from broad community technology contributions, educational materials, etc
- 5. Containers vs VMs VMs • Isolated Environment • Single-service execution environment • Per-VM guest OS • One VM/core • Examples: Xen, KVM,… Containers • Isolated Environment • Single-service execution environment • Single OS • 10-20 containers/core • Examples: Vservers, LXC, BSD Jails
- 6. Ex: GEE Portal • GEE Portal (http://www.gee-project.org) runs in a container system on a VM – Each instance is two containers (webserver + scripts, portal DB) – Currently running three instances • Production Server, Dev Server, Ignite Feature Dev Server • All in one VM! Before containers, we’d have needed 6.
- 7. • A GEE “slicelet” is a set of containers connected by a private network • With slicelets, can easily deploy network services on GENI and other IaaS – “Five minute rule” for running Hello World – Use a custom image in your slicelet • Bootstrap a set of building-block services – Running inside other GEE slicelets – Deploy in your slicelet using orchestration tools
- 8. GEE Basic Tools • – Automate application deployment inside LXC – Image hosting on Docker Hub • – Intuitive IT configuration and automation – Declarative (YAML) instead of imperative • Fabric – Python wrapper around parallel SSH – Create CLI commands that are remotely executed
- 9. FAD Architecture Web server DB Event daemon sliceletslicelet slicelet Docker daemon GEE VM sliceletslicelet slicelet Docker daemon GEE VM . . . GEE portal Ansible GEE private network Fabric
- 10. Typical GEE Use Scenario • Log in to GEE Portal with GENI credentials • Allocate a slice • When Slice is “Ready” (usually < 5 minutes) download tarball with – ssh key for authentication – ssh configuration file – Ansible hosts file and pre-written “playbooks” – fabfile with configuration and pre-written commands • Write/run fabric/Ansible code to load, configure, deploy application on nodes
- 11. What This Means To A User • Two-minute allocation of “VMs” across the continent, using only a web browser • Single-pane-of-glass control of continent-wide service from a standard Unix/Linux/Cygwin shell • Minimal software to install (Ansible and/or Fabric)
- 12. Example: Ignite Visualizer Deployment • Distributed Two Web Servers, Special-Purpose DB Engine, 11 GB database • Installed 10 packages • Total time: 15 minutes • Total code: 20 lines of Ansible script
- 14. GEE Services: In Progress • GEE Storage Service – Leverages Syndicate storage service – UNIX file interface on top of a CDN • GEE Reverse Proxy Service – Everyone wants port 80 • Lively Web on GEE • Custom slicelet images on GEE – Now beta
- 15. • Five minutes to “Hello World!” on GENI • Implementation leverages modern tools – FAD: Fabric, Ansible, Docker • Set of building block services – Storage, messaging, reverse proxy, Lively http://gee-project.org
Editor's Notes
- What is GEE at a high level? The main abstraction is a “slicelet” – this is more lightweight than a GENI slice. Using slicelets, users can easily deploy… The GEE platform can run anywhere where you can get a VM with a public IP address and sufficient storage.
- How many people have used Docker? Docker is changing the way we deploy applications. You should check it out. Ansible is about automating IT configuration tasks. One nice thing about it is that you declare what state you want the system to be in, rather than what to do. Fabric is a tool that lets you easily write and invoke Python programs that execute parallel SSH commands on remote hosts. The basic idea is, if you built PlanetLab today, you would probably heavily leverage these tools or others like them. The world has come a long way since 2002 and PlanetLab hasn’t always kept pace.
- GEE’s architecture leverages all these tools. A user logs into the GEE portal and a slicelet record gets written to the GEE database. The event daemon sees it and uses Ansible to create new Docker containers inside the GEE VMs. There’s the user and here is his slicelet. The GEE portal also generates some helper files that the user downloads once the slicelet is created. For instance, it provides a template Fabfile that can be used to interact with the slicelet.
- Here are some other services that are in varying stages of completion … Reverse Proxy… demultiplex incoming HTTP requests to the correct slicelet Lively Web – quickly build dynamic web pages – talk to Rick…