Using GOCA to Manage OpenNebula - Iguane Solutions

Iguane Solutions | Public
Iguane Solutions ©2019

Iguane Solutions
Iguane Solutions ©2019 - all rights reserved
2
Our contribution to “Goca”
Jean-Philippe Fourès

Iguane Solutions
Agenda
3
● Iguane Solutions small introduction
● Goca contribution
○ Goca introduction
○ Details of our contribution
○ Roadmap
● Example of applications using Goca

1. Company Introduction
4
Iguane Solutions

Iguane Solutions
5
•Network & CDN
•>200 racks in 3 DCs
•OpenNebula clusters
• Infrastructure Expert (Network, Hardware, Storage)
• “DevOps” culture: everything is automated
• Our tools are Open Source based:
CI/CD, Infra as code, metrology, Cloud
• 1 OpenNebula cluster (2 very soon ☺) in Paris:
○ 10 KVM nodes (580 vCPU, 3.6 TB RAM)
○ Storpool for performance Storage
• Cloud Hybridation specialist:
Public / Private or Physical / Virtual
•Network & CDN
•>5 racks
•Network & CDN
•>10 racks
•Network & CDN
•>5 racks
•Network & CDN
•>10 racks
1.1 Who are we ?
Founded in 2000, HQ in Paris, France

66
Iguane Solutions
● For 7 years, IG1 cloud has been using VMware but:
○ Licence are expensive
○ Troubleshooting VSAN is a nightmare (and it is a black box)
○ VMware has lots of features => complexity
1.2 Iguane and One
}
● In 2018, development of a new cloud service
○ Simpler than VMware
○ API friendly
○ Easy to deploy, to extend, to manage, to maintain and to hack
around
○ Able to manage: VMware clusters and Public clouds
=> End of 2018: deployment of our ﬁrst cluster based on OpenNebula
In parallel to the deployment: Contribution to Goca and development tools using it

7
Iguane Solutions
2. Goca

88
Iguane Solutions
2.1 Introduction to Goca
OCA (OpenNebula Cloud API)
(ruby, java, python, golang)

99
Iguane Solutions
2.1 Introduction to Goca
What is Goca ? Why Goca instead of Ruby or
Python API ?
Golang
Wrapper on top of XML RPC
API
Written in Golang
Go + OCA = GOCA
Add helpers to ease XML RPC
instructions
Python API not maintained
Not Ruby friendly:
Sorry Guys
Golang: growing language in
area of backend development
https://talks.golang.org/2014/gocon-tokyo.slide#16

1010
Iguane Solutions
2.2 Our contribution
Rewrite XML mapping to parse and build
entities data
● Change the XML handling package and add
structures to map XML content
● Add dynamic management of template parts
Rework Architecture: adding a controller layer
● Better split of concerns, client and controller
● Modify the API to access the resources in a
controller centric way
Rework of the XML-RPC client
● Give the control of the client to the user:
○ Client externalization,
○ Possible to extend it
● Leverage the use of the initial XML-RPC:
○ Manage only XML-RPC in XML-RPC package
○ Manage HTTP in the client directly
Rewrite the error management
● Enable ﬁne-grained management of errors with
types and codes
● Better error handling:
Don’t kill everyone on a package error

1111
Iguane Solutions
2.3 Goca Organization
Goca subpackages
OpenNebula cluster
“errors”:
Error management package
“schema”:
Package containing all XML
structures deﬁned in
different packages. Each
package matches an
OpenNebula object
“goca”:
It contains a ﬁle per
object family api call.
client.go manages
HTTP connection
controller.go
manages resources

1212
Iguane Solutions
2.4 Goca workﬂow
How it works ?
OpenNebula
/RPC2
client
Model
VM
Controller
group
...
ask client to call
one.vm.info
with value ‘42’
Controller eases objects manipulation
and parsing.
An application only needs to “play”
with the controller to perform queries
to OpenNebula
Model, here is the client which
performs HTTP requests to
OpenNebula.
It builds XML requests (session, XML
formatting) and HTTP layer.
# application code to get
# info for VM 42
cli = goca.NewClient(...)
c =
goca.NewController(cli)
vm = c.VM(42).Info() HTTP request to
OpenNebula XML
RPC endpoint
Goca

1313
Iguane Solutions
2.5 Roadmap Ideas
Improve Tests - CI/CD
● Add more test cases to improve reliability and code
coverage on missing entities
● Rewrite existing tests according to Go coding style, and
consistently across entities
● Add a complete CI to test some scenarios with HTTP
request to a miniOne in a container
Increase the coverage of the XML-RPC API
● Highlights non-covered methods (missing tests case)
● Highlights unimplemented methods (dead code)
● Detect breaking changes, implement fallback methods
● Bug ﬁxes
Increase the user friendliness of the code
● Make the code more readable, improve code comments
● Try to minimize boilerplate code on developer side.
For instance: type conversions
● Minimize the user need to read the documentation.
For instance: remove variable parameters handling
New functionalities
● Allow to ﬁlter entities on various criterias as template
elements for instance
● Contribute to the development of OpenNebula new
functionalities
Suggestions are welcomed

3. Applications
Iguane Solutions
14

1515
Iguane Solutions
3.1 Terraform Provider
Important note:
Runtastic and Blackberry developed an OpenNebula provider using XML RPC calls directly before
goca merge into One repository.
● Software developed by Hashicorp:
Commonly used to deploy
Infrastructure on Public clouds thanks
to a large ecosystem of providers:
AWS, GCP, Azure, …
● Written in Golang, it offers APIs to
write custom providers easily
Thanks to the Goca, we initiated the
development of the OpenNebula Provider.
Tested with
● Terraform version: 0.12+
● OpenNebula 5.8
+
Ofﬁcial OpenNebula Add-on since June 2019!

1616
Iguane Solutions
Group Image Virtual
Network
Security
Group
VDC VM
Template
VM
For more details, issues, roadmap, contribution are welcomed:
https://github.com/OpenNebula/terraform-provider-opennebula
October 2018 June 2019 September 2019
Major steps history
Internal beta release
Added as
OpenNebula Add-on
Submission to
Terraform
Contribution to Goca
Validation of ﬁrst resources
Resources currently supported:
Ofﬁcial in Terraform
Hopefully:
December 2019

1717
Iguane Solutions
Goca used subpackages
OpenNebula cluster
schema:
Terraform provider uses schema
packages linked to goca ﬁles and
permissions.go to manage object
permissions
quota.goto manage group quotas
lock.goto manage lock on images
“goca”:
Each terraform resources
matches with an
OpenNebula object

1818
Iguane Solutions
Sismograf:
● Metrology agent, developed
internally, deployed on each
monitored host or VM
● Metrics are then collected by our
Metrology tool: Sismology
● Several plugins available to
monitor system and common
application metrics:
Sismograf plugin for OpenNebula
has been developed to get usage
metrics of our OpenNebula
clusters
3.2 Sismograf Plugin
OpenNebula Metrics into Iguane Solutions Metrology
OpenNebula Fronts
Sismograf agents to
collect OpenNebula
metrics
VMs
on OpenNebula
Sismograf agents to
collect VM metrics
Iguane Sismology:
Metrology system

1919
Iguane Solutions
3.2 Sismograf Plugin
Goca used subpackages
OpenNebula cluster
“goca”:
It is used in the Sismograf
plugin dedicated to
OpenNebula.
It gets all metrics of our
cloud.
It mainly use Info API calls.
schema:
Sismograf OpenNebula plugin uses
schema packages linked to goca to get
metrics on the various objects
it also uses quota.goand snapshot.go
because we also have metrics on quotas
and size of a snapshot.

Questions ?
Iguane Solutions
20

Using GOCA to Manage OpenNebula - Iguane Solutions

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Using GOCA to Manage OpenNebula - Iguane Solutions

Similar to Using GOCA to Manage OpenNebula - Iguane Solutions (20)

Recently uploaded

Recently uploaded (20)

Using GOCA to Manage OpenNebula - Iguane Solutions