Download to read offline
![Writing the driver
We need to create a NodeDriver based driver
class NephoscaleNodeDriver(NodeDriver):
"""Nephoscale node driver class"""
type = Provider.NEPHOSCALE
api_name = 'nephoscale'
name = 'NephoScale'
website = 'http://www.nephoscale.com'
connectionCls = NephoscaleConnection
features = {'create_node': ['ssh_key']}
def list_locations(self):
...](https://image.slidesharecdn.com/howtocreatelibclouddriver-140602091258-phpapp02/85/How-to-create-a-libcloud-driver-from-scratch-10-320.jpg)
![Writing the Connection class
and write a Connection class
class NephoscaleConnection(ConnectionUserAndKey):
host = API_HOST
responseCls = NephoscaleResponse
def add_default_headers(self, headers):
user_b64 = base64.b64encode(b('%s:%s' % (self.user_id,
self.key)))
headers['Authorization'] = 'Basic %s' %
(user_b64.decode('utf-8'))
return headers](https://image.slidesharecdn.com/howtocreatelibclouddriver-140602091258-phpapp02/85/How-to-create-a-libcloud-driver-from-scratch-11-320.jpg)
![Testing the driver
Test your driver often to make sure it works
user@user:~/dev/libcloud$ python
>>> from libcloud.compute.types import Provider
>>> from libcloud.compute.providers import get_driver
>>> driver = get_driver(Provider.NEPHOSCALE)
>>> conn = driver('user','correct_password')
>>> conn.list_nodes()
[<Node: uuid=e20bdbf7ef6890645f5b217e0bd2b5912b969cc1,
name=nepho-7, state=0, public_ips=['198.89.109.116'],
provider=NephoScale ...>]](https://image.slidesharecdn.com/howtocreatelibclouddriver-140602091258-phpapp02/85/How-to-create-a-libcloud-driver-from-scratch-14-320.jpg)
Uploaded byMike Muzurakis
How to create a libcloud driver from scratch
The document outlines the process of creating a custom Libcloud driver for the Nephoscale cloud provider, detailing tasks like listing nodes, creating and managing nodes, and integrating API functionality. It emphasizes the importance of following Libcloud conventions, testing the driver, and contributing back to the Libcloud repository. Resources and best practices for developing and contributing to the driver are provided, including links to documentation and sample code.
More Related Content
How to create a libcloud driver from scratch
- 1. How to createa libcloud driver from scratch the NephoScale case
- 2. Libcloud one library torule them all
- 3.
- 4. What if yourprovider is not supported? ● Clone ● Branch ● Write ● Test ● Merge back Time to contribute
- 5. Tasklist Backend actions ● list_nodes:list all nodes, including ip addresses, node state, metadata, etc ● list_images: list all the images the provider supports ● list_sizes: list different plans, providing data such as CPU, ram and disk size ● list_locations: List all available provider zones when applicable
- 6. Tasklist Node actions ● create_node:create a node specifying it’s size, image, name and location ● deploy_node: create a node and deploy a public ssh key and optionally run a deploy script ● reboot_node/shut down_node ● start_node/stop_node: start a stopped node, or stop a started one ● destroy_node: Delete the node
- 7. Getting ready APIs anddrivers ● Familiarize yourself with the provider API ● Choose a recent libcloud driver to base your code on ● Don’t break conventions. Do it the libcloud way
- 8. libcloud repo Clone andextend ● Clone libcloud from https://github.com/apache/libcloud ● Libcloud drivers live in libcloud/compute/drivers ● They all inherit from the base driver at libcloud/compute/base.py ● Create your git branch ● Add your driver on libcloud/compute/providers.py
- 9. Create driver file Createlibcloud/compute/drivers/nephoscale.py Specify the cloud provider API endpoint API_HOST = 'api.nephoscale.com' and the possible node states NODE_STATE_MAP = { 'on': NodeState.RUNNING, 'off': NodeState.UNKNOWN, 'unknown': NodeState.UNKNOWN, }
- 10. Writing the driver Weneed to create a NodeDriver based driver class NephoscaleNodeDriver(NodeDriver): """Nephoscale node driver class""" type = Provider.NEPHOSCALE api_name = 'nephoscale' name = 'NephoScale' website = 'http://www.nephoscale.com' connectionCls = NephoscaleConnection features = {'create_node': ['ssh_key']} def list_locations(self): ...
- 11. Writing the Connectionclass and write a Connection class class NephoscaleConnection(ConnectionUserAndKey): host = API_HOST responseCls = NephoscaleResponse def add_default_headers(self, headers): user_b64 = base64.b64encode(b('%s:%s' % (self.user_id, self.key))) headers['Authorization'] = 'Basic %s' % (user_b64.decode('utf-8')) return headers
- 12. Writing the Responseclass Response classes derive from JsonResponse or XMLRPCResponse class NephoscaleResponse(JsonResponse): """Nephoscale API Response""" def parse_error(self): if self.status == httplib.UNAUTHORIZED: raise InvalidCredsError('Authorization Failed') if self.status == httplib.NOT_FOUND: raise Exception("The resource you are looking for is not found.") return self.body
- 13. Keep consistency Keep consistency,hardcode things when necessary ● Most of the times you don’t need to override deploy_node ● Libcloud standardizes things for providers (nodes/sizes/images/locations) ● When a provider does non-standard stuff, override ● Some providers don’t return enough details ● Hardcode only when necessary
- 14. Testing the driver Testyour driver often to make sure it works user@user:~/dev/libcloud$ python >>> from libcloud.compute.types import Provider >>> from libcloud.compute.providers import get_driver >>> driver = get_driver(Provider.NEPHOSCALE) >>> conn = driver('user','correct_password') >>> conn.list_nodes() [<Node: uuid=e20bdbf7ef6890645f5b217e0bd2b5912b969cc1, name=nepho-7, state=0, public_ips=['198.89.109.116'], provider=NephoScale ...>]
- 15. Contributing to libcloud Someground rules and contribution workflow: ● Open a ticket on the tracker to notify any interested parties ● Write unit tests ● Follow code style (follow PEP8, use spaces for tabs, etc) ● Commit and open a pull request ● Wait for a review and make any changes pointed ● Create a patch with your changes and add it to your ticket
- 16. Thank you! To checkthe driver in action, visit https://mist.io More resources: ● https://libcloud.readthedocs.org/en/latest/development.html ● https://github.com/apache/libcloud ● https://libcloud.readthedocs.org/en/latest/developer_information.html Special thanks to Tomaz, maintainer of libcloud and a very helpful guy :)
Editor's Notes
- #3 Libcloud allows users to manage four different cloud resources: Cloud Servers, Cloud Storage, Load Balancers as a Service and DNS as a Service. Cloud Servers is the oldest and more mature part of the library and currently supports more than 40 providers.
- #4 Many cloud providers eg Amazon, Rackspace, Openstack etc, are already supported by libcloud. But what if your favorite provider isn’t?
- #5 We’ll just create one ourselves. Libcloud is an open-source project with solid documentation and lots of code examples.
- #6 The are two main sets of actions we need to support. Backend actions that are provider/account specific.
- #7 and node actions that refer to a specific node (Virtual Machine). Most of them our self-explanatory. deploy_node runs create node and supplies a public ssh key for server access. It can optionally wait for the server to come online and deploy a script.
- #8 We’ll need access to our provider API. In this scenario, the Nephoscale API. We will use the Digital Ocean driver as a template as it is one of the most recent, well written and up-to-date drivers. There is an effort by libcloud to standardize the basic functionality for all the compute drivers, and update the oldest ones.
- #9 We’ll start by cloning the libcloud repo. All our drivers are in /libcloud/compute/drivers Every driver inherits from the main base driver at libcloud/compute/base.py We’ll create a git branch and add our driver on the DRIVERS dict on libcloud/compute/providers.py And our Provider class on libcloud/compute/types.py We will use the Digital Ocean driver as a template as it is one of the most recent, well written and up-to-date drivers. There is an effort by libcloud to standardize the basic functionality for all the compute drivers, and update the oldest ones.
- #10 On libcloud/compute/drivers/ we create a file with the name of our provider: nephoscale.py in our case. After the initial imports that we copy from the Digital Ocean driver, we specify the API endpoint of our cloud provider and a dict with the states that the nodes can have. This differs from provider to provider, with some providers having just a few states -running, stopped - and others having intermediates ones -e.g. rebooting, shutting down etc.
- #11 Here we specify the type (same as on libcloud/compute/types.py), the connection class (NephoscaleConnection) and a features dict with the way deploy_node will try to authenticate to the created node after create_node has run . Inside the NephoscaleNodeDriver is where all our functions will live. We’ll write all functions we want to implement there, list_images, list_nodes, reboot_node etc. To keep consistency, we make sure that list_nodes returns a list of Node objects, list_images a list of NodeImage objects, list_sizes a list of NodeSize objects and list_locations a list of NodeLocation objects.
- #12 NephoscaleConnection is the Connection class that will handle connection, authentication and send the request to the NephoScale API endpoint. All drivers implement a Connection class that is responsible for sending the request data, adding HTTP headers or params and also encoding the request body.The API host endpoint is specified there, along with the HTTP headers or params that are used to authenticate for each provider. NephoScale API calls require HTTP Basic Authentication with the user/password base64 encoded on every request, so we’ll add it to add_default_params. Other providers handle authentication with requiring the params on each request -API key, password, secret etc.
- #13 All drivers also need a response class that handles responses of the API endpoint, parses body and returns Exceptions or the actual return content. Response classes for the libcloud drivers derive from JsonResponse or XMLRPCResponse, depending on the response type of the cloud provider -Json or XML. Both derive from Response class that lives in libcloud/common/base.py. We want to make sure here that in the case of errors we throw the correct exceptions - e.g. InvalidCredsError for failed authentication
- #14 Most of the drivers of libcloud implement create_node to return a Node object, of the newly created host. Function deploy_node is not overridden , but instead gets called from the NodeDriver, since it does a lot of things that need not be rewritten, such as checks that the node public ip is up, that the machine can be accessible, that it can authenticate via password or ssh key, and optionally runs a deploy script. One of the cool things with libcloud is that it standardizes things. It enforces the use of basic entities -nodes, sizes, images, locations- for all cloud providers it supports. SoftLayer for example does not provide a size entity through it’s API, but rather expects that we specify CPU, RAM and disk size when we create a node. The way libcloud driver is implemented, one can either pass these manually, or simply select one of the sizes provided by list_sizes. Ideally all size/image/location related data should be fetched by asking the provider. When this is not possible, for example when the provider does not implement a request, we need to hard code these settings. For example the pricing info for some of the providers isn’t returned while asking for the sizes (list_sizes) and thus is hard coded on libcloud/data/pricing.json
- #16 Having created the compute driver for our cloud provider and having tested it’s functionality, it is time to commit it to libcloud. In this case we need to write tests -ideally for all functionality, add some fixtures and make sure the tests pass. Libcloud contains unit tests for nearly everything and enforces testing for new functionality. Use the codestyle proposed by libcloud, follow the workflow and your patch will be accepted to trunk.
- #17 Have fun developing your driver! To check this driver in action log in to mist.io Special thanks to Tomaz, maintainer of liblcoud and it’s bigger laborer of love, for proof reading this text.