Chef on Python and MongoDB

Rick Copeland @rick446 Arborian Consulting, LLC

  Infrastructure as Code   Resources & Providers   Cookbooks, Recipes, Clients, and Nodes

CouchDB Solr Solr Indexer Chef Server RabbitMQ RabbitMQ RabbitMQ (API) HTTP REST API Chef Server knife chef-‐client (Web UI)

Build, register, and authenticate the node Synchronize cookbooks Build resource collection (run the recipes in order) Conﬁgure node (“converge”) Run notiﬁcation handlers

CouchDB Solr Ruby Solr Indexer Chef Server RabbitMQ RabbitMQ RabbitMQ (API) Ruby Ruby HTTP REST API Chef Server knife chef-‐client (Web UI)

  Chef assumes a bootstrapped node exists   Chef doesn’t keep release notes   Code and infrastructure are versioned diﬀerently   Solution: Web app to manage deployments & generate release notes

MonQ MongoDB Solr Indexer Python Chef Server Solr (API + web) Ruby HTTP REST API knife chef-‐client

  Reduce # of processes & technologies   Don’t know Ruby well   Keep private keys out of the system   Integrate with existing authentication   Performance

/clients /nodes /data /environments /roles /sandboxes /cookbooks /search

  Mostly JSON  almost BSON   References to Ruby files stored separately { "name": "allura-‐0.0.1”,… "json_class": "Chef::CookbookVersion", Ruby files stored on S3 "attributes": [ { "name": "default.rb", "url": "https://s3.amazonaws.com/opscode-‐platform-‐production-‐data/… "checksum": "749a3a328d6c47a32d134b336183981f", "path": "attributes/default.rb", "specificity": "default”…

MongoDB Validator role = collection( chef.role, doc_session, Field(_id, S.ObjectId()), Field(account_id, S.ObjectId(if_missing=None)), Shorthand with Field(name, str), Python Types Field(description, str), Field(default_attributes, str), Field(override_attributes, str), Field(run_list, [ str ] ), Field(env_run_lists, { str: [ str ]}), Index(account_id, name, unique=True)) Embedded Index Deﬁnitions Documents

Models know where they live class Role(object): def url(self): return request.relative_url( config.chef_api_root + /roles/ + self.name) Models can be def __json__(self): return dict( turned into dict (to chef_type=role, be JSONiﬁed) json_class=Chef::Role,! … default_attributes=loads(self.default_attributes),! …) Models can be def update(self, d): self.name = d[name] updated from dict … self.default_attributes = dumps(d[default_attributes]) self.override_attributes = dumps(d[override_attributes])! …

class RoleSchema(JSONModelSchema):! model_class=CM.role! chef_type=role’! json_class=Chef::Role’! exclude_fields=[_id, account_id]

  /foo/bar/baz  root.foo.bar.baz   Lots of decorators   Validation (params/body  **kwargs)   Authorization   Rendering   HTTP method lookup   GET /foo/bar  root.foo.bar._get()

Returns JSON class RolesController(RESTController): @expose(template_engine=json) def _get(self):… Only admins can access @expose(template_engine=json, acl=[CACE.admin(True), ACE.any(False)], schema=cv.RoleSchema) Convert and def _post(self, **kwargs):… Validate POST def __getattr__(self, name): return RoleController(name) Continue dotted lookup

class RoleController(RESTController): @expose(template_engine=json) def _get(self): … PUT looks just like a POST @expose(template_engine=json, acl=[CACE.admin(True), ACE.any(False)], schema=cv.RoleSchema) def _put(self, name, **kwargs): … @expose(template_engine=json, acl=[CACE.admin(True), ACE.any(False)]) def _delete(self): …

class RoleController(RESTController):! AttributeError  def __init__(self, name): HTTP 404 Not Found self._role = CM.Role.query.get( account_id=c.account._id, name=name) if self._role is None: raise AttributeError, name! @expose(template_engine=json) def _get(self): return self._role! Auto-‐JSONify

class RoleController(RESTController):! … @expose(template_engine=json, acl=[CACE.admin(True), ACE.any(False)], Update model schema=cv.RoleSchema) from kwargs def _put(self, name, **kwargs): assert name == self._role.name self._role.update(kwargs) return self._role! @expose(template_engine=json, acl=[CACE.admin(True), ACE.any(False)]) def _delete(self): self._role.delete() return self._role

  Don’t trust the docs   Don’t trust the docs ▪  Don’t trust the docs   Use fat models   Framework support for REST & JSON   You’re gonna have to learn some Ruby anyway   JSON != BSON

  Better test coverage   Search support (SOLR / ElasticSearch)   More testing with real-‐world deployments   Finalize integration with deployment manager

Rick Copeland @rick446 Arborian Consulting, LLC

  http://openmymind.net/2011/10/28/ CouchDB-‐And-‐MongoDB-‐Performance/   MongoDB is 14x faster   http://www.snailinaturtleneck.com/blog/ 2009/06/29/couchdb-‐vs-‐mongodb-‐ benchmark/

http://www.10gen.com	176
http://www.arborian.com	113
http://www.linkedin.com	46
http://lanyrd.com	26
http://duckduckgo.com	3
http://drupal1.10gen.cc	3
http://feeds.feedburner.com	2
http://w.mongodb.org	1
http://www.mongodb.orgwww.mongodb.org	1
http://www.hanrss.com	1
http://fiberization26.katinia.com	1

Chef on Python and MongoDB

by Rick Copeland

Accessibility

Categories

Upload Details

Usage Rights

11 Embeds 373

Statistics