SlideShare a Scribd company logo
1 of 22
Download to read offline
Rick	
  Copeland	
  @rick446	
  
Arborian	
  Consulting,	
  LLC	
  
  Infrastructure	
  as	
  Code	
  


  Resources	
  &	
  Providers	
  


  Cookbooks,	
  Recipes,	
  Clients,	
  and	
  Nodes	
  
CouchDB	
                                                     Solr	
  



                                                          Solr	
  Indexer	
  


Chef	
  Server	
  	
  
                                        RabbitMQ	
  
                                          RabbitMQ	
  
                                           RabbitMQ	
  
  (API)	
  



              HTTP	
  REST	
  API	
  


Chef	
  Server	
  
                                          knife	
          chef-­‐client	
  
 (Web	
  UI)	
  
Build,	
  register,	
  and	
  authenticate	
  the	
  node	
  

                   Synchronize	
  cookbooks	
  

Build	
  resource	
  collection	
  (run	
  the	
  recipes	
  in	
  order)	
  


                Configure	
  node	
  (“converge”)	
  

                  Run	
  notification	
  handlers	
  
CouchDB	
                                                            Solr	
  


                                                                 Ruby	
  
                                                                            Solr	
  Indexer	
  


           Chef	
  Server	
  	
  
                                                   RabbitMQ	
  
                                                     RabbitMQ	
  
                                                      RabbitMQ	
  
             (API)	
  
                                                                                 Ruby	
  
Ruby	
  
                         HTTP	
  REST	
  API	
  


           Chef	
  Server	
  
                                                     knife	
                 chef-­‐client	
  
            (Web	
  UI)	
  
  Chef	
  assumes	
  a	
  bootstrapped	
  node	
  exists	
  


  Chef	
  doesn’t	
  keep	
  release	
  notes	
  


  Code	
  and	
  infrastructure	
  are	
  versioned	
  
  differently	
  

  Solution:	
  Web	
  app	
  to	
  manage	
  deployments	
  &	
  
  generate	
  release	
  notes	
  
MonQ	
  
         MongoDB	
  
                                                                      Solr	
  Indexer	
  



                                                         Python	
  

        Chef	
  Server	
  	
  
                                                   Solr	
  
        (API	
  +	
  web)	
  

                                   Ruby	
  
HTTP	
  REST	
  API	
  



                       knife	
                    chef-­‐client	
  
  Reduce	
  #	
  of	
  processes	
  &	
  technologies	
  


  Don’t	
  know	
  Ruby	
  well	
  


  Keep	
  private	
  keys	
  out	
  of	
  the	
  system	
  

  Integrate	
  with	
  existing	
  authentication	
  


  Performance	
  
/clients	
         /nodes	
  


   /data	
       /environments	
  


   /roles	
       /sandboxes	
  


/cookbooks	
         /search	
  
  Mostly	
  JSON	
  	
  almost	
  BSON	
  
   References	
  to	
  Ruby	
  files	
  stored	
  separately	
  

{	
  
	
  	
  "name":	
  "allura-­‐0.0.1”,…	
  
	
  	
  "json_class":	
  "Chef::CookbookVersion",	
  
                                                                          Ruby	
  files	
  stored	
  on	
  S3	
  
	
  	
  "attributes":	
  [	
  
	
  	
  	
  	
  {	
  
	
  	
  	
  	
  	
  	
  "name":	
  "default.rb",	
  
	
  	
  	
  	
  	
  	
  "url":	
  "https://s3.amazonaws.com/opscode-­‐platform-­‐production-­‐data/…	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  
	
  	
  	
  	
  	
  	
  "checksum":	
  "749a3a328d6c47a32d134b336183981f",	
  
	
  	
  	
  	
  	
  	
  "path":	
  "attributes/default.rb",	
  
	
  	
  	
  	
  	
  	
  "specificity":	
  "default”…	
  
MongoDB	
  Validator	
  
role = collection(
     'chef.role', doc_session,
     Field('_id', S.ObjectId()),
     Field('account_id', S.ObjectId(if_missing=None)),
                                          Shorthand	
  with	
  
     Field('name', str),
                                           Python	
  Types	
  
     Field('description', str),
     Field('default_attributes', str),
     Field('override_attributes', str),
     Field('run_list', [ str ] ),
     Field('env_run_lists', { str: [ str ]}),
     Index('account_id', 'name', unique=True))


                                                Embedded	
  
                 Index	
  Definitions	
  
                                                Documents	
  
Models	
  know	
  
                                where	
  they	
  live	
  
class Role(object):

      def url(self):
         return request.relative_url(
             config.chef_api_root + '/roles/' + self.name)
                                           Models	
  can	
  be	
  
      def __json__(self):
         return dict(
                                   turned	
  into	
  dict	
  (to	
  
             chef_type='role',        be	
  JSONified)	
  
             json_class='Chef::Role',!
             …
             default_attributes=loads(self.default_attributes),!
             …)
                                                  Models	
  can	
  be	
  
      def update(self, d):
         self.name = d['name']         updated	
  from	
  dict	
  
         …
         self.default_attributes = dumps(d['default_attributes'])
         self.override_attributes = dumps(d['override_attributes'])!
         …
class RoleSchema(JSONModelSchema):!

     model_class=CM.role!

     chef_type='role’!

     json_class='Chef::Role’!

     exclude_fields=['_id', 'account_id']
  /foo/bar/baz	
  	
  root.foo.bar.baz	
  


  Lots	
  of	
  decorators	
  
     Validation	
  (params/body	
  	
  **kwargs)	
  
     Authorization	
  
     Rendering	
  


  HTTP	
  method	
  lookup	
  
     GET	
  /foo/bar	
  	
  root.foo.bar._get()	
  
Returns	
  JSON	
  
class RolesController(RESTController):

      @expose(template_engine='json')
      def _get(self):…
                                               Only	
  admins	
  can	
  
                                                     access	
  
      @expose(template_engine='json',
              acl=[CACE.admin(True), ACE.any(False)],
              schema=cv.RoleSchema)
                                         Convert	
  and	
  
      def _post(self, **kwargs):…       Validate	
  POST	
  

      def __getattr__(self, name):
          return RoleController(name)            Continue	
  dotted	
  
                                                     lookup	
  
class RoleController(RESTController):

     @expose(template_engine='json')
     def _get(self):
         …                       PUT	
  looks	
  just	
  like	
  a	
  POST	
  

     @expose(template_engine='json',
             acl=[CACE.admin(True), ACE.any(False)],
             schema=cv.RoleSchema)
     def _put(self, name, **kwargs):
        …

     @expose(template_engine='json',
             acl=[CACE.admin(True), ACE.any(False)])
     def _delete(self):
        …  
class RoleController(RESTController):!
                                                AttributeError	
  	
  	
  
      def __init__(self, name):
                                   HTTP	
  404	
  Not	
  Found	
  
         self._role = CM.Role.query.get(
             account_id=c.account._id,
             name=name)
         if self._role is None:
             raise AttributeError, name!

     @expose(template_engine='json')
     def _get(self):
         return self._role!
                                                     Auto-­‐JSONify	
  
class RoleController(RESTController):!

      …
     @expose(template_engine='json',
             acl=[CACE.admin(True), ACE.any(False)],
                                         Update	
  model	
  
             schema=cv.RoleSchema)        from	
  kwargs	
  
     def _put(self, name, **kwargs):
         assert name == self._role.name
         self._role.update(kwargs)
         return self._role!

     @expose(template_engine='json',
             acl=[CACE.admin(True), ACE.any(False)])
     def _delete(self):
         self._role.delete()
         return self._role
    Don’t	
  trust	
  the	
  docs	
  
       Don’t	
  trust	
  the	
  docs	
  
        ▪  Don’t	
  trust	
  the	
  docs	
  

    Use	
  fat	
  models	
  

    Framework	
  support	
  for	
  REST	
  &	
  JSON	
  

    You’re	
  gonna	
  have	
  to	
  learn	
  some	
  Ruby	
  anyway	
  

    JSON	
  !=	
  BSON	
  
  Better	
  test	
  coverage	
  


  Search	
  support	
  (SOLR	
  /	
  ElasticSearch)	
  


  More	
  testing	
  with	
  real-­‐world	
  deployments	
  

  Finalize	
  integration	
  with	
  deployment	
  
  manager	
  
Rick	
  Copeland	
  @rick446	
  
Arborian	
  Consulting,	
  LLC	
  
  http://openmymind.net/2011/10/28/
 CouchDB-­‐And-­‐MongoDB-­‐Performance/	
  
    MongoDB	
  is	
  14x	
  faster	
  
  http://www.snailinaturtleneck.com/blog/
 2009/06/29/couchdb-­‐vs-­‐mongodb-­‐
 benchmark/	
  

More Related Content

What's hot

Scala ActiveRecord
Scala ActiveRecordScala ActiveRecord
Scala ActiveRecordscalaconfjp
 
O que há de novo no Rails 3 - Ruby on Rails no Mundo Real - 23may2010
O que há de novo no Rails 3 - Ruby on Rails no Mundo Real - 23may2010O que há de novo no Rails 3 - Ruby on Rails no Mundo Real - 23may2010
O que há de novo no Rails 3 - Ruby on Rails no Mundo Real - 23may2010Plataformatec
 
JavaScript Unit Testing with Jasmine
JavaScript Unit Testing with JasmineJavaScript Unit Testing with Jasmine
JavaScript Unit Testing with JasmineRaimonds Simanovskis
 
Ruby/Rails
Ruby/RailsRuby/Rails
Ruby/Railsrstankov
 
A tour on ruby and friends
A tour on ruby and friendsA tour on ruby and friends
A tour on ruby and friends旻琦 潘
 
To Batch Or Not To Batch
To Batch Or Not To BatchTo Batch Or Not To Batch
To Batch Or Not To BatchLuca Mearelli
 
Controlling The Cloud With Python
Controlling The Cloud With PythonControlling The Cloud With Python
Controlling The Cloud With PythonLuca Mearelli
 
The road to Ember.js 2.0
The road to Ember.js 2.0The road to Ember.js 2.0
The road to Ember.js 2.0Codemotion
 
REST APIs in Laravel 101
REST APIs in Laravel 101REST APIs in Laravel 101
REST APIs in Laravel 101Samantha Geitz
 
RESTful API 제대로 만들기
RESTful API 제대로 만들기RESTful API 제대로 만들기
RESTful API 제대로 만들기Juwon Kim
 
Pourquoi ruby et rails déchirent
Pourquoi ruby et rails déchirentPourquoi ruby et rails déchirent
Pourquoi ruby et rails déchirentNicolas Ledez
 
Bootstrat REST APIs with Laravel 5
Bootstrat REST APIs with Laravel 5Bootstrat REST APIs with Laravel 5
Bootstrat REST APIs with Laravel 5Elena Kolevska
 
JRuby, Ruby, Rails and You on the Cloud
JRuby, Ruby, Rails and You on the CloudJRuby, Ruby, Rails and You on the Cloud
JRuby, Ruby, Rails and You on the CloudHiro Asari
 

What's hot (20)

Scala ActiveRecord
Scala ActiveRecordScala ActiveRecord
Scala ActiveRecord
 
Scala active record
Scala active recordScala active record
Scala active record
 
O que há de novo no Rails 3 - Ruby on Rails no Mundo Real - 23may2010
O que há de novo no Rails 3 - Ruby on Rails no Mundo Real - 23may2010O que há de novo no Rails 3 - Ruby on Rails no Mundo Real - 23may2010
O que há de novo no Rails 3 - Ruby on Rails no Mundo Real - 23may2010
 
Play vs Rails
Play vs RailsPlay vs Rails
Play vs Rails
 
JavaScript Unit Testing with Jasmine
JavaScript Unit Testing with JasmineJavaScript Unit Testing with Jasmine
JavaScript Unit Testing with Jasmine
 
Ruby/Rails
Ruby/RailsRuby/Rails
Ruby/Rails
 
A tour on ruby and friends
A tour on ruby and friendsA tour on ruby and friends
A tour on ruby and friends
 
To Batch Or Not To Batch
To Batch Or Not To BatchTo Batch Or Not To Batch
To Batch Or Not To Batch
 
Why ruby
Why rubyWhy ruby
Why ruby
 
Controlling The Cloud With Python
Controlling The Cloud With PythonControlling The Cloud With Python
Controlling The Cloud With Python
 
The road to Ember.js 2.0
The road to Ember.js 2.0The road to Ember.js 2.0
The road to Ember.js 2.0
 
REST APIs in Laravel 101
REST APIs in Laravel 101REST APIs in Laravel 101
REST APIs in Laravel 101
 
Laravel intake 37 all days
Laravel intake 37 all daysLaravel intake 37 all days
Laravel intake 37 all days
 
RESTful API 제대로 만들기
RESTful API 제대로 만들기RESTful API 제대로 만들기
RESTful API 제대로 만들기
 
Fatc
FatcFatc
Fatc
 
Symfony2 meets propel 1.5
Symfony2 meets propel 1.5Symfony2 meets propel 1.5
Symfony2 meets propel 1.5
 
Pourquoi ruby et rails déchirent
Pourquoi ruby et rails déchirentPourquoi ruby et rails déchirent
Pourquoi ruby et rails déchirent
 
Bootstrat REST APIs with Laravel 5
Bootstrat REST APIs with Laravel 5Bootstrat REST APIs with Laravel 5
Bootstrat REST APIs with Laravel 5
 
JRuby, Ruby, Rails and You on the Cloud
JRuby, Ruby, Rails and You on the CloudJRuby, Ruby, Rails and You on the Cloud
JRuby, Ruby, Rails and You on the Cloud
 
AkJS Meetup - ES6++
AkJS Meetup -  ES6++AkJS Meetup -  ES6++
AkJS Meetup - ES6++
 

Similar to Chef on Python and MongoDB

Fast Web Applications Development with Ruby on Rails on Oracle
Fast Web Applications Development with Ruby on Rails on OracleFast Web Applications Development with Ruby on Rails on Oracle
Fast Web Applications Development with Ruby on Rails on OracleRaimonds Simanovskis
 
Connecting the Worlds of Java and Ruby with JRuby
Connecting the Worlds of Java and Ruby with JRubyConnecting the Worlds of Java and Ruby with JRuby
Connecting the Worlds of Java and Ruby with JRubyNick Sieger
 
Using Ruby on Rails with legacy Oracle databases
Using Ruby on Rails with legacy Oracle databasesUsing Ruby on Rails with legacy Oracle databases
Using Ruby on Rails with legacy Oracle databasesRaimonds Simanovskis
 
Building an aws sdk for Perl - Granada Perl Workshop 2014
Building an aws sdk for Perl - Granada Perl Workshop 2014Building an aws sdk for Perl - Granada Perl Workshop 2014
Building an aws sdk for Perl - Granada Perl Workshop 2014Jose Luis Martínez
 
Extending Oracle E-Business Suite with Ruby on Rails
Extending Oracle E-Business Suite with Ruby on RailsExtending Oracle E-Business Suite with Ruby on Rails
Extending Oracle E-Business Suite with Ruby on RailsRaimonds Simanovskis
 
SproutCore and the Future of Web Apps
SproutCore and the Future of Web AppsSproutCore and the Future of Web Apps
SproutCore and the Future of Web AppsMike Subelsky
 
Ruby on Rails Oracle adaptera izstrāde
Ruby on Rails Oracle adaptera izstrādeRuby on Rails Oracle adaptera izstrāde
Ruby on Rails Oracle adaptera izstrādeRaimonds Simanovskis
 
JRuby + Rails = Awesome Java Web Framework at Jfokus 2011
JRuby + Rails = Awesome Java Web Framework at Jfokus 2011JRuby + Rails = Awesome Java Web Framework at Jfokus 2011
JRuby + Rails = Awesome Java Web Framework at Jfokus 2011Nick Sieger
 
Intro to fog and openstack jp
Intro to fog and openstack jpIntro to fog and openstack jp
Intro to fog and openstack jpSatoshi Konno
 
PofEAA and SQLAlchemy
PofEAA and SQLAlchemyPofEAA and SQLAlchemy
PofEAA and SQLAlchemyInada Naoki
 
Learning to code for startup mvp session 3
Learning to code for startup mvp session 3Learning to code for startup mvp session 3
Learning to code for startup mvp session 3Henry S
 
Ember Data and JSON API
Ember Data and JSON APIEmber Data and JSON API
Ember Data and JSON APIyoranbe
 
Rails web api 开发
Rails web api 开发Rails web api 开发
Rails web api 开发shaokun
 
Javascript Frameworks for Well Architected, Immersive Web Apps
Javascript Frameworks for Well Architected, Immersive Web AppsJavascript Frameworks for Well Architected, Immersive Web Apps
Javascript Frameworks for Well Architected, Immersive Web Appsdnelson-cs
 
Ruby conf 2011, Create your own rails framework
Ruby conf 2011, Create your own rails frameworkRuby conf 2011, Create your own rails framework
Ruby conf 2011, Create your own rails frameworkPankaj Bhageria
 
Building web framework with Rack
Building web framework with RackBuilding web framework with Rack
Building web framework with Racksickill
 

Similar to Chef on Python and MongoDB (20)

Fast Web Applications Development with Ruby on Rails on Oracle
Fast Web Applications Development with Ruby on Rails on OracleFast Web Applications Development with Ruby on Rails on Oracle
Fast Web Applications Development with Ruby on Rails on Oracle
 
Rails on Oracle 2011
Rails on Oracle 2011Rails on Oracle 2011
Rails on Oracle 2011
 
Oracle adapters for Ruby ORMs
Oracle adapters for Ruby ORMsOracle adapters for Ruby ORMs
Oracle adapters for Ruby ORMs
 
Connecting the Worlds of Java and Ruby with JRuby
Connecting the Worlds of Java and Ruby with JRubyConnecting the Worlds of Java and Ruby with JRuby
Connecting the Worlds of Java and Ruby with JRuby
 
Using Ruby on Rails with legacy Oracle databases
Using Ruby on Rails with legacy Oracle databasesUsing Ruby on Rails with legacy Oracle databases
Using Ruby on Rails with legacy Oracle databases
 
Building an aws sdk for Perl - Granada Perl Workshop 2014
Building an aws sdk for Perl - Granada Perl Workshop 2014Building an aws sdk for Perl - Granada Perl Workshop 2014
Building an aws sdk for Perl - Granada Perl Workshop 2014
 
Extending Oracle E-Business Suite with Ruby on Rails
Extending Oracle E-Business Suite with Ruby on RailsExtending Oracle E-Business Suite with Ruby on Rails
Extending Oracle E-Business Suite with Ruby on Rails
 
SproutCore and the Future of Web Apps
SproutCore and the Future of Web AppsSproutCore and the Future of Web Apps
SproutCore and the Future of Web Apps
 
Ruby on Rails Oracle adaptera izstrāde
Ruby on Rails Oracle adaptera izstrādeRuby on Rails Oracle adaptera izstrāde
Ruby on Rails Oracle adaptera izstrāde
 
JRuby + Rails = Awesome Java Web Framework at Jfokus 2011
JRuby + Rails = Awesome Java Web Framework at Jfokus 2011JRuby + Rails = Awesome Java Web Framework at Jfokus 2011
JRuby + Rails = Awesome Java Web Framework at Jfokus 2011
 
Intro to fog and openstack jp
Intro to fog and openstack jpIntro to fog and openstack jp
Intro to fog and openstack jp
 
PofEAA and SQLAlchemy
PofEAA and SQLAlchemyPofEAA and SQLAlchemy
PofEAA and SQLAlchemy
 
Learning to code for startup mvp session 3
Learning to code for startup mvp session 3Learning to code for startup mvp session 3
Learning to code for startup mvp session 3
 
DevOps
DevOpsDevOps
DevOps
 
Ember Data and JSON API
Ember Data and JSON APIEmber Data and JSON API
Ember Data and JSON API
 
Rails web api 开发
Rails web api 开发Rails web api 开发
Rails web api 开发
 
Javascript Frameworks for Well Architected, Immersive Web Apps
Javascript Frameworks for Well Architected, Immersive Web AppsJavascript Frameworks for Well Architected, Immersive Web Apps
Javascript Frameworks for Well Architected, Immersive Web Apps
 
Ruby conf 2011, Create your own rails framework
Ruby conf 2011, Create your own rails frameworkRuby conf 2011, Create your own rails framework
Ruby conf 2011, Create your own rails framework
 
Building web framework with Rack
Building web framework with RackBuilding web framework with Rack
Building web framework with Rack
 
Chef introduction
Chef introductionChef introduction
Chef introduction
 

More from Rick Copeland

Python Functions (PyAtl Beginners Night)
Python Functions (PyAtl Beginners Night)Python Functions (PyAtl Beginners Night)
Python Functions (PyAtl Beginners Night)Rick Copeland
 
Schema Design at Scale
Schema Design at ScaleSchema Design at Scale
Schema Design at ScaleRick Copeland
 
Building Your First MongoDB Application
Building Your First MongoDB ApplicationBuilding Your First MongoDB Application
Building Your First MongoDB ApplicationRick Copeland
 
Rapid and Scalable Development with MongoDB, PyMongo, and Ming
Rapid and Scalable Development with MongoDB, PyMongo, and MingRapid and Scalable Development with MongoDB, PyMongo, and Ming
Rapid and Scalable Development with MongoDB, PyMongo, and MingRick Copeland
 
Scaling with MongoDB
Scaling with MongoDBScaling with MongoDB
Scaling with MongoDBRick Copeland
 
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.ioReal-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.ioRick Copeland
 
Rapid and Scalable Development with MongoDB, PyMongo, and Ming
Rapid and Scalable Development with MongoDB, PyMongo, and MingRapid and Scalable Development with MongoDB, PyMongo, and Ming
Rapid and Scalable Development with MongoDB, PyMongo, and MingRick Copeland
 
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQRealtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQRick Copeland
 
Rapid, Scalable Web Development with MongoDB, Ming, and Python
Rapid, Scalable Web Development with MongoDB, Ming, and PythonRapid, Scalable Web Development with MongoDB, Ming, and Python
Rapid, Scalable Web Development with MongoDB, Ming, and PythonRick Copeland
 
Allura - an Open Source MongoDB Based Document Oriented SourceForge
Allura - an Open Source MongoDB Based Document Oriented SourceForgeAllura - an Open Source MongoDB Based Document Oriented SourceForge
Allura - an Open Source MongoDB Based Document Oriented SourceForgeRick Copeland
 
MongoATL: How Sourceforge is Using MongoDB
MongoATL: How Sourceforge is Using MongoDBMongoATL: How Sourceforge is Using MongoDB
MongoATL: How Sourceforge is Using MongoDBRick Copeland
 

More from Rick Copeland (11)

Python Functions (PyAtl Beginners Night)
Python Functions (PyAtl Beginners Night)Python Functions (PyAtl Beginners Night)
Python Functions (PyAtl Beginners Night)
 
Schema Design at Scale
Schema Design at ScaleSchema Design at Scale
Schema Design at Scale
 
Building Your First MongoDB Application
Building Your First MongoDB ApplicationBuilding Your First MongoDB Application
Building Your First MongoDB Application
 
Rapid and Scalable Development with MongoDB, PyMongo, and Ming
Rapid and Scalable Development with MongoDB, PyMongo, and MingRapid and Scalable Development with MongoDB, PyMongo, and Ming
Rapid and Scalable Development with MongoDB, PyMongo, and Ming
 
Scaling with MongoDB
Scaling with MongoDBScaling with MongoDB
Scaling with MongoDB
 
Real-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.ioReal-Time Python Web: Gevent and Socket.io
Real-Time Python Web: Gevent and Socket.io
 
Rapid and Scalable Development with MongoDB, PyMongo, and Ming
Rapid and Scalable Development with MongoDB, PyMongo, and MingRapid and Scalable Development with MongoDB, PyMongo, and Ming
Rapid and Scalable Development with MongoDB, PyMongo, and Ming
 
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQRealtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
Realtime Analytics Using MongoDB, Python, Gevent, and ZeroMQ
 
Rapid, Scalable Web Development with MongoDB, Ming, and Python
Rapid, Scalable Web Development with MongoDB, Ming, and PythonRapid, Scalable Web Development with MongoDB, Ming, and Python
Rapid, Scalable Web Development with MongoDB, Ming, and Python
 
Allura - an Open Source MongoDB Based Document Oriented SourceForge
Allura - an Open Source MongoDB Based Document Oriented SourceForgeAllura - an Open Source MongoDB Based Document Oriented SourceForge
Allura - an Open Source MongoDB Based Document Oriented SourceForge
 
MongoATL: How Sourceforge is Using MongoDB
MongoATL: How Sourceforge is Using MongoDBMongoATL: How Sourceforge is Using MongoDB
MongoATL: How Sourceforge is Using MongoDB
 

Recently uploaded

Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl
 
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024Alkin Tezuysal
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIVijayananda Mohire
 
How to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxHow to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxKaustubhBhavsar6
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Libraryshyamraj55
 
Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNeo4j
 
UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2DianaGray10
 
UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4DianaGray10
 
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTSIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTxtailishbaloch
 
3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud DataEric D. Schabell
 
Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Muhammad Tiham Siddiqui
 
Extra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfExtra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfInfopole1
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingFrancesco Corti
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0DanBrown980551
 
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInOutage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInThousandEyes
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3DianaGray10
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdfThe Good Food Institute
 
Patch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 updatePatch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 updateadam112203
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveIES VE
 

Recently uploaded (20)

Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile Brochure
 
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAI
 
How to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxHow to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptx
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Library
 
Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4j
 
UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2
 
UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4UiPath Studio Web workshop series - Day 4
UiPath Studio Web workshop series - Day 4
 
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTSIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
 
3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data
 
Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)
 
Extra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfExtra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdf
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is going
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0
 
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInOutage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf
 
Patch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 updatePatch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 update
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
 

Chef on Python and MongoDB

  • 1. Rick  Copeland  @rick446   Arborian  Consulting,  LLC  
  • 2.   Infrastructure  as  Code     Resources  &  Providers     Cookbooks,  Recipes,  Clients,  and  Nodes  
  • 3. CouchDB   Solr   Solr  Indexer   Chef  Server     RabbitMQ   RabbitMQ   RabbitMQ   (API)   HTTP  REST  API   Chef  Server   knife   chef-­‐client   (Web  UI)  
  • 4. Build,  register,  and  authenticate  the  node   Synchronize  cookbooks   Build  resource  collection  (run  the  recipes  in  order)   Configure  node  (“converge”)   Run  notification  handlers  
  • 5. CouchDB   Solr   Ruby   Solr  Indexer   Chef  Server     RabbitMQ   RabbitMQ   RabbitMQ   (API)   Ruby   Ruby   HTTP  REST  API   Chef  Server   knife   chef-­‐client   (Web  UI)  
  • 6.   Chef  assumes  a  bootstrapped  node  exists     Chef  doesn’t  keep  release  notes     Code  and  infrastructure  are  versioned   differently     Solution:  Web  app  to  manage  deployments  &   generate  release  notes  
  • 7. MonQ   MongoDB   Solr  Indexer   Python   Chef  Server     Solr   (API  +  web)   Ruby   HTTP  REST  API   knife   chef-­‐client  
  • 8.   Reduce  #  of  processes  &  technologies     Don’t  know  Ruby  well     Keep  private  keys  out  of  the  system     Integrate  with  existing  authentication     Performance  
  • 9. /clients   /nodes   /data   /environments   /roles   /sandboxes   /cookbooks   /search  
  • 10.   Mostly  JSON    almost  BSON     References  to  Ruby  files  stored  separately   {      "name":  "allura-­‐0.0.1”,…      "json_class":  "Chef::CookbookVersion",   Ruby  files  stored  on  S3      "attributes":  [          {              "name":  "default.rb",              "url":  "https://s3.amazonaws.com/opscode-­‐platform-­‐production-­‐data/…                                  "checksum":  "749a3a328d6c47a32d134b336183981f",              "path":  "attributes/default.rb",              "specificity":  "default”…  
  • 11. MongoDB  Validator   role = collection( 'chef.role', doc_session, Field('_id', S.ObjectId()), Field('account_id', S.ObjectId(if_missing=None)), Shorthand  with   Field('name', str), Python  Types   Field('description', str), Field('default_attributes', str), Field('override_attributes', str), Field('run_list', [ str ] ), Field('env_run_lists', { str: [ str ]}), Index('account_id', 'name', unique=True)) Embedded   Index  Definitions   Documents  
  • 12. Models  know   where  they  live   class Role(object): def url(self): return request.relative_url( config.chef_api_root + '/roles/' + self.name) Models  can  be   def __json__(self): return dict( turned  into  dict  (to   chef_type='role', be  JSONified)   json_class='Chef::Role',! … default_attributes=loads(self.default_attributes),! …) Models  can  be   def update(self, d): self.name = d['name'] updated  from  dict   … self.default_attributes = dumps(d['default_attributes']) self.override_attributes = dumps(d['override_attributes'])! …
  • 13. class RoleSchema(JSONModelSchema):! model_class=CM.role! chef_type='role’! json_class='Chef::Role’! exclude_fields=['_id', 'account_id']
  • 14.   /foo/bar/baz    root.foo.bar.baz     Lots  of  decorators     Validation  (params/body    **kwargs)     Authorization     Rendering     HTTP  method  lookup     GET  /foo/bar    root.foo.bar._get()  
  • 15. Returns  JSON   class RolesController(RESTController): @expose(template_engine='json') def _get(self):… Only  admins  can   access   @expose(template_engine='json', acl=[CACE.admin(True), ACE.any(False)], schema=cv.RoleSchema) Convert  and   def _post(self, **kwargs):… Validate  POST   def __getattr__(self, name): return RoleController(name) Continue  dotted   lookup  
  • 16. class RoleController(RESTController): @expose(template_engine='json') def _get(self): … PUT  looks  just  like  a  POST   @expose(template_engine='json', acl=[CACE.admin(True), ACE.any(False)], schema=cv.RoleSchema) def _put(self, name, **kwargs): … @expose(template_engine='json', acl=[CACE.admin(True), ACE.any(False)]) def _delete(self): …  
  • 17. class RoleController(RESTController):! AttributeError       def __init__(self, name): HTTP  404  Not  Found   self._role = CM.Role.query.get( account_id=c.account._id, name=name) if self._role is None: raise AttributeError, name! @expose(template_engine='json') def _get(self): return self._role! Auto-­‐JSONify  
  • 18. class RoleController(RESTController):! … @expose(template_engine='json', acl=[CACE.admin(True), ACE.any(False)], Update  model   schema=cv.RoleSchema) from  kwargs   def _put(self, name, **kwargs): assert name == self._role.name self._role.update(kwargs) return self._role! @expose(template_engine='json', acl=[CACE.admin(True), ACE.any(False)]) def _delete(self): self._role.delete() return self._role
  • 19.   Don’t  trust  the  docs     Don’t  trust  the  docs   ▪  Don’t  trust  the  docs     Use  fat  models     Framework  support  for  REST  &  JSON     You’re  gonna  have  to  learn  some  Ruby  anyway     JSON  !=  BSON  
  • 20.   Better  test  coverage     Search  support  (SOLR  /  ElasticSearch)     More  testing  with  real-­‐world  deployments     Finalize  integration  with  deployment   manager  
  • 21. Rick  Copeland  @rick446   Arborian  Consulting,  LLC  
  • 22.   http://openmymind.net/2011/10/28/ CouchDB-­‐And-­‐MongoDB-­‐Performance/     MongoDB  is  14x  faster     http://www.snailinaturtleneck.com/blog/ 2009/06/29/couchdb-­‐vs-­‐mongodb-­‐ benchmark/  

Editor's Notes

  1. Keep your infrastructure definitions in source controlResources = users, packages, services,files, etc. Providers = How you build a resource on a particular platformCookbook = Collection of resources, providers, templates, libraries, and filesRecipe = Ruby script defining the configuration of some resourcesClient = anyone talking to the chef serverNode = a machine instance managed by chef
  2. “url” field Opscode stores everything on S3, I store it all in GridFS
  3. Note the ‘loads’  there IS valid JSON that is not a valid BSON doc for a collection. Dotted key names, for example.