Content Mirror

Plone Content Serialization to a
Structured/Relational Database
 Who am I

 Kapil Thangavelu

     Lives in Washington, DC

        Works with Python
    
            Uses zope3, sqlalch...
What is this talk about  

  Rethinking How to Use Plone
  Data Portability

  Developer Productivity
What its not about

Using relational database with Plone
Product vs. Framework

2003, First Plone conference NOLA, paul everitt posed the
question.
The Product Won

Plone does alot of out the box, with alot of knobs and
customization possible through the web.
More Complexity

 - Features are great if your using them.
 - Size Matters
 - Orthogonality Maters
Complexity Measured

quot;I have measured out my life in coffee spoonsquot;, The Love
song of J Alrefd Prufrock, TS. Eliot...
Simplicity is Good
- Pay for What You Use     - Developer Productivity
                           and the Learning Curve
-...
Separating Deployment

2am December 6th 2002, Amsterdam, infrae zope3 sprint
Static Deployment

CMFDeployment - Flexible Static Deployment, Filtering,
transforms, deployment skins, rendering, url
res...
Data Deployment

Entransit, provides data deployment, with features like
rollback in a workspace based site structure.

.....
Too Complex

Still too complex, needs to work out of the box.

operation and setup need to be dirt simple regardless of th...
ContentMirror

 quot;A facility for mirroring the content of a Plone site into a
structured external datastore.quot;
 
 - ...
ContentMirror

Supports Plone 2.5 - Plone   Work out of the Box
3.1
                             Strong Test Coverage
Supp...
Bootstrapping

 - Load the Archetypes
 - Transform to Database Schema
 - Create Peer Class
Schema Transformation
transform archetypes
schemas into relational tables
 
Using SQLAlchemy
 
Uses a default schema (
dub...
Peer Based
Each content type's class has a corresponding,
database persisted class (orm mapped), with
corresponding attrib...
Event Based

In order to serialize content as changes are happening in
the CMS, we integrate into the application server's...
Files

Don't copy files without need
 
Stored in the database by default
Containment

Adjacency List Model 
 
Modified Preorder Tree Traversal
Lightweight Mapper for Site Maps (use eager join opt...
Relations

Archetypes              Content Mirror

Bidirectional           Relations Table

Generic Between Types   Doesn'...
Installation   
<configure xmlns=quot;http://namespaces.zope.org/zopequot;
           xmlns:db=quot;http://namespaces.obje...
Finishing Installation

# generate a schema
./bin/zopectl run Products/ContentMirror/ddl.py postgres
 
# create the databa...
Playing with Content Demo

 SQL Command Line

 Django Admin Interface
Resources

Source Code, Issue Tracker, Documentation
http://code.google.com/p/contentmirror
 
Mailing List
http://groups.g...
Questions   


   ?
Upcoming SlideShare
Loading in …5
×

Content Mirror

1,999 views
1,931 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,999
On SlideShare
0
From Embeds
0
Number of Embeds
128
Actions
Shares
0
Downloads
14
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Content Mirror

  1. 1. Content Mirror Plone Content Serialization to a Structured/Relational Database
  2. 2.  Who am I Kapil Thangavelu     Lives in Washington, DC        Works with Python                Uses zope3, sqlalchemy, repoze, z3c
  3. 3. What is this talk about   Rethinking How to Use Plone Data Portability Developer Productivity
  4. 4. What its not about Using relational database with Plone
  5. 5. Product vs. Framework 2003, First Plone conference NOLA, paul everitt posed the question.
  6. 6. The Product Won Plone does alot of out the box, with alot of knobs and customization possible through the web.
  7. 7. More Complexity - Features are great if your using them. - Size Matters - Orthogonality Maters
  8. 8. Complexity Measured quot;I have measured out my life in coffee spoonsquot;, The Love song of J Alrefd Prufrock, TS. Eliot (1m x 40r x 5d x 50w )/60 = 166h The Plone Tax, aka time spent waiting on restarts
  9. 9. Simplicity is Good - Pay for What You Use - Developer Productivity and the Learning Curve - We can build fast, Matters  interactive applications   - Simple things have a better chance of being correct
  10. 10. Separating Deployment 2am December 6th 2002, Amsterdam, infrae zope3 sprint
  11. 11. Static Deployment CMFDeployment - Flexible Static Deployment, Filtering, transforms, deployment skins, rendering, url resolution. http://plone.org/products/cmfdeployment From Plone 1.0 - Plone 3.1
  12. 12. Data Deployment Entransit, provides data deployment, with features like rollback in a workspace based site structure. .. requires particular site structure/layout according to workgroups, provides advanced features like rollback of a deployment.
  13. 13. Too Complex Still too complex, needs to work out of the box. operation and setup need to be dirt simple regardless of the internal complexity (like a product ;-)
  14. 14. ContentMirror  quot;A facility for mirroring the content of a Plone site into a structured external datastore.quot;    - automated, transparent, synchronous serialization.     - default source, Archetypes Content    - default target, Relational Databases
  15. 15. ContentMirror Supports Plone 2.5 - Plone Work out of the Box 3.1   Strong Test Coverage Support different Databases ( tested with Easy to Extend Postgresql & Mysql )
  16. 16. Bootstrapping  - Load the Archetypes  - Transform to Database Schema  - Create Peer Class
  17. 17. Schema Transformation transform archetypes schemas into relational tables   Using SQLAlchemy   Uses a default schema ( dublin core, relations, files, identity )   each type gets its own table with common fields factored out
  18. 18. Peer Based Each content type's class has a corresponding, database persisted class (orm mapped), with corresponding attributes. The peer/domain class can be specified or autogenerated (default).
  19. 19. Event Based In order to serialize content as changes are happening in the CMS, we integrate into the application server's event stream and subscribe to content events. The subscribers in turn create sync operations via an operation factory.
  20. 20. Files Don't copy files without need   Stored in the database by default
  21. 21. Containment Adjacency List Model    Modified Preorder Tree Traversal Lightweight Mapper for Site Maps (use eager join option )
  22. 22. Relations Archetypes Content Mirror Bidirectional Relations Table Generic Between Types Doesn't support custom attributes without Custom Attributes customization.
  23. 23. Installation    <configure xmlns=quot;http://namespaces.zope.org/zopequot;            xmlns:db=quot;http://namespaces.objectrealms.net/rdbquot;            xmlns:ore=quot;http://namespaces.objectrealms. net/mirrorquot;>     <!-- setup a database connection -->    <db:engine url=quot;postgres://localhost/contentmirrorquot;              name=quot;mirror-dbquot;              echo=quot;Truequot;/>    <!-- associate the connection to the db structures -->    <db:bind engine=quot;mirror-dbquot;            metadata=quot;ore.contentmirror.schema.metadataquot; /> </configure>
  24. 24. Finishing Installation # generate a schema ./bin/zopectl run Products/ContentMirror/ddl.py postgres   # create the database ./bin/createdb mirror-example   # load up the schema ./bin/psql mirror-example < db.sql   # bulk load the existing content ./bin/zopectl run Products/ContentMirror/bulk.py portal
  25. 25. Playing with Content Demo  SQL Command Line  Django Admin Interface
  26. 26. Resources Source Code, Issue Tracker, Documentation http://code.google.com/p/contentmirror   Mailing List http://groups.google.com/group/contentmirror   99% unit test coverage / 48 tests / Doctests
  27. 27. Questions    ?

×