This document discusses Content Mirror, a tool that mirrors Plone content into a structured database. It serializes Plone content as it changes and stores it in a relational database for improved developer productivity and simpler deployment. It transforms Archetypes schemas into database tables, uses events to synchronize changes, and has options for installation, bulk loading content, and interacting with the mirrored data.

1. Content Mirror
Plone Content Serialization to a
Structured/Relational Database

2. Who am I
Kapil Thangavelu
Lives in Washington, DC
Works with Python
Uses zope3, sqlalchemy, repoze, z3c

3. What is this talk about
Rethinking How to Use Plone
Data Portability
Developer Productivity

4. What its not about
Using relational database with Plone

5. Product vs. Framework
2003, First Plone conference NOLA, paul everitt posed the
question.

6. The Product Won
Plone does alot of out the box, with alot of knobs and
customization possible through the web.

7. More Complexity
- Features are great if your using them.
- Size Matters
- Orthogonality Maters

8. Complexity Measured
quot;I have measured out my life in coffee spoonsquot;, The Love
song of J Alrefd Prufrock, TS. Eliot
(1m x 40r x 5d x 50w )/60 = 166h
The Plone Tax, aka time spent waiting on restarts

9. Simplicity is Good
- Pay for What You Use - Developer Productivity
and the Learning Curve
- We can build fast, Matters
interactive applications
- Simple things have a
better chance of being
correct

10. Separating Deployment
2am December 6th 2002, Amsterdam, infrae zope3 sprint

11. Static Deployment
CMFDeployment - Flexible Static Deployment, Filtering,
transforms, deployment skins, rendering, url
resolution.
http://plone.org/products/cmfdeployment
From Plone 1.0 - Plone 3.1

12. Data Deployment
Entransit, provides data deployment, with features like
rollback in a workspace based site structure.
.. requires particular site structure/layout according to
workgroups, provides advanced features like rollback of a
deployment.

13. Too Complex
Still too complex, needs to work out of the box.
operation and setup need to be dirt simple regardless of the
internal complexity (like a product ;-)

14. ContentMirror
quot;A facility for mirroring the content of a Plone site into a
structured external datastore.quot;
- automated, transparent, synchronous serialization.
- default source, Archetypes Content
- default target, Relational Databases

15. ContentMirror
Supports Plone 2.5 - Plone Work out of the Box
3.1
Strong Test Coverage
Support different
Databases ( tested with Easy to Extend
Postgresql & Mysql )

16. Bootstrapping
- Load the Archetypes
- Transform to Database Schema
- Create Peer Class

17. Schema Transformation
transform archetypes
schemas into relational tables
Using SQLAlchemy
Uses a default schema (
dublin core, relations, files,
identity )
each type gets its own table
with common fields factored
out

18. Peer Based
Each content type's class has a corresponding,
database persisted class (orm mapped), with
corresponding attributes. The peer/domain class can be
specified or autogenerated (default).

19. Event Based
In order to serialize content as changes are happening in
the CMS, we integrate into the application server's event
stream and subscribe to content events. The subscribers in
turn create sync operations via an operation factory.

20. Files
Don't copy files without need
Stored in the database by default

21. Containment
Adjacency List Model
Modified Preorder Tree Traversal
Lightweight Mapper for Site Maps (use eager join option )

22. Relations
Archetypes Content Mirror
Bidirectional Relations Table
Generic Between Types Doesn't support custom
attributes without
Custom Attributes customization.

23. Installation
<configure xmlns=quot;http://namespaces.zope.org/zopequot;
xmlns:db=quot;http://namespaces.objectrealms.net/rdbquot;
xmlns:ore=quot;http://namespaces.objectrealms.
net/mirrorquot;>
<!-- setup a database connection -->
<db:engine url=quot;postgres://localhost/contentmirrorquot;
name=quot;mirror-dbquot;
echo=quot;Truequot;/>
<!-- associate the connection to the db structures -->
<db:bind engine=quot;mirror-dbquot;
metadata=quot;ore.contentmirror.schema.metadataquot; />
</configure>

24. Finishing Installation
# generate a schema
./bin/zopectl run Products/ContentMirror/ddl.py postgres
# create the database
./bin/createdb mirror-example
# load up the schema
./bin/psql mirror-example < db.sql
# bulk load the existing content
./bin/zopectl run Products/ContentMirror/bulk.py portal

25. Playing with Content Demo
SQL Command Line
Django Admin Interface

26. Resources
Source Code, Issue Tracker, Documentation
http://code.google.com/p/contentmirror
Mailing List
http://groups.google.com/group/contentmirror
99% unit test coverage / 48 tests / Doctests

27. Questions
?