0
Clayton Parker, Senior Web Developer
Laying Pipe with
Transmogrifier
PLONE CONFERENCE 2010
Thursday, October 28, 2010
PLONE CONFERENCE 2010Who Am I?
• claytron
• Python dev since 2003
• Plone Core Committer
• Foundation Member
Thursday, Oct...
PLONE CONFERENCE 2010What Will We
Learn?
• What is Transmogrifier
• How to use it in Plone
• How to package a migration
Thu...
PLONE CONFERENCE 2010Migrations
• One off scripts
• In multiple places
• No re-use
Thursday, October 28, 2010
PLONE CONFERENCE 2010Transmogrifier
• A framework for migrations
• Re-usable parts
Thursday, October 28, 2010
PLONE CONFERENCE 2010Basics
• Pipeline
• Blueprints
• Sources
Thursday, October 28, 2010
PLONE CONFERENCE 2010Pipeline[transmogrifier]
pipeline =
blog
title
text
text-mimetype
date
type
transitions
urlnormalizer...
PLONE CONFERENCE 2010Blueprints
• Python code
• Packages
Thursday, October 28, 2010
PLONE CONFERENCE 2010Sources
• A blueprint
• First item in your pipeline
Thursday, October 28, 2010
PLONE CONFERENCE 2010configure.zcml
<configure
xmlns="http://namespaces.zope.org/zope">
<include package="collective.transm...
PLONE CONFERENCE 2010blueprint.py
from zope.interface import implements
from zope.interface import classProvides
from coll...
PLONE CONFERENCE 2010Items
• Each item is a mapping
• Keys are fields
• Keys with a leading underscore are controllers
Thur...
PLONE CONFERENCE 2010Pipeline Settings
• Use annotations
• Globally imported settings
Thursday, October 28, 2010
PLONE CONFERENCE 2010GenericSetup
• Make migration part of your release
• Ability to package migrations
Thursday, October ...
PLONE CONFERENCE 2010Package Layout
my.migration
!"" __init__.py
!"" configure.zcml
!"" import
#   !"" __init__.py
#   !""...
PLONE CONFERENCE 2010import/configure.zcml
<configure
xmlns="http://namespaces.zope.org/zope"
xmlns:transmogrifier="http://...
PLONE CONFERENCE 2010transmogrifier.txt
my_migration
Thursday, October 28, 2010
PLONE CONFERENCE 2010Example Pipeline
[transmogrifier]
pipeline =
csv_file
constructor
schemaupdater
[csv_file]
blueprint ...
PLONE CONFERENCE 2010my_items.csv
_path,_type,title,description
/folder1,Folder,First Folder,This is folder One
/folder2,F...
PLONE CONFERENCE 2010The Result
Thursday, October 28, 2010
PLONE CONFERENCE 2010Stock Blueprints
• Constructor section
• Folders section
• Codec section
• Inserter section
• Conditi...
PLONE CONFERENCE 2010Useful Sources and
Blueprints
• plone.app.transmogrifier
• transmogrify.filesystem
• transmogrify.sqlal...
PLONE CONFERENCE 2010
plone.app.transmogrifier
• ATSchema updater section
• UID updater section
• Workflow updater section
•...
PLONE CONFERENCE 2010Links
• collective.transmogrifier
http://pypi.python.org/pypi/collective.transmogrifier/
• plone.app.tr...
Check out
sixfeetup.com/demos
Thursday, October 28, 2010
Upcoming SlideShare
Loading in...5
×

Laying Pipe with Transmogrifier

2,240

Published on

Transmogrifier is a tool that gives you the ability to create a configurable pipeline to import and export from various systems. Each section of the pipe can be turned into a re-usable blueprint that can be given back to the community. This gives us the ability to quickly create migration paths for existing systems and maintain them as Python packages.

Published in: Business, Technology
1 Comment
1 Like
Statistics
Notes
No Downloads
Views
Total Views
2,240
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
40
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Laying Pipe with Transmogrifier"

  1. 1. Clayton Parker, Senior Web Developer Laying Pipe with Transmogrifier PLONE CONFERENCE 2010 Thursday, October 28, 2010
  2. 2. PLONE CONFERENCE 2010Who Am I? • claytron • Python dev since 2003 • Plone Core Committer • Foundation Member Thursday, October 28, 2010
  3. 3. PLONE CONFERENCE 2010What Will We Learn? • What is Transmogrifier • How to use it in Plone • How to package a migration Thursday, October 28, 2010
  4. 4. PLONE CONFERENCE 2010Migrations • One off scripts • In multiple places • No re-use Thursday, October 28, 2010
  5. 5. PLONE CONFERENCE 2010Transmogrifier • A framework for migrations • Re-usable parts Thursday, October 28, 2010
  6. 6. PLONE CONFERENCE 2010Basics • Pipeline • Blueprints • Sources Thursday, October 28, 2010
  7. 7. PLONE CONFERENCE 2010Pipeline[transmogrifier] pipeline = blog title text text-mimetype date type transitions urlnormalizer path constructor schemaupdate comment-insert comments workflow reindexobject [blog] blueprint = transmogrify.blogger filename = acost.policy.import:export.xml [title] blueprint = collective.transmogrifier.sections.manipulator keys = _transmogrify.blogger.title destination = string:title delete = ${title:keys} Thursday, October 28, 2010
  8. 8. PLONE CONFERENCE 2010Blueprints • Python code • Packages Thursday, October 28, 2010
  9. 9. PLONE CONFERENCE 2010Sources • A blueprint • First item in your pipeline Thursday, October 28, 2010
  10. 10. PLONE CONFERENCE 2010configure.zcml <configure xmlns="http://namespaces.zope.org/zope"> <include package="collective.transmogrifier" /> <utility component=".blueprint.MySection" name="my.section" /> </configure> Thursday, October 28, 2010
  11. 11. PLONE CONFERENCE 2010blueprint.py from zope.interface import implements from zope.interface import classProvides from collective.transmogrifier.interfaces import ISection from collective.transmogrifier.interfaces import ISectionBlueprint class MySection(object): """A blueprint for importing something into plone """ classProvides(ISectionBlueprint) implements(ISection) def __iter___(self): # add any other sources into the stream for item in self.previous: # Do something with each item # Pass it on to the next blueprint yield item Thursday, October 28, 2010
  12. 12. PLONE CONFERENCE 2010Items • Each item is a mapping • Keys are fields • Keys with a leading underscore are controllers Thursday, October 28, 2010
  13. 13. PLONE CONFERENCE 2010Pipeline Settings • Use annotations • Globally imported settings Thursday, October 28, 2010
  14. 14. PLONE CONFERENCE 2010GenericSetup • Make migration part of your release • Ability to package migrations Thursday, October 28, 2010
  15. 15. PLONE CONFERENCE 2010Package Layout my.migration !"" __init__.py !"" configure.zcml !"" import #   !"" __init__.py #   !"" configure.zcml #   !"" my_items.csv #   %"" my_migration.cfg %"" profiles %"" default %"" transmogrifier.txt Thursday, October 28, 2010
  16. 16. PLONE CONFERENCE 2010import/configure.zcml <configure xmlns="http://namespaces.zope.org/zope" xmlns:transmogrifier="http://namespaces.plone.org/transmogrifier"> <include package="collective.transmogrifier"/> <include package="collective.transmogrifier" file="meta.zcml"/> <transmogrifier:registerConfig name="my_migration" title="Import items based on CSV file" description="" configuration="my_migration.cfg" /> </configure> Thursday, October 28, 2010
  17. 17. PLONE CONFERENCE 2010transmogrifier.txt my_migration Thursday, October 28, 2010
  18. 18. PLONE CONFERENCE 2010Example Pipeline [transmogrifier] pipeline = csv_file constructor schemaupdater [csv_file] blueprint = collective.transmogrifier.sections.csvsource filename = my.migration.import:my_items.csv [constructor] blueprint = collective.transmogrifier.sections.constructor [schemaupdater] blueprint = plone.app.transmogrifier.atschemaupdater Thursday, October 28, 2010
  19. 19. PLONE CONFERENCE 2010my_items.csv _path,_type,title,description /folder1,Folder,First Folder,This is folder One /folder2,Folder,Second Folder,This is folder Two /folder1/foo,Document,One Foo,A document named foo /folder2/foo,Document,Two Foo,Another doc named foo Thursday, October 28, 2010
  20. 20. PLONE CONFERENCE 2010The Result Thursday, October 28, 2010
  21. 21. PLONE CONFERENCE 2010Stock Blueprints • Constructor section • Folders section • Codec section • Inserter section • Condition section • Manipulator section • Splitter section • Savepoint section • CSV source section Thursday, October 28, 2010
  22. 22. PLONE CONFERENCE 2010Useful Sources and Blueprints • plone.app.transmogrifier • transmogrify.filesystem • transmogrify.sqlalchemy • transmogrify.webcrawler • wordpress / zine / blogger Thursday, October 28, 2010
  23. 23. PLONE CONFERENCE 2010 plone.app.transmogrifier • ATSchema updater section • UID updater section • Workflow updater section • Browser default section • Criterion adder section • Portal Transforms section • URL Normalizer section • Mime encapsulator section • Indexing section Thursday, October 28, 2010
  24. 24. PLONE CONFERENCE 2010Links • collective.transmogrifier http://pypi.python.org/pypi/collective.transmogrifier/ • plone.app.transmogrifier http://pypi.python.org/pypi/plone.app.transmogrifier/ Thursday, October 28, 2010
  25. 25. Check out sixfeetup.com/demos Thursday, October 28, 2010
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×