• Like

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Laying Pipe with Transmogrifier

  • 2,176 views
Uploaded on

Transmogrifier is a tool that gives you the ability to create a configurable pipeline to import and export from various systems. Each section of the pipe can be turned into a re-usable blueprint that …

Transmogrifier is a tool that gives you the ability to create a configurable pipeline to import and export from various systems. Each section of the pipe can be turned into a re-usable blueprint that can be given back to the community. This gives us the ability to quickly create migration paths for existing systems and maintain them as Python packages.

More in: Business , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
2,176
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
39
Comments
1
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Laying Pipe with Transmogrifier Clayton Parker, Senior Web Developer PLONE CONFERENCE 2010 Thursday, October 28, 2010
  • 2. Who Am I? PLONE CONFERENCE 2010 • claytron • Python dev since 2003 • Plone Core Committer • Foundation Member Thursday, October 28, 2010
  • 3. What Will We PLONE CONFERENCE 2010 Learn? • What is Transmogrifier • How to use it in Plone • How to package a migration Thursday, October 28, 2010
  • 4. Migrations PLONE CONFERENCE 2010 • One off scripts • In multiple places • No re-use Thursday, October 28, 2010
  • 5. Transmogrifier PLONE CONFERENCE 2010 • A framework for migrations • Re-usable parts Thursday, October 28, 2010
  • 6. Basics PLONE CONFERENCE 2010 • Pipeline • Blueprints • Sources Thursday, October 28, 2010
  • 7. Pipeline [transmogrifier] PLONE CONFERENCE 2010 pipeline = blog title text text-mimetype date type transitions urlnormalizer path constructor schemaupdate comment-insert comments workflow reindexobject [blog] blueprint = transmogrify.blogger filename = acost.policy.import:export.xml [title] blueprint = collective.transmogrifier.sections.manipulator keys = _transmogrify.blogger.title destination = string:title delete = ${title:keys} Thursday, October 28, 2010
  • 8. Blueprints PLONE CONFERENCE 2010 • Python code • Packages Thursday, October 28, 2010
  • 9. Sources PLONE CONFERENCE 2010 • A blueprint • First item in your pipeline Thursday, October 28, 2010
  • 10. configure.zcml PLONE CONFERENCE 2010 <configure xmlns="http://namespaces.zope.org/zope"> <include package="collective.transmogrifier" /> <utility component=".blueprint.MySection" name="my.section" /> </configure> Thursday, October 28, 2010
  • 11. blueprint.py PLONE CONFERENCE 2010 from zope.interface import implements from zope.interface import classProvides from collective.transmogrifier.interfaces import ISection from collective.transmogrifier.interfaces import ISectionBlueprint class MySection(object): """A blueprint for importing something into plone """ classProvides(ISectionBlueprint) implements(ISection) def __iter___(self): # add any other sources into the stream for item in self.previous: # Do something with each item # Pass it on to the next blueprint yield item Thursday, October 28, 2010
  • 12. Items PLONE CONFERENCE 2010 • Each item is a mapping • Keys are fields • Keys with a leading underscore are controllers Thursday, October 28, 2010
  • 13. Pipeline Settings PLONE CONFERENCE 2010 • Use annotations • Globally imported settings Thursday, October 28, 2010
  • 14. GenericSetup PLONE CONFERENCE 2010 • Make migration part of your release • Ability to package migrations Thursday, October 28, 2010
  • 15. Package Layout PLONE CONFERENCE 2010 my.migration !"" __init__.py !"" configure.zcml !"" import #   !"" __init__.py #   !"" configure.zcml #   !"" my_items.csv #   %"" my_migration.cfg %"" profiles %"" default %"" transmogrifier.txt Thursday, October 28, 2010
  • 16. import/configure.zcml PLONE CONFERENCE 2010 <configure xmlns="http://namespaces.zope.org/zope" xmlns:transmogrifier="http://namespaces.plone.org/transmogrifier"> <include package="collective.transmogrifier"/> <include package="collective.transmogrifier" file="meta.zcml"/> <transmogrifier:registerConfig name="my_migration" title="Import items based on CSV file" description="" configuration="my_migration.cfg" /> </configure> Thursday, October 28, 2010
  • 17. transmogrifier.txt PLONE CONFERENCE 2010 my_migration Thursday, October 28, 2010
  • 18. Example Pipeline PLONE CONFERENCE 2010 [transmogrifier] pipeline = csv_file constructor schemaupdater [csv_file] blueprint = collective.transmogrifier.sections.csvsource filename = my.migration.import:my_items.csv [constructor] blueprint = collective.transmogrifier.sections.constructor [schemaupdater] blueprint = plone.app.transmogrifier.atschemaupdater Thursday, October 28, 2010
  • 19. my_items.csv PLONE CONFERENCE 2010 _path,_type,title,description /folder1,Folder,First Folder,This is folder One /folder2,Folder,Second Folder,This is folder Two /folder1/foo,Document,One Foo,A document named foo /folder2/foo,Document,Two Foo,Another doc named foo Thursday, October 28, 2010
  • 20. The Result PLONE CONFERENCE 2010 Thursday, October 28, 2010
  • 21. Stock Blueprints PLONE CONFERENCE 2010 • Constructor section • Folders section • Codec section • Inserter section • Condition section • Manipulator section • Splitter section • Savepoint section • CSV source section Thursday, October 28, 2010
  • 22. Useful Sources and PLONE CONFERENCE 2010 Blueprints • plone.app.transmogrifier • transmogrify.filesystem • transmogrify.sqlalchemy • transmogrify.webcrawler • wordpress / zine / blogger Thursday, October 28, 2010
  • 23. plone.app.transmogrifier PLONE CONFERENCE 2010 • ATSchema updater section • UID updater section • Workflow updater section • Browser default section • Criterion adder section • Portal Transforms section • URL Normalizer section • Mime encapsulator section • Indexing section Thursday, October 28, 2010
  • 24. Links PLONE CONFERENCE 2010 • collective.transmogrifier http://pypi.python.org/pypi/collective.transmogrifier/ • plone.app.transmogrifier http://pypi.python.org/pypi/plone.app.transmogrifier/ Thursday, October 28, 2010
  • 25. Check out .co m/d emos s ixfeetup Thursday, October 28, 2010