Migrations with  Transmogrifier Rok Garbas,  http://www.garbas.si , European Plone Symposium May 2010 - Sorrento
What is transmogrifier? Rok Garbas,  http://www.garbas.si , European Plone Symposium May 2010 - Sorrento original intent w...
Simple Example of Config Rok Garbas,  http://www.garbas.si , European Plone Symposium May 2010 - Sorrento [transmogrifier]...
Create your own blueprint /1 Rok Garbas,  http://www.garbas.si , European Plone Symposium May 2010 - Sorrento class Exampl...
Create your own blueprint /2 Rok Garbas,  http://www.garbas.si , European Plone Symposium May 2010 - Sorrento from collect...
Register custom Blueprint Rok Garbas,  http://www.garbas.si , European Plone Symposium May 2010 - Sorrento <configure xmln...
Base  Blueprints  /1 Rok Garbas,  http://www.garbas.si , European Plone Symposium May 2010 - Sorrento collective.transmogr...
Base Blueprints /2 Rok Garbas,  http://www.garbas.si , European Plone Symposium May 2010 - Sorrento collective.transmogrif...
Some Useful Bluprints  Rok Garbas,  http://www.garbas.si , European Plone Symposium May 2010 - Sorrento plone.app.transmog...
Plone 2.0 > Plone 4.0 Rok Garbas,  http://www.garbas.si , European Plone Symposium May 2010 - Sorrento >100.000 content ob...
Lessons Learned Rok Garbas,  http://www.garbas.si , European Plone Symposium May 2010 - Sorrento make migration repeatable...
Plone 2.0 > Plone 4.0 Rok Garbas,  http://www.garbas.si , European Plone Symposium May 2010 - Sorrento Migrations with  Tr...
Whats missing? Wishlist!? Future! Rok Garbas,  http://www.garbas.si , European Plone Symposium May 2010 - Sorrento Migrati...
Thank you Rok Garbas,  http://www.garbas.si , European Plone Symposium May 2010 - Sorrento Migrations with  Transmogrifier...
Upcoming SlideShare
Loading in...5
×

Migrations With Transmogrifier

2,755

Published on

Published in: Technology
2 Comments
1 Like
Statistics
Notes
  • np ... here is latest slideshow i have about this ... http://garbas.github.com/migrations-with-transmogrifier/ ... and hopefully there will be a full tutorial how to approach this.

    here are some other docs i have about blueprints ai created:
    - http://garbas.github.com/collective.blueprint.jsonmigrator/
    - http://garbas.github.com/collective.blueprint.usersandgroups/

    enjoy and let me know if you need some help ...
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Thanks you!!!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
2,755
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
21
Comments
2
Likes
1
Embeds 0
No embeds

No notes for slide

Migrations With Transmogrifier

  1. 1. Migrations with Transmogrifier Rok Garbas, http://www.garbas.si , European Plone Symposium May 2010 - Sorrento
  2. 2. What is transmogrifier? Rok Garbas, http://www.garbas.si , European Plone Symposium May 2010 - Sorrento original intent was to provide pluggable way to import content into plone Migrations with Transmogrifier help us construct pipeline , from ordered sections (blueprints), where each section proccesses data in pipe can be also used for other tasks that require data proccessing tools. nice example which shows power of transmogrifier is collective.funnelweb
  3. 3. Simple Example of Config Rok Garbas, http://www.garbas.si , European Plone Symposium May 2010 - Sorrento [transmogrifier] pipeline = source transform 1 transform 2 create [source] blueprint = example. transmogrifier .source size = 5 [transform 1] blueprint = example.transmogrifier.transform from_field = text to_ field = text transform = rst condition = python:item['filename'].endswith('.rst') [transform 2] blueprint = example.transmogrifier.transform from_field = text to_ field = description transform = get_first_paragraph [create] blueprint = collective.transmogrifier.constructor Migrations with Transmogrifier
  4. 4. Create your own blueprint /1 Rok Garbas, http://www.garbas.si , European Plone Symposium May 2010 - Sorrento class ExampleSource(object): classProvides(ISectionBlueprint) implements(ISection) def __init__(self, transmogrifier, name, options, previous): self.previous = previous self.name = name def __iter__(self): for item in self.previous: yield item for record in ReadJSONFile(): yield dict(_path=normalize(record['title']), _type=”Document”, title =record['title'], text=record['text']) Migrations with Transmogrifier
  5. 5. Create your own blueprint /2 Rok Garbas, http://www.garbas.si , European Plone Symposium May 2010 - Sorrento from collective.transmogrifier.utils import Condition class ExampleTranform(object): classProvides(ISectionBlueprint) implements(ISection) def __init__(self, transmogrifier, name, options, previous): self.previous = previous self.from_ field = options['from_field'] self.to_ field = options['to_field'] self.tranform = options['tranform'] self.condition = Condition(condition, transmogrifier, name, options) def __iter__(self): for item in self.previous: if self.condition(item): if 'rst' == self.param: item[self.to_field] = TransformFromRSTToHTML(item[self.from_field ]) elif 'get_first_paragraph' == self.param: item[self.to_field] = GetFirstParagraph(item[self.to_field]) yield item Migrations with Transmogrifier
  6. 6. Register custom Blueprint Rok Garbas, http://www.garbas.si , European Plone Symposium May 2010 - Sorrento <configure xmlns=&quot;http://namespaces.zope.org/zope&quot; xmlns:tg=&quot;http://namespaces.plone.org/transmogrifier&quot; i18n_domain=&quot;example.transmogrifier&quot;> <tg:registerConfig name=&quot;exampleconfig&quot; title=&quot;Example&quot; description=&quot;Example pipeline config&quot; configuration=&quot; example .cfg&quot; /> <utility component=&quot;.source.ExampleSource&quot; name=&quot;example.transmogrifier.source&quot; /> <utility component=&quot;.transform.ExampleTransform&quot; name=&quot;example.transmogrifier.transform&quot; /> </configure> Migrations with Transmogrifier
  7. 7. Base Blueprints /1 Rok Garbas, http://www.garbas.si , European Plone Symposium May 2010 - Sorrento collective.transmogrifier.sections.constructor Migrations with Transmogrifier collective.transmogrifier.sections.savepoint contstructs plone content (_type, _path) commits a savepoint which as result free up memory collective.transmogrifier.sections.savepoint commits a savepoint which as result free up memory collective.transmogrifier.sections.csvsource reads csv file and puts lines of it in pipeline collective.transmogrifier.sections.splitter splits pipeline, which lets you proccess data flow separatly
  8. 8. Base Blueprints /2 Rok Garbas, http://www.garbas.si , European Plone Symposium May 2010 - Sorrento collective.transmogrifier.sections.manipulator Migrations with Transmogrifier rename, copy, delete data in pipeline collective.transmogrifier.sections.condition Lets you discard data from pipeline collective.transmogrifier.sections.inserter Lets you create new fields for data (from other fields) collective.transmogrifier.sections.codec Changes encoding for data in pipeline collective.transmogrifier.sections.constructor Creates parent folders when item needs them
  9. 9. Some Useful Bluprints Rok Garbas, http://www.garbas.si , European Plone Symposium May 2010 - Sorrento plone.app.transmogrifier Migrations with Transmogrifier collective.funellweb transmogrify.siteanalyser transmogrify.filesystem transmogrify.ploneremote transmogrify.webcrawler transmogrify.htmltesting transmogrify.htmlcontentextractor transmogrify.pathsorter quintagroup.transmogrifier collective.blueprint.dancing transmogrify.sqlalchemy collective.blueprint.translationlinker
  10. 10. Plone 2.0 > Plone 4.0 Rok Garbas, http://www.garbas.si , European Plone Symposium May 2010 - Sorrento >100.000 content objects Migrations with Transmogrifier >40GB in size many custom products usual migration/upgrade failed
  11. 11. Lessons Learned Rok Garbas, http://www.garbas.si , European Plone Symposium May 2010 - Sorrento make migration repeatable (eg: using GenericSetup upgrade steps) Migrations with Transmogrifier migrate parts of page separatly split migration into export and import part monitor and time migration don't complicate in making blueprints extra reusable
  12. 12. Plone 2.0 > Plone 4.0 Rok Garbas, http://www.garbas.si , European Plone Symposium May 2010 - Sorrento Migrations with Transmogrifier <10GB of “Data.fs” (RelStorage) + 20GB of blobs less custom code, better re-usage of Plone's default features using less resources, noticably faster thanks to Plone4 happy client
  13. 13. Whats missing? Wishlist!? Future! Rok Garbas, http://www.garbas.si , European Plone Symposium May 2010 - Sorrento Migrations with Transmogrifier Better support for monitoring what goes inside pipeline Web GUI for base blueprints Making it not dependable on plone
  14. 14. Thank you Rok Garbas, http://www.garbas.si , European Plone Symposium May 2010 - Sorrento Migrations with Transmogrifier Rok Garbas, rok@garbas.si http://www.garbas.si http://www.twitter.com/garbas
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×