Transmogrifier: Migrating to Plone with less pain

4,168 views
3,879 views

Published on

Transmogrifier is a migration framework written by Martijn Pieters, Jarn, that makes migrating to Plone fun again. This talk gives an introduction to Transmogrifier, an explanation of how it works and some hints for usage.

Published in: Technology, Education
1 Comment
6 Likes
Statistics
Notes
No Downloads
Views
Total views
4,168
On SlideShare
0
From Embeds
0
Number of Embeds
26
Actions
Shares
0
Downloads
48
Comments
1
Likes
6
Embeds 0
No embeds

No notes for slide

Transmogrifier: Migrating to Plone with less pain

  1. 1. Transmogrifier Lennart Regebro Plone Conference 2009, Budapest
  2. 2. collective.transmogrifier Created by Martijn Pieters, Jarn in 2008 Released in version 1.0 August 2009
  3. 3. collective.transmogrifier Import/migration framework Both from Plone 2 and from other websites or data
  4. 4. Super-simple architecture You set up a pipeline of sections Each section send items to the next section. Sections do something with the items (or not)
  5. 5. What are you talking about?
  6. 6. A simple example 1. A section that reads a CSV file with a list of events and creates items based on that data 2. A section that will generate some plone-specific data for each item, like path and type. 3. A section that creates the Plone event 4. A section that updates the Archetypes fields 5. A section that publishes the event
  7. 7. The CSV file title,startDate,endDate Plone Conference,2009-10-28 09:00,2009-10-30 18:00 Christmas,2009-12-24 00:00,2009-12-26:23:59
  8. 8. Setting it up 1. Define a pipeline in a .cfg file 2. Register it as a name in ZCML 3. Call Transmogrifier from Python
  9. 9. The configuration file [transmogrifier] pipeline = section1 section2 [section1] blueprint = name.of.the.blueprint size = 5 [section2] blueprint = another.blueprint
  10. 10. ZCML <configure xmlns:transmogrifier= "http://namespaces.plone.org/transmogrifier"> <transmogrifier:registerConfig name="Zap event import" title="Import of events" description="" configuration="transmogrifier.cfg" /> </configure>
  11. 11. setuphandlers.py from collective.transmogrifier.transmogrifier import Transmogrifier def setupVarious(context): if context.readDataFile('zap_various.txt') is None: return transmogrifier = Transmogrifier(context.getSite()) transmogrifier('Zap event import')
  12. 12. What is a generator def my_generator(): i=1 while i < 1000: yield i i=i*2 for x in my_generator(): print x
  13. 13. Never ending story def my_generator(): i=1 while True: yield i i=i*2 for x in my_generator(): print x
  14. 14. EventSource class EventSource(object): def __iter__(self): for event is csv.DictReader(open(self.options['csvfile'])) yield event for item in self.previous: yield item
  15. 15. EventSource class EventSource(object): def __iter__(self): for event is csv.DictReader(open(self.options['csvfile'])) yield event for item in self.previous: yield item
  16. 16. EventSource class EventSource(object): def __iter__(self): for event is csv.DictReader(open(self.options['csvfile'])) yield event for item in self.previous: yield item
  17. 17. The item { 'title': 'Plone Conference', 'startDate': '2009-10-27 09:00', 'endDate': '2009-10-30 18:00', }
  18. 18. EventUpdater class EventUpdater(object): def __iter__(self): for item in self.previous: id = item['title'].lower().replace(' ', '-') item['_path'] = '/events/'+id item['_type'] = 'Event' item['_transitions'] = ('publish',) yield item
  19. 19. The item { '_type': 'Event', '_path': '/events/plone-conference', '_transitions': ('publish,), 'title': 'Plone Conference', 'startDate': '2009-10-27 09:00', 'endDate': '2009-10-30 18:00', }
  20. 20. Registering the section <utility component=".eventsource.EventSource" name="zap.eventsource" /> <utility component=".eventsource.EventUpdater" name="zap.eventupdater" />
  21. 21. The configuration file [transmogrifier] pipeline = eventsource eventupdater [eventsource] blueprint = zap.eventsource csvfile = /path/to/events.csv [eventupdater] blueprint = zap.eventupdater
  22. 22. The configuration file cont. [constructor] blueprint = collective.transmogrifier.sections.constructor [schemaupdater] blueprint = plone.app.transmogrifier.atschemaupdater [workflow] blueprint = plone.app.transmogrifier.workflowupdater
  23. 23. The configuration file cont. [transmogrifier] pipeline = eventsource eventupdater constructor schemaupdater workflow
  24. 24. Sections Create items Modify items Drop items Split the pipeline in two Construct content Modify content
  25. 25. Migrating from another CMS Similar to the simple example but using SQL instead of CSV
  26. 26. Migration from Plone 2 1. Write an export script 2. Write an import section
  27. 27. Migrate from Plone 2 importsource constructor criterionadder schemaupdater browserdefault workflow savepoint
  28. 28. Migrating from HTML 1. Get HTML from web 2. Extract fields like title, publish date, etc. 3. Make all urls either relative or into UID's
  29. 29. From small to huge Largest migration: 28 sections 527 lines of configuration 16 custom sections
  30. 30. Caveat Emptor Don't migrate from HTML on disk
  31. 31. Caveat Emptor Newly created objects are not indexed.
  32. 32. Caveat Emptor Debugging is backwards
  33. 33. Lennart Regebro regebro@gmail.com IRC nick: regebro

×