Transmogrifier: Migrating to Plone with less pain

Transmogrifier Lennart Regebro Plone Conference 2009, Budapest

collective.transmogrifier Created by Martijn Pieters, Jarn in 2008 Released in version 1.0 August 2009

collective.transmogrifier Import/migration framework Both from Plone 2 and from other websites or data

Super-simple architecture You set up a pipeline of sections Each section send items to the next section. Sections do something with the items (or not)

What are you talking about?

A simple example 1. A section that reads a CSV file with a list of events and creates items based on that data 2. A section that will generate some plone-specific data for each item, like path and type. 3. A section that creates the Plone event 4. A section that updates the Archetypes fields 5. A section that publishes the event

The CSV file title,startDate,endDate Plone Conference,2009-10-28 09:00,2009-10-30 18:00 Christmas,2009-12-24 00:00,2009-12-26:23:59

Setting it up 1. Define a pipeline in a .cfg file 2. Register it as a name in ZCML 3. Call Transmogrifier from Python

The configuration file [transmogrifier] pipeline = section1 section2 [section1] blueprint = name.of.the.blueprint size = 5 [section2] blueprint = another.blueprint

ZCML <configure xmlns:transmogrifier= "http://namespaces.plone.org/transmogrifier"> <transmogrifier:registerConfig name="Zap event import" title="Import of events" description="" configuration="transmogrifier.cfg" /> </configure>

setuphandlers.py from collective.transmogrifier.transmogrifier import Transmogrifier def setupVarious(context): if context.readDataFile('zap_various.txt') is None: return transmogrifier = Transmogrifier(context.getSite()) transmogrifier('Zap event import')

What is a generator def my_generator(): i=1 while i < 1000: yield i i=i*2 for x in my_generator(): print x

Never ending story def my_generator(): i=1 while True: yield i i=i*2 for x in my_generator(): print x

EventSource class EventSource(object): def __iter__(self): for event is csv.DictReader(open(self.options['csvfile'])) yield event for item in self.previous: yield item

The item { 'title': 'Plone Conference', 'startDate': '2009-10-27 09:00', 'endDate': '2009-10-30 18:00', }

EventUpdater class EventUpdater(object): def __iter__(self): for item in self.previous: id = item['title'].lower().replace(' ', '-') item['_path'] = '/events/'+id item['_type'] = 'Event' item['_transitions'] = ('publish',) yield item

The item { '_type': 'Event', '_path': '/events/plone-conference', '_transitions': ('publish,), 'title': 'Plone Conference', 'startDate': '2009-10-27 09:00', 'endDate': '2009-10-30 18:00', }

Registering the section <utility component=".eventsource.EventSource" name="zap.eventsource" /> <utility component=".eventsource.EventUpdater" name="zap.eventupdater" />

The configuration file [transmogrifier] pipeline = eventsource eventupdater [eventsource] blueprint = zap.eventsource csvfile = /path/to/events.csv [eventupdater] blueprint = zap.eventupdater

The configuration file cont. [constructor] blueprint = collective.transmogrifier.sections.constructor [schemaupdater] blueprint = plone.app.transmogrifier.atschemaupdater [workflow] blueprint = plone.app.transmogrifier.workflowupdater

The configuration file cont. [transmogrifier] pipeline = eventsource eventupdater constructor schemaupdater workflow

Sections Create items Modify items Drop items Split the pipeline in two Construct content Modify content

Migrating from another CMS Similar to the simple example but using SQL instead of CSV

Migration from Plone 2 1. Write an export script 2. Write an import section

Migrate from Plone 2 importsource constructor criterionadder schemaupdater browserdefault workflow savepoint

Migrating from HTML 1. Get HTML from web 2. Extract fields like title, publish date, etc. 3. Make all urls either relative or into UID's

From small to huge Largest migration: 28 sections 527 lines of configuration 16 custom sections

Caveat Emptor Don't migrate from HTML on disk

Caveat Emptor Newly created objects are not indexed.

Caveat Emptor Debugging is backwards

Lennart Regebro regebro@gmail.com IRC nick: regebro

Transmogrifier: Migrating to Plone with less pain

by Lennart Regebro on Oct 30, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 16

Statistics

Transmogrifier: Migrating to Plone with less pain — Presentation Transcript