Your SlideShare is downloading. ×
0
Transmogrifier

       Lennart Regebro
Plone Conference 2009, Budapest
collective.transmogrifier

                 Created by
              Martijn Pieters, Jarn
                    in 2008
   ...
collective.transmogrifier

                Import/migration
                   framework
               Both from Plone 2
...
Super-simple architecture
                  You set up a
               pipeline of sections
                Each section ...
What are you talking about?
A simple example

1. A section that reads a CSV file with a list of
   events and creates items based on that data
2. A se...
The CSV file



title,startDate,endDate
Plone Conference,2009-10-28 09:00,2009-10-30 18:00
Christmas,2009-12-24 00:00,2009...
Setting it up


 1. Define a pipeline in a .cfg file

 2. Register it as a name in ZCML

3. Call Transmogrifier from Python
The configuration file
[transmogrifier]
pipeline =
    section1
    section2

[section1]
blueprint = name.of.the.blueprint...
ZCML
<configure
   xmlns:transmogrifier=
     "http://namespaces.plone.org/transmogrifier">

 <transmogrifier:registerConf...
setuphandlers.py

from collective.transmogrifier.transmogrifier import 
 Transmogrifier

def setupVarious(context):

   if...
What is a generator

def my_generator():
    i=1
    while i < 1000:
        yield i
        i=i*2

for x in my_generator(...
Never ending story

def my_generator():
    i=1
    while True:
        yield i
        i=i*2

for x in my_generator():
  ...
EventSource


class EventSource(object):

   def __iter__(self):
      for event is csv.DictReader(open(self.options['csvf...
EventSource


class EventSource(object):

   def __iter__(self):
      for event is csv.DictReader(open(self.options['csvf...
EventSource


class EventSource(object):

   def __iter__(self):
      for event is csv.DictReader(open(self.options['csvf...
The item


{
 'title': 'Plone Conference',
 'startDate': '2009-10-27 09:00',
 'endDate': '2009-10-30 18:00',
}
EventUpdater

class EventUpdater(object):

   def __iter__(self):
      for item in self.previous:
         id = item['tit...
The item

{
 '_type': 'Event',
 '_path': '/events/plone-conference',
 '_transitions': ('publish,),
 'title': 'Plone Confer...
Registering the section

<utility
  component=".eventsource.EventSource"
  name="zap.eventsource"
  />

<utility
  compone...
The configuration file
[transmogrifier]
pipeline =
    eventsource
    eventupdater

[eventsource]
blueprint = zap.eventso...
The configuration file cont.

[constructor]
blueprint = collective.transmogrifier.sections.constructor

[schemaupdater]
bl...
The configuration file cont.

[transmogrifier]
pipeline =
    eventsource
    eventupdater
    constructor
    schemaupdat...
Sections
           Create items
           Modify items
            Drop items
     Split the pipeline in
              t...
Migrating from another CMS




  Similar to the simple example
  but using SQL instead of CSV
Migration from Plone 2




  1. Write an export script
 2. Write an import section
Migrate from Plone 2

     importsource
      constructor
    criterionadder
    schemaupdater
    browserdefault
       w...
Migrating from HTML


           1. Get HTML from web

2. Extract fields like title, publish date, etc.

3. Make all urls ...
From small to huge



   Largest migration:
       28 sections
527 lines of configuration
   16 custom sections
Caveat Emptor




Don't migrate from HTML on disk
Caveat Emptor




Newly created objects are not indexed.
Caveat Emptor




Debugging is backwards
Lennart Regebro

regebro@gmail.com

 IRC nick: regebro
Upcoming SlideShare
Loading in...5
×

Transmogrifier: Migrating to Plone with less pain

3,608

Published on

Transmogrifier is a migration framework written by Martijn Pieters, Jarn, that makes migrating to Plone fun again. This talk gives an introduction to Transmogrifier, an explanation of how it works and some hints for usage.

Published in: Technology, Education

Transcript of "Transmogrifier: Migrating to Plone with less pain"

  1. 1. Transmogrifier Lennart Regebro Plone Conference 2009, Budapest
  2. 2. collective.transmogrifier Created by Martijn Pieters, Jarn in 2008 Released in version 1.0 August 2009
  3. 3. collective.transmogrifier Import/migration framework Both from Plone 2 and from other websites or data
  4. 4. Super-simple architecture You set up a pipeline of sections Each section send items to the next section. Sections do something with the items (or not)
  5. 5. What are you talking about?
  6. 6. A simple example 1. A section that reads a CSV file with a list of events and creates items based on that data 2. A section that will generate some plone-specific data for each item, like path and type. 3. A section that creates the Plone event 4. A section that updates the Archetypes fields 5. A section that publishes the event
  7. 7. The CSV file title,startDate,endDate Plone Conference,2009-10-28 09:00,2009-10-30 18:00 Christmas,2009-12-24 00:00,2009-12-26:23:59
  8. 8. Setting it up 1. Define a pipeline in a .cfg file 2. Register it as a name in ZCML 3. Call Transmogrifier from Python
  9. 9. The configuration file [transmogrifier] pipeline = section1 section2 [section1] blueprint = name.of.the.blueprint size = 5 [section2] blueprint = another.blueprint
  10. 10. ZCML <configure xmlns:transmogrifier= "http://namespaces.plone.org/transmogrifier"> <transmogrifier:registerConfig name="Zap event import" title="Import of events" description="" configuration="transmogrifier.cfg" /> </configure>
  11. 11. setuphandlers.py from collective.transmogrifier.transmogrifier import Transmogrifier def setupVarious(context): if context.readDataFile('zap_various.txt') is None: return transmogrifier = Transmogrifier(context.getSite()) transmogrifier('Zap event import')
  12. 12. What is a generator def my_generator(): i=1 while i < 1000: yield i i=i*2 for x in my_generator(): print x
  13. 13. Never ending story def my_generator(): i=1 while True: yield i i=i*2 for x in my_generator(): print x
  14. 14. EventSource class EventSource(object): def __iter__(self): for event is csv.DictReader(open(self.options['csvfile'])) yield event for item in self.previous: yield item
  15. 15. EventSource class EventSource(object): def __iter__(self): for event is csv.DictReader(open(self.options['csvfile'])) yield event for item in self.previous: yield item
  16. 16. EventSource class EventSource(object): def __iter__(self): for event is csv.DictReader(open(self.options['csvfile'])) yield event for item in self.previous: yield item
  17. 17. The item { 'title': 'Plone Conference', 'startDate': '2009-10-27 09:00', 'endDate': '2009-10-30 18:00', }
  18. 18. EventUpdater class EventUpdater(object): def __iter__(self): for item in self.previous: id = item['title'].lower().replace(' ', '-') item['_path'] = '/events/'+id item['_type'] = 'Event' item['_transitions'] = ('publish',) yield item
  19. 19. The item { '_type': 'Event', '_path': '/events/plone-conference', '_transitions': ('publish,), 'title': 'Plone Conference', 'startDate': '2009-10-27 09:00', 'endDate': '2009-10-30 18:00', }
  20. 20. Registering the section <utility component=".eventsource.EventSource" name="zap.eventsource" /> <utility component=".eventsource.EventUpdater" name="zap.eventupdater" />
  21. 21. The configuration file [transmogrifier] pipeline = eventsource eventupdater [eventsource] blueprint = zap.eventsource csvfile = /path/to/events.csv [eventupdater] blueprint = zap.eventupdater
  22. 22. The configuration file cont. [constructor] blueprint = collective.transmogrifier.sections.constructor [schemaupdater] blueprint = plone.app.transmogrifier.atschemaupdater [workflow] blueprint = plone.app.transmogrifier.workflowupdater
  23. 23. The configuration file cont. [transmogrifier] pipeline = eventsource eventupdater constructor schemaupdater workflow
  24. 24. Sections Create items Modify items Drop items Split the pipeline in two Construct content Modify content
  25. 25. Migrating from another CMS Similar to the simple example but using SQL instead of CSV
  26. 26. Migration from Plone 2 1. Write an export script 2. Write an import section
  27. 27. Migrate from Plone 2 importsource constructor criterionadder schemaupdater browserdefault workflow savepoint
  28. 28. Migrating from HTML 1. Get HTML from web 2. Extract fields like title, publish date, etc. 3. Make all urls either relative or into UID's
  29. 29. From small to huge Largest migration: 28 sections 527 lines of configuration 16 custom sections
  30. 30. Caveat Emptor Don't migrate from HTML on disk
  31. 31. Caveat Emptor Newly created objects are not indexed.
  32. 32. Caveat Emptor Debugging is backwards
  33. 33. Lennart Regebro regebro@gmail.com IRC nick: regebro
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×