fast content import
in Plone
how to import 120k documents
in 8.43 minutes?
http://www.flickr.com/photos/luciano_meirelles/3461046001/sizes/o/

The problem
✓ we have 36.000 documents to migrate (which gives 120.000
with all their children)
✓ we don’t want to wait few hours to do that
✓ users shouldn’t be aware of the migration process

The Idea
✓ Why not just create brains
and let users decide which
document fully migrate?

http://www.flickr.com/photos/oscarmota/1155257083
It’s quite simple

First: Metadata
✓ dummy_object is a simple python object
with minimal metadata
>>> pp dummy_object.__dict__
{'title': u'dummy title',
'id': u'simple_id',
'review_state': 'private',
'path': '/plone/importfolder/ToBeMigrated_simple_id'}

Second: import script
✓ import script which creates brains in
portal_catalog from our dummy object
pc = getToolByName(self.context, 'portal_catalog')
pc.catalog_object(dummy_object, dummy_object.path)

Third: Traverser
✓ Now we need to allow users to see them.
✓ So we created traverser in our import folder
def __bobo_traverse__(self, REQUEST, name):
if name.startswith('ToBeMigrated'):
view = getMultiAdapter(
(self, self.REQUEST),
name='to_be_migrated')
view.setBrainId(name)
return view

At the End
✓ The view informs end-
user that this document
need to be migrated
✓ When user decides to
import it - we will start
the import process for
selected brain.uid

At the End
✓ The rest is pure Plone
folder_contents will lists
'to_be_migrated' brains in
import folder for you
✓ Simple and much faster

Thanks!
Andrew Mleczko
Plone archaeologist
andrzej.mleczko@redturtle.net
http://blog.redturtle.it