Using Plone to devolve Web site management and maintenance Vancouver Python Workshop 2004 Dominic Hiles Web Developer

Overview Who am I ? Project background Devolving content maintenance Business roles Workflow Editing content Devolving user management MembershipTool limitations Notifications mechanism Transfer of object ownership on account deletion

Who am I ?

Project background

Devolving content maintenance

Business roles

Workflow

Editing content

Devolving user management

MembershipTool limitations

Notifications mechanism

Transfer of object ownership on account deletion

Who am I ? Web developer within the Internet Development (ID) group of the Institute for Learning and Research Technology (ILRT) ID group has 10+ staff: usability engineers, designers, developers, project managers Started Zope/Python development April 2003 A Plone "newbie" – started September 2003

Web developer within the Internet Development (ID) group of the Institute for Learning and Research Technology (ILRT)

ID group has 10+ staff: usability engineers, designers, developers, project managers

Started Zope/Python development April 2003

A Plone "newbie" – started September 2003

Church of England Project Proposal finalised Feb 2003 Graphic design agreed Sep 2003 Plone development started October 2003 Development happened in tandem with Plone 2 release cycle

Proposal finalised Feb 2003

Graphic design agreed Sep 2003

Plone development started October 2003

Development happened in tandem with Plone 2 release cycle

Background - content 2000 static HTML pages 350 images, 450 "text" files ( e.g. PDFs, MS Word docs) 4 ASP Web applications, serving data from around 20,000 database records, held in 4 different databases

2000 static HTML pages

350 images, 450 "text" files ( e.g. PDFs, MS Word docs)

4 ASP Web applications, serving data from around 20,000 database records, held in 4 different databases

Migration of static content HTML pages passed through HTMLTidy to convert to XHTML All pages then undergo manual editing whilst on the file system Pages, folders and files (e.g. PDFs, images etc .) imported into ZODB via script and created as Plone objects

HTML pages passed through HTMLTidy to convert to XHTML

All pages then undergo manual editing whilst on the file system

Pages, folders and files (e.g. PDFs, images etc .) imported into ZODB via script and created as Plone objects

Migration of Web applications 3 MS Access databases exported as CSV files and imported into new Archetypes objects 1 remaining database: Data must be maintained elsewhere Previously exported as CSV, manually imported into MS Access, then displayed via ASP Now, CSV file uploaded "Through The Plone" (TTP) and data are automatically imported into PostgreSQL

3 MS Access databases exported as CSV files and imported into new Archetypes objects

1 remaining database:

Data must be maintained elsewhere

Previously exported as CSV, manually imported into MS Access, then displayed via ASP

Now, CSV file uploaded "Through The Plone" (TTP) and data are automatically imported into PostgreSQL

Devolving content maintenance All content can now be maintained TTP but… … up to 100 users with responsibility for content in different areas of the site

All content can now be maintained TTP but…

… up to 100 users with responsibility for content in different areas of the site

Talk focus Providing the mechanisms for these users to easily maintain the content for which they’re responsible Providing the tools to manage these users

Providing the mechanisms for these users to easily maintain the content for which they’re responsible

Providing the tools to manage these users

Content maintenance Create the business roles – what should people be able to do to the content? Create the workflow – provides the mechanism to underpin these roles Provide the GUI for maintaining the content

Create the business roles – what should people be able to do to the content?

Create the workflow – provides the mechanism to underpin these roles

Provide the GUI for maintaining the content

Content maintenance - roles Broadly, 4 different business roles: Editor Creates and edits content in an area, submitting it for publishing Reviewer Reviews submitted content, altering, rejecting or publishing it as appropriate

Broadly, 4 different business roles:

Editor

Creates and edits content in an area, submitting it for publishing

Reviewer

Reviews submitted content, altering, rejecting or publishing it as appropriate

Content maintenance – roles (2) Administrator Able to perform functions of Editor and Reviewer Grants roles to other users in the area for which they’re responsible Site Manager Able to perform functions of Administrator Manages users Uses other custom tools ( e.g. import of CSV data)

Administrator

Able to perform functions of Editor and Reviewer

Grants roles to other users in the area for which they’re responsible

Site Manager

Able to perform functions of Administrator

Manages users

Uses other custom tools ( e.g. import of CSV data)

Content maintenance – roles (3) 4 Business roles distilled to 3 Zope roles: Editor, Reviewer, Administrator "Site Manager" = Administrator role granted at site root Groups were created and named according to a folder-dependent role e.g. info_editors (editors of the info folder) Users were placed in group(s) according to their role(s) in a given content area

4 Business roles distilled to 3 Zope roles: Editor, Reviewer, Administrator

"Site Manager" = Administrator role granted at site root

Groups were created and named according to a folder-dependent role

e.g. info_editors (editors of the info folder)

Users were placed in group(s) according to their role(s) in a given content area

Workflow Business roles implemented by designing appropriate workflow Approx. 40% of Plone development time spent on aspects of workflow design and implementation

Business roles implemented by designing appropriate workflow

Approx. 40% of Plone development time spent on aspects of workflow design and implementation

Private Pending Published Reject Retract Submit Publish Retract Publish

Workflow – key requirements All actions should be collaborative Editors and reviewers must be able to work together on content in folders for which they're responsible No notion of ownership – content is owned by the "business" Creator, workflow history and edit history should still exist Not the traditional Plone ethos! Solution: transition guards and state permissions don't reference Owner role

All actions should be collaborative

Editors and reviewers must be able to work together on content in folders for which they're responsible

No notion of ownership – content is owned by the "business"

Creator, workflow history and edit history should still exist

Not the traditional Plone ethos!

Solution: transition guards and state permissions don't reference Owner role

Workflow – key requirements Revisioning (aka simple Staging) The ability to work on a draft copy of a published object, before committing this copy to a new publishing cycle

Revisioning (aka simple Staging)

The ability to work on a draft copy of a published object, before committing this copy to a new publishing cycle

Workflow - revisioning Re-edit transition available on objects in published state Takes a copy of the published object, and attaches it to that published object as an attribute, with a fixed id Published object: index.html Revision: ~~revision~~ Publish Re-edit

Re-edit transition available on objects in published state

Takes a copy of the published object, and attaches it to that published object as an attribute, with a fixed id

Workflow - revisioning

Workflow – key requirements Versioning Ability to view and revert to previously published versions of the same document

Versioning

Ability to view and revert to previously published versions of the same document

Workflow - versioning Publish transition creates a versions container on the published item Old copy is placed in this container each time the object is published Published object: index.html Versions container Version 1 Version 4 Version 3 Version 2 Publish Revert

Publish transition creates a versions container on the published item

Old copy is placed in this container each time the object is published

Workflow - versioning The versions container class HistoricalVersions (SimpleItem): """Manages historical versions of an object""" security = ClassSecurityInfo() meta_type = 'Historical Content Versions' def __init__( self, id ): self.id = id self._versions = Folder('_versions') self._delete_history=() Note use of _versions so that old versions items aren't accessible via traversal

The versions container

class HistoricalVersions (SimpleItem):

"""Manages historical versions of an object"""

security = ClassSecurityInfo()

meta_type = 'Historical Content Versions'

def __init__( self, id ):

self.id = id

self._versions = Folder('_versions')

self._delete_history=()

Note use of _versions so that old versions items aren't accessible via traversal

Workflow - versioning

Re-edit (copy) Versioned Revert Private Pending Published Reject Retract Submit Publish Retract Publish

Workflow – key requirements Notifications The closest, most appropriate person needs to be notified of actions on content Consider a folder : /info/transport/trains Bert is a Reviewer of /info/transport Fred is a Reviewer of /info/transport/trains If an editor submits content for publishing in /info/transport/trains, Fred should be notified but Bert should not Solution: skin scripts, triggered by transitions, send mail appropriately

Notifications

The closest, most appropriate person needs to be notified of actions on content

Consider a folder : /info/transport/trains

Bert is a Reviewer of /info/transport

Fred is a Reviewer of /info/transport/trains

If an editor submits content for publishing in /info/transport/trains, Fred should be notified but Bert should not

Solution: skin scripts, triggered by transitions, send mail appropriately

Workflow – under the bonnet Uses DCWorkflow DCWorkflow allows scripts to be triggered on state transition By default, scripts or external methods must be added to ZODB Subclassed DCWorkflowDefinition to allow Use of FSPythonScripts, accessible via skin and maintainable in CVS Use of (unprotected) methods held within Product module (to provide versioning and revisioning) Added custom states, permissions & guard conditions

Uses DCWorkflow

DCWorkflow allows scripts to be triggered on state transition

By default, scripts or external methods must be added to ZODB

Subclassed DCWorkflowDefinition to allow

Use of FSPythonScripts, accessible via skin and maintainable in CVS

Use of (unprotected) methods held within Product module (to provide versioning and revisioning)

Added custom states, permissions & guard conditions

Workflow – key requirements Email management - "Great workflow, but I don't want all of this Junk mail!" Prioritise Not all emails require action, e.g. "An item is awaiting your review" vs. "An item has been published" Users should be able to subscribe only to emails requiring an action Provide a digest option Provide an unsubscribe option Users may go on holiday or want to hide! Other users need to be notified to take action instead

Email management - "Great workflow, but I don't want all of this Junk mail!"

Prioritise

Not all emails require action, e.g.

"An item is awaiting your review" vs. "An item has been published"

Users should be able to subscribe only to emails requiring an action

Provide a digest option

Provide an unsubscribe option

Users may go on holiday or want to hide!

Other users need to be notified to take action instead

Workflow – email management Solution: two user options Email frequency Immediate (default) Digest Disabled Email categories Info ( e.g. "An item has been published in your area") Warning ( e.g. "An item is awaiting your review") Critical ( e.g. "Your account has been deleted") Critical mails always sent immediately Users can just subscribe to warning emails, or receive info emails as well

Solution: two user options

Email frequency

Immediate (default)

Digest

Disabled

Email categories

Info ( e.g. "An item has been published in your area")

Warning ( e.g. "An item is awaiting your review")

Critical ( e.g. "Your account has been deleted")

Critical mails always sent immediately

Users can just subscribe to warning emails, or receive info emails as well

Workflow – email management digest_mail tool subclasses MailHost Possible workflow actions are set on the tool by the developer, grouped into the three categories Workflow emails are sent with: sendTriggeredMail(self, messageText, mto_userid, mfrom=None, subject=None, encode=None, action=None) sendTriggeredMail() looks up user preferences from mto_userid , cross-checks them with the action category and then handles email appropriately Digest mails are created on the FS, the tool sends them when sendQueuedMail() is called ( e.g. by a cron job)

digest_mail tool subclasses MailHost

Possible workflow actions are set on the tool by the developer, grouped into the three categories

Workflow emails are sent with: sendTriggeredMail(self, messageText, mto_userid, mfrom=None, subject=None, encode=None, action=None)

sendTriggeredMail() looks up user preferences from mto_userid , cross-checks them with the action category and then handles email appropriately

Digest mails are created on the FS, the tool sends them when sendQueuedMail() is called ( e.g. by a cron job)

Workflow – key requirements Deletion must be managed A workflow concept, but normally handled outside of DCWorkflow Should hook into standard notifications machinery Should be possible to restore deleted content State dependant e.g. editors should be able to delete objects in their Private state but not when published

Deletion must be managed

A workflow concept, but normally handled outside of DCWorkflow

Should hook into standard notifications machinery

Should be possible to restore deleted content

State dependant e.g. editors should be able to delete objects in their Private state but not when published

Workflow – managing deletion Idea was to mimic Windows Recycle Bin folder_delete() skin method provides a simple hook Drop in Product - "PortalWastebasket" Items moved to Wastebasket on deletion Items restored to original location on demand State dependency handled by controlling "Delete objects" permission with workflow

Idea was to mimic Windows Recycle Bin

folder_delete() skin method provides a simple hook

Drop in Product - "PortalWastebasket"

Items moved to Wastebasket on deletion

Items restored to original location on demand

State dependency handled by controlling "Delete objects" permission with workflow

Workflow – managing deletion Essentially just another folder but… Must be collaborative – anyone should be able to restore content in areas for which they have responsibility Original security context lost when item is moved to folder, so how do you set e.g. View permissions? Objects must be moved to private state when restored Must be possible to delete more than one object with the same id, or the same object more than once!

Essentially just another folder but…

Must be collaborative – anyone should be able to restore content in areas for which they have responsibility

Original security context lost when item is moved to folder, so how do you set e.g. View permissions?

Objects must be moved to private state when restored

Must be possible to delete more than one object with the same id, or the same object more than once!

Workflow – managing deletion Solution: Move object to Wastebasket, holding its original id/path in a registry Transition object to private state en route to Wastebasket (now invisible to Anonymous) Grant users with Authenticated role View permission on wastebasket items When restoring the object, use workflow definition to reinstate security in correct context

Solution:

Move object to Wastebasket, holding its original id/path in a registry

Transition object to private state en route to Wastebasket (now invisible to Anonymous)

Grant users with Authenticated role View permission on wastebasket items

When restoring the object, use workflow definition to reinstate security in correct context

Workflow – managing deletion

Workflow - summary 3 roles: Editor, Reviewer and Administrator 3 states: Private, Pending and Published Collaborative workflow ( i.e. no "Owner") Revisioning and versioning system Email notifications Email management Wastebasket ( cf. Windows Recycle Bin) …

3 roles: Editor, Reviewer and Administrator

3 states: Private, Pending and Published

Collaborative workflow ( i.e. no "Owner")

Revisioning and versioning system

Email notifications

Email management

Wastebasket ( cf. Windows Recycle Bin)

Maintaining the content WYSIWYG editor Why not Epoz/Kupu? Both were immature when project started Inconsistent tidying from MS Word No spell checker Inconsistent XHTML quality No support for Mac OS X in Kupu Previous experience of edit-On Pro

WYSIWYG editor

Why not Epoz/Kupu?

Both were immature when project started

Inconsistent tidying from MS Word

No spell checker

Inconsistent XHTML quality

No support for Mac OS X in Kupu

Previous experience of edit-On Pro

edit-On Pro Commercial WYSIWYG editor, produced by RealObjects Java applet Created Zope Product to integrate into Plone Biggest problem was/is requirement for recent JRE to be installed locally Kupu looking increasingly attractive!

Commercial WYSIWYG editor, produced by RealObjects

Java applet

Created Zope Product to integrate into Plone

Biggest problem was/is requirement for recent JRE to be installed locally

Kupu looking increasingly attractive!

edit-On Pro in action

Devolving content maintenance Summary Decide on the business roles Create the workflow to facilitate the business roles Provide appropriate editing tools

Decide on the business roles

Create the workflow to facilitate the business roles

Provide appropriate editing tools

Devolving user management Good TTP interface, but there are some problems MembershipTool permissions and local role methods make delegation hard, in effect, requiring Zope "Manager" role No notification mechanism for changes in users’ status No mechanism for transferring ownership upon user deletion

Good TTP interface, but there are some problems

MembershipTool permissions and local role methods make delegation hard, in effect, requiring Zope "Manager" role

No notification mechanism for changes in users’ status

No mechanism for transferring ownership upon user deletion

User management - solutions MembershipTool Subclassed tool and redefined permissions where necessary Overloaded local role methods to handle custom roles

MembershipTool

Subclassed tool and redefined permissions where necessary

Overloaded local role methods to handle custom roles

User management – solutions (2) Notification Hooked user management methods to fire emails when users are added, deleted or change their group memberships Used digest_mail tool

Notification

Hooked user management methods to fire emails when users are added, deleted or change their group memberships

Used digest_mail tool

User management – solutions (3) Ownership Transfer When a user is deleted: Find objects owned by user Transfer ownership to the closest appropriate person in context of each object Notify any users to whom ownership has been transferred Notify the user who has been deleted

Ownership Transfer

When a user is deleted:

Find objects owned by user

Transfer ownership to the closest appropriate person in context of each object

Notify any users to whom ownership has been transferred

Notify the user who has been deleted

User management – summary Effective delegation of user management is impossible in "out-the-box" Plone Plone can be extended to add this functionality CMFMember looks promising for providing a more generic solution

Effective delegation of user management is impossible in "out-the-box" Plone

Plone can be extended to add this functionality

CMFMember looks promising for providing a more generic solution

Final review Effectively devolving content maintenance was hard, principally because of the workflow design and development Lots of open-source Products now appearing to tackle complex workflows, including versioning and revisioning ( e.g. Occams ) Devolving user management was much easier – CMFMember may make it easier still Extensible Zope/Plone architecture allowed functionality to be added as required

Effectively devolving content maintenance was hard, principally because of the workflow design and development

Lots of open-source Products now appearing to tackle complex workflows, including versioning and revisioning ( e.g. Occams )

Devolving user management was much easier – CMFMember may make it easier still

Extensible Zope/Plone architecture allowed functionality to be added as required

Questions? Slides available at: http://www.ilrt.bris.ac.uk/publications/conf/vanPy2004/slides.ppt