Surfacing External Data Through Magnolia


Published on

A technical presentation focusing on how to bring data from other systems into a Magnolia CMS and present it.

Most current version at

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Introduce selves Discuss needs for getting external data: Other systems (HR for directory, feeds for news, Twitter for twitterstuff, order system for sales data) Going to discuss Three Approaches: Standard JSP/Servlet Data Module RSS Aggregator
  • We give our users something very simple…
  • … and give them a lot in return. Nicely formatted, auto-updates, microformats Explain the basic mechanics.
  • Code for custom control to display department list The XML-RPC client library expects any parameters to be stored into a Vector object.
  • Actually display the information
  • Tell what the data module is, its raison d'etre. "DMS for structured data"
  • Jan working on Advanced Data Module Example -- ready to go yet? Provide an aside on the scheduler module and how lovely it is.
  • 1. Define new data type for Special events root path: /events   Mention Silk project as source for icons 2. Show default dialog definition and dialog that results. Add field for start-date. Show updated dialog. Enter conference data. Enter birthday. Show data in JCR browser.
  • All ImportTarget does is provide a container for any options you want to specify. It has no logic! ImportHandler *must* be subclassed so that it actually does something.
  • Discuss what each of the options do   cron format: this is the Java version (6 fields), not the Unix-standard version (5 fields). Warning: lockDuringImport -- can leave your database bollixed up and needing a export/import. ImportTarget: name, targetPath SimpleImportTarget: adds FileURL, ZipFileURL
  • Simply a data structure. This is the *entire* code.   If you need to specify options that aren't already there, subclass this and add your own private fields and accessors for them.
  • This is mandatory for any import job Use the "parent" parameter, rather than target.targetPath(), because we may be importing into a temporary location to be moved on completion. uuids is an empty set to start with.
  • Demo: SQL version: select * from event XPATH version: //element(*, event)
  • Be sure to use the name you defined for your data type when using the getChildren() call.
  • In order to bootstrap data, you must load the data module before you load your own module. Declare this in your pom.xml file.
  • Tell about our RSS implementation and why the RSS module is better.
  • Talk about ROME project a bit Mention that the module can also be used to generate RSS feeds, though that's a bit beyond the scope of this presentation.
  • Go to Data -> RSS Aggregators Create a new aggregator   Use description = ^[^@]*$ to exclude Twitter replies --- Go to RSS Config, trigger fetch Show Data in JCR Browser --- Go to demo-features -> Aggregation Paragraphs -> latest-news Add a paragraph of each type to show how they work
  • Surfacing External Data Through Magnolia

    1. 1. Surfacing External Data Through Magnolia Sean McMains & Jeff Snider September 10, 2009
    2. 2. 3 Approaches: JSP/Servlet Data Module RSS Module
    3. 3. JSP/Servlet Approach or "Teaching an  Old Dog New Tricks" Photo Credit: Stephen Poff
    4. 4. Department Directory
    5. 5. Department Directory XML-RPC Methods: getDepartmentList() getResults()
    6. 6. Implementing a Custom Control <ul><li>    <jsp:scriptlet>         DialogControlImpl control =  </li></ul><ul><li>            (DialogControlImpl)pageContext.getRequest().getAttribute(&quot;dialogObject&quot;);   </li></ul><ul><li>        XmlRpcClientLite client =             new XmlRpcClientLite(&quot;;);         String method = &quot;getDepartmentList&quot;;         Vector params = new Vector();         Vector result = (Vector)client.execute( method, params );         ListIterator i = result.listIterator(); </li></ul><ul><li>        Select selectControl = new Select( control.getName(), control.getWebsiteNode() );         selectControl.setCssClass( CssConstants.CSSCLASS_SELECT ); </li></ul><ul><li>        while (i.hasNext() )         {             String department = (String);             SelectOption option = new SelectOption( department, department );             if ( department.equals( control.getValue() ) ) option.setSelected( true );             selectControl.setOptions( option );         }        out.println( selectControl.getHtml() );     </jsp:scriptlet> </li></ul>
    7. 7. Presenting the Directory Info <ul><li>XmlRpcClientLite client =  </li></ul><ul><li>  new XmlRpcClientLite(&quot;;); </li></ul><ul><li>  </li></ul><ul><li>String department = </li></ul><ul><li>  Resource.getLocalContentNode(request).getNodeData(&quot;department&quot;).getString(); </li></ul><ul><li>String filter =  </li></ul><ul><li>  Resource.getLocalContentNode(request).getNodeData(&quot;filter&quot;).getString();  </li></ul><ul><li>String method = &quot;getResults&quot;; </li></ul><ul><li>Vector params = new Vector(); </li></ul><ul><li>String searchTerm = &quot;department=&quot;&quot; + department + &quot;&quot;&quot;; </li></ul><ul><li>params.addElement(searchTerm); </li></ul><ul><li>  </li></ul><ul><li>Vector result = (Vector)client.execute( method, params ); </li></ul><ul><li>  </li></ul><ul><li>ListIterator i = result.listIterator(); </li></ul><ul><li>while ( i.hasNext() ) { </li></ul><ul><li>  Hashtable person = (Hashtable); </li></ul><ul><li>  ... </li></ul><ul><li>  String firstName = (String)person.get(&quot;firstname&quot;); </li></ul><ul><li>  String lastName = (String)person.get(&quot;lastname&quot;);   html.append( &quot;<div class=&quot;txst-departmentdirectory-name fn n&quot;> </li></ul><ul><li>    <span class=&quot;given-name&quot;>&quot; + firstName + &quot;</span> <span class=&quot;family-name&quot;>&quot; + </li></ul><ul><li>    lastName + &quot;</span>&nbsp;</div>&quot; ); </li></ul><ul><li>  ... </li></ul><ul><li>}                                      </li></ul>
    8. 8. JSP/Servlet Approach <ul><li>Pros: </li></ul><ul><ul><li>Quick & easy </li></ul></ul><ul><ul><li>No additional modules needed, aside from whatever you use to actually fetch the data </li></ul></ul><ul><ul><li>Uses familiar techniques </li></ul></ul><ul><li>Cons: </li></ul><ul><ul><li>Page caching issues </li></ul></ul><ul><ul><li>Performance </li></ul></ul><ul><ul><li>At mercy of connection to data source </li></ul></ul><ul><ul><li>You have to write it all yourself! </li></ul></ul>
    9. 9. The Data Module or &quot;How I Learned to Stop Worrying and Love the JCR&quot; Photo Credit: Kjunstorm
    10. 10. The Data Module <ul><li>Info: </li></ul><ul><ul><li>Data Module Docs </li></ul></ul><ul><ul><li>Footers Example </li></ul></ul><ul><ul><li>Advanced Data Module Example (?) </li></ul></ul><ul><li>  </li></ul><ul><li>Requirements: </li></ul><ul><ul><li>Magnolia 4.x </li></ul></ul><ul><ul><li>Scheduler Module </li></ul></ul><ul><ul><li>A Bit of Patience </li></ul></ul>
    11. 11. Creating a New Data Type Demonstration
    12. 12. Import Handlers <ul><ul><li>Built on scheduler module </li></ul></ul><ul><ul><li>Provide an automated means to suck in new data </li></ul></ul><ul><ul><li>At /modules/data/config/importers </li></ul></ul>
    13. 13. Using Import Handlers
    14. 14. Configuring an Import Handler
    15. 15. ImportTarget Code public class ImportTarget {     protected String name;     protected String targetPath;     public void setName(String name) { = name;     }     public void setTargetPath(String targetPath) {         this.targetPath = targetPath;     }     public ImportTarget() {     }     public String getTargetPath() {         return targetPath;     }     public String getName() {         return name;     } }
    16. 16. ImportHandler Sample <ul><li>public class DateTimeImportHandler extends ImportHandler {     ...     protected Set doImport(ImportTarget target, Content parent, Set uuids) throws ImportException { </li></ul><ul><li>            ...             String date = dateFormat.format( new Date() );             String name = &quot;ImportedSampleData&quot; + parent.getChildren(&quot;example&quot;).size(); </li></ul><ul><li>            // create new data node.             // FYI: if the name matches existing node and &quot;deleteExisting&quot; is             // set to true, old copy of the node will be automatically deleted.             Content child = parent.createContent( name, &quot;example&quot; ); </li></ul><ul><li>            // set properties for newly created node             NodeDataUtil.getOrCreate( child, &quot;name&quot;).setValue(name);             NodeDataUtil.getOrCreate(child, &quot;comment&quot;).setValue(&quot;Example of data import. </li></ul><ul><li>              Generated on &quot; + date);             NodeDataUtil.getOrCreate(child, &quot;data&quot;).setValue(date); </li></ul><ul><li>  ; </li></ul><ul><li>            uuids.add(child.getUUID()); </li></ul><ul><li>            ...         return uuids;     } } </li></ul>
    17. 17. JCR Query <ul><ul><li>XPath or SQL-style </li></ul></ul><ul><ul><li>Powerful selection capabilities </li></ul></ul><ul><ul><li>Can be complicated </li></ul></ul><ul><li>  </li></ul><ul><li>Additional Info: </li></ul><ul><ul><li>  JSR-170 Spec </li></ul></ul>
    18. 18. Using the Node Object <ul><li>            <jsp:scriptlet> </li></ul><ul><li>                final String BR = &quot;&lt;br/&gt;&quot;;                                 HierarchyManager dataHierarchyManager =  </li></ul><ul><li>                     MgnlContext.getHierarchyManager( &quot;data&quot; );                 Content eventsNode = dataHierarchyManager.getContent( &quot;/event&quot; );                 Iterator eventsNodeIterator = eventsNode.getChildren( &quot;event&quot; ).iterator();                                 while ( eventsNodeIterator.hasNext() ) {                     Content event = (Content);                     out.write( event.getNodeData( &quot;name&quot; ).getString() + BR );                     out.write( event.getNodeData( &quot;date&quot; ).getString() + BR + BR );                 }                             </jsp:scriptlet> </li></ul>
    19. 19. Caveat: Bootstrapping Data <ul><li>Add to your project's pom.xml: </li></ul><ul><li>… </li></ul><ul><li><dependencies>     <dependency>       <name>data</name>       <version>1.2.1/*</version>     </dependency> </dependencies> </li></ul><ul><li>… </li></ul>
    20. 20. Data Module <ul><li>Pros: </li></ul><ul><ul><li>Data is cached in JCR </li></ul></ul><ul><ul><li>Resilient to temporary disconnect from data </li></ul></ul><ul><ul><li>Can enter data manually through AdminCentral or import automatically </li></ul></ul><ul><li>Cons: </li></ul><ul><ul><li>You still have to write it all yourself </li></ul></ul><ul><ul><li>Requires learning another framework </li></ul></ul>
    21. 21. Bringing in RSS Content or &quot;All the news that fits, we print!&quot; Photo Credit: tantrum_dan
    22. 22. The RSS Module <ul><li>Info: </li></ul><ul><ul><li>Official Documentation </li></ul></ul><ul><ul><li>Jan's Weblog Entry </li></ul></ul><ul><li>  </li></ul><ul><li>Requires: </li></ul><ul><ul><li>Magnolia 4.x </li></ul></ul><ul><ul><li>Scheduler Module </li></ul></ul><ul><ul><li>Data Module </li></ul></ul><ul><ul><li>Built on ROME Project </li></ul></ul>
    23. 23. Sightseeing in the RSS Module Add new feeds Configure RSS Module
    24. 24. Setting up RSS Feeds Demonstration
    25. 25. RSS Module <ul><li>Pros: </li></ul><ul><ul><li>Easy set up </li></ul></ul><ul><ul><li>Includes several useful paragraphs </li></ul></ul><ul><ul><li>Can use it without writing any additional code </li></ul></ul><ul><li>Cons: </li></ul><ul><ul><li>Can only use content that's published in RSS format </li></ul></ul>
    26. 26. Questions? <ul><li>Photo Credit: coldtaxi </li></ul>
    27. 27. Photo Credit: damaradeaella