Talk CalDAV To Me: Integrating Bedework into OAE<br />Chris Tweney <chris@media.berkeley.edu><br />Senior Software Develop...
myBerkeley<br />Instance of Sakai OAE with customizations<br />Esp. those that allow advisors to communicate to students<b...
myBerkeley widgets<br />
myBerkeley widgets<br />
myBerkeley Notifications<br />
Notification Dependencies<br />
Calendaring Fundamentals<br />iCalendar specification<br />iCal4j<br />CalDAV protocol<br />WebDAV protocol<br />
iCalendar<br />Simple text-based format for calendar data exchange<br />iCalendar's fundamental objects are:<br />Events (...
iCalendar<br />Usually you encounter iCalendar data in the form of files with a “.ics” extension.  Here’s a typical (abbre...
iCalendar Parts<br />
ical4j<br />Open-source Java library to wrangle messy iCalendar data<br /> iCalendar records with time zone data get very ...
ical4j Java Example<br />protected Calendar buildVTodo(String summary) {<br />CalendarBuilder builder = new CalendarBuilde...
ical4j pitfall: Line folding<br />Per the iCalendar RFC, long lines of text get newlines inserted at the 75th column<br />...
CalDAV<br />Dialect of WebDAV that provides calendar functionality<br />Much syntax and structure is inherited from its un...
Sakai 2 CalDAV<br />GA Tech had a CalDAV project in Sakai 2 that was never released due to gaps in Zimbra's API<br />Zach ...
CalDAV, WebDAV, HTTP<br />
Learn by snooping<br />
Speaking CalDAV with curl<br />Doing a PROPFIND is a shortcut way to get a user's whole calendar:<br />curl –u user:pass -...
Speaking CalDAV with curl<br />curl –u user:pass -X PROPFIND http://localhost:8080/ucaldav/user/300846/calendar/<br />HTTP...
CalDAV and WebDAV challenges<br />Not your everyday HTTP request<br />Weird methods (PROPFIND, OPTIONS, PUT, etc)<br />Mul...
Fortunately…<br />Jackrabbit, on which OAE is based, ships with a reasonable WebDAV Java client<br />We can use this to ma...
CalDavConnector.getCalendarUris()<br />getCalendarUris() is our simplest method<br />Just return URI wrapper objects for a...
Complexity's a problem<br />CalDAV and WebDAV are<br />Complicated<br />Not noob-friendly<br />RFCs are almost the only do...
Impatience is a virtue<br />Need lots of trial and error<br />Redeploying a server takes 60-90s<br />Running a test takes ...
Integration-test-driven development<br />JUnit to the rescue!<br />"Unit" tests that actually talk to a running Bedework s...
Write the test first…<br />Before doing anything else I write a test: <br />public class CalDavConnectorTest() {<br />  @T...
…Then make it pass<br />Now we'll make it pass in the hackiest way imaginable<br />public class CalDavConnector() {<br /> ...
…Iterate until done<br />Keep adding implementation code until it does what's needed: <br />      PropFindMethod propFind ...
Unit testing heresy<br />These tests talk to and expect a running Bedework server<br />This makes them heretical according...
Gracefully failing test<br />This test will succeed even if the Bedework server does not respond (because we catch the IOE...
PROPFIND only gives you URIs…<br />curl –u user:pass -X PROPFIND http://localhost:8080/ucaldav/user/300846/calendar/<br />...
…which you then must get with a REPORT method<br />curl –u user:pass -X REPORT http://localhost:8080/ucaldav/user/mtwain/c...
REPORT method's response<br /><DAV:response><br />    <DAV:href>/ucaldav/user/mtwain/calendar/02bfa2cd-6c04-40f4-93da-896d...
Hairy REPORT Syntax<br />CalDAV's REPORT syntax is hairy XML<br />Have to extend Jackrabbit's ReportInfo classes<br />This...
REPORT is foundation for search<br />REPORT methods with parameters let you search by date<br />
Bedework difficulties<br />Bedework's search implementation spotty<br />No support for search on X-props<br />No support f...
Filtering on myBerkeley server<br />Due to Bedework bugs we're forced to do some filtering on the myBerkeley server side<b...
Caching on myBerkeley server<br />CalendarURI wrapper class<br />URI: Locates calendar<br />Etag: Uniquely identifies its ...
Implementing CalDavProxyServlet<br />Also done with test-driven development<br />Write tests against JSON files that conta...
Improvements to Nakamura's CalendarService<br />Future work: Store and search calendars using Nakamura's CalendarService<b...
Beyond CalendarService<br />Future: Create a full-blown CalDAV Calendar Provider component for Nakamura<br />Store calenda...
Upcoming SlideShare
Loading in...5
×

Integrating Bedework, a CalDAV Calendar Server, into OAE

4,570

Published on

In this session I'll share my experience integrating Bedework, a CalDAV calendar server, into the Berkeley version of the Sakai OAE. Users can see their calendars inside OAE or in the CalDAV client of their choice (e.g. Apple iCal). The presentation will go into heavy technical detail on the iCalendar data specification, CalDAV, WebDAV, and related protocols. I'll also focus on how to use JUnit to write integration tests that help you learn and exercise complex functionality in external systems, one tiny feature at a time. Audience members should have some background in Java programming and XML.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
4,570
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
32
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Integrating Bedework, a CalDAV Calendar Server, into OAE

  1. 1. Talk CalDAV To Me: Integrating Bedework into OAE<br />Chris Tweney <chris@media.berkeley.edu><br />Senior Software Developer<br />Educational Technology Services<br />UC Berkeley<br />June 14, 2011<br />
  2. 2. myBerkeley<br />Instance of Sakai OAE with customizations<br />Esp. those that allow advisors to communicate to students<br />
  3. 3. myBerkeley widgets<br />
  4. 4. myBerkeley widgets<br />
  5. 5. myBerkeley Notifications<br />
  6. 6. Notification Dependencies<br />
  7. 7. Calendaring Fundamentals<br />iCalendar specification<br />iCal4j<br />CalDAV protocol<br />WebDAV protocol<br />
  8. 8. iCalendar<br />Simple text-based format for calendar data exchange<br />iCalendar's fundamental objects are:<br />Events (VEVENT)<br />To-dos (VTODO)<br />Journal entries (VJOURNAL)<br />Free-busy info (VFREEBUSY)<br />
  9. 9. iCalendar<br />Usually you encounter iCalendar data in the form of files with a “.ics” extension. Here’s a typical (abbreviated) example:<br />BEGIN:VCALENDAR<br />PRODID:-//Ben Fortuna//iCal4j 1.0//EN<br />VERSION:2.0<br />CALSCALE:GREGORIAN<br />BEGIN:VTODO<br />DTSTAMP:20110603T222847Z<br />DTSTART:20110505T151506<br />DUE:20110505T151506<br />SUMMARY:Pay your bill<br />UID:f84347ab-575b-4274-9436-a5ac906381f9<br />DESCRIPTION:Pay your bill by the deadline of May 5. <br />END:VTODO<br />END:VCALENDAR<br />iCalendar is defined in RFC 5545<br />http://tools.ietf.org/html/rfc5545<br />
  10. 10. iCalendar Parts<br />
  11. 11. ical4j<br />Open-source Java library to wrangle messy iCalendar data<br /> iCalendar records with time zone data get very complicated; ical4j makes it pretty simple to work with them<br />ical4j home page:<br />http://wiki.modularity.net.au/ical4j/index.php?title=Main_Page<br />
  12. 12. ical4j Java Example<br />protected Calendar buildVTodo(String summary) {<br />CalendarBuilder builder = new CalendarBuilder();<br /> Calendar calendar = new Calendar();<br />calendar.getProperties().add(newProdId("-//Ben Fortuna//iCal4j 1.0//EN"));<br /> calendar.getProperties().add(Version.VERSION_2_0);<br />calendar.getProperties().add(CalScale.GREGORIAN);<br />TimeZoneRegistry registry = builder.getRegistry();<br />VTimeZonetz = registry.getTimeZone("America/Los_Angeles").getVTimeZone();<br />calendar.getComponents().add(tz);<br />DateTime due = new DateTime(DateUtils.addDays(new Date(), new Random().nextInt(28)));<br />VToDovtodo = new VToDo(due, due, summary);<br />vtodo.getProperties().add(newUid(UUID.randomUUID().toString()));<br />vtodo.getProperties().add(CalDavConnector.MYBERKELEY_REQUIRED);<br />vtodo.getProperties().add(newDescription("this is the description, it is long enough to<br /> wrap at the ical specified standard 75th column"));<br />vtodo.getProperties().add(Status.VTODO_NEEDS_ACTION);<br />calendar.getComponents().add(vtodo);<br /> return calendar;<br />}<br />
  13. 13. ical4j pitfall: Line folding<br />Per the iCalendar RFC, long lines of text get newlines inserted at the 75th column<br />Sadly, ical4j does not handle this by default<br />Need to specify “ical4j.unfolding.relaxed=true” in an ical4j.properties file<br />
  14. 14. CalDAV<br />Dialect of WebDAV that provides calendar functionality<br />Much syntax and structure is inherited from its underlying specifications: WebDAV and iCalendar<br />CalDAV is defined in RFC 4791:<br />http://tools.ietf.org/html/rfc4791<br />
  15. 15. Sakai 2 CalDAV<br />GA Tech had a CalDAV project in Sakai 2 that was never released due to gaps in Zimbra's API<br />Zach Thomas left excellent docs that hugely helped our efforts<br />Sakai 2 CalDAV Doc Page:<br />https://confluence.sakaiproject.org/display/CALDAV/Developer's+Guide<br />
  16. 16. CalDAV, WebDAV, HTTP<br />
  17. 17. Learn by snooping<br />
  18. 18. Speaking CalDAV with curl<br />Doing a PROPFIND is a shortcut way to get a user's whole calendar:<br />curl –u user:pass -X PROPFIND http://localhost:8080/ucaldav/user/300846/calendar/<br />To curl, "-X PROPFIND" means do an HTTP PROPFIND method.<br />HTTP has several funky methods you've never heard of <br />unless you've worked with WebDAV. <br />
  19. 19. Speaking CalDAV with curl<br />curl –u user:pass -X PROPFIND http://localhost:8080/ucaldav/user/300846/calendar/<br />HTTP/1.1 207 Multi-Status<br /><?xml version="1.0" encoding="UTF-8" ?><br /><DAV:multistatus xmlns:DAV="DAV:"<br /> xmlns="urn:ietf:params:xml:ns:caldav"<br /> xmlns:ical="http://www.w3.org/2002/12/cal/ical#"><br /> <DAV:response><br /><DAV:href>/ucaldav/user/300846/calendar/00137ac8-1638-4193-ac3e-c172dfe3fd35.ics</DAV:href><br /> <DAV:propstat><br /> <DAV:prop><br /> <DAV:getcontenttype>text/calendar; charset=UTF-8</DAV:getcontenttype><br /> <DAV:getcontentlength>339</DAV:getcontentlength><br /> <DAV:getlastmodified>Tue, 31 May 2011 21:23:32 +0000</DAV:getlastmodified><br /> <DAV:displayname>00137ac8-1638-4193-ac3e-c172dfe3fd35.ics</DAV:displayname><br /><DAV:getetag>"20110531T212332Z-0"</DAV:getetag><br /> <DAV:current-user-principal><br /> <DAV:href>/ucaldav/principals/users/admin/</DAV:href><br /> </DAV:current-user-principal><br /> <DAV:resourcetype/><br /> <DAV:getcontentlanguage>en</DAV:getcontentlanguage><br /> <DAV:creationdate>20110531T212332Z</DAV:creationdate><br /> </DAV:prop><br /> <DAV:status>HTTP/1.1 200 ok</DAV:status><br /> </DAV:propstat><br /> </DAV:response><br /> … more entries snipped …<br /></DAV:multistatus><br />
  20. 20. CalDAV and WebDAV challenges<br />Not your everyday HTTP request<br />Weird methods (PROPFIND, OPTIONS, PUT, etc)<br />Multi-status responses (HTTP 207)<br />
  21. 21. Fortunately…<br />Jackrabbit, on which OAE is based, ships with a reasonable WebDAV Java client<br />We can use this to make our basic CalDAV connections <br />
  22. 22. CalDavConnector.getCalendarUris()<br />getCalendarUris() is our simplest method<br />Just return URI wrapper objects for a user's whole calendar<br />
  23. 23. Complexity's a problem<br />CalDAV and WebDAV are<br />Complicated<br />Not noob-friendly<br />RFCs are almost the only docs available<br />
  24. 24. Impatience is a virtue<br />Need lots of trial and error<br />Redeploying a server takes 60-90s<br />Running a test takes 2s<br />2s is too little time to get distracted<br />
  25. 25. Integration-test-driven development<br />JUnit to the rescue!<br />"Unit" tests that actually talk to a running Bedework server are the solution<br />
  26. 26. Write the test first…<br />Before doing anything else I write a test: <br />public class CalDavConnectorTest() {<br /> @Test()<br /> public void getCalendars() throws CalDavException { <br /> List<CalendarURI> uris = <br /> this.calDavConnector.getCalendarUris();<br /> }<br />}<br />
  27. 27. …Then make it pass<br />Now we'll make it pass in the hackiest way imaginable<br />public class CalDavConnector() {<br /> public void getCalendars() throws CalDavException { <br /> return new ArrayList<CalendarURI>();<br /> }<br />}<br />
  28. 28. …Iterate until done<br />Keep adding implementation code until it does what's needed: <br /> PropFindMethod propFind = executeMethod(new PropFindMethod(this.userHome.toString()));<br /> MultiStatusResponse[] responses = propFind.getResponseBodyAsMultiStatus().getResponses();<br /> for (MultiStatusResponse response : responses) {<br /> if (response.getHref().endsWith(".ics")) {<br /> Status[] status = response.getStatus();<br /> if (status.length == 1 && status[0].getStatusCode() == HttpServletResponse.SC_OK) {<br /> DavPropertySet propSet = response.getProperties(HttpServletResponse.SC_OK);<br />DavProperty etag = propSet.get(DavPropertyName.GETETAG);<br /> try {<br /> CalendarURI calUri = new CalendarURI(<br /> new URI(this.serverRoot, response.getHref(), false),<br /> etag.getValue().toString());<br /> uris.add(calUri);<br /> } catch (ParseException pe) {<br /> throw new CalDavException("Invalid etag date", pe);<br /> }<br /> }<br /> }<br /> }<br />
  29. 29. Unit testing heresy<br />These tests talk to and expect a running Bedework server<br />This makes them heretical according to true unit test dogma<br />Tests are not supposed to require an external server<br />
  30. 30. Gracefully failing test<br />This test will succeed even if the Bedework server does not respond (because we catch the IOException):<br />public class CalDavConnectorTest() {<br /> @Test()<br /> public void getCalendars() throws CalDavException {<br /> try {<br /> List<CalendarURI> uris = <br /> this.calDavConnector.getCalendarUris();<br /> } catch (IOException ioe) {<br /> LOGGER.error("Trouble contacting server", ioe);<br /> }<br /> }<br />}<br />
  31. 31. PROPFIND only gives you URIs…<br />curl –u user:pass -X PROPFIND http://localhost:8080/ucaldav/user/300846/calendar/<br />HTTP/1.1 207 Multi-Status<br /><?xml version="1.0" encoding="UTF-8" ?><br /><DAV:multistatus xmlns:DAV="DAV:"<br /> xmlns="urn:ietf:params:xml:ns:caldav"<br /> xmlns:ical="http://www.w3.org/2002/12/cal/ical#"><br /> <DAV:response><br /> <DAV:href>/ucaldav/user/300846/calendar/00137ac8-1638-4193-ac3e-c172dfe3fd35.ics</DAV:href><br /> <DAV:propstat><br /> <DAV:prop><br /> <DAV:getcontenttype>text/calendar; charset=UTF-8</DAV:getcontenttype><br /> <DAV:getcontentlength>339</DAV:getcontentlength><br /> <DAV:getlastmodified>Tue, 31 May 2011 21:23:32 +0000</DAV:getlastmodified><br /> <DAV:displayname>00137ac8-1638-4193-ac3e-c172dfe3fd35.ics</DAV:displayname><br /> <DAV:getetag>"20110531T212332Z-0"</DAV:getetag><br /> <DAV:current-user-principal><br /> <DAV:href>/ucaldav/principals/users/admin/</DAV:href><br /> </DAV:current-user-principal><br /> <DAV:resourcetype/><br /> <DAV:getcontentlanguage>en</DAV:getcontentlanguage><br /> <DAV:creationdate>20110531T212332Z</DAV:creationdate><br /> </DAV:prop><br /> <DAV:status>HTTP/1.1 200 ok</DAV:status><br /> </DAV:propstat><br /> </DAV:response><br /> … more entries snipped …<br /></DAV:multistatus><br />
  32. 32. …which you then must get with a REPORT method<br />curl –u user:pass -X REPORT http://localhost:8080/ucaldav/user/mtwain/calendar/ -d "<?xml version="1.0" encoding="UTF-8"?><br /><C:calendar-multiget xmlns:C="urn:ietf:params:xml:ns:caldav" xmlns:D="DAV:"><br /> <D:prop><br /> <D:getetag/><br /> <C:calendar-data/><br /> </D:prop><br /> <D:href>http://localhost:8080/ucaldav/user/mtwain/calendar/d12ad881-5769-4d17-85ac-fa0a0196ec04.ics<br /> </D:href><br /></C:calendar-multiget>"<br />(Note: Double quotes not escaped for clarity)<br />
  33. 33. REPORT method's response<br /><DAV:response><br /> <DAV:href>/ucaldav/user/mtwain/calendar/02bfa2cd-6c04-40f4-93da-896d54fc2987.ics</DAV:href><br /> <DAV:propstat><br /> <DAV:prop><br /> <DAV:getetag>"20110602T211046Z-0"</DAV:getetag><br /> <calendar-data><![CDATA[BEGIN:VCALENDAR<br />PRODID://Bedework.org//BedeWork V3.7//EN<br />VERSION:2.0<br />BEGIN:VTODO<br />CATEGORIES:MyBerkeley-Required<br />CATEGORIES:MyBerkeley-Archived<br />CREATED:20110601T171120Z<br />DESCRIPTION:test<br />… [ snip ]<br />]]></calendar-data><br /> </DAV:prop><br /> <DAV:status>HTTP/1.1 200 ok</DAV:status><br /> </DAV:propstat><br /></DAV:response><br />
  34. 34. Hairy REPORT Syntax<br />CalDAV's REPORT syntax is hairy XML<br />Have to extend Jackrabbit's ReportInfo classes<br />This blog entry by Ricardo Martin Camarero has very useful starter code:<br />http://blogs.nologin.es/rickyepoderi/index.php?/archives/15-Introducing-CalDAV-Part-II.html<br />
  35. 35. REPORT is foundation for search<br />REPORT methods with parameters let you search by date<br />
  36. 36. Bedework difficulties<br />Bedework's search implementation spotty<br />No support for search on X-props<br />No support for search on CATEGORIES<br />
  37. 37. Filtering on myBerkeley server<br />Due to Bedework bugs we're forced to do some filtering on the myBerkeley server side<br />E.g. Required/Not Required fields<br />
  38. 38. Caching on myBerkeley server<br />CalendarURI wrapper class<br />URI: Locates calendar<br />Etag: Uniquely identifies its contents (sort of like a SHA hash)<br />Caching not implemented yet, but easy enough since all calendars are keyed by CalendarURI<br />
  39. 39. Implementing CalDavProxyServlet<br />Also done with test-driven development<br />Write tests against JSON files that contain servlet's expected inputs<br />When servlet's finished, UI devs use those JSON files as an API reference<br />
  40. 40. Improvements to Nakamura's CalendarService<br />Future work: Store and search calendars using Nakamura's CalendarService<br />Add support for VTODO to CalendarService<br />
  41. 41. Beyond CalendarService<br />Future: Create a full-blown CalDAV Calendar Provider component for Nakamura<br />Store calendars externally, transparently refer to them within Nakamura code<br />
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×