Copy is a 4 letter word


Published on

This presentation was delivered at the 2012 Blackboard Developer's Conference in New Orleans. It details lessons learned when creating custom content types that survive course copy.

A common Building Block task is to create a new custom content type in Blackboard. The new OpenDatabase allows you to easily create content with much greater functionality and awareness of other parts of Blackboard. You can even tidy up properly afterwards. Surely copying this content wouldn't prove too tricky? I was so wrong. This session explains how Blackboard expect this to happen, names the (undocumented) APIs and walks you through the lifecycle of a custom content item. The open source SignUp List building block is used as a case study.

Published in: Education, Technology
1 Comment
  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Lessons Learned Creating Custom Content Types that Survive Course CopyA common Building Block task is to create a new custom content type in Blackboard. The new OpenDatabase allows you to easily create content with much greater functionality and awareness of other parts of Blackboard. You can even tidy up properly afterwards. Surely copying this content wouldn't prove too tricky? I was so wrong. This session explains how Blackboard expect this to happen, names the (undocumented) APIs and walks you through the lifecycle of a custom content item. The open source SignUp List building block is used as a case study.Starts shallow, but gets deep at the end. If you want to find out how to intercept and shape the Course Copy action, stay tuned…
  • Many building blocks written to extend the functionality of blackboard require defining a customContent type. These fall into two – simple and complex. Simple ones usually survive the course copy process to work again the next year. Complex ones often don’t, complaining of broken links, throwing null pointer exceptions, reporting objects not found or access denied. All in all a very poor show and likely to trigger lots of support calls. Content mangled in the copy process is what I’m terming the Xerox problem.
  • Let’s try and avoid the mangled content now and in the future.The question is how?
  • This presentation will take examine two facets. Why do some content items copy without a problem?We will need to look at the copy process and understand what happens and why.Can we then design these approaches into our building blocks?Equally, can we identify examples where content gets mangles after a copy and establish why?Are there ways we can code a solution, or code round the issue?Are there limitations in the current API? If so can we identify these and submit enhancement request?Are there areas of the documentation that need improved (oh yes!)
  • Start with what works – I’m going to use an example of a custom content item I have been developing with colleagues in the Library. It survives course copy. Accident or design? I’ll leave that for you to decide!
  • An instructor in a Blackboard course can request a Reading List item using a custom building block. This passes the course code (e.g. ANTH1234) to the Library Database which returns the reading list in XML format. The building block parses the XML, displays a list to the user and they then select the item(s) they want to add. Each item is added to the course as a discrete custom content item. The content item contains a link that the user can click to serve them the appropriate library content, e.g. a PDF.Not shown here is the workflow required for edit options – that’s may require a trip back to the database to check the citation is up to date!
  • If a user clicks the content item title, the resulting action is defined in the bb-manifest file. Each content-handler has a view entry – in the simplest case this would take you to a JSP, in this example it uses a servlet. Without you doing anything the servlet is passed two parameters, both derived when the page is rendered: course_id and content_id. As we’ll see in a minute this isn’t quite enough to get to the Resource…In passing note that you can set a can-copy flag in the manifest. If false, this stops instructors copying individual content items to another location in the course (duplication) or to another course. The value you choose here matters if you want to use the CxComponent route to better copying (discussed later).
  • If a user clicks the title link, e.g. “Old man and wolves” in this example, they are taken to the servlet (ViewServlet) and passed two parameters. These are added automatically by the Context - course_id and content_id. We need to obtain two more values to uniquely identify the library resource and so know where to redirect the user. As we have no way of adding other variables to the URL, we need to use the servlet to obtain them. As this is a Content item, a sensible place to store extra data is in the folder associated with this content item in the Course folder on the server (N.B. NOT the Content System). To do this we need to use a bit of code…
  • ViewServlet can use the two variables: course_id and content_id together with a CourseContentFileManager to access the folder we need. Inside this folder we write a small properties file when the content item is created. This stores any data we can’t persist via the Content object itself. The advantage of this location is that itt stays with the content item when it is moved or copied, and is removed when the content is deleted.Tip: don’t use the course_id or content_id as the name of this file!Once we have all four variables we can log access and then using a custom LinkResolver object (not shown here) to redirect the user to the current URL for this item – be it a publisher’s website, a PDF in the eReserve folder, etc. Thus we are only coding what we need, and avoid any volatile data such as Publisher URLs.By using this folder, we can work around the fact that the link by default only passes us the content_id and the course_id.This approach survives move and copying.
  • If instead of clicking on the title, a user clicks the “View Resource” link in the body text (remember body text is static), they are redirected to a custom servlet. To keep things simple for me, it goes to the same servlet that is the link for the title link discussed earlier, but it doesn’t have to be.. This time, as we have more control over the link, the servlet (ViewServlet) is passed all four parameters. Two are derived when the page is rendered (course_id and content_id) using Blackboard’s template variables, two are hard coded and relate to the library resource (and so shouldn’t change).ViewServlet logs access and then redirects the user to the current URL for this item – be it a publisher’s website, a PDF in the eReserve folder, etc. using a custom LinkResolver object not shown here. If this content is copied or moved, the course_id and content_id will be updated to reflect the new values when the page is rendered Warning– Blackboard’s template variables only work in Course Content Items (and portal modules).
  • This approach allows us to copy simple content items and is (I think) how Blackboard expects us to use this.At this point we have approached the edge of the cliff that is “functioning as designed”. We will be jumping off this cliff shortly…
  • Often this approach is not enough. The functional spec requires a more complicated extension, possibly with multiple custom database tables. That’s where the fun really begins.First though we are going to address a question that is often raised when copies go wrong…
  • Copying complicated building blocks can easily cause errors, but understanding why moving items often doesn’t, may provide some useful insight into the reasons for paper-mangling.
  • Let’s begin by creating a simple Content item in a course. What really happens?We specify some information – title, title colour, possibly some text, set availability, etc. – all the usual stuff.
  • Upon persistence, these values are all stored in a table – COURSE_CONTENTSThe primary key to this entry (PK1) is an integer that is passed from page to page in a String representation as the content_id value.We also create a folder on the file system of the server (not in the Content System). This is usually empty. Shown here are two variants for the path on a linux install, which differ depending what you call the database.Note that the file path points to the location we obtained earlier using the blackboard.platform.filesystem.manager.CourseContentFileManagerobject, the folder _1969676_1 is where we stored the extra data (record_id and material_type) in a properties file.
  • If we consider the simplest possibly copy – simply duplicating the item immediately below the original – i.e. same parent location in the same course. The screenshot shows the result.Basically we copy the entire table row, and only update the PK1 (new unique content_id needed) and increment the POSITION value by 1 to show it sits below the original on the page.Note that the DTCREATED and DTMODIFIED fields remain the same! You might want to ask whether they should…Copying the file to another location would change the PARENT_ID (this is essentially the content_id of another place – be it a folder or course menu item). It might also change the value of POSITION.Copying it to another course would mean we’d change the CRSMAIN_PK1 value to match the new course_id (and it would also have a new PARENT_ID) . It might also change the value of POSITION.
  • If we move an item instead of copying, we do not need to create a new table row, we just amend the exiting one.The key point about a move is that the content_id DOES NOT CHANGE – the PK1 value shown here is the same before and after the move.If we move the item to a new location in the same course the PARENT_ID and possibly POSITION will change, but the file system entry will be the same – the URL to the custom folder remains unchanged.If we move the item to another course then the CRSMAIN_PK1 will change, as will the the PARENT_ID and possibly POSITION. In this case the path to the custom folder will change, but the only change will be that the COURSEID (code) value in the path will be updated to reflect that of the new course.Moving often works, because the PK1 (content_id) remains the same, preserving any lookup that rely on this value
  • OK, so lets look at more complicated custom content. Why would they fail and what can we do to help fix that?Many such examples take advantage of the ability to create custom tables in the Blackboard database.
  • I’m going to be using this example for the rest of the presentation. The source code is available for inspection at OSCELOT – go take a look.
  • It exposes a custom content item. Clicking the title link offers a range of extra functions, depending on your role in the course. If you want to know more it is all documented on OSCELOT.
  • One thing that is different about this tool is that the Content item is linked to other objects – CalendarEntries, Tasks, Groups and custom membership lists. These screensots of the create process show this in more detail.Step 1 simply shows content persisted into the COURSE_CONTENT table (though we do manipulate the body text before persisting it to add details of the list size and free places).
  • Everything seen on this page is stored in a custom table – OSLT_SIGNUP_LISTIt uses the content_id (PK1) as the primary key for the entry as it is guaranteed unique and identifies the relationship.If calendarEntry and Task objects are created, we save the corresponding PK1 in the OSLT_SIGNUP_LIST table in case wen need to update them later.
  • Similarly, if the user wants to link the list to a Group (new or existing) then we store the Group’s PK1 in the OSLT_SIGNUP_LIST table
  • The full database structure is defined in the Building Block’s schema object (an XML file). This includes details of primary keys, foreign keys, indexes and any actions to perform if a foreign key is deleted.
  • Here is the structure of the table OSLT_SIGNUP_LISTYou can see the corresponding entries for the Group, two CalendarEntries and a Task.
  • In the first version of the tool, all the data was stored in XML files, this was fragile and frustrating.Now we can store the list of members in a separate table – OSLT_SIGNUP_MEMBERThis has it’s own PK1 but uses the SIGNUP_PK1 (i.e. the content_id) as a foreign keyDelete the parent content item and the entries in OSLT_SIGNUP_LIST and OSLT_SIGNUP_MEMBERare automatically deleted too – nice eh?
  • You can see we also use the USER_PK1 to link to a specific user signed up to the list, and their CU_PK1 to identify the CourseMembership object.Why both – well CourseMemberships are more fragile than users. If someone leaves, it is useful to be sure you know who they are!This gives you a lot of power – you can react when students signed up to existing lists are removed from a course, or indeed if their account is deleted from the server.Note that this may still require some user action – do they want to fill in any gaps? At least now we can recognise these changes and act on them.Deceptively simple looking tasks like checking to see if the lists are full can now utilize the database functionality designed for a multiple transactional model, rather than relying on some horrible lock on a local XML file solutionOK, we can see the benefits of using this sort of approach, but back to the subject of this session: copying…
  • Given that the custom tables use the content_id as their key, you might have hoped that new database table entries would be created when SignUp Lists are copied.Sadly they aren’t. Blackboard copies across the simple content item, but none of the corresponding custom database table entries. As such without further action (code) on our part, the copied SignUp Lists are useless. It gets worse, any linked Tasks or CalendarEntries are also pointing at the wrong thing or just don’t work at all. At this point I decided to put in a support ticket.
  • This was a very carefully worded, supportive and informative reply to my query, it just didn’t help get me out of this hole I’d dug myself into. It seems that what I was doing with content items, they only expected people to do with course tools.It seems that Blackboard have added some listener type objects that intercept attempts to copy course tools and so provide the developer a chance to code their way to a successful copy. Not so the humble custom content item. It’s only a minor simplification of the process to describe it as “Title, colour, body text, goodbye”.  Looks like I was stuck and would have to add a dummy course tool that did nothing except allow copy to proceed
  • We have to add an application-def entry to the manifest.We need to set is-course-tool and is-org-tool to true, so the CxComponent (listener we define in a minute) is triggered on copy (and import and export).Note the handle value used is important – it needs to match that declared at the top of the manifest.Also the required application does nothing via the user interface, we can leave it with no links
  • Even with no link, the tool can appear in the list available in courses.Happily this can be resolved with a quick trip to the Sys Admin panel…
  • To intercept copy, import and export we need to define a custom class that IMPLEMENTS in the manifest.
  • The class that implements CxComponent must override certain methods:getComponentHandle() allows it to identify the relevant content items in the COURSE_CONTENT tablegetApplicationHandle() helps it build ythe URL to any servlets etc. needed
  • We need to specify the name of the items that this class will attempt to copy/import/exportIt would appear to the user before actions such as Exporting a Course, depending on the Usage valueThe Usage value should be one of two types - CONFIGURABLE or ALWAYS.I have selected ALWAYS so it isnever shown in the list to the user and always runs tointercept a copy/import/export even if the user didn’t tick the box in Step 3 – and remember most of us think these are really Content items, not Tools!As an aside, note that the CxComponent is NOT invoked if a user clicks the Copy contextmenu for a content item, this is why the manifest has can-copyset to false.
  • Essentially three use cases – key is the CopyControl/ImportControl/ExportControl object which we look at now:
  • The puzzle we are trying to solve now is to map known data (source course) against unknown – the target course.Specifically we are asking How do we match up the custom table rows used by our extension and identify the new CONTENT_PK1 values to insert as the primary keys?
  • The CopyControl object contains information to help you map items from the old course to the new and provide access to the log.None of these are documented. They are used by Blackboard extensions such as blogs and wikis.Most are fairly self-evident from their name.The isExact() method can be used to differentiate between normal copies (e.g. a copy into another course) and complete copies with usersNever used the copyVTBEText() method so don’t know what it does, or the arguments it takes!Similar methods for export and import (not shown here)ImportControl has a isRestore() method – allows you to differentiate between Imports and Restores – i.e. will it have user?Similarly ExportControl has a isArchive() method – export or archive action?
  • This slide shows the basic logic used to process a copy. Nothing clever – load all the SignUp List objects and then process then one at a time, trying to match them to an entry in the new course and then copy the data.Note this model is a simplification of the actual code used, as the latter has to differentiate between different types of SignUp Lists – a complication we are spared here.
  • The mapping has limits…. It doesn’t work for Groups, Tasks and CalendarEntry objects for example – time for an enhancement request.Other matching currently has to use some horrible best-efforts text-based match (e.g. Group Name).Still a risk of jam…
  • The CxComponent object allows us to intercept copy, import and export, which helps copy custom table entries that use the content_id as a PK1 value.Unfortunately if you link to other Blackboard classes e.g. Tasks, Groups, CalendarEntries. Then you are still stuck rolling your own solution.That still leaves plenty of scope for paper jams.
  • A parting thought…
  • Feel free to contact me using any of the above details.
  • Copy is a 4 letter word

    1. 1. Copy is afour letter word! Dr Malcolm Murray 9th July 2012 10.00 am – 10.45 am
    2. 2. The “Xerox” problem…Shiny new Custom Content in Year 1… …breaks when copied in Year 2 2 Image source:!B9lsCzQEGk~$(KGrHqMOKjsEzIgY)E(wBM6TBcmD!!~~0_35.JPG
    3. 3. Make Custom Content copyable always & forever Image source: 3
    4. 4. How do we get there?Copied MangledUnderstand what works and why Identify the limitationsUse these patterns where we can Avoidance Strategies Work arounds Documentation Enhancement requests 4
    5. 5. Make your Content copyableWhy do “simple” copies survive? 5
    6. 6. The Library Resource model COURSE CONTENT ITEM(S) Reading XML Lists PDF 6
    7. 7. Links from the Title 7
    8. 8. Links from the TitleViewServlet ?course_id=_490_1&content_id=_1424_1 course_id content_id record_id material_type 8
    9. 9. Accessing the Content File Areablackboard.platform.filesystem.manager.CourseContentFileManagerCourseContentFileManager fManager = new CourseContentFileManager();File contentDir = null;try { course_id content_id contentDir = fManager.getRootDirectory(course_id, content_id);} catch (blackboard.platform.filesystem.FileSystemException fE){ // deal with this} catch (java.lang.NullPointerException nE){ // deal with this}File propertiesFile = new File(contentDir, PROPERTIES_FILENAME);// extract the record_id and material_type from the file 9
    10. 10. Links in the Static Body TextViewServlet ?course_id=_490_1&content_id=_1424_1&record_id=b26936641&material_type=c course_id content_id ?course_id=@X@course.pk_string@X@&content_id=@X@content.pk_string@X@ record_id material_type &record_id=b26936641&material_type=c 10
    11. 11. Successful StrategiesCopying Custom Content works if we keep things very simple Use Template variables where possible: ?course_id=@X@course.pk_string@X@&content_id=@X@content.pk_string@X@ Store extra data in the FileSystem: contentDir = fManager.getRootDirectory(course_id, content_id); 11
    12. 12. Image source: is rarely good enough… 12
    13. 13. So why does Move usually work? Image source: 13
    14. 14. Creating a Content Item 14
    15. 15. Creating a Content ItemCOURSE_CONTENTS/local/bboard/blackboard/content/vi/bb_bb60/courses/1/FLOR3100_2011/content/_1969676_1/local/bboard/blackboard/content/vi/BBLEARN/courses/1/FLOR3100_2011/content/_1969676_1 15
    16. 16. Copying a Content Item/local/bboard/blackboard/content/vi/bb_bb60/courses/1/FLOR3100_2011/content/_1969676_1/local/bboard/blackboard/content/vi/bb_bb60/courses/1/FLOR3100_2011/content/_1969 16704_1
    17. 17. Moving a Content Item/local/bboard/blackboard/content/vi/bb_bb60/courses/1/FLOR3100_2011/content/_1969676_1 17
    18. 18. Blackboard OpenDBNeed more, need a database 18
    19. 19. Getting Complicated: SignUp ToolSource code available on OSCELOT 19
    20. 20. Custom Content Item 20
    21. 21. Multiple Links 21
    22. 22. Bespoke Fields 22
    23. 23. Linking to Groups, etc. 23
    24. 24. We need to add a custom schema 24
    25. 25. What the schema bringsSignUp List CONTENT_PK1 GROUP_PK1 CAL_PK1 CAL2_PK1 TASK_PK1 25
    26. 26. Simplifying Membership Lists 26
    27. 27. Link to Users and CourseMemberships SignUp List Member SIGNUP_PK1 (CONTENT_PK1) USER_PK1 CU_PK1<foreign-key name="oslt_signup_member_fk2" reference-table="users" on-delete="cascade”> <columnref name="user_pk1" /></foreign-key> 27
    28. 28. So this will all copy over too, right? Image source: 28
    29. 29. [ insert four-letter word here ]Case Summary: Exception thrown when attempting to copy a courseNote Detail: {Hi Malcolm,The <application value="Bb-wiki"/> tag was added to the manifest to avoid the entries appearing multiple times in the More Tools menu. (LRN-46682-Wikis linkdisplayed twice under tools contextual menu while trying to click on More Tools link)Now, I have had long discussions on this issue so i may wish to call you to get your take on it.So there is a fundamental design difference between the content-type(Content item) and application(tool) concept.Content items (content-type) be simplistic links to either course_content table fields (information held to either Content items (content-type) are really designed to are really designed to be simplistic links against the content_id and anythingcourse_content tableinfields… …orlinks to theto the more(application-defs).tools (application-defs). stored within the [content_id] folder the file system) or links more complex tools complex So when a Content item (content-type) is copied it does a pure copy of the course_contents record and the [content_id] folder and asumes that all refrences areSo whencontent_id and any item (content-type) is copied it does a pure copy of the relative to the a Content links include template variables e.g. recordthat can then be[content_id] folder anddesigned to be athat all references are what Tools (application-defs) create tool items and the linked to from content items. These are asumes more complex container for data and thatsrelativeblogs, wikis etc are tools rather than just basic content items.way of copying/exporting/importing all the data needed for that tool to work in a course. the blackboard.platform.cxComponent works on, allowing a customised This is why, to the content_id and any links include template variables e.g. cxComponent is designed to copy/export/import all the tools items (application-defs) in the operation there isnt a way of copying a single item.Copying the Content items link (content-type) doesnt touch the cxComponent.Copying Content item links (content-type) without the associated tools result in broken links in the new course, this is true of any tool including, Wiki, Blog, andTools (application-defs) create tool items that can then be linked to from content items.collab.These areyour tool does andto idea of SignUp Listscomplex container for adata content-type, especially as it thecustom databaseLooking at what designed the be a more i would consider this a tool rather than simple and thats what has aschema.blackboard.platform.cxComponent works on, allowing a customised way ofcopying/exporting/importing easylike totool by creatingbut iapplicationyour SignUptool hook to the control and course. Thislisted byI have seen some cheats to make a content-type behave the data needed for that tool to work in panel just links to a page thatshows instructions on how to use the tool. This is an all fix a your situation an still think where the Lists are squarely a tool, a should be able to be iswhy, blogs, wikis etc are tools rather groups. just basic content items.the instructor in one place like blogs, wikis, Assignments or even thanI am available to chat about this if you wish. 29
    30. 30. Defining a Tool doing Nothing 30
    31. 31. The Hidden Tool 31
    32. 32. Defining a Custom CxComponent 32
    33. 33. Accessing Handles@Overridepublic String getComponentHandle() { // “resource/” + contentHandle return “resource/signUp-single” ;} @Override public String getApplicationHandle() { // vendorId + "-" + handle return “oslt-signUpList”; } 33
    34. 34. Defining the Usage @Override public String getName() { // might want to localise this  return "SignUp Lists"; } @Override public Usage getUsage() { // either Usage. CONFIGURABLE // or Usage.ALWAYS return Usage.ALWAYS; } 34
    35. 35. Processing content @Override public void doCopy(CopyControl copyControl) { // does the copy back-end magic } @Override public void doImport(ImportControl importControl) { // does the import/restore back-end magic } @Override public void doExport(ExportControl exportControl) { // does the export/archive back-end magic } 35
    36. 36. The ChallengeSource CourseTarget Course How do we match up the table rows and identify the new CONTENT_PK1 ? 36
    37. 37. Controls – spanning old & new getSourceCourseId() : Course Id - source getDestinationCourseId() : Course Id - target lookupIdMapping(Id sourceContentId) : targetContentId copyVTBEText(String ?, Id ?, String ?) : String getLogger() : CxComponentLogger isExact() : boolean – true if Exact Copy (with Users) 37
    38. 38. doCopy() logic 1. Establish the CRS_MAIN_PK1 for the old & new courses CopyControl.getDestinationCourseId() CopyControl.getSourceCourseId() 2. Load all the old SignUpList objects from my custom table using the old CRS_MAIN_PK1 SingleSignUpListLoader.getInstance() .loadByCourseId(CopyControl.getSourceCourseId()); 3. Iterate through this Collection Establish the new CONTENT_ID for each SignUpList lookupIdMapping(sourceContentId) Copy the old table row, swap the CRS_MAIN_PK1 and CONTENT_ID values and persist it SingleSignUpListPersister.persist(newSignUpList); 38
    39. 39. Paper JamSignUp List CONTENT_PK1 ✔ GROUP_PK1 ✗ CAL_PK1 ✗ CAL2_PK1 ✗ TASK_PK1 ✗SignUp List Member SIGNUP_PK1 (CONTENT_PK1) USER_PK1 CU_PK1 39
    40. 40. Still room for improvement Image source:
    41. 41. 41
    42. 42. @malcolmmurray
    43. 43. Slides available on dropbox at We value your feedback! Please fill out a session evaluation. We value documentation!Bb please fill the gaps in the javadocs. 43