Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Caliper: tapping into the Learn Activity Stream

290 views

Published on

Presentation about Blackboard's development efforts around certification for the IMS Global Caliper standard and how institutions and tool providers can contribute to and tap into the event data - presented at #BbWorld 16 on July 12, 2016

Published in: Education
  • Be the first to comment

  • Be the first to like this

Caliper: tapping into the Learn Activity Stream

  1. 1. Caliper: tapping into the Learn Activity Stream Daniel Knight Product Development @ Blackboard Dan Rinzel Product Manager for Analytics @ Blackboard
  2. 2. Agenda – Q’s we will hope to A today • What is IMS Global, Caliper Learning Analytics, why does it matter and how is Blackboard involved? • What is the Learn activity stream? • What do stream events look like? • How can schools use the stream? • How can a tool provider participate in the stream?
  3. 3. Statements regarding our product development initiatives, including new products and future product upgrades, updates or enhancements represent our current intentions, but may be modified, delayed or abandoned without prior notice and there is no assurance that such offering, upgrades, updates or functionality will become available unless and until they have been made generally available to our customers.
  4. 4. Blackboard & Caliper Work in Progress
  5. 5. Caliper Learning Analytics • IMS Global as an organization • Interoperability Standards – LTI, LIS, Common Cartridge, Extended Transcript, Caliper • Create a viable tool ecosystem for .edu that avoids vendor lock-in • LTI focuses on launch semantics and SSO, includes grade post-backs • Caliper focuses on learning event interoperability, to get away from the data silo problem
  6. 6. Activity Stream: Service Architecture Old School Enterprise approach Record activity in the transactional database and run reports directly, or execute an expensive nightly ETL
  7. 7. Activity Stream: Service Architecture Cloud approach Send atomic events to a separately elastic telemetry service dedicated to collection and rollup
  8. 8. Activity Stream: Event Instrumentation • More events, with better context, reducing ”noise” • Compliant with & extends initial IMS Caliper standard • Available to Learn SaaS deployments and planned for Enterprise 9.1 if hosted with Blackboard • Exploring ways around technical constraints for self- hosted customers
  9. 9. The Goal Integrated insights for teachers into what students are doing across tools and experiences Usage data first, progress data to come
  10. 10. So what does that really mean? A look behind the curtain
  11. 11. Caliper Metric Profiles • Learn SaaS was the first LMS certified against the Caliper 1.0 specification in October 2015, for 5 metric profiles: Session, Assessment, AssessmentItem, Assignable, Outcome • Other metric profiles exist in the spec, specific to types of tools (e.g. MediaEvent, ReaderEvent) The goal is to encapsulate events & context in a shared semantic model so that there is consistency in what can be expected in each event
  12. 12. Caliper Event Syntax • JSON-LD (JSON for Linked Data) • Defines a context and the relationships between involved entities (e.g between a user and an assessment) • Transport-compliant custom events (e.g ForumEvent) • Extensions allow BB to include extra specific data in related entities
  13. 13. Example Stream Event Syntax { "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/ 881b88b1-bd43-41b1-a38e-6c643f1fe54b/ sessions/21622015C7998FEA7A7824F053D514C8" }
  14. 14. Example Stream Event Syntax { "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/ 881b88b1-bd43-41b1-a38e-6c643f1fe54b/ sessions/21622015C7998FEA7A7824F053D514C8" } { "@id": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1- a38e-6c643f1fe54b/users/fd6adc5678514c25bd2569c266c9bf65", "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type":"http://purl.imsglobal.org/caliper/v1/lis/Person", "extensions":{ "bb:user.id":"_50_1", "bb:user.externalId":"demo-39" }, }
  15. 15. Example Stream Event Syntax { "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/ 881b88b1-bd43-41b1-a38e-6c643f1fe54b/ sessions/21622015C7998FEA7A7824F053D514C8" } { "@id": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1- a38e-6c643f1fe54b/gradableItems/_7219_1", "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type”: "http://purl.imsglobal.org/caliper/v1/ AssignableDigitalResource", "name": ”Chapter 1 Test", "dateToSubmit":"2016-07-28T13:04:50.963Z", "maxAttempts":0, "maxSubmits":0, "maxScore":25 }
  16. 16. Example Stream Event Syntax { "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/ 881b88b1-bd43-41b1-a38e-6c643f1fe54b/ sessions/21622015C7998FEA7A7824F053D514C8" } { "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@id": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1- a38e-6c643f1fe54b/gradableItems/_7219_1/attempts/_13000_1", "@type":"http://purl.imsglobal.org/caliper/v1/Attempt", "assignable": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1- a38e-6c643f1fe54b/gradableItems/_7219_1", "actor":"https://caliper- mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1- a38e-6c643f1fe54b/users/fd6adc5678514c25bd2569c266c9bf65", "count":1, "startedAtTime":"2016-06-02T15:25:47.815Z", }
  17. 17. Example Stream Event Syntax { "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/ 881b88b1-bd43-41b1-a38e-6c643f1fe54b/ sessions/21622015C7998FEA7A7824F053D514C8" } { "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@id": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1- a38e-6c643f1fe54b/applications/learn", "@type": "http://purl.imsglobal.org/caliper/v1/SoftwareApplication” }
  18. 18. Example Stream Event Syntax { "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/ 881b88b1-bd43-41b1-a38e-6c643f1fe54b/ sessions/21622015C7998FEA7A7824F053D514C8" } { "@id": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1- a38e-6c643f1fe54b/courses/f85ce1b9dfc740968fd91a38d4b20817", "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/lis/CourseOffering", "extensions":{ "bb:course.id":"_2972_1" }, "courseNumber":"GAT-1", }
  19. 19. Example Stream Event Syntax { "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@type": "http://purl.imsglobal.org/caliper/v1/AssignableEvent", "actor":{ …Person…}, "action": "http://purl.imsglobal.org/vocab/caliper/v1/action#Completed", "object":{ …AssignableDigitalResource…}, "generated”:{ … Attempt…}, "eventTime":"2016-06-02T15:25:47.815Z", "edApp":{ …SoftwareApplication…}, "group":{ …CourseOffering…}, "membership":{…Membership…}, "federatedSession": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/ 881b88b1-bd43-41b1-a38e-6c643f1fe54b/ sessions/21622015C7998FEA7A7824F053D514C8" } { "@context": "http://purl.imsglobal.org/ctx/caliper/v1/Context", "@id": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1- a38e-6c643f1fe54b/courses/f85ce1b9dfc740968fd91a38d4b20817 /members/fd6adc5678514c25bd2569c266c9bf65", "@type":"http://purl.imsglobal.org/caliper/v1/lis/Membership", "extensions":{ "bb:user.externalId":"demo-39", "bb:user.id":"_50_1", "bb:course.id":"_2972_1", "bb:course.externalId":"GAT-1" }, "member": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1- a38e-6c643f1fe54b/users/fd6adc5678514c25bd2569c266c9bf65", "organization": "https://caliper- mapping.cloudbb.blackboard.com/v1/sites/881b88b1-bd43-41b1- a38e-6c643f1fe54b/courses/f85ce1b9dfc740968fd91a38d4b20817", "roles": [http://purl.imsglobal.org/vocab/lis/v2/membership#Learner], "status":"http://purl.imsglobal.org/vocab/lis/v2/status#Active" }
  20. 20. How do we join the ecosystem?
  21. 21. Configuring Learn SaaS to relay events to an external Caliper record store
  22. 22. Enabling the Caliper “handshake” between Learn and an LTI Tool New bb-manifest.xml permission block <permission type="blackboard.platform.blti.CaliperPro fileCallbackPermission" name="*" actions="*"/></permissions>
  23. 23. Event Instrumentation If your system already has event instrumentation: • Map existing event instrumentation to Caliper metric profiles • Transform events to be compliant with the Caliper standard event syntax If you do not currently record events: • Determine what events are meaningful • Investigate the available Caliper Metric Profiles
  24. 24. Caliper Sensor Implementations Standard implementations of the Caliper Sensor are available on GitHub: • Java: https://github.com/IMSGlobal/caliper-java-public • Node.js: https://github.com/IMSGlobal/caliper-js-public • PHP: https://github.com/IMSGlobal/caliper-php-public • Python: https://github.com/IMSGlobal/caliper-python-public • Ruby: https://github.com/IMSGlobal/caliper-ruby-public • .NET: https://github.com/IMSGlobal/caliper-net-public
  25. 25. xAPI & the future There is a similar specification in the world of SCORM – Blackboard is working with both IMS & ADL to drive towards a convergence that we hope will make it easier on everyone in the ecosystem. Other goals for Blackboard & Caliper • Support for Caliper 1.1 Discussion profiles • Certification as a record store when that is available • Convergence or “ambidexterity” • Support for 9.1 Enterprise releases
  26. 26. Questions? Daniel.Knight@blackboard.com Product Development @ Blackboard Dan.Rinzel@blackboard.com Product Manager for Analytics @ Blackboard

×