Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

[Srijan Wednesday Webinars] Using Drupal as Data Pipeline for Digital Signage

87 views

Published on

Speaker: Mike Madison, Acquia

Digital signs are a completely different creature than standard websites, mobile applications, or really just about anything else you might be building. In particular, delivery mechanisms for digital signage must be perfectly engineered to ensure fast, accurate, and consistent data flow.

This session will be focused on making a data pipeline for digital signage successful. We will discuss available technologies to build a pipeline, testing methodologies, and how to include Drupal in this pipeline.

Key Takeaways:

- Powering digital experiences beyond websites; specifically digital signage
- Evaluating delivery mechanisms like REST and IoT
- Testing with Behat and PHPUnit
- Integrating Drupal into the mix, leveraging capabilities and contrib modules like Workflow, Roles & Permissions, Digital Asset Management etc.

Explore all Srijan webinars: https://www.srijan.net/webinar/past-webinars

Published in: Software
  • Be the first to comment

  • Be the first to like this

[Srijan Wednesday Webinars] Using Drupal as Data Pipeline for Digital Signage

  1. 1. Using Drupal for a Digital Signage Pipeline Mike Madison | Acquia #SrijanWW | @srijan
  2. 2. #SrijanWW | @srijan
  3. 3. #SrijanWW | @srijan
  4. 4. #SrijanWW | @srijan
  5. 5. #SrijanWW | @srijan
  6. 6. #SrijanWW | @srijan About Me Technical Architect @ Acquia Organizer of Drupal GovCon Maintainer of BLT + COD D.O: mikemadison Github: mikemadison13 LinkedIn: mikemadison
  7. 7. #SrijanWW | @srijan Today’s Agenda – Intro to Data Pipelines – Methodologies for building a data pipeline – Testing with Behat / PHPunit – Integrating Drupal and its features / capabilities – Powering Digital Experiences (broadly)
  8. 8. Today’s Talk – Advanced Topic… Intermediate Level – Architectural Discussion #SrijanWW | @srijan
  9. 9. Disclaimer: Technology #SrijanWW | @srijan
  10. 10. #SrijanWW | @srijan Image Source: http://www.nydailynews.com/
  11. 11. BUZZWORDS ● Decoupled ● Headless ● Digital Experience #SrijanWW | @srijan
  12. 12. Why do you need a Pipeline? “T , .” #SrijanWW | @srijan
  13. 13. What Does a Pipeline Do? ● Pushes Data Automatically ● Limits Human Interaction ● Streamlines Data Collection ● Broadcasts to Multiple Location #SrijanWW | @srijan
  14. 14. What Does a Pipeline Do? ● Limits what/who can interact with system ● Provides fault tolerance #SrijanWW | @srijan
  15. 15. ©2018 Acquia Inc. — Confidential and Proprietary #SrijanWW | @srijan
  16. 16. ©2018 Acquia Inc. — Confidential and Proprietary #SrijanWW | @srijan
  17. 17. ©2018 Acquia Inc. — Confidential and Proprietary Methodologies for Building a Data Pipeline #SrijanWW | @srijan
  18. 18. ©2018 Acquia Inc. — Confidential and Proprietary Methodologies for Building a Data Pipeline 1. What is coming into the pipeline? 2. What is the pipeline powering? 3. How is the <whatever> displaying the <whatever>? #SrijanWW | @srijan
  19. 19. ©2018 Acquia Inc. — Confidential and Proprietary Data #SrijanWW | @srijan
  20. 20. ©2018 Acquia Inc. — Confidential and Proprietary – What is it? – How often does it update? – Are there failover URLs? – What format is it in? – Do you have to authenticate to get to it? – How big is the data dump? – Are there API limitations / requirements? Details #SrijanWW | @srijan
  21. 21. ©2018 Acquia Inc. — Confidential and Proprietary Example 1. What is the data -- Mass Transit Data 2. How often does it update -- Every few seconds 3. Are there failover URLs -- Yes (multiple) 4. What format is it in -- JSON 5. Do you have to authenticate to get to it -- Yes 6. How big is the data dump - <10mb #SrijanWW | @srijan
  22. 22. ©2018 Acquia Inc. — Confidential and Proprietary Example Impact 1. Mass Transit Data NOT something you can screw up (people might die) 2. Every few seconds, <10mb, JSON FAST processing time in a standard format (small) 3. Multiple URLS, Authenticated Requests Secure + Robust #SrijanWW | @srijan
  23. 23. ©2018 Acquia Inc. — Confidential and Proprietary Digital Experiences #SrijanWW | @srijan
  24. 24. ©2018 Acquia Inc. — Confidential and Proprietary – What is the Digital Experience? – Can people interact with it? – Is there any personalization / contextualization? – How often does it need to update? – Is it language dependent? – Is it ADA compliant? Details #SrijanWW | @srijan
  25. 25. ©2018 Acquia Inc. — Confidential and Proprietary – What is the Digital Experience? Arrival Sign – Can people interact with it? No – Is there any personalization / contextualization? Contextualization based on Physical Location – How often does it need to update? Every 5-8 secs – Is it language dependent? No – Is it ADA compliant? Yes Example #SrijanWW | @srijan
  26. 26. ©2018 Acquia Inc. — Confidential and Proprietary – Arrival Sign, Every 5-8 secs Old data not useful (real world evidence if right / wrong) – Contextualization based on Physical Location Sign must display appropriate location data – No Interaction Sign has to just “work” Impact #SrijanWW | @srijan
  27. 27. ©2018 Acquia Inc. — Confidential and Proprietary – Not language dependent BUT IT COULD BE – ADA Compliant Heavily limits how much “stuff” can be on the screen at any one time Impact #SrijanWW | @srijan
  28. 28. ©2018 Acquia Inc. — Confidential and Proprietary Front End #SrijanWW | @srijan
  29. 29. ©2018 Acquia Inc. — Confidential and Proprietary Cool! Now what? #SrijanWW | @srijan
  30. 30. ©2018 Acquia Inc. — Confidential and Proprietary Architecting a Solution – Pipeline: – Acquire Data – Organize Data – Send data #SrijanWW | @srijan
  31. 31. ©2018 Acquia Inc. — Confidential and Proprietary Architecting a Solution – Context: – Structure for organizing??? #SrijanWW | @srijan
  32. 32. ©2018 Acquia Inc. — Confidential and Proprietary Architecting a Solution – Communication with Devices: – How do you keep devices connected? – How do you get data from the pipeline to the device? #SrijanWW | @srijan
  33. 33. ©2018 Acquia Inc. — Confidential and Proprietary Pipeline: Data Acquisition – Authenticate – Get Data From <source> – Get Data from <other source> – Validate Data #SrijanWW | @srijan
  34. 34. ©2018 Acquia Inc. — Confidential and Proprietary Pipeline: Categorize Data – Isolate Data for Each <whatever> – Break Up Data Into Smaller Payloads #SrijanWW | @srijan
  35. 35. ©2018 Acquia Inc. — Confidential and Proprietary Pipeline: Sender – Authenticate Into Appropriate Service – Send the Data #SrijanWW | @srijan
  36. 36. ©2018 Acquia Inc. — Confidential and Proprietary Pipeline #SrijanWW | @srijan
  37. 37. ©2018 Acquia Inc. — Confidential and Proprietary Pipeline #SrijanWW | @srijan
  38. 38. ©2018 Acquia Inc. — Confidential and Proprietary Pipeline #SrijanWW | @srijan
  39. 39. ©2018 Acquia Inc. — Confidential and Proprietary Lots of Signs Means... https://www.flickr.com/photos/83428770@N06/ #SrijanWW | @srijan
  40. 40. ©2018 Acquia Inc. — Confidential and Proprietary Lots of Little Details ● Size / Orientation ● Physical Location ● Facing Direction ● IP Address #SrijanWW | @srijan
  41. 41. ©2018 Acquia Inc. — Confidential and Proprietary OK Fine, Let’s Use Drupal – Drupal can manage this structure very easily #SrijanWW | @srijan
  42. 42. ©2018 Acquia Inc. — Confidential and Proprietary Remember... – Workflow – Permissions / Roles – WYSIWYG / Easy to Use Forms – Contrib – Security – Media – Views / Reporting Drupal as a Framework #SrijanWW | @srijan
  43. 43. ©2018 Acquia Inc. — Confidential and Proprietary Content Types to Define Relationships - Route Groups - Routes - Stations - Platforms - Signs #SrijanWW | @srijan
  44. 44. ©2018 Acquia Inc. — Confidential and Proprietary Content Types to Define Relationships - Route Groups - Routes - Stations - Platforms - Signs #SrijanWW | @srijan
  45. 45. ©2018 Acquia Inc. — Confidential and Proprietary Entity Forms to Capture Relevant Info #SrijanWW | @srijan
  46. 46. ©2018 Acquia Inc. — Confidential and Proprietary Configuration Provider - Sign Config - Dynamic Updates to System #SrijanWW | @srijan
  47. 47. ©2018 Acquia Inc. — Confidential and Proprietary Pipeline #SrijanWW | @srijan
  48. 48. ©2018 Acquia Inc. — Confidential and Proprietary Categorizer ● Markers in Data + Drupal provided Entities ● Can be updated on the fly #SrijanWW | @srijan
  49. 49. ©2018 Acquia Inc. — Confidential and Proprietary Performance #SrijanWW | @srijan
  50. 50. ©2018 Acquia Inc. — Confidential and Proprietary Pipeline #SrijanWW | @srijan
  51. 51. ©2018 Acquia Inc. — Confidential and Proprietary Categorizer Requires Drupal ● Drupal has many moving parts ● Lots of structure / config leads to big / complex queries ● Big / Complex queries are SLOW #SrijanWW | @srijan
  52. 52. ©2018 Acquia Inc. — Confidential and Proprietary Cache EVERYTHING ● Pipeline accesses the cache ● Cache is regenerated only as needed ● Cache could be: ○ File Based ○ DB Based ○ Memcache Based #SrijanWW | @srijan
  53. 53. ©2018 Acquia Inc. — Confidential and Proprietary Did Anything Change??? ● Once Data is categorized… should we send it? #SrijanWW | @srijan
  54. 54. ©2018 Acquia Inc. — Confidential and Proprietary How Do You Show Data On A Sign? #SrijanWW | @srijan
  55. 55. ©2018 Acquia Inc. — Confidential and Proprietary Transmission #SrijanWW | @srijan
  56. 56. ©2018 Acquia Inc. — Confidential and Proprietary How Do You Show Data On A Sign? #SrijanWW | @srijan
  57. 57. ©2018 Acquia Inc. — Confidential and Proprietary Sending Data – Normal Websites: – Requests occur as user’s navigate – New data on page load – Asynchronous communication #SrijanWW | @srijan
  58. 58. ©2018 Acquia Inc. — Confidential and Proprietary Sending Data – For Signs: – Persistent Connection – Fast – Lightweight #SrijanWW | @srijan
  59. 59. ©2018 Acquia Inc. — Confidential and Proprietary MQTT Message Queuing Telemetry Transport “publish-subscribe-based messaging protocol. It works on top of the TCP/IP protocol. It is designed for connections with remote locations where a "small code footprint" is required or the network bandwidth is limited. The publish-subscribe messaging pattern requires a message broker.” https://en.wikipedia.org/wiki/MQTT #SrijanWW | @srijan
  60. 60. ©2018 Acquia Inc. — Confidential and Proprietary Image Source: https://www.survivingwithandroid.com #SrijanWW | @srijan
  61. 61. ©2018 Acquia Inc. — Confidential and Proprietary #SrijanWW | @srijan
  62. 62. ©2018 Acquia Inc. — Confidential and Proprietary How Do You Show Data On A Sign? #SrijanWW | @srijan
  63. 63. ©2018 Acquia Inc. — Confidential and Proprietary Testing Plan (TDD) #SrijanWW | @srijan
  64. 64. ©2018 Acquia Inc. — Confidential and Proprietary Acquia Build and Launch Tool (BLT) https://blt.readthedocs.io #SrijanWW | @srijan
  65. 65. ©2018 Acquia Inc. — Confidential and Proprietary Unit Testing PHPUnit – Ensures functionality at a method level – Ensures classes / components function in a vacuum #SrijanWW | @srijan
  66. 66. ©2018 Acquia Inc. — Confidential and Proprietary (Not) Unit Testing PHPUnit – Bootstrap Drupal / Drush – Execute Scripts / Caching #SrijanWW | @srijan
  67. 67. ©2018 Acquia Inc. — Confidential and Proprietary Functional Testing Behat – Ensure fields / entities are functional – Test roles / permissions / workflows #SrijanWW | @srijan
  68. 68. ©2018 Acquia Inc. — Confidential and Proprietary Pipeline Testing Behat - Use Feature Context to execute elements of Pipeline - Send data to IOT based on test script - Retrieve data from IOT and compare with expected value(s) #SrijanWW | @srijan
  69. 69. ©2018 Acquia Inc. — Confidential and Proprietary SLOW but FIERCE Automated tests take ~20 minutes… 1. Communication with IOT 2. Results of Pipeline for ALL Data Providers 3. Queue Runners’ Functionality 4. The usual unit + functional testing #SrijanWW | @srijan
  70. 70. ©2018 Acquia Inc. — Confidential and Proprietary Pipeline #SrijanWW | @srijan
  71. 71. ©2018 Acquia Inc. — Confidential and Proprietary Not much of a Drupal talk... #SrijanWW | @srijan
  72. 72. ©2018 Acquia Inc. — Confidential and Proprietary Pipeline #SrijanWW | @srijan
  73. 73. ©2018 Acquia Inc. — Confidential and Proprietary Drupal as a Data Provider – Workflow – Permissions / Roles – WYSIWYG / Easy to Use Forms – Contrib – Security – Media – Views / Reporting Drupal as a CMS #SrijanWW | @srijan
  74. 74. ©2018 Acquia Inc. — Confidential and Proprietary Questions #SrijanWW | @srijan

×