Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Boundary Front end tech talk: how it works


Published on

Published in: Technology, Business
  • Be the first to comment

Boundary Front end tech talk: how it works

  1. 1. Tech Talk - FrontendStephen Boak ( Mahoney ( King (
  2. 2. When we say real-time we mean it: JSON beingpushed to the browser once a second andvisualizations built to constantly move and adaptto streaming data
  3. 3. Boundary | Architecture
  4. 4. Meters ! Lightweight and highly scalable, these sit on the cloud as well as private data centers across virtual or physical servers.
  5. 5. Intercept Meter data via TransportCollectors Layer Security (TLS) Authentication.
  6. 6. Collect all of the data at high resolution to gain rich insight into complexData Store environments & problems before they can impact critical business services.
  7. 7. Real-time continuous data streaming atStreaming high resolution, low latency intervals.
  8. 8. Stream data to your dashboard with sub-second latency,Streaming UI providing you with intuitive, powerful dashboard.
  9. 9. How We Get Data• CometD server (streaker)!• Pulls in data from multiple backend services and streams it out to clients!• Can also aggregate and filter data on demand
  10. 10. Subscriptions 1 2 3 4 5 Make a Get back a Subscribe to Receive Receivesubscription request subscription ID new query state-dump add/remove (query and filters) (Query ID) (unique channel) (inserts & schema) messages !! ! ! ! !
  11. 11. Data Structure• Schema and key (which fields are concatenated for each record)• State dump with schema for requested time window (up to 100kb state dumps)• Save bandwidth with N-tuples indexed by the schema• Adds/removes update the state with keys (subset of schema)
  12. 12. JSONExample port:protocol subscription State Object{ { channel:""/query/ac3941b8924a2f73/custom9 "1331234598000:80:6":"{ query92f536bbc93f159418b98f1a973b5dc4e78379 "epochMillis":=1331234598000, filter979564d6c94c9394f479abae906ff95a3411e9 "portProtocol":="80:6", aggregation9b5cc18d49098d949c29a08497037b08a30e6" "ingressPackets":"100, data:"{ "ingressOctets":"8463, insert:"[[1331234598000,"80:6,"100,"8463,"93," "egressPackets":"93, 9672],"[1331234925000,"8080:6,"242,"39657,"243," "egressOctets":"9672 40865],"…], }, keys:"[epochMillis,"portProtocol], "1331234925000:8080:6":"{ schema:"[epochMillis,"portProtocol,"ingressPackets," "epochMillis":=1331234925000, ingressOctets,"egressPackets,"egressOctets], "portProtocol":="8080:6", timestamp:"1331235119001 "ingressPackets":"242, } "ingressOctets":"39657,} "egressPackets":"243, "egressOctets":"40865 } … }
  13. 13. DataSource.js• Negotiates between multiple data sources and subscribers on a page Subscriptions• One data source for N subscribers Data Sources• Abstracts CometD subscription process for JS development Subscribers• Smart enough to resubscribe and notifies subscribers (so that state dumps aren’t doubled up)
  14. 14. bndry.datasource repo//"Receives"object"with"properties"for"state,"inserts"and"removes."function"updateData(data)"{"" console.log(data.added);"}"//"Create"a"handle"to"a"new"data"source"var"volumeDataSource"="bndry.dataSource(volume_1s);"//"Add"a"subscriber"and"start"getting"updates."volumeDataSource.addSubscriber(updateData);
  15. 15. Demo Time
  16. 16. Data and Subcription Problems• Monolithic, Multi-Purpose Queries• Large State Dumps cause CometD timeout (5-10MBs per second)• No Resubscription• Aggregated all data on front-end
  17. 17. Data and Subcription Solutions• Stratified, Filterable Queries and server-side aggregation• Top-N Limitations (constrained output)• Resubscription & filtering• Web workers haven’t helped much• Time smoothing (1-second ticks)
  18. 18. VisualizationDOM manipulation is expensive is tweening SVG Canvas
  19. 19. What’s Next• WebSockets• Historical Data and long term data storage• HTML5 local storage (store data)• Machine-learning for real-time network visualization
  20. 20. Thank You