SlideShare a Scribd company logo
Tech Talk - Frontend
Stephen Boak (steve@boundary.com)
Mark Mahoney (mark@boundary.com)
Matt King (m@boundary.com)
When we say real-time we mean it: JSON being
pushed to the browser once a second and
visualizations built to constantly move and adapt
to streaming data
Boundary | Architecture
Meters                                     !
         Lightweight and highly scalable, these sit on the cloud as well as
         private data centers across virtual or physical servers.
Intercept Meter data via Transport
Collectors   Layer Security (TLS) Authentication.
Collect all of the data at high resolution to gain rich insight into complex
Data Store   environments & problems before they can impact critical business services.
Real-time continuous data streaming at
Streaming   high resolution, low latency intervals.
Stream data to your dashboard with sub-second latency,
Streaming UI   providing you with intuitive, powerful dashboard.
How We Get Data

• CometD server (streaker)!

• Pulls in data from multiple backend services and
  streams it out to clients!

• Can also aggregate and filter data on demand
Subscriptions
         1                   2                  3                   4                5

       Make a            Get back a        Subscribe to           Receive          Receive
subscription request   subscription ID      new query           state-dump       add/remove
  (query and filters)     (Query ID)      (unique channel)   (inserts & schema)    messages




                                                                                   !!
       !




                                              !
                           !




                                                                 !
Data Structure

• Schema and key
  (which fields are concatenated for each record)

• State dump with schema for requested time window
  (up to 100kb state dumps)

• Save bandwidth with N-tuples indexed by the schema

• Adds/removes update the state with keys (subset of schema)
JSON
Example port:protocol subscription                           State Object
{                                                            {
    channel:""/query/ac3941b8924a2f73/custom9                    "1331234598000:80:6":"{
    query92f536bbc93f159418b98f1a973b5dc4e78379                    "epochMillis":=1331234598000,
    filter979564d6c94c9394f479abae906ff95a3411e9                   "portProtocol":="80:6",
    aggregation9b5cc18d49098d949c29a08497037b08a30e6"              "ingressPackets":"100,
    data:"{                                                        "ingressOctets":"8463,
      insert:"[[1331234598000,"80:6,"100,"8463,"93,"               "egressPackets":"93,
      9672],"[1331234925000,"8080:6,"242,"39657,"243,"             "egressOctets":"9672
      40865],"…],                                                },
      keys:"[epochMillis,"portProtocol],                         "1331234925000:8080:6":"{
      schema:"[epochMillis,"portProtocol,"ingressPackets,"         "epochMillis":=1331234925000,
      ingressOctets,"egressPackets,"egressOctets],                 "portProtocol":="8080:6",
      timestamp:"1331235119001                                     "ingressPackets":"242,
    }                                                              "ingressOctets":"39657,
}                                                                  "egressPackets":"243,
                                                                   "egressOctets":"40865
                                                                 }
                                                                 …
                                                             }
DataSource.js
• Negotiates between multiple data
  sources and subscribers on a page        Subscriptions

• One data source for N subscribers                Data
                                                Sources

• Abstracts CometD subscription
  process for JS development
                                            Subscribers
• Smart enough to resubscribe and
  notifies subscribers
 (so that state dumps aren’t doubled up)
bndry.datasource repo

//"Receives"object"with"properties"for"state,"inserts"and"removes."
function"updateData(data)"{"
" console.log(data.added);"
}"

//"Create"a"handle"to"a"new"data"source"
var"volumeDataSource"="bndry.dataSource('volume_1s');"

//"Add"a"subscriber"and"start"getting"updates."
volumeDataSource.addSubscriber(updateData);
Demo Time
Data and Subcription Problems

• Monolithic, Multi-Purpose Queries
• Large State Dumps cause CometD timeout
  (5-10MBs per second)
• No Resubscription
• Aggregated all data on front-end
Data and Subcription Solutions

• Stratified, Filterable Queries and server-side aggregation
• Top-N Limitations
  (constrained output)
• Resubscription & filtering
• Web workers haven’t helped much
• Time smoothing (1-second ticks)
Visualization

DOM manipulation is expensive         ...so is tweening




                                SVG                       Canvas
What’s Next

• WebSockets

• Historical Data and long term data storage

• HTML5 local storage (store data)

• Machine-learning for real-time network visualization
Thank You

More Related Content

What's hot

Mongo db – document oriented database
Mongo db – document oriented databaseMongo db – document oriented database
Mongo db – document oriented databaseWojciech Sznapka
 
MongoDB - An Introduction
MongoDB - An IntroductionMongoDB - An Introduction
MongoDB - An Introduction
sethfloydjr
 
Data day texas: Cassandra and the Cloud
Data day texas: Cassandra and the CloudData day texas: Cassandra and the Cloud
Data day texas: Cassandra and the Cloud
jbellis
 
Real-Time Integration Between MongoDB and SQL Databases
Real-Time Integration Between MongoDB and SQL Databases Real-Time Integration Between MongoDB and SQL Databases
Real-Time Integration Between MongoDB and SQL Databases MongoDB
 
MongoDB + Spring
MongoDB + SpringMongoDB + Spring
MongoDB + Spring
Norberto Leite
 
Conceptos básicos. Seminario web 5: Introducción a Aggregation Framework
Conceptos básicos. Seminario web 5: Introducción a Aggregation FrameworkConceptos básicos. Seminario web 5: Introducción a Aggregation Framework
Conceptos básicos. Seminario web 5: Introducción a Aggregation Framework
MongoDB
 
Data Management 2: Conquering Data Proliferation
Data Management 2: Conquering Data ProliferationData Management 2: Conquering Data Proliferation
Data Management 2: Conquering Data Proliferation
MongoDB
 
Mongo DB: Fundamentals & Basics/ An Overview of MongoDB/ Mongo DB tutorials
Mongo DB: Fundamentals & Basics/ An Overview of MongoDB/ Mongo DB tutorialsMongo DB: Fundamentals & Basics/ An Overview of MongoDB/ Mongo DB tutorials
Mongo DB: Fundamentals & Basics/ An Overview of MongoDB/ Mongo DB tutorials
SpringPeople
 
Data Management 3: Bulletproof Data Management
Data Management 3: Bulletproof Data ManagementData Management 3: Bulletproof Data Management
Data Management 3: Bulletproof Data Management
MongoDB
 
Back to Basics, webinar 2: La tua prima applicazione MongoDB
Back to Basics, webinar 2: La tua prima applicazione MongoDBBack to Basics, webinar 2: La tua prima applicazione MongoDB
Back to Basics, webinar 2: La tua prima applicazione MongoDB
MongoDB
 
Conceptos básicos. Seminario web 4: Indexación avanzada, índices de texto y g...
Conceptos básicos. Seminario web 4: Indexación avanzada, índices de texto y g...Conceptos básicos. Seminario web 4: Indexación avanzada, índices de texto y g...
Conceptos básicos. Seminario web 4: Indexación avanzada, índices de texto y g...
MongoDB
 
Yes, Sql!
Yes, Sql!Yes, Sql!
Yes, Sql!
Uri Cohen
 
Back to Basics Webinar 2: Your First MongoDB Application
Back to Basics Webinar 2: Your First MongoDB ApplicationBack to Basics Webinar 2: Your First MongoDB Application
Back to Basics Webinar 2: Your First MongoDB Application
MongoDB
 
Tms training
Tms trainingTms training
Tms training
Chi Lee
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
Mike Dirolf
 
JSONiq - The SQL of NoSQL
JSONiq - The SQL of NoSQLJSONiq - The SQL of NoSQL
JSONiq - The SQL of NoSQL
William Candillon
 
Lecture 40 1
Lecture 40 1Lecture 40 1
Lecture 40 1
patib5
 
Druid realtime indexing
Druid realtime indexingDruid realtime indexing
Druid realtime indexing
Seoeun Park
 
Mongo db
Mongo dbMongo db
Mongo db
Noman Ellahi
 

What's hot (20)

Mongo db – document oriented database
Mongo db – document oriented databaseMongo db – document oriented database
Mongo db – document oriented database
 
MongoDB - An Introduction
MongoDB - An IntroductionMongoDB - An Introduction
MongoDB - An Introduction
 
Data day texas: Cassandra and the Cloud
Data day texas: Cassandra and the CloudData day texas: Cassandra and the Cloud
Data day texas: Cassandra and the Cloud
 
Real-Time Integration Between MongoDB and SQL Databases
Real-Time Integration Between MongoDB and SQL Databases Real-Time Integration Between MongoDB and SQL Databases
Real-Time Integration Between MongoDB and SQL Databases
 
MongoDB + Spring
MongoDB + SpringMongoDB + Spring
MongoDB + Spring
 
Conceptos básicos. Seminario web 5: Introducción a Aggregation Framework
Conceptos básicos. Seminario web 5: Introducción a Aggregation FrameworkConceptos básicos. Seminario web 5: Introducción a Aggregation Framework
Conceptos básicos. Seminario web 5: Introducción a Aggregation Framework
 
Data Management 2: Conquering Data Proliferation
Data Management 2: Conquering Data ProliferationData Management 2: Conquering Data Proliferation
Data Management 2: Conquering Data Proliferation
 
Mongo DB: Fundamentals & Basics/ An Overview of MongoDB/ Mongo DB tutorials
Mongo DB: Fundamentals & Basics/ An Overview of MongoDB/ Mongo DB tutorialsMongo DB: Fundamentals & Basics/ An Overview of MongoDB/ Mongo DB tutorials
Mongo DB: Fundamentals & Basics/ An Overview of MongoDB/ Mongo DB tutorials
 
Data Management 3: Bulletproof Data Management
Data Management 3: Bulletproof Data ManagementData Management 3: Bulletproof Data Management
Data Management 3: Bulletproof Data Management
 
Back to Basics, webinar 2: La tua prima applicazione MongoDB
Back to Basics, webinar 2: La tua prima applicazione MongoDBBack to Basics, webinar 2: La tua prima applicazione MongoDB
Back to Basics, webinar 2: La tua prima applicazione MongoDB
 
Conceptos básicos. Seminario web 4: Indexación avanzada, índices de texto y g...
Conceptos básicos. Seminario web 4: Indexación avanzada, índices de texto y g...Conceptos básicos. Seminario web 4: Indexación avanzada, índices de texto y g...
Conceptos básicos. Seminario web 4: Indexación avanzada, índices de texto y g...
 
Yes, Sql!
Yes, Sql!Yes, Sql!
Yes, Sql!
 
Back to Basics Webinar 2: Your First MongoDB Application
Back to Basics Webinar 2: Your First MongoDB ApplicationBack to Basics Webinar 2: Your First MongoDB Application
Back to Basics Webinar 2: Your First MongoDB Application
 
Tms training
Tms trainingTms training
Tms training
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
 
Mongo db basics
Mongo db basicsMongo db basics
Mongo db basics
 
JSONiq - The SQL of NoSQL
JSONiq - The SQL of NoSQLJSONiq - The SQL of NoSQL
JSONiq - The SQL of NoSQL
 
Lecture 40 1
Lecture 40 1Lecture 40 1
Lecture 40 1
 
Druid realtime indexing
Druid realtime indexingDruid realtime indexing
Druid realtime indexing
 
Mongo db
Mongo dbMongo db
Mongo db
 

Viewers also liked

Boundary for puppet @ puppet conf2012
Boundary for puppet @ puppet conf2012Boundary for puppet @ puppet conf2012
Boundary for puppet @ puppet conf2012
Boundary
 
Distributed systems-radiology
Distributed systems-radiologyDistributed systems-radiology
Distributed systems-radiology
Boundary
 
Invoke dynamic your api to hotspot
Invoke dynamic your api to hotspotInvoke dynamic your api to hotspot
Invoke dynamic your api to hotspotBoundary
 
Cad cam
Cad camCad cam
Cad cam
sateesh jala
 
PDM - Precedence Diagram Method
PDM - Precedence Diagram MethodPDM - Precedence Diagram Method
PDM - Precedence Diagram Method
phantai24
 
Solid modelling cg
Solid modelling cgSolid modelling cg
Solid modelling cg
Nareek
 
Introduction to solid modeling
Introduction to solid modelingIntroduction to solid modeling
Introduction to solid modeling
*noT yeT workinG! !M stilL studyinG*
 
Solid modeling
Solid modelingSolid modeling
Solid modelingKRvEsL
 

Viewers also liked (8)

Boundary for puppet @ puppet conf2012
Boundary for puppet @ puppet conf2012Boundary for puppet @ puppet conf2012
Boundary for puppet @ puppet conf2012
 
Distributed systems-radiology
Distributed systems-radiologyDistributed systems-radiology
Distributed systems-radiology
 
Invoke dynamic your api to hotspot
Invoke dynamic your api to hotspotInvoke dynamic your api to hotspot
Invoke dynamic your api to hotspot
 
Cad cam
Cad camCad cam
Cad cam
 
PDM - Precedence Diagram Method
PDM - Precedence Diagram MethodPDM - Precedence Diagram Method
PDM - Precedence Diagram Method
 
Solid modelling cg
Solid modelling cgSolid modelling cg
Solid modelling cg
 
Introduction to solid modeling
Introduction to solid modelingIntroduction to solid modeling
Introduction to solid modeling
 
Solid modeling
Solid modelingSolid modeling
Solid modeling
 

Similar to Boundary Front end tech talk: how it works

Real-Time Streaming: Move IMS Data to Your Cloud Data Warehouse
Real-Time Streaming: Move IMS Data to Your Cloud Data WarehouseReal-Time Streaming: Move IMS Data to Your Cloud Data Warehouse
Real-Time Streaming: Move IMS Data to Your Cloud Data Warehouse
Precisely
 
Streaming Operational Data with MariaDB MaxScale
Streaming Operational Data with MariaDB MaxScaleStreaming Operational Data with MariaDB MaxScale
Streaming Operational Data with MariaDB MaxScale
MariaDB plc
 
Jump Start with Apache Spark 2.0 on Databricks
Jump Start with Apache Spark 2.0 on DatabricksJump Start with Apache Spark 2.0 on Databricks
Jump Start with Apache Spark 2.0 on Databricks
Databricks
 
From Kafka to BigQuery - Strata Singapore
From Kafka to BigQuery - Strata SingaporeFrom Kafka to BigQuery - Strata Singapore
From Kafka to BigQuery - Strata Singapore
Ofir Sharony
 
Materials Project Validation, Provenance, and Sandboxes by Dan Gunter
Materials Project Validation, Provenance, and Sandboxes by Dan GunterMaterials Project Validation, Provenance, and Sandboxes by Dan Gunter
Materials Project Validation, Provenance, and Sandboxes by Dan Gunter
Dan Gunter
 
Real-Time Big Data with Storm, Kafka and GigaSpaces
Real-Time Big Data with Storm, Kafka and GigaSpacesReal-Time Big Data with Storm, Kafka and GigaSpaces
Real-Time Big Data with Storm, Kafka and GigaSpaces
Oleksii Diagiliev
 
Making sense of your data jug
Making sense of your data   jugMaking sense of your data   jug
Making sense of your data jug
Gerald Muecke
 
Realtime Analytics on AWS
Realtime Analytics on AWSRealtime Analytics on AWS
Realtime Analytics on AWS
Sungmin Kim
 
High Performance, Scalable MongoDB in a Bare Metal Cloud
High Performance, Scalable MongoDB in a Bare Metal CloudHigh Performance, Scalable MongoDB in a Bare Metal Cloud
High Performance, Scalable MongoDB in a Bare Metal CloudMongoDB
 
Chicago Kafka Meetup
Chicago Kafka MeetupChicago Kafka Meetup
Chicago Kafka Meetup
Cliff Gilmore
 
Agility and Scalability with MongoDB
Agility and Scalability with MongoDBAgility and Scalability with MongoDB
Agility and Scalability with MongoDB
MongoDB
 
Big Data Solutions in Azure - David Giard
Big Data Solutions in Azure - David GiardBig Data Solutions in Azure - David Giard
Big Data Solutions in Azure - David Giard
ITCamp
 
Data Analytics Service Company and Its Ruby Usage
Data Analytics Service Company and Its Ruby UsageData Analytics Service Company and Its Ruby Usage
Data Analytics Service Company and Its Ruby Usage
SATOSHI TAGOMORI
 
Spark Summit - Stratio Streaming
Spark Summit - Stratio Streaming Spark Summit - Stratio Streaming
Spark Summit - Stratio Streaming
Stratio
 
Riga dev day: Lambda architecture at AWS
Riga dev day: Lambda architecture at AWSRiga dev day: Lambda architecture at AWS
Riga dev day: Lambda architecture at AWS
Antons Kranga
 
Introduce Apache Cassandra - JavaTwo Taiwan, 2012
Introduce Apache Cassandra - JavaTwo Taiwan, 2012Introduce Apache Cassandra - JavaTwo Taiwan, 2012
Introduce Apache Cassandra - JavaTwo Taiwan, 2012Boris Yen
 
Enabling Key Business Advantage from Big Data through Advanced Ingest Process...
Enabling Key Business Advantage from Big Data through Advanced Ingest Process...Enabling Key Business Advantage from Big Data through Advanced Ingest Process...
Enabling Key Business Advantage from Big Data through Advanced Ingest Process...
StampedeCon
 
Spark + Cassandra = Real Time Analytics on Operational Data
Spark + Cassandra = Real Time Analytics on Operational DataSpark + Cassandra = Real Time Analytics on Operational Data
Spark + Cassandra = Real Time Analytics on Operational Data
Victor Coustenoble
 
Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...
Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...
Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...
Databricks
 
Building a Scalable Real-Time Fleet Management IoT Data Tracker with Kafka St...
Building a Scalable Real-Time Fleet Management IoT Data Tracker with Kafka St...Building a Scalable Real-Time Fleet Management IoT Data Tracker with Kafka St...
Building a Scalable Real-Time Fleet Management IoT Data Tracker with Kafka St...
HostedbyConfluent
 

Similar to Boundary Front end tech talk: how it works (20)

Real-Time Streaming: Move IMS Data to Your Cloud Data Warehouse
Real-Time Streaming: Move IMS Data to Your Cloud Data WarehouseReal-Time Streaming: Move IMS Data to Your Cloud Data Warehouse
Real-Time Streaming: Move IMS Data to Your Cloud Data Warehouse
 
Streaming Operational Data with MariaDB MaxScale
Streaming Operational Data with MariaDB MaxScaleStreaming Operational Data with MariaDB MaxScale
Streaming Operational Data with MariaDB MaxScale
 
Jump Start with Apache Spark 2.0 on Databricks
Jump Start with Apache Spark 2.0 on DatabricksJump Start with Apache Spark 2.0 on Databricks
Jump Start with Apache Spark 2.0 on Databricks
 
From Kafka to BigQuery - Strata Singapore
From Kafka to BigQuery - Strata SingaporeFrom Kafka to BigQuery - Strata Singapore
From Kafka to BigQuery - Strata Singapore
 
Materials Project Validation, Provenance, and Sandboxes by Dan Gunter
Materials Project Validation, Provenance, and Sandboxes by Dan GunterMaterials Project Validation, Provenance, and Sandboxes by Dan Gunter
Materials Project Validation, Provenance, and Sandboxes by Dan Gunter
 
Real-Time Big Data with Storm, Kafka and GigaSpaces
Real-Time Big Data with Storm, Kafka and GigaSpacesReal-Time Big Data with Storm, Kafka and GigaSpaces
Real-Time Big Data with Storm, Kafka and GigaSpaces
 
Making sense of your data jug
Making sense of your data   jugMaking sense of your data   jug
Making sense of your data jug
 
Realtime Analytics on AWS
Realtime Analytics on AWSRealtime Analytics on AWS
Realtime Analytics on AWS
 
High Performance, Scalable MongoDB in a Bare Metal Cloud
High Performance, Scalable MongoDB in a Bare Metal CloudHigh Performance, Scalable MongoDB in a Bare Metal Cloud
High Performance, Scalable MongoDB in a Bare Metal Cloud
 
Chicago Kafka Meetup
Chicago Kafka MeetupChicago Kafka Meetup
Chicago Kafka Meetup
 
Agility and Scalability with MongoDB
Agility and Scalability with MongoDBAgility and Scalability with MongoDB
Agility and Scalability with MongoDB
 
Big Data Solutions in Azure - David Giard
Big Data Solutions in Azure - David GiardBig Data Solutions in Azure - David Giard
Big Data Solutions in Azure - David Giard
 
Data Analytics Service Company and Its Ruby Usage
Data Analytics Service Company and Its Ruby UsageData Analytics Service Company and Its Ruby Usage
Data Analytics Service Company and Its Ruby Usage
 
Spark Summit - Stratio Streaming
Spark Summit - Stratio Streaming Spark Summit - Stratio Streaming
Spark Summit - Stratio Streaming
 
Riga dev day: Lambda architecture at AWS
Riga dev day: Lambda architecture at AWSRiga dev day: Lambda architecture at AWS
Riga dev day: Lambda architecture at AWS
 
Introduce Apache Cassandra - JavaTwo Taiwan, 2012
Introduce Apache Cassandra - JavaTwo Taiwan, 2012Introduce Apache Cassandra - JavaTwo Taiwan, 2012
Introduce Apache Cassandra - JavaTwo Taiwan, 2012
 
Enabling Key Business Advantage from Big Data through Advanced Ingest Process...
Enabling Key Business Advantage from Big Data through Advanced Ingest Process...Enabling Key Business Advantage from Big Data through Advanced Ingest Process...
Enabling Key Business Advantage from Big Data through Advanced Ingest Process...
 
Spark + Cassandra = Real Time Analytics on Operational Data
Spark + Cassandra = Real Time Analytics on Operational DataSpark + Cassandra = Real Time Analytics on Operational Data
Spark + Cassandra = Real Time Analytics on Operational Data
 
Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...
Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...
Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...
 
Building a Scalable Real-Time Fleet Management IoT Data Tracker with Kafka St...
Building a Scalable Real-Time Fleet Management IoT Data Tracker with Kafka St...Building a Scalable Real-Time Fleet Management IoT Data Tracker with Kafka St...
Building a Scalable Real-Time Fleet Management IoT Data Tracker with Kafka St...
 

Recently uploaded

Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
BookNet Canada
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
DanBrown980551
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
Elena Simperl
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
Bhaskar Mitra
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Product School
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
Product School
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
Abida Shariff
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
Fwdays
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
Alison B. Lowndes
 

Recently uploaded (20)

Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
 

Boundary Front end tech talk: how it works

  • 1.
  • 2. Tech Talk - Frontend Stephen Boak (steve@boundary.com) Mark Mahoney (mark@boundary.com) Matt King (m@boundary.com)
  • 3. When we say real-time we mean it: JSON being pushed to the browser once a second and visualizations built to constantly move and adapt to streaming data
  • 5. Meters ! Lightweight and highly scalable, these sit on the cloud as well as private data centers across virtual or physical servers.
  • 6. Intercept Meter data via Transport Collectors Layer Security (TLS) Authentication.
  • 7. Collect all of the data at high resolution to gain rich insight into complex Data Store environments & problems before they can impact critical business services.
  • 8. Real-time continuous data streaming at Streaming high resolution, low latency intervals.
  • 9. Stream data to your dashboard with sub-second latency, Streaming UI providing you with intuitive, powerful dashboard.
  • 10. How We Get Data • CometD server (streaker)! • Pulls in data from multiple backend services and streams it out to clients! • Can also aggregate and filter data on demand
  • 11. Subscriptions 1 2 3 4 5 Make a Get back a Subscribe to Receive Receive subscription request subscription ID new query state-dump add/remove (query and filters) (Query ID) (unique channel) (inserts & schema) messages !! ! ! ! !
  • 12. Data Structure • Schema and key (which fields are concatenated for each record) • State dump with schema for requested time window (up to 100kb state dumps) • Save bandwidth with N-tuples indexed by the schema • Adds/removes update the state with keys (subset of schema)
  • 13. JSON Example port:protocol subscription State Object { { channel:""/query/ac3941b8924a2f73/custom9 "1331234598000:80:6":"{ query92f536bbc93f159418b98f1a973b5dc4e78379 "epochMillis":=1331234598000, filter979564d6c94c9394f479abae906ff95a3411e9 "portProtocol":="80:6", aggregation9b5cc18d49098d949c29a08497037b08a30e6" "ingressPackets":"100, data:"{ "ingressOctets":"8463, insert:"[[1331234598000,"80:6,"100,"8463,"93," "egressPackets":"93, 9672],"[1331234925000,"8080:6,"242,"39657,"243," "egressOctets":"9672 40865],"…], }, keys:"[epochMillis,"portProtocol], "1331234925000:8080:6":"{ schema:"[epochMillis,"portProtocol,"ingressPackets," "epochMillis":=1331234925000, ingressOctets,"egressPackets,"egressOctets], "portProtocol":="8080:6", timestamp:"1331235119001 "ingressPackets":"242, } "ingressOctets":"39657, } "egressPackets":"243, "egressOctets":"40865 } … }
  • 14. DataSource.js • Negotiates between multiple data sources and subscribers on a page Subscriptions • One data source for N subscribers Data Sources • Abstracts CometD subscription process for JS development Subscribers • Smart enough to resubscribe and notifies subscribers (so that state dumps aren’t doubled up)
  • 17. Data and Subcription Problems • Monolithic, Multi-Purpose Queries • Large State Dumps cause CometD timeout (5-10MBs per second) • No Resubscription • Aggregated all data on front-end
  • 18. Data and Subcription Solutions • Stratified, Filterable Queries and server-side aggregation • Top-N Limitations (constrained output) • Resubscription & filtering • Web workers haven’t helped much • Time smoothing (1-second ticks)
  • 19. Visualization DOM manipulation is expensive ...so is tweening SVG Canvas
  • 20. What’s Next • WebSockets • Historical Data and long term data storage • HTML5 local storage (store data) • Machine-learning for real-time network visualization