SlideShare a Scribd company logo
PNDA
• Volume of network data into terabytes
• Siloed data limits ability to perform
correlation and causal analysis
• Relational databases limit the ability to
• Application of big data analytics to the
network dataset is key to providing both real-
time and historical insights
• Data science is driving the bifurcation of the
OSS stack
Network data is becoming a big data problem
3-fold increase
in total IP Traffic
>60% increase
in devices and
connections
Telemetry data
streamed in near
real-time
Source: Cisco VNI/GCI Global IP Traffic Forecast
What changes?
PMO FMO
Orientation Single domain Cross domain
Realisation Small data, tool driven Big data, data driven
Data collection Polled Streamed
Data aggregation and
analysis
Coupled Decoupled
Domain data schema Schema-on-write Schema-on-read
Analysis Prescriptive Prescriptive + Stochastic +
ML
Customisation Design time Run time
• Tight coupling of data
aggregation/store/
analysis
• Multiple analytics
pipelines implemented
from open source
components
• Common design
patterns ~75% of effort
wasted / duplicated
• Siloes limit the potential
of big data analytics and
lead to industry
divergence
Today’s siloed analytics pipelines
Telemetry
Metrics
Data sources
HDFS
Data store
Spark
Streaming
MapR
Data analysis
Hbase
Storm
Kafka
Streamsets
Data aggregation
Kafka
Impala
Query
Outputs
Dashboard &
ReportingNiFi
Logs
What is PNDA?
PNDA brings together a number of open source technologies to
provide a simple, scalable open big data analytics Platform for
Network Data Analytics
Linux Foundation Collaborative Project based on the Apache
ecosystem
• Simple, scalable open data
platform
• Provides a common set of
services for developing
analytics applications
• Accelerates the process of
developing big data analytics
applications whilst significantly
reducing the TCO
• PNDAprovides a platform for
convergence of network data
analytics
PNDA
PNDA
Plugins
ODL
Logstash
OpenBPM
pmacct
XR Telemetry
Real
-time
DataDistribution
File
Store
Platform Services: Installation, Mgmt,
Security, Data Privacy
App Packaging
and Mgmt
Stream
Batch
Processing
SQL
Query
OLAP
Cube
Search/
Lucene
NoSQL Time
Series
Data
Exploration
Metric
Visualisation
Event
Visualisation PNDA
Mnged App
PNDA
Mnged App
Unmnged
App
Unmnged
App
Query
Visualisation
and Exploration
PNDA
Applications
PNDA
Producer API
PNDA
Consumer API
• Horizontally scalable platform for
analytics and data processing
applications
• Support for near-real-time stream
processing and in-depth batch analysis
on massive datasets
• PNDA decouples data aggregation from
data analysis
• Consuming applications can be either
platform apps developed for PNDA or
client apps integrated with PNDA
• Client apps can use one of several
structured query interfaces or consume
streams directly.
• Leverages best current practise in big
data analytics
PNDA
PNDA
Plugins
ODL
Logstash
OpenBPM
pmacct
XR Telemetry
Real
-time
DataDistribution
File
Store
Platform Services: Installation, Mgmt,
Security, Data Privacy
App Packaging
and Mgmt
Stream
Batch
Processing
SQL
Query
OLAP
Cube
Search/
Lucene
NoSQL Time
Series
Data
Exploration
Metric
Visualisation
Event
Visualisation PNDA
Mnged App
PNDA
Mnged App
Unmnged
App
Unmnged
App
Query
Visualisation
and Exploration
PNDA
Applications
PNDA
Producer API
PNDA
Consumer API
Why PNDA?
There are a bewildering number of big data technologies out
there, so how do you decide what to use?
We've evaluated and chosen the best tools, based on technical
capability and community support.
PNDA combines them to streamline the process of developing
data processing applications.
PNDA Technologies
Why PNDA?
Innovation in the big data space is extremely rapid, but combining
multiple technologies into an end-to-end solution can be
extremely complex and time-consuming
PNDA removes this complexity and allows you to focus on
developing the analytics applications, not on developing the
pipeline – significantly reducing the effort required and time-to-
insight
PNDA Software Components
• Platform for data aggregation,
distribution, processing and storage
• Automated installation, creation, and
configuration
• Openstack, AWS and baremetal
• Typical install ~1hr
• Modular install
• Open producer and consumer APIs
• Avro platform schema
• Plugins for Logstash, pmacct,
OpenBMP, OpenDaylight, Cisco XR-
telemetry, bulk ingest …
• Data distribution – Apache Kafka
• Data store:
• Automated data partitioning and storage
(HDFS)
• OpenTSDB – time series analysis
• Hbase - NoSQL
• Support for batch and stream processing:
• Apache Spark and Spark Streaming
• Jupyter notebook server for app
prototyping and data exploration
• Impala-based SQL query support
• Grafana for time series visualisation
• PNDAapplication packaging
• PNDAmanagement and dashboard
PNDA 3.4 Capabilities
• The PNDA console
provides a dashboard
across all components
in a cluster
• Inbuilt platform test
agents verify the
operation of all
components
• Active platform testing
verifies the end-to-end
data pipeline
PNDA Console
• Ingested data should be
encapsulated in PNDA Avro
schema and published on a
pre-defined Kafka topic or set
of topics
Publishing Data to PNDA
PNDA Plugins
Data Type Data Aggregator Data Aggregator Reference PNDA Producer Plugin Reference
BGP (inc. BGP LS) OpenBMP http://www.openbmp.org/#!index.md#Usi
ng_Kafka_for_Collector_Integration
http://pnda.io/pnda-
guide/producer/openbmp.html
BGP PMACCT (BGP listener) http://www.pmacct.net/ http://pnda.io/pnda-
guide/producer/pmacct.html
Bulk Ingest PNDA Bulk Ingest Tool http://pnda.io/pnda-guide/bulkingest/
ISIS PMACCT (ISIS listener) http://www.pmacct.net/ http://pnda.io/pnda-
guide/producer/pmacct.html
Cisco XR streaming telemetry Pipeline https://github.com/cisco/bigmuddy-
network-telemetry-collector
CollectD (CollectD supports multiple
plugins as listed
here https://collectd.org/wiki/index.php/T
able_of_Plugins)
Logstash https://www.elastic.co/guide/en/logstash/
current/plugins-codecs-collectd.html
http://pnda.io/pnda-guide/repos/prod-
logstash-codec-avro/
IoT sensor via HTTP Node-RED https://nodered.org
Logstash (Logstash supports multiple
plugins as listed
here https://www.elastic.co/guide/en/logs
tash/current/input-plugins.html)
Logstash http://pnda.io/pnda-guide/repos/prod-
logstash-codec-avro/
NETCONF Notifications ODL http://www.opendaylight.org/ http://pnda.io/pnda-
guide/producer/opendl.html
Netflow / IPFIX Logstash https://www.elastic.co/guide/en/logstash/
current/plugins-codecs-netflow.html
http://pnda.io/pnda-guide/repos/prod-
logstash-codec-avro/
Netflow / IPFIX / sFlow pmacct http://www.pmacct.net/ http://pnda.io/pnda-
guide/producer/pmacct.html
Openstack Work in progress
sFlow Logstash https://github.com/ashangit/logstash-
codec-sflow
http://pnda.io/pnda-guide/repos/prod-
logstash-codec-avro/
SNMP Metrics and Traps ODL https://wiki.opendaylight.org/view/SNMP_
Plugin:Getting_Started
http://pnda.io/pnda-
guide/producer/opendl.html
SNMP Traps Logstash https://www.elastic.co/guide/en/logstash/
current/plugins-inputs-snmptrap.html
http://pnda.io/pnda-guide/repos/prod-
logstash-codec-avro/
Syslog Logstash https://www.elastic.co/guide/en/logstash/
current/plugins-inputs-syslog.html
http://pnda.io/pnda-guide/repos/prod-
logstash-codec-avro/
Syslog (RFC3164 or RFC5424 - needed
for newer IOS/IOS XR/ NX OS etc.)
Logstash https://gist.github.com/donaldh/89b73049
81f96497c94fe4d98bb03d71
http://pnda.io/pnda-guide/repos/prod-
logstash-codec-avro/
Design time vs. runtime
pico
standard
BGP Analytics Pipeline
Open BPM
Collector
BGPBGP
BMP
Logstash
PNDA Cluster
Gobblin
HDFS
Kafka
Spark
OpenTSDB
BGP Data
Service
UI
Impala
PNDA Applied to NFV
Infrastruct
ure
Analytics
Data	Aggregators
Open	Data	Platform	(PNDA)
Analytics	Applications
Open
Source
Custom Licensed
Alerts
Metrics
Telemetry
Logs
Data	Sources
Inventory
Orchestration
NFVO
VNFM
VIM
NFVI
VNF
Data	CenterCoreUser
State
Data
Access Aggregation
Related as loosely coupled
systems
Context
Network
Control
Convergence of network data analytics
Operational
Intelligence
Planning
Intelligence
Security
Intelligence
• PNDA 3.5
• ElasticSearch integration
• CentOS / RHEL
• Offline install
• Future
• Apache Kylin
• Apache Ambari
• Containerisation
• Deep-learning framework
• Red PNDA – the smallest PNDA
yet!
What’s coming?
Come and join us!
PNDA - Platform for Network Data Analytics

More Related Content

What's hot

Cisco Data Center Orchestration Solution
Cisco Data Center Orchestration SolutionCisco Data Center Orchestration Solution
Cisco Data Center Orchestration Solution
Cisco Canada
 
Introducing Databricks Delta
Introducing Databricks DeltaIntroducing Databricks Delta
Introducing Databricks Delta
Databricks
 
Data Mesh
Data MeshData Mesh
BGP Dynamic Routing and Neutron
BGP Dynamic Routing and NeutronBGP Dynamic Routing and Neutron
BGP Dynamic Routing and Neutron
rktidwell
 
Cisco Live Milan 2015 - BGP advance
Cisco Live Milan 2015 - BGP advanceCisco Live Milan 2015 - BGP advance
Cisco Live Milan 2015 - BGP advance
Bertrand Duvivier
 
SDN Architecture & Ecosystem
SDN Architecture & EcosystemSDN Architecture & Ecosystem
SDN Architecture & Ecosystem
Kingston Smiler
 
Cloudera Customer Success Story
Cloudera Customer Success StoryCloudera Customer Success Story
Cloudera Customer Success Story
Xpand IT
 
Software Defined WAN – SD-WAN
Software Defined WAN – SD-WANSoftware Defined WAN – SD-WAN
Software Defined WAN – SD-WAN
MarketingArrowECS_CZ
 
5G Network Slicing Using Mininet
5G Network Slicing Using Mininet5G Network Slicing Using Mininet
5G Network Slicing Using Mininet
Mohammed Abuibaid
 
OpenFlow
OpenFlowOpenFlow
OpenFlow
Kingston Smiler
 
Understanding Cisco’ Next Generation SD-WAN Technology
Understanding Cisco’ Next Generation SD-WAN TechnologyUnderstanding Cisco’ Next Generation SD-WAN Technology
Understanding Cisco’ Next Generation SD-WAN Technology
Cisco Canada
 
vSAN Beyond The Basics
vSAN Beyond The BasicsvSAN Beyond The Basics
vSAN Beyond The Basics
Sumit Lahiri
 
Big Data Business Wins: Real-time Inventory Tracking with Hadoop
Big Data Business Wins: Real-time Inventory Tracking with HadoopBig Data Business Wins: Real-time Inventory Tracking with Hadoop
Big Data Business Wins: Real-time Inventory Tracking with Hadoop
DataWorks Summit
 
ThousandEyes Overview
ThousandEyes Overview ThousandEyes Overview
ThousandEyes Overview
ThousandEyes
 
Windows 2019
Windows 2019Windows 2019
Windows 2019
Gary Williams
 
Introduction to vxlan
Introduction to vxlanIntroduction to vxlan
Introduction to vxlan
Mohammed Umair
 
Introduction to OpenFlow
Introduction to OpenFlowIntroduction to OpenFlow
Introduction to OpenFlow
Joel W. King
 
cloud storage
cloud storagecloud storage
cloud storage
obrita youkhane
 
Cisco Security portfolio update
Cisco Security portfolio updateCisco Security portfolio update
Cisco Security portfolio update
Atanas Gergiminov
 
Splunk Overview
Splunk OverviewSplunk Overview
Splunk Overview
Splunk
 

What's hot (20)

Cisco Data Center Orchestration Solution
Cisco Data Center Orchestration SolutionCisco Data Center Orchestration Solution
Cisco Data Center Orchestration Solution
 
Introducing Databricks Delta
Introducing Databricks DeltaIntroducing Databricks Delta
Introducing Databricks Delta
 
Data Mesh
Data MeshData Mesh
Data Mesh
 
BGP Dynamic Routing and Neutron
BGP Dynamic Routing and NeutronBGP Dynamic Routing and Neutron
BGP Dynamic Routing and Neutron
 
Cisco Live Milan 2015 - BGP advance
Cisco Live Milan 2015 - BGP advanceCisco Live Milan 2015 - BGP advance
Cisco Live Milan 2015 - BGP advance
 
SDN Architecture & Ecosystem
SDN Architecture & EcosystemSDN Architecture & Ecosystem
SDN Architecture & Ecosystem
 
Cloudera Customer Success Story
Cloudera Customer Success StoryCloudera Customer Success Story
Cloudera Customer Success Story
 
Software Defined WAN – SD-WAN
Software Defined WAN – SD-WANSoftware Defined WAN – SD-WAN
Software Defined WAN – SD-WAN
 
5G Network Slicing Using Mininet
5G Network Slicing Using Mininet5G Network Slicing Using Mininet
5G Network Slicing Using Mininet
 
OpenFlow
OpenFlowOpenFlow
OpenFlow
 
Understanding Cisco’ Next Generation SD-WAN Technology
Understanding Cisco’ Next Generation SD-WAN TechnologyUnderstanding Cisco’ Next Generation SD-WAN Technology
Understanding Cisco’ Next Generation SD-WAN Technology
 
vSAN Beyond The Basics
vSAN Beyond The BasicsvSAN Beyond The Basics
vSAN Beyond The Basics
 
Big Data Business Wins: Real-time Inventory Tracking with Hadoop
Big Data Business Wins: Real-time Inventory Tracking with HadoopBig Data Business Wins: Real-time Inventory Tracking with Hadoop
Big Data Business Wins: Real-time Inventory Tracking with Hadoop
 
ThousandEyes Overview
ThousandEyes Overview ThousandEyes Overview
ThousandEyes Overview
 
Windows 2019
Windows 2019Windows 2019
Windows 2019
 
Introduction to vxlan
Introduction to vxlanIntroduction to vxlan
Introduction to vxlan
 
Introduction to OpenFlow
Introduction to OpenFlowIntroduction to OpenFlow
Introduction to OpenFlow
 
cloud storage
cloud storagecloud storage
cloud storage
 
Cisco Security portfolio update
Cisco Security portfolio updateCisco Security portfolio update
Cisco Security portfolio update
 
Splunk Overview
Splunk OverviewSplunk Overview
Splunk Overview
 

Viewers also liked

OpenMAMA Governance
OpenMAMA GovernanceOpenMAMA Governance
OpenMAMA Governance
OpenMAMA
 
Stream Analytics with SQL on Apache Flink
 Stream Analytics with SQL on Apache Flink Stream Analytics with SQL on Apache Flink
Stream Analytics with SQL on Apache Flink
Fabian Hueske
 
Remote Control Architecture: How We Are Building The World’s Fastest Remote C...
Remote Control Architecture: How We Are Building The World’s Fastest Remote C...Remote Control Architecture: How We Are Building The World’s Fastest Remote C...
Remote Control Architecture: How We Are Building The World’s Fastest Remote C...
Kaseya
 
Advanced Administration: Kaseya Traverse
Advanced Administration: Kaseya TraverseAdvanced Administration: Kaseya Traverse
Advanced Administration: Kaseya Traverse
Kaseya
 
Kaseya Connect 2013: Kaseya IT Services – Virtual Engineers! Come See them in...
Kaseya Connect 2013: Kaseya IT Services – Virtual Engineers! Come See them in...Kaseya Connect 2013: Kaseya IT Services – Virtual Engineers! Come See them in...
Kaseya Connect 2013: Kaseya IT Services – Virtual Engineers! Come See them in...
Kaseya
 
Kaseya Monitoring Suite Overview
Kaseya Monitoring Suite OverviewKaseya Monitoring Suite Overview
Kaseya Monitoring Suite Overview
Kaseya
 
Automated Out-of-Band management with Ansible and Redfish
Automated Out-of-Band management with Ansible and RedfishAutomated Out-of-Band management with Ansible and Redfish
Automated Out-of-Band management with Ansible and Redfish
Jose De La Rosa
 
A Tale of Three Apache Spark APIs: RDDs, DataFrames, and Datasets with Jules ...
A Tale of Three Apache Spark APIs: RDDs, DataFrames, and Datasets with Jules ...A Tale of Three Apache Spark APIs: RDDs, DataFrames, and Datasets with Jules ...
A Tale of Three Apache Spark APIs: RDDs, DataFrames, and Datasets with Jules ...
Databricks
 

Viewers also liked (8)

OpenMAMA Governance
OpenMAMA GovernanceOpenMAMA Governance
OpenMAMA Governance
 
Stream Analytics with SQL on Apache Flink
 Stream Analytics with SQL on Apache Flink Stream Analytics with SQL on Apache Flink
Stream Analytics with SQL on Apache Flink
 
Remote Control Architecture: How We Are Building The World’s Fastest Remote C...
Remote Control Architecture: How We Are Building The World’s Fastest Remote C...Remote Control Architecture: How We Are Building The World’s Fastest Remote C...
Remote Control Architecture: How We Are Building The World’s Fastest Remote C...
 
Advanced Administration: Kaseya Traverse
Advanced Administration: Kaseya TraverseAdvanced Administration: Kaseya Traverse
Advanced Administration: Kaseya Traverse
 
Kaseya Connect 2013: Kaseya IT Services – Virtual Engineers! Come See them in...
Kaseya Connect 2013: Kaseya IT Services – Virtual Engineers! Come See them in...Kaseya Connect 2013: Kaseya IT Services – Virtual Engineers! Come See them in...
Kaseya Connect 2013: Kaseya IT Services – Virtual Engineers! Come See them in...
 
Kaseya Monitoring Suite Overview
Kaseya Monitoring Suite OverviewKaseya Monitoring Suite Overview
Kaseya Monitoring Suite Overview
 
Automated Out-of-Band management with Ansible and Redfish
Automated Out-of-Band management with Ansible and RedfishAutomated Out-of-Band management with Ansible and Redfish
Automated Out-of-Band management with Ansible and Redfish
 
A Tale of Three Apache Spark APIs: RDDs, DataFrames, and Datasets with Jules ...
A Tale of Three Apache Spark APIs: RDDs, DataFrames, and Datasets with Jules ...A Tale of Three Apache Spark APIs: RDDs, DataFrames, and Datasets with Jules ...
A Tale of Three Apache Spark APIs: RDDs, DataFrames, and Datasets with Jules ...
 

Similar to PNDA - Platform for Network Data Analytics

Stream Data Processing at Big Data Landscape by Oleksandr Fedirko
Stream Data Processing at Big Data Landscape by Oleksandr Fedirko Stream Data Processing at Big Data Landscape by Oleksandr Fedirko
Stream Data Processing at Big Data Landscape by Oleksandr Fedirko
GlobalLogic Ukraine
 
Devfest uk & ireland using apache nifi with apache pulsar for fast data on-r...
Devfest uk & ireland  using apache nifi with apache pulsar for fast data on-r...Devfest uk & ireland  using apache nifi with apache pulsar for fast data on-r...
Devfest uk & ireland using apache nifi with apache pulsar for fast data on-r...
Timothy Spann
 
Whither the Hadoop Developer Experience, June Hadoop Meetup, Nitin Motgi
Whither the Hadoop Developer Experience, June Hadoop Meetup, Nitin MotgiWhither the Hadoop Developer Experience, June Hadoop Meetup, Nitin Motgi
Whither the Hadoop Developer Experience, June Hadoop Meetup, Nitin Motgi
Felicia Haggarty
 
Tim Pugh-SPEDDEXES 2014
Tim Pugh-SPEDDEXES 2014Tim Pugh-SPEDDEXES 2014
Tim Pugh-SPEDDEXES 2014
aceas13tern
 
ClickHouse Paris Meetup. Pragma Analytics Software Suite w/ClickHouse, by Mat...
ClickHouse Paris Meetup. Pragma Analytics Software Suite w/ClickHouse, by Mat...ClickHouse Paris Meetup. Pragma Analytics Software Suite w/ClickHouse, by Mat...
ClickHouse Paris Meetup. Pragma Analytics Software Suite w/ClickHouse, by Mat...
Altinity Ltd
 
XDF 2019 Xilinx Accelerated Database and Data Analytics Ecosystem
XDF 2019 Xilinx Accelerated Database and Data Analytics EcosystemXDF 2019 Xilinx Accelerated Database and Data Analytics Ecosystem
XDF 2019 Xilinx Accelerated Database and Data Analytics Ecosystem
Dan Eaton
 
Big data analytics and machine intelligence v5.0
Big data analytics and machine intelligence   v5.0Big data analytics and machine intelligence   v5.0
Big data analytics and machine intelligence v5.0
Amr Kamel Deklel
 
Pivotal Real Time Data Stream Analytics
Pivotal Real Time Data Stream AnalyticsPivotal Real Time Data Stream Analytics
Pivotal Real Time Data Stream Analytics
kgshukla
 
Cardinality-HL-Overview
Cardinality-HL-OverviewCardinality-HL-Overview
Cardinality-HL-Overview
Harry Frost
 
Splunk App for Stream
Splunk App for StreamSplunk App for Stream
Splunk App for Stream
Splunk
 
Unbundling the Modern Streaming Stack With Dunith Dhanushka | Current 2022
Unbundling the Modern Streaming Stack With Dunith Dhanushka | Current 2022Unbundling the Modern Streaming Stack With Dunith Dhanushka | Current 2022
Unbundling the Modern Streaming Stack With Dunith Dhanushka | Current 2022
HostedbyConfluent
 
Решения WANDL и NorthStar для операторов
Решения WANDL и NorthStar для операторовРешения WANDL и NorthStar для операторов
Решения WANDL и NorthStar для операторов
TERMILAB. Интернет - лаборатория
 
20160331 sa introduction to big data pipelining berlin meetup 0.3
20160331 sa introduction to big data pipelining berlin meetup   0.320160331 sa introduction to big data pipelining berlin meetup   0.3
20160331 sa introduction to big data pipelining berlin meetup 0.3
Simon Ambridge
 
Streaming Data Ingest and Processing with Apache Kafka
Streaming Data Ingest and Processing with Apache KafkaStreaming Data Ingest and Processing with Apache Kafka
Streaming Data Ingest and Processing with Apache Kafka
Attunity
 
Introduction to Apache Kafka
Introduction to Apache KafkaIntroduction to Apache Kafka
Introduction to Apache Kafka
Ricardo Bravo
 
Big Data for Testing - Heading for Post Process and Analytics
Big Data for Testing - Heading for Post Process and AnalyticsBig Data for Testing - Heading for Post Process and Analytics
Big Data for Testing - Heading for Post Process and Analytics
OPNFV
 
Data Pipelines with Spark & DataStax Enterprise
Data Pipelines with Spark & DataStax EnterpriseData Pipelines with Spark & DataStax Enterprise
Data Pipelines with Spark & DataStax Enterprise
DataStax
 
Big Data Berlin v8.0 Stream Processing with Apache Apex
Big Data Berlin v8.0 Stream Processing with Apache Apex Big Data Berlin v8.0 Stream Processing with Apache Apex
Big Data Berlin v8.0 Stream Processing with Apache Apex
Apache Apex
 
Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...
Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...
Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...
Dataconomy Media
 
Cloud Lambda Architecture Patterns
Cloud Lambda Architecture PatternsCloud Lambda Architecture Patterns
Cloud Lambda Architecture Patterns
Asis Mohanty
 

Similar to PNDA - Platform for Network Data Analytics (20)

Stream Data Processing at Big Data Landscape by Oleksandr Fedirko
Stream Data Processing at Big Data Landscape by Oleksandr Fedirko Stream Data Processing at Big Data Landscape by Oleksandr Fedirko
Stream Data Processing at Big Data Landscape by Oleksandr Fedirko
 
Devfest uk & ireland using apache nifi with apache pulsar for fast data on-r...
Devfest uk & ireland  using apache nifi with apache pulsar for fast data on-r...Devfest uk & ireland  using apache nifi with apache pulsar for fast data on-r...
Devfest uk & ireland using apache nifi with apache pulsar for fast data on-r...
 
Whither the Hadoop Developer Experience, June Hadoop Meetup, Nitin Motgi
Whither the Hadoop Developer Experience, June Hadoop Meetup, Nitin MotgiWhither the Hadoop Developer Experience, June Hadoop Meetup, Nitin Motgi
Whither the Hadoop Developer Experience, June Hadoop Meetup, Nitin Motgi
 
Tim Pugh-SPEDDEXES 2014
Tim Pugh-SPEDDEXES 2014Tim Pugh-SPEDDEXES 2014
Tim Pugh-SPEDDEXES 2014
 
ClickHouse Paris Meetup. Pragma Analytics Software Suite w/ClickHouse, by Mat...
ClickHouse Paris Meetup. Pragma Analytics Software Suite w/ClickHouse, by Mat...ClickHouse Paris Meetup. Pragma Analytics Software Suite w/ClickHouse, by Mat...
ClickHouse Paris Meetup. Pragma Analytics Software Suite w/ClickHouse, by Mat...
 
XDF 2019 Xilinx Accelerated Database and Data Analytics Ecosystem
XDF 2019 Xilinx Accelerated Database and Data Analytics EcosystemXDF 2019 Xilinx Accelerated Database and Data Analytics Ecosystem
XDF 2019 Xilinx Accelerated Database and Data Analytics Ecosystem
 
Big data analytics and machine intelligence v5.0
Big data analytics and machine intelligence   v5.0Big data analytics and machine intelligence   v5.0
Big data analytics and machine intelligence v5.0
 
Pivotal Real Time Data Stream Analytics
Pivotal Real Time Data Stream AnalyticsPivotal Real Time Data Stream Analytics
Pivotal Real Time Data Stream Analytics
 
Cardinality-HL-Overview
Cardinality-HL-OverviewCardinality-HL-Overview
Cardinality-HL-Overview
 
Splunk App for Stream
Splunk App for StreamSplunk App for Stream
Splunk App for Stream
 
Unbundling the Modern Streaming Stack With Dunith Dhanushka | Current 2022
Unbundling the Modern Streaming Stack With Dunith Dhanushka | Current 2022Unbundling the Modern Streaming Stack With Dunith Dhanushka | Current 2022
Unbundling the Modern Streaming Stack With Dunith Dhanushka | Current 2022
 
Решения WANDL и NorthStar для операторов
Решения WANDL и NorthStar для операторовРешения WANDL и NorthStar для операторов
Решения WANDL и NorthStar для операторов
 
20160331 sa introduction to big data pipelining berlin meetup 0.3
20160331 sa introduction to big data pipelining berlin meetup   0.320160331 sa introduction to big data pipelining berlin meetup   0.3
20160331 sa introduction to big data pipelining berlin meetup 0.3
 
Streaming Data Ingest and Processing with Apache Kafka
Streaming Data Ingest and Processing with Apache KafkaStreaming Data Ingest and Processing with Apache Kafka
Streaming Data Ingest and Processing with Apache Kafka
 
Introduction to Apache Kafka
Introduction to Apache KafkaIntroduction to Apache Kafka
Introduction to Apache Kafka
 
Big Data for Testing - Heading for Post Process and Analytics
Big Data for Testing - Heading for Post Process and AnalyticsBig Data for Testing - Heading for Post Process and Analytics
Big Data for Testing - Heading for Post Process and Analytics
 
Data Pipelines with Spark & DataStax Enterprise
Data Pipelines with Spark & DataStax EnterpriseData Pipelines with Spark & DataStax Enterprise
Data Pipelines with Spark & DataStax Enterprise
 
Big Data Berlin v8.0 Stream Processing with Apache Apex
Big Data Berlin v8.0 Stream Processing with Apache Apex Big Data Berlin v8.0 Stream Processing with Apache Apex
Big Data Berlin v8.0 Stream Processing with Apache Apex
 
Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...
Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...
Thomas Weise, Apache Apex PMC Member and Architect/Co-Founder, DataTorrent - ...
 
Cloud Lambda Architecture Patterns
Cloud Lambda Architecture PatternsCloud Lambda Architecture Patterns
Cloud Lambda Architecture Patterns
 

Recently uploaded

Nipissing University degree offer Nipissing diploma Transcript
Nipissing University degree offer Nipissing diploma TranscriptNipissing University degree offer Nipissing diploma Transcript
Nipissing University degree offer Nipissing diploma Transcript
zyqedad
 
Mumbai Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service And ...
Mumbai Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service And ...Mumbai Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service And ...
Mumbai Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service And ...
norina2645
 
Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...
Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...
Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...
birajmohan012
 
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
sharonblush
 
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
45unexpected
 
Welcome back to Instagram. Sign in to check out what your
Welcome back to Instagram. Sign in to check out what yourWelcome back to Instagram. Sign in to check out what your
Welcome back to Instagram. Sign in to check out what your
Virni Arrora
 
Data analytics and Access Program Recommendations
Data analytics and Access Program RecommendationsData analytics and Access Program Recommendations
Data analytics and Access Program Recommendations
hemantsharmaus
 
BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...
BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...
BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...
fatima shekh$A17
 
DataScienceConcept_Kanchana_Weerasinghe.pptx
DataScienceConcept_Kanchana_Weerasinghe.pptxDataScienceConcept_Kanchana_Weerasinghe.pptx
DataScienceConcept_Kanchana_Weerasinghe.pptx
Kanchana Weerasinghe
 
bai-tap-tieng-anh-lop-12-unit-4-the-mass-media (1).doc
bai-tap-tieng-anh-lop-12-unit-4-the-mass-media (1).docbai-tap-tieng-anh-lop-12-unit-4-the-mass-media (1).doc
bai-tap-tieng-anh-lop-12-unit-4-the-mass-media (1).doc
PhngThLmHnh
 
CHAPTER-1-Introduction-to-Marketing.pptx
CHAPTER-1-Introduction-to-Marketing.pptxCHAPTER-1-Introduction-to-Marketing.pptx
CHAPTER-1-Introduction-to-Marketing.pptx
girewiy968
 
Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...
Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...
Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...
sheetal singh$A17
 
Introduction to the Red Hat Portfolio.pdf
Introduction to the Red Hat Portfolio.pdfIntroduction to the Red Hat Portfolio.pdf
Introduction to the Red Hat Portfolio.pdf
kihus38
 
Coimbatore Girls call Service 000XX00000 Provide Best And Top Girl Service An...
Coimbatore Girls call Service 000XX00000 Provide Best And Top Girl Service An...Coimbatore Girls call Service 000XX00000 Provide Best And Top Girl Service An...
Coimbatore Girls call Service 000XX00000 Provide Best And Top Girl Service An...
vrvipin164
 
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
ginni singh$A17
 
all about the data science process, covering the steps present in almost ever...
all about the data science process, covering the steps present in almost ever...all about the data science process, covering the steps present in almost ever...
all about the data science process, covering the steps present in almost ever...
palaniappancse
 
MUMBAI MONTHLY RAINFALL CAPSTONE PROJECT
MUMBAI MONTHLY RAINFALL CAPSTONE PROJECTMUMBAI MONTHLY RAINFALL CAPSTONE PROJECT
MUMBAI MONTHLY RAINFALL CAPSTONE PROJECT
GaneshGanesh399816
 
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
tanupasswan6
 
Amul goes international: Desi dairy giant to launch fresh ...
Amul goes international: Desi dairy giant to launch fresh ...Amul goes international: Desi dairy giant to launch fresh ...
Amul goes international: Desi dairy giant to launch fresh ...
chetankumar9855
 
🚂🚘 Premium Girls Call Bangalore 🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
🚂🚘 Premium Girls Call Bangalore  🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...🚂🚘 Premium Girls Call Bangalore  🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
🚂🚘 Premium Girls Call Bangalore 🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
bhupeshkumar0889
 

Recently uploaded (20)

Nipissing University degree offer Nipissing diploma Transcript
Nipissing University degree offer Nipissing diploma TranscriptNipissing University degree offer Nipissing diploma Transcript
Nipissing University degree offer Nipissing diploma Transcript
 
Mumbai Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service And ...
Mumbai Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service And ...Mumbai Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service And ...
Mumbai Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service And ...
 
Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...
Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...
Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...
 
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
Best Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Service And ...
 
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
 
Welcome back to Instagram. Sign in to check out what your
Welcome back to Instagram. Sign in to check out what yourWelcome back to Instagram. Sign in to check out what your
Welcome back to Instagram. Sign in to check out what your
 
Data analytics and Access Program Recommendations
Data analytics and Access Program RecommendationsData analytics and Access Program Recommendations
Data analytics and Access Program Recommendations
 
BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...
BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...
BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...
 
DataScienceConcept_Kanchana_Weerasinghe.pptx
DataScienceConcept_Kanchana_Weerasinghe.pptxDataScienceConcept_Kanchana_Weerasinghe.pptx
DataScienceConcept_Kanchana_Weerasinghe.pptx
 
bai-tap-tieng-anh-lop-12-unit-4-the-mass-media (1).doc
bai-tap-tieng-anh-lop-12-unit-4-the-mass-media (1).docbai-tap-tieng-anh-lop-12-unit-4-the-mass-media (1).doc
bai-tap-tieng-anh-lop-12-unit-4-the-mass-media (1).doc
 
CHAPTER-1-Introduction-to-Marketing.pptx
CHAPTER-1-Introduction-to-Marketing.pptxCHAPTER-1-Introduction-to-Marketing.pptx
CHAPTER-1-Introduction-to-Marketing.pptx
 
Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...
Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...
Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...
 
Introduction to the Red Hat Portfolio.pdf
Introduction to the Red Hat Portfolio.pdfIntroduction to the Red Hat Portfolio.pdf
Introduction to the Red Hat Portfolio.pdf
 
Coimbatore Girls call Service 000XX00000 Provide Best And Top Girl Service An...
Coimbatore Girls call Service 000XX00000 Provide Best And Top Girl Service An...Coimbatore Girls call Service 000XX00000 Provide Best And Top Girl Service An...
Coimbatore Girls call Service 000XX00000 Provide Best And Top Girl Service An...
 
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
Celebrity Girls Call Noida 9873940964 Unlimited Short Providing Girls Service...
 
all about the data science process, covering the steps present in almost ever...
all about the data science process, covering the steps present in almost ever...all about the data science process, covering the steps present in almost ever...
all about the data science process, covering the steps present in almost ever...
 
MUMBAI MONTHLY RAINFALL CAPSTONE PROJECT
MUMBAI MONTHLY RAINFALL CAPSTONE PROJECTMUMBAI MONTHLY RAINFALL CAPSTONE PROJECT
MUMBAI MONTHLY RAINFALL CAPSTONE PROJECT
 
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
 
Amul goes international: Desi dairy giant to launch fresh ...
Amul goes international: Desi dairy giant to launch fresh ...Amul goes international: Desi dairy giant to launch fresh ...
Amul goes international: Desi dairy giant to launch fresh ...
 
🚂🚘 Premium Girls Call Bangalore 🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
🚂🚘 Premium Girls Call Bangalore  🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...🚂🚘 Premium Girls Call Bangalore  🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
🚂🚘 Premium Girls Call Bangalore 🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
 

PNDA - Platform for Network Data Analytics

  • 2. • Volume of network data into terabytes • Siloed data limits ability to perform correlation and causal analysis • Relational databases limit the ability to • Application of big data analytics to the network dataset is key to providing both real- time and historical insights • Data science is driving the bifurcation of the OSS stack Network data is becoming a big data problem 3-fold increase in total IP Traffic >60% increase in devices and connections Telemetry data streamed in near real-time Source: Cisco VNI/GCI Global IP Traffic Forecast
  • 3. What changes? PMO FMO Orientation Single domain Cross domain Realisation Small data, tool driven Big data, data driven Data collection Polled Streamed Data aggregation and analysis Coupled Decoupled Domain data schema Schema-on-write Schema-on-read Analysis Prescriptive Prescriptive + Stochastic + ML Customisation Design time Run time
  • 4. • Tight coupling of data aggregation/store/ analysis • Multiple analytics pipelines implemented from open source components • Common design patterns ~75% of effort wasted / duplicated • Siloes limit the potential of big data analytics and lead to industry divergence Today’s siloed analytics pipelines Telemetry Metrics Data sources HDFS Data store Spark Streaming MapR Data analysis Hbase Storm Kafka Streamsets Data aggregation Kafka Impala Query Outputs Dashboard & ReportingNiFi Logs
  • 5. What is PNDA? PNDA brings together a number of open source technologies to provide a simple, scalable open big data analytics Platform for Network Data Analytics Linux Foundation Collaborative Project based on the Apache ecosystem
  • 6. • Simple, scalable open data platform • Provides a common set of services for developing analytics applications • Accelerates the process of developing big data analytics applications whilst significantly reducing the TCO • PNDAprovides a platform for convergence of network data analytics PNDA PNDA Plugins ODL Logstash OpenBPM pmacct XR Telemetry Real -time DataDistribution File Store Platform Services: Installation, Mgmt, Security, Data Privacy App Packaging and Mgmt Stream Batch Processing SQL Query OLAP Cube Search/ Lucene NoSQL Time Series Data Exploration Metric Visualisation Event Visualisation PNDA Mnged App PNDA Mnged App Unmnged App Unmnged App Query Visualisation and Exploration PNDA Applications PNDA Producer API PNDA Consumer API
  • 7. • Horizontally scalable platform for analytics and data processing applications • Support for near-real-time stream processing and in-depth batch analysis on massive datasets • PNDA decouples data aggregation from data analysis • Consuming applications can be either platform apps developed for PNDA or client apps integrated with PNDA • Client apps can use one of several structured query interfaces or consume streams directly. • Leverages best current practise in big data analytics PNDA PNDA Plugins ODL Logstash OpenBPM pmacct XR Telemetry Real -time DataDistribution File Store Platform Services: Installation, Mgmt, Security, Data Privacy App Packaging and Mgmt Stream Batch Processing SQL Query OLAP Cube Search/ Lucene NoSQL Time Series Data Exploration Metric Visualisation Event Visualisation PNDA Mnged App PNDA Mnged App Unmnged App Unmnged App Query Visualisation and Exploration PNDA Applications PNDA Producer API PNDA Consumer API
  • 8. Why PNDA? There are a bewildering number of big data technologies out there, so how do you decide what to use? We've evaluated and chosen the best tools, based on technical capability and community support. PNDA combines them to streamline the process of developing data processing applications.
  • 10. Why PNDA? Innovation in the big data space is extremely rapid, but combining multiple technologies into an end-to-end solution can be extremely complex and time-consuming PNDA removes this complexity and allows you to focus on developing the analytics applications, not on developing the pipeline – significantly reducing the effort required and time-to- insight
  • 12. • Platform for data aggregation, distribution, processing and storage • Automated installation, creation, and configuration • Openstack, AWS and baremetal • Typical install ~1hr • Modular install • Open producer and consumer APIs • Avro platform schema • Plugins for Logstash, pmacct, OpenBMP, OpenDaylight, Cisco XR- telemetry, bulk ingest … • Data distribution – Apache Kafka • Data store: • Automated data partitioning and storage (HDFS) • OpenTSDB – time series analysis • Hbase - NoSQL • Support for batch and stream processing: • Apache Spark and Spark Streaming • Jupyter notebook server for app prototyping and data exploration • Impala-based SQL query support • Grafana for time series visualisation • PNDAapplication packaging • PNDAmanagement and dashboard PNDA 3.4 Capabilities
  • 13. • The PNDA console provides a dashboard across all components in a cluster • Inbuilt platform test agents verify the operation of all components • Active platform testing verifies the end-to-end data pipeline PNDA Console
  • 14. • Ingested data should be encapsulated in PNDA Avro schema and published on a pre-defined Kafka topic or set of topics Publishing Data to PNDA
  • 15. PNDA Plugins Data Type Data Aggregator Data Aggregator Reference PNDA Producer Plugin Reference BGP (inc. BGP LS) OpenBMP http://www.openbmp.org/#!index.md#Usi ng_Kafka_for_Collector_Integration http://pnda.io/pnda- guide/producer/openbmp.html BGP PMACCT (BGP listener) http://www.pmacct.net/ http://pnda.io/pnda- guide/producer/pmacct.html Bulk Ingest PNDA Bulk Ingest Tool http://pnda.io/pnda-guide/bulkingest/ ISIS PMACCT (ISIS listener) http://www.pmacct.net/ http://pnda.io/pnda- guide/producer/pmacct.html Cisco XR streaming telemetry Pipeline https://github.com/cisco/bigmuddy- network-telemetry-collector CollectD (CollectD supports multiple plugins as listed here https://collectd.org/wiki/index.php/T able_of_Plugins) Logstash https://www.elastic.co/guide/en/logstash/ current/plugins-codecs-collectd.html http://pnda.io/pnda-guide/repos/prod- logstash-codec-avro/ IoT sensor via HTTP Node-RED https://nodered.org Logstash (Logstash supports multiple plugins as listed here https://www.elastic.co/guide/en/logs tash/current/input-plugins.html) Logstash http://pnda.io/pnda-guide/repos/prod- logstash-codec-avro/ NETCONF Notifications ODL http://www.opendaylight.org/ http://pnda.io/pnda- guide/producer/opendl.html Netflow / IPFIX Logstash https://www.elastic.co/guide/en/logstash/ current/plugins-codecs-netflow.html http://pnda.io/pnda-guide/repos/prod- logstash-codec-avro/ Netflow / IPFIX / sFlow pmacct http://www.pmacct.net/ http://pnda.io/pnda- guide/producer/pmacct.html Openstack Work in progress sFlow Logstash https://github.com/ashangit/logstash- codec-sflow http://pnda.io/pnda-guide/repos/prod- logstash-codec-avro/ SNMP Metrics and Traps ODL https://wiki.opendaylight.org/view/SNMP_ Plugin:Getting_Started http://pnda.io/pnda- guide/producer/opendl.html SNMP Traps Logstash https://www.elastic.co/guide/en/logstash/ current/plugins-inputs-snmptrap.html http://pnda.io/pnda-guide/repos/prod- logstash-codec-avro/ Syslog Logstash https://www.elastic.co/guide/en/logstash/ current/plugins-inputs-syslog.html http://pnda.io/pnda-guide/repos/prod- logstash-codec-avro/ Syslog (RFC3164 or RFC5424 - needed for newer IOS/IOS XR/ NX OS etc.) Logstash https://gist.github.com/donaldh/89b73049 81f96497c94fe4d98bb03d71 http://pnda.io/pnda-guide/repos/prod- logstash-codec-avro/
  • 16. Design time vs. runtime pico standard
  • 17. BGP Analytics Pipeline Open BPM Collector BGPBGP BMP Logstash PNDA Cluster Gobblin HDFS Kafka Spark OpenTSDB BGP Data Service UI Impala
  • 18. PNDA Applied to NFV Infrastruct ure Analytics Data Aggregators Open Data Platform (PNDA) Analytics Applications Open Source Custom Licensed Alerts Metrics Telemetry Logs Data Sources Inventory Orchestration NFVO VNFM VIM NFVI VNF Data CenterCoreUser State Data Access Aggregation Related as loosely coupled systems Context Network Control
  • 19. Convergence of network data analytics Operational Intelligence Planning Intelligence Security Intelligence
  • 20. • PNDA 3.5 • ElasticSearch integration • CentOS / RHEL • Offline install • Future • Apache Kylin • Apache Ambari • Containerisation • Deep-learning framework • Red PNDA – the smallest PNDA yet! What’s coming?