As data sources such as sensors, social media, mobile, cloud, and machine logs become pervasive within the enterprise and the 4Vs of data managed by modern business applications increase exponentially, organizations need faster and better visualizations to explore data at scale and discover game-changing insights.
Organizations are increasingly adopting modern technologies such as GPUs to interactively visualize billions of data elements in real-time and take faster business decisions.
Augment your existing business applications and visualization capabilities with GPU-accelerated rendering of maps and accompanying dashboards for spatial awareness and location-based analytics
GPU’s massive parallelization minimizes the need for data preparation and sampling and delivers brute force capabilities to interactively visualize the fast-moving location and time-series data such as trading, traffic, social media, and vehicle telematics at scale and with speed.
The Ultimate Guide to Choosing WordPress Pros and Cons
GTC-DC 2017 Session: Advanced Analytics and Machine Learning with Geospatial Data: A World of Possibilities
1. 1
Advanced Analytics and Machine Learning with Geospatial Data: A World of Possibilities
Amit Vij | CEO and Co-founder | avij@kinetica.com
2. Challenge Faced by the US Army INSCOM
2
Move from document-based search to entity based search
200 sources of streaming data – SIGINT, ISR, HUMINT,
CYBINT
Finding the needle in the haystack from streaming big data
3. INTELLIGENCE | US Army - INSCOM
Oracle Spatial
(92 Minutes)
42x Lower Space
28x Lower Cost
38x Lower Power Cost
U.S Army INSCOM Shift from Oracle to Kinetica
Kinetica
(20ms)
1 Kinetica server vs 42 servers with Oracle 10gR2 (2012)
Available technologies didn’t stack up
3
Military leadership pushed industry to
create better solutions
4. 4
IoT Data Challenges with Geospatial and AI
REAL-TIME DEMANDS
Current Technology:
I/O Bound
Compute Bound
EXPLOSION OF DATA
Structured and unstructured
Devices, Sensors
Industrial IoT
EXISTING SOLUTIONS NOT WORKING
Too Complex
Batch Processing
Duct taping 5-10 technologies
4
5. Kinetica: GPU-Accelerated, Distributed, In-Memory Database
5
GPU-accelerated
database operations
NLP-based full
text search
Native GIS and IP-
address object support
Deep integration
Hadoop, Spark, NiFi,
Accumulo, Tableau,
Kibana
Linear scale out on
premise or in the cloud
Distributed visualization
pipeline
6. In-Database Analytics Architecture
ETL / STREAM
PROCESSING
ON DEMAND SCALE OUT +
1TB MEM / 2 GPU CARDS
SQL
Native
APIs
PARALLELINGEST
Geospatial
WMS
Custom
Connectors
In-Database Processing
CUSTOM LOGIC
BIDMach
MLLibs
BI DASHBOARDS
BI / GIS / APPS
CUSTOM APPS
& GEOSPATIAL
KINETICA ‘REVEAL’
STREAMINGDATAERP/CRM/
TRANSACTIONALDATA
UDFs
6
7. Michelangelo - Uber’s Machine Learning Platform
7
Intelligence
Streaming Data
Feedback
loop
• Two Separate Cassandra Clusters
• Online Cassandra Cluster
• Offline Cassandra Cluster
• Hadoop Data Lake
• Hive Feature Store
• Spark SQL
• Kafka
• Spark Streaming
• Machine Learning Libraries
8. Various
ETL/ELT
Head
Node
Worker
1
KINETICA: 10 Node Cluster
Worker
9
Fact and dimensions tables for various Use Cases
Billions of rows
Massive Stream
Ingestion
Massive Fast
Analytics
Apache Tomcat Applications Servers
• Spring Endpoint oriented architecture
• Horizontal elastic scaling
Full Model Pipeline 1
Various
ETL/ELT
Full Model Pipeline N
Kinetica’s Real-Time Machine Learning Database Platform
8
GPU Accelerated Database
and Machine Learning
Platform in Single Solution
Data Locality in a
Converged Architecture
Execute Machine Learning
algorithms via SQL
Open standard adapters
for data ingestion and
extraction
9. Real-time Route Optimization
USPS is the single largest logistics
company in the United States
CASE STUDY
NEW CAPABILITIES DELIVERED
• Real-time delivery and pickup notifications,
shipment routing, just-in-time supplies
• Real-time route optimization - route planning,
rerouting
• Geospatial analytics to uncover overlapping
coverage areas, uncovered areas, and
distribution bottlenecks
SOLUTION OVERVIEW
• Collect, process, and analyze 200,000
messages per minute for real-time streaming
analytics. 15,000 daily sessions with 5 9’s uptim
9
10. Kinetica Database | Geospatial 101
10
Geospatial Objects
z
Points
Lines
Polygons
Tracks
Labels
Spatial Operations
Accelerated Spatial Operations
SQL Expression & API Support
Spatial Queries, Filters & Joins
Geospatial Event Triggers
Geospatial Visualization
Server-side Rendering Vector data via
WMS
Complex Symbology Support
Several Built-in Geospatial Renderers
1
2
3
11. 11
Accelerated Geospatial with Kinetica | Fast, Scalable, Flexible
Solution
• Full data provisioning
• Scale and speed
• Flexibility
• Simplicity
Bonus
• Converge AI and BI
• Streaming Analytics
13. PREDICTIVE INFRASTRUCTURE MANAGEMENT
CASE STUDY
23
Kinetica operates as a speed-layer with
ESRI to monitor, manage, and predict
infrastructure health.
• Interactively view health of over 3 million poles
at all zoom levels
• Assess poles located within hazardous
environments (fire, snow and corrosion)
• Real-time calculations of pole statistics that
enable PG&E based on real-time map updates
• Identify failed assets and spatially locate closest
asset in same asset class
14. PIPE LINE & WELL RESEARCH | Location-based analytics
14
NEW CAPABILITIES DELIVERED
• Geospatial visualization and analytics of massive number of
wells, pipelines by land ownership, region etc.
• Custom visualizations and charts for data-driven insights
• Embedded solution with seamless Node.js integration, GPU
acceleration
SOLUTION OVERVIEW
• Kinetica running in RSEG’s Amazon Web Services VPC
deployment
15. Silicon Valley Validation- Kinetica Raises $50M Series A
15
“We posed what must have seemed to be a daunting technical challenge to Amit and Nima back in 2009. They rose
to the occasion and developed the precursor to what you see today as the Kinetica database platform when every
other commercial and open-source solution failed to meet the mission objectives. They are an example of brilliant
technical minds putting their talents to work in service to our country.”
Former Director of the NSA - GEN Keith B. Alexander