The document discusses a briefing on harnessing real-time data analytics and features insights from industry experts, including David Loshin of Knowledge Integrity and Dan Potter of Datawatch. The briefing covers a range of topics including the characteristics of enterprise software, the evolution of visual analytics platforms, and the significance of streaming data in various sectors like finance and healthcare. It emphasizes the increasing need for advanced analytic capabilities to handle the growing volume and complexity of real-time data.

1. Grab some
coffee and
enjoy the
pre-show
banter before
the top of the
hour!

2. The Briefing Room
Moving Targets: Harnessing Real-Time Value from Data in Motion

3. Twitter Tag: #briefr The Briefing Room
Welcome
Host:
Eric Kavanagh
eric.kavanagh@bloorgroup.com
@eric_kavanagh

4. Twitter Tag: #briefr The Briefing Room
Reveal the essential characteristics of enterprise
software, good and bad
Provide a forum for detailed analysis of today s innovative
technologies
Give vendors a chance to explain their product to savvy
analysts
Allow audience members to pose serious questions... and
get answers!
Mission

5. Twitter Tag: #briefr The Briefing Room
Topics
February: DATA IN MOTION
March: BI/ANALYTICS
April: BIG DATA

6. Twitter Tag: #briefr The Briefing Room
Parmenides and the Truth of Now
There is no tomorrow
There is no yesterday
There is only today
There is only now

7. Twitter Tag: #briefr The Briefing Room
Analyst: David Loshin
David Loshin, president of Knowledge
Integrity, Inc, is a thought leader and
expert consultant in the areas of data
quality, master data management, and
business intelligence. David is the
author of numerous books and papers
on data management, including the
“Practitioner’s Guide to Data Quality
Improvement.” David is a frequent
speaker at conferences and in web
seminars. His best-selling book, “Master
Data Management,” has been endorsed
by data management industry leaders.
David can be reached at
loshin@knowledge-integrity.com, or at
(301) 754-6350.

8. Twitter Tag: #briefr The Briefing Room
Datawatch
Datawatch began as a BI tool and has developed into a
visual analytics platform
The platform provides visual data analytics and discovery on
any type of data, including streaming data
The suite of products are Datawatch Desktop, Datawatch
Server, Datawatch Report Mining Server and Datawatch
Modeler

9. Twitter Tag: #briefr The Briefing Room
Guest: Dan Potter
Dan Potter is the Vice President of
Product Marketing at Datawatch
Corporation. In this role, Dan leads
the product marketing and go-to-
market strategy for Datawatch. Prior
to Datawatch, Dan held senior roles at
IBM, Oracle, Progress Software and
Attunity where he was responsible for
identifying and launching solutions
across a variety of emerging markets,
including cloud computing, visual data
discovery, real-time data streaming,
federated data and e-commerce.

10. VISUAL DATA DISCOVERY
& STREAMING DATA
New Technologies for Real-Time Analytics
Dan Potter
Vice President, Product Marketing

11. NASDAQ: DWCH
Pioneer in real-time visual data discovery and self-service data
preparation
Global operations and support
§ US, UK, Germany, France, Australia, Singapore, Philippines
Extensive global customer base
§ 93 of the Fortune 100
§ 12 of the 15 largest financial institutions
Embedded and resold by leading vendors
About Datawatch

12. DISCOVER
GOVERN
ACQUIRE
PREPARE
AUTOMATE
Visual Analytics Platform
For Any Data at Any Speed

13. Where Do Real-Time Streams Come From?
• Internet of Things
• Machine data / log files
• Web clickstreams
• Enterprise applications
• Human generated
• Commercial data

14. Streaming Visualization Examples
Capital
Markets
§ Transac'on
Cost
Analysis
§ Analyze
market
data
at
ultra-­‐low
latencies
§ Momentum
Calculator
Fraud
preven2on
§ Detec'ng
mul'-­‐party
fraud
§ Real
'me
fraud
preven'on
e-­‐Science
§ Space
weather
predic'on
§ Detec'on
of
transient
events
§ Synchrotron
atomic
research
§ Genomic
Research
Transporta2on
§ Intelligent
traffic
management
§ Automo've
Telema'cs
Energy
&
U2li2es
§ Transac've
control
§ Phasor
Monitoring
Unit
§ Down
hole
sensor
monitoring
Natural
Systems
§ Wildfire
management
§ Water
management
Other
§ Manufacturing
§ ERP
for
Commodi'es
§ Real-­‐'me
mul'modal
surveillance
§ Situa'onal
awareness
§ Cyber
security
detec'on
§ Emergency
Evacua'on
Law
Enforcement,
Defense
&
Cyber
Security
Health
&
Life
Sciences
§ ICU
monitoring
§ Epidemic
early
warning
§ Remote
healthcare
monitoring
Telephony
§ CDR
processing
§ Social
analysis
§ Churn
predic'on
§ Geomapping

15. Visual Data Discovery
• Easy for users to author,
customize and share
• Interactive exploration &
visually filter results
• Quickly identify
anomalies and outliers
with large or in-motion
datasets
• Rich palette of
visualizations for static
and time series data

16. Visualize Any Data at Any Speed
Stream
Rela2onal
NoSQL
OLAP
Warehouse
Hadoop
Content
Connect,
Federate,
Visualize

17. Data Architectures Evolving
Database
Distributed
or
Hybrid
Database
In-­‐Memory
Database
Streaming
Analy'cs
Faster
Speed,
Faster
Insights

18. Data
at
Rest
Limitations of Traditional BI
Database
Distributed
or
Hybrid
Database
In-­‐Memory
Database
Streaming
Analy'cs

19. Data
at
Rest
Streaming Data Visualization
Database
Distributed
or
Hybrid
Database
In-­‐Memory
Database
Streaming
Analy'cs

20. Datawatch Streaming Data Visualization
• Connect directly to data in motion
• CEP (IBM Streams, Informatica Rulepoint, Tibco Streambase)
• Hosted IoT platforms (Amazon Kinesis, PTC ThingWorx)
• Message Bus (Informatica UltraMessaging, WebSphere MQ)
• Operational Intelligence Systems (OSIsoft Pi)
• Purpose built data model optimized for both caching
and persistence
• High density visuals with rendering in milliseconds
Monitor
Analyze
Take
Ac2on

21. Time Series Data
• Traditional BI only looks at buckets of
time
• Day, week, month, year
• Streaming data is a continuous and has
different requirements
• Second, millisecond, nanosecond
• Time windows
• Time slices
• Playback
• Complete situational awareness
• Now (streaming)
• Intra-day
• Historic

22. Predictive & Advanced Analytics
• Connect to R (Rserv) and
Python (Pyro) servers
• Transform using R and
Python
• Many use cases in IoT (e.g.
predictive maintenance,
smart logistics, clinical
pattern detection etc.)

23. Modeled
and
transformed
for
analysis
Complex File Formats
• Sensor and machine data often in multi-structured format
• Need to transform, enrich and prepare data
• Almost no metadata
• For example, wave form visualization from JSON arrays
stored in MongoDB and streaming
23
Log
Files
HTML,
XML
JSON
PDFs

24. Real-Time Geospatial & Location
• Real-time (stream) plotting
• Street-level geo maps or
custom SVG files
• Time-series playback
Healthcare
Retail
Logis'cs
U'li'es

25. Customer Challenge
Dozens of risk management
systems generating data silos of
operational information
Server based solution to
visualize integrated risk
information in real-time to
identify trends and anomalies
Analyze patterns in physiological
data that may detect and
eventually to predict deadly
clinical events
Visualize large volumes of
streaming, unstructured data
from multiple devices in real-time
Improve yield production and
enhance machine reliability in
contact lens manufacturing
process
Flexible visualization solution
highlighting production line
yield, leading to a 2% yield
increase and 750,000 additional
units produced
Real-World, Real-Time Examples
Process and visualize billions of
streaming trades per day for
leading surveillance and
compliance platform
Fully embedded visual data
discovery solution that delivers a
single consolidated real-time
view of trading across venues

27. Twitter Tag: #briefr The Briefing Room
Perceptions & Questions
Analyst:
David Loshin

28. Brie%ing
Room
02-­‐17-­‐2015:
Considerations
for
Streaming
Analytics
2015-­‐02-­‐17
David
Loshin
Knowledge
Integrity,
Inc.
loshin@knowledge-­‐integrity.com
©
2015
Knowledge
Integrity,
Inc
loshin@knowledge-­‐integrity.com
(301)
754-­‐6350
28

29. Technology
Convergence
&
Stream
Analysis
• Discovery
&
Streaming
Analy'cs
employs
a
number
of
key
evolving
technologies
beyond
the
expected
“repor'ng
&
analy'cs”:
– Data
virtualiza'on
and
federa'on
– Text
parsing
and
text
analy'cs
– Seman'c
models
– Real-­‐'me
data
inges'on
– Event
stream
processing
– Embedded
rules
for
monitoring,
no'fica'on,
and
alerts
– In-­‐memory
processing
– Visualiza'on
• Con'nued
improvements
in
these
technologies
will
automa'cally
improve
the
quality
and
speed
of
real-­‐'me
stream
analy'cs
©
2015
Knowledge
Integrity,
Inc
loshin@knowledge-­‐integrity.com
(301)
754-­‐6350
29

30. Future
Direction
of
Connected
Devices?
• More
“things”
will
be
networked
– Who’d
a
thunk
that
thermostats
would
be
in
the
first
wave
of
smart
devices?
• Networked
“things”
will
be
gegng
“smarter”
– More
&
beher
resources
at
the
device
• Increased
open-­‐source
standardiza'on
– Including
the
hardware!
• Increased
ease
of
programmability
expands
the
community
of
developers
– A
12-­‐year
old
can
program
this
©
2015
Knowledge
Integrity,
Inc
loshin@knowledge-­‐integrity.com
(301)
754-­‐6350
30

31. Considerations
• The
volume
and
variety
of
human-­‐generated
content
will
con'nue
to
explode
– This
will
require
increased
analy7c
intelligence
for
parsing
and
filtering
within
the
network
• Par'al
analy'c
computa'ons
can
be
pushed
out
to
the
devices
– Move
the
applica7on
to
the
data,
not
the
data
to
the
applica7on
• Alerts
and
no'fica'ons
base
on
the
results
of
intermediate
analyses
can
provide
advantage
in
mul'ple
ways
– The
same
data
streams
can
feed
a
wide
variety
of
consumer
communi7es
• Streaming
analy'cs
will
become
relevant
at
the
personal
as
well
as
the
business
level
– Enable
personalized
algorithmic
stream
blending,
analysis,
and
monitoring/no7fica7on
at
the
mobile
device
©
2015
Knowledge
Integrity,
Inc
loshin@knowledge-­‐integrity.com
(301)
754-­‐6350
31

32. Questions
to
Explore
• What
has
predicated
the
growth
in
demand
for
analyzing
streaming
data
in
recent
years?
• What
are
the
types
of
streaming
data
that
are
most
frequently
subjected
to
analysis?
• What
are
the
features
of
your
product
that
have
been
most
valuable
to
your
customer
community,
and
why?
• How
does
your
product
help
business
users
dis'nguish
relevant
streaming
content
from
the
“noise”?
• Can
you
share
some
insight
into
how
your
tool
uses
in-­‐memory
processing
and
manages
data
in
memory?
• What
fundamental
differences
do
you
see
between
the
ability
to
enable
analysis
of
human-­‐generated
content
vs.
machine-­‐generated
streaming
content?
• Can
you
share
thoughts
about
external
constraints
that
prevent
the
best
opportuni'es
for
using
streaming
analy'cs
and
discovery?
• What
do
you
see
as
the
next
hurdles
in
enabling
business
consumers
in
adop'ng
discovery
analy'cs
for
streaming
data?
• Who
are
the
compe'tors
and
what
do
you
see
as
the
advantages
your
tool
provides
over
your
compe'tors?
©
2015
Knowledge
Integrity,
Inc
loshin@knowledge-­‐integrity.com
(301)
754-­‐6350
32

33. Following
up…
• www.knowledge-­‐integrity.com
• www.dataqualitybook.com
• www.decisionworx.com
• If
you
have
ques'ons,
comments,
or
sugges'ons,
please
contact
me
David
Loshin
301-­‐754-­‐6350
loshin@knowledge-­‐integrity.com
©
2015
Knowledge
Integrity,
Inc
loshin@knowledge-­‐integrity.com
(301)
754-­‐6350
33

34. Twitter Tag: #briefr The Briefing Room

35. Twitter Tag: #briefr The Briefing Room
Upcoming Topics
www.insideanalysis.com
February: DATA IN MOTION
March: BI/ANALYTICS
April: BIG DATA

36. Twitter Tag: #briefr The Briefing Room
THANK YOU
for your
ATTENTION!
Some images provided courtesy of
Wikimedia Commons and Wikipedia