MongoDB as a universal data store for
Process Data

Munich
2013
Munich
2013

The talk in brief
It is all about two technology standards (coming together)…
Munich
2013

Timo Klingenmeier
Co-founder, GM and Technical Lead inmation.com
timo.klingenmeier@inmation.com

89-93

EMP (...
Munich
2013
Munich
2013
Munich
2013

Realtime

1ms – 1sec
Munich
2013

ERP / B2B / B2C
MES and PIMS
Control & Automation
Munich
2013

MES
and

PIMS
ACRONYM ALERT!
MES = Manufacturing Execution System
PIMS = Process Information Management Syste...
Munich
2013

1
Refining Company
15
Sites
2000
Applications

CAPEX

OPEX (Maintenance Contracts, etc.)
Munich
2013

Control

Typical Production Process Applications
Advanced Process Control

Mass Balancing

Laboratory Informa...
Munich
2013

The evolution of
industrial system integration
(a brief history of time)
Munich
2013

Industry 3.0

Emptying the parking lots

Source: flickr.com/acarlos1000
Munich
2013
Munich
2013

Today

Distributed
Control Systems

Entire Sites

Countless Items
and Actors
Munich
2013

The Spinal Cord in Production:
Contextualized Time-Series
historization of numerical data
and process events.
Munich
2013

The easiest way to
improve a prediction is
to add data. You can’t
infer without data. So,
store the data now ...
Munich
2013

The
Babylonian
Era
Munich
2013

Protocol Differences

Information
Contextualization

Bandwidth and
Throughput

Implementation
Complexity

Rob...
Munich
2013
Munich
2013

Birth of a standard
Munich
2013
Munich
2013

You can talk
DCOM to me.
I will present you a qualified and more or
less structured namespace, consisting of
...
Munich
2013

You can talk
DCOM to me.
On request, I will constantly deliver new
Alarms & Events of any kind. Depending on
...
Munich
2013

You can talk
DCOM to me.
Similar to my DA colleague on the left, I
maintain a structured namespace of tags.
U...
Munich
2013

You can talk SOAP / XML to me.
I can do what the DA guy does, but a little
simplified.
Munich
2013

You can talk either binary TCP or XML to me.
I offer various options for secure communications.
I have many d...
Munich
2013
Munich
2013

You can talk to me.
(Everybody knows that)
Very similar to the SQL language, I’m not
young of age but definit...
Munich
2013

One standard – many faces
Munich
2013

Globally standardized

Universal Real-time
Data Access

Unknown to broader
audience
Munich
2013

Industrial IT Data Storage

Strategies Today

Proprietary High-Frequency
Time-Series Data Formats

SQL
Databa...
Munich
2013

Application Differences

Supported
Interfaces

Data Scope
Extraction

Data Format
Storage

Interfaces to
next...
Munich
2013

Too
many
data silos
Munich
2013

Data Reduction & Loss
Munich
2013

How can industries create
affordable, maintainable, open
data stores which allow for the
“Merriman paradigm” ...
Munich
2013

the
approach
Munich
2013

The problem to solve
Munich
2013

OS Platform? Scalability? Make your choice!
...or simply grow as required
Munich
2013

How we use MongoDB in our product
12 months ago, there were SQL parts
There was homegrown data serialization
...
Munich
2013

BSON

Command Processor

MongoDB

Core
cmd
cmd_errors

inSys
cmd_processed

BSON

Working Model

Working Mode...
Munich
2013

Remote
OPC UA Server

Endpoint
Remote
XML-DA Server

Endpoint
Remote/Local
OPC DA Server

Endpoint
Remote/Loc...
Munich
2013

Remote
OPC UA Server

Endpoint
Remote
XML-DA Server

SPROX Protocol (Single Port TCP)
Secure Prioritized Real...
Munich
2013

Remote
OPC UA Server

Endpoint
Remote
XML-DA Server

Schema Design:
• Multiple Databases
• Multiple Collectio...
Munich
2013

Third-Party Stack (API)
Command Application
•
•
•
•
•
•

Realtime Data Access
Component Registration
Historic...
Munich
2013

DataStudio
•
•
•
•
•

Component Registration
Component Configuration
System Health Monitoring
Item Monitoring...
Munich
2013

OPC UA Server

DataProxy

BOX Protocol (Single Port TCP)
BSON Object Xchange

Connector Service

Core Service...
Munich
2013
Munich
2013
Munich
2013

Using MongoDB-based
Real-time Data
(and a little bit of Google, too)
Munich
2013
Munich
2013

Summary – MongoDB qualifies for
process data storage
It has the performance and the scalability options
requi...
Munich
2013

Thank you very much for your
attention
Upcoming SlideShare
Loading in...5
×

Mongo db as a universal data store for process data

977

Published on

Published in: Technology, Travel
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
977
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
76
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Mongo db as a universal data store for process data

  1. 1. MongoDB as a universal data store for Process Data Munich 2013
  2. 2. Munich 2013 The talk in brief It is all about two technology standards (coming together)…
  3. 3. Munich 2013 Timo Klingenmeier Co-founder, GM and Technical Lead inmation.com timo.klingenmeier@inmation.com 89-93 EMP (Engineering Company) Software Developer 93-03 GM, Founder iDV/best solutions AG Industrial System Integration, Custom SW Development 03-10 GM, Matrikon Deutschland AG Industrial Software Specialist, Real-Time Connectivity, Alarm Management, Performance Monitoring 11/12 Consultant Offshore Wind Projects 13 inmation.com NextGen Information Management Software
  4. 4. Munich 2013
  5. 5. Munich 2013
  6. 6. Munich 2013 Realtime 1ms – 1sec
  7. 7. Munich 2013 ERP / B2B / B2C MES and PIMS Control & Automation
  8. 8. Munich 2013 MES and PIMS ACRONYM ALERT! MES = Manufacturing Execution System PIMS = Process Information Management System
  9. 9. Munich 2013 1 Refining Company 15 Sites 2000 Applications CAPEX OPEX (Maintenance Contracts, etc.)
  10. 10. Munich 2013 Control Typical Production Process Applications Advanced Process Control Mass Balancing Laboratory Information Management Production Scheduling Alarm Management Quality Management Operational Excellence Performance, Reliability, Safety Management ERP Continuous Improvement (Generic Data-Mining)
  11. 11. Munich 2013 The evolution of industrial system integration (a brief history of time)
  12. 12. Munich 2013 Industry 3.0 Emptying the parking lots Source: flickr.com/acarlos1000
  13. 13. Munich 2013
  14. 14. Munich 2013 Today Distributed Control Systems Entire Sites Countless Items and Actors
  15. 15. Munich 2013 The Spinal Cord in Production: Contextualized Time-Series historization of numerical data and process events.
  16. 16. Munich 2013 The easiest way to improve a prediction is to add data. You can’t infer without data. So, store the data now and analyze it later,…[] Dwight Merriman, MongoDB evangelist
  17. 17. Munich 2013 The Babylonian Era
  18. 18. Munich 2013 Protocol Differences Information Contextualization Bandwidth and Throughput Implementation Complexity Robustness
  19. 19. Munich 2013
  20. 20. Munich 2013 Birth of a standard
  21. 21. Munich 2013
  22. 22. Munich 2013 You can talk DCOM to me. I will present you a qualified and more or less structured namespace, consisting of symbolically named items (tags). Each item may have additional properties. I can give you the actual value, the milisecond accuracy and the value quality.
  23. 23. Munich 2013 You can talk DCOM to me. On request, I will constantly deliver new Alarms & Events of any kind. Depending on the subordinated control systems I am connected to, the detail content of a single event record may vary. I can either supply all or filtered events.
  24. 24. Munich 2013 You can talk DCOM to me. Similar to my DA colleague on the left, I maintain a structured namespace of tags. Unlike this guy, I know about the entire history of their values. On request I will return raw values and statistical aggregates for any period of time.
  25. 25. Munich 2013 You can talk SOAP / XML to me. I can do what the DA guy does, but a little simplified.
  26. 26. Munich 2013 You can talk either binary TCP or XML to me. I offer various options for secure communications. I have many different profiles and facets. (Some people are confused about me) My smallest incarnation can work in a single chip solution, while I’m still qualifying for an enterprise-wide service. Obviously – Unified Architecture – I can supply all services of those little guys in one.
  27. 27. Munich 2013
  28. 28. Munich 2013 You can talk to me. (Everybody knows that) Very similar to the SQL language, I’m not young of age but definitely not willing to retire!
  29. 29. Munich 2013 One standard – many faces
  30. 30. Munich 2013 Globally standardized Universal Real-time Data Access Unknown to broader audience
  31. 31. Munich 2013 Industrial IT Data Storage Strategies Today Proprietary High-Frequency Time-Series Data Formats SQL Databases
  32. 32. Munich 2013 Application Differences Supported Interfaces Data Scope Extraction Data Format Storage Interfaces to next level
  33. 33. Munich 2013 Too many data silos
  34. 34. Munich 2013 Data Reduction & Loss
  35. 35. Munich 2013 How can industries create affordable, maintainable, open data stores which allow for the “Merriman paradigm” in the specific context of industrial data mining requirements ?
  36. 36. Munich 2013 the approach
  37. 37. Munich 2013 The problem to solve
  38. 38. Munich 2013 OS Platform? Scalability? Make your choice! ...or simply grow as required
  39. 39. Munich 2013 How we use MongoDB in our product 12 months ago, there were SQL parts There was homegrown data serialization Today, we only use MongoDB for any kind of data storage All network transports use inner BSON chunks, extended for efficient real-time object communication
  40. 40. Munich 2013 BSON Command Processor MongoDB Core cmd cmd_errors inSys cmd_processed BSON Working Model Working Model Core Object Connector Object BSON BSON Connector Object Endpoint Object Endpoint Object Endpoint Connector
  41. 41. Munich 2013 Remote OPC UA Server Endpoint Remote XML-DA Server Endpoint Remote/Local OPC DA Server Endpoint Remote/Local OPC A&E Server Endpoint Remote /Local OPC HDA Server Endpoint Connector Service
  42. 42. Munich 2013 Remote OPC UA Server Endpoint Remote XML-DA Server SPROX Protocol (Single Port TCP) Secure Prioritized Realtime Object Xchange Endpoint Remote/Local OPC DA Server Endpoint Remote/Local OPC A&E Server Endpoint Remote /Local OPC HDA Server Endpoint Connector Service Core Service
  43. 43. Munich 2013 Remote OPC UA Server Endpoint Remote XML-DA Server Schema Design: • Multiple Databases • Multiple Collections Database Design: • Replication (Redundancy) • Port TCP) SPROX Protocol (Single Sharding (Horizontal Scaling) Secure Prioritized Realtime Object Xchange Endpoint Remote/Local OPC DA Server Endpoint Connector Service MongoDB Remote/Local OPC A&E Server Different OPC Servers Endpoint Remote /Local OPC HDA Server Endpoint Mongo BSON Object Bulk Inserts Core Service
  44. 44. Munich 2013 Third-Party Stack (API) Command Application • • • • • • Realtime Data Access Component Registration Historical Data Access ComponentEvents Alarms and Configuration System Health Monitoring Any MongoDB Driver C++, C#, Java, … Any MongoDB Driver C++, C#, Java, … Connector Service Core Service MongoDB
  45. 45. Munich 2013 DataStudio • • • • • Component Registration Component Configuration System Health Monitoring Item Monitoring Log Analysis BOX Protocol (Single Port TCP) BSON Object Xchange Connector Service Core Service MongoDB
  46. 46. Munich 2013 OPC UA Server DataProxy BOX Protocol (Single Port TCP) BSON Object Xchange Connector Service Core Service MongoDB BSON MongoDB
  47. 47. Munich 2013
  48. 48. Munich 2013
  49. 49. Munich 2013 Using MongoDB-based Real-time Data (and a little bit of Google, too)
  50. 50. Munich 2013
  51. 51. Munich 2013 Summary – MongoDB qualifies for process data storage It has the performance and the scalability options required to store hi-freq data in huge amounts Timestamp accuracy is sufficient Schema (-less) flexibility fits the variant data structures, originated in (OPC) source systems Unbeatable value offer (both software and hardware utilization) It exposes ‘natural’ data structures which make any kind of analysis fun It satisfies IT people and engineers
  52. 52. Munich 2013 Thank you very much for your attention
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×