Intelie is a Brazilian data technology company focused on optimizing operations through stream data analytics, helping clients like Petrobras and Shell in various industries worldwide. They provide real-time data monitoring and analytics solutions, particularly for well operations in the Libra oil field, utilizing a flexible technology that allows engineers to create custom analyses without needing extensive programming skills. The document emphasizes the importance of collaboration between different technologies and teams to effectively address real-world challenges in data analytics.

1. Beyond Stream Analytics
Ricardo Clemente
ricardo@intelie.com
Joining different technologies and smart people
together to solve real problems
Based on a real case application on a major operator

2. Intelie is a data technology company originally founded in Brazil,
with presence in Rio de Janeiro, São Paulo, Macaé and Houston.
Our mission is to optimize critical operations
through technological solutions specialized in
stream data analytics associated with high data
volumes and high data throughput.
Since the beginning, Intelie has built a solid customer base by
delivering results in high-profile clients in different industries and
countries like USA, Malaysia, China and Brazil.
INTELIE

3. STREAM ANALYTICS INTO PRACTICE
Libra oil field is a large ultra-
deepwater oil prospect located in
the Santos Basin, about 230
kilometres (140 mi) off the coast of
Rio de Janeiro, Brazil. The most
probable estimate being 7.9 billion
barrels.
Petrobras (operator, with 40%),
Shell (20%), Total (20%), CNOOC
(10%) and CNPC (10%)
Real-time monitoring and analytics
for well operations (drilling,
cementing, interventions,
completion)
Case started at R&D center and are being used for Libra field exploration

4. LEARNING FROM THE TRENCHES
Empower engineers and R&D:
• Flexibility to create new analysis and visualization without the
need of software development
• Extensibility to reduce the lead time of new data-related
heuristics, algorithms, visualization and applications;
Make the best tools and people to collaborate:
• There is no silver bullet. The best tools to address the each real
problem in corporation scale must work together and
collaborate.
• Make it easier for people to consume and collaborate.
Join engineering approach with machine learning approach:
• Use what is best for each problem;
• Try new approach to problems;
• When possible combine them into one solution

5. Stream analytics definition:
Software that can filter, aggregate, enrich, and analyze a
high throughput of data from multiple, disparate live data
sources and in any data format to identify simple and
complex patterns to provide applications with context to
detect opportune situations, automate immediate actions,
and dynamically adapt.
STREAM ANALYTICS - REAL TIME BIG DATA
The Forrester Wave™: Big Data Streaming Analytics, Q1 2016

6. STREAM ANALYTICS - the upside-down database
The upside-down database analogy
...
data
query 1 query n
continuous
query n
continuous
query 1 ...
relational model stream model
data
data

7. STREAM ANALYTICS - Deeper understanding
Babcock, Brian, et al. "Models and issues in data stream systems."Proceedings of the
twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems.
ACM, 2002.

8. ARCHITECTURAL VIEW

9. STREAM ANALYTICS FOR DRILLING
pressure woh
flow
continuous
queries
rop
temp
gama-ray
bit depth
WITSML
protocol Data preparation
Data distribution
Stream intelligence
Simple analysis
Patterns
Machine Learning
Visualization

10. Read data from rigs and
wells using industry
protocol (WITSML)
Ensure the right
mnemonics semantic and
units use
Manage and notify data
suppliers promptly
DATA CAPTURE, NORMALIZATION AND QUALITY
Data preparation
Service companies have different
names for variables and different units;
Hundreds of sensors per rig, coming
from different systems;
Communication stalls, sensors breaks,
service fails, ...
Challenges faced
Mapping and identifying
changes
Event-driven and pub / sub
Data fusion, quality
monitoring
Solution

11. IN-MEMORY DATA STREAM ANALYSIS ENGINE
Enabling user defined patterns
INTELIE PIPES
A specific language and in-memory engine
to analyze data stream from sensors.
Filter rig42
Detect pressure
increase (A)
Detect Bit Depth
decrease (B)
(A) AND (B)
Chained logic model
Stream intelligence

12. IN-MEMORY DATA STREAM ANALYSIS ENGINE
Enabling user defined patterns
INTELIE PIPES
A specific language and in-memory engine
to analyze data stream from sensors.
rig42
=> [ @filter mnemonic:SPP
=> avg(value#) as last_pressure every 20 minutes
=> expand *, _ > :prev(1) and :prev(1) > :prev(2) and :prev(2) > :prev(3) as
pressure_increase,
:prev(3) as first_pressure, last_pressure-:prev(3) as pressure_increase_amount
=> expand last(*) every 5 minutes
join
@filter mnemonic:DBTM
=> avg(value#) as last_bit_depth every 5 minutes
=> expand *, :prev - _ >= 5 and as bit_depth_decrease, :prev as first_bit_depth, :prev
- _ as bit_depth_decrease_amount]
=> @filter bit_depth_decrease and pressure_increase
filter rig42
bit depth decrease (B)
pressure increase (A)
A AND B
Stream intelligence

13. DISTRIBUTED QUERY FOR REAL-TIME ADVANCED ANALYTICS
Sensor

14. DISTRIBUTED QUERY FOR REAL-TIME ADVANCED ANALYTICS
MAP
REDUCER
REDUCERREDUCER
HyperLogLog1 probabilistic data structure to
optimize memory consumption
1- Flajolet, P.; Fusy, E.; Gandouet, O.; Meunier, F. (2007). "HyperLogLog:the analysis of a near-optimal cardinality
estimation algorithm“. AOFA ’07: Proceedings of the 2007 International Conference on the Analysis of Algorithms.
Hidden complexity

15. VISUAL EXTENSIBILITY
Enabling user defined visualization
Stream intelligence
The “look what I did” effect!

16. ANALITICAL EXTENSIBILITY
Weight On Bit
Hole Depth
Rotary Speed
Rate Of Penetration
Heave Bit Depth
Flow Rate
Block Position
Weight On Hook
Custom
classification
algorithm,
extending
Intelie PIPES
INPUT SIGNALS IDENTIFIED OPERATION
Enabling user defined patterns and visual extensibility
Using Java write the algorithm to extend Intelie PIPE functions
Stream intelligence
Let Phd´s and data scientist focus on the core

17. ANALITICAL EXTENSIBILITY
Enabling user defined patterns and visual extensibility
Stream intelligence

18. MACHINE LEARNING INTO PRACTICE
Mud weight prediction for pre-salt drilling
zones
Stream analytics can prepare data and embed powerful algorithms
Data Stream Anomaly Detection through
Principal Subspace Tracking
*independent software
Stream intelligence
*not applied in O&G yet

19. STREAM ANALYTICS + MACHINE LEARNING
Stream intelligence
“The main area of research for the
future is to investigate the
possibility to build a learning
module to detect anomalies in an
unsupervised manner, as proven by
the HOLMES project”
Kazarov, A., G. Lehmann Miotto, and L. Magnoni. "The AAL project: automated monitoring and
intelligent analysis for the ATLAS data taking infrastructure." Journal of Physics: Conference
Series. Vol. 368. No. 1. IOP Publishing, 2012.

20. CUSTOM APP EXAMPLE
BOP monitoring and right EDS selection control
loading ...
Stream intelligence

21. BOPs Become the Focus of Data-Driven Scrutiny
(Data-Driven BOP on July JPT edition)

22. APPs INTEGRATION
Pronova integration
Data distribution

23. (APPs INTEGRATION)
Slack application example on Github
Data distribution
https://github.com/intelie/plugin-slack

24. APPs INTEGRATION
PWDa (Petrobras in-house anomaly detection software) two-way integration
Data distribution

25. APPs INTEGRATION
Data distribution
Intelie Live Platform
Python
Application
Matlab set of
algorithms
ProNova
Pub/Sub
WEBMANAGEMENT
streaming data

26. Real-time analytics in O&G / E&P
“Simple” foundation problems like data integration, data quality,
data governance and monitoring still have to be solved. Do not
underestimate them.
Empower, using a platform approach, your company, or suppliers,
engineers, devs and data scientist to solve real-time analytics
related problems. It´s all about good people doing their jobs.
Embrace the best tools in the market to solve specific analytics
related problems. Make them also to collaborate,
There is no artificial intelligence silver bullet.
What I have learned... and would like for you to keep in mind!

27. Future
Drone Pilot
Jet Pilot

28. Ricardo Clemente
ricardo@intelie.com
Q&A
@rick