A look at the effects of big data on the energy sector

1. Big data in the energy sector
By John Tkaczewski

2. Agenda
| © Copyright FileCatalyst, 20142
The Big Data in Oil & GasThe Big Data in Oil & Gas
explorationexploration datadata
drillingdrilling datadata
productionproduction datadata
Data Supply ChainData Supply Chain
source to data centre forsource to data centre for analysisanalysis
toto stakeholdersstakeholders
Case StudyCase Study
Big data movement from vesselsBig data movement from vessels
to London data center toto London data center to
global stakeholdersglobal stakeholders

3. © Copyright FileCatalyst, 2014
Data-driven Oilfields and the data we gather during production.

4. © Copyright FileCatalyst, 2014
Digital Data gathered during exploration

5. © Copyright FileCatalyst, 2014
Big data during exploration…
• Logging While Drilling or Measurements While Drilling (LWD MWD)
• Hole size and shape
• structural analysis: Rock density and porosity
• Acoustic scanning for rock faults
• reservoir simulation
• seismic analysis
• fluid dynamics
• Engineering and construction

6. © Copyright FileCatalyst, 2014
... So we collected big data now what?
Need send it for further processing
• Analysis
• Simulation
• Interpretation
• High Performance Computing (HPC) Centers
• Data Distribution to stake holders

7. © Copyright FileCatalyst, 2014
Are we still shipping hard drives???

8. © Copyright FileCatalyst, 2014
Or should we use the Internet!
- FTP, Email, File Sharing
- Cloud
- CIFS

9. © Copyright FileCatalyst, 2014
• Last mile. Slower links between the operations and main data centers.
Satellite, 3G/4G/LTE, Microwave – slower unreliable links
• High speed fat pipes between data centers. 100 Mbps fiber connections. The
connections are fast but very far apart. (example Houston – London)
Sending big data – 2 fold problem

10. © Copyright FileCatalyst, 2014
Traditional file transfers will remain slow, regardless of the size of the link.
For example:
Transferring a 10GB file from London to Houston can take several hours.
The slow file transfer in turn will affect production and exploration
schedules.
Why traditional file transfers are so slow on the Internet?
• Latency (or delay on the network)
• Packet loss (percentage of data lost during transmission)

11. TCP Overview
What is TCP? Transmission Control Protocol
• Works well for most internet traffic, email, web browsing
small ad-hoc transfers (HTTP, FTP, SFTP, SMTP etc)
• Provides reliability, error checking, ordered packets in a
network stream
• Congestion control built in
• Internet could not survive without it
© Copyright FileCatalyst, 2014

12. File Transfer Issues with TCP
• Flow control limits transmission window, causes “dead air” on
the line when high latency present
• Very aggressive in response to network congestion
• Result is less than ideal performance on wireless, satellite, or
long haul links
• TCP Can be tuned but still not ideal
© Copyright FileCatalyst, 2014
TCP is a very serial process.
Each packet of data must be received before a
new packet is sent = Decreased transfer speed
Source File
Destination
File
Acknowledgments
Data Packet

13. File transfer acceleration via UDP
• Ideal for bulk file transfer
• Predictable - Can send at a perfect rate
• Not affected by latency or packet loss
• Congestion Control implemented in application layer
• Tunable congestion control aggression
• Instantly detect link capacity
© Copyright FileCatalyst, 2014
Source File
Destination
File
Acknowledgments
FileCatalyst completely saturates the pipe by
sending multiple blocks of data = Increased
transfer speed

14. © Copyright FileCatalyst, 2014
*NA = North America
Bandwidth Improvement example
Accelerating File Transfers

15. © Copyright FileCatalyst, 2014
• WAN optimization hardware
• Commercial off-the shelf (COTS) file transfer acceleration
software
• Open source UDP based file transfer accelerations applications
File transfer acceleration options:

16. © Copyright FileCatalyst, 2014
WAN optimizers
• Pros:
• Almost Transparent
• Optimizes all the network traffic including FTP
• Hardware solution
• Cons:
• Hardware solution
• Cost
• Lack of file transfer tools such as automation, monitoring
and QoS

17. © Copyright FileCatalyst, 2014
File transfer COTS
• Pros:
• Flexible software only deployment
• Enterprise tools for transferring files are included
• Web applications that include accelerated file transfer
• Cons:
• Less expensive then hardware but not free
• Limited to file transfers only
• Some initial configuration required

18. © Copyright FileCatalyst, 2014
Open Source file transfer applications
• Pros:
• Free
• Flexible software only deployment
• Cons:
• Not adaptable to changing network conditions
• no tools advanced tools (command line or API’s only)
• Complex initial configuration required

19. Considerations for FTP Acceleration
• Plays fair with other traffic on network
• Top speed at 10 Gbps file transfer, single stream.
• Maintain high speed even with high latency and packet loss.
• High performance for many small files
• Security, reliability, automatic re-try and resume
• Automation tools, SDK, central management and configuration
• Rsync and Compression built-in
© Copyright FileCatalyst, 2014

20. © Copyright FileCatalyst, 2014
Case study: marine exploration
Project Requirements:
• Multiple ships delivering data in real time to London
• London HPC center provides data analysis
• Final data to be delivered to major markets in Houston and
Malaysia
Using FTP:
• Data from ships was not deliverable
• Final data deliveries taking hours or not arriving at all

21. © Copyright FileCatalyst, 2014
Case study: marine exploration

22. © Copyright FileCatalyst, 2014
Case study: marine exploration
Results:
• More control over file transfers and a dramatic speed
improvement .
• 10 Gbps line was finally used to its full extent
• All file transfers are a simple and straightforward experience,
logging all file transfers and exceptions
• End user experience for both clients and back office processes
dramatically improved

23. © Copyright FileCatalyst, 2014
Forecast for tomorrow
• Big data will only get bigger
• Global connectivity will get faster and more reliable
• Latency will remain as we can’t increase the speed of light
• Progressive tech will shorten the exploration cycle and ensure
continuous success of oil and gas extraction

24. Questions?