This document discusses securely networking big data environments. It provides an overview of how Hadoop Distributed File System (HDFS) works with different phases of data processing and movement. It then examines the network characteristics needed to support big data like availability, burst handling, latency, scale, and security. Traditional networks are prone to issues supporting big data workloads due to oversubscription and lack of security controls. Network fabrics provide better availability, burst handling, latency, scale, and security through features like ECMP, low oversubscription, and microsegmentation. The document recommends using a network fabric built on software-defined networking and network functions virtualization to securely network big data environments and workloads.

1. Securely Networking Big Data
Environments
Stephen Hampton
CTO, Hutchinson Networks

2. Big Data On The Network – An Example
• HDFS (Hadoop Distributed File System)
• Name Node (HDFS Brain, List of Blocks & Meta Data)
• Data Node (Independent Nodes, Capable of Executing Workloads)
• Process
• Write Data (Data Blocks Uploaded)
• Workload Execution
• Map Phase (Little or no data)
• Shuffle Phase (Data on the network)
• Reduce Phase (Little or no data)
• Output Replication (Data on the network)
• Reading Data (Data Read By Application)

3. Big Data Network Characteristics
• Availability – The loss of a portion of the cluster will impact performance.
• Burst Handling – Queue depth and low over-subscription ratio are very important.
• Latency – There is delay in the processing on the data node, so this is not an issue.
• Jitter (Variation in Delay) – Some workloads are highly synchronous, so deterministic latency is important.
• Scale – The environment should easily scale and retract to fit requirements.
• Security & Multi-Tenancy – A single environment for multiple logical work loads is more efficient.
• Performant – Server 10GE connectivity is common-place and likely to increase to 25 GE, 40 GE and 50 GE.

4. Big Data On Traditional Networks
• Availability – Resilient but slow to converge and prone to
Layer 2 problems (Loops and Broadcast Storms).
• Burst Handling – Queue depths are good but over-
subscription ratios can be high.
• Latency/Jitter – Over-subscription can lead with jitter.
• Scale – Difficult to scale without manual configuration
• Security – Firewall is a bottleneck and DMZs are difficult to
design.
• Performant – High speeds available but these are not the
most efficient or cost effective.
Core Layer
Distribution Layer
Access Layer
Big Data Cluster Nodes
1 Gbps1 Gbps
10 Gbps
1 Gbps1 Gbps
Firewall Layer

5. Big Data On Network Fabrics
• Availability – ECMP with sub-second failover and
flow-lets.
• Burst Handling – Low over-subscription ratios and
optimised use of bandwidth.
• Latency/Jitter – Predictable latency throughout.
• Scale – Very easy to scale, add more leafs and deploy
via controller.
• Security – Secure micro-segmentation, DMZs
everywhere.
• Performant – Multiple 40/100 GE ports in fabric with
1/10 GE to servers.
Big Data Cluster Nodes
1/10 Gbps
40 Gbps
Ethernet Fabric

6. What Is Networking Your Big Data?
Whether it’s on premise in your own data centre, co-location or cloud, check that your
network is…
• An Network Fabric
• Supports ECMP (Equal Cost Multi-Pathing)
• Provide 40 GE (future 100 GE Support), along with 10GE edge support
• Is built on Software Defined Network
• Uses Network Functions Virtualisation for security
• Implements secure micro-segmentation

7. What About Big Data for Networking?
• SIEM
• Infrastructure Analytics
• Monitoring & Troubleshooting
• Intent Based Automation

8. Securely Networking Big Data
Environments
Stephen Hampton
CTO, Hutchinson Networks