This document discusses how CloudFlare uses Salt and NAPALM for network automation and configuration management across their global network of over 80 points of presence. Some key points:
- CloudFlare routes web traffic through their global network of over 80 locations serving over 4 million domains and handling over 43 billion DNS queries per day.
- They use Salt and NAPALM to automate the deployment, configuration, monitoring and maintenance of their large network, including replacing equipment and deploying new points of presence.
- NAPALM integrates directly with Salt and provides vendor-agnostic modules to control network devices, retrieve information and enforce configurations across different device types and vendors.
Network Automation with Salt and NAPALM: a self-resilient networkCloudflare
This document discusses using Salt and NAPALM for network automation. Salt is used as the automation framework due to its scalability, concurrency, configurability and other features. NAPALM is used to provide vendor-agnostic network drivers and modules. Together, Salt and NAPALM allow for automating tasks like deploying new network sites, monitoring links and devices, maintaining consistent configurations, and improving recovery times from outages or equipment replacements. Examples shown include using Salt to schedule regular configuration checks, deploy probes to monitor transit providers, retrieve probe results, and set up alerts. Contributions to the open source Salt and NAPALM projects are encouraged to advance the goal of self-resilient
Network Automation with Salt and NAPALM: a self-resilient networkAPNIC
This document discusses using Salt and NAPALM for network automation. Salt is used as an orchestrator to manage network devices at scale through NAPALM, which provides vendor-agnostic APIs and drivers. Key points include:
- Salt and NAPALM allow Cloudflare to automate tasks like deploying new network locations, reducing human errors and speeding recovery.
- NAPALM integrates with Salt to provide vendor-agnostic methods for tasks like configuration management, CLI execution, and collecting operational data from devices.
- Together Salt and NAPALM allow Cloudflare to manage thousands of devices through a single framework, deploy configurations consistently, and monitor network performance through
1. The document discusses OpenStack networking-sfc and flow analysis. It provides details on setting up an OpenStack environment with networking-sfc, including creating ports, virtual networks, and VMs for a service function chaining scenario. 2. Flow analysis is shown for the br-int and br-tun bridges, including resubmitting packets between tables based on port numbers or MAC address. 3. Key steps shown include installing networking-sfc, creating a virtual router, generating ports for each VM, and booting VMs with dual interfaces for the service function VMs.
This document provides instructions for setting up a single server SDN testbed environment using Open vSwitch. It describes installing Ubuntu, configuring networking, installing necessary programs like Open vSwitch and DevStack, and configuring Open vSwitch bridges, tunnels, and virtual machines to emulate an SDN network on a single physical server.
HaProxy is a free, very fast, and highly available load balancer and proxy. It can balance loads and act as a proxy for TCP and HTTP(s) traffic across multiple systems. The document discusses HaProxy's capabilities and configurations. Configurations include global settings, defaults, backends, frontends, and listens. Examples are provided for load balancing HTTP, HTTPS, MySQL, FTP, and RDP traffic. Sticky sessions and DDoS protection configurations are also covered. The document concludes with a summary of HaProxy's performance, configurability, documentation, multi-system support, and statistics/monitoring tools.
Debugging Distributed Systems - Velocity Santa Clara 2016Donny Nadolny
Despite our best efforts, our systems fail. Sometimes it’s our fault—code that we wrote, bugs that we caused. But sometimes the fault is with systems that we have no direct control over. Distributed systems are hard. They are complicated, hard to understand, and very challenging to manage. But they are critical to modern software, and when they have problems, we need to fix them.
ZooKeeper is a very useful distributed system that is often used as a building block for other distributed systems like Kafka and Spark. It is used by PagerDuty for many critical systems, and for five months it failed a lot. Donny Nadolny looks at what it takes to debug a problem in a distributed system like ZooKeeper, walking attendees through the process of finding and fixing one cause of many of these failures. Donny explains how to use various tools to stress test the network, some intricate details of how ZooKeeper works, and possibly more than you will want to know about TCP, including an example of machines having a different view of the state of a TCP stream.
http://conferences.oreilly.com/velocity/devops-web-performance-ca/public/schedule/detail/50058
Ipv6 test plan for opnfv poc v2.2 spirent-vctlabIben Rodriguez
This document outlines test plans and requirements for testing IPv6 in an OPNFV PoC v2.0 environment using OpenStack Liberty and ODL Lithium SR2. It details:
(1) Setting up an IPv6 service VM in OpenStack with ODL controller capability for IPv6 routing and address advertisement.
(2) A test design and steps for setting up infrastructure, ODL and OpenStack controllers, and compute nodes.
(3) Positive test cases to validate IPv6 and IPv4 connectivity between VMs, routers and external DNS via ping, traceroute from the VM and service VM.
(4) References for IPv6 configuration and testing in Linux.
How deep is your buffer – Demystifying buffers and application performanceCumulus Networks
Packet buffer memory is among the oldest topics in networking, and yet it never seems to fade in popularity. Starting from the days of buffers sized by the bandwidth delay product to what is now called "buffer bloat", from the days of 10Mbps to 100Gbps, the discussion around how deep should the buffers be never ceases to evoke opinionated responses.
In this webinar we will be joined by JR Rivers, co-founder and CTO of Cumulus Networks, a man who has designed many ultra-successful switching chips, switch products, and compute platforms, to discuss the innards of buffering. This webinar will cover data path theory, tools to evaluate network data path behavior, and the configuration variations that affect application visible outcomes.
Network Automation with Salt and NAPALM: a self-resilient networkCloudflare
This document discusses using Salt and NAPALM for network automation. Salt is used as the automation framework due to its scalability, concurrency, configurability and other features. NAPALM is used to provide vendor-agnostic network drivers and modules. Together, Salt and NAPALM allow for automating tasks like deploying new network sites, monitoring links and devices, maintaining consistent configurations, and improving recovery times from outages or equipment replacements. Examples shown include using Salt to schedule regular configuration checks, deploy probes to monitor transit providers, retrieve probe results, and set up alerts. Contributions to the open source Salt and NAPALM projects are encouraged to advance the goal of self-resilient
Network Automation with Salt and NAPALM: a self-resilient networkAPNIC
This document discusses using Salt and NAPALM for network automation. Salt is used as an orchestrator to manage network devices at scale through NAPALM, which provides vendor-agnostic APIs and drivers. Key points include:
- Salt and NAPALM allow Cloudflare to automate tasks like deploying new network locations, reducing human errors and speeding recovery.
- NAPALM integrates with Salt to provide vendor-agnostic methods for tasks like configuration management, CLI execution, and collecting operational data from devices.
- Together Salt and NAPALM allow Cloudflare to manage thousands of devices through a single framework, deploy configurations consistently, and monitor network performance through
1. The document discusses OpenStack networking-sfc and flow analysis. It provides details on setting up an OpenStack environment with networking-sfc, including creating ports, virtual networks, and VMs for a service function chaining scenario. 2. Flow analysis is shown for the br-int and br-tun bridges, including resubmitting packets between tables based on port numbers or MAC address. 3. Key steps shown include installing networking-sfc, creating a virtual router, generating ports for each VM, and booting VMs with dual interfaces for the service function VMs.
This document provides instructions for setting up a single server SDN testbed environment using Open vSwitch. It describes installing Ubuntu, configuring networking, installing necessary programs like Open vSwitch and DevStack, and configuring Open vSwitch bridges, tunnels, and virtual machines to emulate an SDN network on a single physical server.
HaProxy is a free, very fast, and highly available load balancer and proxy. It can balance loads and act as a proxy for TCP and HTTP(s) traffic across multiple systems. The document discusses HaProxy's capabilities and configurations. Configurations include global settings, defaults, backends, frontends, and listens. Examples are provided for load balancing HTTP, HTTPS, MySQL, FTP, and RDP traffic. Sticky sessions and DDoS protection configurations are also covered. The document concludes with a summary of HaProxy's performance, configurability, documentation, multi-system support, and statistics/monitoring tools.
Debugging Distributed Systems - Velocity Santa Clara 2016Donny Nadolny
Despite our best efforts, our systems fail. Sometimes it’s our fault—code that we wrote, bugs that we caused. But sometimes the fault is with systems that we have no direct control over. Distributed systems are hard. They are complicated, hard to understand, and very challenging to manage. But they are critical to modern software, and when they have problems, we need to fix them.
ZooKeeper is a very useful distributed system that is often used as a building block for other distributed systems like Kafka and Spark. It is used by PagerDuty for many critical systems, and for five months it failed a lot. Donny Nadolny looks at what it takes to debug a problem in a distributed system like ZooKeeper, walking attendees through the process of finding and fixing one cause of many of these failures. Donny explains how to use various tools to stress test the network, some intricate details of how ZooKeeper works, and possibly more than you will want to know about TCP, including an example of machines having a different view of the state of a TCP stream.
http://conferences.oreilly.com/velocity/devops-web-performance-ca/public/schedule/detail/50058
Ipv6 test plan for opnfv poc v2.2 spirent-vctlabIben Rodriguez
This document outlines test plans and requirements for testing IPv6 in an OPNFV PoC v2.0 environment using OpenStack Liberty and ODL Lithium SR2. It details:
(1) Setting up an IPv6 service VM in OpenStack with ODL controller capability for IPv6 routing and address advertisement.
(2) A test design and steps for setting up infrastructure, ODL and OpenStack controllers, and compute nodes.
(3) Positive test cases to validate IPv6 and IPv4 connectivity between VMs, routers and external DNS via ping, traceroute from the VM and service VM.
(4) References for IPv6 configuration and testing in Linux.
How deep is your buffer – Demystifying buffers and application performanceCumulus Networks
Packet buffer memory is among the oldest topics in networking, and yet it never seems to fade in popularity. Starting from the days of buffers sized by the bandwidth delay product to what is now called "buffer bloat", from the days of 10Mbps to 100Gbps, the discussion around how deep should the buffers be never ceases to evoke opinionated responses.
In this webinar we will be joined by JR Rivers, co-founder and CTO of Cumulus Networks, a man who has designed many ultra-successful switching chips, switch products, and compute platforms, to discuss the innards of buffering. This webinar will cover data path theory, tools to evaluate network data path behavior, and the configuration variations that affect application visible outcomes.
Extensible Messaging and Presence Protocol (XMPP)Sean Tsai
The document provides information about the Extensible Messaging and Presence Protocol (XMPP):
- XMPP is an open-source communication protocol for message-oriented middleware based on XML. It allows for near-real-time exchange of structured yet extensible data between any two or more network entities.
- XMPP supports instant messaging, presence, multi-party chat, voice and video calls, collaboration, lightweight middleware, content syndication, and generalized routing of XML data.
- The document discusses XMPP architecture, standards, security mechanisms like TLS and SASL, and the basic client-server communication flow when establishing an XMPP connection.
This document discusses Flowspec, a mechanism for filtering traffic flows using BGP. It can be used to easily rate limit or discard traffic based on attributes like source/destination addresses and ports. The document provides sample configuration examples and notes some limitations like lack of SNMP support for counters. It also shows graphs of attacks detected and mitigated using Flowspec rules.
OpenStack DVR (Distributed Virtual Router) allows L3 routing functions to be distributed across compute nodes by creating router namespaces on each compute node. This avoids bottlenecks and single points of failure at network nodes. DVR supports east-west inter-subnet routing, SNAT for external access without floating IPs, and floating IPs associated with internal VMs for direct external access. Traffic flows are encapsulated in VXLAN/GRE tunnels between compute nodes and routed appropriately within each node's router namespace.
(SDD419) Amazon EC2 Networking Deep Dive and Best Practices | AWS re:Invent 2014Amazon Web Services
Amazon EC2 instances give customers a variety of high-bandwidth networking choices. In this session, we discuss how to choose among Amazon EC2 networking technologies and examine how to get the best performance out of Amazon EC2 enhanced networking and cluster networking. We also share best practices and useful tips for success.
This document provides an overview of ONOS (Open Network Operating System) including:
- What ONOS is and its architectural tenets of high availability, scalability, and modularity
- ONOS's distributed architecture with core subsystems and components running on multiple nodes
- The SDN-IP application which allows ONOS to communicate with external IP networks
- Guidelines for deploying SDN-IP including physical setup and basic workflow
- Using SDN-IP and ONOS for an SDX use case including route validation with RPKI
- A tutorial demonstrating setting up an SDN-IP environment in Mininet and ONOS
(NET301) New Capabilities for Amazon Virtual Private CloudAmazon Web Services
Amazon's Virtual Private Cloud (Amazon VPC) continues to evolve with new capabilities and enhancements. These features give you increasingly greater isolation, control, and visibility at the all-important networking layer. In this session, we review some of the latest changes, discuss their value, and describe their use cases.
this slide is created for understand open vswitch more easily.
so I tried to make it practical. if you just follow up this scenario, then you will get some knowledge about OVS.
In this document, I mainly use only two command "ip" and "ovs-vsctl" to show you the ability of these commands.
Opensample: A Low-latency, Sampling-based Measurement Platform for Software D...Junho Suh
In this paper we propose, implement and evaluate OpenSample: a low-latency, sampling-based network measure- ment platform targeted at building faster control loops for software-defined networks. OpenSample leverages sFlow packet sampling to provide near–real-time measurements of both net- work load and individual flows. While OpenSample is useful in any context, it is particularly useful in an SDN environment where a network controller can quickly take action based on the data it provides. Using sampling for network monitoring allows OpenSample to have a 100 millisecond control loop rather than the 1–5 second control loop of prior polling-based approaches. We implement OpenSample in the Floodlight OpenFlow controller and evaluate it both in simulation and on a testbed comprised of commodity switches. When used to inform traffic engineering, OpenSample provides up to a 150% throughput improvement over both static equal-cost multi-path routing and a polling-based solution with a one second control loop.
The document discusses using Ixia's IxVM virtual machine for network testing in Cisco's Modeling Lab (CML) virtual environment. Specifically:
- IxVM is a virtual machine version of Ixia's network testing hardware that runs on Linux. Its OVA file can be added to CML.
- The OVA file contains disk files that are extracted and registered with CML's OpenStack environment to launch the IxVM VM.
- Example shows configuring a router and IxVM VMs in CML, and using IxNetwork software to generate OSPF traffic between the VMs and router ports for testing.
2015.7.17 JANOG36 BGP Flowspec Interoperability Test @ Interop Tokyo 2015 Sho...Shuichi Ohkubo
The document summarizes an interoperability test of BGP Flowspec functionality across Cisco, Huawei, and Juniper routers at Interop Tokyo 2015. The test confirmed basic BGP Flowspec action rules like drop, rate-limiting, and VRF redirection worked across all vendors. Some differences in NLRI formats and match bits were found for TCP flag and fragment match types. The document also provides an example use case of filtering SSH brute-force attacks and discusses additional configuration needed for Juniper routers.
This document discusses how to configure a multi-homed router to connect to multiple internet service providers (ISPs) simultaneously. It describes using multiple routing tables associated with each network interface to direct traffic. Rules are used to classify traffic and mark packets so they are routed to the appropriate table. Network address translation (NAT) is configured to map internal IP addresses to external IP addresses for each ISP connection. Additional documentation and tools are provided for monitoring link status and load balancing across connections.
HAProxy is a free, open source load balancer and proxy server that provides high availability, load balancing, and proxying for TCP and HTTP-based applications. It can be used to improve fault tolerance, distribute load, and optimize resource usage by terminating TCP connections and proxying requests to multiple backend servers. The document provides information on installing HAProxy, configuring the HAProxy configuration file to define frontend and backend settings, and log files for monitoring load balancing activity and troubleshooting issues.
[2018.10.19] Andrew Kong - Tunnel without tunnel (Seminar at OpenStack Korea ...OpenStack Korea Community
The document discusses network architectures in OpenStack. It provides diagrams to illustrate the networking components including compute nodes, virtual machines, linux bridges, agents, and routers. MPLS is introduced as a solution to address issues with tenant network separation and performance challenges with other approaches like VxLAN. MPLS uses label switching to encapsulate and forward packets instead of relying on IP routing and overlays, improving east-west traffic performance between tenants.
This document provides an overview of advanced load balancing capabilities in Apache HTTP Server 2.2 using the mod_proxy module. Key points include:
- Mod_proxy allows Apache to function as a reverse proxy or load balancer for backend servers.
- New in 2.2 are improvements like large file support, graceful stop, mod_dbd integration, and better debugging.
- Load balancing is implemented through balancer providers that can be customized. Default providers balance by requests, traffic, or server busyness.
- Features like connection pooling, sticky sessions, failover clusters, and an embedded admin interface provide robust load balancing functionality.
This presentation, DEFEATING THE NETWORK SECURITY INFRASTRUCTURE v1.0.pdf, was made after some brainstorming
with some friends. The techniques used are not new and the tools readily available for download. The purpose of the discussion however
is to debate how internal enterprise resources might be (in)adversely exposed to the internet by in an insider using a combination of common techniques such as SSH and SSL.
PuppetConf 2016: Why Network Automation Matters, and What You Can Do About It...Puppet
Here are the slides from Rick Sherman's PuppetConf 2016 presentation called Why Network Automation Matters, and What You Can Do About It. Watch the videos at https://www.youtube.com/playlist?list=PLV86BgbREluVjwwt-9UL8u2Uy8xnzpIqa
This document discusses tools for automating network interconnection and capacity planning decisions. It describes the complexity of manual processes and how automation can help by providing consistency, speed, ease of support, and compliance. NETCONF is presented as a standard for automating device configuration. Automating common tasks like adding ports and BGP sessions can help provision capacity based on traffic data. Total pluggability of providers is suggested to further streamline automation.
The Rules of Network Automation - Interop/NYC 2014Jeremy Schulman
The document discusses network automation, noting that while network operations are currently very painful, automation can provide business benefits like velocity, agility, stability and lower costs. It evaluates options for automation, from vendor products to building from scratch, and advocates learning from other fields like DevOps that have successfully adopted automation. The document concludes by urging readers to start planning their network automation initiatives now while keeping in mind that culture change and seeing results will take time.
Extensible Messaging and Presence Protocol (XMPP)Sean Tsai
The document provides information about the Extensible Messaging and Presence Protocol (XMPP):
- XMPP is an open-source communication protocol for message-oriented middleware based on XML. It allows for near-real-time exchange of structured yet extensible data between any two or more network entities.
- XMPP supports instant messaging, presence, multi-party chat, voice and video calls, collaboration, lightweight middleware, content syndication, and generalized routing of XML data.
- The document discusses XMPP architecture, standards, security mechanisms like TLS and SASL, and the basic client-server communication flow when establishing an XMPP connection.
This document discusses Flowspec, a mechanism for filtering traffic flows using BGP. It can be used to easily rate limit or discard traffic based on attributes like source/destination addresses and ports. The document provides sample configuration examples and notes some limitations like lack of SNMP support for counters. It also shows graphs of attacks detected and mitigated using Flowspec rules.
OpenStack DVR (Distributed Virtual Router) allows L3 routing functions to be distributed across compute nodes by creating router namespaces on each compute node. This avoids bottlenecks and single points of failure at network nodes. DVR supports east-west inter-subnet routing, SNAT for external access without floating IPs, and floating IPs associated with internal VMs for direct external access. Traffic flows are encapsulated in VXLAN/GRE tunnels between compute nodes and routed appropriately within each node's router namespace.
(SDD419) Amazon EC2 Networking Deep Dive and Best Practices | AWS re:Invent 2014Amazon Web Services
Amazon EC2 instances give customers a variety of high-bandwidth networking choices. In this session, we discuss how to choose among Amazon EC2 networking technologies and examine how to get the best performance out of Amazon EC2 enhanced networking and cluster networking. We also share best practices and useful tips for success.
This document provides an overview of ONOS (Open Network Operating System) including:
- What ONOS is and its architectural tenets of high availability, scalability, and modularity
- ONOS's distributed architecture with core subsystems and components running on multiple nodes
- The SDN-IP application which allows ONOS to communicate with external IP networks
- Guidelines for deploying SDN-IP including physical setup and basic workflow
- Using SDN-IP and ONOS for an SDX use case including route validation with RPKI
- A tutorial demonstrating setting up an SDN-IP environment in Mininet and ONOS
(NET301) New Capabilities for Amazon Virtual Private CloudAmazon Web Services
Amazon's Virtual Private Cloud (Amazon VPC) continues to evolve with new capabilities and enhancements. These features give you increasingly greater isolation, control, and visibility at the all-important networking layer. In this session, we review some of the latest changes, discuss their value, and describe their use cases.
this slide is created for understand open vswitch more easily.
so I tried to make it practical. if you just follow up this scenario, then you will get some knowledge about OVS.
In this document, I mainly use only two command "ip" and "ovs-vsctl" to show you the ability of these commands.
Opensample: A Low-latency, Sampling-based Measurement Platform for Software D...Junho Suh
In this paper we propose, implement and evaluate OpenSample: a low-latency, sampling-based network measure- ment platform targeted at building faster control loops for software-defined networks. OpenSample leverages sFlow packet sampling to provide near–real-time measurements of both net- work load and individual flows. While OpenSample is useful in any context, it is particularly useful in an SDN environment where a network controller can quickly take action based on the data it provides. Using sampling for network monitoring allows OpenSample to have a 100 millisecond control loop rather than the 1–5 second control loop of prior polling-based approaches. We implement OpenSample in the Floodlight OpenFlow controller and evaluate it both in simulation and on a testbed comprised of commodity switches. When used to inform traffic engineering, OpenSample provides up to a 150% throughput improvement over both static equal-cost multi-path routing and a polling-based solution with a one second control loop.
The document discusses using Ixia's IxVM virtual machine for network testing in Cisco's Modeling Lab (CML) virtual environment. Specifically:
- IxVM is a virtual machine version of Ixia's network testing hardware that runs on Linux. Its OVA file can be added to CML.
- The OVA file contains disk files that are extracted and registered with CML's OpenStack environment to launch the IxVM VM.
- Example shows configuring a router and IxVM VMs in CML, and using IxNetwork software to generate OSPF traffic between the VMs and router ports for testing.
2015.7.17 JANOG36 BGP Flowspec Interoperability Test @ Interop Tokyo 2015 Sho...Shuichi Ohkubo
The document summarizes an interoperability test of BGP Flowspec functionality across Cisco, Huawei, and Juniper routers at Interop Tokyo 2015. The test confirmed basic BGP Flowspec action rules like drop, rate-limiting, and VRF redirection worked across all vendors. Some differences in NLRI formats and match bits were found for TCP flag and fragment match types. The document also provides an example use case of filtering SSH brute-force attacks and discusses additional configuration needed for Juniper routers.
This document discusses how to configure a multi-homed router to connect to multiple internet service providers (ISPs) simultaneously. It describes using multiple routing tables associated with each network interface to direct traffic. Rules are used to classify traffic and mark packets so they are routed to the appropriate table. Network address translation (NAT) is configured to map internal IP addresses to external IP addresses for each ISP connection. Additional documentation and tools are provided for monitoring link status and load balancing across connections.
HAProxy is a free, open source load balancer and proxy server that provides high availability, load balancing, and proxying for TCP and HTTP-based applications. It can be used to improve fault tolerance, distribute load, and optimize resource usage by terminating TCP connections and proxying requests to multiple backend servers. The document provides information on installing HAProxy, configuring the HAProxy configuration file to define frontend and backend settings, and log files for monitoring load balancing activity and troubleshooting issues.
[2018.10.19] Andrew Kong - Tunnel without tunnel (Seminar at OpenStack Korea ...OpenStack Korea Community
The document discusses network architectures in OpenStack. It provides diagrams to illustrate the networking components including compute nodes, virtual machines, linux bridges, agents, and routers. MPLS is introduced as a solution to address issues with tenant network separation and performance challenges with other approaches like VxLAN. MPLS uses label switching to encapsulate and forward packets instead of relying on IP routing and overlays, improving east-west traffic performance between tenants.
This document provides an overview of advanced load balancing capabilities in Apache HTTP Server 2.2 using the mod_proxy module. Key points include:
- Mod_proxy allows Apache to function as a reverse proxy or load balancer for backend servers.
- New in 2.2 are improvements like large file support, graceful stop, mod_dbd integration, and better debugging.
- Load balancing is implemented through balancer providers that can be customized. Default providers balance by requests, traffic, or server busyness.
- Features like connection pooling, sticky sessions, failover clusters, and an embedded admin interface provide robust load balancing functionality.
This presentation, DEFEATING THE NETWORK SECURITY INFRASTRUCTURE v1.0.pdf, was made after some brainstorming
with some friends. The techniques used are not new and the tools readily available for download. The purpose of the discussion however
is to debate how internal enterprise resources might be (in)adversely exposed to the internet by in an insider using a combination of common techniques such as SSH and SSL.
PuppetConf 2016: Why Network Automation Matters, and What You Can Do About It...Puppet
Here are the slides from Rick Sherman's PuppetConf 2016 presentation called Why Network Automation Matters, and What You Can Do About It. Watch the videos at https://www.youtube.com/playlist?list=PLV86BgbREluVjwwt-9UL8u2Uy8xnzpIqa
This document discusses tools for automating network interconnection and capacity planning decisions. It describes the complexity of manual processes and how automation can help by providing consistency, speed, ease of support, and compliance. NETCONF is presented as a standard for automating device configuration. Automating common tasks like adding ports and BGP sessions can help provision capacity based on traffic data. Total pluggability of providers is suggested to further streamline automation.
The Rules of Network Automation - Interop/NYC 2014Jeremy Schulman
The document discusses network automation, noting that while network operations are currently very painful, automation can provide business benefits like velocity, agility, stability and lower costs. It evaluates options for automation, from vendor products to building from scratch, and advocates learning from other fields like DevOps that have successfully adopted automation. The document concludes by urging readers to start planning their network automation initiatives now while keeping in mind that culture change and seeing results will take time.
Anas Tarsha presented on using Ansible for network automation. Ansible is an open source automation tool that is agentless and uses simple YAML files called playbooks to execute tasks sequentially. It can be used to generate device configurations, push configurations, collect running configs, upgrade devices, and more. Ansible modules run Python code directly on network devices to perform tasks. The demo showed using Ansible modules like ping, ios_command, and junos_command to execute show commands and change the hostname on both IOS and Junos devices. Additional resources were provided to learn more about using Ansible for network automation.
The document provides tips and tricks for network automation. It discusses:
- Why network automation is important, even for small networks, including consistency, scalability, fast iteration, and enjoyment.
- Simplifying network configuration before automation by exploiting regularity, automating patterns, using "cookie cutter" configurations, and minimizing duplication.
- Tips for network automation including using simple consistent toolchains like Ansible, employing unnumbered interfaces to reduce complexity, using interface names instead of IP addresses, and configuring routing protocols under interfaces instead of using network statements.
- Additional tips are using the device hostname to derive unique IDs, loopback IPs, ASNs, and other attributes to simplify automation.
The document discusses Shapeways' use of Puppet and Cumulus Linux for network automation. Some key points:
- Shapeways uses Puppet to define networking configurations with a common language, gain situational awareness of their network, and orchestrate changes intelligently.
- They chose Cumulus Linux as the network OS because it allowed them to leverage their Linux expertise and provided reliability benefits over traditional network OSes.
- Puppet enables them to build abstractions that simplify complex implementations, focusing on the controls that matter and reusing configurations across environments.
- Their process automates network topology generation and uses custom scripts to build switch configuration from this data. Now their network is fast, reliable, and
SolarWinds Scalability for the EnterpriseSolarWinds
Listen to the SolarWinds product management team as they show you how SolarWinds affordable, powerful, and easy-to-use solutions can scale your monitoring and management capabilities to hundreds of thousands of network devices, servers, or applications.
Murakumo is an open-source IaaS cloud controller and API orchestrator developed in 2012 to manage virtual machines, storage, and networks. It uses a thin controller and rich node agent architecture with asynchronous job queue processing. It supports Linux KVM and uses a simple design intended for easy operation and maintenance.
OpenSource API Server based on Node.js API framework built on supported Node.js platform with Tooling and DevOps. Use cases are Omni-channel API Server, Mobile Backend as a Service (mBaaS) or Next Generation Enterprise Service Bus. Key functionality include built in enterprise connectors, ORM, Offline Sync, Mobile and JS SDKs, Isomorphic JavaScript and Graphical API creation tool.
The Real World - Plugging the Enterprise Into It (nodejs)Aman Kohli
This document discusses using Node.js as the foundation for building applications that connect the physical world to enterprise systems through mobile devices and sensors. It describes initial work done to build a proxy and protocol for handling requests and addresses challenges with authentication, scalability, and performance testing. The document shares results from benchmarking the system under different network conditions and outlines next steps to improve concurrency, security, and infrastructure elasticity.
Node is used to build a reverse proxy to provide secure access to internal web resources and sites for mobile clients within a large enterprise. Performance testing shows the proxy can handle over 1000 requests per second with latency under 1 second. Code quality analysis tools like Plato and testing frameworks like Jest are useful for maintaining high quality code. Scalability is achieved through auto-scaling virtual machine instances with a load balancer and configuration management.
Apache Pulsar with MQTT for Edge Computing - Pulsar Summit Asia 2021StreamNative
This document discusses using Apache Pulsar with MQTT for edge computing. It provides an overview of Apache Pulsar and how it enables message queuing and data streaming with features like pub-sub, geo-replication, and multi-protocol support including MQTT. It also discusses edge computing characteristics and challenges, and how running Apache Pulsar on edge devices can address these by extending data processing to the edge and integrating with sensors using the MQTT protocol. Examples are provided of ingesting IoT data into Pulsar from Python and using NVIDIA Jetson devices with Pulsar.
Unified Stream Processing at Scale with Apache Samza - BDS2017Jacob Maes
The shift to stream processing at LinkedIn has accelerated over the past few years. We now have over 200 Samza applications in production processing more than 260B events per day. Many of these are new applications, but there have also been more migrations from existing online and offline applications. To support the influx of new use cases, we have improved the flexibility, efficiency and reliability of Apache Samza.
In this talk, we will take a brief look at the broader streaming ecosystem at LinkedIn, then we will zoom in on a few representative use cases and explain how they are powered by recent advancements to Apache Samza including a unified high level API, flexible deployment model, batch processing, and more.
Exploring the Final Frontier of Data Center Orchestration: Network Elements -...Puppet
The document discusses network element automation using Puppet. It provides context on the challenges of manual network configuration including lack of agility, reliability issues from errors, and time spent on basic tasks. Puppet can automate network elements similar to how it automates servers, reducing errors and improving speed/productivity. The Cisco Nexus platform and NXAPI enable programmatic access for automation using Puppet through technologies like onePK and LXC containers running on the switch.
NetBrain is network automation software that allows users to:
- Create network diagrams and maps automatically in seconds to visualize network topology and troubleshoot issues
- Continuously monitor networks and automatically detect changes across devices
- Automate complex network changes with a single click and automatically document changes
- Perform in-depth historical analysis and use automated diagnostics apps to identify root causes of issues quickly
What is a Service Mesh and what can it do for your MicroservicesMatt Turner
e’ll explore what a service mesh is and what it can do for your microservices. Are the claims of observability, resiliency, and WAF features real? Are they useful during development, production, or both? Using pictures and demos, we’ll find out!
This session will also briefly cover how a service mesh works, giving us a mental model with which to explore and evaluate after the talk. Matt will show a simple installation and demo, giving us all the knowledge to go home and try for ourself.
Strata Singapore: GearpumpReal time DAG-Processing with Akka at ScaleSean Zhong
Gearpump is a Akka based realtime streaming engine, it use Actor to model everything. It has super performance and flexibility. It has performance of 18000000 messages/second and latency of 8ms on a cluster of 4 machines.
Cassandra Tools and Distributed Administration (Jeffrey Berger, Knewton) | C*...DataStax
At Knewton we operate across five different VPCs a total of 29 clusters, each ranging from 3 nodes to 24 nodes. For a team of three to maintain this is not herculean, however good tools to diagnose issues and gather information in a distributed manner are vital to moving quickly and minimizing engineering time spent.
The database team at Knewton has been successfully using a combination of Ansible and custom open sourced tools to maintain and improve the Cassandra deployment at Knewton. I will be talking about several of these tools and giving examples of how we are using them. Specifically I will discuss the cassandra-tracing tool, which analyzes the contents of the system_traces keyspace, and the cassandra-stat tool, which gives real-time output of the operations of a cassandra cluster. Distributed administration with ad-hoc Ansible will also be covered and I will walk through examples of using these commands to identify and remediate clusterwide issues.
About the Speaker
Jeffrey Berger Lead Database Engineer, Knewton
Dr. Jeffrey Berger is currently the lead database engineer at Knewton, an education tech startup in NYC. He joined the tech scene in NYC in 2013 and spent two years working with MongoDB, becoming a certified MongoDB administrator and a MongoDB Master. He received his Cassandra Administrator certification at Cassandra Summit 2015. He holds a Ph.D. in Theoretical Physics from Penn State and spent several years working on high energy nuclear interactions.
Cotopaxi - IoT testing toolkit (Black Hat Asia 2019 Arsenal)Jakub Botwicz
Presentation about Cotopaxi toolkit from Black Hat Asia 2019 Arsenal session. Author: Jakub Botwicz
https://www.blackhat.com/asia-19/arsenal/schedule/index.html#cotopaxi-iot-protocols-security-testing-toolkit-14325
This document discusses zero downtime architectures. It defines zero downtime as services being available to end users at all times. It identifies sources of planned and unplanned downtime. It proposes concepts like independent application groups, redundant infrastructure within and between datacenters, and replicating data between datacenters to reduce downtime. It provides examples of implementing high availability for networks, applications, and databases. It also discusses development guidelines and monitoring to support zero downtime operations.
Hpe service virtualization 3.8 what's new chicago admJeffrey Nunn
Service Virtualization is an HPE branded solution that helps simulate and emulate the behavior of specific components in heterogeneous component-based applications such as API-driven apps, ERP apps, cloud-based apps, and web services/service-oriented architectures (SOA).
Value Proposition
Empowers developers and testers to easily automate, predict, accelerate and scale their application testing and delivery through virtualization and simulation of dependent components and services that are either off limits, unavailable, inaccessible, or with costly fees to access.
Smuggling Multi-Cloud Support into Cloud-native Applications using Elastic Co...Nane Kratzke
Elastic container platforms (like Kubernetes, Docker Swarm, Apache Mesos) fit very well with existing cloud-native application architecture approaches. So it is more than astonishing, that these already existing and open source available elastic platforms are not considered more consequently for multi-cloud approaches. Elastic container platforms provide inherent multi-cloud support that can be easily accessed. We present a solution proposal of a control process which is able to scale (and migrate as a side effect) elastic container platforms across different public and private cloud-service providers. This control loop can be used in an execution phase of self-adaptive auto-scaling MAPE loops (monitoring, analysis, planning, execution). Additionally, we present several lessons learned from our prototype implementation which might be of general interest for researchers and practitioners. For instance, to describe only the intended state of an elastic platform and let a single control process take care to reach this intended state is far less complex than to define plenty of specific and necessary multi-cloud aware workflows to deploy, migrate, terminate, scale up and scale down elastic platforms or applications.
Monitoring as Code: Getting to Monitoring-Driven Development - DEV314 - re:In...Amazon Web Services
“Infrastructure as Code” has changed not only how we think about configuring infrastructure, but about the infrastructure itself. AWS has been at the core of this movement, enabling your infrastructure teams to benefit from software engineering best practices such as CI/CD, automated testing, and repeatable deployments. Now that you have mastered the art of managing your infrastructure as code, it’s time to leverage these same lessons for monitoring and metrics. In this session, we dive into how you can leverage tooling such as AWS, Terraform, and Datadog to programmatically define your monitoring so that you that you can scale your organizational observability along with your infrastructure, and attain consistency from local development all the way through production.
Session sponsored by Datadog, Inc.
For the Docker users out there, Sematext's DevOps Evangelist, Stefan Thies, goes through a number of different Docker monitoring options, points out their pros and cons, and offers solutions for Docker monitoring. Webinar contains actionable content, diagrams and how-to steps.
A presentation on how applying Cloud Architecture Patterns using Docker Swarm as orchestrator is possible to create reliable, resilient and scalable FIWARE platforms.
Similar to Network Automation with Salt and NAPALM: Introuction (20)
Ready to Unlock the Power of Blockchain!Toptal Tech
Imagine a world where data flows freely, yet remains secure. A world where trust is built into the fabric of every transaction. This is the promise of blockchain, a revolutionary technology poised to reshape our digital landscape.
Toptal Tech is at the forefront of this innovation, connecting you with the brightest minds in blockchain development. Together, we can unlock the potential of this transformative technology, building a future of transparency, security, and endless possibilities.
HijackLoader Evolution: Interactive Process HollowingDonato Onofri
CrowdStrike researchers have identified a HijackLoader (aka IDAT Loader) sample that employs sophisticated evasion techniques to enhance the complexity of the threat. HijackLoader, an increasingly popular tool among adversaries for deploying additional payloads and tooling, continues to evolve as its developers experiment and enhance its capabilities.
In their analysis of a recent HijackLoader sample, CrowdStrike researchers discovered new techniques designed to increase the defense evasion capabilities of the loader. The malware developer used a standard process hollowing technique coupled with an additional trigger that was activated by the parent process writing to a pipe. This new approach, called "Interactive Process Hollowing", has the potential to make defense evasion stealthier.
Discover the benefits of outsourcing SEO to Indiadavidjhones387
"Discover the benefits of outsourcing SEO to India! From cost-effective services and expert professionals to round-the-clock work advantages, learn how your business can achieve digital success with Indian SEO solutions.
Network Automation with Salt and NAPALM: Introuction
1. Network Automation with Salt and
NAPALM (or how we control 100’s of PoPs around the world)
Mircea Ulinic
CloudFlare, London
RIPE 72 Copenhagen
May 2016
2. 2
CloudFlare (a quick background)
● Once a website is part of the CloudFlare community, its web traffic is routed
through our global network of 80+ locations
● How big?
○ Four+ million zones/domains
○ Authoritative for ~40% of Alexa top 1 million
○ 43+ billion DNS queries/day
■ Second only to Verisign
● 80+ anycast locations globally
○ 40 countries (and growing)
● Origin CA
3. Our big network challenges
● Deploy new PoPs
● Human error factor
● Replace equipment
● Monitor
3
6. Opinions
6
Ryan D Lane
Jens Rantil
“The learning curve for Salt is higher and the intro docs are rough, but in the long-term Salt’s docs are
much better than Ansible’s, because they’re way more complete (which is also why they’re much worse
as intro docs).”
“To me, Ansible was a great introduction to automated server configuration and deployment. Moving
forward, the scalability, speed and architecture of Salt has it going for it. For cloud deployments I find
the Salt architecture to be a better fit. I would not hesitate to use Salt in the future.”
7. Salt: the “unwanted child” of network automation
7
https://opennxos.cisco.com/public/getting-started
https://forums.juniper.net/t5/Automation-Programmability/Automation-with-Chef-Puppet-and-Ansible/ba-p/261773
8. Why?
● Old references
● No feature for net devices as of yesterday
● Not well informed
● Not suitable for tiny VM networks
8
9. Salt at CloudFlare: used for years
Many thousands of servers already using Salt
Same tool for both servers and net devices
9
10. 10
Salt
(what fits the best our needs)
Ansible
(most used in network automation)
● Long standing sessions
● 20 types of modules
● Customizable
● Many thousands of CloudFlare
servers
● Comes embedded with features and
tools
● Native config enforcement logic
● Real-time job
● Job scheduling
● Runner as a module
● REST API
● High Availability
● GPG encryption
● Pull from Git, SVN
● open/close session per module
● 1 type of module
● Customizable
● ?
● Need to install separate packages
(“roles”) that are not necessarily
dependent
● Real-time job (Tower: $$)
● Job Scheduling (Tower: $$)
● Runner as a class
● REST API (Ansible Tower: $$)
● HA (Tower > Enterprise edition: $$$$)
● Security (Tower: $$)
● Pull from Git, SVN (Tower, $$)
11. Salt module types (selection)
● Execution modules
● Grains
● States
● Runners
● Pillars
● Returners
11
31. Unique ASNs per geographic area
31
# salt-run bgp.asns_per_area
Canada : 96
Brazil : 167
Australia : 113
Peru : 4
USA : 410
Africa : 21
Asia : 362
Europe : 1004
North America : 421
South America : 183
Oceania : 162
Colombia : 5
Chile : 5
Argentina : 21
Execution time: 2.84680294991 s
#
Execution
module
Runner Pillars
Grains
State
36. How can you use it?
# apt-get install salt-master (install guide)
# pip install napalm
Examples:
https://github.com/napalm-automation/napalm-salt
36
37. How can you contribute?
● NAPALM Automation:
https://github.com/napalm-automation
● SaltStack
https://github.com/saltstack/salt
37