SM
Uploaded byShannon McFarland
PDF, PPTX1,148 views

Openstack Summit Vancouver 2018 - Multicloud Networking

This document discusses connecting private clouds located on-premises to public clouds using multicloud networking. It begins with an overview of hybrid vs multicloud networking and reasons for using multiple cloud providers. It then covers options for extending an on-premises private cloud to a public cloud like AWS, GCP, or Azure using their native VPN services. The document discusses adding more public cloud providers and the need for automation. It introduces dynamic multipoint VPN (DMVPN) as a way to enable dynamic multicloud connectivity and extend routing protocols between sites.

Embed presentation

Download as PDF, PPTX
Multicloud Networking – Connecting OpenStack Private Clouds to Public Clouds Shannon McFarland – CCIE #5245 Distinguished Consulting Engineer @eyepv6
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Agenda • Multicloud Networking Overview • Extending On-Prem Private Clouds to a Public Cloud • Adding More Public Cloud Providers to the Mix • DMVPN • Amazon Web Services • Google Cloud Platform • Microsoft Azure • Automation • Conclusion 2
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Disclaimer • There are a gazillion ways to accomplish the same thing for ALL of this • You can build multicloud connections using software, hardware, commercial and open source gadgets • You or someone you work with needs to know IPsec/IKE, BGP, OSPF, EIGRP and FHRP stuff • Dead Peer Detection • IPsec SA lifetimes • IPsec SA replay window-size • Perfect Forward Secrecy (PFS) • BGP timers, Local Preference, MED, inbound soft reset (check if cloud provider supports dynamic inbound soft reset) • BGP graceful restart - Note: Each cloud provider uses BGP graceful restart with default timers (120 sec) – My configs do not show that due to slide space but know that it is enabled on each on-prem router • IGP timers, tuning • FHRP (HSRP, GLBP, VRRP) timers, tracking 3 router bgp 65002 bgp log-neighbor-changes bgp graceful-restart restart-time 120 bgp graceful-restart stalepath-time 360 bgp graceful-restart
Multicloud Networking Overview
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Hybrid vs Multicloud Networking • Hybrid Cloud Networking = Network transport from on-premises (on-prem) to a single public cloud provider • Multicloud Networking = Network transport from on-prem to multiple public cloud providers and/or between multiple public cloud providers • The technologies used can be identical for every connection or they can be per-provider, per-region, per-project, etc.. • Common network transport ingredients for hybrid and multicloud: • Encryption (IPsec/IKEv2/IKEv2, SSL, PKI) • Routing (Static, BGP and with supported public cloud-hosted routers: OSPF, EIGRP) • Tunneling (IPsec tunnel mode, GRE, mGRE, MPLS, segment routing, etc..) • Common network endpoint options: • Native VPN (IPsec over Internet) using public cloud provider services that connect to on-prem router/firewall • Commercial/Open Source VPN platform hosted on the public cloud provider connecting to an on-prem router/firewall • Colocation/Direct Peering: Service from public cloud provider to on-prem via a 3rd party colo facility • Google Cloud Platform Dedicated Internconnect/Direct Peering/Carrier Peering: https://cloud.google.com/interconnect/ • Amazon Web Services Direct Connect/PrivateLink: https://aws.amazon.com/directconnect/ • Microsoft Azure ExpressRoute: https://azure.microsoft.com/en-us/services/expressroute/ 5
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Why Would You Use Multiple Cloud Providers? • Cloud provider high availability • M&A may dictate public cloud provider preference (for a time) • Regional cloud provider access • Feature disparity between providers, regions and/or services • Per-project service requirements 6
Extending On-Prem Private Cloud to a Public Cloud
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Options – IPsec-over-the-Internet or Dedicated Connections 8 VPC Network 10.138.0.0/20 IPsec/IKEv2 BGP/OSPF/EIGRP On-Prem Private Cloud Google Cloud VPN Google Cloud Router VPC Network 10.138.0.0/20 BGP/OSPF/EIGRP On-Prem Private Cloud Google Cloud VPN VPC Network 10.138.0.0/20 Google Cloud Router Cloud Partner Interconnect Colocation Facility IPsec VPN + Internet Colocation Commercial/Open Source & Native OpenStack VPNaaS Private Network 192.168.200.0/24 Private Network 192.168.200.0/24
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Multicloud Topologies With OpenStack 9 OpenStack VM Neutron Router + VPNaaS VPNaaS Based Multicloud Networking Data Center Infra. TOR(s) Internet Edge Infra. VPN/CoLo Virtual Router Based Multicloud Networking Data Center Infra. OpenStack TOR(s) Internet Edge Infra. VPN/CoLo VM Virtual Router Neutron Router OpenStack VM Neutron Router Hardware Based Multicloud Networking Data Center Infra. TOR(s) Internet Edge Infra. VPN/CoLo *Also, provider networks
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Public Cloud Provider - Native VPN Services • Google Cloud Platform (GCP): • VPN: https://cloud.google.com/compute/docs/vpn/overview • Dedicated Interconnect: https://cloud.google.com/interconnect/ • Amazon Web Services (AWS): • VPN: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpn-connections.html • Direct Connect: https://aws.amazon.com/directconnect/ • Microsoft Azure: • VPN: https://docs.microsoft.com/en-us/azure/vpn-gateway/ • ExpressRoute: https://azure.microsoft.com/en-us/services/expressroute/ • OpenStack public cloud goodness: https://www.openstack.org/passport The Big Three 10 Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Starting Simple Public Cloud Provider Native IPsec VPN Service 11 VPC Network 10.138.0.0/20 IPsec/IKEv2 BGP/OSPF/EIGRP eBGP<>IGP Redistribution On-Prem Private Cloud Google Cloud VPN Google Cloud Router BGP AS65000 BGP AS65003 Private Network 192.168.200.0/24Cisco ASR1000
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Add More On-Prem Stuff Public Cloud Provider Native IPsec VPN Service 12 VPC Network 10.138.0.0/20 BGP AS65000 BGP AS65003 Routes this side should see: 10.138.0.0/20 Private Network 192.168.100.0/24 BGP AS65002 On-Prem Tenant 1 On-Prem Tenant 2Routes this side should see: 192.168.100.0/24 192.168.200.0/24 Google Cloud VPN Google Cloud Router BGP/OSPF/EIGRP BGP/OSPF/EIGRP Private Network 192.168.200.0/24 Cisco ASR1000 Cisco ASR1000
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Physical/Virtual Public Cloud Provider Native IPsec VPN Service 13 VPC Network 10.138.0.0/20 Virtual Router Physical Firewall Google Cloud VPN Google Cloud Router CSR 1000v ASA Firewall Private Network 192.168.100.0/24 Private Network 192.168.200.0/24
Add More Public Cloud Providers to the Mix
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Stepping into Multicloud Networking Multiple Native IPsec VPN Services 15 Private Network 192.168.200.0/24 VPC Network 10.138.0.0/20 BGP/OSPF/EIGRP On-Prem Private Cloud Google Cloud VPN Google Cloud Router VPC Network 172.31.0.0/16 VPN Gateway VPC Router
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Stepping into Multicloud Networking Multiple Native IPsec VPN Services 16 VPC Network 10.138.0.0/20 BGP/OSPF/EIGRP On-Prem Private Cloud Google Cloud VPN Google Cloud Router VPC Network 172.31.0.0/16 VPN Gateway VPC Router As the number of these connections increase and/or change frequently... You can see where this is going Private Network 192.168.200.0/24
Site-to-Site + Manual Configuration per Site = Unpleasant Times 17
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Example – OpenStack VPNaaS 18 • Lots of sites and lots of variation in policies can lead to lots of human errors • Per-Cloud provider IKE/IPsec compatibility polices required • Automation helps but only with the configuration challenge [root@mc-os-q-aio-sm ~]# openstack vpn ike policy create ikepolicy [root@mc-os-q-aio-sm ~]# openstack vpn ipsec policy create ipsecpolicy [root@mc-os-q-aio-sm ~]# openstack vpn service create vpn > --router a6c58be0-7e32-4a14-b648-8b8178f8de8c [root@mc-os-q-aio-sm ~]# openstack vpn endpoint group create ep_subnet > --type subnet > --value 7fe62bea-49ee-42a0-8c6a-5ec982983e98 [root@mc-os-q-aio-sm ~]# openstack vpn endpoint group create ep_cidr > --type cidr > --value 10.0.1.0/24 [root@mc-os-q-aio-sm ~]# openstack vpn ipsec site connection create GCP-Conn --vpnservice vpn --ikepolicy ikepolicy --ipsecpolicy ipsecpolicy --peer-address 35.xx.xx.xx --peer-id 35.xx.xx.xx --psk demo-secret --local-endpoint-group ep_subnet --peer-endpoint-group ep_cidr Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Moving Away From Native VPN Services • Large Site-to-Site designs suck due to configuration complexity (even with Heat or other automation) • If on-prem routers/firewalls are behind NAT – Check for provider support of NAT-T • You need to extend your on-prem IGP (OSPF/EIGRP) into the public cloud • Operational consistency • You need SSL-based VPNs • You need MPLS VPN • QoS, specific network monitoring (IP SLA, NetFlow), Enterprise toolsets for configuration and monitoring What Conditions Cause a Change in Design? 19
DMVPN – Dynamic Multipoint VPN
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public DMVPN – Enable Dynamic Multicloud Networking 21 VNet Network 10.50.0.0/16 DMVPN BGP/OSPF/EIGRP On-Prem Private CloudVPC Network 172.31.0.0/16 Cisco CSR1000v Cisco CSR1000v DMVPN: https://www.cisco.com/c/en/us/products/security/ dynamic-multipoint-vpn-dmvpn/index.html Hub Spoke Spoke Cisco ASR1000 Private Network 192.168.200.0/24
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public DMVPN – Enable Dynamic Multicloud Networking 22 VNet Network 10.50.0.0/16 DMVPN FHRP On-Prem Private Cloud VPC Network 172.31.0.0/16 Cisco CSR1000v Cisco CSR1000v IGP Support: OSPF, EIGRP, iBGP QoS Policies IP SLA, NetFlow NAT-T (Transparency) MPLS etc... Hub Spoke Spoke Cisco ASR1000 Private Network 192.168.200.0/24
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public DMVPN (Dynamic Multipoint VPN) • DMVPN is a Cisco innovation for building GRE/mGRE + IPsec VPN connections in a dynamic and scalable manner • Cisco DMVPN • https://www.cisco.com/c/en/us/products/security/dynamic-multipoint-vpn-dmvpn/index.html • Cisco IWAN CVD • https://www.cisco.com/c/en/us/solutions/design-zone/networking-design-guides/branch- wan-edge.html • OpenNHRP: • https://sourceforge.net/projects/opennhrp/ • https://wiki.alpinelinux.org/wiki/Dynamic_Multipoint_VPN_(DMVPN) 23
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 24 Terminology and Features 192.168.102.0/24 Hub1 Spoke 1 Hub 2 Spoke 2 192.168.101.0/24 192.168.1.0/24 192.168.2.0/24 Tunnel: 10.0.0.101 Physical: 172.16.101.1 Tunnel: 10.0.0.1 Physical: 172.16.1.1 Tunnel: 10.0.0.2 Physical: 172.16.2.1 Overlay Addresses NBMA Address Core Network 192.168.128.0/17 On Demand Spoke Tunnels Tunnel Address Tunnel: 10.0.0.102 Physical: 172.16.102.1 GRE/IPsec Tunnels
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public DMVPN Components • Next Hop Resolution Protocol (NHRP) • Creates a distributed (NHRP) mapping database of all the spoke’s tunnel to real (public interface) addresses • Multipoint GRE Tunnel Interface (mGRE) • Single GRE interface to support multiple GRE/IPsec tunnels • Simplifies size and complexity of configuration • IPsec tunnel protection • Dynamically creates and applies encryption policies • Routing • Dynamic advertisement of branch networks; almost all routing protocols (EIGRP, RIP, OSPF, BGP, ODR) are supported 25 Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public DMVPN Implementation 26 . Hub and spoke (Phase 1) Spoke-to-spoke (Phase 2) Server Load Balancing Hierarchical (Phase 3) VRF-lite 2547oDMVPN Spoke-to-hub tunnels Spoke-to-spoke tunnels 2547oDMVPN tunnels
Amazon Web Services – Cisco CSR & DMVPN
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public AWS with Cisco CSR 1000v Support • Amazon Web Services Marketplace + Cisco CSR: • https://aws.amazon.com/marketplace/search/results?x=0&y=0&searchTerms=csr&page=1&ref_=nav_search_ box • Cisco CSR for AWS Deployment • DMVPN https://www.cisco.com/c/en/us/td/docs/solutions/Hybrid_Cloud/Intercloud/CSR/AWS/CSRAWS/CSRAWS_3. html • Deployment https://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/aws/b_csraws.html • Cisco Live Session for AWS with Cisco CSR: • https://www.ciscolive.com/global/on-demand-library/?search=brkarc- 2023#/session/1486155288098001AhER • Transit VPC with CSR: http://d2zmdbbm9feqrf.cloudfront.net/2017/usa/pdf/BRKARC-2749.pdf 28
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public AWS CSR to On-Prem ASR – DMVPN 29 VPC Network 172.16.2.0/24 VPC Router 192.xxx.xxx.x52.xxx.xxx.x Routes this side should see: 192.168.200.0/24 Routes this side should see: 172.16.2.0/16 Public-side Network 172.16.1.0/24 Cisco CSR1000v DMVPN Hub Tunnel: 10.1.0.2 Spoke Tunnel: 10.1.0.4 OSPF Private Network 192.168.200.0/24 OSPF 10 Area 0 OpenStack VM .30 DataCenter Infra. Provider Networks with VLANs Example Cisco ASR1000
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 30 AWS CLI: Create VPC, Subnets and Internet GW Create a new AWS VPC (vpc) # aws ec2 create-vpc --cidr-block 172.16.0.0/16 Create a new subnet in the VPC (this one will be used for the CSR’s ’outside’ interface) # aws ec2 create-subnet --vpc-id vpc-66a0a102 --cidr-block 172.16.1.0/24 Create another new subnet in the VPC (this one will be used for the CSR’s ‘inside’ interface) # aws ec2 create-subnet --vpc-id vpc-66a0a102 --cidr-block 172.16.2.0/24 Create a new AWS Internet Gateway (igw) # aws ec2 create-internet-gateway Attach the Internet gateway to the VPC # aws ec2 attach-internet-gateway --vpc-id vpc-66a0a102 --internet-gateway-id igw-591fba3d Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 31 AWS CLI: Create Route Tables Create a new route table in the VPC (rtb) that will be used by the CSR’s ‘outside’ subnet # aws ec2 create-route-table --vpc-id vpc-66a0a102 Create a new default route in the route table and point it to the Internet gateway # aws ec2 create-route --route-table-id rtb-aaa37dcd --destination-cidr-block 0.0.0.0/0 --gateway-id igw-591fba3d Associate the new routable with the ‘outside’ VPC subnet # aws ec2 associate-route-table --subnet-id subnet-0c15b86b --route-table-id rtb-aaa37dcd Create a new route table in the VPC (rtb) that will be used by the CSR’s ‘inside’ subnet # aws ec2 create-route-table --vpc-id vpc-66a0a102 Create a new default route in the route table for the ‘inside’ subnet and point it to the Internet gateway # aws ec2 create-route --route-table-id rtb-3741e750 --destination-cidr-block 0.0.0.0/0 --gateway-id igw-591fba3d Create a new default route in the route table and point it to the Internet gateway # aws ec2 create-route --route-table-id rtb-3741e750 --destination-cidr-block 192.168.200.0/24 --network-interface-id eni-af67db80 Associate the new route table with the ‘inside’ VPC subnet # aws ec2 associate-route-table --subnet-id subnet-c617baa1 --route-table-id rtb-3741e750 Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public AWS CLI: Create a Security Group/Rules 32 Create a new security group for the ‘outside’ facing interface (Optional: You can just use an existing group) # aws ec2 create-security-group --group-name csr --description csr-rules --vpc-id vpc-66a0a102 Create a new security group rule for SSH to the CSR # aws ec2 authorize-security-group-ingress --group-id sg-65c39b03 --protocol tcp --port 22 --cidr 0.0.0.0/0 Create a new security group rule for ICMP from the other CSRs (On-Prem and GCP CSR [optional: Just showing the format for your use]) # aws ec2 authorize-security-group-ingress --group-id sg-65c39b03 --ip-permissions '[{"IpProtocol": "icmp", "FromPort": -1, "ToPort": -1, "IpRanges": [{"CidrIp": "192.x.x.x/32"}, {"CidrIp": "35.x.x.x/32"}]}] Create a new security group rule for ESP (IP 50) from the other CSRs # aws ec2 authorize-security-group-ingress --group-id sg-65c39b03 --ip-permissions '[{"IpProtocol": "50", "IpRanges": [{"CidrIp": ”192.x.x.x/32"}, {"CidrIp": "35.x.x.x/32"}]}]' Create a new security group rule for IKE from the other CSRs # aws ec2 authorize-security-group-ingress --group-id sg-65c39b03 --ip-permissions '[{"IpProtocol": "17", "FromPort": 500, "ToPort": 500, "IpRanges": [{"CidrIp": "192.x.x.x/32"}, {"CidrIp": "35.x.x.x/32"}]}]' Create a new security group rule for IKE/NAT-T from the other CSRs # aws ec2 authorize-security-group-ingress --group-id sg-65c39b03 --ip-permissions '[{"IpProtocol": "17", "FromPort": 4500, "ToPort": 4500, "IpRanges": [{"CidrIp": "192.x.x.x/32"}, {"CidrIp": "35.x.x.x/32"}]}]’ Optional: You may want to create a security group just for the ’inside’ subnet that has different rules than the one for the ‘outside’ subnet Create a new security group for the ‘outside’ facing interface (Optional: You can just use an existing group) # aws ec2 authorize-security-group-ingress --group-id sg-84aef7e2 --protocol all --cidr 192.168.200.0/24 Create a new security group for the ‘outside’ facing interface (Optional: You can just use an existing group) # aws ec2 authorize-security-group-ingress --group-id sg-84aef7e2 --protocol all --cidr 172.16.2.0/24 Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public AWS CLI: Run a new CSR Instance Using Previous Parameters 33 { "ImageId": "ami-99e5d0f9", "InstanceType": "t2.medium", "KeyName": "mc-aws-key", "NetworkInterfaces": [ { "DeviceIndex": 0, "Description": "Primary network interface", "Groups": [ "sg-65c39b03" ], "PrivateIpAddresses": [ { "Primary": true, "PrivateIpAddress": "172.16.1.10" } ], "SubnetId": "subnet-0c15b86b" }, { "DeviceIndex": 1, "PrivateIpAddresses": [ { "Primary": true, "PrivateIpAddress": "172.16.2.10" } ], "SubnetId": "subnet-c617baa1" } ] } csr-create.json Create a CSR instance using the JSON file shown to the left # aws ec2 run-instances --cli-input-json file://csr-create.json Create a tag/name and associate it with the CSR (Optional) # aws ec2 create-tags --resources i-0f2a0ee857e9c2540 --tags Key=Name,Value=csr-aws-01 Create a new External IP (EIP) allocation (or use an existing one) # aws ec2 allocate-address eipalloc-ab35cb96 vpc 52.xxx.xxx.x Associate the EIP with the ’outside’ interface of the CSR (GigabitEthernet 1) # aws ec2 associate-address --allocation-id eipalloc-ab35cb96 --network-interface-id eni-dd5bd6f2 Modify the ’inside’ subnet to disable source/destination checking # aws ec2 modify-network-interface-attribute --network-interface-id eni-af67db80 --source-dest-check "{"Value": false}" A note about NAT: If you plan to use the CSR for NAT operation, you must disable source/destination checking on the outside CSR interface/subnet http://docs.aws.amazon.com/AmazonVPC/latest/UserG uide/VPC_NAT_Instance.html#EIP_Disable_SrcDestCh eck Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 34 Connect to the AWS CSR – Enable Interfaces # ssh -i "mc-aws-key.pem" ec2-user@52.xxx.xxx.x csr-aws-01#configure terminal Enter configuration commands, one per line. End with CNTL/Z. csr-aws-01(config)#interface gigabitEthernet 2 csr-aws-01(config-if)#ip address dhcp csr-aws-01(config-if)#no shutdown Connect to the new AWS-hosted CSR and enable the GigabitEthernet 2 interface for DHCP csr-aws-01#show ip interface brief Interface IP-Address OK? Method Status Protocol GigabitEthernet1 172.16.1.10 YES DHCP up up GigabitEthernet2 172.16.2.10 YES DHCP up up VirtualPortGroup0 192.168.35.1 YES TFTP up up Wait a few seconds and check to make sure that both interfaces on the CSR are up and have the correct IP addresses: Note: This can all be automated (along with the DMVPN configs) by creating AWS CloudFormation templates Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS AWS Cisco CSR DMVPN Config Spoke 35BRKCLD-3440 crypto ikev2 proposal AES/GCM/256 encryption aes-gcm-256 prf sha512 group 19 ! crypto ikev2 policy AES/GCM/256 match fvrf any proposal AES/GCM/256 ! crypto ikev2 keyring DMVPN-KEYRING peer ANY address 0.0.0.0 0.0.0.0 pre-shared-key <PSK_PASSWORD_GOES_HERE> ! crypto ikev2 profile DMVPN-IKEv2-PROFILE description PSK Profile match identity remote address 0.0.0.0 identity local address 52.xxx.xxx.x authentication remote pre-share authentication local pre-share keyring local DMVPN-KEYRING dpd 40 5 on-demand ! crypto ipsec security-association replay window-size 1024 ! crypto ipsec transform-set AES256/GCM/TRANSFORM esp-gcm 256 mode transport ! crypto ipsec profile DMVPN-IPSEC-PROFILE set transform-set AES256/GCM/TRANSFORM set ikev2-profile DMVPN-IKEv2-PROFILE ... Output summarized interface Tunnel0 description DMVPN ip address 10.1.0.4 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication <NHRP_PASSWORD> ip nhrp network-id 100 ip nhrp nhs 10.1.0.2 nbma 192.xxx.xxx.x multicast ip tcp adjust-mss 1360 ip ospf authentication-key 7 <OSPF_PASSWORD> ip ospf network point-to-multipoint ip ospf hello-interval 10 tunnel source GigabitEthernet1 tunnel mode gre multipoint tunnel key 100 tunnel protection ipsec profile DMVPN-IPSEC-PROFILE ! interface GigabitEthernet1 description Internet ip address dhcp no ip redirects no ip unreachables no ip proxy-arp negotiation auto ! router ospf 10 router-id 10.1.0.4 network 172.16.2.0 0.0.0.255 area 2 network 10.1.0.0 0.0.0.255 area 0 ! ip route 0.0.0.0 0.0.0.0 172.16.1.1 Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS On-Prem Cisco ASR DMVPN Config Hub – Nothing ever changes on the hub for each example 36BRKCLD-3440 crypto ikev2 proposal AES/GCM/256 encryption aes-gcm-256 prf sha512 group 19 ! crypto ikev2 policy AES/GCM/256 match fvrf any proposal AES/GCM/256 ! crypto ikev2 keyring DMVPN-KEYRING peer ANY address 0.0.0.0 0.0.0.0 pre-shared-key <PSK_PASSWORD_GOES_HERE> ! crypto ikev2 profile DMVPN-IKEv2-PROFILE description PSK Profile match identity remote address 0.0.0.0 identity local address 192.xxx.xxx.x authentication remote pre-share authentication local pre-share keyring local DMVPN-KEYRING dpd 40 5 on-demand ! crypto ipsec security-association replay window-size 1024 ! crypto ipsec transform-set AES256/GCM/TRANSFORM esp-gcm 256 mode transport ! crypto ipsec profile DMVPN-IPSEC-PROFILE set transform-set AES256/GCM/TRANSFORM set ikev2-profile DMVPN-IKEv2-PROFILE ... Output summarized interface Tunnel0 description DMVPN ip address 10.1.0.2 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication <NHRP_PASSWORD> ip nhrp map multicast dynamic ip nhrp network-id 100 ip nhrp redirect ip tcp adjust-mss 1360 ip ospf authentication-key 7 <OSPF_PASSWORD> ip ospf network point-to-multipoint ip ospf hello-interval 10 tunnel source GigabitEthernet0/0/0 tunnel mode gre multipoint tunnel key 100 tunnel protection ipsec profile DMVPN-IPSEC-PROFILE ! interface GigabitEthernet0/0/0 description Internet ip address 192.xxx.xxx.x 255.255.255.0 no ip redirects no ip unreachables no ip proxy-arp negotiation auto ! router ospf 10 router-id 10.1.0.2 network 10.1.0.0 0.0.0.255 area 0 network 192.168.200.0 0.0.0.255 area 0 ! ip route 0.0.0.0 0.0.0.0 192.xxx.xxx.x Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 37 Verify Routing and Reachability ... Output summarized [ec2-user@ip-172-16-2-192 ~]$ ping 192.168.200.30 PING 192.168.200.30 (192.168.200.30) 56(84) bytes of data. 64 bytes from 192.168.200.30: icmp_seq=1 ttl=62 time=2.75 ms 64 bytes from 192.168.200.30: icmp_seq=2 ttl=62 time=2.93 ms 64 bytes from 192.168.200.30: icmp_seq=3 ttl=62 time=2.75 ms Connect to an AWS instances and ping to the on-prem private network asr-mc-01#show ip route | i 172.16.2.0 O IA 172.16.2.0 [110/1001] via 10.1.0.4, 00:11:41, Tunnel0 On the on-prem ASR check the route for the AWS VPC network 172.16.2.0/24 csr-aws-01#show ip route | i 192.168.200.0 O 192.168.200.0/24 [110/1001] via 10.1.0.2, 6d17h, Tunnel0 On AWS check for the route for the on-prem network (192.168.200.0/24) VPC Network 172.16.2.0/24 .10.192 OSPF Hub Tunnel: 10.1.0.2 Spoke Tunnel: 10.1.0.4 Cisco CSR1000v Reference Private Network 192.168.200.0/24 OSPF 10 Area 0 VM .30 DataCenter Infra. Provider Networks with VLANs Example Cisco ASR1000
Google Cloud Platform– Cisco CSR & DMVPN
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 39 Private Network 192.168.200.0/24inside-network 10.0.1.0/24 OSPF 10 Area 0 Routes this side should see: 10.0.1.0/24 Routes this side should see: 192.168.200.0/24 Cisco CSR1000v OpenStack Default Network 10.138.0.0/20 .100.3 .2 .1 VM .30192.xxx.xxx.x35.xxx.xxx.x DMVPN Hub Tunnel: 10.1.0.2 Spoke Tunnel: 10.1.0.1 GCP CSR to On-Prem ASR – DMVPN Coming in 16.9.1 Release OSPF Compute Engine 2 1 DataCenter Infra. Provider Networks with VLANs Example Cisco ASR1000
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 40 gcloud – Create the GCP External IP, Inside VPC Network & Route Create a new external IP reservation that will be used for the GCP CSR NATed connection (or use an existing one) # gcloud compute addresses create csr-to-csr-ext-ip --region us-west1 Capture the external IP address # gcloud compute addresses list --filter="csr-to-csr-ext-ip" NAME REGION ADDRESS STATUS csr-to-csr-ext-ip us-west1 35.xxx.xxx.x RESERVED Create a new GCP inside network that will be attached to the ‘inside’ interface of the CSR # gcloud compute networks create inside-network --subnet-mode=custom Create a new GCP inside subnet - Associate it with the inside network # gcloud compute networks subnets create inside-subnet --network=inside-network --range=10.0.1.0/24 Create a new GCP route from the CSR inside network to the on-prem private network which routes through the IPsec VPN # gcloud compute routes create inside-to-csr-private --network=inside-network --destination-range=192.168.200.0/24 --next-hop-address=10.0.1.2 Coming in 16.9.1 Release Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 41 gcloud – Create GCP Firewall Rules Create a new GCP firewall rule to allow traffic into the inside CSR network from the default network # gcloud compute firewall-rules create allow-default-to-csr-inside --direction=INGRESS --network=inside-network --action=ALLOW --rules=all --source-ranges=0.0.0.0/0 Create a new GCP firewall rule to allow traffic between the default network and the on-prem ASR public IP for IKE, IPsec # gcloud compute firewall-rules create csr-csr-vpn --direction=INGRESS --network=default --action=ALLOW --rules=udp:500,udp:4500,esp --source-ranges=192.xxx.xxx.x Coming in 16.9.1 Release Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 42 gcloud – Create CSR and Test Instances Create a new GCE CSR instance and set fixed IPv4 addresses to each of the two interfaces # gcloud compute instances create "csr-gcp-01" --zone "us-west1-a" --machine-type "n1-standard-4" --network-interface subnet="default",private-network-ip="10.138.0.100",address="35.xxx.xxx.x" --can-ip-forward --network-interface subnet="inside-subnet",private-network-ip="10.0.1.2",no-address --image ”name_of_csr_image" --boot-disk-size "10" --boot-disk-type "pd-standard" --boot-disk-device-name "csr-gcp-01" Create a new GCE test instance that will be used to validate the VPN and routing # gcloud compute instances create "csr-inside-vm" --zone "us-west1-a" --machine-type "g1-small" --subnet "inside-subnet" --private-network-ip "10.0.1.3" --image "debian-9-stretch-v20170918" --image-project "debian-cloud" --boot-disk-size "10" --boot-disk-type "pd-standard" --boot-disk-device-name "csr-inside-vm" Coming in 16.9.1 Release Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Connect to the GCP CSR – Enable Interfaces 43 # gcloud compute ssh cisco-user@csr-gcp-01 csr1kv-gcp#configure terminal Enter configuration commands, one per line. End with CNTL/Z. csr1kv-gcp(config)#interface gigabitEthernet 2 csr1kv-gcp(config-if)#ip address dhcp csr1kv-gcp(config-if)#no shutdown ... Output summarized Connect to the new GCP-hosted CSR and enable the GigabitEthernet 2 interface for DHCP csr1kv-gcp#show ip interface brief Interface IP-Address OK? Method Status Protocol GigabitEthernet1 10.138.0.100 YES TFTP up up GigabitEthernet2 10.0.1.2 YES DHCP up up Wait a few seconds and check to make sure that both interfaces on the CSR are up and with the correct IP addresses: Coming in 16.9.1 Release Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public GCP Cisco CSR DMVPN Config Spoke 44 crypto ikev2 proposal AES/GCM/256 encryption aes-gcm-256 prf sha512 group 19 ! crypto ikev2 policy AES/GCM/256 match fvrf any proposal AES/GCM/256 ! crypto ikev2 keyring DMVPN-KEYRING peer ANY address 0.0.0.0 0.0.0.0 pre-shared-key <PSK_PASSWORD_GOES_HERE> ! crypto ikev2 profile DMVPN-IKEv2-PROFILE description PSK Profile match identity remote address 0.0.0.0 identity local address 35.xxx.xxx.x authentication remote pre-share authentication local pre-share keyring local DMVPN-KEYRING dpd 40 5 on-demand ! crypto ipsec security-association replay window-size 1024 ! crypto ipsec transform-set AES256/GCM/TRANSFORM esp-gcm 256 mode transport ! crypto ipsec profile DMVPN-IPSEC-PROFILE set transform-set AES256/GCM/TRANSFORM set ikev2-profile DMVPN-IKEv2-PROFILE ... Output summarized interface Tunnel0 description DMVPN ip address 10.1.0.1 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication <NHRP_PASSWORD> ip nhrp network-id 100 ip nhrp nhs 10.1.0.2 nbma 192.xxx.xxx.x multicast ip tcp adjust-mss 1360 ip ospf authentication-key 7 <OSPF_PASSWORD> ip ospf network point-to-multipoint ip ospf hello-interval 10 tunnel source GigabitEthernet1 tunnel mode gre multipoint tunnel key 100 tunnel protection ipsec profile DMVPN-IPSEC-PROFILE ! interface GigabitEthernet1 description Internet ip address 10.138.0.100 255.255.255.0 no ip redirects no ip unreachables no ip proxy-arp negotiation auto ! router ospf 10 router-id 10.1.0.1 network 10.0.1.0 0.0.0.255 area 1 network 10.1.0.0 0.0.0.255 area 0 ! ip route 0.0.0.0 0.0.0.0 138.0.0.1 Coming in 16.9.1 Release Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public On-Prem Cisco ASR DMVPN Config Hub 45 crypto ikev2 proposal AES/GCM/256 encryption aes-gcm-256 prf sha512 group 19 ! crypto ikev2 policy AES/GCM/256 match fvrf any proposal AES/GCM/256 ! crypto ikev2 keyring DMVPN-KEYRING peer ANY address 0.0.0.0 0.0.0.0 pre-shared-key <PSK_PASSWORD_GOES_HERE> ! crypto ikev2 profile DMVPN-IKEv2-PROFILE description PSK Profile match identity remote address 0.0.0.0 identity local address 192.xxx.xxx.x authentication remote pre-share authentication local pre-share keyring local DMVPN-KEYRING dpd 40 5 on-demand ! crypto ipsec security-association replay window-size 1024 ! crypto ipsec transform-set AES256/GCM/TRANSFORM esp-gcm 256 mode transport ! crypto ipsec profile DMVPN-IPSEC-PROFILE set transform-set AES256/GCM/TRANSFORM set ikev2-profile DMVPN-IKEv2-PROFILE ... Output summarized interface Tunnel0 description DMVPN ip address 10.1.0.2 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication <NHRP_PASSWORD> ip nhrp map multicast dynamic ip nhrp network-id 100 ip nhrp redirect ip tcp adjust-mss 1360 ip ospf authentication-key 7 <OSPF_PASSWORD> ip ospf network point-to-multipoint ip ospf hello-interval 10 tunnel source GigabitEthernet0/0/0 tunnel mode gre multipoint tunnel key 100 tunnel protection ipsec profile DMVPN-IPSEC-PROFILE ! interface GigabitEthernet0/0/0 description Internet ip address 192.xxx.xxx.x 255.255.255.0 no ip redirects no ip unreachables no ip proxy-arp negotiation auto ! router ospf 10 router-id 10.1.0.2 network 10.1.0.0 0.0.0.255 area 0 network 192.168.200.0 0.0.0.255 area 0 ! ip route 0.0.0.0 0.0.0.0 192.xxx.xxx.x Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Verify Routing and Reachability 46 ... Output summarized # gcloud compute ssh "csr-inside-vm“ shmcfarl@csr-inside-vm:~$ ping 192.168.200.30 PING 192.168.200.30 (192.168.200.30) 56(84) bytes of data. 64 bytes from 192.168.200.30: icmp_seq=1 ttl=62 time=22.1 ms 64 bytes from 192.168.200.30: icmp_seq=2 ttl=62 time=23.3 ms 64 bytes from 192.168.200.30: icmp_seq=3 ttl=62 time=23.6 ms Connect to the GCP test instance that was created earlier and ping to the on-prem private network csr1kv-gcp#show ip route | i 192.168.200.0 . . . O 192.168.200.0/24 [110/1001] via 10.1.0.2, 00:09:51, Tunnel0 On the GCP CSR, check for the private network route from the on-prem side(192.168.200.0/24) asr-mc-01#show ip route | i 10.0.1.0 . . . O IA 10.0.1.0/24 [110/1001] via 10.1.0.1, 00:40:08, Tunnel0 On the on-prem ASR, check for the VPC inside network route (10.1.0/24) csr1kv-gcp#show ip nhrp 10.1.0.2/32 via 10.1.0.2 Tunnel0 created 5d14h, never expire Type: static, Flags: NBMA address: 192.xxx.xxx.x Check the DMVPN Next-Hop Resolution Protocol (NHRP) Status asr-mc-01#show ip nhrp 10.1.0.1/32 via 10.1.0.1 Tunnel0 created 00:40:25, expire 00:08:20 Type: dynamic, Flags: registered used nhop NBMA address: 35.xxx.xxx.x (Claimed NBMA address: 10.138.0.100) Coming in 16.9.1 Release Reference
Microsoft Azure – Cisco CSR and DMVPN
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 48 Azure CSR to On-Prem ASR – DMVPN 192.xxx.xxx.x40.xxx.xxx.x Routes this side should see: 192.168.200.0/24 Routes this side should see: 10.10.1.0/24 Cisco CSR1000v DMVPN Hub Tunnel: 10.1.0.2 Spoke Tunnel: 10.1.0.6 OSPF Inside Subnet 10.10.1.0/24 Outside Subnet 10.10.0.0/24 Cisco ASR1000 Private Network 192.168.200.0/24 OSPF 10 Area 0 OpenStack VM .30 DataCenter Infra. Provider Networks with VLANs Example
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Microsoft Azure with Cisco CSR 1000v • Microsoft Azure Marketplace • https://azuremarketplace.microsoft.com/en- us/marketplace/apps/cisco.cisco-csr-basic-template • https://github.com/Azure/azure-quickstart-templates/tree/master/cisco- csr-1000v • Cisco CSR 1000v with Azure Deployment • https://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/azu/b_c sr1000config-azure.html 49
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 50 Azure CLI: Create Resource Group, Networks, Subnets Create a new Azure Resource Group (rg) # az group create --name multicloud-rg --location westus Create a new public (external IP) IPv4 address to be used for the CSR’s ‘outside’ interface # az network public-ip create --resource-group multicloud-rg --name csr-azure-01-eip --allocation-method static Create a new virtual network (vnet) and a subnet to be used for the CSR’s ‘outside interface # az network vnet create --resource-group multicloud-rg --name mc-csr-vnet --address-prefix 10.10.0.0/16 --subnet-name csr-outside --subnet-prefix 10.10.0.0/24 Create a new subnet for the CSR’s ‘inside’ interface and associate it with the vnet created above # az network vnet subnet create --resource-group multicloud-rg --vnet-name mc-csr-vnet --name csr-inside --address-prefix 10.10.1.0/24 Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS 51BRKCLD-3440 Azure CLI: Create Route Tables Create a new route table (rt) that will be used for the CSR’s ’outside’ subnet # az network route-table create --resource-group multicloud-rg --name csr-outside-rt Create a new route table that will used for the CSR’s ‘inside’ subnet # az network route-table create --resource-group multicloud-rg --name csr-inside-rt Create a new route table entry for the ‘inside’ subnet to reach the on-prem network (192.168.200.0) via the CSR’s IP (10.10.1.4) # az network route-table route create --resource-group multicloud-rg --name csr-to-on-prem-route --route-table-name csr-inside-rt --address-prefix 192.168.200.0/24 --next-hop-type VirtualAppliance --next-hop-ip-address 10.10.1.4 Associate the ‘outside’ route table with the ‘outside’ subnet # az network vnet subnet update --resource-group multicloud-rg --vnet-name mc-csr-vnet --name csr-outside --route-table csr-outside-rt Associate the ‘inside’ route table with the ‘inside’ subnet # az network vnet subnet update --resource-group multicloud-rg --vnet-name mc-csr-vnet --name csr-inside --route-table csr-inside-rt Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS 52BRKCLD-3440 Azure CLI: Create Network Security Group (NSG) Create a new Network Security Group (NSG) to be used for the ‘outside’ CSR interface # az network nsg create --resource-group multicloud-rg --name csr-nsg-outside Create a new NSG rule to allow inbound SSH access to the CSR (can make it specific to an IP/Prefix) # az network nsg rule create --resource-group multicloud-rg --nsg-name csr-nsg-outside --name SSHRule --priority 100 --source-address-prefixes 'Internet' --source-port-ranges '*' --destination-address-prefixes '*' --destination-port-ranges 22 --access Allow --protocol Tcp --direction inbound Create a new NSG rule to allow inbound UDP 500 (IKE) traffic to the CSR (can make it specific to an IP/Prefix) # az network nsg rule create --resource-group multicloud-rg --nsg-name csr-nsg-outside --name UDP-500 --priority 101 --source-address-prefixes 'Internet' --source-port-ranges '*' --destination-address-prefixes '*' --destination-port-ranges 500 --access Allow --protocol Udp --direction inbound Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS 53BRKCLD-3440 Azure CLI: Create NSG Rule & NICs Create a new Network Security Group (NSG) to be used for the ‘outside’ CSR interface # az network nsg rule create --resource-group multicloud-rg --nsg-name csr-nsg-outside --name UDP-4500 --priority 102 --source-address-prefixes 'Internet' --source-port-ranges '*' --destination-address-prefixes '*' --destination-port-ranges 4500 --access Allow --protocol Udp --direction inbound Create a new NIC to be used by the CSR’s ‘outside’ interface. Associate the NIC with the NSG, Subnet, Public IP & enable forwarding # az network nic create --resource-group multicloud-rg --name csr-nic-g1 --vnet-name mc-csr-vnet --subnet csr-outside --network-security-group csr-nsg-outside --ip-forwarding true --public-ip-address csr-azure-01-eip Create a new NIC to be used by the CSR’s ‘inside’ interface. Associate the NIC with the NSG, Subnet and enable forwarding # az network nic create --resource-group multicloud-rg --name csr-nic-g2 --vnet-name mc-csr-vnet --subnet csr-inside --ip-forwarding true Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 54 Azure CLI: Run a new CSR Instance Using Previous Parameters Create a CSR VM using a Azure Marketplace image. Associate the VM with two the NICs created earlier. # Note: The VM can be created with a large number of options to include SSH keys, image (BYOL, # of NICs), and size # az vm create --resource-group multicloud-rg --name csr-azure-01 --admin-username csr-azure --admin-password <PASSWORD> --authentication-type password --image cisco:cisco-csr-1000v:16_6:16.6.120170804 <<< Change image based on current release --nics csr-nic-g1 csr-nic-g2 --size Standard_D2_v2
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 55 Connect to the Azure CSR – Enable Interfaces # ssh csr-azure@40.xxx.xxx.x csr-azure-01#configure terminal Enter configuration commands, one per line. End with CNTL/Z. csr-azure-01(config)#interface gigabitEthernet 2 csr-azure-01(config-if)#ip address dhcp csr-azure-01(config-if)#no shutdown Connect to the new Azure-hosted CSR and enable the GigabitEthernet 2 interface for DHCP csr-azure-01#show ip interface brief Interface IP-Address OK? Method Status Protocol GigabitEthernet1 10.10.0.4 YES DHCP up up GigabitEthernet2 10.10.1.4 YES DHCP up up VirtualPortGroup0 192.168.35.1 YES TFTP up up Wait a few seconds and check to make sure that both interfaces on the CSR are up and have the correct IP addresses: Note: This can all be automated (along with the DMVPN configs) by creating Azure Automation/Resource Manager Reference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS Azure Cisco CSR DMVPN Config Spoke crypto ikev2 proposal AES/GCM/256 encryption aes-gcm-256 prf sha512 group 19 ! crypto ikev2 policy AES/GCM/256 match fvrf any proposal AES/GCM/256 ! crypto ikev2 keyring DMVPN-KEYRING peer ANY address 0.0.0.0 0.0.0.0 pre-shared-key <PSK_PASSWORD_GOES_HERE> ! crypto ikev2 profile DMVPN-IKEv2-PROFILE description PSK Profile match identity remote address 0.0.0.0 identity local address 40.xxx.xxx.x authentication remote pre-share authentication local pre-share keyring local DMVPN-KEYRING dpd 40 5 on-demand ! crypto ipsec security-association replay window-size 1024 ! crypto ipsec transform-set AES256/GCM/TRANSFORM esp-gcm 256 mode transport ! crypto ipsec profile DMVPN-IPSEC-PROFILE set transform-set AES256/GCM/TRANSFORM set ikev2-profile DMVPN-IKEv2-PROFILE ... Output summarized interface Tunnel0 description DMVPN ip address 10.1.0.6 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication <NHRP_PASSWORD> ip nhrp network-id 100 ip nhrp nhs 10.1.0.2 nbma 192.xxx.xxx.x multicast ip tcp adjust-mss 1360 ip ospf authentication-key 7 <OSPF_PASSWORD> ip ospf network point-to-multipoint ip ospf hello-interval 10 tunnel source GigabitEthernet1 tunnel mode gre multipoint tunnel key 100 tunnel protection ipsec profile DMVPN-IPSEC-PROFILE ! interface GigabitEthernet1 description Internet ip address dhcp no ip redirects no ip unreachables no ip proxy-arp negotiation auto ! router ospf 10 router-id 10.1.0.6 network 10.1.0.0 0.0.0.255 area 0 network 10.10.1.0 0.0.0.255 area 3 ! ip route 0.0.0.0 0.0.0.0 10.10.0.1
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS 57BRKCLD-3440 On-Prem Cisco ASR DMVPN Config Hub - Nothing ever changes on the hub for each example crypto ikev2 proposal AES/GCM/256 encryption aes-gcm-256 prf sha512 group 19 ! crypto ikev2 policy AES/GCM/256 match fvrf any proposal AES/GCM/256 ! crypto ikev2 keyring DMVPN-KEYRING peer ANY address 0.0.0.0 0.0.0.0 pre-shared-key <PSK_PASSWORD_GOES_HERE> ! crypto ikev2 profile DMVPN-IKEv2-PROFILE description PSK Profile match identity remote address 0.0.0.0 identity local address 192.xxx.xxx.x authentication remote pre-share authentication local pre-share keyring local DMVPN-KEYRING dpd 40 5 on-demand ! crypto ipsec security-association replay window-size 1024 ! crypto ipsec transform-set AES256/GCM/TRANSFORM esp-gcm 256 mode transport ! crypto ipsec profile DMVPN-IPSEC-PROFILE set transform-set AES256/GCM/TRANSFORM set ikev2-profile DMVPN-IKEv2-PROFILE ... Output summarized interface Tunnel0 description DMVPN ip address 10.1.0.2 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication <NHRP_PASSWORD> ip nhrp map multicast dynamic ip nhrp network-id 100 ip nhrp redirect ip tcp adjust-mss 1360 ip ospf authentication-key 7 <OSPF_PASSWORD> ip ospf network point-to-multipoint ip ospf hello-interval 10 tunnel source GigabitEthernet0/0/0 tunnel mode gre multipoint tunnel key 100 tunnel protection ipsec profile DMVPN-IPSEC-PROFILE ! interface GigabitEthernet0/0/0 description Internet ip address 192.xxx.xxx.x 255.255.255.0 no ip redirects no ip unreachables no ip proxy-arp negotiation auto ! router ospf 10 router-id 10.1.0.2 network 10.1.0.0 0.0.0.255 area 0 network 192.168.200.0 0.0.0.255 area 0 ! ip route 0.0.0.0 0.0.0.0 192.xxx.xxx.x
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 58 Verify Routing and Reachability ... Output summarized shmcfarl@AzTestVm:~$ping 192.168.200.30 PING 192.168.200.30 (192.168.200.30) 56(84) bytes of data. 64 bytes from 192.168.200.30: icmp_seq=2 ttl=62 time=3.99 ms 64 bytes from 192.168.200.30: icmp_seq=3 ttl=62 time=6.44 ms Connect to an AWS instances and ping to the on-prem private network asr-mc-01#show ip route | i 10.10.1.0 O IA 10.10.1.0/24 [110/1001] via 10.1.0.6, 00:19:15, Tunnel0 On the on-prem CSR check the route for the AWS VPC network 172.16.2.0/24 csr-azure-01#show ip route | i 192.168.200.0 O 192.168.200.0/24 [110/1001] via 10.1.0.2, 00:17:57, Tunnel0 On AWS check for the route for the on-prem network (192.168.200.0/24) Inside Subnet 10.10.1.0/24 .4.5 OSPF Hub Tunnel: 10.1.0.2 Spoke Tunnel: 10.1.0.6 Cisco CSR1000v Reference Cisco ASR1000 Private Network 192.168.200.0/24 OSPF 10 Area 0 VM .30 DataCenter Infra. Provider Networks with VLANs Example
Linking DMVPN Sites
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public DMVPN – Enable Dynamic Multicloud Networking 60 VNet Network 10.10.1.0/24 DMVPN VPC Network 172.16.2.0/24 Cisco CSR1000v Cisco CSR1000v VPC Network 10.0.1.0/24 Cisco CSR1000v Cisco ASR1000 Hub Tunnel: 10.1.0.2 Spoke Tunnel: 10.1.0.1 Spoke Tunnel: 10.1.0.4 Spoke Tunnel: 10.1.0.6 Private Network 192.168.200.0/24 OSPF 10 Area 0 OpenStack VM .30 DataCenter Infra. Provider Networks with VLANs Example
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS BRKCLD-3440 General Guidelines for DMVPN Between Clouds • Set the VPC routes for each site • Set the firewall/security groups/network security groups for each site/protocol gcloud compute routes create inside-to-aws --network=csr-inside-network --destination-range=172.16.2.0/24 --next-hop-address=10.0.1.2 gcloud compute routes create inside-to-azure --network=csr-inside-network --destination-range=10.10.1.0/24 --next-hop-address=10.0.1.2 Create a very specific per-site route (AWS example allowing UDP500 from each cloud provider public IP) aws ec2 authorize-security-group-ingress --group-id sg-65c39b03 --ip-permissions '[{"IpProtocol": "17", "FromPort": 500, "ToPort": 500, "IpRanges": [{"CidrIp": "192.x.x.x/32"}, {"CidrIp": "35.x.x.x/32"}, {"CidrIp": "40.x.x.x/32"}]}]’ Alternatively, you can open it up (Azure example) az network nsg rule create --resource-group multicloud-rg --nsg-name csr-nsg-outside --name UDP-4500 --priority 102 --source-address-prefixes 'Internet' --source-port-ranges '*' --destination-address-prefixes '*' --destination-port-ranges 4500 --access Allow --protocol Udp --direction inbound
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Routing Example – All Sites 62 • For spoke-to-spoke direct routing with DMVPN/NHRP: • ‘ip nhrp redirect’ on the hubs • ‘ip nhrp shortcut’ on the spokes asr-mc-01#show ip route ospf 10.0.0.0/8 is variably subnetted, 7 subnets, 2 masks O IA 10.0.1.0/24 [110/1001] via 10.1.0.1, 02:40:45, Tunnel0 O 10.1.0.1/32 [110/1000] via 10.1.0.1, 02:40:45, Tunnel0 O 10.1.0.4/32 [110/1000] via 10.1.0.4, 01:18:49, Tunnel0 O 10.1.0.6/32 [110/1000] via 10.1.0.6, 00:56:19, Tunnel0 O IA 10.10.1.0/24 [110/1001] via 10.1.0.6, 00:55:34, Tunnel0 172.16.0.0/24 is subnetted, 1 subnets O IA 172.16.2.0 [110/1001] via 10.1.0.4, 01:18:49, Tunnel0 ... Output summarized Hub On-Prem CSR csr1kv-gcp#show ip route ospf 10.0.0.0/8 is variably subnetted, 10 subnets, 2 masks O 10.1.0.2/32 [110/1000] via 10.1.0.2, 02:43:14, Tunnel0 O % 10.1.0.4/32 [110/2000] via 10.1.0.2, 01:21:14, Tunnel0 O % 10.1.0.6/32 [110/2000] via 10.1.0.2, 00:58:47, Tunnel0 O IA% 10.10.1.0/24 [110/2001] via 10.1.0.2, 00:58:00, Tunnel0 172.16.0.0/24 is subnetted, 1 subnets O IA% 172.16.2.0 [110/2001] via 10.1.0.2, 01:21:14, Tunnel0 O 192.168.200.0/24 [110/1001] via 10.1.0.2, 02:43:14, Tunnel0 Spoke – Google Cloud Platform CSR csr-aws-01#show ip route ospf 10.0.0.0/8 is variably subnetted, 7 subnets, 2 masks O IA% 10.0.1.0/24 [110/2001] via 10.1.0.2, 01:21:32, Tunnel0 O % 10.1.0.1/32 [110/2000] via 10.1.0.2, 01:21:32, Tunnel0 O 10.1.0.2/32 [110/1000] via 10.1.0.2, 01:21:32, Tunnel0 O % 10.1.0.6/32 [110/2000] via 10.1.0.2, 00:59:01, Tunnel0 O IA% 10.10.1.0/24 [110/2001] via 10.1.0.2, 00:58:14, Tunnel0 O 192.168.200.0/24 [110/1001] via 10.1.0.2, 01:21:32, Tunnel0 Spoke – Amazon Web Services CSR csr-azure-01#show ip route ospf 10.0.0.0/8 is variably subnetted, 10 subnets, 2 masks O IA% 10.0.1.0/24 [110/2001] via 10.1.0.2, 00:58:44, Tunnel0 O % 10.1.0.1/32 [110/2000] via 10.1.0.2, 00:58:44, Tunnel0 O 10.1.0.2/32 [110/1000] via 10.1.0.2, 00:58:44, Tunnel0 O % 10.1.0.4/32 [110/2000] via 10.1.0.2, 00:58:44, Tunnel0 172.16.0.0/24 is subnetted, 1 subnets O IA% 172.16.2.0 [110/2001] via 10.1.0.2, 00:58:44, Tunnel0 O 192.168.200.0/24 [110/1001] via 10.1.0.2, 00:58:44, Tunnel0 Spoke – Azure CSR IA - OSPF inter area % - next hop override
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public#CLUS BRKCLD-3440 NHRP Example – Hub/Spoke asr-mc-01#show ip nhrp 10.1.0.1/32 via 10.1.0.1 Tunnel0 created 02:02:42, expire 00:08:17 Type: dynamic, Flags: registered used nhop NBMA address: 35.xxx.xxx.x (Claimed NBMA address: 10.138.0.100) 10.1.0.4/32 via 10.1.0.4 Tunnel0 created 00:42:52, expire 00:09:17 Type: dynamic, Flags: registered used nhop NBMA address: 52.xxx.xxx.x (Claimed NBMA address: 172.16.1.10) 10.1.0.6/32 via 10.1.0.6 Tunnel0 created 00:18:12, expire 00:08:26 Type: dynamic, Flags: registered used nhop NBMA address: 40.xxx.xxx.x (Claimed NBMA address: 10.10.0.4) asr-mc-01#show ip nhrp multicast I/F NBMA address Tunnel0 35.xxx.xxx.x Flags: dynamic (Enabled) Tunnel0 52.xxx.xxx.x Flags: dynamic (Enabled) Tunnel0 40.xxx.xxx.x Flags: dynamic (Enabled) Hub On-Prem ASR csr-azure-01#show ip nhrp 10.0.1.0/24 via 10.1.0.1 Tunnel0 created 00:06:26, expire 00:03:32 Type: dynamic, Flags: router rib nho NBMA address: 35.xxx.xxx.x (Claimed NBMA address: 10.138.0.100) 10.1.0.1/32 via 10.1.0.1 Tunnel0 created 00:06:26, expire 00:03:32 Type: dynamic, Flags: router nhop rib nho NBMA address: 35.xxx.xxx.x (Claimed NBMA address: 10.138.0.100) 10.1.0.2/32 via 10.1.0.2 Tunnel0 created 00:21:28, never expire Type: static, Flags: NBMA address: 192.xxx.xxx.x 10.1.0.4/32 via 10.1.0.4 Tunnel0 created 00:12:29, expire 00:02:40 Type: dynamic, Flags: router nhop rib nho NBMA address: 52.xxx.xxx.x (Claimed NBMA address: 172.16.1.10) 10.10.1.0/24 via 10.1.0.6 Tunnel0 created 00:08:30, expire 00:03:33 Type: dynamic, Flags: router unique local NBMA address: 10.10.0.4 (no-socket) 172.16.2.0/24 via 10.1.0.4 Tunnel0 created 00:07:19, expire 00:02:40 Type: dynamic, Flags: router rib nho NBMA address: 52.xxx.xxx.x (Claimed NBMA address: 172.16.1.10) csr-azure-01#show ip nhrp multicast I/F NBMA address Tunnel0 192.xxx.xxx.x Flags: nhs (Enabled) Spoke – Azure CSR shmcfarl@AzureTestVm:~$ traceroute 10.0.1.3 traceroute to 10.0.1.3 (10.0.1.3), 30 hops max, 60 byte packets 1 10.10.1.4 (10.10.1.4) 1.220 ms 1.192 ms 1.328 ms 2 10.0.1.3 (10.0.1.3) 25.794 ms * 25.782 ms Spoke – Azure VM
Demo 64Presentation ID
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public DMVPN – Enable Dynamic Multicloud Networking 65 VNet Network 10.10.1.0/24 DMVPN OpenStack Private Cloud VPC Network 172.16.2.0/24 Cisco CSR1000v Cisco CSR1000v VPC Network 10.0.1.0/24 Cisco CSR1000v Cisco ASR1000 Hub Tunnel: 10.1.0.2 Spoke Tunnel: 10.1.0.1 Spoke Tunnel: 10.1.0.4 Spoke Tunnel: 10.1.0.6 192.168.200.0/24 OSPF 10 Area 0 VM.110 DataCenter Infra. 10.40.0.0/24 .6 Neutron Router
Split- Tunneling/Routing Options
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Split-Tunnel/Routing Options 67 • All three public cloud providers allow for either split-tunneling or forced/direct routing • Split-tunneling: • Public cloud resources (instances/VMs, container clusters) will use the default VPC gateway for non-on-prem routes • Public cloud resources will use the on-prem-specific routes advertised by the CSR • Forced/Direct routing – All public cloud resources will use the VPN connection as their default route for ALL traffic (forces traffic through the on-prem site) BGP Google Cloud VPN Google Cloud Router Cisco ASR1000 10.0.0.1 VPC Subnetwork GW External/NAT Routing 192.xxx.xxx.x 35.xxx.xxx.x10.0.0.5 Compute Engine 2 1
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Dealing with Split Routes • OpenStack with two possible routes: • Typically the Neutron L3 agent is the default route for VMs on the Private-Network (172.16.0.1) • Adding a CSR for GCP-facing connections requires route changes: • Static definition or dynamically learned via Neutron BGP service 68 [centos@c7-os-vm1 ~]$ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 172.16.0.1 0.0.0.0 UG 0 0 0 eth0 10.138.0.0 172.16.0.11 255.255.240.0 UG 0 0 0 eth0 169.254.169.254 172.16.0.1 255.255.255.255 UGH 0 0 0 eth0 172.16.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 openstack subnet set --host-route destination=10.138.0.0/20,gateway=172.16.0.11 Private-Subnet Default 10.138.0.0/20
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Summary • Public cloud native IPsec VPN support is good, but it is always point-to-point, does not have consistent support for NAT and lacks network-rich features • DMVPN can greatly improve the deployment, HA, scalability and operations of the VPN connections • Multicloud between multiple public cloud providers and on-prem look like distinctly separate hybrid cloud deployments but.. • You have to take into consideration: • Team knowledge of public cloud operations, tools, automation • Cross cloud tools and automation • Diversity of network designs, protocols, security • Multi-region designs • Availability zones within and across providers 69
Reference Slides 71
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public A Note On MTU • All three providers recommend a different size interface MTU for the IPsec tunnel interface: • Google recommends 1460 on the tunnel: https://cloud.google.com/vpn/docs/concepts/advanced#mtu • AWS recommends 1399 on the tunnel: https://docs.aws.amazon.com/AmazonVPC/latest/NetworkAdminGuide/Introduction.html • Azure recommends 1400 on the tunnel: https://docs.microsoft.com/en-us/azure/vpn- gateway/vpn-gateway-about-vpn-devices • In addition to MTU, you need to set and test your TCP MSS values • In my testing, an IP MTU of 1400 and TCP MSS of 1360 worked for all sites but this may need to change based on your applications and if you are adding other encaps like MPLS 72
Automation Challenges
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Automating the Multicloud Network • Challenges: • Different toolsets for different jobs (Ansible, Python, Bash scripts, Terraform, etc..) • Different toolsets for different clouds (Heat for OpenStack, CloudFormation for AWS, Deployment Manager for GCP, Azure Automation) • Different toolsets for different vendor products (Cisco NSO, CloudCenter, Prime, YANG development kit, etc..) • There is no silver bullet - Start simple: • Use what your team knows – Perform a gap analysis on what you have against what you need • Initially, automate the things that hurt a lot to do by hand and that change frequently – I use free tools but that doesn’t mean the process is free J • I use public cloud clients (gcloud, aws cli, azure cli) for services that don’t change frequently or that need very unique/non- repeatable configurations • I use public cloud provider automation tools (GCP Deployment Manager) for in-project work (new instances with new networks for a GCP-only project) • I use REST for things that change a lot • When you want to stop pulling your hair out, move to something that can front-end each API that you need to talk to and treat the environment as a whole – Cisco CloudCenter: https://www.cisco.com/c/en/us/products/cloud-systems- management/cloudcenter/index.html 74
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Amazon CloudFormation • https://aws.amazon.com/cloudformation/ • Template-based (JSON/YAML) – Build a stack(s) from a template file • Sometimes you need to run more than one stack (in order) to get what you need • Race conditions: Understand ‘DependsOn’ and the use of the wait condition • If you need to use more than one stack, use “Outputs” to export values that the next stack will need to build the next set of resources • Example template: https://github.com/shmcfarl/multicloud/tree/master/aws/cloudformation 75
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Google Cloud Platform – Deployment Manager • https://cloud.google.com/deployment-manager/ • Configuration files (YAML), Templates (Python/Jinja2), Schema files (JSON) • Sometimes you need to run more than one stack (in order) to get what you need • Race conditions • Use “Outputs” to export values that the next stack will need to build the next set of resources • Example templates: https://github.com/shmcfarl/multicloud/tree/master/gcp/deployment- manager • Make your own changes to the files: <ZONE>, <PROJECT>, <IMAGE>, etc. • Deploy the main stack: • Deploy any custom routes that may be needed for other sites 76 gcloud deployment-manager deployments create gcp-stack --config gcp_main_stack.yaml --automatic-rollback-on-error gcloud deployment-manager deployments create gcp-stack-route --config inside-private-routes.yaml --automatic-rollback-on-error
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Microsoft Azure Automation/Resource Manager • https://azure.microsoft.com/en-us/services/automation/ • Runbooks (create graphically, PowerShell, Python) • Read and select these carefully: https://docs.microsoft.com/en- us/azure/automation/automation-runbook-types • Resource Manager: https://docs.microsoft.com/en-us/azure/azure-resource- manager/resource-group-overview • https://github.com/Azure/azure-quickstart-templates/tree/master/cisco-csr-1000v • Example template: https://github.com/shmcfarl/multicloud/blob/master/azure/resource-manager/az- arm-csr-cleaned.json 77
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Call APIs Directly • Google Cloud Platform: https://cloud.google.com/compute/docs/reference/latest/ • Amazon Web Services: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/Welc ome.html • Microsoft Azure: https://docs.microsoft.com/en-us/rest/api/ 78
Google VPN – Creating Google VPN, Router, IPsec, BGP via REST APIs
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Google Cloud API – Creating GCP Cloud VPN/Routers • Assumptions/environment: • Understand how to authenticate to GCP APIs: https://cloud.google.com/docs/authentication/ • In this example, the Paw application was used to craft GET, POST and PATCH calls • Some configurations have been sanitized for security purposes • Have on-prem Cloud infrastructure deployed and a CSR/ASR configured (can be done after GCP side is deployed) • In this example, the configuration will be deployed against the OpenStack use case discussed in the earlier slides • In this example, the default network created by GCP will be used • Note: gcloud has VERY long delays in commands if you have IPv6 enabled on your local machine – set to “link-local” mode on your Mac 80
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 81 Reference Topology for GCP API Example Private Network 172.16.0.0/24 .11Default Network 10.138.0.0/20 IPsec/IKEv2 Tunnel Mode OSPF 10 Area 0 OSPF<>BGP Redistribution 192.yyy.yyy.y 35.yyy.yyy.y Routes this side should see: 10.138.0.0/20 Routes this side should see: 172.16.0.0/24 On-Prem Cloud Google Cloud VPN Google Cloud Router BGP AS65000 BGP AS65003 169.254.0.5 169.254.0.6
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public GCP API (1) – Create VPN GW and External IP POST /compute/v1/projects/<gcp_project_number>/regions/us-west1/targetVpnGateways HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close Content-Length: 138 { "name": "csr-gcp-os-aio-gw", "network": "projects/<gcp_project_number>/global/networks/default", "region": "projects/<gcp_project_number>/regions/us-west1" } POST /compute/v1/projects/<gcp_project_number>/regions/us-west1/addresses HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close Content-Length: 29 { "name": "gcp-to-os-dmz" } GET /compute/v1/projects/<gcp_project_number>/regions/us-west1/addresses/gcp-to-os-dmz HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close RESPONSE - SUMMARIZED: "name": "gcp-to-os-dmz", "description": "", "address": ”35.yyy.yyy.y", "status": "RESERVED", ... Output summarized POST: Create VPN Gateway POST: Create External IP Address GET: Get the External IP Address
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public GCP API (2) – Create Forwarding Rules 83 POST /compute/v1/projects/<gcp_project_number>/regions/us-west1/forwardingRules HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close Content-Length: 257 { "name": "csr-gcp-os-aio-rule-esp", "IPProtocol": "ESP", "IPAddress": "35.yyy.yyy.y", "region": "projects/<gcp_project_number>/regions/us-west1", "target": "https://www.googleapis.com/compute/v1/projects/<gcp_project_number>/regions/us-west1/targetVpnGateways/csr-gcp-os-aio-gw" } POST /compute/v1/projects/<gcp_project_number>/regions/us-west1/forwardingRules HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close Content-Length: 278 { "name": "csr-gcp-os-aio-rule-udp500", "IPProtocol": "UDP", "IPAddress": "35.yyy.yyy.y", "region": "projects/<gcp_project_number>/regions/us-west1", "target": "https://www.googleapis.com/compute/v1/projects/<gcp_project_number>/regions/us-west1/targetVpnGateways/csr-gcp-os-aio-gw", "portRange": "500" } POST /compute/v1/projects/<gcp_project_number>/regions/us-west1/forwardingRules HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close Content-Length: 280 { "name": "csr-gcp-os-aio-rule-udp4500", "IPProtocol": "UDP", "IPAddress": "35.yyy.yyy.y", "region": "projects/<gcp_project_number>/regions/us-west1", "target": "https://www.googleapis.com/compute/v1/projects/<gcp_project_number>/regions/us-west1/targetVpnGateways/csr-gcp-os-aio-gw", "portRange": "4500" } POST: Create Forwarding rule for ESP ... Output summarized POST: Create Forwarding rule for UDP 500 POST: Create Forwarding rule for UDP 4500
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public GCP API (3) – Create Cloud Router & BGP Session 84 POST /compute/v1/projects/<gcp_project_number>/regions/us-west1/routers HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close Content-Length: 574 { "name": "csr-gcp-os-bgp-rtr", "bgp": { "asn": "65000" }, "interfaces": [ { "name": "if-csr-gcp-os-bgp-rtr-02", "linkedVpnTunnel": "https://www.googleapis.com/compute/v1/projects/<gcp_project_number>/regions/us-west1/vpnTunnels/csr-gcp-os-aio-gw-tunnel-1", "ipRange": "169.254.0.5/30" } ], "bgpPeers": [ { "name": "csr-gcp-os-bgp-peer", "interfaceName": "if-csr-gcp-os-bgp-rtr-02", "ipAddress": "169.254.0.5", "peerIpAddress": "169.254.0.6", "peerAsn": "65003" } ], "region": "projects/<gcp_project_number>/regions/us-west1", "network": "https://www.googleapis.com/compute/v1/projects/<gcp_project_number>/global/networks/default" } POST: Create Cloud Router, BGP session and link to the Cloud VPN tunnel ... Output summarized
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public GCP API (5) – Create Cloud VPN Tunnel 85 POST /compute/v1/projects/<gcp_project_number>/regions/us-west1/vpnTunnels HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close Content-Length: 417 { "name": "csr-gcp-os-aio-gw-tunnel-1", "sharedSecret": " <pre-shared-password-goes-here> ", "router": "https://www.googleapis.com/compute/v1/projects/<gcp_project_number>/regions/us-west1/routers/csr-gcp-os-bgp-rtr", "peerIp": "192.yyy.yyy.y", "region": "projects/<gcp_project_number>/regions/us-west1", "ikeVersion": "2", "targetVpnGateway": "https://www.googleapis.com/compute/v1/projects/<gcp_project_number>/regions/us-west1/targetVpnGateways/csr-gcp-os-aio-gw" } POST: Create a Cloud VPN tunnel and associated it with the Cloud router ... Output summarized

More Related Content

PDF
[2018] 오픈스택 5년 운영의 경험
byNHN FORWARD
 
PDF
Kubernetes networking
bySim Janghoon
 
PDF
Open shift 4 infra deep dive
byWinton Winton
 
PDF
[오픈소스컨설팅] Open Stack Ceph, Neutron, HA, Multi-Region
byJi-Woong Choi
 
PDF
Accelerating Envoy and Istio with Cilium and the Linux Kernel
byThomas Graf
 
PPTX
Meetup 23 - 02 - OVN - The future of networking in OpenStack
byVietnam Open Infrastructure User Group
 
PDF
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
byEdureka!
 
PPTX
OVN - Basics and deep dive
byTrinath Somanchi
 
[2018] 오픈스택 5년 운영의 경험
byNHN FORWARD
 
Kubernetes networking
bySim Janghoon
 
Open shift 4 infra deep dive
byWinton Winton
 
[오픈소스컨설팅] Open Stack Ceph, Neutron, HA, Multi-Region
byJi-Woong Choi
 
Accelerating Envoy and Istio with Cilium and the Linux Kernel
byThomas Graf
 
Meetup 23 - 02 - OVN - The future of networking in OpenStack
byVietnam Open Infrastructure User Group
 
Kubernetes Architecture | Understanding Kubernetes Components | Kubernetes Tu...
byEdureka!
 
OVN - Basics and deep dive
byTrinath Somanchi
 

What's hot

PDF
[MeetUp][2nd] 오리뎅이의_쿠버네티스_네트워킹_v1.2
byInfraEngineer
 
PDF
stupid-simple-kubernetes-final.pdf
byDaniloQueirozMota
 
PDF
Kubernetes Networking
byCJ Cullen
 
PDF
Open vSwitch 패킷 처리 구조
bySeung-Hoon Baek
 
PDF
OpenShift-Technical-Overview.pdf
byJuanSalinas593459
 
PPTX
Kubernetes PPT.pptx
byssuser0cc9131
 
PDF
OpenStack Networking
byIlya Shakhat
 
PPTX
Kubernetes Networking 101
byWeaveworks
 
PDF
Kubernetes Architecture and Introduction
byStefan Schimanski
 
PPTX
Adopting OpenTelemetry
byVincent Behar
 
PPTX
쿠버네티스 ( Kubernetes ) 소개 자료
byOpennaru, inc.
 
PDF
Deep dive into Kubernetes Networking
bySreenivas Makam
 
PDF
Cilium - Network security for microservices
byThomas Graf
 
PDF
Kubernetes - A Comprehensive Overview
byBob Killen
 
PPTX
Service mesh
byArnab Mitra
 
PDF
Hands-On Introduction to Kubernetes at LISA17
byRyan Jarvinen
 
PPTX
Everything You Need To Know About Persistent Storage in Kubernetes
byThe {code} Team
 
PDF
Kubernetes Basics
byEueung Mulyana
 
PPTX
Service Discovery In Kubernetes
byKnoldus Inc.
 
PDF
Open vSwitch - Stateful Connection Tracking & Stateful NAT
byThomas Graf
 
[MeetUp][2nd] 오리뎅이의_쿠버네티스_네트워킹_v1.2
byInfraEngineer
 
stupid-simple-kubernetes-final.pdf
byDaniloQueirozMota
 
Kubernetes Networking
byCJ Cullen
 
Open vSwitch 패킷 처리 구조
bySeung-Hoon Baek
 
OpenShift-Technical-Overview.pdf
byJuanSalinas593459
 
Kubernetes PPT.pptx
byssuser0cc9131
 
OpenStack Networking
byIlya Shakhat
 
Kubernetes Networking 101
byWeaveworks
 
Kubernetes Architecture and Introduction
byStefan Schimanski
 
Adopting OpenTelemetry
byVincent Behar
 
쿠버네티스 ( Kubernetes ) 소개 자료
byOpennaru, inc.
 
Deep dive into Kubernetes Networking
bySreenivas Makam
 
Cilium - Network security for microservices
byThomas Graf
 
Kubernetes - A Comprehensive Overview
byBob Killen
 
Service mesh
byArnab Mitra
 
Hands-On Introduction to Kubernetes at LISA17
byRyan Jarvinen
 
Everything You Need To Know About Persistent Storage in Kubernetes
byThe {code} Team
 
Kubernetes Basics
byEueung Mulyana
 
Service Discovery In Kubernetes
byKnoldus Inc.
 
Open vSwitch - Stateful Connection Tracking & Stateful NAT
byThomas Graf
 

Similar to Openstack Summit Vancouver 2018 - Multicloud Networking

PDF
AWS VPC
byKiranChinnagangannag
 
PDF
AWS Direct Connect 및 VPN을 이용한 클라우드 아키텍쳐 설계:: Steve Seymour :: AWS Summit Seou...
byAmazon Web Services Korea
 
PPT
Cisco MPLS
bywebhostingguy
 
PPTX
Introduction to AWS VPC & Networking
byMichael Pearce
 
PPTX
CCNP ROUTE V7 CH6
byChaing Ravuth
 
PDF
cisco csr1000v
byMing914298
 
PPTX
ENSA_Module_8.pptx
bySkyBlue659156
 
PDF
How to Connect Your On-Premises Site to AWS Using Site-to-Site VPN.pdf
byTamanna
 
PPTX
GDSC Google Cloud Study Jams Session - 3
bySahithiGurlinka
 
PPTX
GCCP - Session #3
byGDSCIIITDHARWAD
 
PDF
AWS VPC NOTES _ LEARN AWS EFFECTIVELY and Easily
byakramemohemat
 
PPTX
AWS Hybrid Cloud Connectivity - VPN Solutions
byKent Plummer
 
PDF
Fostering the Evolution of Network Based Cloud Service Providers.
byCisco Service Provider
 
PDF
Private cloud networking_cloudstack_days_austin
byChiradeep Vittal
 
PPTX
GCCP Session 3
byDSCIITPatna
 
PDF
Cisco connect montreal 2018 vision mondiale analyse locale
byCisco Canada
 
PDF
World of many (OpenStack) clouds - the Making of the Intercloud
byLew Tucker
 
PDF
VMworld 2014: vCloud Hybrid Service Networking Technical Deep Dive
byVMworld
 
PPTX
AWS Certified Solutions Architect Professional Course S6-S9
byNeal Davis
 
PPTX
Productos de redes con AWS
byAmazon Web Services LATAM
 
AWS VPC
byKiranChinnagangannag
 
AWS Direct Connect 및 VPN을 이용한 클라우드 아키텍쳐 설계:: Steve Seymour :: AWS Summit Seou...
byAmazon Web Services Korea
 
Cisco MPLS
bywebhostingguy
 
Introduction to AWS VPC & Networking
byMichael Pearce
 
CCNP ROUTE V7 CH6
byChaing Ravuth
 
cisco csr1000v
byMing914298
 
ENSA_Module_8.pptx
bySkyBlue659156
 
How to Connect Your On-Premises Site to AWS Using Site-to-Site VPN.pdf
byTamanna
 
GDSC Google Cloud Study Jams Session - 3
bySahithiGurlinka
 
GCCP - Session #3
byGDSCIIITDHARWAD
 
AWS VPC NOTES _ LEARN AWS EFFECTIVELY and Easily
byakramemohemat
 
AWS Hybrid Cloud Connectivity - VPN Solutions
byKent Plummer
 
Fostering the Evolution of Network Based Cloud Service Providers.
byCisco Service Provider
 
Private cloud networking_cloudstack_days_austin
byChiradeep Vittal
 
GCCP Session 3
byDSCIITPatna
 
Cisco connect montreal 2018 vision mondiale analyse locale
byCisco Canada
 
World of many (OpenStack) clouds - the Making of the Intercloud
byLew Tucker
 
VMworld 2014: vCloud Hybrid Service Networking Technical Deep Dive
byVMworld
 
AWS Certified Solutions Architect Professional Course S6-S9
byNeal Davis
 
Productos de redes con AWS
byAmazon Web Services LATAM
 

Recently uploaded

PDF
December Patch Tuesday
byIvanti
 
PDF
Digit Expo 2025 - EICC Edinburgh 27th November
byRay Bugg
 
PDF
Unser Jahresrückblick – MarvelClient in 2025
bypanagenda
 
PDF
Agentic Automation for Testers – A Hands-On Deep Dive
byUiPathCommunity
 
PDF
Vibe Coding vs. Spec-Driven Development [Free Meetup]
byHaim Michael
 
PDF
Igniting the Future: Copilot trends, agentic transformation and product roadm...
byUni Systems S.M.S.A.
 
PPTX
Basics of Identity Access Management In mordern Infrastructure
byPrinceXavier18
 
PPTX
The Landscape of Computer Software Development Companies
byYash Singh
 
PDF
Decoding the DNA: The Digital Networks Act, the Open Internet, and IP interco...
byCSUC - Consorci de Serveis Universitaris de Catalunya
 
PDF
Session 1 - Solving Semi-Structured Documents with Document Understanding
byDianaGray10
 
PDF
Safeguarding AI-Based Financial Infrastructure
byYasir Naveed Riaz
 
PDF
DIGITAL FORENSICS - Notes for Everything.pdf
bypankajkumavatbeit
 
PPTX
THIS IS CYBER SECURITY NOTES USED IN CLASS ON VARIOUS TOPICS USED IN CYBERSEC...
byMelissaGblinwon
 
PDF
The major tech developments for 2026 by Pluralsight, a research and training ...
byChris Skinner
 
PDF
TrustArc Webinar - Looking Ahead: The 2026 Privacy Landscape
byTrustArc
 
PDF
Generative AI in 2026: Hype, Bubble, Winter or The Real Deal?
byDr. Tathagat Varma
 
PDF
Incident Response Planning with a Foundation Model
byKim Hammar
 
PDF
Greetings All Students Update 3 by Mia Corp
by©LDMMIA, ©Reiki Yoga
 
PDF
The Ultimate Guide to Problem Management Dashboards for IT Teams.pdf
byomnex systems
 
PDF
Recursive Self Improvement vs Continuous Learning
byBob Marcus
 
December Patch Tuesday
byIvanti
 
Digit Expo 2025 - EICC Edinburgh 27th November
byRay Bugg
 
Unser Jahresrückblick – MarvelClient in 2025
bypanagenda
 
Agentic Automation for Testers – A Hands-On Deep Dive
byUiPathCommunity
 
Vibe Coding vs. Spec-Driven Development [Free Meetup]
byHaim Michael
 
Igniting the Future: Copilot trends, agentic transformation and product roadm...
byUni Systems S.M.S.A.
 
Basics of Identity Access Management In mordern Infrastructure
byPrinceXavier18
 
The Landscape of Computer Software Development Companies
byYash Singh
 
Decoding the DNA: The Digital Networks Act, the Open Internet, and IP interco...
byCSUC - Consorci de Serveis Universitaris de Catalunya
 
Session 1 - Solving Semi-Structured Documents with Document Understanding
byDianaGray10
 
Safeguarding AI-Based Financial Infrastructure
byYasir Naveed Riaz
 
DIGITAL FORENSICS - Notes for Everything.pdf
bypankajkumavatbeit
 
THIS IS CYBER SECURITY NOTES USED IN CLASS ON VARIOUS TOPICS USED IN CYBERSEC...
byMelissaGblinwon
 
The major tech developments for 2026 by Pluralsight, a research and training ...
byChris Skinner
 
TrustArc Webinar - Looking Ahead: The 2026 Privacy Landscape
byTrustArc
 
Generative AI in 2026: Hype, Bubble, Winter or The Real Deal?
byDr. Tathagat Varma
 
Incident Response Planning with a Foundation Model
byKim Hammar
 
Greetings All Students Update 3 by Mia Corp
by©LDMMIA, ©Reiki Yoga
 
The Ultimate Guide to Problem Management Dashboards for IT Teams.pdf
byomnex systems
 
Recursive Self Improvement vs Continuous Learning
byBob Marcus
 

Openstack Summit Vancouver 2018 - Multicloud Networking

  • 1.
    Multicloud Networking – ConnectingOpenStack Private Clouds to Public Clouds Shannon McFarland – CCIE #5245 Distinguished Consulting Engineer @eyepv6
  • 2.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Agenda • Multicloud Networking Overview • Extending On-Prem Private Clouds to a Public Cloud • Adding More Public Cloud Providers to the Mix • DMVPN • Amazon Web Services • Google Cloud Platform • Microsoft Azure • Automation • Conclusion 2
  • 3.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Disclaimer • There are a gazillion ways to accomplish the same thing for ALL of this • You can build multicloud connections using software, hardware, commercial and open source gadgets • You or someone you work with needs to know IPsec/IKE, BGP, OSPF, EIGRP and FHRP stuff • Dead Peer Detection • IPsec SA lifetimes • IPsec SA replay window-size • Perfect Forward Secrecy (PFS) • BGP timers, Local Preference, MED, inbound soft reset (check if cloud provider supports dynamic inbound soft reset) • BGP graceful restart - Note: Each cloud provider uses BGP graceful restart with default timers (120 sec) – My configs do not show that due to slide space but know that it is enabled on each on-prem router • IGP timers, tuning • FHRP (HSRP, GLBP, VRRP) timers, tracking 3 router bgp 65002 bgp log-neighbor-changes bgp graceful-restart restart-time 120 bgp graceful-restart stalepath-time 360 bgp graceful-restart
  • 4.
  • 5.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Hybrid vs Multicloud Networking • Hybrid Cloud Networking = Network transport from on-premises (on-prem) to a single public cloud provider • Multicloud Networking = Network transport from on-prem to multiple public cloud providers and/or between multiple public cloud providers • The technologies used can be identical for every connection or they can be per-provider, per-region, per-project, etc.. • Common network transport ingredients for hybrid and multicloud: • Encryption (IPsec/IKEv2/IKEv2, SSL, PKI) • Routing (Static, BGP and with supported public cloud-hosted routers: OSPF, EIGRP) • Tunneling (IPsec tunnel mode, GRE, mGRE, MPLS, segment routing, etc..) • Common network endpoint options: • Native VPN (IPsec over Internet) using public cloud provider services that connect to on-prem router/firewall • Commercial/Open Source VPN platform hosted on the public cloud provider connecting to an on-prem router/firewall • Colocation/Direct Peering: Service from public cloud provider to on-prem via a 3rd party colo facility • Google Cloud Platform Dedicated Internconnect/Direct Peering/Carrier Peering: https://cloud.google.com/interconnect/ • Amazon Web Services Direct Connect/PrivateLink: https://aws.amazon.com/directconnect/ • Microsoft Azure ExpressRoute: https://azure.microsoft.com/en-us/services/expressroute/ 5
  • 6.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Why Would You Use Multiple Cloud Providers? • Cloud provider high availability • M&A may dictate public cloud provider preference (for a time) • Regional cloud provider access • Feature disparity between providers, regions and/or services • Per-project service requirements 6
  • 7.
  • 8.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Options – IPsec-over-the-Internet or Dedicated Connections 8 VPC Network 10.138.0.0/20 IPsec/IKEv2 BGP/OSPF/EIGRP On-Prem Private Cloud Google Cloud VPN Google Cloud Router VPC Network 10.138.0.0/20 BGP/OSPF/EIGRP On-Prem Private Cloud Google Cloud VPN VPC Network 10.138.0.0/20 Google Cloud Router Cloud Partner Interconnect Colocation Facility IPsec VPN + Internet Colocation Commercial/Open Source & Native OpenStack VPNaaS Private Network 192.168.200.0/24 Private Network 192.168.200.0/24
  • 9.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Multicloud Topologies With OpenStack 9 OpenStack VM Neutron Router + VPNaaS VPNaaS Based Multicloud Networking Data Center Infra. TOR(s) Internet Edge Infra. VPN/CoLo Virtual Router Based Multicloud Networking Data Center Infra. OpenStack TOR(s) Internet Edge Infra. VPN/CoLo VM Virtual Router Neutron Router OpenStack VM Neutron Router Hardware Based Multicloud Networking Data Center Infra. TOR(s) Internet Edge Infra. VPN/CoLo *Also, provider networks
  • 10.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Public Cloud Provider - Native VPN Services • Google Cloud Platform (GCP): • VPN: https://cloud.google.com/compute/docs/vpn/overview • Dedicated Interconnect: https://cloud.google.com/interconnect/ • Amazon Web Services (AWS): • VPN: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpn-connections.html • Direct Connect: https://aws.amazon.com/directconnect/ • Microsoft Azure: • VPN: https://docs.microsoft.com/en-us/azure/vpn-gateway/ • ExpressRoute: https://azure.microsoft.com/en-us/services/expressroute/ • OpenStack public cloud goodness: https://www.openstack.org/passport The Big Three 10 Reference
  • 11.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Starting Simple Public Cloud Provider Native IPsec VPN Service 11 VPC Network 10.138.0.0/20 IPsec/IKEv2 BGP/OSPF/EIGRP eBGP<>IGP Redistribution On-Prem Private Cloud Google Cloud VPN Google Cloud Router BGP AS65000 BGP AS65003 Private Network 192.168.200.0/24Cisco ASR1000
  • 12.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Add More On-Prem Stuff Public Cloud Provider Native IPsec VPN Service 12 VPC Network 10.138.0.0/20 BGP AS65000 BGP AS65003 Routes this side should see: 10.138.0.0/20 Private Network 192.168.100.0/24 BGP AS65002 On-Prem Tenant 1 On-Prem Tenant 2Routes this side should see: 192.168.100.0/24 192.168.200.0/24 Google Cloud VPN Google Cloud Router BGP/OSPF/EIGRP BGP/OSPF/EIGRP Private Network 192.168.200.0/24 Cisco ASR1000 Cisco ASR1000
  • 13.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Physical/Virtual Public Cloud Provider Native IPsec VPN Service 13 VPC Network 10.138.0.0/20 Virtual Router Physical Firewall Google Cloud VPN Google Cloud Router CSR 1000v ASA Firewall Private Network 192.168.100.0/24 Private Network 192.168.200.0/24
  • 14.
    Add More Public CloudProviders to the Mix
  • 15.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Stepping into Multicloud Networking Multiple Native IPsec VPN Services 15 Private Network 192.168.200.0/24 VPC Network 10.138.0.0/20 BGP/OSPF/EIGRP On-Prem Private Cloud Google Cloud VPN Google Cloud Router VPC Network 172.31.0.0/16 VPN Gateway VPC Router
  • 16.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Stepping into Multicloud Networking Multiple Native IPsec VPN Services 16 VPC Network 10.138.0.0/20 BGP/OSPF/EIGRP On-Prem Private Cloud Google Cloud VPN Google Cloud Router VPC Network 172.31.0.0/16 VPN Gateway VPC Router As the number of these connections increase and/or change frequently... You can see where this is going Private Network 192.168.200.0/24
  • 17.
    Site-to-Site + ManualConfiguration per Site = Unpleasant Times 17
  • 18.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Example – OpenStack VPNaaS 18 • Lots of sites and lots of variation in policies can lead to lots of human errors • Per-Cloud provider IKE/IPsec compatibility polices required • Automation helps but only with the configuration challenge [root@mc-os-q-aio-sm ~]# openstack vpn ike policy create ikepolicy [root@mc-os-q-aio-sm ~]# openstack vpn ipsec policy create ipsecpolicy [root@mc-os-q-aio-sm ~]# openstack vpn service create vpn > --router a6c58be0-7e32-4a14-b648-8b8178f8de8c [root@mc-os-q-aio-sm ~]# openstack vpn endpoint group create ep_subnet > --type subnet > --value 7fe62bea-49ee-42a0-8c6a-5ec982983e98 [root@mc-os-q-aio-sm ~]# openstack vpn endpoint group create ep_cidr > --type cidr > --value 10.0.1.0/24 [root@mc-os-q-aio-sm ~]# openstack vpn ipsec site connection create GCP-Conn --vpnservice vpn --ikepolicy ikepolicy --ipsecpolicy ipsecpolicy --peer-address 35.xx.xx.xx --peer-id 35.xx.xx.xx --psk demo-secret --local-endpoint-group ep_subnet --peer-endpoint-group ep_cidr Reference
  • 19.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Moving Away From Native VPN Services • Large Site-to-Site designs suck due to configuration complexity (even with Heat or other automation) • If on-prem routers/firewalls are behind NAT – Check for provider support of NAT-T • You need to extend your on-prem IGP (OSPF/EIGRP) into the public cloud • Operational consistency • You need SSL-based VPNs • You need MPLS VPN • QoS, specific network monitoring (IP SLA, NetFlow), Enterprise toolsets for configuration and monitoring What Conditions Cause a Change in Design? 19
  • 20.
  • 21.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public DMVPN – Enable Dynamic Multicloud Networking 21 VNet Network 10.50.0.0/16 DMVPN BGP/OSPF/EIGRP On-Prem Private CloudVPC Network 172.31.0.0/16 Cisco CSR1000v Cisco CSR1000v DMVPN: https://www.cisco.com/c/en/us/products/security/ dynamic-multipoint-vpn-dmvpn/index.html Hub Spoke Spoke Cisco ASR1000 Private Network 192.168.200.0/24
  • 22.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public DMVPN – Enable Dynamic Multicloud Networking 22 VNet Network 10.50.0.0/16 DMVPN FHRP On-Prem Private Cloud VPC Network 172.31.0.0/16 Cisco CSR1000v Cisco CSR1000v IGP Support: OSPF, EIGRP, iBGP QoS Policies IP SLA, NetFlow NAT-T (Transparency) MPLS etc... Hub Spoke Spoke Cisco ASR1000 Private Network 192.168.200.0/24
  • 23.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public DMVPN (Dynamic Multipoint VPN) • DMVPN is a Cisco innovation for building GRE/mGRE + IPsec VPN connections in a dynamic and scalable manner • Cisco DMVPN • https://www.cisco.com/c/en/us/products/security/dynamic-multipoint-vpn-dmvpn/index.html • Cisco IWAN CVD • https://www.cisco.com/c/en/us/solutions/design-zone/networking-design-guides/branch- wan-edge.html • OpenNHRP: • https://sourceforge.net/projects/opennhrp/ • https://wiki.alpinelinux.org/wiki/Dynamic_Multipoint_VPN_(DMVPN) 23
  • 24.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 24 Terminology and Features 192.168.102.0/24 Hub1 Spoke 1 Hub 2 Spoke 2 192.168.101.0/24 192.168.1.0/24 192.168.2.0/24 Tunnel: 10.0.0.101 Physical: 172.16.101.1 Tunnel: 10.0.0.1 Physical: 172.16.1.1 Tunnel: 10.0.0.2 Physical: 172.16.2.1 Overlay Addresses NBMA Address Core Network 192.168.128.0/17 On Demand Spoke Tunnels Tunnel Address Tunnel: 10.0.0.102 Physical: 172.16.102.1 GRE/IPsec Tunnels
  • 25.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public DMVPN Components • Next Hop Resolution Protocol (NHRP) • Creates a distributed (NHRP) mapping database of all the spoke’s tunnel to real (public interface) addresses • Multipoint GRE Tunnel Interface (mGRE) • Single GRE interface to support multiple GRE/IPsec tunnels • Simplifies size and complexity of configuration • IPsec tunnel protection • Dynamically creates and applies encryption policies • Routing • Dynamic advertisement of branch networks; almost all routing protocols (EIGRP, RIP, OSPF, BGP, ODR) are supported 25 Reference
  • 26.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public DMVPN Implementation 26 . Hub and spoke (Phase 1) Spoke-to-spoke (Phase 2) Server Load Balancing Hierarchical (Phase 3) VRF-lite 2547oDMVPN Spoke-to-hub tunnels Spoke-to-spoke tunnels 2547oDMVPN tunnels
  • 27.
    Amazon Web Services –Cisco CSR & DMVPN
  • 28.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public AWS with Cisco CSR 1000v Support • Amazon Web Services Marketplace + Cisco CSR: • https://aws.amazon.com/marketplace/search/results?x=0&y=0&searchTerms=csr&page=1&ref_=nav_search_ box • Cisco CSR for AWS Deployment • DMVPN https://www.cisco.com/c/en/us/td/docs/solutions/Hybrid_Cloud/Intercloud/CSR/AWS/CSRAWS/CSRAWS_3. html • Deployment https://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/aws/b_csraws.html • Cisco Live Session for AWS with Cisco CSR: • https://www.ciscolive.com/global/on-demand-library/?search=brkarc- 2023#/session/1486155288098001AhER • Transit VPC with CSR: http://d2zmdbbm9feqrf.cloudfront.net/2017/usa/pdf/BRKARC-2749.pdf 28
  • 29.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public AWS CSR to On-Prem ASR – DMVPN 29 VPC Network 172.16.2.0/24 VPC Router 192.xxx.xxx.x52.xxx.xxx.x Routes this side should see: 192.168.200.0/24 Routes this side should see: 172.16.2.0/16 Public-side Network 172.16.1.0/24 Cisco CSR1000v DMVPN Hub Tunnel: 10.1.0.2 Spoke Tunnel: 10.1.0.4 OSPF Private Network 192.168.200.0/24 OSPF 10 Area 0 OpenStack VM .30 DataCenter Infra. Provider Networks with VLANs Example Cisco ASR1000
  • 30.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 30 AWS CLI: Create VPC, Subnets and Internet GW Create a new AWS VPC (vpc) # aws ec2 create-vpc --cidr-block 172.16.0.0/16 Create a new subnet in the VPC (this one will be used for the CSR’s ’outside’ interface) # aws ec2 create-subnet --vpc-id vpc-66a0a102 --cidr-block 172.16.1.0/24 Create another new subnet in the VPC (this one will be used for the CSR’s ‘inside’ interface) # aws ec2 create-subnet --vpc-id vpc-66a0a102 --cidr-block 172.16.2.0/24 Create a new AWS Internet Gateway (igw) # aws ec2 create-internet-gateway Attach the Internet gateway to the VPC # aws ec2 attach-internet-gateway --vpc-id vpc-66a0a102 --internet-gateway-id igw-591fba3d Reference
  • 31.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 31 AWS CLI: Create Route Tables Create a new route table in the VPC (rtb) that will be used by the CSR’s ‘outside’ subnet # aws ec2 create-route-table --vpc-id vpc-66a0a102 Create a new default route in the route table and point it to the Internet gateway # aws ec2 create-route --route-table-id rtb-aaa37dcd --destination-cidr-block 0.0.0.0/0 --gateway-id igw-591fba3d Associate the new routable with the ‘outside’ VPC subnet # aws ec2 associate-route-table --subnet-id subnet-0c15b86b --route-table-id rtb-aaa37dcd Create a new route table in the VPC (rtb) that will be used by the CSR’s ‘inside’ subnet # aws ec2 create-route-table --vpc-id vpc-66a0a102 Create a new default route in the route table for the ‘inside’ subnet and point it to the Internet gateway # aws ec2 create-route --route-table-id rtb-3741e750 --destination-cidr-block 0.0.0.0/0 --gateway-id igw-591fba3d Create a new default route in the route table and point it to the Internet gateway # aws ec2 create-route --route-table-id rtb-3741e750 --destination-cidr-block 192.168.200.0/24 --network-interface-id eni-af67db80 Associate the new route table with the ‘inside’ VPC subnet # aws ec2 associate-route-table --subnet-id subnet-c617baa1 --route-table-id rtb-3741e750 Reference
  • 32.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public AWS CLI: Create a Security Group/Rules 32 Create a new security group for the ‘outside’ facing interface (Optional: You can just use an existing group) # aws ec2 create-security-group --group-name csr --description csr-rules --vpc-id vpc-66a0a102 Create a new security group rule for SSH to the CSR # aws ec2 authorize-security-group-ingress --group-id sg-65c39b03 --protocol tcp --port 22 --cidr 0.0.0.0/0 Create a new security group rule for ICMP from the other CSRs (On-Prem and GCP CSR [optional: Just showing the format for your use]) # aws ec2 authorize-security-group-ingress --group-id sg-65c39b03 --ip-permissions '[{"IpProtocol": "icmp", "FromPort": -1, "ToPort": -1, "IpRanges": [{"CidrIp": "192.x.x.x/32"}, {"CidrIp": "35.x.x.x/32"}]}] Create a new security group rule for ESP (IP 50) from the other CSRs # aws ec2 authorize-security-group-ingress --group-id sg-65c39b03 --ip-permissions '[{"IpProtocol": "50", "IpRanges": [{"CidrIp": ”192.x.x.x/32"}, {"CidrIp": "35.x.x.x/32"}]}]' Create a new security group rule for IKE from the other CSRs # aws ec2 authorize-security-group-ingress --group-id sg-65c39b03 --ip-permissions '[{"IpProtocol": "17", "FromPort": 500, "ToPort": 500, "IpRanges": [{"CidrIp": "192.x.x.x/32"}, {"CidrIp": "35.x.x.x/32"}]}]' Create a new security group rule for IKE/NAT-T from the other CSRs # aws ec2 authorize-security-group-ingress --group-id sg-65c39b03 --ip-permissions '[{"IpProtocol": "17", "FromPort": 4500, "ToPort": 4500, "IpRanges": [{"CidrIp": "192.x.x.x/32"}, {"CidrIp": "35.x.x.x/32"}]}]’ Optional: You may want to create a security group just for the ’inside’ subnet that has different rules than the one for the ‘outside’ subnet Create a new security group for the ‘outside’ facing interface (Optional: You can just use an existing group) # aws ec2 authorize-security-group-ingress --group-id sg-84aef7e2 --protocol all --cidr 192.168.200.0/24 Create a new security group for the ‘outside’ facing interface (Optional: You can just use an existing group) # aws ec2 authorize-security-group-ingress --group-id sg-84aef7e2 --protocol all --cidr 172.16.2.0/24 Reference
  • 33.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public AWS CLI: Run a new CSR Instance Using Previous Parameters 33 { "ImageId": "ami-99e5d0f9", "InstanceType": "t2.medium", "KeyName": "mc-aws-key", "NetworkInterfaces": [ { "DeviceIndex": 0, "Description": "Primary network interface", "Groups": [ "sg-65c39b03" ], "PrivateIpAddresses": [ { "Primary": true, "PrivateIpAddress": "172.16.1.10" } ], "SubnetId": "subnet-0c15b86b" }, { "DeviceIndex": 1, "PrivateIpAddresses": [ { "Primary": true, "PrivateIpAddress": "172.16.2.10" } ], "SubnetId": "subnet-c617baa1" } ] } csr-create.json Create a CSR instance using the JSON file shown to the left # aws ec2 run-instances --cli-input-json file://csr-create.json Create a tag/name and associate it with the CSR (Optional) # aws ec2 create-tags --resources i-0f2a0ee857e9c2540 --tags Key=Name,Value=csr-aws-01 Create a new External IP (EIP) allocation (or use an existing one) # aws ec2 allocate-address eipalloc-ab35cb96 vpc 52.xxx.xxx.x Associate the EIP with the ’outside’ interface of the CSR (GigabitEthernet 1) # aws ec2 associate-address --allocation-id eipalloc-ab35cb96 --network-interface-id eni-dd5bd6f2 Modify the ’inside’ subnet to disable source/destination checking # aws ec2 modify-network-interface-attribute --network-interface-id eni-af67db80 --source-dest-check "{"Value": false}" A note about NAT: If you plan to use the CSR for NAT operation, you must disable source/destination checking on the outside CSR interface/subnet http://docs.aws.amazon.com/AmazonVPC/latest/UserG uide/VPC_NAT_Instance.html#EIP_Disable_SrcDestCh eck Reference
  • 34.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 34 Connect to the AWS CSR – Enable Interfaces # ssh -i "mc-aws-key.pem" ec2-user@52.xxx.xxx.x csr-aws-01#configure terminal Enter configuration commands, one per line. End with CNTL/Z. csr-aws-01(config)#interface gigabitEthernet 2 csr-aws-01(config-if)#ip address dhcp csr-aws-01(config-if)#no shutdown Connect to the new AWS-hosted CSR and enable the GigabitEthernet 2 interface for DHCP csr-aws-01#show ip interface brief Interface IP-Address OK? Method Status Protocol GigabitEthernet1 172.16.1.10 YES DHCP up up GigabitEthernet2 172.16.2.10 YES DHCP up up VirtualPortGroup0 192.168.35.1 YES TFTP up up Wait a few seconds and check to make sure that both interfaces on the CSR are up and have the correct IP addresses: Note: This can all be automated (along with the DMVPN configs) by creating AWS CloudFormation templates Reference
  • 35.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public#CLUS AWS Cisco CSR DMVPN Config Spoke 35BRKCLD-3440 crypto ikev2 proposal AES/GCM/256 encryption aes-gcm-256 prf sha512 group 19 ! crypto ikev2 policy AES/GCM/256 match fvrf any proposal AES/GCM/256 ! crypto ikev2 keyring DMVPN-KEYRING peer ANY address 0.0.0.0 0.0.0.0 pre-shared-key <PSK_PASSWORD_GOES_HERE> ! crypto ikev2 profile DMVPN-IKEv2-PROFILE description PSK Profile match identity remote address 0.0.0.0 identity local address 52.xxx.xxx.x authentication remote pre-share authentication local pre-share keyring local DMVPN-KEYRING dpd 40 5 on-demand ! crypto ipsec security-association replay window-size 1024 ! crypto ipsec transform-set AES256/GCM/TRANSFORM esp-gcm 256 mode transport ! crypto ipsec profile DMVPN-IPSEC-PROFILE set transform-set AES256/GCM/TRANSFORM set ikev2-profile DMVPN-IKEv2-PROFILE ... Output summarized interface Tunnel0 description DMVPN ip address 10.1.0.4 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication <NHRP_PASSWORD> ip nhrp network-id 100 ip nhrp nhs 10.1.0.2 nbma 192.xxx.xxx.x multicast ip tcp adjust-mss 1360 ip ospf authentication-key 7 <OSPF_PASSWORD> ip ospf network point-to-multipoint ip ospf hello-interval 10 tunnel source GigabitEthernet1 tunnel mode gre multipoint tunnel key 100 tunnel protection ipsec profile DMVPN-IPSEC-PROFILE ! interface GigabitEthernet1 description Internet ip address dhcp no ip redirects no ip unreachables no ip proxy-arp negotiation auto ! router ospf 10 router-id 10.1.0.4 network 172.16.2.0 0.0.0.255 area 2 network 10.1.0.0 0.0.0.255 area 0 ! ip route 0.0.0.0 0.0.0.0 172.16.1.1 Reference
  • 36.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public#CLUS On-Prem Cisco ASR DMVPN Config Hub – Nothing ever changes on the hub for each example 36BRKCLD-3440 crypto ikev2 proposal AES/GCM/256 encryption aes-gcm-256 prf sha512 group 19 ! crypto ikev2 policy AES/GCM/256 match fvrf any proposal AES/GCM/256 ! crypto ikev2 keyring DMVPN-KEYRING peer ANY address 0.0.0.0 0.0.0.0 pre-shared-key <PSK_PASSWORD_GOES_HERE> ! crypto ikev2 profile DMVPN-IKEv2-PROFILE description PSK Profile match identity remote address 0.0.0.0 identity local address 192.xxx.xxx.x authentication remote pre-share authentication local pre-share keyring local DMVPN-KEYRING dpd 40 5 on-demand ! crypto ipsec security-association replay window-size 1024 ! crypto ipsec transform-set AES256/GCM/TRANSFORM esp-gcm 256 mode transport ! crypto ipsec profile DMVPN-IPSEC-PROFILE set transform-set AES256/GCM/TRANSFORM set ikev2-profile DMVPN-IKEv2-PROFILE ... Output summarized interface Tunnel0 description DMVPN ip address 10.1.0.2 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication <NHRP_PASSWORD> ip nhrp map multicast dynamic ip nhrp network-id 100 ip nhrp redirect ip tcp adjust-mss 1360 ip ospf authentication-key 7 <OSPF_PASSWORD> ip ospf network point-to-multipoint ip ospf hello-interval 10 tunnel source GigabitEthernet0/0/0 tunnel mode gre multipoint tunnel key 100 tunnel protection ipsec profile DMVPN-IPSEC-PROFILE ! interface GigabitEthernet0/0/0 description Internet ip address 192.xxx.xxx.x 255.255.255.0 no ip redirects no ip unreachables no ip proxy-arp negotiation auto ! router ospf 10 router-id 10.1.0.2 network 10.1.0.0 0.0.0.255 area 0 network 192.168.200.0 0.0.0.255 area 0 ! ip route 0.0.0.0 0.0.0.0 192.xxx.xxx.x Reference
  • 37.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 37 Verify Routing and Reachability ... Output summarized [ec2-user@ip-172-16-2-192 ~]$ ping 192.168.200.30 PING 192.168.200.30 (192.168.200.30) 56(84) bytes of data. 64 bytes from 192.168.200.30: icmp_seq=1 ttl=62 time=2.75 ms 64 bytes from 192.168.200.30: icmp_seq=2 ttl=62 time=2.93 ms 64 bytes from 192.168.200.30: icmp_seq=3 ttl=62 time=2.75 ms Connect to an AWS instances and ping to the on-prem private network asr-mc-01#show ip route | i 172.16.2.0 O IA 172.16.2.0 [110/1001] via 10.1.0.4, 00:11:41, Tunnel0 On the on-prem ASR check the route for the AWS VPC network 172.16.2.0/24 csr-aws-01#show ip route | i 192.168.200.0 O 192.168.200.0/24 [110/1001] via 10.1.0.2, 6d17h, Tunnel0 On AWS check for the route for the on-prem network (192.168.200.0/24) VPC Network 172.16.2.0/24 .10.192 OSPF Hub Tunnel: 10.1.0.2 Spoke Tunnel: 10.1.0.4 Cisco CSR1000v Reference Private Network 192.168.200.0/24 OSPF 10 Area 0 VM .30 DataCenter Infra. Provider Networks with VLANs Example Cisco ASR1000
  • 38.
  • 39.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 39 Private Network 192.168.200.0/24inside-network 10.0.1.0/24 OSPF 10 Area 0 Routes this side should see: 10.0.1.0/24 Routes this side should see: 192.168.200.0/24 Cisco CSR1000v OpenStack Default Network 10.138.0.0/20 .100.3 .2 .1 VM .30192.xxx.xxx.x35.xxx.xxx.x DMVPN Hub Tunnel: 10.1.0.2 Spoke Tunnel: 10.1.0.1 GCP CSR to On-Prem ASR – DMVPN Coming in 16.9.1 Release OSPF Compute Engine 2 1 DataCenter Infra. Provider Networks with VLANs Example Cisco ASR1000
  • 40.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 40 gcloud – Create the GCP External IP, Inside VPC Network & Route Create a new external IP reservation that will be used for the GCP CSR NATed connection (or use an existing one) # gcloud compute addresses create csr-to-csr-ext-ip --region us-west1 Capture the external IP address # gcloud compute addresses list --filter="csr-to-csr-ext-ip" NAME REGION ADDRESS STATUS csr-to-csr-ext-ip us-west1 35.xxx.xxx.x RESERVED Create a new GCP inside network that will be attached to the ‘inside’ interface of the CSR # gcloud compute networks create inside-network --subnet-mode=custom Create a new GCP inside subnet - Associate it with the inside network # gcloud compute networks subnets create inside-subnet --network=inside-network --range=10.0.1.0/24 Create a new GCP route from the CSR inside network to the on-prem private network which routes through the IPsec VPN # gcloud compute routes create inside-to-csr-private --network=inside-network --destination-range=192.168.200.0/24 --next-hop-address=10.0.1.2 Coming in 16.9.1 Release Reference
  • 41.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 41 gcloud – Create GCP Firewall Rules Create a new GCP firewall rule to allow traffic into the inside CSR network from the default network # gcloud compute firewall-rules create allow-default-to-csr-inside --direction=INGRESS --network=inside-network --action=ALLOW --rules=all --source-ranges=0.0.0.0/0 Create a new GCP firewall rule to allow traffic between the default network and the on-prem ASR public IP for IKE, IPsec # gcloud compute firewall-rules create csr-csr-vpn --direction=INGRESS --network=default --action=ALLOW --rules=udp:500,udp:4500,esp --source-ranges=192.xxx.xxx.x Coming in 16.9.1 Release Reference
  • 42.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 42 gcloud – Create CSR and Test Instances Create a new GCE CSR instance and set fixed IPv4 addresses to each of the two interfaces # gcloud compute instances create "csr-gcp-01" --zone "us-west1-a" --machine-type "n1-standard-4" --network-interface subnet="default",private-network-ip="10.138.0.100",address="35.xxx.xxx.x" --can-ip-forward --network-interface subnet="inside-subnet",private-network-ip="10.0.1.2",no-address --image ”name_of_csr_image" --boot-disk-size "10" --boot-disk-type "pd-standard" --boot-disk-device-name "csr-gcp-01" Create a new GCE test instance that will be used to validate the VPN and routing # gcloud compute instances create "csr-inside-vm" --zone "us-west1-a" --machine-type "g1-small" --subnet "inside-subnet" --private-network-ip "10.0.1.3" --image "debian-9-stretch-v20170918" --image-project "debian-cloud" --boot-disk-size "10" --boot-disk-type "pd-standard" --boot-disk-device-name "csr-inside-vm" Coming in 16.9.1 Release Reference
  • 43.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Connect to the GCP CSR – Enable Interfaces 43 # gcloud compute ssh cisco-user@csr-gcp-01 csr1kv-gcp#configure terminal Enter configuration commands, one per line. End with CNTL/Z. csr1kv-gcp(config)#interface gigabitEthernet 2 csr1kv-gcp(config-if)#ip address dhcp csr1kv-gcp(config-if)#no shutdown ... Output summarized Connect to the new GCP-hosted CSR and enable the GigabitEthernet 2 interface for DHCP csr1kv-gcp#show ip interface brief Interface IP-Address OK? Method Status Protocol GigabitEthernet1 10.138.0.100 YES TFTP up up GigabitEthernet2 10.0.1.2 YES DHCP up up Wait a few seconds and check to make sure that both interfaces on the CSR are up and with the correct IP addresses: Coming in 16.9.1 Release Reference
  • 44.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public GCP Cisco CSR DMVPN Config Spoke 44 crypto ikev2 proposal AES/GCM/256 encryption aes-gcm-256 prf sha512 group 19 ! crypto ikev2 policy AES/GCM/256 match fvrf any proposal AES/GCM/256 ! crypto ikev2 keyring DMVPN-KEYRING peer ANY address 0.0.0.0 0.0.0.0 pre-shared-key <PSK_PASSWORD_GOES_HERE> ! crypto ikev2 profile DMVPN-IKEv2-PROFILE description PSK Profile match identity remote address 0.0.0.0 identity local address 35.xxx.xxx.x authentication remote pre-share authentication local pre-share keyring local DMVPN-KEYRING dpd 40 5 on-demand ! crypto ipsec security-association replay window-size 1024 ! crypto ipsec transform-set AES256/GCM/TRANSFORM esp-gcm 256 mode transport ! crypto ipsec profile DMVPN-IPSEC-PROFILE set transform-set AES256/GCM/TRANSFORM set ikev2-profile DMVPN-IKEv2-PROFILE ... Output summarized interface Tunnel0 description DMVPN ip address 10.1.0.1 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication <NHRP_PASSWORD> ip nhrp network-id 100 ip nhrp nhs 10.1.0.2 nbma 192.xxx.xxx.x multicast ip tcp adjust-mss 1360 ip ospf authentication-key 7 <OSPF_PASSWORD> ip ospf network point-to-multipoint ip ospf hello-interval 10 tunnel source GigabitEthernet1 tunnel mode gre multipoint tunnel key 100 tunnel protection ipsec profile DMVPN-IPSEC-PROFILE ! interface GigabitEthernet1 description Internet ip address 10.138.0.100 255.255.255.0 no ip redirects no ip unreachables no ip proxy-arp negotiation auto ! router ospf 10 router-id 10.1.0.1 network 10.0.1.0 0.0.0.255 area 1 network 10.1.0.0 0.0.0.255 area 0 ! ip route 0.0.0.0 0.0.0.0 138.0.0.1 Coming in 16.9.1 Release Reference
  • 45.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public On-Prem Cisco ASR DMVPN Config Hub 45 crypto ikev2 proposal AES/GCM/256 encryption aes-gcm-256 prf sha512 group 19 ! crypto ikev2 policy AES/GCM/256 match fvrf any proposal AES/GCM/256 ! crypto ikev2 keyring DMVPN-KEYRING peer ANY address 0.0.0.0 0.0.0.0 pre-shared-key <PSK_PASSWORD_GOES_HERE> ! crypto ikev2 profile DMVPN-IKEv2-PROFILE description PSK Profile match identity remote address 0.0.0.0 identity local address 192.xxx.xxx.x authentication remote pre-share authentication local pre-share keyring local DMVPN-KEYRING dpd 40 5 on-demand ! crypto ipsec security-association replay window-size 1024 ! crypto ipsec transform-set AES256/GCM/TRANSFORM esp-gcm 256 mode transport ! crypto ipsec profile DMVPN-IPSEC-PROFILE set transform-set AES256/GCM/TRANSFORM set ikev2-profile DMVPN-IKEv2-PROFILE ... Output summarized interface Tunnel0 description DMVPN ip address 10.1.0.2 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication <NHRP_PASSWORD> ip nhrp map multicast dynamic ip nhrp network-id 100 ip nhrp redirect ip tcp adjust-mss 1360 ip ospf authentication-key 7 <OSPF_PASSWORD> ip ospf network point-to-multipoint ip ospf hello-interval 10 tunnel source GigabitEthernet0/0/0 tunnel mode gre multipoint tunnel key 100 tunnel protection ipsec profile DMVPN-IPSEC-PROFILE ! interface GigabitEthernet0/0/0 description Internet ip address 192.xxx.xxx.x 255.255.255.0 no ip redirects no ip unreachables no ip proxy-arp negotiation auto ! router ospf 10 router-id 10.1.0.2 network 10.1.0.0 0.0.0.255 area 0 network 192.168.200.0 0.0.0.255 area 0 ! ip route 0.0.0.0 0.0.0.0 192.xxx.xxx.x Reference
  • 46.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Verify Routing and Reachability 46 ... Output summarized # gcloud compute ssh "csr-inside-vm“ shmcfarl@csr-inside-vm:~$ ping 192.168.200.30 PING 192.168.200.30 (192.168.200.30) 56(84) bytes of data. 64 bytes from 192.168.200.30: icmp_seq=1 ttl=62 time=22.1 ms 64 bytes from 192.168.200.30: icmp_seq=2 ttl=62 time=23.3 ms 64 bytes from 192.168.200.30: icmp_seq=3 ttl=62 time=23.6 ms Connect to the GCP test instance that was created earlier and ping to the on-prem private network csr1kv-gcp#show ip route | i 192.168.200.0 . . . O 192.168.200.0/24 [110/1001] via 10.1.0.2, 00:09:51, Tunnel0 On the GCP CSR, check for the private network route from the on-prem side(192.168.200.0/24) asr-mc-01#show ip route | i 10.0.1.0 . . . O IA 10.0.1.0/24 [110/1001] via 10.1.0.1, 00:40:08, Tunnel0 On the on-prem ASR, check for the VPC inside network route (10.1.0/24) csr1kv-gcp#show ip nhrp 10.1.0.2/32 via 10.1.0.2 Tunnel0 created 5d14h, never expire Type: static, Flags: NBMA address: 192.xxx.xxx.x Check the DMVPN Next-Hop Resolution Protocol (NHRP) Status asr-mc-01#show ip nhrp 10.1.0.1/32 via 10.1.0.1 Tunnel0 created 00:40:25, expire 00:08:20 Type: dynamic, Flags: registered used nhop NBMA address: 35.xxx.xxx.x (Claimed NBMA address: 10.138.0.100) Coming in 16.9.1 Release Reference
  • 47.
  • 48.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 48 Azure CSR to On-Prem ASR – DMVPN 192.xxx.xxx.x40.xxx.xxx.x Routes this side should see: 192.168.200.0/24 Routes this side should see: 10.10.1.0/24 Cisco CSR1000v DMVPN Hub Tunnel: 10.1.0.2 Spoke Tunnel: 10.1.0.6 OSPF Inside Subnet 10.10.1.0/24 Outside Subnet 10.10.0.0/24 Cisco ASR1000 Private Network 192.168.200.0/24 OSPF 10 Area 0 OpenStack VM .30 DataCenter Infra. Provider Networks with VLANs Example
  • 49.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Microsoft Azure with Cisco CSR 1000v • Microsoft Azure Marketplace • https://azuremarketplace.microsoft.com/en- us/marketplace/apps/cisco.cisco-csr-basic-template • https://github.com/Azure/azure-quickstart-templates/tree/master/cisco- csr-1000v • Cisco CSR 1000v with Azure Deployment • https://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/azu/b_c sr1000config-azure.html 49
  • 50.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 50 Azure CLI: Create Resource Group, Networks, Subnets Create a new Azure Resource Group (rg) # az group create --name multicloud-rg --location westus Create a new public (external IP) IPv4 address to be used for the CSR’s ‘outside’ interface # az network public-ip create --resource-group multicloud-rg --name csr-azure-01-eip --allocation-method static Create a new virtual network (vnet) and a subnet to be used for the CSR’s ‘outside interface # az network vnet create --resource-group multicloud-rg --name mc-csr-vnet --address-prefix 10.10.0.0/16 --subnet-name csr-outside --subnet-prefix 10.10.0.0/24 Create a new subnet for the CSR’s ‘inside’ interface and associate it with the vnet created above # az network vnet subnet create --resource-group multicloud-rg --vnet-name mc-csr-vnet --name csr-inside --address-prefix 10.10.1.0/24 Reference
  • 51.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public#CLUS 51BRKCLD-3440 Azure CLI: Create Route Tables Create a new route table (rt) that will be used for the CSR’s ’outside’ subnet # az network route-table create --resource-group multicloud-rg --name csr-outside-rt Create a new route table that will used for the CSR’s ‘inside’ subnet # az network route-table create --resource-group multicloud-rg --name csr-inside-rt Create a new route table entry for the ‘inside’ subnet to reach the on-prem network (192.168.200.0) via the CSR’s IP (10.10.1.4) # az network route-table route create --resource-group multicloud-rg --name csr-to-on-prem-route --route-table-name csr-inside-rt --address-prefix 192.168.200.0/24 --next-hop-type VirtualAppliance --next-hop-ip-address 10.10.1.4 Associate the ‘outside’ route table with the ‘outside’ subnet # az network vnet subnet update --resource-group multicloud-rg --vnet-name mc-csr-vnet --name csr-outside --route-table csr-outside-rt Associate the ‘inside’ route table with the ‘inside’ subnet # az network vnet subnet update --resource-group multicloud-rg --vnet-name mc-csr-vnet --name csr-inside --route-table csr-inside-rt Reference
  • 52.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public#CLUS 52BRKCLD-3440 Azure CLI: Create Network Security Group (NSG) Create a new Network Security Group (NSG) to be used for the ‘outside’ CSR interface # az network nsg create --resource-group multicloud-rg --name csr-nsg-outside Create a new NSG rule to allow inbound SSH access to the CSR (can make it specific to an IP/Prefix) # az network nsg rule create --resource-group multicloud-rg --nsg-name csr-nsg-outside --name SSHRule --priority 100 --source-address-prefixes 'Internet' --source-port-ranges '*' --destination-address-prefixes '*' --destination-port-ranges 22 --access Allow --protocol Tcp --direction inbound Create a new NSG rule to allow inbound UDP 500 (IKE) traffic to the CSR (can make it specific to an IP/Prefix) # az network nsg rule create --resource-group multicloud-rg --nsg-name csr-nsg-outside --name UDP-500 --priority 101 --source-address-prefixes 'Internet' --source-port-ranges '*' --destination-address-prefixes '*' --destination-port-ranges 500 --access Allow --protocol Udp --direction inbound Reference
  • 53.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public#CLUS 53BRKCLD-3440 Azure CLI: Create NSG Rule & NICs Create a new Network Security Group (NSG) to be used for the ‘outside’ CSR interface # az network nsg rule create --resource-group multicloud-rg --nsg-name csr-nsg-outside --name UDP-4500 --priority 102 --source-address-prefixes 'Internet' --source-port-ranges '*' --destination-address-prefixes '*' --destination-port-ranges 4500 --access Allow --protocol Udp --direction inbound Create a new NIC to be used by the CSR’s ‘outside’ interface. Associate the NIC with the NSG, Subnet, Public IP & enable forwarding # az network nic create --resource-group multicloud-rg --name csr-nic-g1 --vnet-name mc-csr-vnet --subnet csr-outside --network-security-group csr-nsg-outside --ip-forwarding true --public-ip-address csr-azure-01-eip Create a new NIC to be used by the CSR’s ‘inside’ interface. Associate the NIC with the NSG, Subnet and enable forwarding # az network nic create --resource-group multicloud-rg --name csr-nic-g2 --vnet-name mc-csr-vnet --subnet csr-inside --ip-forwarding true Reference
  • 54.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 54 Azure CLI: Run a new CSR Instance Using Previous Parameters Create a CSR VM using a Azure Marketplace image. Associate the VM with two the NICs created earlier. # Note: The VM can be created with a large number of options to include SSH keys, image (BYOL, # of NICs), and size # az vm create --resource-group multicloud-rg --name csr-azure-01 --admin-username csr-azure --admin-password <PASSWORD> --authentication-type password --image cisco:cisco-csr-1000v:16_6:16.6.120170804 <<< Change image based on current release --nics csr-nic-g1 csr-nic-g2 --size Standard_D2_v2
  • 55.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 55 Connect to the Azure CSR – Enable Interfaces # ssh csr-azure@40.xxx.xxx.x csr-azure-01#configure terminal Enter configuration commands, one per line. End with CNTL/Z. csr-azure-01(config)#interface gigabitEthernet 2 csr-azure-01(config-if)#ip address dhcp csr-azure-01(config-if)#no shutdown Connect to the new Azure-hosted CSR and enable the GigabitEthernet 2 interface for DHCP csr-azure-01#show ip interface brief Interface IP-Address OK? Method Status Protocol GigabitEthernet1 10.10.0.4 YES DHCP up up GigabitEthernet2 10.10.1.4 YES DHCP up up VirtualPortGroup0 192.168.35.1 YES TFTP up up Wait a few seconds and check to make sure that both interfaces on the CSR are up and have the correct IP addresses: Note: This can all be automated (along with the DMVPN configs) by creating Azure Automation/Resource Manager Reference
  • 56.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public#CLUS Azure Cisco CSR DMVPN Config Spoke crypto ikev2 proposal AES/GCM/256 encryption aes-gcm-256 prf sha512 group 19 ! crypto ikev2 policy AES/GCM/256 match fvrf any proposal AES/GCM/256 ! crypto ikev2 keyring DMVPN-KEYRING peer ANY address 0.0.0.0 0.0.0.0 pre-shared-key <PSK_PASSWORD_GOES_HERE> ! crypto ikev2 profile DMVPN-IKEv2-PROFILE description PSK Profile match identity remote address 0.0.0.0 identity local address 40.xxx.xxx.x authentication remote pre-share authentication local pre-share keyring local DMVPN-KEYRING dpd 40 5 on-demand ! crypto ipsec security-association replay window-size 1024 ! crypto ipsec transform-set AES256/GCM/TRANSFORM esp-gcm 256 mode transport ! crypto ipsec profile DMVPN-IPSEC-PROFILE set transform-set AES256/GCM/TRANSFORM set ikev2-profile DMVPN-IKEv2-PROFILE ... Output summarized interface Tunnel0 description DMVPN ip address 10.1.0.6 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication <NHRP_PASSWORD> ip nhrp network-id 100 ip nhrp nhs 10.1.0.2 nbma 192.xxx.xxx.x multicast ip tcp adjust-mss 1360 ip ospf authentication-key 7 <OSPF_PASSWORD> ip ospf network point-to-multipoint ip ospf hello-interval 10 tunnel source GigabitEthernet1 tunnel mode gre multipoint tunnel key 100 tunnel protection ipsec profile DMVPN-IPSEC-PROFILE ! interface GigabitEthernet1 description Internet ip address dhcp no ip redirects no ip unreachables no ip proxy-arp negotiation auto ! router ospf 10 router-id 10.1.0.6 network 10.1.0.0 0.0.0.255 area 0 network 10.10.1.0 0.0.0.255 area 3 ! ip route 0.0.0.0 0.0.0.0 10.10.0.1
  • 57.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public#CLUS 57BRKCLD-3440 On-Prem Cisco ASR DMVPN Config Hub - Nothing ever changes on the hub for each example crypto ikev2 proposal AES/GCM/256 encryption aes-gcm-256 prf sha512 group 19 ! crypto ikev2 policy AES/GCM/256 match fvrf any proposal AES/GCM/256 ! crypto ikev2 keyring DMVPN-KEYRING peer ANY address 0.0.0.0 0.0.0.0 pre-shared-key <PSK_PASSWORD_GOES_HERE> ! crypto ikev2 profile DMVPN-IKEv2-PROFILE description PSK Profile match identity remote address 0.0.0.0 identity local address 192.xxx.xxx.x authentication remote pre-share authentication local pre-share keyring local DMVPN-KEYRING dpd 40 5 on-demand ! crypto ipsec security-association replay window-size 1024 ! crypto ipsec transform-set AES256/GCM/TRANSFORM esp-gcm 256 mode transport ! crypto ipsec profile DMVPN-IPSEC-PROFILE set transform-set AES256/GCM/TRANSFORM set ikev2-profile DMVPN-IKEv2-PROFILE ... Output summarized interface Tunnel0 description DMVPN ip address 10.1.0.2 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication <NHRP_PASSWORD> ip nhrp map multicast dynamic ip nhrp network-id 100 ip nhrp redirect ip tcp adjust-mss 1360 ip ospf authentication-key 7 <OSPF_PASSWORD> ip ospf network point-to-multipoint ip ospf hello-interval 10 tunnel source GigabitEthernet0/0/0 tunnel mode gre multipoint tunnel key 100 tunnel protection ipsec profile DMVPN-IPSEC-PROFILE ! interface GigabitEthernet0/0/0 description Internet ip address 192.xxx.xxx.x 255.255.255.0 no ip redirects no ip unreachables no ip proxy-arp negotiation auto ! router ospf 10 router-id 10.1.0.2 network 10.1.0.0 0.0.0.255 area 0 network 192.168.200.0 0.0.0.255 area 0 ! ip route 0.0.0.0 0.0.0.0 192.xxx.xxx.x
  • 58.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 58 Verify Routing and Reachability ... Output summarized shmcfarl@AzTestVm:~$ping 192.168.200.30 PING 192.168.200.30 (192.168.200.30) 56(84) bytes of data. 64 bytes from 192.168.200.30: icmp_seq=2 ttl=62 time=3.99 ms 64 bytes from 192.168.200.30: icmp_seq=3 ttl=62 time=6.44 ms Connect to an AWS instances and ping to the on-prem private network asr-mc-01#show ip route | i 10.10.1.0 O IA 10.10.1.0/24 [110/1001] via 10.1.0.6, 00:19:15, Tunnel0 On the on-prem CSR check the route for the AWS VPC network 172.16.2.0/24 csr-azure-01#show ip route | i 192.168.200.0 O 192.168.200.0/24 [110/1001] via 10.1.0.2, 00:17:57, Tunnel0 On AWS check for the route for the on-prem network (192.168.200.0/24) Inside Subnet 10.10.1.0/24 .4.5 OSPF Hub Tunnel: 10.1.0.2 Spoke Tunnel: 10.1.0.6 Cisco CSR1000v Reference Cisco ASR1000 Private Network 192.168.200.0/24 OSPF 10 Area 0 VM .30 DataCenter Infra. Provider Networks with VLANs Example
  • 59.
  • 60.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public DMVPN – Enable Dynamic Multicloud Networking 60 VNet Network 10.10.1.0/24 DMVPN VPC Network 172.16.2.0/24 Cisco CSR1000v Cisco CSR1000v VPC Network 10.0.1.0/24 Cisco CSR1000v Cisco ASR1000 Hub Tunnel: 10.1.0.2 Spoke Tunnel: 10.1.0.1 Spoke Tunnel: 10.1.0.4 Spoke Tunnel: 10.1.0.6 Private Network 192.168.200.0/24 OSPF 10 Area 0 OpenStack VM .30 DataCenter Infra. Provider Networks with VLANs Example
  • 61.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public#CLUS BRKCLD-3440 General Guidelines for DMVPN Between Clouds • Set the VPC routes for each site • Set the firewall/security groups/network security groups for each site/protocol gcloud compute routes create inside-to-aws --network=csr-inside-network --destination-range=172.16.2.0/24 --next-hop-address=10.0.1.2 gcloud compute routes create inside-to-azure --network=csr-inside-network --destination-range=10.10.1.0/24 --next-hop-address=10.0.1.2 Create a very specific per-site route (AWS example allowing UDP500 from each cloud provider public IP) aws ec2 authorize-security-group-ingress --group-id sg-65c39b03 --ip-permissions '[{"IpProtocol": "17", "FromPort": 500, "ToPort": 500, "IpRanges": [{"CidrIp": "192.x.x.x/32"}, {"CidrIp": "35.x.x.x/32"}, {"CidrIp": "40.x.x.x/32"}]}]’ Alternatively, you can open it up (Azure example) az network nsg rule create --resource-group multicloud-rg --nsg-name csr-nsg-outside --name UDP-4500 --priority 102 --source-address-prefixes 'Internet' --source-port-ranges '*' --destination-address-prefixes '*' --destination-port-ranges 4500 --access Allow --protocol Udp --direction inbound
  • 62.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Routing Example – All Sites 62 • For spoke-to-spoke direct routing with DMVPN/NHRP: • ‘ip nhrp redirect’ on the hubs • ‘ip nhrp shortcut’ on the spokes asr-mc-01#show ip route ospf 10.0.0.0/8 is variably subnetted, 7 subnets, 2 masks O IA 10.0.1.0/24 [110/1001] via 10.1.0.1, 02:40:45, Tunnel0 O 10.1.0.1/32 [110/1000] via 10.1.0.1, 02:40:45, Tunnel0 O 10.1.0.4/32 [110/1000] via 10.1.0.4, 01:18:49, Tunnel0 O 10.1.0.6/32 [110/1000] via 10.1.0.6, 00:56:19, Tunnel0 O IA 10.10.1.0/24 [110/1001] via 10.1.0.6, 00:55:34, Tunnel0 172.16.0.0/24 is subnetted, 1 subnets O IA 172.16.2.0 [110/1001] via 10.1.0.4, 01:18:49, Tunnel0 ... Output summarized Hub On-Prem CSR csr1kv-gcp#show ip route ospf 10.0.0.0/8 is variably subnetted, 10 subnets, 2 masks O 10.1.0.2/32 [110/1000] via 10.1.0.2, 02:43:14, Tunnel0 O % 10.1.0.4/32 [110/2000] via 10.1.0.2, 01:21:14, Tunnel0 O % 10.1.0.6/32 [110/2000] via 10.1.0.2, 00:58:47, Tunnel0 O IA% 10.10.1.0/24 [110/2001] via 10.1.0.2, 00:58:00, Tunnel0 172.16.0.0/24 is subnetted, 1 subnets O IA% 172.16.2.0 [110/2001] via 10.1.0.2, 01:21:14, Tunnel0 O 192.168.200.0/24 [110/1001] via 10.1.0.2, 02:43:14, Tunnel0 Spoke – Google Cloud Platform CSR csr-aws-01#show ip route ospf 10.0.0.0/8 is variably subnetted, 7 subnets, 2 masks O IA% 10.0.1.0/24 [110/2001] via 10.1.0.2, 01:21:32, Tunnel0 O % 10.1.0.1/32 [110/2000] via 10.1.0.2, 01:21:32, Tunnel0 O 10.1.0.2/32 [110/1000] via 10.1.0.2, 01:21:32, Tunnel0 O % 10.1.0.6/32 [110/2000] via 10.1.0.2, 00:59:01, Tunnel0 O IA% 10.10.1.0/24 [110/2001] via 10.1.0.2, 00:58:14, Tunnel0 O 192.168.200.0/24 [110/1001] via 10.1.0.2, 01:21:32, Tunnel0 Spoke – Amazon Web Services CSR csr-azure-01#show ip route ospf 10.0.0.0/8 is variably subnetted, 10 subnets, 2 masks O IA% 10.0.1.0/24 [110/2001] via 10.1.0.2, 00:58:44, Tunnel0 O % 10.1.0.1/32 [110/2000] via 10.1.0.2, 00:58:44, Tunnel0 O 10.1.0.2/32 [110/1000] via 10.1.0.2, 00:58:44, Tunnel0 O % 10.1.0.4/32 [110/2000] via 10.1.0.2, 00:58:44, Tunnel0 172.16.0.0/24 is subnetted, 1 subnets O IA% 172.16.2.0 [110/2001] via 10.1.0.2, 00:58:44, Tunnel0 O 192.168.200.0/24 [110/1001] via 10.1.0.2, 00:58:44, Tunnel0 Spoke – Azure CSR IA - OSPF inter area % - next hop override
  • 63.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public#CLUS BRKCLD-3440 NHRP Example – Hub/Spoke asr-mc-01#show ip nhrp 10.1.0.1/32 via 10.1.0.1 Tunnel0 created 02:02:42, expire 00:08:17 Type: dynamic, Flags: registered used nhop NBMA address: 35.xxx.xxx.x (Claimed NBMA address: 10.138.0.100) 10.1.0.4/32 via 10.1.0.4 Tunnel0 created 00:42:52, expire 00:09:17 Type: dynamic, Flags: registered used nhop NBMA address: 52.xxx.xxx.x (Claimed NBMA address: 172.16.1.10) 10.1.0.6/32 via 10.1.0.6 Tunnel0 created 00:18:12, expire 00:08:26 Type: dynamic, Flags: registered used nhop NBMA address: 40.xxx.xxx.x (Claimed NBMA address: 10.10.0.4) asr-mc-01#show ip nhrp multicast I/F NBMA address Tunnel0 35.xxx.xxx.x Flags: dynamic (Enabled) Tunnel0 52.xxx.xxx.x Flags: dynamic (Enabled) Tunnel0 40.xxx.xxx.x Flags: dynamic (Enabled) Hub On-Prem ASR csr-azure-01#show ip nhrp 10.0.1.0/24 via 10.1.0.1 Tunnel0 created 00:06:26, expire 00:03:32 Type: dynamic, Flags: router rib nho NBMA address: 35.xxx.xxx.x (Claimed NBMA address: 10.138.0.100) 10.1.0.1/32 via 10.1.0.1 Tunnel0 created 00:06:26, expire 00:03:32 Type: dynamic, Flags: router nhop rib nho NBMA address: 35.xxx.xxx.x (Claimed NBMA address: 10.138.0.100) 10.1.0.2/32 via 10.1.0.2 Tunnel0 created 00:21:28, never expire Type: static, Flags: NBMA address: 192.xxx.xxx.x 10.1.0.4/32 via 10.1.0.4 Tunnel0 created 00:12:29, expire 00:02:40 Type: dynamic, Flags: router nhop rib nho NBMA address: 52.xxx.xxx.x (Claimed NBMA address: 172.16.1.10) 10.10.1.0/24 via 10.1.0.6 Tunnel0 created 00:08:30, expire 00:03:33 Type: dynamic, Flags: router unique local NBMA address: 10.10.0.4 (no-socket) 172.16.2.0/24 via 10.1.0.4 Tunnel0 created 00:07:19, expire 00:02:40 Type: dynamic, Flags: router rib nho NBMA address: 52.xxx.xxx.x (Claimed NBMA address: 172.16.1.10) csr-azure-01#show ip nhrp multicast I/F NBMA address Tunnel0 192.xxx.xxx.x Flags: nhs (Enabled) Spoke – Azure CSR shmcfarl@AzureTestVm:~$ traceroute 10.0.1.3 traceroute to 10.0.1.3 (10.0.1.3), 30 hops max, 60 byte packets 1 10.10.1.4 (10.10.1.4) 1.220 ms 1.192 ms 1.328 ms 2 10.0.1.3 (10.0.1.3) 25.794 ms * 25.782 ms Spoke – Azure VM
  • 64.
  • 65.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public DMVPN – Enable Dynamic Multicloud Networking 65 VNet Network 10.10.1.0/24 DMVPN OpenStack Private Cloud VPC Network 172.16.2.0/24 Cisco CSR1000v Cisco CSR1000v VPC Network 10.0.1.0/24 Cisco CSR1000v Cisco ASR1000 Hub Tunnel: 10.1.0.2 Spoke Tunnel: 10.1.0.1 Spoke Tunnel: 10.1.0.4 Spoke Tunnel: 10.1.0.6 192.168.200.0/24 OSPF 10 Area 0 VM.110 DataCenter Infra. 10.40.0.0/24 .6 Neutron Router
  • 66.
  • 67.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Split-Tunnel/Routing Options 67 • All three public cloud providers allow for either split-tunneling or forced/direct routing • Split-tunneling: • Public cloud resources (instances/VMs, container clusters) will use the default VPC gateway for non-on-prem routes • Public cloud resources will use the on-prem-specific routes advertised by the CSR • Forced/Direct routing – All public cloud resources will use the VPN connection as their default route for ALL traffic (forces traffic through the on-prem site) BGP Google Cloud VPN Google Cloud Router Cisco ASR1000 10.0.0.1 VPC Subnetwork GW External/NAT Routing 192.xxx.xxx.x 35.xxx.xxx.x10.0.0.5 Compute Engine 2 1
  • 68.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Dealing with Split Routes • OpenStack with two possible routes: • Typically the Neutron L3 agent is the default route for VMs on the Private-Network (172.16.0.1) • Adding a CSR for GCP-facing connections requires route changes: • Static definition or dynamically learned via Neutron BGP service 68 [centos@c7-os-vm1 ~]$ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 172.16.0.1 0.0.0.0 UG 0 0 0 eth0 10.138.0.0 172.16.0.11 255.255.240.0 UG 0 0 0 eth0 169.254.169.254 172.16.0.1 255.255.255.255 UGH 0 0 0 eth0 172.16.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 openstack subnet set --host-route destination=10.138.0.0/20,gateway=172.16.0.11 Private-Subnet Default 10.138.0.0/20
  • 69.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Summary • Public cloud native IPsec VPN support is good, but it is always point-to-point, does not have consistent support for NAT and lacks network-rich features • DMVPN can greatly improve the deployment, HA, scalability and operations of the VPN connections • Multicloud between multiple public cloud providers and on-prem look like distinctly separate hybrid cloud deployments but.. • You have to take into consideration: • Team knowledge of public cloud operations, tools, automation • Cross cloud tools and automation • Diversity of network designs, protocols, security • Multi-region designs • Availability zones within and across providers 69
  • 71.
  • 72.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public A Note On MTU • All three providers recommend a different size interface MTU for the IPsec tunnel interface: • Google recommends 1460 on the tunnel: https://cloud.google.com/vpn/docs/concepts/advanced#mtu • AWS recommends 1399 on the tunnel: https://docs.aws.amazon.com/AmazonVPC/latest/NetworkAdminGuide/Introduction.html • Azure recommends 1400 on the tunnel: https://docs.microsoft.com/en-us/azure/vpn- gateway/vpn-gateway-about-vpn-devices • In addition to MTU, you need to set and test your TCP MSS values • In my testing, an IP MTU of 1400 and TCP MSS of 1360 worked for all sites but this may need to change based on your applications and if you are adding other encaps like MPLS 72
  • 73.
  • 74.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Automating the Multicloud Network • Challenges: • Different toolsets for different jobs (Ansible, Python, Bash scripts, Terraform, etc..) • Different toolsets for different clouds (Heat for OpenStack, CloudFormation for AWS, Deployment Manager for GCP, Azure Automation) • Different toolsets for different vendor products (Cisco NSO, CloudCenter, Prime, YANG development kit, etc..) • There is no silver bullet - Start simple: • Use what your team knows – Perform a gap analysis on what you have against what you need • Initially, automate the things that hurt a lot to do by hand and that change frequently – I use free tools but that doesn’t mean the process is free J • I use public cloud clients (gcloud, aws cli, azure cli) for services that don’t change frequently or that need very unique/non- repeatable configurations • I use public cloud provider automation tools (GCP Deployment Manager) for in-project work (new instances with new networks for a GCP-only project) • I use REST for things that change a lot • When you want to stop pulling your hair out, move to something that can front-end each API that you need to talk to and treat the environment as a whole – Cisco CloudCenter: https://www.cisco.com/c/en/us/products/cloud-systems- management/cloudcenter/index.html 74
  • 75.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Amazon CloudFormation • https://aws.amazon.com/cloudformation/ • Template-based (JSON/YAML) – Build a stack(s) from a template file • Sometimes you need to run more than one stack (in order) to get what you need • Race conditions: Understand ‘DependsOn’ and the use of the wait condition • If you need to use more than one stack, use “Outputs” to export values that the next stack will need to build the next set of resources • Example template: https://github.com/shmcfarl/multicloud/tree/master/aws/cloudformation 75
  • 76.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Google Cloud Platform – Deployment Manager • https://cloud.google.com/deployment-manager/ • Configuration files (YAML), Templates (Python/Jinja2), Schema files (JSON) • Sometimes you need to run more than one stack (in order) to get what you need • Race conditions • Use “Outputs” to export values that the next stack will need to build the next set of resources • Example templates: https://github.com/shmcfarl/multicloud/tree/master/gcp/deployment- manager • Make your own changes to the files: <ZONE>, <PROJECT>, <IMAGE>, etc. • Deploy the main stack: • Deploy any custom routes that may be needed for other sites 76 gcloud deployment-manager deployments create gcp-stack --config gcp_main_stack.yaml --automatic-rollback-on-error gcloud deployment-manager deployments create gcp-stack-route --config inside-private-routes.yaml --automatic-rollback-on-error
  • 77.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Microsoft Azure Automation/Resource Manager • https://azure.microsoft.com/en-us/services/automation/ • Runbooks (create graphically, PowerShell, Python) • Read and select these carefully: https://docs.microsoft.com/en- us/azure/automation/automation-runbook-types • Resource Manager: https://docs.microsoft.com/en-us/azure/azure-resource- manager/resource-group-overview • https://github.com/Azure/azure-quickstart-templates/tree/master/cisco-csr-1000v • Example template: https://github.com/shmcfarl/multicloud/blob/master/azure/resource-manager/az- arm-csr-cleaned.json 77
  • 78.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Call APIs Directly • Google Cloud Platform: https://cloud.google.com/compute/docs/reference/latest/ • Amazon Web Services: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/Welc ome.html • Microsoft Azure: https://docs.microsoft.com/en-us/rest/api/ 78
  • 79.
    Google VPN – CreatingGoogle VPN, Router, IPsec, BGP via REST APIs
  • 80.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public Google Cloud API – Creating GCP Cloud VPN/Routers • Assumptions/environment: • Understand how to authenticate to GCP APIs: https://cloud.google.com/docs/authentication/ • In this example, the Paw application was used to craft GET, POST and PATCH calls • Some configurations have been sanitized for security purposes • Have on-prem Cloud infrastructure deployed and a CSR/ASR configured (can be done after GCP side is deployed) • In this example, the configuration will be deployed against the OpenStack use case discussed in the earlier slides • In this example, the default network created by GCP will be used • Note: gcloud has VERY long delays in commands if you have IPv6 enabled on your local machine – set to “link-local” mode on your Mac 80
  • 81.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public 81 Reference Topology for GCP API Example Private Network 172.16.0.0/24 .11Default Network 10.138.0.0/20 IPsec/IKEv2 Tunnel Mode OSPF 10 Area 0 OSPF<>BGP Redistribution 192.yyy.yyy.y 35.yyy.yyy.y Routes this side should see: 10.138.0.0/20 Routes this side should see: 172.16.0.0/24 On-Prem Cloud Google Cloud VPN Google Cloud Router BGP AS65000 BGP AS65003 169.254.0.5 169.254.0.6
  • 82.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public GCP API (1) – Create VPN GW and External IP POST /compute/v1/projects/<gcp_project_number>/regions/us-west1/targetVpnGateways HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close Content-Length: 138 { "name": "csr-gcp-os-aio-gw", "network": "projects/<gcp_project_number>/global/networks/default", "region": "projects/<gcp_project_number>/regions/us-west1" } POST /compute/v1/projects/<gcp_project_number>/regions/us-west1/addresses HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close Content-Length: 29 { "name": "gcp-to-os-dmz" } GET /compute/v1/projects/<gcp_project_number>/regions/us-west1/addresses/gcp-to-os-dmz HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close RESPONSE - SUMMARIZED: "name": "gcp-to-os-dmz", "description": "", "address": ”35.yyy.yyy.y", "status": "RESERVED", ... Output summarized POST: Create VPN Gateway POST: Create External IP Address GET: Get the External IP Address
  • 83.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public GCP API (2) – Create Forwarding Rules 83 POST /compute/v1/projects/<gcp_project_number>/regions/us-west1/forwardingRules HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close Content-Length: 257 { "name": "csr-gcp-os-aio-rule-esp", "IPProtocol": "ESP", "IPAddress": "35.yyy.yyy.y", "region": "projects/<gcp_project_number>/regions/us-west1", "target": "https://www.googleapis.com/compute/v1/projects/<gcp_project_number>/regions/us-west1/targetVpnGateways/csr-gcp-os-aio-gw" } POST /compute/v1/projects/<gcp_project_number>/regions/us-west1/forwardingRules HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close Content-Length: 278 { "name": "csr-gcp-os-aio-rule-udp500", "IPProtocol": "UDP", "IPAddress": "35.yyy.yyy.y", "region": "projects/<gcp_project_number>/regions/us-west1", "target": "https://www.googleapis.com/compute/v1/projects/<gcp_project_number>/regions/us-west1/targetVpnGateways/csr-gcp-os-aio-gw", "portRange": "500" } POST /compute/v1/projects/<gcp_project_number>/regions/us-west1/forwardingRules HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close Content-Length: 280 { "name": "csr-gcp-os-aio-rule-udp4500", "IPProtocol": "UDP", "IPAddress": "35.yyy.yyy.y", "region": "projects/<gcp_project_number>/regions/us-west1", "target": "https://www.googleapis.com/compute/v1/projects/<gcp_project_number>/regions/us-west1/targetVpnGateways/csr-gcp-os-aio-gw", "portRange": "4500" } POST: Create Forwarding rule for ESP ... Output summarized POST: Create Forwarding rule for UDP 500 POST: Create Forwarding rule for UDP 4500
  • 84.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public GCP API (3) – Create Cloud Router & BGP Session 84 POST /compute/v1/projects/<gcp_project_number>/regions/us-west1/routers HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close Content-Length: 574 { "name": "csr-gcp-os-bgp-rtr", "bgp": { "asn": "65000" }, "interfaces": [ { "name": "if-csr-gcp-os-bgp-rtr-02", "linkedVpnTunnel": "https://www.googleapis.com/compute/v1/projects/<gcp_project_number>/regions/us-west1/vpnTunnels/csr-gcp-os-aio-gw-tunnel-1", "ipRange": "169.254.0.5/30" } ], "bgpPeers": [ { "name": "csr-gcp-os-bgp-peer", "interfaceName": "if-csr-gcp-os-bgp-rtr-02", "ipAddress": "169.254.0.5", "peerIpAddress": "169.254.0.6", "peerAsn": "65003" } ], "region": "projects/<gcp_project_number>/regions/us-west1", "network": "https://www.googleapis.com/compute/v1/projects/<gcp_project_number>/global/networks/default" } POST: Create Cloud Router, BGP session and link to the Cloud VPN tunnel ... Output summarized
  • 85.
    © 2018 Ciscoand/or its affiliates. All rights reserved. Cisco Public GCP API (5) – Create Cloud VPN Tunnel 85 POST /compute/v1/projects/<gcp_project_number>/regions/us-west1/vpnTunnels HTTP/1.1 Authorization: Bearer XXXX Content-Type: application/json; charset=utf-8 Host: www.googleapis.com Connection: close Content-Length: 417 { "name": "csr-gcp-os-aio-gw-tunnel-1", "sharedSecret": " <pre-shared-password-goes-here> ", "router": "https://www.googleapis.com/compute/v1/projects/<gcp_project_number>/regions/us-west1/routers/csr-gcp-os-bgp-rtr", "peerIp": "192.yyy.yyy.y", "region": "projects/<gcp_project_number>/regions/us-west1", "ikeVersion": "2", "targetVpnGateway": "https://www.googleapis.com/compute/v1/projects/<gcp_project_number>/regions/us-west1/targetVpnGateways/csr-gcp-os-aio-gw" } POST: Create a Cloud VPN tunnel and associated it with the Cloud router ... Output summarized