IBM Cloud: Direct Link Guide

© 2020 IBM Corporation
IBM Cloud
Direct Link Guide
2020/11/02
Cloud Computing
IBM Japan
Tomoyuki Niijima

Direct Link Overview
▪ IBM Cloud offers Direct Link services to allow customers to establish private connectivity
from their premise.
▪ There are the following types.
– Direct Link Exchange 1.0
• Single connection over Exchange service provided at PoP by 3rd party service
providers such as ECX (Equinix Cloud Exchange). Cost effective solution especially
for those who will connect multiple clouds.
– Direct Link Connect 1.0 and 2.0
• Provided by partnership with various network service providers.
– Direct Link Dedicated 1.0 and 2.0
• Use this if you need a dedicated port
• Use this if you need a connection at a DC location
CAUTION: Now BGP is mandatory to manage routing via Direct Link. An account that orders
Direct Link will be migrated to VRF environment. Inter-account VLAN spanning will not be
allowed in VRF environment. VPN service will be limited.
You can configure redundant links by ordering two Direct Links. (See P.18)
There is another type called Direct Link Dedicated Hosting which offers racks and cross
connects. Direct Link Dedicated Hosting is out of the scope of this document.
2

Direct Link 1.0 vs 2.0
Direct Link 1.0
▪ Support only IBM Cloud Classic
▪ Unmetered billing only
▪ Dedicated, Exchange and Connect
▪ Manual ordering process (except ECX)
▪ It takes a few business days to assign a
subnet in between CER and XCR
▪ Likely to be sunset once 2.0 coverage will
be good enough.
▪ Direct Link 2.0
▪ Support VPC Gen1 and Gen2 in addition to
Classic
▪ Unmetered and Metered billing
▪ Dedicated and Connect (Exchange was
integrated into Connect)
▪ Will be upgraded to support automated
ordering process using API
▪ Immediate assignment of a subnet in
between CER and XCR. Even a user can
define it by themselves.
▪ Coverage of DL partners are still limited
3
Direct Link Connect 2.0 is still new. If you find a location and a provider available on the IBM Cloud portal
it doesn’t necessarily means that someone already proved that it works. You might want to consult your
IBM sales for the availability of the service at your preferred location and a provider.
user
XCRCER Classic IaaS
user
XCRCER Classic Iaas
VPC Gen2DL
VRF
VPC Gen1

© 2020 IBM Corporation4
Direct Link Exchange 1.0 overview
▪ Single connection over Exchange service provided at PoP by 3rd party service providers
such as ECX (Equinix Cloud Exchange) or Verison.
▪ Customers needs to cover process and cost to establish connect from their premise to the
data center where a target PoP is located, including that of exchange service.
▪ Typically requires 30 to 60 days to complete the link.
▪ 50M, 100M, 200M, 500M, 1G, 2G and 5Gbps speeds are available.
▪ Subnets required in between routers will be provided by IBM Cloud.
▪ 1000BASE-LX or10GBASE-LR, both with single mode fibre will be used for the link.
▪ VLAN trunking of 802.1Q or 802.1ad will be used.
Customer
PoP
DC of a PoP IBM Cloud
DC
BCR
VM
Private VLAN
Public VLAN
Internet
CX
802.1Q
Or
802.1ad
802.1Q
Customer’s responsibility IBM Cloud’s responsibility
CER XCR
BGP
VC

Direct Link Exchange 1.0 ordering process (except via ECX)
1. Sign up for IBM Cloud.
2. Order a server on IBM Cloud to find a subnet on a Private VLAN.
3. Order the following to the exchange service provider.
– Co-location space to install the customer’s router at PoP location
– Exchange service
– Cross connect to the exchange service
4. Request IBM Cloud to migrate the account to VRF.
5. Order a Direct Link for Cloud to confirm connection parameters such as Service Key, CID, IP address
and BGP ASN.
6. Order a VC to the exchange provider.
1. Specify VLAN type
2. Specify VLAN ID for the customer’ s side
3. Specify the Service Key and the CID.
4. Notify IBM Cloud of the VC order completion.
7. Configure the customer’s side router.
8. Verify connectivity in between CERs and XCRs.
9. Configure NAT gateway or GRE tunnels based on a design.
10. Verify end to end testing.

Direct Link Exchange 1.0 ordering process (via ECX)
Before following the process, ECX account must be ready.
4. Order a VC under the menu of Direct Link Exchange on the ECX portal
1. Specify destination as one of IBM Cloud PoPs
2. Specify the account number of IBM Cloud to be connected
3. Specify buyer’s port to be used
4. Specify VLAN ID on user end
5. Specify VC Speed
6. Specify BGP ASN
5. Order a Direct Link Exchange by accepting the VC request on the IBM Cloud portal
6. Confirm defined IP addresses and BGP ASN.
8. Verify connectivity in between CERs and XCRs after VC will be connected status.
10.Verify end to end testing.

Direct Link Exchange physical wiring
▪ From customer’s premise to the target PoP
– Remote network from the customer’s premise to the target PoP location will be provided
by a carrier, and co-location space at the data center will be provided by a data center
provider, based on the customer’s order.
▪ In house connection at the target DC
– 1000BASE-LX or 10GBASE-LR with single mode fibre
– The customer is responsible to the customer’s side connection.
– From the customer’s rack to the exchange service will be connected by a data center
provider based on the customer’s order.
– Connection in between the exchange service and IBM Cloud is done by IBM Cloud.
– Redundant link will need two links. Exchange service providers and IBM Cloud has two
links in between two pair of equipments.
IBM Cloud DC
A Data center of a PoP location
CX
Remote lines
To customer’s place
IBM Cloud’s
responsibility
IBM Cloud PoP
CER01 XCR01
CER02 XCR02
Customer’s
responsibility

Direct Link Exchange: ports of ECX
▪ ECX has two service nodes, SE01 and SE02 in each location.
▪ There are a pair of links in between ECX and IBM Cloud using two seller’s ports. XCR01 is connected to
SE01 via the primary seller’s port, XCR02 is to SE02 via the secondary seller’s port.
▪ Name of two seller’s ports includes -PRI- or -SEC- as the following example.
– SOFTLAYER-TY2-CX-PRI-01
– SOFTLAYER-TY2-CX-SEC-01
▪ Customers can not specify a seller’s port when they order their first Direct Link Exchange via ECX at a
PoP, but they can see the assigned port name on the edit screen of the link, that can be opened by using
action > edit on the list of Direct Link Exchange.
▪ Customers can specify a seller’s port when they order their second or newer links on the order screen.
They can also confirm number of existing link on each ports.
▪ Customers will need to carefully specify a seller’s port so that two links will be provisioned on separate
XCRs.
▪ They will also use the port information to properly specify buyer’s port when they order a VC for each link.
Please be noted that buyer’s port with a name including -PRI- is not always connected to SE01. Please
verify mapping in between buyer’s ports and service nodes with Equinix.
CER01 XCR01
SE02
ECX
CER02 XCR02
SE01
Seller’s portBuyer’s port
-PRI-
-SEC-
-PRI-
-SEC-

Direct Link Connect 1.0 and 2.0 overview
▪ Provided by partnerships with carriers who offer network connectivity service with cloud
connectivity option.
▪ Physical connections in between IBM Cloud and partner carriers are already in place.
Customers can order virtual links with specified bandwidth.
▪ Customers must be existing user of those partner’s network connectivity service or will have
to order one of those network connectivity service.
▪ 50M, 100M, 200M, 500M, 1G, 2G and 5Gbps speeds are available.
▪ Peer router of customer’s side will be a pair of PE routers managed by a carrier or
customer’s CER router depending on types of the network service. (See P.10)
▪ Availability of single or redundant link options depends on each partner’s service model.

Direct Link Connect 1.0 and 2.0 overview (continued)
▪ Peer routers will be a pair of PE routers managed by carriers when the network service is
providing layer 3 network.
▪ Peer routers will be a pair of CER routers when the network service is providing layer 2
network.
List of L2 and L3 carriers: https://cloud.ibm.com/docs/infrastructure/direct-link?topic=direct-link-comparing-layer-2-layer-3
Users
PECE XCR
PoP
IBM Cloud DC
BCR
VM
Private VLAN
Public VLAN
Internet
Customer’s scope IBM’s scope
BGP
VC
Carrier’s
L3
Network
Users
SWCER XCR
PoP
IBM Cloud DC
BCR
VM
Private VLAN
Public VLAN
Internet
Customer’s scope IBM’s scope
BGP
VC
Users
CE
Carrier’s L2 Network

Direct Link Connect 1.0 and 2.0 ordering process
3. Order the following to the exchange service provider.
– Network service from the customer’s location to a PoP location of IBM Cloud
– Cloud connectivity service by the Network Service Provider
5. Order a Direct Link Connect to confirm connection parameters such as Service Key, CID, IP address
and BGP ASN.
6. Order a VC to the Network Service Provider.
1. Specify VLAN type
2. Specify VLAN ID for the customer’ s side
3. Specify the Service Key and the CID.
4. Notify IBM Cloud of the VC order completion.
8. Verify connectivity in between CERs and XCRs.
10. Verify end to end testing.
*The ordering process described on this page is a subject to change once a service provider will be ready to
upgrade the process into automation.

Direct Link Dedicated 1.0 and 2.0 overview
▪ Customers needs to cover process and cost to establish connect from their premise to the
data center where a target PoP is located, including that of cross connect.
▪ 1G, 2G, 5G and 10Gbps speeds are available.
▪ 1000BASE-LX or10GBASE-LR, both with single mode fibre will be used for the link.
Customer
NAT
PoP
Data Center
IBM Cloud
Data center
BCR
VM
Private VLAN
Public VLAN
Internet
CER XCRBGP

Direct Link Dedicated ordering process
3. Order a co-location space to install the customer’s router at PoP location
4. Order a Direct Link Dedicated to get LOA (Letter of Agreement) and other information.
5. Order connections from customer’s premise to the data center.
6. Order cross connects to the data center provider.
7. Configure the customer’s router (CER).
8. Send IBM Cloud a completion notice for the cross connects issued by the data center
provider via a ticket of the Direct Link order.
9. Verify connectivity in between CER and XCR.
10.Configure NAT gateway or GRE tunnels based on a design.
11.Verify end to end testing.
12.Notify completion of the connection via ticket.
13.Charge for the link will be started.

Direct Link Dedicated physical wiring
▪ From customer’s premise to the target PoP location
– Remote network from the customer’s premise to the target PoP location will be provided
by a carrier, and co-location space at the data center will be provided by a data center
provider, based on the customer’s order.
▪ In house connection at the target PoP location
– 1000BASE-LX or 10GBASE-LR with single mode fibre
– The customer is responsible to the customer’s side connection.
– From the customer’s rack to IBM Cloud’s patch panel will be connected by a data center
provider based on the customer’s order.
– IBM Cloud is responsible to connect from IBM Cloud’s side patch panels to IBM Cloud’s
routers.
CER
Patch panel
XCR
Patch panel
A Data center of a
PoP locationCustomer’s
responsibility
IBM Cloud’s
responsibility
IBM Cloud PoP
IBM Cloud DC
Remote lines
To customer’s place

Contracts and payments around PoP/DC
▪ Remote connection
– Carrier will provide a quote
– Data center provider will provide a quote for a co-location space to install customer’s
routers.
▪ Cross connect at PoP/DC
– Data center provider will provide a quote for cross connects
▪ Direct Link
– Direct Link should be ordered on the IBM Cloud portal
Customer
NAT
IBM Cloud’s scope
VM
Private VLAN
Public VLAN
Internet
CER XCR

Direct Link routing in between Classic IaaS and on-prem
▪ A customer’s router at PoP/DC is called as CER (Customer’s Endpoint Router). A IBM Cloud’s side router
is called as XCR (Cross Connect Router).
▪ IBM Cloud provides subnet (10.254.x.x/31*) in between CER and XCR. As for Direct Link 2.0, a subnet
from 169.254.0.0/16 range will be provided. Alternatively, you can manually define a subnet from one of
10.254.0.0/16, 172.16.0.0/12, or 192.168.0.0/16 ranges.
▪ VRF environment will be configured around XCR and BCR which enables routing from the private VLAN
to the customer’s network.
▪ BGP need to be configured in between CER and XCR.
▪ CER advertise routes to customer’s network to XCR, XCR advertise IBM Cloud’s network to CER.
▪ CER is inhibited to advertise the following subnets which is reserved by IBM Cloud.
10.0.0.0/14, 10.200.0.0/14, 10.198.0.0/15, 169.254.0.0/16, 224.0.0.0/4
Subnet for Direct Link (/31 or /30 size subnet from 10.254.0.0/16 range)
Subnets on Private VLANs
CER BCR
Private VLAN
Public VLAN
VM
XCR
10.254.x.x/31*
VRF
Customer
IBM Cloud scope
Customer’s address IBM Cloud’s address
*Request /30 subnet if CER will not support /30.
Alternative address or range (172.16.0.0/12) can
be requested if the assigned subnet won’t work.

Direct Link 2.0 routing in between VPC and on-prem
▪ A customer’s router at PoP/DC is called as CER (Customer’s Endpoint Router). A IBM Cloud’s side router
is called as XCR (Cross Connect Router).
▪ IBM Cloud provides a subnet in between CER and XCR from 169.254.0.0/16 range. Alternatively, you can
manually define a subnet from one of 10.254.0.0/16, 172.16.0.0/12, or 192.168.0.0/16 ranges.
▪ DL VRF will be configured on XCR with a VC for each VPC and/or Classic to be connected.
▪ BGP will be configured in between CER and XCR.
▪ CER advertise routes to customer’s network to XCR, XCR advertise IBM Cloud’s network to CER.
▪ There is no routing limitation if you will not connect the link to Classic.
▪ The routing from IBM Cloud to on-prem will be limited as that of Direct Link 1.0 when you will connect to
Classic.
▪ You may want to user Transit Gateway in addition to Direct Link and VC when you want to interconnect in
between multiple VPCs and/or Classic. Just having VCs from a Direct Link will not allow VPCs and/or
Classic to be connected each other.
user
CER
VM
XCR
IBM Cloud’s scope
BGP
User defined address
VC
DL
VRF
VPC
VRF
VM
VPC
VRF
VC

Direct Link: some more details about BGP use
1. What AS number can be used?
IBM Cloud’s ASN is 13884, for both public and private. IBM Cloud will specify private ASNs
as AS PATH attribute for each route advertisements from the range of 65200-65235 and
65400-65435.
Customer can use a public ASN (1 - 64511) or a private ASN (64999 is recommended)
excluding 64512, 64513, 65100, 65201-65234, 65402–65433, 65500, 4201065000-
4201065999.
2. How IBM Cloud advertises routes through BGP from XCR to CER?
For each private subnets on Private VLANs that are assigned to the account and should
be reachable through the Direct Link will be advertised.
3. Which attributes are supported?
Well-known mandatory and Well-known discretionary are supported.
4. Supported capabilities?
The following capabilities are confirmed to be supported. XCR will just ignore non-
supported capabilities sent through a BGP session and will keep the session.
Multi protocol Extensions Capability (address family IPv4 unicast only)
Route-Refresh Capability
4 Octets-AS Capability
5. keepalive/hold timer setting?
keepalive 30sec / hold time 90sec

Direct Link routing (optional)
Address conflict in between customer’s network and IBM Cloud network may be avoided by
the following method.
▪ Configure GW using VRA or Linux server on IBM Cloud.
▪ Establish GRE or IPsec tunnel in between GW and CER, or a router that can be reachable
by using a route which is advertised in BGP.
▪ Use customer’s address from customer’s side up to GW through the tunnel, use IBM Cloud’s
address from CER to IBM Cloud side out side of the tunnel.
▪ NAT still may be required at GW depending on IP addresses involved on both end.
CER
Private VLAN
Public VLAN
VMGW
NAT
XCR
10.254.x.x/31Customer
IBM Cloud scope
Customer’s address
IBM Cloud’s address
BGP
GRE tunnel

Direct Link: Redundant path configuration
1. Any of Direct Link Exchange, Direct Link Connect, and Direct Link Dedicated will provide single link for
each one order.
2. Some carriers require 2 Direct Link Connect orders at the same PoP.
3. Other than the case in 2., any two or more Direct Links can be used to configure redundant path.
4. An recommended method is to configure active/standby path using BGP AS PATH.
a. Configure primary CER to advertise routes with AS PATH=64999, while secondary CER to
configure to advertise the same routes with AS PATH=“64999 64999”.
b. This will route packets from IBM Cloud to on-premise via primary CER when the primary is
available and route them via secondary when the primary down.
c. Path from from on-premise to IBM Cloud is typically managed by having a virtual gateway address
in between primary and secondary CERs on their interfaces for the on-premise side.
5. All the XCRs will advertise the same routes for all the private subnets to each peer CER.
CER1
CER2
XCR1
XCR2
IBM Cloud
Customer’s
Network
BGPgateway
address
AS PATH=64999
AS PATH=“64999 64999”
AS 13884AS 64999

References
▪ IBM Cloud / Direct Link
https://cloud.ibm.com/docs/direct-link
21

IBM Cloud: Direct Link Guide

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to IBM Cloud: Direct Link Guide

Similar to IBM Cloud: Direct Link Guide (20)

Recently uploaded

Recently uploaded (20)

IBM Cloud: Direct Link Guide