MPLS VPN Torn apart

MPLS is one of the topic where there is too much entry barrier as far as information is concerned. You have to deal with new protocols, different label types, MP BGP, VRFs, etc. on top of the fact that we don’t get exposure to it in our day to day networking (except if you work for an ISP). When I was taught MPLS the first time, it was very difficult for me to visualize (I rely heavily on visualization to understand a topic) the flow as labels had been flying all over the place and pushed and popped and what not. So this is my attempt to explain MPLS VPN by tearing open the packets and showing you what takes place across the network. I will try my best to make it as graphical as possible.

The Setup

The topology consists of 5 7200s as represented in the diagram. The ip addressing of the links are as mentioned in the diagram. OSPF is running on all links of Service Provider routers using network 0.0.0.0 255.255.255.255 area 0 command. In order to keep the topology simple, I have kept 2 CE routers, 2 PE Routers and a single P router. The PE-CE routing protocol is EIGRP.

setup

I will not explain the configuration as it is obvious from the diagram. I will post the configuration though at the end of the post. Also, I will not touch the CE part as that will be moving away from the topic as it does not run MPLS.

LDP

We are assuming that all layer 3 reachability in the ISP domain is up so that we can concentrate on LDP. We have enabled LDP on all interfaces of Service Provider.

LDP

Just like OSPF, LDP also uses the highest ip address of a loopback interface as router ID. In the case of the P router, there was no loopback, so it took the ip address of the highest addressed interface. The disadvantage to this is if the interface connecting to PE1 fails, so will LDP neighborship between P and PE2. The labels originated by the 3 MPLS enabled devices will be as follows PE1 1000-1999, P 2000-2999 and PE2 3000-3999. This has been done to know which routers originated which Labels to ensure simplicity.

LDP session establishment

From the above capture taken between PE1 and P, we come to know that PE1 initiated the LDP session with P because it had the highest router ID amongst the 2. Also, PE1 initiated a TCP session with P on destination port 646 which is the port for LDP. You can see the session establishment in the capture with SYN, SYN ACK and ACK messages. After all the messages are exchanged, the session is established and comes the time for Label exchange.LDP Label generation

As can be seen in the capture, PE1 is sending its label binding to P router. It contains all prefixes which are learnt via IGP and connected routes. The example of one such prefix is given which is 100.100.100.2 with a label of 1001 (as we have restricted labels from 1000 to 1999). Similarly, the labels for other subnets is 3 for 100.100.100.1, 3 for 10.1.1.0 and 1000 for 10.1.2.0. The Label value of 3 on 2 of the subnets is for Implicit Null as these are locally originated prefixes. The relevant show command on PE1 is as mentioned below

PE1#show mpls ldp bindings local
tib entry: 10.1.1.0/24, rev 2
local binding: tag: imp-null
tib entry: 10.1.2.0/24, rev 6
local binding: tag: 1000
tib entry: 100.100.100.1/32, rev 4
local binding: tag: imp-null
tib entry: 100.100.100.2/32, rev 8
local binding: tag: 1001

A similar process is repeated by all MPLS enabled routers to exchange the Labels so that a table is generated from it on each router. This is the MPLS forwarding table. The MPLS forwarding table on PE1 is as shown below

PE1#show mpls forwarding
Local Outgoing    Prefix              Bytes tag     Outgoing      Next Hop
tag     tag or VC   or Tunnel Id    switched       interface
1000  Pop tag      10.1.2.0/24            0                Gi2/0        10.1.1.2
1001  2001           100.100.100.2/32 0                 Gi2/0        10.1.1.2

The labels generated by LDP are used to transport packets across the MPLS Network and that is the reason they are called Transport labels.

MP BGP

We are not running ipv4 BGP as we do not require it. We are only running MP BGP between PE1 and PE2 as this required to exchange customer labels. The VRF for the customer is “cust” with RD = 100:1 and RT = 100:1.MP BGP Label exchange

The capture above depicts the transaction taking place between PE1 and PE2 after MP BGP comes up. The BGP update message which PE2 sends to PE1 contains the customer prefix (172.16.22.0), its label 3002 (since all labels originated by PE2 has to be between 3000 and 3999), RD of 100:1 and RT of 100:1. Also note that this label is marked as “bottom” which means it is at the bottom of the label stack. This label will only be used after the packet traverses the MPLS cloud and reaches the destination PE where the Transport label will be stripped (It can also be stripped before the destination depending upon whether Penultimate Hop Popping is enabled or not) and the packet sent out to customer interface depending upon the Label. This is the reason why this label is called VPN Label. Similar to the example of prefix 172.16.22.0, the update message will contain other prefix like 192.168.10.0. (Note: There is no field in the packet where the customer VRF is mentioned.)

After the MP BGP peers exchange updates, a BGP Label table is formed on each peer per vrf. The example table on PE1 is as given below

PE1#show ip bgp vpnv4 vrf cust labels
Network                              Next Hop                          In label/Out label
Route Distinguisher: 100:1 (cust)
1.1.40.0/24                         172.16.11.2                     1003/nolabel
172.16.11.0/24                    0.0.0.0                            1002/aggregate(cust)
172.16.22.0/24                    100.100.100.2                 nolabel/3002
192.168.10.0                       100.100.100.2                 nolabel/3003

Analyzing a typical traffic flow

Now that we have seen the 2 types of labels i.e. Transport and VPN labels, we will see how the 2 labels work together to transport packet to the destination. For testing this, we will send an icmp packet from CE1 to CE2. I am pinging from the host facing interface on CE1 to the host facing interface on CE2.

CE1#ping 192.168.10.1 source 1.1.40.1 repeat 1

Type escape sequence to abort.
Sending 1, 100-byte ICMP Echos to 192.168.10.1, timeout is 2 seconds:
Packet sent with a source address of 1.1.40.1
!
Success rate is 100 percent (1/1), round-trip min/avg/max = 84/84/84 ms

When we enter the above command, an icmp packet is created with source ip address of 1.1.40.1 and destination ip address of 192.168.10.1. When this packet reaches PE1 on a VRF enabled interface, the VRF routing table for VRF “cust” is checked and the destination address of the packet is matched. From the “sh ip bgp vpnv4 vrf cust labels” output on PE1, we see that 192.168.10.0 should be encapsulated with outgoing label of 3003 and the next hop is 100.100.100.2. The label value of 100.100.100.2 can be obtained from “sh mpls forwarding” command on PE1. The outgoing label is 2001 so the packet is encapsulated with an outer label of 2001.

CE1 to CE2 before P

As can be seen in the packet capture between PE1 and P, the transport label of 2001 is set and a VPN label of 3003 is set. Also, the VPN label is at the bottom of the label stack as indicated by the S bit set in the packet. When this packet reaches P router with an incoming label of 2001, the label is removed as per the MPLS forwarding table and sent out the outgoing interface. This is because Penultimate Hop Popping is enabled by default. This is depicted in the capture below where there is no outer label but the VPN Label is intact. (Note: Had there been more hops between PE1 and PE2, label 2001 would have been replaced by another label as per the switching table on P router)CE1 to CE2 before PE2

When this packet reaches PE2, the label value of 3003 is utilized to send the packet to the interface 172.16.22.2 (as per the switching table) and the label is stripped (as it has to be sent untagged as per switching table). The packet reaches CE2 and completes the one way communication. The output of the switching table on PE2 is as below

PE2#sh mpl for
Local                  Outgoing                   Prefix              Bytes tag        Outgoing     Next Hop
tag                       tag or VC                 or Tunnel Id    switched          interface
3000                        Pop tag                10.1.1.0/24              0                Gi1/0         10.1.2.2
3001                        2000                     100.100.100.1/32    0                Gi1/0         10.1.2.2
3002                      Aggregate               172.16.22.0/24[V]   0
3003                      Untagged                192.168.10.0/24[V] 0               Gi2/0    172.16.22.2

On the return journey, the source is 192.168.10.1 and destination is 1.1.40.1. When this packet reaches PE2, the destination address of 1.1.40.0 is matched with a VPN label of 1003 from the BGP Label table. The next hop of 100.100.100.1 is reached via a Transport label of 2000 from the MPLS forwarding table. The packet created is as seen in the below given capture.CE2 to CE1 on PE2

On reaching P router, the outer label is stripped due to PHP and the traffic is sent out the interface connecting PE1 to P. This can be seen in the below capture.CE2 to CE1 on P

When this packet reaches PE1, the label of 1003 is utilized to send the packet out interface connecting PE1 to CE1 with a next hop 172.16.11.2 and the label is stripped. This is as per the sh ip bgp vpnv4 vrf cust labels output on PE1. The packet reaches the destination and you see the reply.

Relevant Router Configurations

CE1

interface GigabitEthernet1/0
ip address 172.16.11.2 255.255.255.0
!
interface GigabitEthernet2/0
ip address 1.1.40.1 255.255.255.0
!
router eigrp 100
network 1.1.40.0 0.0.0.255
network 172.16.11.0 0.0.0.255
no auto-summary

PE1

ip vrf cust
rd 100:1
route-target export 100:1
route-target import 100:1
!
mpls label range 1000 1999

interface Loopback100
ip address 100.100.100.1 255.255.255.255
!
interface GigabitEthernet1/0
ip vrf forwarding cust
ip address 172.16.11.1 255.255.255.0
!
interface GigabitEthernet2/0
ip address 10.1.1.1 255.255.255.0
mpls ip
!
router eigrp 65000
auto-summary
!
address-family ipv4 vrf cust
redistribute bgp 100 metric 1 1 1 1 1
network 172.16.11.0 0.0.0.255
no auto-summary
autonomous-system 100
exit-address-family

router ospf 1
log-adjacency-changes
network 0.0.0.0 255.255.255.255 area 0
!
router bgp 100
no bgp default ipv4-unicast
bgp log-neighbor-changes
neighbor 100.100.100.2 remote-as 100
neighbor 100.100.100.2 update-source Loopback100
!
address-family vpnv4
neighbor 100.100.100.2 activate
neighbor 100.100.100.2 send-community extended
exit-address-family
!
address-family ipv4 vrf cust
redistribute eigrp 100
no auto-summary
no synchronization
exit-address-family

P

mpls label range 2000 2999

interface GigabitEthernet1/0
ip address 10.1.1.2 255.255.255.0
mpls ip
!
interface GigabitEthernet2/0
ip address 10.1.2.2 255.255.255.0
mpls ip
!
router ospf 1
log-adjacency-changes
network 0.0.0.0 255.255.255.255 area 0

PE2

ip vrf cust
rd 100:1
route-target export 100:1
route-target import 100:1
!
mpls label range 3000 3999

interface Loopback100
ip address 100.100.100.2 255.255.255.255

interface GigabitEthernet1/0
ip address 10.1.2.1 255.255.255.0
mpls ip
!
interface GigabitEthernet2/0
ip vrf forwarding cust
ip address 172.16.22.1 255.255.255.0
!
router eigrp 65000
auto-summary
!
address-family ipv4 vrf cust
redistribute bgp 100 metric 1 1 1 1 1
network 172.16.22.0 0.0.0.255
no auto-summary
autonomous-system 100
exit-address-family
!
router ospf 1
log-adjacency-changes
network 0.0.0.0 255.255.255.255 area 0

router bgp 100
no bgp default ipv4-unicast
bgp log-neighbor-changes
neighbor 100.100.100.1 remote-as 100
neighbor 100.100.100.1 update-source Loopback100
!
address-family vpnv4
neighbor 100.100.100.1 activate
neighbor 100.100.100.1 send-community extended
exit-address-family
!
address-family ipv4 vrf cust
redistribute eigrp 100
no auto-summary
no synchronization
exit-address-family

I hope my post has been helpful in your life but the only guide which can help you in the hereafter is the Qur’an. You can download the English translation of the Qur’an here.

Advertisements

One thought on “MPLS VPN Torn apart

  1. Pingback: Troubleshooting MPLS VPN | Baba AweSam

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s