If you are a service provider providing a managed MPLS based IP-VPN service some of the things you may worry about could include scaling your customer provisioning process. If you have a customer that has 10 sites and requires BGP routing to CE devices at each site besides dealing with interface addressing allocations and so on - technically you may need to also allow unique BGP ASNs per site/CE as well.
There are a few techniques that I know that allows us to allocate the same BGP ASN to every router.
Let's start with a simple topology..
CE1
S0/0===S0/0 PE1
Fa0/0 ==Fa0/0 PE2
S0/0 ===S0/0 CE2
PE1 and PE1 are running OSPF, MPLS and LDP between themselves and MP-iBGP on ASN 666 to carry the customer routing information
CE1 has an EBGP connection to PE1 while CE2 has an EBGP connection to PE2 and each of these are redistributing a number of loopbacks into bgp
PE1 Config:
hostname PE1!ip vrf CustA rd 666:1 route-target export 666:1 route-target import 666:1!
mpls label protocol ldp
!
interface Loopback0
ip address 1.1.1.1 255.255.255.255
!
interface FastEthernet0/0
description To PE2
ip address 192.168.255.253 255.255.255.252
duplex auto
speed auto
mpls ip!interface Serial0/0 ip vrf forwarding CustA ip address 192.168.12.2 255.255.255.252 clock rate 128000!router ospf 100 router-id 1.1.1.1 log-adjacency-changes network 1.1.1.1 0.0.0.0 area 0 network 192.168.255.253 0.0.0.0 area 0!router bgp 666 no bgp default ipv4-unicast bgp log-neighbor-changes neighbor 2.2.2.2 remote-as 666 neighbor 2.2.2.2 update-source Loopback0 ! address-family vpnv4 neighbor 2.2.2.2 activate neighbor 2.2.2.2 send-community extended exit-address-family ! address-family ipv4 vrf CustA neighbor 192.168.12.1 remote-as 64512 neighbor 192.168.12.1 activate no synchronization exit-address-family
!
mpls ldp router-id Loopback0
PE2 Config:
hostname PE2
!
ip vrf CustA
rd 666:1
route-target export 666:1
route-target import 666:1
!
mpls label protocol ldp
!
interface Loopback0
ip address 2.2.2.2 255.255.255.255
!
interface FastEthernet0/0
description To PE1
ip address 192.168.255.254 255.255.255.252
duplex auto
speed auto
mpls ip
!
interface Serial0/0
ip vrf forwarding CustA
ip address 192.168.34.2 255.255.255.0
clock rate 128000
!
router ospf 100
router-id 2.2.2.2
log-adjacency-changes
network 2.2.2.2 0.0.0.0 area 0
network 192.168.255.254 0.0.0.0 area 0
!
router bgp 666
no bgp default ipv4-unicast
bgp log-neighbor-changes
neighbor 1.1.1.1 remote-as 666
neighbor 1.1.1.1 update-source Loopback0
!
address-family vpnv4
neighbor 1.1.1.1 activate
neighbor 1.1.1.1 send-community extended
exit-address-family
!
address-family ipv4 vrf CustA
neighbor 192.168.34.1 remote-as 64512
neighbor 192.168.34.1 activate
no synchronization
exit-address-family
!
mpls ldp router-id Loopback0
CE1 Config:
hostname CE1!interface Loopback0 ip address 1.2.3.4 255.255.255.255!interface Loopback1 ip address 2.3.4.5 255.255.255.255!interface Loopback2 ip address 3.4.5.6 255.255.255.255!interface Loopback3 ip address 4.5.6.7 255.255.255.255!interface Serial0/0 ip address 192.168.12.1 255.255.255.252!router bgp 64512 no synchronization bgp log-neighbor-changes redistribute connected metric 100 neighbor 192.168.12.2 remote-as 666 neighbor 192.168.12.2 soft-reconfiguration inbound
no auto-summary!
CE2 Config:
hostname CE2!interface Loopback0 ip address 9.8.7.6 255.255.255.255!interface Loopback1 ip address 8.7.6.5 255.255.255.255!interface Loopback2 ip address 7.6.5.4 255.255.255.255!interface Loopback3 ip address 6.5.4.3 255.255.255.255!interface Serial0/0 ip address 192.168.34.1 255.255.255.252!router bgp 64512 no synchronization bgp log-neighbor-changes redistribute connected metric 100 neighbor 192.168.34.2 remote-as 666 neighbor 192.168.34.2 soft-reconfiguration inbound
no auto-summary!
Lets see if we can see our routes from the other CE router:
CE1#sh ip route bgp
CE2#sh ip route bgp
Nothing... not a good sign, is BGP running?
CE1#sh ip bgp summBGP router identifier 4.5.6.7, local AS number 64512BGP table version is 6, main routing table version 65 network entries using 600 bytes of memory5 path entries using 260 bytes of memory2/1 BGP path/bestpath attribute entries using 248 bytes of memory0 BGP route-map cache entries using 0 bytes of memory0 BGP filter-list cache entries using 0 bytes of memoryBitfield cache entries: current 1 (at peak 1) using 32 bytes of memoryBGP using 1140 total bytes of memoryBGP activity 15/10 prefixes, 15/10 paths, scan interval 60 secsNeighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd192.168.12.2 4 666 71 52 6 0 0 00:03:57 0
CE2#sh ip bgp summBGP router identifier 9.8.7.6, local AS number 64512BGP table version is 6, main routing table version 65 network entries using 600 bytes of memory5 path entries using 260 bytes of memory2/1 BGP path/bestpath attribute entries using 248 bytes of memory0 BGP route-map cache entries using 0 bytes of memory0 BGP filter-list cache entries using 0 bytes of memoryBitfield cache entries: current 1 (at peak 2) using 32 bytes of memoryBGP using 1140 total bytes of memoryBGP activity 5/0 prefixes, 5/0 paths, scan interval 60 secsNeighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd192.168.34.2 4 666 53 89 6 0 0 00:00:27 0
Yes, we're just not seeing any prefixs coming in - can we see them on the PE routers?
PE1#sh ip route vrf CustA
Routing Table: CustA
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route
Gateway of last resort is not set
192.168.12.0/30 is subnetted, 1 subnets
C 192.168.12.0 is directly connected, Serial0/0
1.0.0.0/32 is subnetted, 1 subnets
B 1.2.3.4 [20/100] via 192.168.12.1, 00:08:15
2.0.0.0/32 is subnetted, 1 subnets
B 2.3.4.5 [20/100] via 192.168.12.1, 00:08:15
3.0.0.0/32 is subnetted, 1 subnets
B 3.4.5.6 [20/100] via 192.168.12.1, 00:08:15
4.0.0.0/32 is subnetted, 1 subnets
B 4.5.6.7 [20/100] via 192.168.12.1, 00:08:15
6.0.0.0/32 is subnetted, 1 subnets
B 6.5.4.3 [200/100] via 2.2.2.2, 00:01:49
7.0.0.0/32 is subnetted, 1 subnets
B 7.6.5.4 [200/100] via 2.2.2.2, 00:01:49
8.0.0.0/32 is subnetted, 1 subnets
B 8.7.6.5 [200/100] via 2.2.2.2, 00:01:49
9.0.0.0/32 is subnetted, 1 subnets
B 9.8.7.6 [200/100] via 2.2.2.2, 00:01:49
192.168.34.0/30 is subnetted, 1 subnets
B 192.168.34.0 [200/100] via 2.2.2.2, 00:01:49
PE2#sh ip route vrf CustARouting Table: CustACodes: C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route o - ODR, P - periodic downloaded static routeGateway of last resort is not set 192.168.12.0/30 is subnetted, 1 subnetsB 192.168.12.0 [200/100] via 1.1.1.1, 00:08:09 1.0.0.0/32 is subnetted, 1 subnetsB 1.2.3.4 [200/100] via 1.1.1.1, 00:08:09 2.0.0.0/32 is subnetted, 1 subnetsB 2.3.4.5 [200/100] via 1.1.1.1, 00:08:09 3.0.0.0/32 is subnetted, 1 subnetsB 3.4.5.6 [200/100] via 1.1.1.1, 00:08:09 4.0.0.0/32 is subnetted, 1 subnetsB 4.5.6.7 [200/100] via 1.1.1.1, 00:08:09 6.0.0.0/32 is subnetted, 1 subnetsB 6.5.4.3 [20/100] via 192.168.34.1, 00:02:00 7.0.0.0/32 is subnetted, 1 subnetsB 7.6.5.4 [20/100] via 192.168.34.1, 00:02:00 8.0.0.0/32 is subnetted, 1 subnetsB 8.7.6.5 [20/100] via 192.168.34.1, 00:02:00 9.0.0.0/32 is subnetted, 1 subnetsB 9.8.7.6 [20/100] via 192.168.34.1, 00:02:00 192.168.34.0/24 is variably subnetted, 2 subnets, 2 masksB 192.168.34.0/30 [20/100] via 192.168.34.1, 00:02:00C 192.168.34.0/24 is directly connected, Serial0/0
Ok, so the PE routers appear to be happy enough at the situation....
What's the problem? Let's have a look at the BGP updates coming in
CE1#debug ip bgp updatesBGP updates debugging is on for address family: IPv4 UnicastCE1#clear ip bgp **Mar 1 01:02:23.675: %BGP-5-ADJCHANGE: neighbor 192.168.12.2 Down User reset*Mar 1 01:02:50.755: %BGP-5-ADJCHANGE: neighbor 192.168.12.2 Up*Mar 1 01:02:50.763: BGP(0): 192.168.12.2 rcv UPDATE w/ attr: nexthop 192.168.12.2, origin ?, originator 0.0.0.0, path 666 64512, community , extended community*Mar 1 01:02:50.763: BGP(0): 192.168.12.2 rcv UPDATE about 192.168.34.0/30 -- DENIED due to: AS-PATH contains our own AS;*Mar 1 01:02:50.763: BGP(0): 192.168.12.2 rcv UPDATE about 9.8.7.6/32 -- DENIED due to: AS-PATH contains our own AS;*Mar 1 01:02:50.763: BGP(0): 192.168.12.2 rcv UPDATE about 8.7.6.5/32 -- DENIED due to: AS-PATH contains our own AS;*Mar 1 01:02:50.767: BGP(0): 192.168.12.2 rcv UPDATE about 7.6.5.4/32 -- DENIED due to: AS-PATH contains our own AS;*Mar 1 01:02:50.767: BGP(0): 192.168.12.2 rcv UPDATE about 6.5.4.3/32 -- DENIED due to: AS-PATH contains our own AS;
Well this explains it - BGP loop avoidance is based on the AS-PATH, since CE1 and CE2 are both allocated the same ASN, we need to fix this.
CE1
EBGP AS 64512 -
EBGP AS 666 PE1
I-MPBGP AS 666 PE2
EBGP AS 666 - EBGP AS 64512 CE2
Option 1 - allow-as-in on the CE
CE1#conf tEnter configuration commands, one per line. End with CNTL/Z.CE1(config)#router bgp 64512CE1(config-router)#neighbor 192.168.12.2 allowas-in ? <1-10> Number of occurances of AS number CE1(config-router)#neighbor 192.168.12.2 allowas-in 1
This allows us to add some flexibility to the AS-PATH checks - in this case if we see our ASN once in the path already, it's ok to allow it in
CE1(config-router)#do clear ip bgp * soft
CE1(config-router)#do sh ip route bgp 6.0.0.0/32 is subnetted, 1 subnetsB 6.5.4.3 [20/0] via 192.168.12.2, 00:00:11 7.0.0.0/32 is subnetted, 1 subnetsB 7.6.5.4 [20/0] via 192.168.12.2, 00:00:11 8.0.0.0/32 is subnetted, 1 subnetsB 8.7.6.5 [20/0] via 192.168.12.2, 00:00:11 9.0.0.0/32 is subnetted, 1 subnetsB 9.8.7.6 [20/0] via 192.168.12.2, 00:00:11 192.168.34.0/30 is subnetted, 1 subnetsB 192.168.34.0 [20/0] via 192.168.12.2, 00:00:11
Lets see the AS-Path
CE1(config-router)#do sh ip bgpBGP table version is 31, local router ID is 4.5.6.7Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S StaleOrigin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path*> 1.2.3.4/32 0.0.0.0 100 32768 ?*> 2.3.4.5/32 0.0.0.0 100 32768 ?*> 3.4.5.6/32 0.0.0.0 100 32768 ?*> 4.5.6.7/32 0.0.0.0 100 32768 ?*> 6.5.4.3/32 192.168.12.2 0 666 64512 ?*> 7.6.5.4/32 192.168.12.2 0 666 64512 ?*> 8.7.6.5/32 192.168.12.2 0 666 64512 ?*> 9.8.7.6/32 192.168.12.2 0 666 64512 ?*> 192.168.12.0/30 0.0.0.0 100 32768 ?*> 192.168.34.0/30 192.168.12.2 0 666 64512 ?
Option 2 - as-override on the PE
An alternate method is to remove the ASN of the other CE from the AS-PATH loop check doesn't fail on the local CE
PE2#conf tEnter configuration commands, one per line. End with CNTL/Z.PE2(config)#router bgp 666PE2(config-router)#address-family ipv4 unicast vrf CustAPE2(config-router-af)#neighbor 192.168.34.1 as-overridePE2(config-router-af)#do clear ip bgp * soft
Now lets see if CE2 is happy:
CE2#sh ip route bgp 192.168.12.0/30 is subnetted, 1 subnetsB 192.168.12.0 [20/0] via 192.168.34.2, 00:02:46 1.0.0.0/32 is subnetted, 1 subnetsB 1.2.3.4 [20/0] via 192.168.34.2, 00:02:46 2.0.0.0/32 is subnetted, 1 subnetsB 2.3.4.5 [20/0] via 192.168.34.2, 00:02:46 3.0.0.0/32 is subnetted, 1 subnetsB 3.4.5.6 [20/0] via 192.168.34.2, 00:02:46 4.0.0.0/32 is subnetted, 1 subnetsB 4.5.6.7 [20/0] via 192.168.34.2, 00:02:46
Yep, and what is the AS path?
CE2#sh ip bgpBGP table version is 21, local router ID is 9.8.7.6Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S StaleOrigin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path*> 1.2.3.4/32 192.168.34.2 0 666 666 ?*> 2.3.4.5/32 192.168.34.2 0 666 666 ?*> 3.4.5.6/32 192.168.34.2 0 666 666 ?*> 4.5.6.7/32 192.168.34.2 0 666 666 ?*> 6.5.4.3/32 0.0.0.0 100 32768 ?*> 7.6.5.4/32 0.0.0.0 100 32768 ?*> 8.7.6.5/32 0.0.0.0 100 32768 ?*> 9.8.7.6/32 0.0.0.0 100 32768 ?*> 192.168.12.0/30 192.168.34.2 0 666 666 ?*> 192.168.34.0/30 0.0.0.0 100 32768 ?
So the final test - L3 data plane connectivity:
CE2#ping 1.2.3.4 source 9.8.7.6
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 1.2.3.4, timeout is 2 seconds:
Packet sent with a source address of 9.8.7.6
!!!!!
It seems like for the CCIE lab there is usually more than one way to do something, and quite often you will be restricted from using particular methods, so it is always handy to have an alternate up your sleeve.