BGP allow-as-in and as-override

If you are a service provider providing a managed MPLS based IP-VPN service some of the things you may worry about could include scaling your customer provisioning process.  If you have a customer that has 10 sites and requires BGP routing to CE devices at each site besides dealing with interface addressing allocations and so on - technically you may need to also allow unique BGP ASNs per site/CE as well.

There are a few techniques that I know that allows us to allocate the same BGP ASN to every router.

Let's start with a simple topology..

CE1 S0/0===S0/0 PE1 Fa0/0 ==Fa0/0 PE2 S0/0 ===S0/0 CE2

PE1 and PE1 are running OSPF, MPLS and LDP between themselves and MP-iBGP on ASN 666 to carry the customer routing information

CE1 has an EBGP connection to PE1 while CE2 has an EBGP connection to PE2 and each of these are redistributing a number of loopbacks into bgp

PE1 Config:
hostname PE1
!
ip vrf CustA
 rd 666:1
 route-target export 666:1
 route-target import 666:1
!

mpls label protocol ldp

!
interface Loopback0
 ip address 1.1.1.1 255.255.255.255
!
interface FastEthernet0/0
 description To PE2
 ip address 192.168.255.253 255.255.255.252
 duplex auto
 speed auto

 mpls ip
!
interface Serial0/0
 ip vrf forwarding CustA
 ip address 192.168.12.2 255.255.255.252
 clock rate 128000
!
router ospf 100
 router-id 1.1.1.1
 log-adjacency-changes
 network 1.1.1.1 0.0.0.0 area 0
 network 192.168.255.253 0.0.0.0 area 0
!
router bgp 666
 no bgp default ipv4-unicast
 bgp log-neighbor-changes
 neighbor 2.2.2.2 remote-as 666
 neighbor 2.2.2.2 update-source Loopback0
 !
 address-family vpnv4
  neighbor 2.2.2.2 activate
  neighbor 2.2.2.2 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf CustA
  neighbor 192.168.12.1 remote-as 64512
  neighbor 192.168.12.1 activate
  no synchronization
 exit-address-family

!

mpls ldp router-id Loopback0


PE2 Config:

hostname PE2
!
ip vrf CustA
 rd 666:1
 route-target export 666:1
 route-target import 666:1
!
mpls label protocol ldp

!
interface Loopback0
 ip address 2.2.2.2 255.255.255.255
!
interface FastEthernet0/0
 description To PE1
 ip address 192.168.255.254 255.255.255.252
 duplex auto
 speed auto

 mpls ip
!
interface Serial0/0
 ip vrf forwarding CustA
 ip address 192.168.34.2 255.255.255.0
 clock rate 128000
!
router ospf 100
 router-id 2.2.2.2
 log-adjacency-changes
 network 2.2.2.2 0.0.0.0 area 0
 network 192.168.255.254 0.0.0.0 area 0
!
router bgp 666
 no bgp default ipv4-unicast
 bgp log-neighbor-changes
 neighbor 1.1.1.1 remote-as 666
 neighbor 1.1.1.1 update-source Loopback0
 !
 address-family vpnv4
  neighbor 1.1.1.1 activate
  neighbor 1.1.1.1 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf CustA
  neighbor 192.168.34.1 remote-as 64512
  neighbor 192.168.34.1 activate
  no synchronization
 exit-address-family

!

mpls ldp router-id Loopback0

CE1 Config:

hostname CE1
!
interface Loopback0
 ip address 1.2.3.4 255.255.255.255
!
interface Loopback1
 ip address 2.3.4.5 255.255.255.255
!
interface Loopback2
 ip address 3.4.5.6 255.255.255.255
!
interface Loopback3
 ip address 4.5.6.7 255.255.255.255
!
interface Serial0/0
 ip address 192.168.12.1 255.255.255.252
!
router bgp 64512
 no synchronization
 bgp log-neighbor-changes
 redistribute connected metric 100
 neighbor 192.168.12.2 remote-as 666
 neighbor 192.168.12.2 soft-reconfiguration inbound
 no auto-summary
!

CE2 Config:

hostname CE2
!
interface Loopback0
 ip address 9.8.7.6 255.255.255.255
!
interface Loopback1
 ip address 8.7.6.5 255.255.255.255
!
interface Loopback2
 ip address 7.6.5.4 255.255.255.255
!
interface Loopback3
 ip address 6.5.4.3 255.255.255.255
!
interface Serial0/0
 ip address 192.168.34.1 255.255.255.252
!
router bgp 64512
 no synchronization
 bgp log-neighbor-changes
 redistribute connected metric 100
 neighbor 192.168.34.2 remote-as 666
 neighbor 192.168.34.2 soft-reconfiguration inbound
 no auto-summary
!


Lets see if we can see our routes from the other CE router:

CE1#sh ip route bgp

CE2#sh ip route bgp

 

Nothing... not a good sign, is BGP running?

CE1#sh ip bgp summ
BGP router identifier 4.5.6.7, local AS number 64512
BGP table version is 6, main routing table version 6
5 network entries using 600 bytes of memory
5 path entries using 260 bytes of memory
2/1 BGP path/bestpath attribute entries using 248 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
Bitfield cache entries: current 1 (at peak 1) using 32 bytes of memory
BGP using 1140 total bytes of memory
BGP activity 15/10 prefixes, 15/10 paths, scan interval 60 secs

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.12.2    4   666      71      52        6    0    0 00:03:57        0

CE2#sh ip bgp summ
BGP router identifier 9.8.7.6, local AS number 64512
BGP table version is 6, main routing table version 6
5 network entries using 600 bytes of memory
5 path entries using 260 bytes of memory
2/1 BGP path/bestpath attribute entries using 248 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
Bitfield cache entries: current 1 (at peak 2) using 32 bytes of memory
BGP using 1140 total bytes of memory
BGP activity 5/0 prefixes, 5/0 paths, scan interval 60 secs

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.34.2    4   666      53      89        6    0    0 00:00:27        0

Yes, we're just not seeing any prefixs coming in - can we see them on the PE routers?

PE1#sh ip route vrf CustA

Routing Table: CustA
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     192.168.12.0/30 is subnetted, 1 subnets
C       192.168.12.0 is directly connected, Serial0/0
     1.0.0.0/32 is subnetted, 1 subnets
B       1.2.3.4 [20/100] via 192.168.12.1, 00:08:15
     2.0.0.0/32 is subnetted, 1 subnets
B       2.3.4.5 [20/100] via 192.168.12.1, 00:08:15
     3.0.0.0/32 is subnetted, 1 subnets
B       3.4.5.6 [20/100] via 192.168.12.1, 00:08:15
     4.0.0.0/32 is subnetted, 1 subnets
B       4.5.6.7 [20/100] via 192.168.12.1, 00:08:15
     6.0.0.0/32 is subnetted, 1 subnets
B       6.5.4.3 [200/100] via 2.2.2.2, 00:01:49
     7.0.0.0/32 is subnetted, 1 subnets
B       7.6.5.4 [200/100] via 2.2.2.2, 00:01:49
     8.0.0.0/32 is subnetted, 1 subnets
B       8.7.6.5 [200/100] via 2.2.2.2, 00:01:49
     9.0.0.0/32 is subnetted, 1 subnets
B       9.8.7.6 [200/100] via 2.2.2.2, 00:01:49
     192.168.34.0/30 is subnetted, 1 subnets
B       192.168.34.0 [200/100] via 2.2.2.2, 00:01:49

PE2#sh ip route vrf CustA

Routing Table: CustA
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     192.168.12.0/30 is subnetted, 1 subnets
B       192.168.12.0 [200/100] via 1.1.1.1, 00:08:09
     1.0.0.0/32 is subnetted, 1 subnets
B       1.2.3.4 [200/100] via 1.1.1.1, 00:08:09
     2.0.0.0/32 is subnetted, 1 subnets
B       2.3.4.5 [200/100] via 1.1.1.1, 00:08:09
     3.0.0.0/32 is subnetted, 1 subnets
B       3.4.5.6 [200/100] via 1.1.1.1, 00:08:09
     4.0.0.0/32 is subnetted, 1 subnets
B       4.5.6.7 [200/100] via 1.1.1.1, 00:08:09
     6.0.0.0/32 is subnetted, 1 subnets
B       6.5.4.3 [20/100] via 192.168.34.1, 00:02:00
     7.0.0.0/32 is subnetted, 1 subnets
B       7.6.5.4 [20/100] via 192.168.34.1, 00:02:00
     8.0.0.0/32 is subnetted, 1 subnets
B       8.7.6.5 [20/100] via 192.168.34.1, 00:02:00
     9.0.0.0/32 is subnetted, 1 subnets
B       9.8.7.6 [20/100] via 192.168.34.1, 00:02:00
     192.168.34.0/24 is variably subnetted, 2 subnets, 2 masks
B       192.168.34.0/30 [20/100] via 192.168.34.1, 00:02:00
C       192.168.34.0/24 is directly connected, Serial0/0

Ok, so the PE routers appear to be happy enough at the situation....


What's the problem? Let's have a look at the BGP updates coming in

CE1#debug ip bgp updates
BGP updates debugging is on for address family: IPv4 Unicast
CE1#clear ip bgp *
*Mar  1 01:02:23.675: %BGP-5-ADJCHANGE: neighbor 192.168.12.2 Down User reset
*Mar  1 01:02:50.755: %BGP-5-ADJCHANGE: neighbor 192.168.12.2 Up
*Mar  1 01:02:50.763: BGP(0): 192.168.12.2 rcv UPDATE w/ attr: nexthop 192.168.12.2, origin ?, originator 0.0.0.0, path 666 64512, community , extended community
*Mar  1 01:02:50.763: BGP(0): 192.168.12.2 rcv UPDATE about 192.168.34.0/30 -- DENIED due to: AS-PATH contains our own AS;
*Mar  1 01:02:50.763: BGP(0): 192.168.12.2 rcv UPDATE about 9.8.7.6/32 -- DENIED due to: AS-PATH contains our own AS;
*Mar  1 01:02:50.763: BGP(0): 192.168.12.2 rcv UPDATE about 8.7.6.5/32 -- DENIED due to: AS-PATH contains our own AS;
*Mar  1 01:02:50.767: BGP(0): 192.168.12.2 rcv UPDATE about 7.6.5.4/32 -- DENIED due to: AS-PATH contains our own AS;
*Mar  1 01:02:50.767: BGP(0): 192.168.12.2 rcv UPDATE about 6.5.4.3/32 -- DENIED due to: AS-PATH contains our own AS;

Well this explains it - BGP loop avoidance is based on the AS-PATH, since CE1 and CE2 are both allocated the same ASN, we need to fix this.


CE1 EBGP AS 64512 - EBGP AS 666 PE1 I-MPBGP AS 666 PE2 EBGP AS 666 - EBGP AS 64512 CE2

Option 1 - allow-as-in on the CE

CE1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
CE1(config)#router bgp 64512
CE1(config-router)#neighbor 192.168.12.2 allowas-in ?
  <1-10>  Number of occurances of AS number
 

CE1(config-router)#neighbor 192.168.12.2 allowas-in 1

This allows us to add some flexibility to the AS-PATH checks - in this case if we see our ASN once in the path already, it's ok to allow it in

CE1(config-router)#do clear ip bgp * soft

CE1(config-router)#do sh ip route bgp
     6.0.0.0/32 is subnetted, 1 subnets
B       6.5.4.3 [20/0] via 192.168.12.2, 00:00:11
     7.0.0.0/32 is subnetted, 1 subnets
B       7.6.5.4 [20/0] via 192.168.12.2, 00:00:11
     8.0.0.0/32 is subnetted, 1 subnets
B       8.7.6.5 [20/0] via 192.168.12.2, 00:00:11
     9.0.0.0/32 is subnetted, 1 subnets
B       9.8.7.6 [20/0] via 192.168.12.2, 00:00:11
     192.168.34.0/30 is subnetted, 1 subnets
B       192.168.34.0 [20/0] via 192.168.12.2, 00:00:11



Lets see the AS-Path

CE1(config-router)#do sh ip bgp
BGP table version is 31, local router ID is 4.5.6.7
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 1.2.3.4/32       0.0.0.0                100         32768 ?
*> 2.3.4.5/32       0.0.0.0                100         32768 ?
*> 3.4.5.6/32       0.0.0.0                100         32768 ?
*> 4.5.6.7/32       0.0.0.0                100         32768 ?
*> 6.5.4.3/32       192.168.12.2                           0 666 64512 ?
*> 7.6.5.4/32       192.168.12.2                           0 666 64512 ?
*> 8.7.6.5/32       192.168.12.2                           0 666 64512 ?
*> 9.8.7.6/32       192.168.12.2                           0 666 64512 ?
*> 192.168.12.0/30  0.0.0.0                100         32768 ?
*> 192.168.34.0/30  192.168.12.2                           0 666 64512 ?

Option 2 - as-override on the PE

An alternate method is to remove the ASN of the other CE from the AS-PATH loop check doesn't fail on the local CE

PE2#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
PE2(config)#router bgp 666
PE2(config-router)#address-family ipv4 unicast vrf CustA
PE2(config-router-af)#neighbor 192.168.34.1 as-override
PE2(config-router-af)#do clear ip bgp * soft


Now lets see if CE2 is happy:

CE2#sh ip route bgp
     192.168.12.0/30 is subnetted, 1 subnets
B       192.168.12.0 [20/0] via 192.168.34.2, 00:02:46
     1.0.0.0/32 is subnetted, 1 subnets
B       1.2.3.4 [20/0] via 192.168.34.2, 00:02:46
     2.0.0.0/32 is subnetted, 1 subnets
B       2.3.4.5 [20/0] via 192.168.34.2, 00:02:46
     3.0.0.0/32 is subnetted, 1 subnets
B       3.4.5.6 [20/0] via 192.168.34.2, 00:02:46
     4.0.0.0/32 is subnetted, 1 subnets
B       4.5.6.7 [20/0] via 192.168.34.2, 00:02:46
 

Yep, and what is the AS path?

CE2#sh ip bgp
BGP table version is 21, local router ID is 9.8.7.6
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 1.2.3.4/32       192.168.34.2                           0 666 666 ?
*> 2.3.4.5/32       192.168.34.2                           0 666 666 ?
*> 3.4.5.6/32       192.168.34.2                           0 666 666 ?
*> 4.5.6.7/32       192.168.34.2                           0 666 666 ?
*> 6.5.4.3/32       0.0.0.0                100         32768 ?
*> 7.6.5.4/32       0.0.0.0                100         32768 ?
*> 8.7.6.5/32       0.0.0.0                100         32768 ?
*> 9.8.7.6/32       0.0.0.0                100         32768 ?
*> 192.168.12.0/30  192.168.34.2                           0 666 666 ?
*> 192.168.34.0/30  0.0.0.0                100         32768 ?

So the final test - L3 data plane connectivity:

CE2#ping 1.2.3.4 source 9.8.7.6

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 1.2.3.4, timeout is 2 seconds:
Packet sent with a source address of 9.8.7.6
!!!!!
 
It seems like for the CCIE lab there is usually more than one way to do something, and quite often you will be restricted from using particular methods, so it is always handy to have an alternate up your sleeve.