Lock and Key (Dynamic Access-Lists)

Lock and Key allows us to dynamically change an access-list using the access-enable command.

Let us consider a simple topology of R4 Connected to R5 Connected to R6

R4 is representing a host device we're using with R5 as its default gateway with R5 and R6 having dynamic routing between them.

Relevant Config:

R4
hostname R4
interface FastEthernet0/0
 ip address 192.168.101.4 255.255.255.0
 duplex auto
 speed auto
!
ip route 0.0.0.0 0.0.0.0 192.168.101.5


R5
hostname R5
username bob password 0 test
!
interface FastEthernet0/1
 description LAN Interface
 ip address 192.168.101.5 255.255.255.0
 ip access-group 104 in
 duplex auto
 speed auto
!
interface Loopback0
 ip address 5.5.5.5 255.255.255.255
!
access-list 104 permit tcp host 192.168.101.4 host 5.5.5.5 eq telnet
access-list 104 dynamic R4Telnet permit tcp host 192.168.101.4 any eq telnet
access-list 104 deny   tcp host 192.168.101.4 any eq telnet
!
line vty 0 4
 login local
 autocommand  access-enable timeout 5
!

We aren't showing R6's config or the routing with R5 as it's not relevant here

The interesting stuff in this example happens on R5

By default the dynamic lines in the access-list are ignored - so we can see that R4 can telnet into R5's Loopback0 but nowhere else.

R5#sh ip access-list
Extended IP access list 104
    10 permit tcp host 192.168.101.4 host 5.5.5.5 eq telnet
    20 Dynamic R4Telnet permit tcp host 192.168.101.4 any eq telnet
    30 deny tcp host 192.168.101.4 any eq telnet

When Someone telnets into R5 thanks to the autocommand line in the line vty 0 4 section, the access-enable command will enable the dynamic entry in the ACL for the period of time defined (5 minutes)

R4>telnet 5.5.5.5
Trying 5.5.5.5 ... Open


User Access Verification

Username: bob
Password:
[Connection to 5.5.5.5 closed by foreign host]

Lets see what has happened with the acl now

R5#sh ip access-list
Extended IP access list 104
    10 permit tcp host 192.168.101.4 host 5.5.5.5 eq telnet (47 matches)
    20 Dynamic R4Telnet permit tcp host 192.168.101.4 any eq telnet
       permit tcp host 192.168.101.4 any eq telnet
    30 deny tcp host 192.168.101.4 any eq telnet


We can see that line 20 is now active and will take precedence to the deny in line 30

R4>telnet 6.6.6.6
Trying 6.6.6.6 ... Open


User Access Verification

Username: bob
Password:

R6>


Looks good.  So What about that timeout business?

R5#sh ip access-list
Extended IP access list 104
    10 permit tcp host 192.168.101.4 host 5.5.5.5 eq telnet (47 matches)
    20 Dynamic R4Telnet permit tcp host 192.168.101.4 any eq telnet
       permit tcp host 192.168.101.4 any eq telnet (28 matches) (time left 297)
    30 deny tcp host 192.168.101.4 any eq telnet

We can see it counting down.  If we had wanted it so that the session would automatically extend beyond the initial 5 minutes indefinitely (but still timeout after 5 minutes of no activity) we add this to R5:

R5(config)#access-list dynamic-extended

Which would restart the time-out every time new traffic matched the dynamic acl entry.

Frame Relay End-To-End Keepalive (FREEK)

Another Frame-Relay Topic...

LMI is pretty good but may not always something you can use.  Say you have back to back routers, R4 and R7 both connected to each other at S0/0 and you don't want to set one of them up to be a Frame Relay switch because sooner or later we'll be hooking them up to a WAN provider that has a Frame Switch.

It would still be very much a good idea to be able to verify that we have end to end service continuity.  Why would we care? It's handy to know if the peer has gone down since we might be using enhanced object tracking.

R4 is the Internet router for R7

R4

hostname R4
interface Serial0/0
 description R7 S0/0
 no ip address
 encapsulation frame-relay
 no keepalive
 clock rate 2000000
 no frame-relay inverse-arp
!
interface Serial0/0.16 point-to-point
 ip address 1.1.1.4 255.255.255.0
 frame-relay interface-dlci 16
!
int loopback 1
 ip address 9.9.9.9 255.255.255.255

R7
hostname R7
interface Serial0/0
 description R4 S0/0

 no ip address
 encapsulation frame-relay
 no keepalive
 no frame-relay inverse-arp
!
interface Serial0/0.16 point-to-point
 ip address 1.1.1.7 255.255.255.0
 frame-relay interface-dlci 16
!
track 1 interface s0/0.16 line-protocol
ip route 0.0.0.0 0.0.0.0 1.1.1.4 track 1

Lets validate that service is operational:

R7#sh frame pvc

PVC Statistics for interface Serial0/0 (Frame Relay DTE)

              Active     Inactive      Deleted       Static
  Local          1            0            0            0
  Switched       0            0            0            0
  Unused         0            0            0            0

DLCI = 16, DLCI USAGE = LOCAL, PVC STATUS = STATIC, INTERFACE = Serial0/0.16

  input pkts 137           output pkts 192          in bytes 13936
  out bytes 18989          dropped pkts 0           in pkts dropped 0
  out pkts dropped 0                out bytes dropped 0
  in FECN pkts 0           in BECN pkts 0           out FECN pkts 0
  out BECN pkts 0          in DE pkts 0             out DE pkts 0
  out bcast pkts 39        out bcast bytes 12714
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
  pvc create time 00:38:57, last time pvc status changed 00:06:55
R7#sh track
Track 1
  Interface Serial0/0.16 line-protocol
  Line protocol is Up
    5 changes, last change 00:06:59
  Tracked by:
    STATIC-IP-ROUTING 0
R7#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 1.1.1.4 to network 0.0.0.0

     1.0.0.0/24 is subnetted, 1 subnets
C       1.1.1.0 is directly connected, Serial0/0.16
S*   0.0.0.0/0 [1/0] via 1.1.1.4
R7#ping 9.9.9.9

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 9.9.9.9, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/5/16 ms


So far so good.  Ideally R7 would be able to withdraw the default route if the PVC to R4 fails - we can quite easily simulate this by shutting down R4's S0/0.16 interface.

R4(config)#int s0/0.16
R4(config-subif)#shut

We wait around for a few minutes and R7 is none the wiser.

R7#sh frame pvc

PVC Statistics for interface Serial0/0 (Frame Relay DTE)

              Active     Inactive      Deleted       Static
  Local          1            0            0            0
  Switched       0            0            0            0
  Unused         0            0            0            0

DLCI = 16, DLCI USAGE = LOCAL, PVC STATUS = STATIC, INTERFACE = Serial0/0.16

  input pkts 147           output pkts 205          in bytes 15810
  out bytes 21391          dropped pkts 0           in pkts dropped 0
  out pkts dropped 0                out bytes dropped 0
  in FECN pkts 0           in BECN pkts 0           out FECN pkts 0
  out BECN pkts 0          in DE pkts 0             out DE pkts 0
  out bcast pkts 44        out bcast bytes 14344
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
  pvc create time 00:43:31, last time pvc status changed 00:11:30
R7#ping 9.9.9.9

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 9.9.9.9, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

Since the physical interface stays up and we don't have LMI letting us know that there's a problem we cant determine that there is a problem with the PVC.  This is where Frame-Relay End-to-end Keepalive (FREEK) comes into play.  There are a number of parameters associated with it but the simple case will be demonstrated here:


R4/R7
map-class frame-relay FREEK
 frame-relay end-to-end keepalive mode bidirectional
interface Serial0/0.16 point-to-point
 frame-relay interface-dlci 16
  class FREEK

While R4 s0/0.16 is still shut down we can see that this appears to be working in that the link is down and the default route is no longer present.

On R7:

*Mar  1 00:49:41.687: %SYS-5-CONFIG_I: Configured from console by console
*Mar  1 00:49:59.715: %FR_EEK-5-FAILED: Interface Serial0/0.16 - DLCI 16
*Mar  1 00:49:59.727: %TRACKING-5-STATE: 1 interface Se0/0.16 line-protocol Up->Down

R7#sh frame pvc

PVC Statistics for interface Serial0/0 (Frame Relay DTE)

              Active     Inactive      Deleted       Static
  Local          0            1            0            0
  Switched       0            0            0            0
  Unused         0            0            0            0

DLCI = 16, DLCI USAGE = LOCAL, PVC STATUS = STATIC (EEK DOWN), INTERFACE = Serial0/0.16

  input pkts 147           output pkts 225          in bytes 15810
  out bytes 24073          dropped pkts 0           in pkts dropped 0
  out pkts dropped 0                out bytes dropped 0
  in FECN pkts 0           in BECN pkts 0           out FECN pkts 0
  out BECN pkts 0          in DE pkts 0             out DE pkts 0
  out bcast pkts 49        out bcast bytes 15974
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
  pvc create time 00:48:21, last time pvc status changed 00:00:12
R7#sh frame end-to-end keepalive

End-to-end Keepalive Statistics for Interface Serial0/0 (Frame Relay DTE)

DLCI = 16, DLCI USAGE = LOCAL, VC STATUS = STATIC (EEK DOWN)

SEND SIDE STATISTICS

Send Sequence Number: 255,      Receive Sequence Number: 0
Configured Event Window: 3,     Configured Error Threshold: 2
Total Observed Events: 4,       Total Observed Errors: 4
Monitored Events: 3,            Monitored Errors: 3
Successive Successes: 0,        End-to-end VC Status: DOWN

RECEIVE SIDE STATISTICS

Send Sequence Number: 0,        Receive Sequence Number: 0
Configured Event Window: 3,     Configured Error Threshold: 2
Total Observed Events: 2,       Total Observed Errors: 2
Monitored Events: 2,            Monitored Errors: 2
Successive Successes: 0,        End-to-end VC Status: DOWN

Failures Since Started: 1,      Last Failure: 00:00:23
R7#sh track
Track 1
  Interface Serial0/0.16 line-protocol
  Line protocol is Down (sw down)
    6 changes, last change 00:00:26
  Tracked by:
    STATIC-IP-ROUTING 0

R7#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

So Lets repair the fault:

R4(config)#int s0/0.16
R4(config-subif)#no shut

On R7:
*Mar  1 00:52:49.131: %TRACKING-5-STATE: 1 interface Se0/0.16 line-protocol Down->Up
R7#sh frame pvc

PVC Statistics for interface Serial0/0 (Frame Relay DTE)

              Active     Inactive      Deleted       Static
  Local          1            0            0            0
  Switched       0            0            0            0
  Unused         0            0            0            0

DLCI = 16, DLCI USAGE = LOCAL, PVC STATUS = STATIC (EEK UP), INTERFACE = Serial0/0.16

  input pkts 157           output pkts 250          in bytes 16892
  out bytes 25205          dropped pkts 0           in pkts dropped 0
  out pkts dropped 0                out bytes dropped 0
  in FECN pkts 0           in BECN pkts 0           out FECN pkts 0
  out BECN pkts 0          in DE pkts 0             out DE pkts 0
  out bcast pkts 52        out bcast bytes 16952
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
  pvc create time 00:51:26, last time pvc status changed 00:00:27
R7#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 1.1.1.4 to network 0.0.0.0

     1.0.0.0/24 is subnetted, 1 subnets
C       1.1.1.0 is directly connected, Serial0/0.16
S*   0.0.0.0/0 [1/0] via 1.1.1.4
R7#ping 9.9.9.9

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 9.9.9.9, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/5/16 ms

So that's the basics on FREEK.

Multilink Frame Relay (MFR) and Multilink PPP (ML-PPP)

This post is about non-etherchannel link bundling a way to increase bandwidth and resiliency with the layer 3 result appearing as a single Layer 3 link between nodes using multiple WAN links.

So lets start with Routers R4 and R7, each has 4 serial interfaces that are back to back (R4 s0/0 to R7 s0/0, R4 s0/1 to R7 s0/1, R4 s1/0 to R7 s1/0 and R4 s1/1 to R7 s1/1)

Start with a multilink frame relay (MFR) for the first two links (since we're not using a Frame Switch here so we're going to turn FR LMI off)

R4
hostname R4
int mfr 1
 no keepalive
 no shut
int s0/0
 shutdown
 encapsulation frame-relay mfr 1
 no shut
int s0/1
 shutdown
 encapsulation frame-relay mfr 1
 no shut
int mfr1.47 point-to-point
 ip address 47.47.47.4 255.255.255.0
 frame-relay interface-dlci 47


R7
hostname R7
int mfr 1
 no keepalive
 no shut
int s0/0
 shutdown
 encapsulation frame-relay mfr 1
 no shut
int s0/1
 shutdown
 encapsulation frame-relay mfr 1
 no shut
int mfr1.47 point-to-point
 ip address 47.47.47.7 255.255.255.0
 frame-relay interface-dlci 47


So lets check that is operating.

R4#sh frame pvc

PVC Statistics for interface MFR1 (Frame Relay DTE)

              Active     Inactive      Deleted       Static
  Local          1            0            0            0
  Switched       0            0            0            0
  Unused         0            0            0            0

DLCI = 47, DLCI USAGE = LOCAL, PVC STATUS = STATIC, INTERFACE = MFR1.47

  input pkts 3             output pkts 3            in bytes 963
  out bytes 963            dropped pkts 0           in pkts dropped 0
  out pkts dropped 0                out bytes dropped 0
  in FECN pkts 0           in BECN pkts 0           out FECN pkts 0
  out BECN pkts 0          in DE pkts 0             out DE pkts 0
  out bcast pkts 3         out bcast bytes 963
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
  pvc create time 00:00:28, last time pvc status changed 00:00:14
R4#ping 47.47.47.7

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 47.47.47.7, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/5/16 ms
R4#sh frame-relay multilink
Bundle: MFR1, State = up, class = A, fragmentation disabled
 BID = MFR1
 Bundle links:
  Serial0/1, HW state = up, link state = Up, LID = Serial0/1
  Serial0/0, HW state = up, link state = Up, LID = Serial0/0

Okay, so multilink frame relay is relatively straight forward to get going.  Lets park that for the moment and look at getting multilink PPP operational

R4
int mfr1.47 point-to-point
 no ip address
interface Multilink1
 ip address 47.47.47.4 255.255.255.0
 ppp multilink
 ppp multilink group 1
int s1/0
 shutdown
 encapsulation ppp
 ppp multilink
 ppp multilink group 1
 no shut
int s1/1
 shutdown
 encapsulation ppp
 ppp multilink
 ppp multilink group 1
 no shut

R7
int mfr1.47 point-to-point
 no ip address
interface Multilink1
 ip address 47.47.47.7 255.255.255.0
 ppp multilink
 ppp multilink group 1
int s1/0
 shutdown
 encapsulation ppp
 ppp multilink
 ppp multilink group 1
 no shut
int s1/1
 shutdown
 encapsulation ppp
 ppp multilink
 ppp multilink group 1
 no shut

So lets check that this is working.
 
R7#ping 47.47.47.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 47.47.47.4, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/9/24 ms
R7#sh ppp multi

Multilink1
  Bundle name: R4
  Remote Endpoint Discriminator: [1] R4
  Local Endpoint Discriminator: [1] R7
  Bundle up for 00:00:26, total bandwidth 3088, load 1/255
  Receive buffer limit 24000 bytes, frag timeout 1000 ms
    0/0 fragments/bytes in reassembly list
    0 lost fragments, 7 reordered
    0/0 discarded fragments/bytes, 0 lost received
    0xF received sequence, 0xD sent sequence
  Member links: 2 active, 0 inactive (max not set, min not set)
    Se1/0, since 00:00:21
    Se1/1, since 00:00:20
No inactive multilink interfaces

No problems, we can see that the bandwidth is 3088kbps (1544 kbps default per serial interface). Ok, so just for fun, how about including the Frame Relay links with this bundle?  It's certainly possible, we just need to put ppp over frame and include it in the multilink group.

R4/47
interface Virtual-Template1
 no ip address
 ppp multilink
 ppp multilink group 1
interface MFR1.47 point-to-point
 frame-relay interface-dlci 47 ppp Virtual-Template1

However it's not quite as simple as it seems to get it going..

*Mar  1 00:14:13.559: %LINK-3-UPDOWN: Interface Virtual-Access2, changed state to up
*Mar  1 00:14:25.547: %FR-3-MLPOFR_ERROR: MLPoFR not configured properly on Link Virtual-Access2 Bundle Multilink1 :Frame Relay traffic shaping must be enabled
*Mar  1 00:14:26.539: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access2, changed state to up

We need some traffic shaping on the FR (MFR1) interface - remember if we just turn traffic shaping on, all PVCs will be shaped at 56kbps.  For this example, we'll assume that there is 256kbps per PVC available

R4/47
map-class frame-relay MFR
 frame-relay traffic-rate 512000
interface MFR1
 frame-relay traffic-shaping
interface MFR1.47 point-to-point
 frame-relay interface-dlci 47 ppp Virtual-Template1
 class MFR

Lets bounce the link and see if it's happy this time round.

R7(config-if)#int mfr1
R7(config-if)#shut
*Mar  1 00:22:30.247: %LINK-5-CHANGED: Interface MFR1, changed state to administratively down
*Mar  1 00:22:30.247: %LINK-3-UPDOWN: Interface Virtual-Access2, changed state to down
*Mar  1 00:22:31.247: %LINEPROTO-5-UPDOWN: Line protocol on Interface MFR1, changed state to down
*Mar  1 00:22:31.251: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access2, changed state to down
*Mar  1 00:22:33.255: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/1, changed state to down
*Mar  1 00:22:33.259: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/0, changed state to down
R7(config-if)#no shut
*Mar  1 00:22:46.619: %LINK-3-UPDOWN: Interface MFR1, changed state to down
*Mar  1 00:22:54.215: %LINK-3-UPDOWN: Interface MFR1, changed state to up
*Mar  1 00:22:54.263: %LINK-3-UPDOWN: Interface Virtual-Access2, changed state to up
*Mar  1 00:22:55.207: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/1, changed state to up
*Mar  1 00:22:55.215: %LINEPROTO-5-UPDOWN: Line protocol on Interface MFR1, changed state to up
*Mar  1 00:22:55.219: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/0, changed state to up
*Mar  1 00:22:55.263: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access2, changed state to up

No complaints this time...

R7(config-if)#do ping 47.47.47.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 47.47.47.4, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/6/16 ms
R7(config-if)#do sh users
    Line       User       Host(s)              Idle       Location
*  0 con 0                idle                 00:00:00

  Interface    User               Mode         Idle     Peer Address
  Vi2                             PPPoFR       00:01:06
  Se1/0                           Sync PPP     00:00:05
  Se1/1                           Sync PPP     00:00:05
  Mu1                             Sync PPP     00:00:54 47.47.47.4

Looks good so far, lets confirm what the multilink PPP interface thinks its bandwidth is (it should be 1544 * 2 for the direct PPP links and 512 for the MFR link)

R7(config-if)#do sh ppp multi

Multilink1
  Bundle name: R4
  Remote Endpoint Discriminator: [1] R4
  Local Endpoint Discriminator: [1] R7
  Bundle up for 00:24:11, total bandwidth 3344, load 1/255
  Receive buffer limit 36000 bytes, frag timeout 1000 ms
    0/0 fragments/bytes in reassembly list
    0 lost fragments, 13 reordered
    0/0 discarded fragments/bytes, 0 lost received
    0x32 received sequence, 0x30 sent sequence
  Member links: 3 active, 1 inactive (max not set, min not set)
    Se1/0, since 00:24:06, 5790 weight, 1496 frag size
    Se1/1, since 00:24:05, 5790 weight, 1496 frag size
    Vi2, since 00:04:21, 960 weight, 950 frag size
    Vt1 (inactive)
No inactive multilink interfaces

We can see that the multilink interface thinks it has 3344 kbps of bandwidth and not 3600.

R7#sh int virtual-access 2
Virtual-Access2 is up, line protocol is up
  Hardware is Virtual Access interface
  MTU 1500 bytes, BW 256 Kbit/sec, DLY 100000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation PPP, LCP Open, multilink Open
  Link is a member of Multilink bundle Multilink1
  PPPoFR vaccess, cloned from Virtual-Template1
  Vaccess status 0x44
  Bound to MFR1.47 DLCI 47, Cloned from Virtual-Template1, loopback not set
  Keepalive set (10 sec)
  DTR is pulsed for 5 seconds on reset
  Last input 00:02:02, output never, output hang never
  Last clearing of "show interface" counters 00:20:59
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     253 packets input, 5890 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     254 packets output, 5393 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions

Virtual-Access2 believes that there is 256kbps of bandwidth available rather than the expected 512kbps.  Lets modify the bandwidth associated with Virtual-Template 1 (since the Virtual-Access interface inherits data from it) and see if that helps.

R7(config-if)#int virtual-template 1
R7(config-if)#bandwidth 512
R7(config-if)#int mfr1
R7(config-if)#shut
R7(config-if)#no shut
*Mar  1 00:40:48.283: %LINK-5-CHANGED: Interface MFR1, changed state to administratively down
*Mar  1 00:40:48.287: %LINK-3-UPDOWN: Interface Virtual-Access2, changed state to down
*Mar  1 00:40:49.295: %LINEPROTO-5-UPDOWN: Line protocol on Interface MFR1, changed state to down
*Mar  1 00:40:49.299: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access2, changed state to down
*Mar  1 00:40:49.647: %LINK-3-UPDOWN: Interface MFR1, changed state to down
*Mar  1 00:40:51.295: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/1, changed state to down
*Mar  1 00:40:51.299: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/0, changed state to down
*Mar  1 00:40:58.259: %LINK-3-UPDOWN: Interface MFR1, changed state to up
*Mar  1 00:40:58.311: %LINK-3-UPDOWN: Interface Virtual-Access2, changed state to up
*Mar  1 00:40:59.251: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/1, changed state to up
*Mar  1 00:40:59.259: %LINEPROTO-5-UPDOWN: Line protocol on Interface MFR1, changed state to up
*Mar  1 00:40:59.267: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/0, changed state to up
*Mar  1 00:40:59.311: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access2, changed state to up
R7(config-if)#do sh ppp multi

Multilink1
  Bundle name: R4
  Remote Endpoint Discriminator: [1] R4
  Local Endpoint Discriminator: [1] R7
  Bundle up for 00:32:49, total bandwidth 3600, load 1/255
  Receive buffer limit 36000 bytes, frag timeout 1000 ms
    0/0 fragments/bytes in reassembly list
    0 lost fragments, 13 reordered
    0/0 discarded fragments/bytes, 0 lost received
    0x3A received sequence, 0x38 sent sequence
  Member links: 3 active, 1 inactive (max not set, min not set)
    Se1/0, since 00:32:44, 5790 weight, 1496 frag size
    Se1/1, since 00:32:43, 5790 weight, 1496 frag size
    Vi2, since 00:00:09, 1920 weight, 1496 frag size
    Vt1 (inactive)
No inactive multilink interfaces

Okay, repeat the same thing on R4 and that should be it.  Would anyone want to do this in the real world?  I certainly hope not!

Routing Protocol Redistribution using Tags and admin distance

Generally when I have been playing around with redistribution I have only been using route tags to eliminate routing loops, however this may not always work.

Below is a redistribution lab topology I have been recently playing with.

The base configurations prior to doing redistribution are shown below:

R4
hostname R4
!
interface Loopback0
 ip address 4.4.4.4 255.255.255.255
!
interface Serial0/0
 description R5 S0/0
 ip address 192.168.45.4 255.255.255.0
 clock rate 2000000
!
interface FastEthernet0/1
 description R6 Fa0/0
 ip address 192.168.46.4 255.255.255.0
 duplex auto
 speed auto
!
router eigrp 45
 network 4.4.4.4 0.0.0.0
 network 192.168.45.4 0.0.0.0
 no auto-summary
!
router ospf 456
 log-adjacency-changes
 network 4.4.4.4 0.0.0.0 area 46
 network 192.168.46.4 0.0.0.0 area 46
!


R5
hostname R5
!
interface Loopback0
 ip address 5.5.5.5 255.255.255.255
!
interface Serial0/0
 description R4 S0/0
 ip address 192.168.45.5 255.255.255.0
!
interface FastEthernet0/1
 description R6 Fa0/1
 ip address 192.168.56.5 255.255.255.0
 duplex auto
 speed auto
!
router eigrp 45
 network 5.5.5.5 0.0.0.0
 network 192.168.45.5 0.0.0.0
 no auto-summary
!
router ospf 456
 log-adjacency-changes
 network 5.5.5.5 0.0.0.0 area 56
 network 192.168.56.5 0.0.0.0 area 56
!


R6
hostname R6
!
interface Loopback0
 ip address 6.6.6.6 255.255.255.255
!
interface FastEthernet0/0
 description R4 Fa0/1
 ip address 192.168.46.6 255.255.255.0
 duplex auto
 speed auto
!
interface FastEthernet0/1
 description R5 Fa0/1
 ip address 192.168.56.6 255.255.255.0
 duplex auto
 speed auto
!
interface Serial0/0
 description R7 S0/0
 ip address 192.168.67.6 255.255.255.0
 clock rate 2000000
!
router ospf 456
 log-adjacency-changes
 network 6.6.6.6 0.0.0.0 area 0
 network 192.168.46.6 0.0.0.0 area 46
 network 192.168.56.6 0.0.0.0 area 56
!
router rip
 version 2
 network 6.0.0.0
 network 192.168.67.0
 no auto-summary
!

R7
hostname R7
!
interface Loopback0
 ip address 7.7.7.7 255.255.255.255
!
interface Serial0/0
 description R6 S0/0
 ip address 192.168.67.7 255.255.255.0
!
router rip
 version 2
 network 7.0.0.0
 network 192.168.67.0
 no auto-summary
!

In this example, I want to have mutual redistribution across all protocols to ensure connectivity regardless as to which link fails.

Redistribution happens at three locations - R6: Between RIP and OSPF while both R4 and R5 redistribute between OSPF and EIGRP

Route-Tags shall be used to ensure that routes originated by a routing protocol are not injected back into it.  To keep things straight forward, I am using the default admin distances for the protocols for their tags.

R6

route-map RIP2OSPF deny 10
 match tag 110
!
route-map RIP2OSPF permit 20
 match tag 90
 set tag 90
!
route-map RIP2OSPF permit 30
 set tag 120

route-map OSPF2RIP deny 10
 match tag 120
!
route-map OSPF2RIP permit 20
 match tag 90
 set tag 90
!
route-map OSPF2RIP permit 30
 set tag 110
!
router rip
 redistribute ospf 456 route-map OSPF2RIP metric 3
router ospf 456
 redistribute rip subnets route-map RIP2OSPF metric 100

R4/R5

route-map OSPF2EIGRP deny 10
 match tag 90
!
route-map OSPF2EIGRP permit 20
 match tag 120
 set tag 120
!
route-map OSPF2EIGRP permit 30
 set tag 110
!
route-map EIGRP2OSPF deny 10
 match tag 110
!
route-map EIGRP2OSPF permit 20
 match tag 120
 set tag 120
!
route-map EIGRP2OSPF permit 30
 set tag 90
!
router eigrp 45
 redistribute ospf 456 route-map OSPF2EIGRP metric 100000 1 255 1 1500
router ospf 456
 redistribute eigrp 45 subnets route-map EIGRP2OSPF metric 100


So far everything should be fine in the quiescent state.  However if we play around with links it is possible to introduce some problems (whenever working on a redistribution lab performing "debug ip routing" can be quite helpful)

If bounce a link between R6 and R5 we can see that the route to R7 Loopback disappears

R6#sh ip route 7.7.7.7
Routing entry for 7.7.7.7/32
  Known via "rip", distance 120, metric 1
  Redistributing via ospf 456, rip
  Advertised by ospf 456 metric 100 subnets route-map RIP2OSPF
  Last update from 192.168.67.7 on Serial0/0, 00:00:01 ago
  Routing Descriptor Blocks:
  * 192.168.67.7, from 192.168.67.7, 00:00:01 ago, via Serial0/0
      Route metric is 1, traffic share count is 1

R6#debug ip routing
IP routing debugging is on
R6#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R6(config)#int fa0/1
R6(config-if)#shut
*Mar  1 00:16:34.343: RT: is_up: FastEthernet0/1 0 state: 6 sub state: 1 line: 1 has_route: True
*Mar  1 00:16:34.343: %OSPF-5-ADJCHG: Process 456, Nbr 5.5.5.5 on FastEthernet0/1 from FULL to DOWN, Neighbor Down: Interface down or detached
*Mar  1 00:16:34.343: RT: interface FastEthernet0/1 removed from routing table
*Mar  1 00:16:34.343: RT: del 192.168.56.0 via 0.0.0.0, connected metric [0/0]
*Mar  1 00:16:34.343: RT: delete network route to 192.168.56.0
*Mar  1 00:16:34.347: RT: NET-RED 192.168.56.0/24
*Mar  1 00:16:34.347: RT: Pruning routes for FastEthernet0/1 (2)
*Mar  1 00:16:34.351: RT: delete route to 192.168.45.0 via 192.168.56.5, FastEthernet0/1
*Mar  1 00:16:34.351: RT: NET-RED 192.168.45.0/24
*Mar  1 00:16:34.351: RT: delete route to 5.5.5.5 via 192.168.56.5, FastEthernet0/1
*Mar  1 00:16:34.351: RT: no routes to 5.5.5.5, flushing
*Mar  1 00:16:34.351: RT: NET-RED 5.5.5.5/32
*Mar  1 00:16:34.351: RT: delete network route to 5.0.0.0
*Mar  1 00:16:34.351: RT: NET-RED 5.0.0.0/8
*Mar  1 00:16:36.327: %LINK-5-CHANGED: Interface FastEthernet0/1, changed state to administratively down
*Mar  1 00:16:36.335: RT: is_up: FastEthernet0/1 0 state: 6 sub state: 1 line: 1 has_route: False
*Mar  1 00:16:37.327: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
*Mar  1 00:16:37.335: RT: is_up: FastEthernet0/1 0 state: 6 sub state: 1 line: 1 has_route: False
*Mar  1 00:16:39.859: RT: add 5.5.5.5/32 via 192.168.46.4, ospf metric [110/100]
*Mar  1 00:16:39.863: RT: NET-RED 5.5.5.5/32

R6(config-if)#no shut
*Mar  1 00:16:46.307: RT: is_up: FastEthernet0/1 1 state: 4 sub state: 1 line: 1 has_route: False
*Mar  1 00:16:46.311: RT: add 192.168.56.0/24 via 0.0.0.0, connected metric [0/0]
*Mar  1 00:16:46.311: RT: NET-RED 192.168.56.0/24
*Mar  1 00:16:46.311: RT: interface FastEthernet0/1 added to routing table
*Mar  1 00:16:48.283: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up
*Mar  1 00:16:48.291: RT: is_up: FastEthernet0/1 1 state: 4 sub state: 1 line: 1 has_route: True
*Mar  1 00:16:49.283: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
*Mar  1 00:16:49.291: RT: is_up: FastEthernet0/1 1 state: 4 sub state: 1 line: 1 has_route: True
*Mar  1 00:16:50.959: %OSPF-5-ADJCHG: Process 456, Nbr 5.5.5.5 on FastEthernet0/1 from LOADING to FULL, Loading Done
*Mar  1 00:16:59.883: RT: del 5.5.5.5/32 via 192.168.46.4, ospf metric [110/100]
*Mar  1 00:16:59.887: RT: add 5.5.5.5/32 via 192.168.56.5, ospf metric [110/11]
*Mar  1 00:16:59.891: RT: NET-RED 5.5.5.5/32
*Mar  1 00:16:59.895: RT: closer admin distance for 7.7.7.7, flushing 1 routes
*Mar  1 00:16:59.899: RT: NET-RED 7.7.7.7/32
*Mar  1 00:16:59.899: RT: add 7.7.7.7/32 via 192.168.56.5, ospf metric [110/100]
*Mar  1 00:16:59.899: RT: NET-RED 7.7.7.7/32
*Mar  1 00:16:59.899: RT: add 192.168.45.0/24 via 192.168.56.5, ospf metric [110/100]
*Mar  1 00:16:59.899: RT: NET-RED 192.168.45.0/24

R6(config-if)#do sh ip route 7.7.7.7
Routing entry for 7.7.7.7/32
  Known via "ospf 456", distance 110, metric 100
  Tag 120, type extern 2, forward metric 10
  Redistributing via rip
  Last update from 192.168.56.5 on FastEthernet0/1, 00:00:16 ago
  Routing Descriptor Blocks:
  * 192.168.56.5, from 5.5.5.5, 00:00:16 ago, via FastEthernet0/1
      Route metric is 100, traffic share count is 1
      Route tag 120

On R6 we can see that the route to 7.7.7.7 is now being learnt through OSPF even though we were the one to put it there in the first place!  The issue here is because of the administration distance of the route.  RIP has an admin distance of 120, OSPF 110 and EIGRP 90.

On R6 our RIP route from R7 is redistributed into OSPF to R4 and R5.  When the link between R6 and R5 is brought down, the only way R5 knows about this route is when it is redistributed by R4 into EIGRP.
When the link between R6 and R5 comes back into service, that route is redistributed from EIGRP into OSPF by R5 and advertised to R6.  For the same route, the lower administration distance, the more belivable and therefore the RIP route is ignored (as can be seen by the debug above)

R6(config-if)#do ping 7.7.7.7

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 7.7.7.7, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

It is no surprise that that the ping to R7 failed, since R6 is the adjacent router to R7 and thinks the way there is via R5 - traffic to 7.7.7.7 will zoom around the network until the TTL expires

If we reset the OSPF process on R6 - things can return to the normal state (the route for 7.7.7.7 originated into OSPF will be withdrawn from EIGRP by R4 and wont be advertised back into OSPF by R5)

R6(config-if)#do clear ip ospf proc
Reset ALL OSPF processes? [no]: yes
*Mar  1 00:19:24.267: %OSPF-5-ADJCHG: Process 456, Nbr 4.4.4.4 on FastEthernet0/0 from FULL to DOWN, Neighbor Down: Interface down or detached
*Mar  1 00:19:24.323: %OSPF-5-ADJCHG: Process 456, Nbr 5.5.5.5 on FastEthernet0/1 from FULL to DOWN, Neighbor Down: Interface down or detached
*Mar  1 00:19:24.331: RT: delete route to 192.168.45.0/24
*Mar  1 00:19:24.331: RT: NET-RED 192.168.45.0/24
*Mar  1 00:19:24.335: RT: delete route to 4.4.4.4/32
*Mar  1 00:19:24.335: RT: NET-RED 4.4.4.4/32
*Mar  1 00:19:24.335: RT: delete network route to 4.0.0.0
*Mar  1 00:19:24.339: RT: NET-RED 4.0.0.0/8
*Mar  1 00:19:24.355: RT: delete route to 5.5.5.5/32
*Mar  1 00:19:24.359: RT: NET-RED 5.5.5.5/32
*Mar  1 00:19:24.359: RT: delete network route to 5.0.0.0
*Mar  1 00:19:24.363: RT: NET-RED 5.0.0.0/8
*Mar  1 00:19:24.371: RT: delete route to 7.7.7.7/32
*Mar  1 00:19:24.371: RT: NET-RED 7.7.7.7/32
*Mar  1 00:19:24.371: RT: delete network route to 7.0.0.0
*Mar  1 00:19:24.371: RT: NET-RED 7.0.0.0/8
*Mar  1 00:19:24.551: %OSPF-5-ADJCHG: Process 456, Nbr 5.5.5.5 on FastEthernet0/1 from LOADING to FULL, Loading Done
*Mar  1 00:19:24.551: %OSPF-5-ADJCHG: Process 456, Nbr 4.4.4.4 on FastEthernet0/0 from LOADING to FULL, Loading Done
*Mar  1 00:19:29.539: RT: add 4.4.4.4/32 via 192.168.46.4, ospf metric [110/11]
*Mar  1 00:19:29.539: RT: NET-RED 4.4.4.4/32
*Mar  1 00:19:29.543: RT: add 5.5.5.5/32 via 192.168.56.5, ospf metric [110/11]
*Mar  1 00:19:29.543: RT: NET-RED 5.5.5.5/32
*Mar  1 00:19:29.547: RT: add 192.168.45.0/24 via 192.168.56.5, ospf metric [110/100]
*Mar  1 00:19:29.547: RT: NET-RED 192.168.45.0/24
*Mar  1 00:19:29.547: RT: add 192.168.45.0/24 via 192.168.46.4, ospf metric [110/100]
*Mar  1 00:19:29.547: RT: NET-RED 192.168.45.0/24
*Mar  1 00:19:49.191: RT: add 7.7.7.7/32 via 192.168.67.7, rip metric [120/1]
*Mar  1 00:19:49.195: RT: NET-RED 7.7.7.7/32


We can see that this fixes things (until the next time we lose one of the OSPF links and it gets restored) We know that the problem is related to the route believability (Admin Distance) so the obvious answer is to modify it.

We have at least two ways to do so:
 Increase the Admin Distance of OSPF
 Decrease the Admin Distance for RIP

Related to the second, I will decrease the admin distance for a specific route (7.7.7.7/32)


R6(config-if)#access-list 1 permit host 7.7.7.7
R6(config)#router rip
R6(config-router)#distance 105 192.168.67.7 0.0.0.0 1

Not too long after after applying this change we can see:

*Mar  1 00:22:10.359: RT: closer admin distance for 7.7.7.7, flushing 1 routes
*Mar  1 00:22:10.363: RT: add 7.7.7.7/32 via 192.168.67.7, rip metric [105/1]
*Mar  1 00:22:10.367: RT: NET-RED 7.7.7.7/32

105 is less than 110, so R6 will not insert the OSPF route into the RIB - lets verify this by bouncing the link between R5 and R6 again:

R6(config-router)#int fa0/1
R6(config-if)#shut
*Mar  1 00:25:56.751: RT: is_up: FastEthernet0/1 0 state: 6 sub state: 1 line: 1 has_route: True
*Mar  1 00:25:56.755: %OSPF-5-ADJCHG: Process 456, Nbr 5.5.5.5 on FastEthernet0/1 from FULL to DOWN, Neighbor Down: Interface down or detached
*Mar  1 00:25:56.763: RT: interface FastEthernet0/1 removed from routing table
*Mar  1 00:25:56.767: RT: del 192.168.56.0 via 0.0.0.0, connected metric [0/0]
*Mar  1 00:25:56.767: RT: delete network route to 192.168.56.0
*Mar  1 00:25:56.767: RT: NET-RED 192.168.56.0/24
*Mar  1 00:25:56.767: RT: Pruning routes for FastEthernet0/1 (2)
*Mar  1 00:25:56.771: RT: delete route to 192.168.45.0 via 192.168.56.5, FastEthernet0/1
*Mar  1 00:25:56.771: RT: NET-RED 192.168.45.0/24
*Mar  1 00:25:56.771: RT: delete route to 5.5.5.5 via 192.168.56.5, FastEthernet0/1
*Mar  1 00:25:56.771: RT: no routes to 5.5.5.5, flushing
*Mar  1 00:25:56.771: RT: NET-RED 5.5.5.5/32
*Mar  1 00:25:56.771: RT: delete network route to 5.0.0.0
*Mar  1 00:25:56.771: RT: NET-RED 5.0.0.0/8
*Mar  1 00:25:58.727: %LINK-5-CHANGED: Interface FastEthernet0/1, changed state to administratively down
*Mar  1 00:25:58.735: RT: is_up: FastEthernet0/1 0 state: 6 sub state: 1 line: 1 has_route: False
*Mar  1 00:25:59.727: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
*Mar  1 00:25:59.735: RT: is_up: FastEthernet0/1 0 state: 6 sub state: 1 line: 1 has_route: False
*Mar  1 00:26:02.275: RT: add 5.5.5.5/32 via 192.168.46.4, ospf metric [110/100]
*Mar  1 00:26:02.275: RT: NET-RED 5.5.5.5/32

R6(config-if)#no shut
*Mar  1 00:26:29.059: RT: is_up: FastEthernet0/1 1 state: 4 sub state: 1 line: 1 has_route: False
*Mar  1 00:26:29.059: RT: add 192.168.56.0/24 via 0.0.0.0, connected metric [0/0]
*Mar  1 00:26:29.063: RT: NET-RED 192.168.56.0/24
*Mar  1 00:26:29.063: RT: interface FastEthernet0/1 added to routing table
*Mar  1 00:26:31.023: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up
*Mar  1 00:26:31.027: RT: is_up: FastEthernet0/1 1 state: 4 sub state: 1 line: 1 has_route: True
*Mar  1 00:26:32.023: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
*Mar  1 00:26:32.031: RT: is_up: FastEthernet0/1 1 state: 4 sub state: 1 line: 1 has_route: True
*Mar  1 00:26:33.995: %OSPF-5-ADJCHG: Process 456, Nbr 5.5.5.5 on FastEthernet0/1 from LOADING to FULL, Loading Done
*Mar  1 00:26:44.575: RT: del 5.5.5.5/32 via 192.168.46.4, ospf metric [110/100]
*Mar  1 00:26:44.579: RT: add 5.5.5.5/32 via 192.168.56.5, ospf metric [110/11]
*Mar  1 00:26:44.579: RT: NET-RED 5.5.5.5/32
*Mar  1 00:26:44.579: RT: add 192.168.45.0/24 via 192.168.56.5, ospf metric [110/100]
*Mar  1 00:26:44.583: RT: NET-RED 192.168.45.0/24

As we can see there has been no change to the routing table for 7.7.7.7

R6(config-if)#do sh ip route 7.7.7.7
Routing entry for 7.7.7.7/32
  Known via "rip", distance 105, metric 1
  Redistributing via ospf 456, rip
  Advertised by ospf 456 metric 100 subnets route-map RIP2OSPF
  Last update from 192.168.67.7 on Serial0/0, 00:00:25 ago
  Routing Descriptor Blocks:
  * 192.168.67.7, from 192.168.67.7, 00:00:25 ago, via Serial0/0
      Route metric is 1, traffic share count is 1

R6(config-if)#do ping 7.7.7.7

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 7.7.7.7, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

While using route tags is one of the tools to control routing loops, understanding the admin distance is also worthwhile knowing.