Multilink Frame Relay (MFR) and Multilink PPP (ML-PPP)

This post is about non-etherchannel link bundling a way to increase bandwidth and resiliency with the layer 3 result appearing as a single Layer 3 link between nodes using multiple WAN links.

So lets start with Routers R4 and R7, each has 4 serial interfaces that are back to back (R4 s0/0 to R7 s0/0, R4 s0/1 to R7 s0/1, R4 s1/0 to R7 s1/0 and R4 s1/1 to R7 s1/1)

Start with a multilink frame relay (MFR) for the first two links (since we're not using a Frame Switch here so we're going to turn FR LMI off)

R4
hostname R4
int mfr 1
 no keepalive
 no shut
int s0/0
 shutdown
 encapsulation frame-relay mfr 1
 no shut
int s0/1
 shutdown
 encapsulation frame-relay mfr 1
 no shut
int mfr1.47 point-to-point
 ip address 47.47.47.4 255.255.255.0
 frame-relay interface-dlci 47


R7
hostname R7
int mfr 1
 no keepalive
 no shut
int s0/0
 shutdown
 encapsulation frame-relay mfr 1
 no shut
int s0/1
 shutdown
 encapsulation frame-relay mfr 1
 no shut
int mfr1.47 point-to-point
 ip address 47.47.47.7 255.255.255.0
 frame-relay interface-dlci 47


So lets check that is operating.

R4#sh frame pvc

PVC Statistics for interface MFR1 (Frame Relay DTE)

              Active     Inactive      Deleted       Static
  Local          1            0            0            0
  Switched       0            0            0            0
  Unused         0            0            0            0

DLCI = 47, DLCI USAGE = LOCAL, PVC STATUS = STATIC, INTERFACE = MFR1.47

  input pkts 3             output pkts 3            in bytes 963
  out bytes 963            dropped pkts 0           in pkts dropped 0
  out pkts dropped 0                out bytes dropped 0
  in FECN pkts 0           in BECN pkts 0           out FECN pkts 0
  out BECN pkts 0          in DE pkts 0             out DE pkts 0
  out bcast pkts 3         out bcast bytes 963
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
  pvc create time 00:00:28, last time pvc status changed 00:00:14
R4#ping 47.47.47.7

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 47.47.47.7, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/5/16 ms
R4#sh frame-relay multilink
Bundle: MFR1, State = up, class = A, fragmentation disabled
 BID = MFR1
 Bundle links:
  Serial0/1, HW state = up, link state = Up, LID = Serial0/1
  Serial0/0, HW state = up, link state = Up, LID = Serial0/0

Okay, so multilink frame relay is relatively straight forward to get going.  Lets park that for the moment and look at getting multilink PPP operational

R4
int mfr1.47 point-to-point
 no ip address
interface Multilink1
 ip address 47.47.47.4 255.255.255.0
 ppp multilink
 ppp multilink group 1
int s1/0
 shutdown
 encapsulation ppp
 ppp multilink
 ppp multilink group 1
 no shut
int s1/1
 shutdown
 encapsulation ppp
 ppp multilink
 ppp multilink group 1
 no shut

R7
int mfr1.47 point-to-point
 no ip address
interface Multilink1
 ip address 47.47.47.7 255.255.255.0
 ppp multilink
 ppp multilink group 1
int s1/0
 shutdown
 encapsulation ppp
 ppp multilink
 ppp multilink group 1
 no shut
int s1/1
 shutdown
 encapsulation ppp
 ppp multilink
 ppp multilink group 1
 no shut

So lets check that this is working.
 
R7#ping 47.47.47.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 47.47.47.4, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/9/24 ms
R7#sh ppp multi

Multilink1
  Bundle name: R4
  Remote Endpoint Discriminator: [1] R4
  Local Endpoint Discriminator: [1] R7
  Bundle up for 00:00:26, total bandwidth 3088, load 1/255
  Receive buffer limit 24000 bytes, frag timeout 1000 ms
    0/0 fragments/bytes in reassembly list
    0 lost fragments, 7 reordered
    0/0 discarded fragments/bytes, 0 lost received
    0xF received sequence, 0xD sent sequence
  Member links: 2 active, 0 inactive (max not set, min not set)
    Se1/0, since 00:00:21
    Se1/1, since 00:00:20
No inactive multilink interfaces

No problems, we can see that the bandwidth is 3088kbps (1544 kbps default per serial interface). Ok, so just for fun, how about including the Frame Relay links with this bundle?  It's certainly possible, we just need to put ppp over frame and include it in the multilink group.

R4/47
interface Virtual-Template1
 no ip address
 ppp multilink
 ppp multilink group 1
interface MFR1.47 point-to-point
 frame-relay interface-dlci 47 ppp Virtual-Template1

However it's not quite as simple as it seems to get it going..

*Mar  1 00:14:13.559: %LINK-3-UPDOWN: Interface Virtual-Access2, changed state to up
*Mar  1 00:14:25.547: %FR-3-MLPOFR_ERROR: MLPoFR not configured properly on Link Virtual-Access2 Bundle Multilink1 :Frame Relay traffic shaping must be enabled
*Mar  1 00:14:26.539: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access2, changed state to up

We need some traffic shaping on the FR (MFR1) interface - remember if we just turn traffic shaping on, all PVCs will be shaped at 56kbps.  For this example, we'll assume that there is 256kbps per PVC available

R4/47
map-class frame-relay MFR
 frame-relay traffic-rate 512000
interface MFR1
 frame-relay traffic-shaping
interface MFR1.47 point-to-point
 frame-relay interface-dlci 47 ppp Virtual-Template1
 class MFR

Lets bounce the link and see if it's happy this time round.

R7(config-if)#int mfr1
R7(config-if)#shut
*Mar  1 00:22:30.247: %LINK-5-CHANGED: Interface MFR1, changed state to administratively down
*Mar  1 00:22:30.247: %LINK-3-UPDOWN: Interface Virtual-Access2, changed state to down
*Mar  1 00:22:31.247: %LINEPROTO-5-UPDOWN: Line protocol on Interface MFR1, changed state to down
*Mar  1 00:22:31.251: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access2, changed state to down
*Mar  1 00:22:33.255: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/1, changed state to down
*Mar  1 00:22:33.259: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/0, changed state to down
R7(config-if)#no shut
*Mar  1 00:22:46.619: %LINK-3-UPDOWN: Interface MFR1, changed state to down
*Mar  1 00:22:54.215: %LINK-3-UPDOWN: Interface MFR1, changed state to up
*Mar  1 00:22:54.263: %LINK-3-UPDOWN: Interface Virtual-Access2, changed state to up
*Mar  1 00:22:55.207: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/1, changed state to up
*Mar  1 00:22:55.215: %LINEPROTO-5-UPDOWN: Line protocol on Interface MFR1, changed state to up
*Mar  1 00:22:55.219: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/0, changed state to up
*Mar  1 00:22:55.263: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access2, changed state to up

No complaints this time...

R7(config-if)#do ping 47.47.47.4

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 47.47.47.4, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/6/16 ms
R7(config-if)#do sh users
    Line       User       Host(s)              Idle       Location
*  0 con 0                idle                 00:00:00

  Interface    User               Mode         Idle     Peer Address
  Vi2                             PPPoFR       00:01:06
  Se1/0                           Sync PPP     00:00:05
  Se1/1                           Sync PPP     00:00:05
  Mu1                             Sync PPP     00:00:54 47.47.47.4

Looks good so far, lets confirm what the multilink PPP interface thinks its bandwidth is (it should be 1544 * 2 for the direct PPP links and 512 for the MFR link)

R7(config-if)#do sh ppp multi

Multilink1
  Bundle name: R4
  Remote Endpoint Discriminator: [1] R4
  Local Endpoint Discriminator: [1] R7
  Bundle up for 00:24:11, total bandwidth 3344, load 1/255
  Receive buffer limit 36000 bytes, frag timeout 1000 ms
    0/0 fragments/bytes in reassembly list
    0 lost fragments, 13 reordered
    0/0 discarded fragments/bytes, 0 lost received
    0x32 received sequence, 0x30 sent sequence
  Member links: 3 active, 1 inactive (max not set, min not set)
    Se1/0, since 00:24:06, 5790 weight, 1496 frag size
    Se1/1, since 00:24:05, 5790 weight, 1496 frag size
    Vi2, since 00:04:21, 960 weight, 950 frag size
    Vt1 (inactive)
No inactive multilink interfaces

We can see that the multilink interface thinks it has 3344 kbps of bandwidth and not 3600.

R7#sh int virtual-access 2
Virtual-Access2 is up, line protocol is up
  Hardware is Virtual Access interface
  MTU 1500 bytes, BW 256 Kbit/sec, DLY 100000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation PPP, LCP Open, multilink Open
  Link is a member of Multilink bundle Multilink1
  PPPoFR vaccess, cloned from Virtual-Template1
  Vaccess status 0x44
  Bound to MFR1.47 DLCI 47, Cloned from Virtual-Template1, loopback not set
  Keepalive set (10 sec)
  DTR is pulsed for 5 seconds on reset
  Last input 00:02:02, output never, output hang never
  Last clearing of "show interface" counters 00:20:59
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     253 packets input, 5890 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     254 packets output, 5393 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions

Virtual-Access2 believes that there is 256kbps of bandwidth available rather than the expected 512kbps.  Lets modify the bandwidth associated with Virtual-Template 1 (since the Virtual-Access interface inherits data from it) and see if that helps.

R7(config-if)#int virtual-template 1
R7(config-if)#bandwidth 512
R7(config-if)#int mfr1
R7(config-if)#shut
R7(config-if)#no shut
*Mar  1 00:40:48.283: %LINK-5-CHANGED: Interface MFR1, changed state to administratively down
*Mar  1 00:40:48.287: %LINK-3-UPDOWN: Interface Virtual-Access2, changed state to down
*Mar  1 00:40:49.295: %LINEPROTO-5-UPDOWN: Line protocol on Interface MFR1, changed state to down
*Mar  1 00:40:49.299: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access2, changed state to down
*Mar  1 00:40:49.647: %LINK-3-UPDOWN: Interface MFR1, changed state to down
*Mar  1 00:40:51.295: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/1, changed state to down
*Mar  1 00:40:51.299: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/0, changed state to down
*Mar  1 00:40:58.259: %LINK-3-UPDOWN: Interface MFR1, changed state to up
*Mar  1 00:40:58.311: %LINK-3-UPDOWN: Interface Virtual-Access2, changed state to up
*Mar  1 00:40:59.251: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/1, changed state to up
*Mar  1 00:40:59.259: %LINEPROTO-5-UPDOWN: Line protocol on Interface MFR1, changed state to up
*Mar  1 00:40:59.267: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/0, changed state to up
*Mar  1 00:40:59.311: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Access2, changed state to up
R7(config-if)#do sh ppp multi

Multilink1
  Bundle name: R4
  Remote Endpoint Discriminator: [1] R4
  Local Endpoint Discriminator: [1] R7
  Bundle up for 00:32:49, total bandwidth 3600, load 1/255
  Receive buffer limit 36000 bytes, frag timeout 1000 ms
    0/0 fragments/bytes in reassembly list
    0 lost fragments, 13 reordered
    0/0 discarded fragments/bytes, 0 lost received
    0x3A received sequence, 0x38 sent sequence
  Member links: 3 active, 1 inactive (max not set, min not set)
    Se1/0, since 00:32:44, 5790 weight, 1496 frag size
    Se1/1, since 00:32:43, 5790 weight, 1496 frag size
    Vi2, since 00:00:09, 1920 weight, 1496 frag size
    Vt1 (inactive)
No inactive multilink interfaces

Okay, repeat the same thing on R4 and that should be it.  Would anyone want to do this in the real world?  I certainly hope not!

Routing Protocol Redistribution using Tags and admin distance

Generally when I have been playing around with redistribution I have only been using route tags to eliminate routing loops, however this may not always work.

Below is a redistribution lab topology I have been recently playing with.

The base configurations prior to doing redistribution are shown below:

R4
hostname R4
!
interface Loopback0
 ip address 4.4.4.4 255.255.255.255
!
interface Serial0/0
 description R5 S0/0
 ip address 192.168.45.4 255.255.255.0
 clock rate 2000000
!
interface FastEthernet0/1
 description R6 Fa0/0
 ip address 192.168.46.4 255.255.255.0
 duplex auto
 speed auto
!
router eigrp 45
 network 4.4.4.4 0.0.0.0
 network 192.168.45.4 0.0.0.0
 no auto-summary
!
router ospf 456
 log-adjacency-changes
 network 4.4.4.4 0.0.0.0 area 46
 network 192.168.46.4 0.0.0.0 area 46
!


R5
hostname R5
!
interface Loopback0
 ip address 5.5.5.5 255.255.255.255
!
interface Serial0/0
 description R4 S0/0
 ip address 192.168.45.5 255.255.255.0
!
interface FastEthernet0/1
 description R6 Fa0/1
 ip address 192.168.56.5 255.255.255.0
 duplex auto
 speed auto
!
router eigrp 45
 network 5.5.5.5 0.0.0.0
 network 192.168.45.5 0.0.0.0
 no auto-summary
!
router ospf 456
 log-adjacency-changes
 network 5.5.5.5 0.0.0.0 area 56
 network 192.168.56.5 0.0.0.0 area 56
!


R6
hostname R6
!
interface Loopback0
 ip address 6.6.6.6 255.255.255.255
!
interface FastEthernet0/0
 description R4 Fa0/1
 ip address 192.168.46.6 255.255.255.0
 duplex auto
 speed auto
!
interface FastEthernet0/1
 description R5 Fa0/1
 ip address 192.168.56.6 255.255.255.0
 duplex auto
 speed auto
!
interface Serial0/0
 description R7 S0/0
 ip address 192.168.67.6 255.255.255.0
 clock rate 2000000
!
router ospf 456
 log-adjacency-changes
 network 6.6.6.6 0.0.0.0 area 0
 network 192.168.46.6 0.0.0.0 area 46
 network 192.168.56.6 0.0.0.0 area 56
!
router rip
 version 2
 network 6.0.0.0
 network 192.168.67.0
 no auto-summary
!

R7
hostname R7
!
interface Loopback0
 ip address 7.7.7.7 255.255.255.255
!
interface Serial0/0
 description R6 S0/0
 ip address 192.168.67.7 255.255.255.0
!
router rip
 version 2
 network 7.0.0.0
 network 192.168.67.0
 no auto-summary
!

In this example, I want to have mutual redistribution across all protocols to ensure connectivity regardless as to which link fails.

Redistribution happens at three locations - R6: Between RIP and OSPF while both R4 and R5 redistribute between OSPF and EIGRP

Route-Tags shall be used to ensure that routes originated by a routing protocol are not injected back into it.  To keep things straight forward, I am using the default admin distances for the protocols for their tags.

R6

route-map RIP2OSPF deny 10
 match tag 110
!
route-map RIP2OSPF permit 20
 match tag 90
 set tag 90
!
route-map RIP2OSPF permit 30
 set tag 120

route-map OSPF2RIP deny 10
 match tag 120
!
route-map OSPF2RIP permit 20
 match tag 90
 set tag 90
!
route-map OSPF2RIP permit 30
 set tag 110
!
router rip
 redistribute ospf 456 route-map OSPF2RIP metric 3
router ospf 456
 redistribute rip subnets route-map RIP2OSPF metric 100

R4/R5

route-map OSPF2EIGRP deny 10
 match tag 90
!
route-map OSPF2EIGRP permit 20
 match tag 120
 set tag 120
!
route-map OSPF2EIGRP permit 30
 set tag 110
!
route-map EIGRP2OSPF deny 10
 match tag 110
!
route-map EIGRP2OSPF permit 20
 match tag 120
 set tag 120
!
route-map EIGRP2OSPF permit 30
 set tag 90
!
router eigrp 45
 redistribute ospf 456 route-map OSPF2EIGRP metric 100000 1 255 1 1500
router ospf 456
 redistribute eigrp 45 subnets route-map EIGRP2OSPF metric 100


So far everything should be fine in the quiescent state.  However if we play around with links it is possible to introduce some problems (whenever working on a redistribution lab performing "debug ip routing" can be quite helpful)

If bounce a link between R6 and R5 we can see that the route to R7 Loopback disappears

R6#sh ip route 7.7.7.7
Routing entry for 7.7.7.7/32
  Known via "rip", distance 120, metric 1
  Redistributing via ospf 456, rip
  Advertised by ospf 456 metric 100 subnets route-map RIP2OSPF
  Last update from 192.168.67.7 on Serial0/0, 00:00:01 ago
  Routing Descriptor Blocks:
  * 192.168.67.7, from 192.168.67.7, 00:00:01 ago, via Serial0/0
      Route metric is 1, traffic share count is 1

R6#debug ip routing
IP routing debugging is on
R6#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R6(config)#int fa0/1
R6(config-if)#shut
*Mar  1 00:16:34.343: RT: is_up: FastEthernet0/1 0 state: 6 sub state: 1 line: 1 has_route: True
*Mar  1 00:16:34.343: %OSPF-5-ADJCHG: Process 456, Nbr 5.5.5.5 on FastEthernet0/1 from FULL to DOWN, Neighbor Down: Interface down or detached
*Mar  1 00:16:34.343: RT: interface FastEthernet0/1 removed from routing table
*Mar  1 00:16:34.343: RT: del 192.168.56.0 via 0.0.0.0, connected metric [0/0]
*Mar  1 00:16:34.343: RT: delete network route to 192.168.56.0
*Mar  1 00:16:34.347: RT: NET-RED 192.168.56.0/24
*Mar  1 00:16:34.347: RT: Pruning routes for FastEthernet0/1 (2)
*Mar  1 00:16:34.351: RT: delete route to 192.168.45.0 via 192.168.56.5, FastEthernet0/1
*Mar  1 00:16:34.351: RT: NET-RED 192.168.45.0/24
*Mar  1 00:16:34.351: RT: delete route to 5.5.5.5 via 192.168.56.5, FastEthernet0/1
*Mar  1 00:16:34.351: RT: no routes to 5.5.5.5, flushing
*Mar  1 00:16:34.351: RT: NET-RED 5.5.5.5/32
*Mar  1 00:16:34.351: RT: delete network route to 5.0.0.0
*Mar  1 00:16:34.351: RT: NET-RED 5.0.0.0/8
*Mar  1 00:16:36.327: %LINK-5-CHANGED: Interface FastEthernet0/1, changed state to administratively down
*Mar  1 00:16:36.335: RT: is_up: FastEthernet0/1 0 state: 6 sub state: 1 line: 1 has_route: False
*Mar  1 00:16:37.327: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
*Mar  1 00:16:37.335: RT: is_up: FastEthernet0/1 0 state: 6 sub state: 1 line: 1 has_route: False
*Mar  1 00:16:39.859: RT: add 5.5.5.5/32 via 192.168.46.4, ospf metric [110/100]
*Mar  1 00:16:39.863: RT: NET-RED 5.5.5.5/32

R6(config-if)#no shut
*Mar  1 00:16:46.307: RT: is_up: FastEthernet0/1 1 state: 4 sub state: 1 line: 1 has_route: False
*Mar  1 00:16:46.311: RT: add 192.168.56.0/24 via 0.0.0.0, connected metric [0/0]
*Mar  1 00:16:46.311: RT: NET-RED 192.168.56.0/24
*Mar  1 00:16:46.311: RT: interface FastEthernet0/1 added to routing table
*Mar  1 00:16:48.283: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up
*Mar  1 00:16:48.291: RT: is_up: FastEthernet0/1 1 state: 4 sub state: 1 line: 1 has_route: True
*Mar  1 00:16:49.283: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
*Mar  1 00:16:49.291: RT: is_up: FastEthernet0/1 1 state: 4 sub state: 1 line: 1 has_route: True
*Mar  1 00:16:50.959: %OSPF-5-ADJCHG: Process 456, Nbr 5.5.5.5 on FastEthernet0/1 from LOADING to FULL, Loading Done
*Mar  1 00:16:59.883: RT: del 5.5.5.5/32 via 192.168.46.4, ospf metric [110/100]
*Mar  1 00:16:59.887: RT: add 5.5.5.5/32 via 192.168.56.5, ospf metric [110/11]
*Mar  1 00:16:59.891: RT: NET-RED 5.5.5.5/32
*Mar  1 00:16:59.895: RT: closer admin distance for 7.7.7.7, flushing 1 routes
*Mar  1 00:16:59.899: RT: NET-RED 7.7.7.7/32
*Mar  1 00:16:59.899: RT: add 7.7.7.7/32 via 192.168.56.5, ospf metric [110/100]
*Mar  1 00:16:59.899: RT: NET-RED 7.7.7.7/32
*Mar  1 00:16:59.899: RT: add 192.168.45.0/24 via 192.168.56.5, ospf metric [110/100]
*Mar  1 00:16:59.899: RT: NET-RED 192.168.45.0/24

R6(config-if)#do sh ip route 7.7.7.7
Routing entry for 7.7.7.7/32
  Known via "ospf 456", distance 110, metric 100
  Tag 120, type extern 2, forward metric 10
  Redistributing via rip
  Last update from 192.168.56.5 on FastEthernet0/1, 00:00:16 ago
  Routing Descriptor Blocks:
  * 192.168.56.5, from 5.5.5.5, 00:00:16 ago, via FastEthernet0/1
      Route metric is 100, traffic share count is 1
      Route tag 120

On R6 we can see that the route to 7.7.7.7 is now being learnt through OSPF even though we were the one to put it there in the first place!  The issue here is because of the administration distance of the route.  RIP has an admin distance of 120, OSPF 110 and EIGRP 90.

On R6 our RIP route from R7 is redistributed into OSPF to R4 and R5.  When the link between R6 and R5 is brought down, the only way R5 knows about this route is when it is redistributed by R4 into EIGRP.
When the link between R6 and R5 comes back into service, that route is redistributed from EIGRP into OSPF by R5 and advertised to R6.  For the same route, the lower administration distance, the more belivable and therefore the RIP route is ignored (as can be seen by the debug above)

R6(config-if)#do ping 7.7.7.7

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 7.7.7.7, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

It is no surprise that that the ping to R7 failed, since R6 is the adjacent router to R7 and thinks the way there is via R5 - traffic to 7.7.7.7 will zoom around the network until the TTL expires

If we reset the OSPF process on R6 - things can return to the normal state (the route for 7.7.7.7 originated into OSPF will be withdrawn from EIGRP by R4 and wont be advertised back into OSPF by R5)

R6(config-if)#do clear ip ospf proc
Reset ALL OSPF processes? [no]: yes
*Mar  1 00:19:24.267: %OSPF-5-ADJCHG: Process 456, Nbr 4.4.4.4 on FastEthernet0/0 from FULL to DOWN, Neighbor Down: Interface down or detached
*Mar  1 00:19:24.323: %OSPF-5-ADJCHG: Process 456, Nbr 5.5.5.5 on FastEthernet0/1 from FULL to DOWN, Neighbor Down: Interface down or detached
*Mar  1 00:19:24.331: RT: delete route to 192.168.45.0/24
*Mar  1 00:19:24.331: RT: NET-RED 192.168.45.0/24
*Mar  1 00:19:24.335: RT: delete route to 4.4.4.4/32
*Mar  1 00:19:24.335: RT: NET-RED 4.4.4.4/32
*Mar  1 00:19:24.335: RT: delete network route to 4.0.0.0
*Mar  1 00:19:24.339: RT: NET-RED 4.0.0.0/8
*Mar  1 00:19:24.355: RT: delete route to 5.5.5.5/32
*Mar  1 00:19:24.359: RT: NET-RED 5.5.5.5/32
*Mar  1 00:19:24.359: RT: delete network route to 5.0.0.0
*Mar  1 00:19:24.363: RT: NET-RED 5.0.0.0/8
*Mar  1 00:19:24.371: RT: delete route to 7.7.7.7/32
*Mar  1 00:19:24.371: RT: NET-RED 7.7.7.7/32
*Mar  1 00:19:24.371: RT: delete network route to 7.0.0.0
*Mar  1 00:19:24.371: RT: NET-RED 7.0.0.0/8
*Mar  1 00:19:24.551: %OSPF-5-ADJCHG: Process 456, Nbr 5.5.5.5 on FastEthernet0/1 from LOADING to FULL, Loading Done
*Mar  1 00:19:24.551: %OSPF-5-ADJCHG: Process 456, Nbr 4.4.4.4 on FastEthernet0/0 from LOADING to FULL, Loading Done
*Mar  1 00:19:29.539: RT: add 4.4.4.4/32 via 192.168.46.4, ospf metric [110/11]
*Mar  1 00:19:29.539: RT: NET-RED 4.4.4.4/32
*Mar  1 00:19:29.543: RT: add 5.5.5.5/32 via 192.168.56.5, ospf metric [110/11]
*Mar  1 00:19:29.543: RT: NET-RED 5.5.5.5/32
*Mar  1 00:19:29.547: RT: add 192.168.45.0/24 via 192.168.56.5, ospf metric [110/100]
*Mar  1 00:19:29.547: RT: NET-RED 192.168.45.0/24
*Mar  1 00:19:29.547: RT: add 192.168.45.0/24 via 192.168.46.4, ospf metric [110/100]
*Mar  1 00:19:29.547: RT: NET-RED 192.168.45.0/24
*Mar  1 00:19:49.191: RT: add 7.7.7.7/32 via 192.168.67.7, rip metric [120/1]
*Mar  1 00:19:49.195: RT: NET-RED 7.7.7.7/32


We can see that this fixes things (until the next time we lose one of the OSPF links and it gets restored) We know that the problem is related to the route believability (Admin Distance) so the obvious answer is to modify it.

We have at least two ways to do so:
 Increase the Admin Distance of OSPF
 Decrease the Admin Distance for RIP

Related to the second, I will decrease the admin distance for a specific route (7.7.7.7/32)


R6(config-if)#access-list 1 permit host 7.7.7.7
R6(config)#router rip
R6(config-router)#distance 105 192.168.67.7 0.0.0.0 1

Not too long after after applying this change we can see:

*Mar  1 00:22:10.359: RT: closer admin distance for 7.7.7.7, flushing 1 routes
*Mar  1 00:22:10.363: RT: add 7.7.7.7/32 via 192.168.67.7, rip metric [105/1]
*Mar  1 00:22:10.367: RT: NET-RED 7.7.7.7/32

105 is less than 110, so R6 will not insert the OSPF route into the RIB - lets verify this by bouncing the link between R5 and R6 again:

R6(config-router)#int fa0/1
R6(config-if)#shut
*Mar  1 00:25:56.751: RT: is_up: FastEthernet0/1 0 state: 6 sub state: 1 line: 1 has_route: True
*Mar  1 00:25:56.755: %OSPF-5-ADJCHG: Process 456, Nbr 5.5.5.5 on FastEthernet0/1 from FULL to DOWN, Neighbor Down: Interface down or detached
*Mar  1 00:25:56.763: RT: interface FastEthernet0/1 removed from routing table
*Mar  1 00:25:56.767: RT: del 192.168.56.0 via 0.0.0.0, connected metric [0/0]
*Mar  1 00:25:56.767: RT: delete network route to 192.168.56.0
*Mar  1 00:25:56.767: RT: NET-RED 192.168.56.0/24
*Mar  1 00:25:56.767: RT: Pruning routes for FastEthernet0/1 (2)
*Mar  1 00:25:56.771: RT: delete route to 192.168.45.0 via 192.168.56.5, FastEthernet0/1
*Mar  1 00:25:56.771: RT: NET-RED 192.168.45.0/24
*Mar  1 00:25:56.771: RT: delete route to 5.5.5.5 via 192.168.56.5, FastEthernet0/1
*Mar  1 00:25:56.771: RT: no routes to 5.5.5.5, flushing
*Mar  1 00:25:56.771: RT: NET-RED 5.5.5.5/32
*Mar  1 00:25:56.771: RT: delete network route to 5.0.0.0
*Mar  1 00:25:56.771: RT: NET-RED 5.0.0.0/8
*Mar  1 00:25:58.727: %LINK-5-CHANGED: Interface FastEthernet0/1, changed state to administratively down
*Mar  1 00:25:58.735: RT: is_up: FastEthernet0/1 0 state: 6 sub state: 1 line: 1 has_route: False
*Mar  1 00:25:59.727: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
*Mar  1 00:25:59.735: RT: is_up: FastEthernet0/1 0 state: 6 sub state: 1 line: 1 has_route: False
*Mar  1 00:26:02.275: RT: add 5.5.5.5/32 via 192.168.46.4, ospf metric [110/100]
*Mar  1 00:26:02.275: RT: NET-RED 5.5.5.5/32

R6(config-if)#no shut
*Mar  1 00:26:29.059: RT: is_up: FastEthernet0/1 1 state: 4 sub state: 1 line: 1 has_route: False
*Mar  1 00:26:29.059: RT: add 192.168.56.0/24 via 0.0.0.0, connected metric [0/0]
*Mar  1 00:26:29.063: RT: NET-RED 192.168.56.0/24
*Mar  1 00:26:29.063: RT: interface FastEthernet0/1 added to routing table
*Mar  1 00:26:31.023: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up
*Mar  1 00:26:31.027: RT: is_up: FastEthernet0/1 1 state: 4 sub state: 1 line: 1 has_route: True
*Mar  1 00:26:32.023: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
*Mar  1 00:26:32.031: RT: is_up: FastEthernet0/1 1 state: 4 sub state: 1 line: 1 has_route: True
*Mar  1 00:26:33.995: %OSPF-5-ADJCHG: Process 456, Nbr 5.5.5.5 on FastEthernet0/1 from LOADING to FULL, Loading Done
*Mar  1 00:26:44.575: RT: del 5.5.5.5/32 via 192.168.46.4, ospf metric [110/100]
*Mar  1 00:26:44.579: RT: add 5.5.5.5/32 via 192.168.56.5, ospf metric [110/11]
*Mar  1 00:26:44.579: RT: NET-RED 5.5.5.5/32
*Mar  1 00:26:44.579: RT: add 192.168.45.0/24 via 192.168.56.5, ospf metric [110/100]
*Mar  1 00:26:44.583: RT: NET-RED 192.168.45.0/24

As we can see there has been no change to the routing table for 7.7.7.7

R6(config-if)#do sh ip route 7.7.7.7
Routing entry for 7.7.7.7/32
  Known via "rip", distance 105, metric 1
  Redistributing via ospf 456, rip
  Advertised by ospf 456 metric 100 subnets route-map RIP2OSPF
  Last update from 192.168.67.7 on Serial0/0, 00:00:25 ago
  Routing Descriptor Blocks:
  * 192.168.67.7, from 192.168.67.7, 00:00:25 ago, via Serial0/0
      Route metric is 1, traffic share count is 1

R6(config-if)#do ping 7.7.7.7

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 7.7.7.7, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

While using route tags is one of the tools to control routing loops, understanding the admin distance is also worthwhile knowing.

Lab Booked and Paid For

It's all paid for so there's no turning back.  Sydney in the middle of June... Booking the R&S Lab it could very well be a traditional expensive lunch since right now I know I am not completely ready right now but I had to put a definite stake in the ground to aim towards, as well as organising the time off work.

To assist with this, I have been fortunate enough to be able to scrounge up the funding to take part in one of Narbik's Bootcamps in Sydney which will take place approximately two weeks before I attempt the lab for the first time (I'm not aiming or planning to fail as such - I just want to prepare myself for the condition that I may not pass).  The Bootcamp and being able to take a large block of annual leave to put work to the side and concentrate on getting my technique and speed honed should help.

If I were to be honest with myself I think parts of multicast, FR QoS and Security particularly around the IOS Firewall and ZBF are my weaker areas that I will be putting extra focus on.

I'm quite looking forward to the bootcamp, I hear they are very intense but worthwhile at the very least I will be able to meet up in person with some more candidates as well as drinking from the fountain of networking related to the CCIE R&S V.40 blueprint.  There always seems to be a near Steve Jobs (I was nearly going to say religious) mania surrounding Narbik from those that have been in his presence, I wonder if I will become one of the zealots :)

Slight workflow increase when using Dynamips

I'm still working my way through the IP Expert Volume 3 Workbooks - primarily going through the troubleshooting sessions at the moment since I don't have the largest amount of time available right now and a 2 hour session under a bit of pressure is a good thing to keep me motivated.

Anyway, the topic for discussion here is about how to slightly speed up getting ready to do a lab - probably a lot of people do something similar already, it's just unfortunate that I have really only woken up to the fact that this can help save about 15-20 minutes and reduce errors.

Firstly - this doesn't help much with when you're using real routers - When I finish up with the lab, I get them erased and reloaded so they're all nice an clear of IOS config and VTP related config

Cat1#wr erase
Erasing the nvram filesystem will remove all configuration files! Continue? [confirm]
[OK]
Erase of nvram: complete
*Mar  1 03:18:23.937: %SYS-7-NV_BLOCK_INIT: Initialized the geometry of nvram
Cat1#del vlan.dat
Delete filename [vlan.dat]?
Delete flash:vlan.dat? [confirm]
Cat1#reload
Proceed with reload? [confirm]
*Mar  1 03:18:31.672: %SYS-5-RELOAD: Reload requested by console. Reload Reason: Reload command.

On my Ubuntu box, I keep all of the configs I use in a directory funnily enough called cisco:

adam@RouterSim:~/cisco$ pwd
/home/adam/cisco

I then have all the related labs living in sub directories - for example the Volume 3 labs are simply unzipped into a directory called ipexpert_vol3

adam@RouterSim:~/cisco$ ls ipexpert_vol3
LAB-10C  LAB-10T  LAB-1C  LAB-1T  LAB-2C  LAB-2T  LAB-3C  LAB-3T  LAB-4C  LAB-4T  LAB-5C  LAB-5T  LAB-6C  LAB-6T  LAB-7C  LAB-7T  LAB-8C  LAB-8T  LAB-9C  LAB-9T  R&S_Volume_3_Diagrams_v11.0.pdf

The dynamips.net config file I use is set up to a generic version what I used previously the main thing I want to highlight is the reference to "currentlab" in the working dir and cnfg lines, the trick I will be using is "softlinks" to point to the specific lab of interest and be able to load up the lab configs without needing a separate .net file per lab when I start dynamips...



start of ipexpert.net
autostart = false
ghostios = true
sparsemem = true
[localhost]
        workingdir = /home/adam/cisco/currentlab/work_ipexpert
        [[7200]]
                image = /opt/ios/unzipped/c7200-adventerprisek9-mz.124-24.T.bin
                npe = npe-400
                ram = 384
                idlepc = 0x6307ad74
                disk0 = 128

        [[2611]]
                image = /opt/ios/unzipped/c2600-js-mz.123-17a.bin
                ram = 64
                idlepc = 0x803266c4
        [[3725]]
                image = /opt/ios/unzipped/c3725-advsecurityk9-mz.124-15.T12.bin
                #/opt/ios/unzipped/c3725-adventerprisek9-mz.124-15.T10.bin
                ram = 128
                #idlepc = 0x6307ad74
                disk0 = 64
                idlepc = 0x60bb28d0

        [[ROUTER R1]]
                model = 7200
                console = 2001
                slot0 = PA-C7200-IO-2FE
                f0/0 = NIO_gen_eth:eth0.101
                f0/1 = NIO_gen_eth:eth0.102
                cnfg = /home/adam/cisco/currentlab/INITIAL/R1.txt

        [[ROUTER R2]]
                model = 7200
                console = 2002
                slot0 = PA-C7200-IO-2FE
                slot2 = PA-4T
                f0/0 = NIO_gen_eth:eth0.201
                f0/1 = NIO_gen_eth:eth0.203
                s2/1 = FR s0/0
                s2/2 = R5 s2/2
                cnfg = /home/adam/cisco/currentlab/INITIAL/R2.txt

        [[ROUTER R4]]
                model = 7200
                console = 2004
                slot0 = PA-C7200-IO-2FE
                slot2 = PA-4T
                f0/0 = NIO_gen_eth:eth0.401
                f0/1 = NIO_gen_eth:eth0.402
                s2/0 = FR s0/1
                cnfg = /home/adam/cisco/currentlab/INITIAL/R4.txt

        [[ROUTER R5]]
                model = 7200
                console = 2005
                slot0 = PA-C7200-IO-2FE
                slot2 = PA-4T
                f0/0 = NIO_gen_eth:eth0.501
                f0/1 = NIO_gen_eth:eth0.502
                s2/1 = FR s0/2
                cnfg = /home/adam/cisco/currentlab/INITIAL/R5.txt

        [[ROUTER R6]]
                model = 7200
                console = 2006
                slot0 = PA-C7200-IO-2FE
                slot2 = PA-4T
                f0/0 = NIO_gen_eth:eth0.601
                f0/1 = NIO_gen_eth:eth0.602
                s2/1 = FR s0/3
                s2/2 = R9 s2/2
                s2/3 = R9 s2/3
                cnfg = /home/adam/cisco/currentlab/INITIAL/R6.txt

        [[ROUTER R7]]
                model = 7200
                console = 2007
                slot0 = PA-C7200-IO-2FE
                slot2 = PA-4T
                f0/0 = NIO_gen_eth:eth0.701
                f0/1 = NIO_gen_eth:eth0.702
                s2/0 = R8 s2/0
                cnfg = /home/adam/cisco/currentlab/INITIAL/R7.txt

        [[ROUTER R8]]
                model = 7200
                console = 2008
                slot0 = PA-C7200-IO-2FE
                slot2 = PA-4T
                f0/0 = NIO_gen_eth:eth0.801
                f0/1 = NIO_gen_eth:eth0.802
                cnfg = /home/adam/cisco/currentlab/INITIAL/R8.txt

        [[ROUTER R9]]
                model = 7200
                console = 2009
                slot0 = PA-C7200-IO-2FE
                slot2 = PA-4T
                f0/0 = NIO_gen_eth:eth0.901
                f0/1 = NIO_gen_eth:eth0.902
                cnfg = /home/adam/cisco/currentlab/INITIAL/R9.txt

        [[ROUTER BB1]]
                model = 2611
                console = 2101
                e0 = NIO_gen_eth:eth0.11
                e1 = NIO_gen_eth:eth1
                cnfg = /home/adam/cisco/currentlab/INITIAL/BB1.txt

        [[ROUTER BB2]]
                model = 2611
                console = 2102
                e0 = NIO_gen_eth:eth0.22
                cnfg = /home/adam/cisco/currentlab/INITIAL/BB2.txt

        [[ROUTER BB3]]
                model = 2611
                console = 2103
                e0 = NIO_gen_eth:eth0.33
                cnfg = /home/adam/cisco/currentlab/INITIAL/BB3.txt

        [[ROUTER FR]]
                model = 3725
                console = 2104
                WIC0/0 = WIC-2T
                WIC0/1 = WIC-2T
                cnfg = /home/adam/cisco/IPExpertFrame.txt
end of ipexpert.net

IPexpertFrame.txt refers to a configuration for the FR Switch rather than use the built in frame relay switch capacity of dynamips:

start of IPExpertFrame.txt
hostname FR
!
interface Serial0/0
 description R2
 no ip address
 encapsulation frame-relay
 clock rate 2000000
 frame-relay lmi-type cisco
 frame-relay intf-type dce
 frame-relay route 104 interface Serial0/1 401
 frame-relay route 105 interface Serial0/2 501
 frame-relay route 106 interface Serial0/3 601
 frame-relay route 204 interface Serial0/1 402
 frame-relay route 205 interface Serial0/2 502
 frame-relay route 206 interface Serial0/3 602
 frame-relay route 214 interface Serial0/1 412
 frame-relay route 215 interface Serial0/2 512
 frame-relay route 216 interface Serial0/3 612
 frame-relay route 224 interface Serial0/1 422
 frame-relay route 225 interface Serial0/2 522
 frame-relay route 226 interface Serial0/3 622
 no shutdown
!
interface Serial0/1
 description R4
 no ip address
 encapsulation frame-relay
 clock rate 2000000
 frame-relay lmi-type cisco
 frame-relay intf-type dce
 frame-relay route 401 interface Serial0/0 104
 frame-relay route 402 interface Serial0/0 204
 frame-relay route 405 interface Serial0/2 504
 frame-relay route 406 interface Serial0/3 604
 frame-relay route 412 interface Serial0/0 214
 frame-relay route 415 interface Serial0/2 514
 frame-relay route 416 interface Serial0/3 614
 frame-relay route 422 interface Serial0/0 224
 frame-relay route 425 interface Serial0/2 524
 frame-relay route 426 interface Serial0/3 624
 no shutdown
!
interface Serial0/2
 description R5
 no ip address
 encapsulation frame-relay
 clock rate 2000000
 frame-relay lmi-type cisco
 frame-relay intf-type dce
 frame-relay route 501 interface Serial0/0 105
 frame-relay route 502 interface Serial0/0 205
 frame-relay route 504 interface Serial0/1 405
 frame-relay route 506 interface Serial0/3 605
 frame-relay route 512 interface Serial0/0 215
 frame-relay route 514 interface Serial0/1 415
 frame-relay route 516 interface Serial0/3 615
 frame-relay route 522 interface Serial0/0 225
 frame-relay route 524 interface Serial0/1 425
 frame-relay route 526 interface Serial0/3 625
 no shutdown
!
interface Serial0/3
 description R6
 no ip address
 encapsulation frame-relay
 clock rate 2000000
 frame-relay lmi-type cisco
 frame-relay intf-type dce
 frame-relay route 601 interface Serial0/0 106
 frame-relay route 602 interface Serial0/0 206
 frame-relay route 604 interface Serial0/1 406
 frame-relay route 605 interface Serial0/2 506
 frame-relay route 612 interface Serial0/0 216
 frame-relay route 614 interface Serial0/1 416
 frame-relay route 615 interface Serial0/2 516
 frame-relay route 622 interface Serial0/0 226
 frame-relay route 624 interface Serial0/1 426
 frame-relay route 625 interface Serial0/2 526
 no shutdown
!
end
end of IPExpertFrame.txt

So the deal with the softlinks is to point "currentlab" to the lab you want to work on for example:

adam@RouterSim:~/cisco$ ln -s ipexpert_vol3/LAB-4T/ currentlab

adam@RouterSim:~/cisco$ ls -l currentlab
lrwxrwxrwx 1 adam adam 21 2011-02-20 18:45 currentlab -> ipexpert_vol3/LAB-4T/

Here are a few house keeping tasks that could be easily scripted away - create a working directory for dynamips files to go into and to rework the standard IPExpert config to match the interfaces in my topology


adam@RouterSim:~/cisco$ mkdir currentlab/work_ipexpert
adam@RouterSim:~/cisco$ cd currentlab/INITIAL/
adam@RouterSim:~/cisco/currentlab/INITIAL$ reworkcfg
adam@RouterSim:~/cisco/currentlab/INITIAL$ cd ../..

To load the config onto the external switches, we just paste the output from

adam@RouterSim:~/cisco$ cat currentlab/INITIAL/Cat1.txt

adam@RouterSim:~/cisco$ cat currentlab/INITIAL/Cat2.txt 

adam@RouterSim:~/cisco$ cat currentlab/INITIAL/Cat3.txt  

adam@RouterSim:~/cisco$ cat currentlab/INITIAL/Cat4.txt

So we just start things up normally and go on our merry lab way

adam@RouterSim:~/cisco$ sudo dynagen ipexpert.net
Reading configuration file...

Network successfully loaded

Dynagen management console for Dynamips and Pemuwrapper 0.11.0
Copyright (c) 2005-2007 Greg Anuzelli, contributions Pavel Skovajsa

=> start /all
100-VM 'R4' started
100-VM 'R5' started
100-VM 'R6' started
100-VM 'R7' started
100-VM 'R1' started
100-VM 'R2' started
100-VM 'FR' started
100-VM 'R8' started
100-VM 'R9' started
100-VM 'BB3' started
100-VM 'BB2' started
100-VM 'BB1' started

Once we have finished the with lab and exited dynagen, we tidy up:

That is write erase/del vlan.dat/reload on the switches and
 
adam@RouterSim:~/cisco$ rm currentlab

Now we're ready for the next lab session:

Back in action

Nearly missed out on a January update here...

Just before Christmas, I sat and happily passed the CCIE R&S Written Exam, which is handy if only to be able to view the Lab exam spots that are open.

In mid January I sat the Alcatel-Lucent Service Router Certification NRS-II Lab Exam in Melbourne.  I think it was a pretty good exam really in the depth and breadth of the material to be covered and time available, I don't know if I have passed it yet, unfortunately the grading of this exam takes somewhat more time than it appears the CCIE Lab exams do.  It's of direct relevance to most of the work I've been involved with lately so I felt the diversion away from the CCIE was easily justified.

Today I have started back up again and have jumped in with the IP Expert Volume 3 Labs.  Having had what probably amounts to a six week break from CCIE studies has been pretty harsh with the speed on the keys and knowing where to look in the doccd, I have been a little slower than I really would have liked but it's actually not a bad thing to get a dose of reality and to identify some areas that I have got a little rusty on already.

Status Update

Things have been a little quiet on the blog lately but I have been pretty much keeping the study momentum going.

I don't think I mentioned it but I haven't actually sat for the CCIE Written Exam yet.  Here in Brisbane there used to be two centres you could sit the exam but one went out of business a few months ago and there is a 4-5 month lead time on doing a computer based exam! In a few weeks I plan to go to Sydney so I will make time to do the exam then.  The CCIE R&S Version 4.0 blueprint for Written and Lab are pretty much in alignment, right now I am moving more to looking at things from what the written exam is interested in.

That said, I'm now at the end of the IPExpert Volume 1 Labs - the final lab has an estimated timelimit of 10 hours and touches on NAT and IOS security topics particularly IOS firewalls both topics which aren't really my strong points.  So I think I'm going to do some more self study in those areas before I face that particular lab.  How do I feel so far about the Volume 1 Workbook?  I've been pretty happy with it, most of the labs have been good but some have been more challenging than others (I guess the areas I spent a lot of time on projects I found particularly easy compared to others I havent had a lot of experience with) - I know I'm going to redo particular labs in the future but it's been good to touch on technology areas.  I'm looking forward to facing the Volume 3 labs. 

One of the things I'm planning on doing is getting used to working to the estimated timeframes for the labs, while I do attempt to black out enough time to sit a lab at home, the reality is there are some interruptions that occur, so I've decided to use a timer program that I can pause when these events crop up.  So far XNote Stopwatch seems fairly reasonable at the moment - I like the idea of being able to set it to be always on top but partly transparent so I can see the time remaining and not lose screen real estate.