In the examples that follow, R5 is configured with no ip routing. R5 is our ignorant host system who will be using the wrong gateway and receiving ICMP redirects.
No FHRP:
For the topology in Figure 1, when R5 pings R10's loopback interface, he receives an ICMP redirect:
R5#ping 192.168.51.10
Here's how VRRP blackholes traffic:
VRRP has given R5 some potentially troublesome information. Consider what happens if R3 fails:
R1#show ip route 192.168.51.10
Routing entry for 192.168.51.10/32
Known via "ospf 1", distance 110, metric 792, type intra area
Last update from 192.168.50.4 on Ethernet0/0, 00:00:13 ago
Routing Descriptor Blocks:
* 192.168.50.4, from 192.168.51.10, 00:00:13 ago, via Ethernet0/0
Route metric is 792, traffic share count is 1
No FHRP:
 |
| Figure 1 |
R5#debug ip icmp
ICMP packet debugging is on
R5#ping 192.168.51.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.51.10, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/8 ms
R5#
*Mar 1 21:57:00.715: ICMP: redirect rcvd from 192.168.50.1- for 192.168.51.10 use gw 192.168.50.3
R1#debug ip icmp
ICMP packet debugging is on
R1#
*Mar 1 02:38:01.123: ICMP: redirect sent to 192.168.50.5 for dest 192.168.51.10, use gw 192.168.50.3
No real surprises here. R1 knows that R3 provides the best path to R10's loopback interface, so he sent a redirect. The redirect:
- was sourced from R1's physical interface address (it seems obvious, but this is important)
- tells R5 to use 192.168.50.3 (R3's physical address) when talking to R10
The result on R5 looks like this:
R5#show ip route
Default gateway is 192.168.50.1
Host Gateway Last Use Total Uses Interface
192.168.51.10 192.168.50.3 0:05 8 Ethernet0/0
Let's switch on VRRP:
Now, let's enable VRRP on R1, and point R5 at R1's VRRP address, rather than the physical interface address. The resulting topology now looks like:
Now, let's enable VRRP on R1, and point R5 at R1's VRRP address, rather than the physical interface address. The resulting topology now looks like:
 |
| Figure 2 |
R5#ping 192.168.51.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.51.10, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/4 ms
R5#
*Mar 1 22:11:54.707: ICMP: redirect rcvd from 192.168.50.101- for 192.168.51.10 use gw 192.168.50.3
R1#
*Mar 1 02:52:55.099: ICMP: redirect sent to 192.168.50.5 for dest 192.168.51.10, use gw 192.168.50.3
R1 didn't mention explicitly it in the debug output, but he changed the address from which he sourced the redirect. This time, the redirect came from the VRRP gateway address 192.168.50.101. It's important that R1 source the redirect from the correct IP, because RFC 1122 section 3.2.2.2 says:
A Redirect message SHOULD be silently discarded if the newR1 is able to determine that R5 knows him by the VRRP address (.101), rather than the physical interface address (.1) because the packet destined for R10 was encapsulated in an Ethernet frame destined for 00:00:5e:00:01:65 (the VRRP MAC address). If, for some reason, a router is unable to determine the IP by which clients know him, he shouldn't send an ICMP redirect.
gateway address it specifies is not on the same connected
(sub-) net through which the Redirect arrived [INTRO:2,
Appendix A], or if the source of the Redirect is not the
current first-hop gateway for the specified destination (see
Section 3.3.1).
RFC 5798 Section 8.1.1 spells this out:
The IPv4 source address of an ICMP redirect should be the address
that the end-host used when making its next-hop routing decision. If
a VRRP router is acting as Master for virtual router(s) containing
addresses it does not own, then it must determine which virtual
router the packet was sent to when selecting the redirect source
address. One method to deduce the virtual router used is to examine
the destination MAC address in the packet that triggered the
redirect.
Note that there are 3 usable addresses on R3:
- The physical interface address 192.168.50.3, which R1 chose to use in the redirect
- The address for VRRP group 201: 192.168.50.201
- The address for VRRP group 202: 192.168.50.202 (currently active on R4)
Here's how VRRP blackholes traffic:
VRRP has given R5 some potentially troublesome information. Consider what happens if R3 fails:
R3#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R3(config)#int eth 0/0
R3(config-if)#shut
R1 reconverged around this failure:
R1#show ip route 192.168.51.10
Routing entry for 192.168.51.10/32
Known via "ospf 1", distance 110, metric 792, type intra area
Last update from 192.168.50.4 on Ethernet0/0, 00:00:13 ago
Routing Descriptor Blocks:
* 192.168.50.4, from 192.168.51.10, 00:00:13 ago, via Ethernet0/0
Route metric is 792, traffic share count is 1
But R5 did not. He's still trying to use R3 (now offline), as previously directed by R1:
R5#ping 192.168.51.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.51.10, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
R5#show ip route
Default gateway is 192.168.50.101
Host Gateway Last Use Total Uses Interface
192.168.51.10 192.168.50.3 0:01 33 Ethernet0/0
Thanks, VRRP!
VRRP routers can certainly hear other VRRP routers on the local segment:
R1#debug vrrp errors
VRRP Errors debugging is on
R1#
*Mar 1 03:29:53.915: VRRP: Advertisement from 192.168.50.4 has an incorrect
group 202 id for interface Et0/0
*Mar 1 03:29:54.039: VRRP: Advertisement from 192.168.50.3 has an incorrect
group 201 id for interface Et0/0
But they don't seem to keep track of groups in which they're not participating:
R1#show vrrp all
Ethernet0/0 - Group 101
State is Master
Virtual IP address is 192.168.50.101
Virtual MAC address is 0000.5e00.0165
Advertisement interval is 1.000 sec
Preemption enabled
Priority is 105
Master Router is 192.168.50.1 (local), priority is 105
Master Advertisement interval is 1.000 sec
Master Down interval is 3.589 sec
So, what about HSRP?
As it turns out, the outcome of the first test (no HSRP on R1, R5 pointed to 192.168.50.1, R3 and R4 running HSRP) is the same: A redirect sourced from R1's physical interface address steers R5 toward 192.168.50.3 for destination 192.168.51.10.
Routers not running HSRP don't keep track of any HSRP groups (probably on a per-interface basis):
R1#show standby all brief
R1#
Now, lets enable HSRP on R1, and point R5 at the HSRP gateway address:
 |
| Figure 3 |
The first thing to notice is that with HSRP running, R1 (group 101) keeps track of what R3 and R4 are doing on HSRP groups 201 and 202. These are groups of which R1 is not a member.
R1#show standby all brief
P indicates configured to preempt.
|
Interface Grp Prio P State Active Standby Virtual IP
Et0/0 101 105 P Active local unknown 192.168.50.101
Et0/0 201 100 Disabled 192.168.50.3 192.168.50.4 192.168.50.201
Et0/0 202 100 Disabled 192.168.50.4 192.168.50.3 192.168.50.202
HSRP redirect behavior:
R5#ping 192.168.51.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.51.10, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/404/1004 ms
R5#
*Mar 1 23:21:25.859: ICMP: redirect rcvd from 192.168.50.101- for 192.168.51.10 use gw 192.168.50.201
R1#debug ip icmp
ICMP packet debugging is on
R1#
*Mar 1 04:02:26.159: ICMP: HSRP changing redirect sent to 192.168.50.5 for dest 192.168.51.10
*Mar 1 04:02:26.159: ICMP: gw 192.168.50.3 -> 192.168.50.201, src 192.168.50.101
*Mar 1 04:02:26.159: ICMP: Use HSRP virtual address 192.168.50.101 as ICMP src
*Mar 1 04:02:26.159: ICMP: redirect sent to 192.168.50.5 for dest 192.168.51.10, use gw 192.168.50.201
This time, R1 changed two parameters in the redirect. First, he sourced the redirect from the 192.168.50.101, which is the IP that R5 knows about, as required by RFC 1122. This is the same as VRRP's behavior. Next, because R1 knows that R3 is running HSRP, and that R3 is active for group 201, he changed the redirect to refer to R3 by its HSRP virtual address, 192.168.50.201.
This is way cool because now if R3 fails, R5 won't lose connectivity with R10.
Somewhat surprisingly, we've added resiliency to this topology by turning up an HSRP process on R1, even though there's no router backing up R1!
Now lets see what happens if we change the HSRP priority so that R4 is the active router for both of his groups (201 and 202), preempting R3:
 |
| Figure 4 |
R4(config)#interface Ethernet0/0
R4(config-if)#standby 201 priority 110
R4(config-if)#standby 201 preempt
*Mar 1 09:46:48.751: %HSRP-5-STATECHANGE: Ethernet0/0 Grp 201 state Standby -> Active
Now R3 remains the best choice for accessing R10, but it isn't the active HSRP router for any groups. What will R1 do?
R5#debug ip icmp
ICMP packet debugging is on
R5#ping 192.168.51.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.51.10, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/5/8 ms
R5#
Huh. No redirect arrived at R5.
R1#debug ip icmp
ICMP packet debugging is on
R1#
*Mar 1 04:49:33.850: ICMP: redirect not sent to 192.168.50.5 for dest 192.168.51.10
*Mar 1 04:49:33.850: ICMP: 192.168.50.3 does not contain an active HSRP group
Cool! Rather than redirecting to R3's physical interface (like VRRP does), or to a virtual router address on R4, R1 elects to suppress the redirect and route the traffic itself.
Okay, one last thing to point out:
As I mentioned earlier, routers aren't supposed to send redirects if they're not sure about the IP by which clients know them. One way to accomplish this is with the standby use-bia command. This causes the router to use its burned-in-address for the HSRP groups. This creates ambiguity about the client's configuration because both the physical interface IP and the HSRP IP are represented by the same MAC address.
R1(config)#interface Ethernet 0/0
R1(config)#interface Ethernet 0/0
R1(config-if)#standby use-bia
We can see the result in R5's show ip arp output, where both of R1's IP addresses can be seen using the same MAC address. R1 now has no way of distinguishing whether R5 knows him as 192.168.50.1 or as 192.168.50.101, making it impossible to know which address should be used in an ICMP redirect.
R5#show ip arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.50.101 3 0050.730d.0d41 ARPA Ethernet0/0
Internet 192.168.50.1 1 0050.730d.0d41 ARPA Ethernet0/0
Nonetheless, R1 generates a redirect, sourced from 192.168.50.1. That's the wrong address because R5's default route points to 192.168.50.101. R5 doesn't seem to mind, and installs the redirect in his routing table anyway, ignoring the advice in RFC 1122.
R5#show ip route
Default gateway is 192.168.50.101
Host Gateway Last Use Total Uses Interface
ICMP redirect cache is empty
R5#ping 192.168.51.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.51.10, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/8 ms
R5#
*Mar 1 23:39:54.667: ICMP: redirect rcvd from 192.168.50.1- for 192.168.51.10 use gw 192.168.50.201
R1(config-if)#
*Mar 1 04:20:54.947: ICMP: HSRP changing redirect sent to 192.168.50.5 for dest 192.168.51.10
*Mar 1 04:20:54.947: ICMP: gw 192.168.50.3 -> 192.168.50.201
*Mar 1 04:20:54.947: ICMP: redirect sent to 192.168.50.5 for dest 192.168.51.10, use gw 192.168.50.201
R5#show ip route
Default gateway is 192.168.50.101
Host Gateway Last Use Total Uses Interface
192.168.51.10 192.168.50.201 0:02 8 Ethernet0/0
R1's decision to send the redirect in spite of the addressing ambiguity contrasts with some Cisco documentation. That document includes the following debug output from a case where a router can't be sure how he's known by a client:
10:43:08: SB: ICMP redirect not sent to 20.0.0.4 for dest 30.0.0.2
10:43:08: SB: could not uniquely determine IP address for mac 00d0.bbd3.bc22
