Now, don't get me wrong, I think the people at Cisco TAC are great. They're an absolutely first class support organization, the standard by which other vendors are judged, and they consistently give me great service.
In spite of their efforts, sometimes things just don't work out. This is one of those times.
I opened SR629007199 on January 31st 2014 after noticing a peculiar problem with an ISR G2 router: Servers couldn't receive packets intended for them because the router was screwing up their traffic. The Ethernet frames carrying these packets included the wrong destination MAC address, so the servers ignored them.
Specifically, the router was screwing up the IP->L2 address mapping required for IPv4 multicast packets. Instead of using 23 bits of the multicast group in the L2 header, that portion of the L2 header was all zeros. It looked like this:
Bogus dMAC on most of these frames |
Those two HSRP packets originated by the router looked okay, and traffic from local sources was okay, but every frame containing packets which had transited the router was wrong. Instead of 01:00:5e:00:00:00, those frames should have been sent to 01:00:5e:78:01:24.
Now, to be fair, my configuration was a little "off in the weeds". Most folks don't route multicast in production, and I was using an unusual combination of interface types. I'm not surprised this is issue isn't widely complained about, even though it's 100% reproducible.
Still, it's a supported configuration, and a whole year later there's no indication that anybody at Cisco has even begun working on the problem. That's the reason for this blog post. I'm complaining about it publicly because I'm frustrated, and because the correct channel has gotten me nowhere for the last year.
TAC has explained to me repeatedly that their hands are tied. It seems that ISR product management doesn't care about this bug. Their "workaround" is suggesting that I disable CEF. Doing so resolves the framing issue, but opens another can of worms in the performance department. It's true, I can disable CEF only for multicast traffic, but that's basically all this router carries. It exists only to support a multicast-based voice/video collaboration application. I can't afford the performance penalty associated with process switching this traffic. TAC, to their credit, didn't take the workaround seriously either.
So, why do I think product management doesn't care? Well, the relevant bug is rated as Severity 6:
Severity: 6 Enhancement |
According to Cisco, this severity rating indicates a request for new functionality or feature improvements:
Cisco Table of Severity Levels |
TAC and I have been trying to make the case for this bug for a year now, but product management has totally ignored it.
So, I give up. Next time TAC calls to ask permission to close the support case, I'm going to let them.