Sebastian described an intriguing Cisco ACI quirk they experienced the privilege of chasing all-around:

We have encountered VM connectivity troubles right after VM movements from one vPC leaf pair to a distinctive vPC leaf pair with ACI. The problem did not occur right away (owing to ACI’s bounce entries) and only sometimes, which built it pretty tricky to reproduce synthetically, but owing to DRS and a big amount of VMs it happened often sufficient, that it was a serious issue for us.

Here’s what they figured out:

The issue was, that at times the COOP database entry (ACI’s individual handle airplane for MACs and host addresses) was not up-to-date the right way to level to the new leaf pair.

That undoubtedly sounds like a bug, and Erik described in a later remark that it was probably preset in the meantime. Having said that, the enjoyment section was that points labored for almost 10 minutes just after the VM migration:

After the bounce entry on the old leaf pair expired (630 seconds by default), targeted visitors to the VM was mostly blackholed, because remote endpoint understanding is disabled on border leafs and constantly forwarded to the spines underlay IP deal with for proxying.

A bounce entry appears to be something like MPLS/VPN PIC Edge – the original swap appreciates in which the MAC tackle has moved to, and redirects the visitors to the new location. Just owning that features can make me worried – opposite to MPLS/VPN networks wherever you could have numerous paths to the exact same prefix (and therefore know the backup path in progress), you will need a bounce entry for a MAC handle only when:

• The first edge gadget appreciates the new swap the moved MAC address is connected to
• Other material customers haven’t realized that nonetheless.
• The interim condition persists prolonged ample to be value the additional energy.

In any case, the corporation struggling with that challenge decided to “solve” it by limiting VM migration to a solitary vPC pair:

In the conclude we gave up and restricted the VM migration area to a solitary VPC leaf pair. VMware suggests a highest variety of 64 hosts per cluster anyway.

Getting superior-availability vSphere clusters and much more than two leaf switches, and restricting the HA domain to a solitary pair of leafs, certainly degrades the resilience of the all round architecture, until they made the decision to limit DRS (automated VM migrations) to a subset of cluster nodes with VM affinity though retaining the benefits of having the high-availability cluster stretched throughout many leaf pairs. It’s unhappy that just one has to go down these types of paths to prevent vendor bugs brought about by as well a lot unwanted complexity.

Want to Know Extra About Cisco ACI? Cisco ACI Introduction and Cisco ACI Deep Dive
Webinars are ready for you )