Advanced uses for the ESXi Learnswitch VIB
VMware has a VIB (vSphere Installation Bundle; essentially a patch) that enables the ability for the vSphere Distributed Switches (VDS) to learn MAC addresses. This makes some very cool scenarios possible.
You see the VDS is essentially a software based layer 2 switch that has the ability to span accross multiple ESXi hosts. But due to the fact that the VDS resides inside the VMware environment it has never had the need to perform MAC learning like a normal physical switch does. This is because the VMware environment already knows what the MAC address is of each vNIC attached to each Virtual Machine. And because of this it will only forward traffic to Virtual Machines for which they're intended.
This poses a problem when people wish to nest ESXi installations. In other words, when someone wants to install an ESXi host inside of another one. You might ask, "when does anyone ever want to do this?" Well most often those who are wanting learn ESXi; VMware virtualization. This also makes for a useful development environment in which you can do whatever you want without effecting any production environments.
So when someone creates an ESXi host VM inside of another you end up with a layering effect. Your ESXi host VM could have a collection of it's own VMs. The only way for external network traffic to reach the nested VMs is to pass through the same virtual port on the VDS. This means that the virtual port on the VDS has multiple MAC addresses hanging off of it.
You can learn more about the learnswitch VIB here: https://labs.vmware.com/flings/learnswitch
William Lam provides a very good esplanation of the feature here: https://www.virtuallyghetto.com/2017/04/esxi-learnswitch-enhancement-to-the-esxi-mac-learn-dvfilter.html
I have come to find that most discussions of the Learnswitch touch on the desire to nest ESXi installations. But I've found that this feature is useful if you wish to nest ANY hypervisor (Linux KVM, VirtualBox, etc.) within an ESXi environment. The concept is the same.
To take things further, I've recenctly been working on a project to extend a distributed portgroup across a large layer 3 network with the use of a VXLAN tunnel provided by OpenVswitch. Not only is this possible but you can use any hypervisor at the distant end as long as you have a VXLAN tunnel end point to provide the transport.
Consider the picture below:
In this picture on the left the CentOS 7 VM running inside the ESXi host is providing a VXLAN tunnel entry point (VTEP) that extends VLAN 119 across a layer 3 network to a separate CentOS 7 KVM server on the right. In this case we have VMware ESXi and Linux KVM working together to host 4 virtual machines and geographically separated by a VXLAN tunnel. VM1, VM2, VM3, VM4 and the workstation hanging off the Nexus 5K layer 2 switch are all in the same subnet. They communicate with each other over layer 2 only. The CentOS 7 host on the right can be located anywhere in the world and as long as the IPs at the VTEPs can ping each other then the VXLAN tunnel will form an the VMs will be able to reach eachother. The ESXi Learnswitch allows the VDS to learn that the MAC addresses of VMs 3 and 4 are reachable through the port that the CentOS 7 VM eth0 interface is attached to.
In other words, when VM1 ARPs for the MAC address of VM3 (10.0.0.3) the ARP will be broadcast onto VLAN 119. It'll enter the eth0 inteface of the CentOS 7 VM and be sent down the VXLAN tunnel and come out on VLAN 119 at the CentOS 7 host on the right. VM3 will see the ARP and respond with it's MAC address. VM3's ARP response will travel back down the VXLAN tunnel and come out of eth0 onto VLAN119. The VDS will update its MAC address table and map VM3's MAC address to the port that eth0 is attached to.