Neutron security groups and OVS, part 1: tap interfaces and VM connectivity

Today I did a little digging into the implementation of security groups when using OpenVSwitch. In particular, I was curious about this: why is it that security groups require the creation of a linux bridge on the compute node? Why can’t we just attach the VM directly to the OVS integration bridge (br-int) and set iptables rules on the VM interface like we otherwise would?

This applies when using the iptables_hybrid firewall driver for Neutron with the ML2+OVS subsystem. If you use the openvswitch firewall driver, these firewall rules are implemented entirely by OpenFlow rules that use the conntrack module in the Kernel.

This was originally going to be one post but I ended up rambling on for so long I opted to split it into a few related posts. This is the first!

Tap interfaces?

Tap interfaces are a virtual interface created by the kernel that allows raw Layer 2 packets to pass between the kernel and a userspace application. Tap interfaces are effectively a pipe – if you write packets into it, someone reading from the interface will see that packet arrive, and vice-versa.

Every NIC for a VM is assigned a tap interface that enables raw network traffic to be passed to the VM from the host – specifically, the traffic leaves the host kernel and is processed by the QEMU process that is running your VM.

You can identify tap interfaces by running ip link on the compute host:

root@cop1 ~]# ip link
65: tape814ad4e-20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master qbre814ad4e-20 state UNKNOWN mode DEFAULT qlen 1000
 link/ether fe:16:3e:b0:fb:42 brd ff:ff:ff:ff:ff:ff

The hexadecimal string after ‘tap’ in the name is the first few characters of the ID of the neutron port assigned to the interface:

# openstack port list -c ID -c MAC\ Address -c Status
| ID                                   | MAC Address       | Status |
... snip ...
| e814ad4e-201b-4139-b20c-7ecf4252133e | fa:16:3e:b0:fb:42 | ACTIVE |

There will be one tap interface per port assigned to the VM, and one port per network the VM is attached to. These tap interfaces will appear in the VM as ‘eth0’, ‘eth1’, etc.

Passing network traffic between host and VM

When the host kernel wants to pass layer 2 traffic to the VM to enable the VM’s network stack to process it, the host kernel writes the traffic to the tap interface. The VM kernel reads from its network device (such as ‘eth0’) to receive those Layer 2 packets. When the VM wants to send a packet over eth0 the VM kernel writes the packet to eth0. The host kernel then reads the packet by reading the from the tap interface and processes it as it would any other packet entering from an interface.

As an aside, a tunnel interface is another type of virtual interface very similar to the tap interface. While tap interfaces pass full layer 2 information, tunnel interfaces only operate at layer 3. That is, they will only accept IP packets both in and out. All network-level information is stripped. Tunnels are quite common for VPN implementations when you don’t need to be on the same layer 2 network as the remote hosts. In these implementations the kernel routes outbound traffic into the tunnel, the packet is read by the VPN software, encrypted, and wrapped into a new IP packet destined for the VPN server. The VPN softtware then writes this new IP packet back to the tunnel where it’s routed over your default gateway for delivery to the VPN server.

If you want to go deep on tun/tap implementation, check out this excellent post.

That’ll do for now. The next post will describe how security groups are implemented as iptables rules, and we’ll take a tour of some of these rules. After that we’ll get into the specifics of how OVS reads packets from interfaces, and finish with why that matters for the implementation of security groups.

Stay tuned!

Leave a Reply

Your email address will not be published. Required fields are marked *