Neutron security groups and OVS, Part 3: tracing OVS’s hooks and claws…

So far we’ve looked at:

  • What tap interfaces are and why the VMs require them for network connectivity. (Part 1)
  • How security groups are implemented as iptables rules. (Part 2)
  • The implementation detail that iptables is just a front-end to the netfilter framework within the kernel, a framework that operates at layer 3.

None of that explains why we need the linux bridge in the middle, however.

In order to perform its function as a userspace switching platform OVS needs to have access to packets at the layer 2 level. That is, it needs to see Ethernet headers and data to decide which actions to take in response to a packet.

To enable such behaviour the kernel exposes receive handlers for each network device on the system. These receive handlers operate at a very low level, a level so low that the handlers are called before the packet is processed by higher level protocols (including netfilter).

A kernel driver – such as the OVS kernel driver – can install a receive handler on an interface. This receive handler will be called for every packet that arrives on that interface. The handler will attempt to process the packet before the kernel passes the packet up the protocol stack. Importantly, the handler can mark the packet as ‘consumed’: when this occurs, the kernel considers that packet fully processed and will not pass it to higher level protocols.

When you attach a system interface – such as a tap interface, one end of a virtual ethernet (veth) pair, or a physical NIC interface – to an OVS bridge, OVS installs a receive handler against that network device to ensure OVS gets hold of the packets at the earliest possible moment to decide how to switch them.

So what?

Bottom line: If we attach the tap interface to an OVS bridge, OVS will install a receive handler against it. For every packet that comes in, OVS will process and mark that packet as consumed. Once the packet is consumed the kernel ceases any further processing – that includes sending it to higher level IP layer. Consequently our now-consumed packet never makes it to the IP layer, never makes it to netfilter, and our security group rules are never run against it.

This is why we need to put something between the tap interface and OVS that allows netfilter to process all the packets coming into or out of the tap interface before passing the result on to OVS. That something is the linux bridge, with the veth pair to transport the packet into OVS.

If using the OVS native firewall driver, then OpenFlow rules are written to achieve the same effect as iptables, with the addition that the kernel conntrack module is involved to enable stateful tracking of connection state.

Tour-de-code

If you’re interested in a more code-level run through of what occurs, take a look at the following links:

OVS: ovs/vport-netdev.c:96 – the ovs_netdev_link function which installs the handler using the netdev_rx_handler_register kernel function.

OVS: ovs/vport-netdev.c:72 – the netdev_frame_hook function, the receive handler installed by the OVS kernel driver within ovs_netdev_link on each system interface. Note how this function returns the RX_HANDLER_CONSUMED return code.

Kernel: net/core/dev.c:4254 – the netdev_rx_handler_register kernel function, which installs the receive handler onto the network device.

Kernel: net/core/dev.c:4330 – the __netif_receive_skb_core function, in particular line 4409 where the receive handler is executed and the return value is checked for RX_HANDLER_CONSUMED.

Leave a Reply

Your email address will not be published. Required fields are marked *