On Linux Network Interfaces and Virtual Bridges

31 August 2020

Suppose you have some virtual machines in a host-only network governed by VirtualBox referenced with the interface vboxnet0 and also some individual Docker containers in their own default network domain, referenced with docker0. Let there be also a QEMU machine with an unhooked interface.

Linux-based operating systems, in my experience, are praised for their network capabilities. With that in mind, it shouldn’t be too hard to “put these devices under a virtual LAN”, should it be?

Could we create a unified virtual network - preferably with an internet connection - in a setting similar to our typical home LAN network setup? Or maybe this action does not make sense at all?

This is the topic that I would like to share my experience on with you.

Understanding the concepts

But what is exactly a network interface? What is the difference between a switch and a router? What is a network bridge? And can I emulate all these ton of devices virtually?

Linux Network interface

A Linux network interface is an OS-level abstraction for a network connection - it encapsulates the concepts of the OSI layers 2 (data link) and 3 (network). I emphasize the phrase abstraction. It is a “thing”, that you can send packets to, assign an IP address to, set it up or down, route, create firewall rules for, and so on. It is also visible to your applications, such as your browser or a server you run.

It is usually backed by either a physical Network Interface Card / NIC (OSI level 1) or in some cases a special software. Typically, a program that creates a network interface for you is some virtualization solution that features network isolation - the appearing new network interface in your host OS is your link to the created virtual network. You can also think of these links as the “virtual cable endings” in your OS, whether there is a physical device or some software on the other end.

Note, that a virtualization solution that creates virtual networks is not forced to give you a link - consider the internal networking mode of VirtualBox. In this scenario, VirtualBox emulates a network bridge for the machines, but it does not connect the host operating systems to this bridge - that is, no new interface is going to appear and you cannot reach the virtual machines from this direction.

Hubs, network bridges and switches

Before understanding the virtual bridges, let me start with the classical concepts. There are multiple ways to connect computers in a way they can communicate with each other.

Hubs and bridges

One of the simplest solutions is using a hub, which is basically a “broadcast box”. All machines that are plugged into the hub receive all the network packets of their neighbors. While it has some negative security-related implications, it can also put a burden on all the connections as every computer receives messages that aren’t actually their business. It can quickly become noisy - like when a lot of people argue in crowded place.

To my best knowledge, they are not widely used anymore.

As the network grows, we can install some bridges here and there to lower traffic and increase security.

A classical bridge has only two ports, but it can decide whether the message it receives on one port should travel to the other side or should get dropped - therefore, messages from two segments only pass through if they are meant to.

Switches

A classical switch is virtually a network bridge with more ports - it can also be thought of as a “smart hub”. Instead of broadcasting all communication to all of its ports, it delivers the packets only to its correct destination. It can operate on either the data link layer (L2) or on the network layer (L3).

Routers

A router is used to forward packets from one network to the other without combining the network segments. It has a routing table that decides which packets are forwarded to what interfaces. Usually it has a firewall and does something called Network Address Translation or NAT for short, a technique, which, amongst others, enables a household to have multiple machines connected to the Internet with one (external) IP address.

That box with the flashing lights on the shelf that we call a “WiFi router” is not only a network router (that forwards data from the WAN and the LAN network), but also an ethernet switch, a wireless access point (a “wireless hub”, more or less) and a bridge that connects the broadcast domain of the LAN ethernet interfaces and the clients of the wireless access points.

Virtual bridges

In Linux, a virtual bridge is the analogous device for a switch. You can “plug” into it an arbitrary number of other interfaces and it will optionally provide your OS another link (br0 in the figure). If you want to access the newly created, combined network, then you can do so by assigning this interface an IP address - now you’re connected to the network that bridge creates. Although these new bridge links cannot be plugged into other virtual bridges again, the virtual bridges can be connected using virtual ethernet pairs (veth pairs).

Virtual ethernet pair

In Linux, a virtual ethernet pair or VETH pair is a virtual network device that gives you two links into the OS. The packets sent into one appear in the other end. You can think of it as a virtual patch cable. It is usually used to connect isolated network namespaces in Linux, but we are also going to use them now to connect multiple bridges to form a single virtual network.

Linux network namespace

Last but not least - we have network namespaces. Keeping it short for this time, a network namespace in Linux enables you to have multiple network stacks on the same machine creating running virtual machines. By different network stack I primarily mean the set of visible network interfaces, the routing table and the IPTables firewall rules. In Linux, container technologies such as Docker makes heavy use of this kernel feature.

We are going to use a namespace for forwarding packets between our virtual appliances and the interface of the physical network card.

The final goal

With all that in mind, our final stop is to create this network in a testing environment. It is a unified “LAN” network for the virtual machines with Internet access.

  1. We are going to combine (bridge) the three networks so that the appliances can reach each other
  2. We are going to provide an Internet connection in a similar way a “household” router device does (the physical full-featured device)

In the next episode we are going to create this network with example machines and containers.