Knowledgebase: Corporate Colocation Essentials
Posted by Neal Godfrey on 25 July 2013 07:02 PM
One common source of confusion amongst those new to the world of datacenter networking involves the use of the terms "layer 2 switching" and "layer 3 switching." Some of you will no doubt be familiar with the seven layer OSI model, which divides computer networking up into seven conceptual layers, based on function. For those of you who aren't, our discussion must begin with an overview of this model. First, some terminology:
Host - An individual computer, connected to a network (or the Internet).
Network Interface Card - The ethernet card or wifi adapter on your computer.
Address - a unique identifier for another computer on a network.
Layer 1 - Physical
The physical layer consists of the ethernet and fiber optic cables that link computers together, as well as the wireless signals in wifi networks; it also consists of the raw hardware that transmits and receives data on each end. An example of a layer 1 device would be a hub; hubs, though now mostly replaced by switches, were at one time commonly used to connect multiple servers or workstations into a network; they are pure layer 1 devices in that they take an input signal and repeat and amplify it through all of the connected interfaces on the device. This introduces performance problems; only one attached host can transmit data at any given time. When two hosts try to transmit simultaneously, the result is a collision, and each host will then use an algorithm to wait a random amount of time before retransmitting (this is called 'back-off'). The more hosts you have connected to a hub, the more collisions that occur. Since the time wasted on collisions, and on waiting to retransmit, is time not spent sending information, the result is that the network becomes slower and slower the more hosts you connect to it, until finally it reaches an manageable state.
Layer 2 - Data Link Layer
This layer is the most minimal logical layer, above the physical layer. At layer 2, devices communicate via MAC address, a value that is generally hardcoded into the Network Interface Card. Each host on a network has a unique MAC address; when one host wishes to send a message to another, it, at the lowest level, sends an Ethernet frame to that specific MAC address. The classic layer 2 device is the common Ethernet switch. A switch, instead of sending all received traffic out all of its interfaces (which is how hubs operate), will instead learn which hosts are behind specific interfaces, and send traffic addressed to those hosts through that interface only. These devices are what are commonly known as "layer 2 switches." They are extremely fast, but have one weakness, which we will explore next.
Layer 3 - Network Layer
This layer deals in packets instead of frames, and IP addresses instead of MAC addresses. Whereas MAC addresses are hardwired unique identifiers built into network interface cards, that are only accessible on a local network, Layer 3 IP addresses are globally routable addresses that allow communication between any two computers connected to the Internet, anywhere in the world. When a computer needs to transmit IP traffic to a specific address, it has to look up the MAC address on its local network that corresponds to this IP address. When you configure an IP address on a computer, you have to set three values: the address, the subnet mask, and the default gateway. When your computer goes to send traffic, first, it determines whether or not the IP address is that of a host on the local network, or on a remote network. It does this using the subnet mask; this value, commonly 255.255.255.0 on residential networks (allowing 256 IP addresses), but frequently other values on production or datacenter networks, helps the computer to demarcate the bounds of the IP network it is immediately connected to. If the IP address it is sending traffic to is on the local network, it sends out a special broadcast ethernet message called an ARP (address resolution protocol) request. This request tells all hosts on the local network that the sending host is trying to learn the MAC address associated with the given recipient IP address so it can send traffic to it. Whichever host actually has that IP address configured on it will then respond with an affirmative answer, allowing the sending host to learn its MAC address. The sending host then proceeds to transmit layer 2 ethernet traffic to that specific MAC address.
This of course exposes the main problem with layer 2 switches. Over time, on extremely large networks, the amount of broadcast traffic due to ARP requests and other related activity will become excessive. This results in a condition not dissimilar to that of an oversaturated layer 1 hub; hosts connected to the network cannot send regular traffic as frequently as they would otherwise, because the network is blocked while the layer 2 switch handles the broadcast traffic. This eventually reaches a point where the layer 2 network's performance is seriously impacted to an adverse degree. Many network engineers prefer to not allow more than 256 hosts on the same layer 2 network, or broadcast domain. 256 hosts is of course a miniscule fraction of the billions of systems connected to the Internet.
Thus, to allow networks to scale, and to let the Internet happen, we have to do something called routing. When you configure an IP address on your computer, the default gateway that you set is the IP address of your router. This router allows your local host to talk to other hosts on the Internet that aren't on your local area network. This works as follows: when your computer wants to send IP traffic to a host, recall that it first sees if the host is on its local network. If it is on the local network, it then broadcasts an ARP request to all the hosts on the network to find the corresponding MAC address. If its not on the local network, your computer will instead broadcast an ARP request, to obtain the MAC address of your router. Your computer will then send the traffic to that specific MAC address.
When the router receives the incoming traffic, it will note that the destination IP address on it is not the same as its local address. It will then forward the traffic to the next router that has a route to that specific address. The way this usually works is that routers are situated so that one of their network interfaces is on a downstream local network, such as the wifi network within your house, and the other interface is configured on an upstream "wide area network", which is just another ethernet network that connects that router to upstream routers, and the rest of the Internet beyond. Routers on end-user networks usually just have a default gateway assigned to them, just like the computers behind them. Routers that connect to multiple upstream networks, like routers near the core of the internet, will use a dynamic routing protocol instead of a statically assigned default gateway; this routing protocol lets multiple routers communicate and exchange information on which router has the best route to any given network. If one router should fail, the other routers communicating using this dynamic routing protocol can automatically bypass it and send traffic via different route.
Routers have one disadvantage: they tend to be much slower than ethernet switches. The processing required to do IP routing was historically implemented in software on a typical router. Each router that separates any two IP addresses on the internet tends to introduce a performance penalty; the more hops in the route, the more latency in the connection, as a general rule. The most commonly used dynamic routing protocol in the Internet backbone, BGP, consequently uses hop counts as its metric for selecting the best route between any two destinations.
So, with this behind us, we now know what layers 1, 2 and 3 of the OSI stack are, we know what a layer 2 switch is, and we know what a router is.
What then is a layer 3 switch?