diff --git a/capacity/datacenter.rst b/capacity/datacenter.rst index 1cf4647..58abbae 100644 --- a/capacity/datacenter.rst +++ b/capacity/datacenter.rst @@ -1,44 +1,39 @@ -|Capacity|.4 Datacenter Networks +|Capacity|.4 Service Differentiation -------------------------------------- While RED and ECN have not enjoyed wide-spread adoption in the Internet at large, and FIFO with tail drop continues to be the default queuing discipline, the mechanisms described in the previous two -sections have proven useful in narrow settings. Datacenter networks -are a noteworthy example, and so we use them to illustrate how these -mechanisms are used in practice. - -Datacenter networks have two properties that make them an ideal -candidate for enhanced resource management. One is that they are -self-contained, under the control of a single organization. Cloud -administrators can unilaterally decide that all routers implement a -particular queuing discipline, set the ECN bit when early signs of -congestion are detected, and enforce good behavior among all edge -hosts (the datacenter servers). The second reason is that datacenters -typically have sub-millisecond RTTs, so there is no worry that some -hosts will require 100ms, or longer, to react to an ECN. +sections have proven useful in several specific settings. We describe +here some of the scenarios where offering different levels of service +to different classes of traffic has proven both tractable and useful. -If anything, datacenters have proven to be such fertile ground for the -mechanisms described in this chapter that there is no single correct -answer; different combinations are packaged in products by vendors and -deployed in datacenter fabrics by clouds providers. It is also an -active area of research, so there is not yet any consensus about -exactly what combination works best. But datacenters do illustrate how -multiple mechanisms described in this chapter are used in concert to -build a complete solution; that coordination is the focus of this -section. - -First, we need a way for edge hosts and routers to exchange -information with each other. As noted in Section -|Capacity|.2, the -*Differentiated Services Code Point (DSCP)* plays this role. DSCP is -defined in RFC 2474, which was written in 1998, and predates modern -datacenters. But it prescribes a general mechanism that works across -many use cases, including datacenters. Specifically, DSCP allows -networks to distinguish between different *classes* of traffic, where -each class corresponds to packets that should be treated the -same. Each class, in turn, is assigned a *per-hop behavior* (PHB), where the -recommended set of behaviors include: +The possibility of sending latency-sensitive traffic over +packet-switched networks was known from at least the 1980s, but +increases in bandwidth in the 1990s drove greater interest in sending +voice and then video traffic over the Internet. Voice over IP (VOIP) +started to emerge as a real possibility and exposed the need to +control the latency experienced by a subset of the traffic passing +through routers and across networks. This interest in controlling the +latency of selected traffic led to a flurry of related research, and +to a set of activities at the IETF to standardize quality of service +mechanisms. The most successful of these was the Differentiated +Services architecture, also known as DiffServ. + +One major contribution of DiffServ was to redefine the ``ToS`` field +of the IP header to allow common behaviors, such as low-latency +queuing, to be requested by setting the *Differentiated Services Code +Point (DSCP)* to a well-known value. In addition, it standardized a +handful of *per-hop behaviors (PHBs)* that could be implemented using +common queuing mechanisms such as those described in Section +|Capacity|.2. With these two features in place, it became reasonably +straightforward to configure networks to support applications such as +VOIP. A device sending VOIP traffic marks the voice packets with the +well-known DSCP value; routers are configured to recognize that value +and place the corresponding packets into a queue that will deliver low +latency, such as a priority queue. + +The commonly used set of per-hop behaviors include: * **Default Forwarding (DF):** Regular best-effort traffic. @@ -56,14 +51,16 @@ backward compatibility with the priority (precedence) specified in the first three bits of the original ``ToS`` field, which continues to be used for high-value network control traffic, such as BGP and OSPF. All told, there are 2\ :sup:`6` = 64 possible DSCP settings, using the -first six bits of the ``ToS`` field; the other two bits are used by -ECN. The details of how a specific network might use these DSCP -settings are spelled out in multiple RFCs, each targeted at a -different use case—e.g., multimedia conferencing, multimedia -streaming, telephony, and so on—but it turns out that datacenter -networks have largely converged on a minimal subset that includes just -one or two AF classes, in addition to the high-priority network -control class and default best-effort class. +first six bits of the ``ToS`` field; the other two bits are available +to be used by ECN. The details of how a specific network might use +these DSCP settings are spelled out in multiple RFCs, each targeted at +a different use case—e.g., multimedia conferencing, multimedia +streaming, VOIP, and so on. + + +The definition of the Differentiated Services field, defining six +bits of the old ``ToS`` byte, is in RFC 2474, while the companion RFC +2575 spells out the larger architectural picture for DiffServ. .. admonition:: Further Reading @@ -76,7 +73,50 @@ control class and default best-effort class. `__. RFC 2475, December 1998. -Second, instead of probabilistically dropping packets, datacenters use +There are some remaining challenges, such as making sure that this +capability is not abused by endpoints that don't require low latency, +since an excessive amount of traffic in a priority queue still won't +receive low latency. There may be certain trusted endpoints, such as +the dedicated IP phones that started appearing in the early days of +VOIP. There are also ways to "police" traffic so that packets marked +with certain DSCP values are either limited in total bandwidth or +completely disallowed on some interfaces. These challenges are +manageable, especially in the enterprise networks where this +technology was particularly popular. While ever-increasing backbone +speeds limited the utility of DiffServ in the core of the Internet, +the capabilities remain available for managing bottleneck links in +enterprises and in the last mile of residential networks. They also +find application in modern datacenters. + +Two properties make datacenter networks ideal candidates for enhanced +resource management. One is that they are self-contained, under the +control of a single organization. Cloud administrators can, for +example, unilaterally decide on a single set of DSCP values and PHBs +to be used. They can arrange that all routers implement a particular +queuing discipline, set the ECN bit when early signs of congestion are +detected, and enforce good behavior among all edge hosts (the +datacenter servers). The second simplifying factor in datacenters is +that they typically have sub-millisecond RTTs, so there reason to +expect that most hosts will react in a timely manner to a +congestion markings. + +If anything, datacenters have proven to be such fertile ground for the +mechanisms described in this chapter that there is no single correct +answer; different combinations are packaged in products by vendors and +deployed in datacenter fabrics by cloud providers. It is also an +active area of research, so there is not yet any consensus about +exactly what combination works best. But datacenters do illustrate how +multiple mechanisms described in this chapter are used in concert to +build a complete solution; that coordination is the focus of this +section. It appears that datacenter networks have largely converged on +a minimal subset that includes just one or two AF classes, in addition +to the high-priority network control class and default best-effort +class. + + +In addition to DiffServ traffic differentiation, datacenters use ECN +in a manner that is slightly different than that described in the +previous section. Instead of probabilistically dropping packets, datacenters use ECN to provide direct feedback. Given the low round-trip times, ECN allows traffic sources to react quickly to queue buildup. But since we have more than one class of service, we need a different way to decide @@ -107,7 +147,7 @@ length calculation. Finally, the in-network mechanisms make assumptions about the edge hosts being well-behaved, which includes applying the right DSCP labels and limiting their sending rate. One general approach is for -the cloud to "police" senders; this acknowledges that a cloud hosts +the cloud to police senders; this acknowledges that a cloud hosts VMs that are able to run arbitrary code. This policing action can be implemented in the hypervisor that sits between the server and the tenant VM; in the NIC that connects the server to the datacenter diff --git a/capacity/design.rst b/capacity/design.rst index 07769a1..a7109bb 100644 --- a/capacity/design.rst +++ b/capacity/design.rst @@ -228,49 +228,35 @@ but when a packet happens to belong to a flow for which the router is currently maintaining soft state, then the router is better able to handle the packet. -|Capacity|.1.5 Differentiated Service +|Capacity|.1.5 Service Differentiation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The final design question is one the original IP specification raised, but didn't fully address: Should routers treat all traffic the same, or should it be possible to offer different levels of service to different classes of traffic? The original spec, RFC 791, defines an -8-bit *Type of Service* field (``ToS``), along with abstract settings -that can be summarized as a 3-bit "Priority" subfield and a 3-bit " -"Quality-of-Service" subfield. The former is used to identify -important packets, such as routing updates, and the latter supports a -three-way tradeoff between low-delay, high-reliability, and -high-throughput. The other two bits were saved for future use. - -The existence of the ``ToS`` field indicates a recognition that there -are good reasons to treat packets differently, but the actual purpose -of the field was to provide routers the information they might need to -properly set parameters on the underlying physical networks they -forwarded packets over. It was assumed those physical networks offered -different levels of service to different types of traffic; it was not -intended that the router itself would change its own packet forwarding -based on the setting. Routers remained purely best-effort, without -favoring one class of traffic over another. To further complicate -things, different vendors ended up using the ``ToS`` bit for different -purposes, and so there wasn't universal agreement on what the bits -meant. - -All of that changed many years after IP became ubiquitous, in ways -that we explore in this chapter. The general approach is often -referred to as *differentiated services*, indicating that not all -packets are treated exactly the same. We are still squarely in the -best-effort domain in that no promises are made, but short of -guaranteeing a throughput rate or an upper bound on jitter, there are -steps the network can take to differentiate the level of service various -classes of traffic receive. - -The following sections show how this is done by repurposing the -``ToS`` field already included in the IP headers. But finding the -header bits to denote how packets are to be treated is the easy -part. The challenge is that, once you decide to differentiate among -different classes of traffic, you need to decide (a) what classes to -support, and (b) how to modify packet forwarding so each class -receives the desired performance. This is a multi-faceted problem, -involving coordination among multiple mechanisms. We address it in the -context of a specific use case—datacenter networks—in Section -|Capacity|.4. +8-bit *Type of Service* field (``ToS``), that could be used to identify +important packets, such as routing updates, or to request low delay, +high reliability, or high throughput. The existence of the ``ToS`` +field indicates a recognition that there may be good reasons to treat +packets differently, but in practice this was rarely done and there +was no consistency in usage. + +Interest in service differentiation started to grow in the 1990s as +several factors arose almost simultaneously. The telcos started to +become very interested in packet-switched networks that could offer +quality-of-service guarantees suitable for high-quality voice and +video communications. The Internet became increasingly ubiquitous, and +link bandwidths increased rapidly to the point where it became much +more realistic to send large volumes of latency-sensitive traffic over +the Internet. Such traffic, it seemed, would not meet the application +needs if it received standard, best-effort service. All of this led to +a great deal of research on how to support service differentiation in +the Internet, and more broadly on packet-switched networks. A lot of +that research ended up being overtaken by events, as well-provisioned, +high-bandwidth networks proliferated, but some of it had a lasting +impact. This is particularly true for environments where one entity +controls the end-points and the bottleneck links, such as enterprise +networks and cloud datacenters. In this chapter we cover the +mechanisms that have actually succeeded in providing different classes +of service in real-world deployments. diff --git a/capacity/scheduler.rst b/capacity/scheduler.rst index c76933e..3f54213 100644 --- a/capacity/scheduler.rst +++ b/capacity/scheduler.rst @@ -88,43 +88,6 @@ policing is managed in a later section, but for now, the key is to recognize the potential value of priority queues, and the complications in using them. -.. sidebar:: Latency-sensitive Traffic and Differentiated Services - - *The possibility of sending latency-sensitive traffic over packet-switched - networks was known from at least the 1980s, but increases in - bandwidth in the 1990s drove greater interest in sending voice and then video - traffic over the Internet. Voice over IP (VOIP) started to emerge - as a real possibility and exposed the need to control the latency - experienced by a subset of the traffic passing through routers and - across networks. This interest in controlling the latency of - selected traffic led to a set of activities at the IETF to - standardize quality of service mechanisms. The most successful of - these was the Differentiated Services architecture, also known as - DiffServ.* - - *One major contribution of DiffServ was to redefine the* ``TOS`` *field - of the IP header to allow common behaviors, such as low-latency - queuing, to be requested by setting the* Differentiated Services Code Point - *(DSCP) to a well-known value. In addition, it standardized a - handful of* per-hop behaviors *(PHBs) that could be implemented - using common queuing mechanisms such as those described in this - section. With these two features in place, it became reasonably - straightforward to configure networks to support applications such - as VOIP. A device sending VOIP traffic marks the voice packets with - the well-known DSCP value; routers are configured to recognize that - value and place the corresponding packets into a queue that will - deliver low latency, such as a priority queue. There are remaining - challenges, such as making sure that this capability is not abused - by endpoints that don't require low latency, since an excessive - amount of traffic in a priority queue still won't receive low - latency. But these challenges are manageable, especially in the - enterprise networks where this technology was particularly - popular. While ever increasing backbone speeds limited the utility - of DiffServ in the core of the Internet, the capabilities remain - available for managing bottleneck links in enterprises and in the - last mile of residential networks. They have also found use if - modern datacenters, as discussed in Section |Capacity|.4.* - |Capacity|.2.3 Fair Queuing ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~