QoS and traffic shaping

Quality of service, from the handset to the far end

I design QoS (quality of service) so calls, video and the applications that matter keep working when links are busy. This page shows how traffic is marked, trusted, queued, dropped, policed and shaped. It follows the markings across Wi-Fi, the campus, MPLS (Multiprotocol Label Switching), SD-WAN (software defined WAN) and tunnels. It also shows how I prove a policy works and find faults.

DSCP and DiffServ
Trust boundary
LLQ and CBWFQ
WRED and ECN
Policing and shaping
Hierarchical QoS
Voice and video
Wi-Fi to SD-WAN

Start with the basics My related work

Live view · calls first, even on a full link

Basics

Where queues form, and why calls suffer first

QoS is a set of tools that decide which traffic goes first, which gets a guaranteed share and which is dropped when there is not enough bandwidth for everything. It never adds bandwidth. It only acts when a queue forms, and queues form at a few predictable places.

Swipe sideways to see the whole diagram.

Step 1

Most queues form where a fast link feeds a slower one. A 1 Gb/s LAN (local area network) feeding a 100 Mb/s WAN (wide area network) circuit is the classic case. When a burst arrives faster than the circuit can send it, packets wait in the router's queue.
Queues also form where many links feed one: 48 access ports sharing a 10 Gb/s uplink, or several servers sending to one storage port. The five minute average looks low, but bursts lasting a few milliseconds still overflow small switch buffers. These are microbursts.
Wi-Fi is shared and half duplex. Every client and access point on a channel takes turns, so traffic queues for airtime as well as inside the access point. WMM (Wi-Fi Multimedia) gives voice a better chance at each turn.
A carrier often delivers a 100 Mb/s service on a 1 Gb/s port and polices anything above the contract. Its policer drops whatever is over the rate, voice included. If I shape to the contract myself, I choose what waits and what drops.
Traffic from a data centre on 1 Gb/s to a branch on 20 Mb/s queues inside the carrier network, before it ever reaches my branch router. The only place I can control it is the sending end, by shaping per site at the hub.
Each queued packet adds delay. Delay that changes from packet to packet is jitter. A full queue drops packets. Voice and video feel all three at once, while a file copy just takes longer. That difference is why QoS exists.

What each kind of traffic can tolerate

Traffic	One way delay	Jitter	Loss	What it means for design
Voice	150 ms or less	Under 30 ms	1% or less	Small, steady packets. Needs a priority queue and a fixed budget per call.
Interactive video	150 ms or less	Under 30 ms	Under 1%	Bursty frames. Sized for the peak, not the average.
Streaming video	Seconds are fine	Hidden by buffering	Low	The player buffers ahead, so it needs bandwidth more than priority.
Business applications	Matters to users	Tolerant	Retransmitted	TCP (Transmission Control Protocol) resends lost data, so loss shows up as slowness.
Bulk and backups	Tolerant	Tolerant	Tolerant	Can be squeezed hard at busy times and catch up later.

Indicative targets from Recommendation G.114 of the International Telecommunication Union (ITU-T), for one way delay, and from Cisco's enterprise QoS design guidance. Each organisation agrees its own.

The QoS toolset, in the order a packet meets it

Classify. Decide which class a packet belongs to, from its marking, its addresses and ports, or the application.
Mark. Write that decision into the packet, as a DSCP (Differentiated Services Code Point) value, so later devices need not work it out again.
Police. Hold a class to a rate at the edge of trust by dropping or re marking the excess.
Queue. Give each class its own queue when an interface is congested.
Schedule. Decide which queue sends next: priority first, then guaranteed shares.
Drop early. Drop or mark a few packets before a queue is full, so TCP slows down smoothly.
Shape. Hold traffic leaving an interface to a lower rate, such as the contracted rate of a circuit.
Prove. Read the counters, capture the markings and test under load, because a policy that is never measured is a guess.

QoS never adds bandwidth

It decides who waits and who is dropped when there is not enough. If a link is full all day, the answer is more capacity, with QoS protecting the important traffic until it arrives.

It only acts in congestion

With no queue, every packet goes straight out and QoS does nothing. That is why a policy can look perfect in a quiet lab and fail at 3 pm on a Monday.

Delay, jitter and loss

Delay is how long a packet takes end to end. Jitter is how much that delay varies. Loss is the share of packets that never arrive. Real time traffic is hurt by all three.

The same ideas sit behind the shorter QoS notes on the switching, routing and WAN pages. This page goes deeper.

Marking

Six bits that carry the priority

A marking is a label in the packet that tells every device on the path how to treat it. At Layer 3 it is the DSCP (Differentiated Services Code Point), six bits in the IP header. At Layer 2 there are three bit fields in the 802.1Q tag, the MPLS label and the Wi-Fi header. DiffServ (differentiated services) works because each device acts on the label alone, with no signalling between them. Each standard treatment is called a per hop behaviour (PHB).

Swipe sideways to see the whole diagram.

DSCP in the IP header. After the version and the IHL (header length), the second byte of the IPv4 header was the type of service (ToS) byte. RFC 2474, one of the Request for Comments (RFC) documents that define internet standards, renamed it the DS (differentiated services) field. Its first six bits are the DSCP, giving 64 values, and RFC 3168 gave the last two bits to ECN (explicit congestion notification). The first three bits are the old IP precedence, which is why class selector values line up with it. A command that asks for a ToS value wants the whole byte, which is the DSCP times four: EF (expedited forwarding), the voice marking, is 184, or 0xB8.

CoS in the 802.1Q tag. The TPID (tag protocol identifier) marks the frame as tagged. The tag then carries a 3 bit priority code point (PCP), usually called CoS (class of service), so Layer 2 switches can queue frames without reading the IP header. It only exists on tagged links: trunks and the voice VLAN (virtual LAN) on a phone port. Cisco switches build CoS from the top three bits of the DSCP by default, so EF becomes CoS 5.

TC in the MPLS label. A provider core switches on labels and never reads my IP header, so the provider edge copies a class into the label's 3 bit traffic class field, once called EXP (experimental). By default it copies IP precedence. S marks the bottom of the label stack and TTL (time to live) stops loops. RFC 3270 describes three ways a provider can treat my DSCP across its network: uniform, pipe and short pipe. The difference decides whether a re mark inside the provider network reaches my packet.

User priority in the Wi-Fi header. WMM uses the 802.11 QoS control field. Its user priority (UP), 0 to 7, travels in the TID (traffic identifier) bits. The UP puts the frame into one of four access categories: voice, video, best effort and background, each with shorter or longer waits for the channel. The other fields, EOSP (end of service period), Ack policy, A-MSDU (aggregate MAC service data unit) and TXOP (transmit opportunity), handle power saving, acknowledgements, aggregation and airtime. Downstream, the access point sets UP from DSCP using RFC 8325. Upstream, clients choose their own UP, and some get it wrong, so I trust the DSCP inside the frame instead.

Animation

The per hop behaviours

Behaviour	Name	DSCP	Binary	What I use it for	Defined in
DF	Default forwarding (also CS0)	0	000000	Best effort: everything not marked otherwise	RFC 2474
CSx	Class selector	8 times x: 8 to 56	xxx000	Matches the old IP precedence. CS6 for network control, CS3 for call signalling	RFC 2474
AFxy	Assured forwarding, class x, drop precedence y	8x plus 2y: 10 to 38	xxxyy0	Four classes of data, each with three drop levels	RFC 2597
EF	Expedited forwarding	46	101110	Voice: low delay, low jitter, low loss	RFC 3246
VA	Voice admit	44	101100	Calls that passed admission control, kept apart from other EF	RFC 5865
LE	Lower effort	1	000001	Scavenger traffic, below best effort	RFC 8622
NQB	Non queue building	45	101101	Small, sparse flows kept out of queues that bulk flows fill	RFC 9956

The assured forwarding grid

Class	Low drop	Medium drop	High drop
AF1 · bulk data	AF11 = 10	AF12 = 12	AF13 = 14
AF2 · transactional data	AF21 = 18	AF22 = 20	AF23 = 22
AF3 · streaming	AF31 = 26	AF32 = 28	AF33 = 30
AF4 · interactive video	AF41 = 34	AF42 = 36	AF43 = 38

Within one AF class, traffic with a higher drop precedence is dropped first when the queue fills. A policer can re mark AF11 to AF12 or AF13 instead of dropping it.

Try it: decode a marking

Choose a name or type any DSCP value from 0 to 63 to see its bits, its ToS byte, what it becomes at Layer 2 and over Wi-Fi, and how to match it. Without scripts, the worked example below shows EF.

Named value

DSCP value

Orange bits are the DSCP. The last two bits belong to ECN.

Name: EF, expedited forwarding
DSCP in binary: 101110
Whole ToS byte: 184 (0xB8), with ECN bits 00
IP precedence: 5
Default CoS and MPLS TC: 5
Wi-Fi (RFC 8325): UP 6, voice access category
Typical use: Voice media, such as Teams audio. Strict priority queue.
Match it: match dscp ef, Wireshark ip.dsfield.dscp == 46, tcpdump ip[1] & 0xfc == 0xb8

EF (46): expedited forwarding, the standard marking for voice.

Mark once, near the source

The phone, the access switch or the endpoint marks. Every device after that only reads the marking, which keeps the policy simple and the core fast.

Layer 2 marks do not travel

CoS lives in the 802.1Q tag, and each routed hop removes it. DSCP survives end to end unless something rewrites it, so DSCP is the marking I design around.

CS1 or LE for scavenger

Most enterprise models, including mine, still mark scavenger traffic CS1. RFC 8622 now recommends LE (DSCP 1), because some networks treat CS1 as better than best effort. I check before handing traffic to another network.

Class models

Few classes, applied the same everywhere

A class model groups applications by what they need from the network. I start with as few classes as the business needs, because every class has to be built, tested and supported on every platform. The models below grow from four to twelve classes.

Swipe sideways to see the whole diagram.

Four classes. The smallest model that still protects calls: one priority queue for voice and interactive video, a small class for routing and call control, one for the applications the business depends on, and everything else. It suits small sites and carriers that sell four classes.

Eight classes. My usual enterprise model. Voice gets a policed priority queue. Interactive video gets a second priority level where the platform has one, so a video burst never delays voice, or a guarantee, as in my IOS XE example. Network control and call signalling get small protected shares. Transactional data gets a guarantee with WRED (weighted random early detection). Scavenger traffic is squeezed first.

Twelve classes. Cisco's version of the twelve service classes in RFC 4594, with call signalling on CS3 and broadcast video on CS5. RFC 4594 itself, and Microsoft Teams, use CS5 for signalling instead. I only go this far where the applications need it, such as broadcast video or a large contact centre, and I write the choice down.

Animation

My eight class model in detail

Class	Examples	DSCP	Queue	Indicative share
Voice	Phone and softphone audio, Teams audio	EF	Priority, policed	10%
Interactive video	Video calls and meetings, Teams video	AF41	Priority level 2, or a guarantee	23%
Streaming video	Training videos, town halls on demand	AF31	Guarantee	10%
Network control	OSPF (Open Shortest Path First), BGP (Border Gateway Protocol), HSRP (Hot Standby Router Protocol) and BFD (bidirectional forwarding detection) packets	CS6	Small guarantee	5%
Call signalling	SIP (Session Initiation Protocol), SCCP (Skinny Client Control Protocol), Teams signalling	CS3, and CS5 from Teams	Small guarantee	2%
Transactional data	ERP and CRM screens, Citrix, Teams screen sharing from Windows	AF21	Guarantee with WRED	24%
Best effort	Web, email, anything unmarked	DF	Fair queue	25%
Scavenger	Backups, updates, peer to peer	CS1	Minimum, dropped first	1%

Indicative shares for a WAN edge. ERP is enterprise resource planning and CRM is customer relationship management. Microsoft recommends AF21 for Teams screen sharing on Windows, so it lands in transactional data. Teams on Mac and mobile devices marks sharing AF41, like video.

Fitting my classes into a carrier's

My class	DSCP leaving my edge	Carrier class (example of a four class service)
Voice	EF	Real time: strict priority up to the contracted rate
Interactive video	AF41	Real time, or the highest data class if real time is too small
Streaming, transactional, signalling, network control	AF31, AF21, CS3, CS6	Business data classes
Best effort and scavenger	DF, CS1	Default

Carriers typically sell between three and six classes. I map mine into theirs at the CE (customer edge) router, and I ask what happens to traffic above each class's contracted rate: some carriers drop it, others re mark it to default.

Fewer classes, fewer faults

Every class must be configured, tested and monitored on every platform. Four well run classes beat twelve that nobody can explain.

Grow by splitting

Each model splits a class of the smaller one, so a site can move from four to eight classes without re marking anything at the edge.

Write the model down

The class table, its DSCP values and its shares are a design standard, owned by the design authority, not a setting someone remembers.

Trust

Mark at the edge, trust only what I marked

Classification decides which class a packet belongs to. Marking writes that decision into the packet. I do both as close to the source as I can, then decide which devices to trust. Everything beyond the trust boundary relies on the marking, so the boundary is a security control as much as a QoS one.

Swipe sideways to see the whole diagram.

Step 1

A Cisco phone marks its voice packets EF (46) and its call signalling CS3 (24), and tags them CoS 5 and CoS 3 on the voice VLAN. A phone is a device I manage, so its markings are ones I can trust.
The PC plugged into the phone's second port is on the data VLAN. Some applications mark their own traffic EF, by design or by mistake, and a user with admin rights can mark anything. If the network believed it, a download could jump ahead of every call.
The access port is the trust boundary. With conditional trust it accepts markings only while CDP (Cisco Discovery Protocol) sees a Cisco phone. An input policy resets the PC's traffic to 0 unless a rule says otherwise, such as Teams media from managed laptops.
Uplinks between my own switches trust DSCP, so distribution and core switches just queue on the marking they receive. Where a link is tagged, CoS is rebuilt from DSCP. No device inside the boundary classifies again.
The WAN router puts each DSCP into its WAN class, shapes to the circuit and queues inside the shaper. It also maps my classes to the carrier's. The carrier's PE (provider edge) router copies IP precedence into the MPLS traffic class, and may police or re mark traffic above each class's contract.
Most marking faults hide here: a carrier or a tunnel resets DSCP to 0 and the far site treats voice as best effort. I capture on both sides of the WAN and compare. EF must leave as EF and arrive as EF.

Ways to classify, from simplest to smartest

Method	How it works	Good for	Watch out for
Trust the marking	Accept the DSCP or CoS the device set	IP phones, access points, my own switches and routers	Anything a user controls can mark itself
Port or VLAN	Everything on the voice VLAN, or on a camera port, gets one class	Phones, cameras and OT (operational technology) devices	A PC moved onto the wrong VLAN or port
ACL (access control list)	Match addresses, protocols and ports	Known servers, Teams port ranges, backup servers	Lists go stale as applications change
NBAR2	Network Based Application Recognition inspects packets to name the application	Mixed internet and SaaS (software as a service) traffic at the WAN edge	CPU (processor) load, encrypted traffic and protocol packs that need updating
Endpoint policy	Windows Group Policy, or the application itself, marks at the source	Managed laptops running Teams	Unmanaged devices, and the network must trust the laptops

Marking Teams media at the source

Teams traffic	Source ports	DSCP	Name
Audio	50000 to 50019	46	EF
Video	50020 to 50039	34	AF41
Screen and application sharing	50040 to 50059	18	AF21
Calling and meeting signalling	50070 to 50089	40	CS5

Microsoft's recommended values for the Teams desktop client on Windows. If the port ranges are changed in Teams, the policies must change with them.

PowerShell on Windows, as Microsoft documents it. Group Policy sets the same rules across many laptops.

New-NetQosPolicy -Name "Teams Audio" -AppPathNameMatchCondition "ms-teams.exe" `
  -IPProtocolMatchCondition Both -IPSrcPortStartMatchCondition 50000 `
  -IPSrcPortEndMatchCondition 50019 -DSCPAction 46 -NetworkProfile All
# Repeat for video (34), sharing (18) and signalling (40), or use Group Policy:
# Computer Configuration > Windows Settings > Policy-based QoS
Get-NetQosPolicy        # confirm what the laptop will mark

Reset what I do not trust

Untrusted ports re mark to 0 or classify by rule. A marking from a guest laptop, a contractor's device or the internet is a request, not an instruction.

Police priority at the edge

Even trusted ports get a policer on the priority class sized for one phone, such as 128 kb/s, so a faulty or spoofed device cannot flood the voice queue for everyone.

Re mark at every border

Traffic arriving from a carrier, a partner or the internet is re marked at my edge to my own model. Their markings meant something in their network, not mine.

Queuing

Who goes next: queues and the scheduler

When packets arrive faster than an interface can send them, they wait in queues. The scheduler decides which queue sends next. That one decision is where QoS turns into lower delay for calls and a fair share for everything else.

Swipe sideways to see the whole diagram.

Step 1

With one queue, packets leave in the order they arrived. A voice packet that lands behind a burst of 1,500 byte data packets waits for all of them, and the wait changes with every burst. That changing wait is jitter.
Every interface has a small hardware transmit ring that feeds the wire. The software queues only fill when that ring is full, which is why QoS only acts during congestion. A very large ring hides congestion from the policy and adds delay of its own.
CBWFQ (class based weighted fair queuing) gives each class its own queue and a guaranteed share of the link when it is busy. A class may use more than its share while others are quiet, so no bandwidth is wasted.
LLQ (low latency queuing) adds a strict priority queue that the scheduler always serves first, so voice waits for at most the packet already being sent. Under congestion it is policed to its configured rate, so it can never starve the other classes.
Inside the default class, fair queuing gives each flow its own turn, so one large download cannot crowd out everybody else's web pages. WRED (weighted random early detection) can then drop early within the class, as the next section shows.
Switches queue in hardware. A Catalyst 9000 port has eight egress queues: up to two strict priority levels and six weighted queues, each with three drop thresholds, written 2P6Q3T. The thresholds let one queue drop its lower priority traffic first.

Try it: share a congested link

Set the link speed and how much each class is trying to send, then compare a QoS policy with none. This is a simple model of priority plus class based queuing, as IOS XE does it by default: each class first gets its guarantee, then spare capacity is shared equally between the classes that still want more. The example policy gives voice a 10% priority queue and the other classes guarantees of 25, 35, 25 and 5%, as indicative shares.

Link speed100 Mb/s

Voice8 Mb/s

Video25 Mb/s

Business data40 Mb/s

Default60 Mb/s

Scavenger20 Mb/s

Class	Guarantee	Wants	Gets	Dropped
Voice	priority 10%	8	8.0	0.0%
Video	25%	25	25.0	0.0%
Business data	35%	40	35.7	10.8%
Default	25%	60	25.7	57.2%
Scavenger	5%	20	5.7	71.7%

Rates in Mb/s. Bars show what each class wants (pink) and gets.

Voice is protected. The link is asked for 153 Mb/s but has 100. Voice loses nothing, and the shortfall falls hardest on scavenger, which loses 71.7%.

Serialisation: why slow links hurt voice

Link speed	Time to send one 1,500 byte packet
768 kb/s	15.6 ms
2 Mb/s	6 ms
10 Mb/s	1.2 ms
100 Mb/s	0.12 ms
1 Gb/s	12 µs (microseconds)
10 Gb/s	1.2 µs

Serialisation delay is the packet size in bits divided by the link speed. On links at or below 768 kb/s, Cisco's guidance was LFI (link fragmentation and interleaving), which chops large packets so voice can slip between the pieces. I rarely meet links that slow now, but the maths still matters on narrow backup links.

Cap priority at a third

Cisco's long standing guidance keeps all priority queues together at no more than about 33% of a link. Above that, data starves whenever real time traffic is busy.

Leave a quarter for best effort

Most applications are unmarked and land in the default class. I keep at least 25% of a WAN link for it, so ordinary work does not grind to a halt.

Buffers are a trade off

Deep buffers absorb bursts but add delay. Shallow ones drop sooner. On busy Catalyst 9000 ports, qos queue-softmax-multiplier 1200 lets a queue borrow more of the shared buffer to ride out microbursts.

WRED and ECN

Dropping early, so TCP slows down smoothly

When a queue fills completely, every arriving packet is dropped and many TCP (Transmission Control Protocol) flows suffer at once. Congestion avoidance drops or marks a few packets early instead, so senders slow down one at a time and the queue never gets long.

Swipe sideways to see the whole diagram.

Step 1

When a queue is full, every new packet is dropped. Many TCP flows lose packets at the same moment, all halve their rate together, then all speed up together. Link use saws up and down, which is called global synchronisation, and the queue stays full, which adds delay.
RED (random early detection) starts dropping a few packets at random once the average queue passes a minimum threshold. The chance rises to a maximum at the maximum threshold, one packet in ten by default on Cisco. Flows slow down at different times, so the link stays busy and the queue stays short.
WRED (weighted RED) gives each marking its own thresholds. AF13 starts dropping first, then AF12, then AF11. A policer can re mark traffic above a contract to a higher drop precedence, so it goes before traffic that kept to its contract. I never use WRED on voice, which sits in the priority queue.
With ECN (explicit congestion notification), a router sets the packet's two ECN bits to CE (congestion experienced) instead of dropping it, when both ends have said they support ECN. The receiver tells the sender, which slows down without losing anything. Data centre fabrics for RDMA (remote direct memory access) rely on it.
Newer active queue management (AQM) watches how long packets sit in the queue rather than how full it is. CoDel (controlled delay) aims for 5 ms, PIE (proportional integral controller enhanced) for 15 ms by default (10 ms in cable modems), and L4S (low latency, low loss, scalable throughput) uses ECN to keep queues almost empty.

Method	What it watches	Where I use it	Standard
Tail drop	Whether the queue is full	The priority queue, which is kept short by its policer	None
WRED	Average queue depth, separately for each marking	WAN data classes and the default class	Cisco's weighted version of RED
ECN	Marks packets instead of dropping them	Data centre fabrics for RoCEv2, and TCP stacks that support it	RFC 3168
CoDel and FQ-CoDel (flow queue CoDel)	How long each packet waited, with fairness between flows	Linux servers, firewalls and internet edges	RFC 8289, RFC 8290
PIE	Estimated queuing delay	Cable broadband (DOCSIS) modems	RFC 8033, RFC 8034
L4S	A separate low delay queue, signalled with ECN	Emerging in broadband access networks	RFC 9330 to RFC 9332

RFC 7567 sets out the Internet Engineering Task Force's current advice on queue management. RoCEv2 is RDMA over Converged Ethernet, version 2. DOCSIS is the Data Over Cable Service Interface Specification.

The two ECN bits

ECN bits	Name	Meaning
00	Not ECT	The sender does not support ECN, so congestion means a drop
10	ECT(0)	ECN capable transport: mark me instead of dropping me
01	ECT(1)	ECN capable, used by L4S to ask for the low delay queue
11	CE	Congestion experienced: a router marked this packet

WRED by drop precedence, with ECN, on Cisco IOS XE.

policy-map WAN-QUEUES
 class BUSINESS
  bandwidth percent 34
  ! drop AF23 before AF22 before AF21
  random-detect dscp-based
  ! mark ECN capable packets instead of dropping them
  random-detect ecn
!
show policy-map interface GigabitEthernet0/0/0 output class BUSINESS

Not for real time traffic

Voice and video do not slow down when a packet is lost, they just sound or look worse. Dropping them early helps nobody, so WRED stays off the priority classes.

TCP is what it helps

WRED and ECN work because TCP reacts to loss or marks by slowing down. UDP (User Datagram Protocol) traffic without its own congestion control ignores them.

Lossless is a special case

Storage and RDMA traffic in a data centre cannot tolerate drops at all. There I use PFC (priority flow control) and ECN together, as the AI infrastructure page shows.

Policing and shaping

Two ways to enforce a rate, with very different results

Policing and shaping both hold traffic to a rate, and both use a token bucket to measure it. A policer drops or re marks traffic above the rate at once. A shaper holds the excess in a queue and sends it later. Policing protects a network from too much traffic. Shaping makes my traffic fit under someone else's policer.

Swipe sideways to see the whole diagram.

Step 1

Tokens drop into a bucket at the CIR (committed information rate). The bucket holds at most Bc (committed burst) worth of tokens, and the interval Tc is Bc divided by CIR. A full bucket lets a burst of up to Bc through at line rate.
A packet may pass if the bucket holds enough tokens for its size, and it takes them out as it goes. Traffic that keeps to the average rate, with bursts no bigger than Bc, always finds tokens and is said to conform.
When the bucket is empty, a policer drops the packet at once. Nothing waits, so no delay is added, but TCP sees the loss and halves its rate. A policer with too small a bucket can deliver far less than its rate to bursty TCP traffic.
Instead of dropping, a policer can re mark excess traffic, for example from AF11 to AF12, and send it. If there is congestion further on, WRED drops the re marked packets first, and if there is none, they still arrive.
A two rate, three colour policer (RFC 2698) has a second bucket filled at the PIR (peak information rate). Up to CIR is green and sent, up to PIR is yellow and re marked, and above PIR is red and dropped. RFC 2697 does the same with one rate and an excess burst.
A shaper uses the same bucket, but a packet that finds no tokens waits in a queue until tokens arrive. The output is smooth at CIR. It adds delay instead of loss, and it only works on traffic leaving an interface.

	Policing	Shaping
Excess traffic	Dropped or re marked at once	Held in a queue and sent later
Direction	Inbound or outbound	Outbound only
Delay	None added	Added while the queue drains
Effect on TCP	Loss, so TCP backs off hard and throughput saws up and down	Smooth: TCP sees a little more delay instead of loss
Where I use it	Guest and scavenger caps, each phone port, the priority queue's own policer, and the carrier's edge (theirs, not mine)	My WAN and internet edges, to fit under a carrier's policer, and at hubs for each branch

Try it: police or shape the same traffic

Choose a traffic pattern, a rate and a bucket size. The chart shows 600 ms of traffic: what was offered, what a policer let through and what a shaper sent. The shaper has a 1 MB queue.

Rate50 Mb/s

Bucket, Bc32 KB

Offered
Policed
Shaped

Offered, average: 45.0 Mb/s
Policed: delivered: 23.9 Mb/s, 47% dropped
Shaped: delivered: 45.0 Mb/s, 0% dropped
Shaper: most delay added: 17 ms

The policer dropped 47% even though the average is under the rate, because each burst is bigger than the bucket. The shaper lost nothing and added up to 17 ms of delay instead.

The token bucket terms

Term	Meaning	Example
CIR	Committed information rate: the long term rate allowed	100 Mb/s
Bc	Committed burst: how many tokens the bucket holds	1,000,000 bits, which is 125 KB
Tc	The interval: Bc divided by CIR	10 ms
Be	Excess burst: extra tokens saved while the line was idle	0 on shapers that carry voice
PIR	Peak information rate, for the second bucket of a two rate policer	150 Mb/s

Cisco's design guidance for shapers carrying voice sets Bc to CIR divided by 100, a 10 ms interval, so the shaper never holds voice back for long. Shaper Bc is usually set in bits and policer Bc in bytes, which is a common source of mistakes.

A policer and a shaper on Cisco IOS XE. Rates are in bits per second.

! Two rate, three colour policer: re mark above CIR, drop above PIR
policy-map POLICE-BULK
 class BULK
  police cir 50000000 pir 80000000
   conform-action transmit
   exceed-action set-dscp-transmit af12
   violate-action drop
!
! Shaper to 100 Mb/s with a 10 ms interval (Bc in bits)
policy-map SHAPE-100M
 class class-default
  shape average 100000000 1000000
!
show policy-map interface GigabitEthernet0/0/0

Hierarchical QoS

Shape to the contract, queue inside the shape

Most WAN and internet circuits arrive on a faster port than the contract, such as a 1 Gb/s port carrying a 200 Mb/s service. The router never sees congestion at 1 Gb/s, so its queues never fill and the carrier's policer decides what is lost. Hierarchical QoS (HQoS) fixes this with a parent shaper at the contracted rate and the class policy as its child.

Swipe sideways to see the whole diagram.

Step 1

The router's port runs at 1 Gb/s but the contract is 200 Mb/s. The router sees no congestion, so its queues stay empty and every packet goes straight out. The carrier polices at 200 Mb/s and drops whatever is over the rate, voice included.
A parent policy shapes everything leaving the port to 200 Mb/s. Now congestion happens inside my router, where I can manage it, instead of at the carrier's policer, where I cannot.
A child policy, nested under the shaper, holds the class queues: voice in the priority queue, then the guaranteed classes. This is hierarchical QoS. The child's percentages are shares of the shaped 200 Mb/s, not of the 1 Gb/s port.
A hub with a 1 Gb/s circuit can send far faster than a 20 Mb/s branch can receive. The queue builds inside the carrier network, in front of the branch, where my policy cannot reach. Calls to that branch suffer whenever someone sends it a large file.
The hub shapes each branch's traffic to that branch's rate, with the class queues inside each shaper. On DMVPN (Dynamic Multipoint Virtual Private Network) this is per tunnel QoS, chosen by an NHRP (Next Hop Resolution Protocol) group that each spoke announces. SD-WAN hubs can shape per tunnel in the same way.
Inside GRE (generic routing encapsulation) or IPsec, the original header is hidden. Cisco routers copy the ToS byte to the outer header, so DSCP based queuing still works across the carrier. To classify on inner addresses or ports instead, I use qos pre-classify or put the policy on the tunnel.

Overheads that change the rate

What is added	Bytes	Notes
Ethernet header and FCS (frame check sequence)	18	Counted by most shapers and policers
802.1Q VLAN tag	4	On tagged handoffs, which most carrier services are
Preamble and gap between frames	20	Uses the wire, but most shapers and policers do not count it
MPLS label	4 per label	Inside the carrier network
GRE	24	New IP header plus the GRE header
IPsec ESP (encapsulating security payload), tunnel mode	About 54 to 57	With AES-GCM (the Advanced Encryption Standard in Galois Counter Mode): new IP header, ESP header, IV (initialisation vector), padding and the check value
NAT (network address translation) traversal	8	A UDP header around ESP, when a NAT sits in the path

Approximate. A router's shaper and a carrier's policer may not count the same bytes, so I ask the carrier how its policer counts and shape a few per cent under the contract if in doubt.

Per tunnel QoS on a DMVPN hub, as Cisco documents it. WAN-QUEUES is the class policy in the configs section below.

! Hub: one shaper per branch size, with the class queues inside it
policy-map SHAPE-20M
 class class-default
  shape average 20000000
  service-policy WAN-QUEUES
interface Tunnel0
 ip nhrp map group BRANCH-20M service-policy output SHAPE-20M
!
! Spoke: tell the hub which group it belongs to
interface Tunnel0
 ip nhrp group BRANCH-20M
!
show dmvpn detail
show policy-map multipoint tunnel 0

Shaping only works outbound

I cannot shape what arrives at my router, because it has already crossed the link. Downloads are shaped at the sending end, which is why hubs shape per branch.

Internet circuits too

An internet circuit at 500 Mb/s on a 1 Gb/s port behaves the same way. I shape outbound to the contract and let SD-WAN steer the traffic that needs a better path.

One parent per contract

Each circuit, tunnel or sub interface with its own contracted rate gets its own parent shaper. The class policy inside can be the same everywhere.

Voice and video

What a call really costs, header by header

Voice is small, steady and impatient. A call sends a packet every 20 ms in each direction, and the headers can be bigger than the speech they carry. I size the priority queue from the real rate per call, including every header on the link, then let call admission control stop the call that would not fit.

Swipe sideways to see the whole diagram.

Step 1

A G.711 codec turns speech into 64 kb/s. Every 20 ms it fills a packet with 160 bytes of speech, so each call sends 50 packets a second in each direction.
RTP (Real-time Transport Protocol) adds a sequence number and a timestamp in 12 bytes, UDP (User Datagram Protocol) adds 8 and IP adds 20. The packet is now 200 bytes, and a fifth of it is header.
Ethernet adds 18 bytes of header and frame check sequence, making 218 bytes. At 50 packets a second that is 87.2 kb/s per call in each direction. This is the number I size the priority queue with.
G.729 squeezes the same 20 ms of speech into 20 bytes. The headers stay the same size, so each packet is 78 bytes on Ethernet and the call needs 31.2 kb/s, not 8 kb/s. Headers are now three quarters of every packet.
Over IPsec with AES-GCM, every packet grows by about 56 bytes more. A G.711 call becomes about 110 kb/s and a G.729 call about 54 kb/s. That is why I size voice classes on the tunnel, not on the codec's headline rate.

Try it: size the voice priority queue

Choose a codec, the packet interval, the link type and any tunnel, then the number of calls at the busiest time. Tunnel sizes are estimates for AES-GCM; other ciphers add a few bytes more.

Codec

Packet interval

Link type

Tunnel

Busy hour calls20

Link speed100 Mb/s

Speech in each packet: 160 bytes
IP packet: 200 bytes
On the wire: 218 bytes, 50 packets a second
Each call, each way: 87.2 kb/s
All calls: 1744 kb/s, 1.7% of the link
Priority to configure: priority 1750 (kb/s)

Fits comfortably. 20 calls need 1.7% of a 100 Mb/s link, well inside the 10% voice class in my model.

In my home lab, the Dubai site has no call servers of its own: its phones register across the MPLS core to the call managers in London and New York. That makes it a good place to see what WAN congestion does to call signalling as well as to speech.

Collaboration traffic, as Microsoft sizes it

Teams traffic, per user	Minimum	Recommended	Best
Audio	10 kb/s	58 kb/s	76 kb/s
Video, one to one	150 kb/s	1.5 Mb/s	4 Mb/s
Video, meetings (up / down)	150 / 200 kb/s	2.5 / 4 Mb/s	4 / 4 Mb/s
Screen sharing, meetings	250 kb/s	2.5 Mb/s	4 Mb/s

Microsoft's published figures for Teams, in each direction unless shown. I use the recommended column for sizing and check it against call records once the service is live.

A voice quality probe on Cisco IOS XE: fifty G.711 sized packets a second, marked EF, measured end to end.

ip sla 10
 udp-jitter 10.2.1.10 16384 codec g711alaw
 ! ToS 184 is EF, so the probe uses the voice queue
 tos 184
 frequency 60
ip sla schedule 10 life forever start-time now
! on the far router
ip sla responder
!
! jitter, loss, delay and an estimated MOS
show ip sla statistics 10

Admission control

The priority queue is policed, so one call too many damages every call on the link. Call admission control (CAC) counts calls per site and rejects or reroutes the call that would not fit. In CUCM (Cisco Unified Communications Manager) this is Enhanced Locations CAC.

Jitter buffers

Phones hold arriving packets in a jitter buffer, typically a few tens of milliseconds, to smooth out uneven arrival. A packet later than the buffer allows is thrown away, so heavy jitter sounds exactly like loss.

Video is bursty

Each video frame leaves as a burst of large packets. I size video classes for the peak, and keep video on a lower priority level than voice, so a burst never delays speech.

Measure what people hear

IP SLA (service level agreement) probes and call records give a MOS (mean opinion score), an estimate of call quality from 1 to 5. Teams adds its Call Quality Dashboard, and Webex its Control Hub analytics.

End to end

One class model, many places to break it

A call crosses Wi-Fi, the campus, the WAN or SD-WAN and someone else's network. Each domain carries the marking in a different field and has its own queues. The class model stays the same everywhere. Only the mapping changes, and every border is a place where a marking can be lost.

Swipe sideways to see the whole diagram.

Wi-Fi. WMM has four access categories, each with its own waiting rules for the channel. Downstream, the access point maps DSCP to UP using RFC 8325: EF to 6, AF41 to 4, CS1 to 1. Upstream, clients pick their own UP and some pick badly (Windows sends voice as UP 5), so a Catalyst 9800 trusts the DSCP inside the frame, which has been its default since release 17.4. The CAPWAP (Control and Provisioning of Wireless Access Points) tunnel copies the marking to its outer header, so switch ports facing access points must trust DSCP. The 9800's platinum, gold, silver and bronze profiles cap the marking on each SSID (service set identifier): platinum at 46, gold at 34, silver at 0 and bronze at 8. Guest SSIDs sit on bronze or silver. Radio design for voice is on the wireless page.

Campus. On Catalyst 9000 switches DSCP is trusted unless a policy says otherwise, so the job is to decide where not to trust it: access ports for PCs and guests re mark, and conditional trust handles phones. Each port has eight egress queues, up to two priority levels and three drop thresholds per queue. AutoQoS builds a sound baseline that I then read and tune. On older Catalyst 3750 switches, turning on mls qos made every port untrusted and reset markings to 0 until trust was set, a classic outage after a change.

WAN edge. This is where congestion usually lives, so it carries the full policy. That means a parent shaper at the contracted rate, my class queues inside it, WRED on the data classes and a protected class for routing. It also translates my class model into the carrier's, and re marks anything arriving from outside back into my model.

Carrier and SD-WAN. On MPLS, the PE copies IP precedence into the label's traffic class by default. RFC 3270 describes three ways a carrier can treat my DSCP: uniform mode lets its re marking reach my packet, while pipe and short pipe leave my DSCP alone. I ask which one I am buying. On Catalyst SD-WAN, a localised policy maps forwarding classes to queues 0 to 7, queue 0 is the low latency queue, and the transport interface carries a shaping rate. DSCP is copied to the IPsec outer header, so the underlay can queue on it. Path choice is on the SD-WAN page.

Data centre. Fast data centre links queue less often for ordinary applications, but storage and RDMA traffic cannot tolerate a single drop. DCB (data centre bridging) adds three tools. PFC (priority flow control, 802.1Qbb) pauses one priority, not the whole link. ETS (enhanced transmission selection, 802.1Qaz) shares bandwidth between priorities. DCBX (DCB exchange) agrees both over LLDP (Link Layer Discovery Protocol). ECN tells RoCEv2 senders to slow down before PFC has to pause. On Nexus, QoS comes in three policy types: qos for classifying, queuing for scheduling and network-qos for system wide settings such as lossless classes. The AI infrastructure page covers lossless fabrics in depth.

Internet and cloud. Once traffic leaves my network over the internet, nobody has agreed to honour my markings, and some networks reset them. I protect what I control: queue and shape at my internet edge, send collaboration traffic straight out locally rather than back through a distant hub, and use SD-WAN to pick the best path. Where a service level matters, private connections such as ExpressRoute or Direct Connect give a defined path, but I still check what each provider does with DSCP before relying on it.

Animation

One class, many fields

Class	DSCP	CoS	MPLS TC (default)	Wi-Fi UP and access category (RFC 8325)
Voice	EF 46	5	5	6 · voice
Interactive video	AF41 34	4	4	4 · video
Streaming video	AF31 26	3	3	4 · video
Network control	CS6 48	6	6	7 · voice, or 0 where only clients sit behind the access point
Call signalling	CS3 24	3	3	4 · video
Transactional data	AF21 18	2	2	3 · best effort
Best effort	DF 0	0	0	0 · best effort
Scavenger	CS1 8	1	1	1 · background

CoS and TC values follow the top three bits of the DSCP, which is the usual default. Clients should never send CS6 upstream, and access points should not pass it on.

Same model, different fields

The class table is written once. Each domain only needs a mapping table, and the mapping is checked at every border.

I ask before I buy

For any carrier or cloud service I ask three things: how many classes, what happens to traffic above each class's rate, and whether my DSCP arrives unchanged.

Steering is not queuing

SD-WAN application aware routing picks a path that meets the target. QoS decides what waits on that path. A busy link needs both.

Configs

The same policy, written for each platform

Every platform has its own syntax and its own defaults, but the building blocks are the same: classify, mark, queue and shape. These are starting points in the shape of my standards, with the commands I use to check each one. Names, addresses and rates are examples.

Cisco IOS XE WAN router

MQC (modular QoS command line) has three parts: class maps to classify, a policy map to act, and a service policy to attach it to an interface and direction.
Queuing policies only run on egress, and only matter when the interface is congested. The shaper in the parent policy is what creates that congestion at the contracted rate.
priority percent is policed during congestion. The bandwidth and priority percentages are shares of the shaped rate.
Routers mark their own routing and HSRP packets CS6 and protect them internally, but on a congested WAN I still give CS6 its own class.

class-map match-any VOICE
 match dscp ef
class-map match-any VIDEO
 match dscp af41 af42 af43
class-map match-any SIGNALLING
 match dscp cs3 cs5
class-map match-any ROUTING
 match dscp cs6
class-map match-any BUSINESS
 match dscp af21 af22 af23 af31 af32 af33
class-map match-any SCAVENGER
 match dscp cs1
!
policy-map WAN-QUEUES
 class VOICE
  priority percent 10
 class VIDEO
  bandwidth percent 23
 class SIGNALLING
  bandwidth percent 2
 class ROUTING
  bandwidth percent 5
 class BUSINESS
  bandwidth percent 34
  random-detect dscp-based
 class SCAVENGER
  bandwidth percent 1
 class class-default
  bandwidth percent 25
  fair-queue
!
policy-map WAN-SHAPE
 class class-default
  shape average 200000000
  service-policy WAN-QUEUES
!
interface GigabitEthernet0/0/0
 service-policy output WAN-SHAPE

Check	Command
Class counters, drops and queue depth	`show policy-map interface GigabitEthernet0/0/0 output`
One class only	`show policy-map interface GigabitEthernet0/0/0 output class VOICE`
Which applications cross the edge	`show ip nbar protocol-discovery interface GigabitEthernet0/0/0 top-n 10`

Cisco Catalyst 9000 access switch

Catalyst 9000 trusts DSCP by default. Policies decide where to re mark, not where to trust.
AutoQoS builds a tested baseline from one interface command. I read what it generated, keep it consistent across the estate and tune from there.
Each port has eight egress queues: up to two priority levels and six weighted queues, with three thresholds in each.
Output drops with a low average rate usually mean microbursts. The per queue counters show which queue is dropping.

interface GigabitEthernet1/0/10
 description Phone with a PC behind it
 switchport access vlan 12
 switchport voice vlan 13
 auto qos voip cisco-phone
!
! AutoQoS adds conditional trust (trust device cisco-phone),
! an input marking policy and an output policy with one
! priority queue and seven other queues
!
! Busy uplinks: let queues borrow more of the shared buffer
qos queue-softmax-multiplier 1200

Check	Command
What AutoQoS built	`show auto qos interface GigabitEthernet1/0/10`
Policy counters	`show policy-map interface GigabitEthernet1/0/10`
Hardware queue drops	`show platform hardware fed switch active qos queue stats interface GigabitEthernet1/0/48`
Is the phone seen	`show cdp neighbors GigabitEthernet1/0/10 detail`

Cisco Nexus 9000 data centre switch

NX-OS splits QoS into three policy types: qos to classify and mark on ingress, queuing to schedule on egress, and network-qos for system wide settings such as MTU (maximum transmission unit) and lossless classes.
Classification sets a qos-group, and the qos-group picks the egress queue.
Every port always has a system queuing policy. I copy the default, change it and apply it under system qos.

class-map type qos match-any VOICE
  match dscp 46
policy-map type qos CLASSIFY-IN
  class VOICE
    set qos-group 5
interface Ethernet1/1
  service-policy type qos input CLASSIFY-IN
!
! qos-group 5 is served by egress queue 5 of the system
! queuing policy (type queuing). Lossless classes and MTU
! are set system wide in the network-qos policy.

Check	Command
Classification counters	`show policy-map interface ethernet 1/1 type qos`
Queue depth and drops	`show queuing interface ethernet 1/1`
System lossless settings	`show policy-map system type network-qos`

Juniper Junos

Junos calls it class of service. Classifiers put traffic into forwarding classes, schedulers set each queue's rate and priority, and a scheduler map ties them to an interface.
By default there are four forwarding classes on queues 0 to 3: best effort, expedited forwarding, assured forwarding and network control. Only best effort (95%) and network control (5%) have schedulers.
The default DSCP classifier only sends AF1x to assured forwarding, so this example adds AF41, AF31 and AF21, and gives every class a scheduler.
Syntax varies between SRX, MX and EX platforms, so I check the platform guide before applying a shaping rate.

class-of-service {
    classifiers {
        dscp WAN-IN {
            import default;
            forwarding-class assured-forwarding {
                loss-priority low code-points [ af41 af31 af21 ];
            }
        }
    }
    schedulers {
        VOICE-S { transmit-rate percent 10; priority strict-high; }
        AF-S    { transmit-rate percent 50; }
        BE-S    { transmit-rate remainder; }
        NC-S    { transmit-rate percent 5; }
    }
    scheduler-maps {
        WAN-MAP {
            forwarding-class expedited-forwarding scheduler VOICE-S;
            forwarding-class assured-forwarding scheduler AF-S;
            forwarding-class best-effort scheduler BE-S;
            forwarding-class network-control scheduler NC-S;
        }
    }
    interfaces {
        ge-0/0/0 {
            scheduler-map WAN-MAP;
            shaping-rate 200m;
            unit 0 { classifiers { dscp WAN-IN; } }
        }
    }
}

Check	Command
Class of service on an interface	`show class-of-service interface ge-0/0/0`
Per queue counters and drops	`show interfaces queue ge-0/0/0`
Forwarding classes and queues	`show class-of-service forwarding-class`

Fortinet FortiGate

FortiOS has three kinds of shaper: shared shapers for a policy, per IP shapers for each source address, and application control shapers.
A traffic shaping policy matches traffic and picks the shaper. Shapers set guaranteed and maximum bandwidth in kb/s, a priority, and can set DSCP.
Application control shapers override shared shapers, which override per IP shapers.

config firewall shaper traffic-shaper
    edit "VOICE-PRIORITY"
        set guaranteed-bandwidth 10000
        set maximum-bandwidth 20000
        set priority high
        set diffserv enable
        set diffservcode 101110
    next
end
config firewall shaping-policy
    edit 1
        set name "Voice media"
        set service "ALL_UDP"
        set dstintf "wan1"
        set srcaddr "VOICE-VLAN"
        set dstaddr "all"
        set traffic-shaper "VOICE-PRIORITY"
        set traffic-shaper-reverse "VOICE-PRIORITY"
    next
end

Check	Command
Shapers and their settings	`diagnose firewall shaper traffic-shaper list`
Shaper statistics	`diagnose firewall shaper traffic-shaper stats`

Windows endpoints and Teams

Windows marks traffic through policy based QoS, set by Group Policy or PowerShell, matched by application, protocol and port.
Teams uses fixed source port ranges per media type, so one rule per type marks audio, video, sharing and signalling.
The network still decides whether to trust these markings. I trust them only from managed laptops, on ports and VLANs I know.

# Mark Teams audio EF on managed laptops (see the trust section for all four rules)
New-NetQosPolicy -Name "Teams Audio" -AppPathNameMatchCondition "ms-teams.exe" `
  -IPProtocolMatchCondition Both -IPSrcPortStartMatchCondition 50000 `
  -IPSrcPortEndMatchCondition 50019 -DSCPAction 46 -NetworkProfile All

# Check what the laptop will do
Get-NetQosPolicy
gpupdate /force
# Group Policy rules land in HKLM\Software\Policies\Microsoft\Windows\QoS

Check	Command
Policies on the laptop	`Get-NetQosPolicy`
Markings on the wire	`Wireshark display filter ip.dsfield.dscp == 46`

Linux servers and edges

Linux queues with qdiscs (queuing disciplines). FQ-CoDel is the default on many distributions and keeps queues short with no tuning.
CAKE combines a shaper, fairness between flows and hosts, DiffServ tins and CoDel style queue management in one qdisc.
Shaping just below the real link rate moves the queue from the modem, where I cannot manage it, into Linux, where I can.

# Shape a 100 Mb/s uplink just under its rate with CAKE:
# fair between flows, four DiffServ tins, CoDel style drops
tc qdisc replace dev eth0 root cake bandwidth 95Mbit diffserv4
tc -s qdisc show dev eth0

# Or FQ-CoDel where no shaper is needed
tc qdisc replace dev eth0 root fq_codel

# Send marked test traffic: 2 Mb/s of UDP marked EF
iperf3 -c 10.2.1.10 -u -b 2M --dscp EF

Check	Command
Qdisc counters, drops and delay	`tc -s qdisc show dev eth0`
Capture EF packets	`tcpdump -ni eth0 'ip[1] & 0xfc == 0xb8'`

Design rules

The rules I design by, and what each one prevents

QoS goes wrong in predictable ways: too many classes, priority with no cap, markings trusted from anywhere, and policies that are never measured. These are the rules I apply to every design, and the trade offs behind them.

Start from the applications. List what must keep working when a link is full, with numbers, before choosing a single tool.
Measure before designing. Find out where queues and drops happen today. QoS on a link that never congests is effort with no benefit.
Mark once, near the source. Phones, access switches and managed endpoints mark. Everything after them trusts the marking.
Keep the class model small. Four to eight classes for most estates, grown by splitting a class rather than starting again.
Cap priority, protect best effort. All priority queues at no more than about a third of a link, and at least a quarter left for the default class.
Protect the network's own traffic. Routing, first hop redundancy and BFD packets get a class that never drops, and CoPP (control plane policing) protects each device's processor.
Shape where someone else polices. A parent shaper at every contracted rate, and per branch shaping at hubs.
Size voice from calls, not codecs. Busy hour calls times the real rate per call, with every header, and call admission control for the call that does not fit.
Same policy everywhere, as code. One template per platform, generated from the source of truth, so no site drifts. The automation page shows how.
Prove it under load. Fill the link in a lab or pilot and check the voice class stays clean, before any user finds out the hard way.

Do I need QoS here?

Situation	Do I need QoS here?	What I do
Link busy for minutes every day, with voice or video on it	Yes	Full class policy, and shaping if the port is faster than the contract
Fast port, slower contracted rate	Yes	Parent shaper at the contract, class policy inside
Campus uplinks rarely above 30%, with output drops	Yes, light	Trust and queue on the uplinks, more buffer for microbursts
Wi-Fi carrying calls	Yes	WMM, RFC 8325 mapping, trusted DSCP upstream, metal QoS per SSID
Data centre with storage or RDMA traffic	Yes, lossless	PFC, ETS and ECN on the classes that need them
Link always full	Not enough on its own	Protect the critical classes now, and order more capacity
Fast link, never congested, no real time traffic	No	Trust and keep the markings, so the policy is ready when it is needed

QoS and security

A marking is a way in

A device that can mark EF can steal priority from every call, or fill the voice queue so its policer drops real calls. Re marking at the trust boundary and policing the priority class per port are security controls.

Scavenger for the unwanted

Traffic that should not be there, or is far above its normal rate, goes to the scavenger class with a tiny share. A worm or a runaway backup then hurts itself first.

Protect the control plane

Even a perfect data plane policy does not protect a router's processor. CoPP polices what reaches it, as the security page describes.

Markings can say too much

Copying the DSCP to a tunnel's outer header lets the carrier see which packets are voice. Most organisations accept that for better calls. Some security policies reset the outer marking instead.

Troubleshooting

Finding a QoS fault hop by hop

QoS faults look like application faults: choppy calls, frozen video and slow transfers at busy times. My method is always the same: prove there is congestion, follow the marking hop by hop, then read the counters of the queue the traffic really used. Choose a symptom to see where it fails and what I run. My general method is on the troubleshooting page.

Swipe sideways to see the whole diagram.

Calls break up at busy times. Voice is not classified at the WAN edge, often because a new phone system, a softphone or Teams marks differently from the class map, so calls land in class-default with everything else. I capture a call to see its real marking, fix the class map or the marking at the access edge, and check the VOICE counters climb during a call.

Every call suffers once a few are up. There are more calls than the priority queue was sized for. Its policer drops voice packets from every call, not just the newest one. I resize the priority class from busy hour calls, using the calculator above, and set call admission control so the next call is refused cleanly instead.

Fine on the LAN, poor across the WAN. Something on the path reset the marking: a carrier in uniform mode, a provider re marking traffic above its real time contract, or a firewall or tunnel that does not copy the DSCP. I compare captures on both sides of each border, agree the mapping with the carrier, and re mark at my far edge as a safety net.

Priority class busy, but few calls. An access port trusts everything, so a PC or an application marked its own traffic EF. I use conditional trust for phones, re mark the data VLAN, and police the priority class on every access port.

Output drops at low average use. Microbursts: many fast sources sending into one port at once, too briefly to show in five minute averages. I give the queue more of the shared buffer, with qos queue-softmax-multiplier 1200 on Catalyst 9000, spread the load over more uplinks, or move the bursty sources.

Transfers stuck well below the rate. A policer whose burst size is much smaller than TCP's bursts, or a rate set in the wrong units: bits against bytes, or kb/s against b/s. Where I control the sender I shape instead, size any policer's burst to the traffic, and read the configured rate back from the counters.

Routing drops when the link is full. Routing hellos are being dropped in a congested queue, or by CoPP on a busy router. I give CS6 a guaranteed class at every congestion point, set BFD timers that suit the link, and tune CoPP from a baseline.

Wired calls fine, Wi-Fi calls poor. The access point is not trusting the client's DSCP, the switch port to the access point re marks the CAPWAP outer header, or the SSID's metal QoS caps the marking. I trust DSCP upstream on the controller, trust the access point ports, use platinum for voice SSIDs, and check UP in an over the air capture. Radio problems look the same, so I check the wireless faults too.

Animation

The tools I reach for

Question	Command or tool	What it tells me
Is the link really congested?	`show interfaces GigabitEthernet0/0/0 \| include rate\|drops`, and flow records	Load and output drops, and who the top talkers are
Which class did the traffic hit?	`show policy-map interface GigabitEthernet0/0/0 output`	Matches, drops and queue depth per class, and the shaper's state
Is a switch queue dropping?	`show platform hardware fed switch active qos queue stats interface Gi1/0/48`	Bytes queued and dropped for each hardware queue and threshold
Same on Nexus and Junos	`show queuing interface ethernet 1/1`, `show interfaces queue ge-0/0/0`	Per queue counters on each platform
Did the marking survive?	Wireshark `ip.dsfield.dscp == 46` on both sides of a border	Exactly where a DSCP was reset
Can I reproduce it?	`iperf3 -c 10.2.1.10 -u -b 2M --dscp EF`, plus bulk traffic to fill the link	Whether marked traffic survives a full link
What do users hear?	IP SLA `udp-jitter` probes, Teams Call Quality Dashboard	Jitter, loss, delay and MOS over time, by site and network

I change one thing at a time and re run the same test after each change. If the voice class shows no drops and calls are still poor, the cause is usually outside QoS: Wi-Fi, a duplex mismatch, an overloaded server or the far end.

In trading networks microbursts last microseconds. How I measure them on the wire is on my low latency trading page.

Engagement

How I run a QoS programme, from first complaint to steady state

QoS work usually starts with a complaint about calls, or with a change that touches every link, such as a new phone system or a WAN migration. I follow the same path each time: agree what must keep working, measure before changing anything, design one model for every domain, prove it under load, then keep it current as applications change.

Swipe sideways to see the whole diagram.

Step 1

I start by agreeing what must keep working when a link is full, and for whom: calls, a contact centre, trading screens or a plant system. I record who decides and who is consulted in a RACI (responsible, accountable, consulted, informed) chart, with the sites and links in scope.
I turn the outcome into requirements for each class: what it must do, and how well, with delay, jitter and loss targets and busy hour volumes. Security sets the trust rules: which devices may mark, and where markings are reset. Each number has an owner who signs it off.
Before designing anything I measure: flow records with the DSCP field, class counters and queue drops, and captures at every border to see which markings survive. I ask each carrier for its class model and what it does with traffic above contract. The findings often change the design.
The high level design sets the class model, the trust boundary and the mapping into Wi-Fi, campus, WAN, SD-WAN and data centre. The low level design turns it into templates for each platform, a shaping rate for every circuit and a test plan with pass marks agreed in advance.
I fill links on purpose in a lab, then at pilot sites, while real and test calls run, and compare the voice class against its pass marks. Only then does the policy roll out in waves through change control, each with a tested rollback and before and after counters.
Before handover, class counters, priority drops and policer hits are in monitoring with thresholds, beside call quality from the call manager or Teams. Support teams get runbooks for the common faults, and hypercare ends only when the agreed criteria have been met for a set period.
Applications, codecs and circuits keep changing, so the policy does too. I review class drops and call quality each quarter, add marking rules for new applications through the design authority, update templates with each software release, and turn sustained congestion into capacity requests.

Functional requirements

What it must do.

Give voice priority on every link where it can meet congestion, from the access port to the far site.
Keep interactive video, call signalling and network control traffic within their targets at full load.
Classify and mark traffic at the edge, and re mark anything from an untrusted source.
Shape every circuit delivered on a faster port to its contracted rate, and shape per branch at hubs.
Map the class model into Wi-Fi, carrier, SD-WAN and data centre domains, and keep markings intact across each border.
Report counters for each class and call quality, per site, to the operations teams.

Non functional requirements

How well it must do it, with a number wherever one can be agreed.

Voice quality: One way delay 150 ms or less, jitter under 30 ms and loss 1% or less, end to end (typical).
Interactive video: The same delay and jitter targets as voice, with loss under 1%, sized for peak bursts (typical).
Priority capacity: All priority queues together at no more than about a third of each link (typical).
Best effort: At least 25% of each WAN link left for the default class (typical).
Control traffic: No routing or first hop redundancy hellos lost with links at 100% load.
Call capacity: Busy hour calls per site plus headroom, enforced by call admission control.
Consistency: One class model and mapping table across every domain, generated as code.
Operability: Counters for each class on every congested interface in monitoring, with alert thresholds.

Numbers are typical starting points, drawn from ITU-T G.114 and Cisco design guidance. Each organisation agrees its own.

Who I talk to

Voice, contact centre and collaboration teams, who know the codecs and call volumes
Owners of the applications most sensitive to delay
Network operations, who see the drops and the complaints
Endpoint and desktop teams, who control laptop marking policy
Carriers and their solution architects, for class models and contracts

How I find out

Flow records at busy hour, broken down by DSCP and application
Class and queue counters on every link that can congest
Captures on both sides of each border to check markings survive
Test calls and marked test traffic while links are deliberately full
Review of carrier class models, contracted rates and how policers count bytes

Discovery tools

Tool or source	What it tells me
NetFlow or IPFIX (IP Flow Information Export) with the DSCP field	Busy hour traffic per application and marking
NBAR2 protocol discovery	Which applications cross each WAN edge
`show policy-map interface` and queue counters	Where drops happen today, and in which class
Wireshark captures at each border	Whether markings survive each hop
Catalyst Center Assurance or SD-WAN Manager	Application health, loss, delay and jitter per site and tunnel
ThousandEyes or IP SLA probes	Delay, jitter and loss along the real path, including the carrier
Teams Call Quality Dashboard or Webex Control Hub	Call quality by building, subnet and network type

Most QoS faults turn out to be a lost marking or a class that is too small, so I check those early, once I have proved the link is really congested.

My approach

Scope. Which calls or applications, which sites, which direction, and since when? What changed: a phone system, a client, a carrier or a policy?
Congestion. Is any link on the path busy at the time of the fault? With no congestion, QoS is not the cause.
Marking. Capture on both sides of each border to see whether the DSCP survives, and whether the traffic was marked at all.
Queue. Read the class counters where the link is congested: matches, priority drops, policer hits and queue depth.
Path. Check which path the traffic really took: a backup link, an SD-WAN path change or a tunnel can bypass the policy.
Fix and prove. Change one thing through change control, repeat the same test, and record the cause for problem management.

Faults I see most

Voice landing in class-default because its marking changed
A carrier or tunnel resetting DSCP to 0
A priority queue too small for the busy hour
A PC or application marking itself EF
Microburst drops on switch ports with low average use
Policers with tiny bursts throttling TCP

Layer by layer

Layer	What I check
1 · Physical	Errors, speed and duplex, and Wi-Fi retries and channel use, which look like QoS faults but are not.
2 · Data link	CoS on trunks and the voice VLAN, WMM user priority over the air, and output drops per switch queue.
3 · Network	DSCP at each hop with captures, re marking by carriers and tunnels, and `show policy-map interface` on routed links.
4 · Transport	TCP retransmissions and windows on slow transfers, and UDP media ports matching the classifiers.
5 to 7 · Application	Codec, jitter buffer and call records with MOS, jitter and loss for each call.

My general method, with worked faults, is on the troubleshooting page.

Projects

The work behind this page

Voice and WAN work runs through my career, from VoIP (voice over IP) across the JJ Food Wholesale group to SD-WAN at every AVEVA site worldwide. My lab carries real calls across an MPLS core, so I can watch what congestion does to them.

Home lab · ongoing

Voice across an MPLS core

Nine IP phones across three sites, with CUCM, Unity Connection, Presence and Contact Center Express running as virtual machines. The Dubai site has no call servers, so its phones rely on the call managers in London and New York across the MPLS core. Each phone port carries a voice VLAN and a data VLAN. See the topology.

JJ Food Wholesale · 2007 to 2011

VoIP across the group

LAN, WAN, VoIP, virtualisation and security across the group, including the warehouse and cold chain systems at the food depots.

BNY Mellon · 2018 to 2019

A refresh across Europe, the Middle East and Africa

Audit, high and low level design and implementation of LAN, WAN, SD-WAN and data centre infrastructure, replacing end of life equipment with minimal impact on global operations.

AVEVA · 2020 to 2021

LAN and WAN on Catalyst SD-WAN

Designed and built LAN and WAN on Catalyst SD-WAN, SD-Access and Catalyst Center, and upgraded every global site remotely during the lockdowns through local enablement, with no delay.

Vodafone · 2024

Planning for low latency workloads

Guided SD-WAN, SD-Access and Catalyst Center infrastructure planning so that backbone, data centre and edge environments could carry high throughput, low latency workloads.

XIDA · 2025 to now

AI ready data centre infrastructure

Designed AI ready data centre infrastructure for high density compute clusters, covering network, power, cooling and structured cabling. The fabric side is on the AI infrastructure page.

Backed by CCIP (Cisco Certified Internetwork Professional), which covered quality of service, CCVP (Cisco Certified Voice Professional) for voice, and CCNP (Cisco Certified Network Professional) Enterprise. See all credentials

Explore

More areas I design and deliver

Every page follows the same idea: how it works, the choices that matter, and where I have used it. The foundations pages explain the basics the others build on.

Quality of service, from the handset to the far end

Where queues form, and why calls suffer first

What each kind of traffic can tolerate

The QoS toolset, in the order a packet meets it

QoS never adds bandwidth

It only acts in congestion

Delay, jitter and loss

Six bits that carry the priority

The per hop behaviours

The assured forwarding grid

Try it: decode a marking

Mark once, near the source

Layer 2 marks do not travel

CS1 or LE for scavenger

Few classes, applied the same everywhere

My eight class model in detail

Fitting my classes into a carrier's

Fewer classes, fewer faults

Grow by splitting

Write the model down

Mark at the edge, trust only what I marked

Ways to classify, from simplest to smartest

Marking Teams media at the source

Reset what I do not trust

Police priority at the edge

Re mark at every border

Who goes next: queues and the scheduler

Try it: share a congested link

Serialisation: why slow links hurt voice

Cap priority at a third

Leave a quarter for best effort

Buffers are a trade off

Dropping early, so TCP slows down smoothly

The two ECN bits

Not for real time traffic

TCP is what it helps

Lossless is a special case

Two ways to enforce a rate, with very different results

Try it: police or shape the same traffic

The token bucket terms

Shape to the contract, queue inside the shape

Overheads that change the rate

Shaping only works outbound

Internet circuits too

One parent per contract

What a call really costs, header by header

Try it: size the voice priority queue

Collaboration traffic, as Microsoft sizes it

Admission control

Jitter buffers

Video is bursty

Measure what people hear

One class model, many places to break it

One class, many fields

Same model, different fields

I ask before I buy

Steering is not queuing

The same policy, written for each platform

Cisco IOS XE WAN router

Cisco Catalyst 9000 access switch

Cisco Nexus 9000 data centre switch

Juniper Junos

Palo Alto Networks firewalls

Fortinet FortiGate

Windows endpoints and Teams

Linux servers and edges

The rules I design by, and what each one prevents

Do I need QoS here?

QoS and security

A marking is a way in

Scavenger for the unwanted

Protect the control plane

Markings can say too much

Finding a QoS fault hop by hop

The tools I reach for

How I run a QoS programme, from first complaint to steady state

Functional requirements

Non functional requirements

Who I talk to

How I find out