Routing decides where every packet goes, and almost every outage I have fixed came down to one router choosing a path nobody expected. This page shows how a router decides, how OSPF, EIGRP and BGP work, and how I find the fault when a route goes missing.
Every forwarding decision follows the same order. First the longest prefix that matches the destination. If two sources offer the same prefix, the lower administrative distance. If one protocol offers two paths, the lower metric. Try it: pick a destination and watch the router choose.
Swipe sideways to see the whole diagram.
| Prefix | Learned by | AD | Matches? |
|---|---|---|---|
| 10.1.20.0/24 | EIGRP | 90 | yes |
| 10.1.0.0/16 | eBGP | 20 | yes |
| 10.0.0.0/8 | OSPF | 110 | yes |
| 0.0.0.0/0 | Static | 1 | yes |
10.1.20.15 matches four routes. The longest is /24, so it goes to the branch office. The low AD (administrative distance) of the other routes does not matter, because they are different prefixes.
When two protocols offer exactly the same prefix, the router trusts the one with the lower administrative distance (AD). These are Cisco's defaults. Juniper calls it route preference and uses different numbers: static 5, OSPF internal 10, BGP 170.
| Source | AD | Source | AD |
|---|---|---|---|
| Connected interface | 0 | OSPF | 110 |
| Static route | 1 | IS-IS | 115 |
| EIGRP summary | 5 | RIP (Routing Information Protocol) | 120 |
| External BGP (eBGP) | 20 | EIGRP external | 170 |
| EIGRP internal | 90 | Internal BGP (iBGP) | 200 |
Swipe sideways to see the whole diagram.
Each routing protocol keeps its own table and offers its best route for each prefix to the routing table. Here EIGRP, OSPF and iBGP all know 10.1.20.0/24.
The routing table, the RIB (routing information base), keeps the offer with the lowest administrative distance. EIGRP at 90 wins. show ip route lists it with the AD and metric in brackets, such as [90/3072], and show ip route 10.1.20.0 spells out the distance and metric.
The winner is copied into the FIB (forwarding information base) with the next hop's MAC (media access control) address already resolved. Cisco calls this CEF (Cisco Express Forwarding). Packets are switched in hardware without the CPU. show ip cef 10.1.20.15 detail shows the result.
If EIGRP loses the route, the next best offer, OSPF at 110, is installed at once. The routes a protocol offers but does not win stay in its own table, ready.
When one protocol has several paths with the same metric, the router installs them all, up to maximum-paths: 4 by default for OSPF and EIGRP on IOS, 1 for BGP. CEF shares traffic per flow by hashing addresses, so packets in one flow stay in order.
A metric only compares routes from the same protocol. OSPF uses cost, EIGRP a composite of bandwidth and delay, BGP its path attributes. Comparing an OSPF cost with an EIGRP metric means nothing.
An IGP (interior gateway protocol) such as OSPF, IS-IS or EIGRP inside one network, for fast convergence. BGP at the edges: providers, the WAN (wide area network), cloud and data centre fabrics, where policy and scale matter more than speed.
A static route never changes by itself. It stays in the table as long as its next hop looks reachable, even if the path beyond has failed. On a branch with two links, I tie the primary route to a probe and keep a floating static as the backup.
Swipe sideways to see the whole diagram.
The primary default route has distance 1 and is tied to track 10. The backup is a floating static: the same prefix with distance 250, so it stays out of the table while the primary is there. An IP SLA (service level agreement) probe pings an address beyond the provider every 5 seconds, sourced from the primary interface.
The provider fails upstream, but the link to its router stays up. A plain static route would stay in the table and black hole traffic. The probes start to fail.
After the down delay, track 10 goes down and the primary route is removed. The floating static, at distance 250, is now the best route and is installed.
Traffic leaves through 4G. On a firewall or NAT (network address translation) edge, sessions may need to restart, because their source address has changed.
When the probes succeed again, an up delay of 30 seconds stops the route flapping before traffic moves back. I also add a host route so the probe always uses the primary link.
ip sla 10 with icmp-echo 203.0.113.10 source-interface Gi0/0/0 and frequency 5, started with ip sla schedule 10 life forever start-time now, then track 10 ip sla 10 reachability with delay down 10 up 30, then ip route 0.0.0.0 0.0.0.0 198.51.100.1 track 10 and the backup ip route 0.0.0.0 0.0.0.0 192.0.2.1 250.
A static route to Null0 discards traffic. A summary pointing at Null0 stops loops when part of the summary is missing, and a static to Null0 plus a BGP network statement advertises a stable aggregate.
A static route through an Ethernet interface alone relies on proxy ARP (address resolution protocol), so I always give a next hop address. BFD (Bidirectional Forwarding Detection) can also watch a static route's next hop and remove it within a second.
OSPF (Open Shortest Path First) is a link state protocol. Every router in an area floods LSAs (link state advertisements) describing its links, so all of them hold the same database, and each runs the SPF (shortest path first) algorithm with itself at the root. Areas keep that flooding contained.
Swipe sideways to see the whole diagram.
Areas and router types. Every area joins area 0, and traffic between areas crosses it. An ABR (area border router) sits in area 0 and another area, and summarises one into the other. An ASBR (autonomous system boundary router) brings in routes from outside OSPF. A change inside area 1 runs SPF only in area 1. Other areas just see a changed type 3 summary, or nothing at all if the ABR summarises.
Stub and totally stubby. A stub area blocks type 4 and type 5 LSAs, and its ABR sends a default route instead. A totally stubby area, a Cisco extension, also blocks type 3 summaries except that default. Routers in the area keep small tables and only need to know the way out. No ASBR may sit in a stub area.
NSSA. A not so stubby area blocks types 4 and 5 like a stub, but allows a local ASBR, for example a router redistributing a partner's static routes. Its routes travel as type 7 LSAs inside the area, and the ABR converts them to type 5 for everyone else. An NSSA gets no default route automatically: I add one with area 2 nssa default-information-originate, or use a totally NSSA with no-summary.
| LSA type | Name | Sent by | Floods to | In the routing table |
|---|---|---|---|---|
| 1 | Router | Every router | Its own area | O |
| 2 | Network | The DR on a shared segment | Its own area | O |
| 3 | Summary | ABR | Other areas | O IA (inter area) |
| 4 | ASBR summary | ABR | Other areas, to say where the ASBR is | not a route itself |
| 5 | AS external | ASBR | Every area except stub and NSSA areas | O E1 or O E2 |
| 6 | Group membership | Multicast OSPF, which was never widely used | Not supported by Cisco | none |
| 7 | NSSA external | ASBR inside an NSSA | The NSSA only, converted to type 5 at the ABR | O N1 or O N2 |
Cost is the reference bandwidth divided by the interface bandwidth. IOS defaults to 100 Mb/s, so 1, 10 and 100 Gb/s links all cost 1. I set auto-cost reference-bandwidth 100000 on every router, the same everywhere. NX-OS defaults to 40 Gb/s.
E2, the default, keeps the cost set at the ASBR. E1 adds the internal cost to reach the ASBR, which picks the nearest exit when two ASBRs advertise the same route. OSPF always prefers intra area, then inter area, then E1, then E2, whatever the cost.
OSPF summarises only at ABRs (area 1 range) and ASBRs (summary-address). The router adds a Null0 discard route for the summary. Two ABRs advertising the same summary can black hole traffic if one loses part of the range.
Routers in one area must hold the same database, so OSPF filters only where it summarises. On an ABR, area 1 filter-list prefix BLOCK in stops chosen type 3 summaries entering area 1, and area 1 range ... not-advertise hides a range. A distribute-list prefix BLOCK in under the process is different: the LSAs still flood and stay in the database, and only this router's own routing table leaves them out.
Every area must touch area 0. When a new area can only reach the backbone through another area, area 1 virtual-link 2.2.2.2 on both ABRs builds a logical area 0 link through area 1, the transit area, which must not be a stub. It also repairs a backbone split in two. I treat it as a temporary fix and plan a real link to area 0.
The router ID comes from the router-id command, else the highest loopback address, else the highest active interface address. It is fixed when the process starts. I always set it to a loopback. Duplicate router IDs cause strange, intermittent faults.
default-information originate advertises a default only while the router has one itself, so it withdraws when the upstream fails. Adding always can black hole traffic. I tie the router's own default to tracking.
A link that stays up but stops forwarding waits out the 40 second dead timer, so I add BFD with ip ospf bfd, with bfd interval 300 min_rx 300 multiplier 3 set on the interface. SPF throttling (timers throttle spf) stops a flapping link causing constant recalculation. Before maintenance, max-metric router-lsa makes traffic route around the router first.
feature ospf first, then a named process such as router ospf UNDERLAY. Interfaces join with ip router ospf UNDERLAY area 0.0.0.0, not network statements.
Two OSPF routers move through a fixed set of states before they share routes. When an adjacency will not form, the state it sticks in tells me where to look. OSPF runs over IP protocol 89, sending to 224.0.0.5 for all OSPF routers and 224.0.0.6 for the DR and BDR.
Swipe sideways to see the whole diagram.
No hello has been heard from the neighbour. On a broadcast link, hellos go every 10 seconds and the neighbour is dead after 40.
R2 has received a hello from R1, but R1 does not yet list R2 in its hellos. To be accepted, the hellos must agree on area, subnet and mask, hello and dead timers, authentication and the stub flags.
Each router sees its own router ID in the other's hello, so the link works both ways. On a shared segment, the DR (designated router) and BDR (backup DR) are elected here. Two routers that are neither stay in 2-Way, which is normal.
The routers agree who leads the database exchange. The higher router ID becomes master. The DBD (database description) packets carry the interface MTU (maximum transmission unit), and a mismatch stalls here.
Each router sends DBD packets listing the headers of every LSA it holds, so the other can see what it is missing.
R1 sends an LSR (link state request) for the LSAs it lacks. R2 answers with an LSU (link state update), and R1 acknowledges it with an LSAck (link state acknowledgement).
Both databases match. The routers run SPF, install routes and keep sending hellos. show ip ospf neighbor shows FULL/DR, FULL/BDR, or FULL/ - on a point to point link.
On a shared segment, every router would otherwise form a full adjacency with every other. Instead all routers go Full with the DR and BDR only. The highest priority wins, 1 by default and 0 for never, then the highest router ID. The election does not preempt, so a new router with a higher priority waits.
Broadcast elects a DR. Point to point does not, and forms the adjacency faster, so I set ip ospf network point-to-point on routed links between two switches. Non broadcast and point to multipoint networks use 30 and 120 second timers.
When a new circuit or subinterface has a different MTU, the adjacency cycles through ExStart and Exchange while traffic falls back to the backup. debug ip ospf adj shows "Nbr ... has larger interface MTU" on the router with the smaller MTU. I fix the MTU and use ip ospf mtu-ignore only as a short, recorded workaround.
EIGRP (Enhanced Interior Gateway Routing Protocol) is Cisco's advanced distance vector protocol. Its DUAL algorithm (diffusing update algorithm) keeps a loop free backup route in its topology table, so it can switch over without asking anyone.
Swipe sideways to see the whole diagram.
A knows three ways to network N. The best total distance, the feasible distance, is 20 through B. B is the successor and its route is installed.
A neighbour is a feasible successor if the distance it reports is lower than A's current feasible distance. C reports 15, under 20, so its path cannot loop back through A. D reports 25, so it might, and is not a feasible successor.
When the link to B fails, A installs the path through C straight away. No queries are needed, which is why EIGRP converges so fast.
If C fails too, there is no feasible successor. The route goes active and A queries its neighbours. When the replies arrive it installs the path through D. A query that never gets a reply leaves the route stuck in active, after 3 minutes by default.
IP protocol 88, multicast 224.0.0.10, hellos every 5 seconds on fast links with a 15 second hold time. The classic metric uses the lowest bandwidth and the total delay along the path (K values K1 and K3). K values must match for neighbours to form. Summaries and stub routers limit how far queries travel.
EIGRP can share traffic over paths with different metrics. variance 2 installs any feasible successor whose metric is under twice the best, and traffic is split in proportion to the metrics. Only feasible successors qualify, so the extra paths are still loop free.
eigrp stub connected summary on a branch router tells its neighbours not to query it when a route goes active. Queries stop at the hub, which keeps convergence fast and stops a remote site causing stuck in active routes. A stub also never offers itself as a transit path between two hubs.
Named mode puts everything under one place: router eigrp CAMPUS, then address-family ipv4 unicast autonomous-system 100, with per interface settings under af-interface. It uses wide 64 bit metrics, so 10 Gb/s, 40 Gb/s and 100 Gb/s links get different metrics instead of all looking the same.
IS-IS (Intermediate System to Intermediate System) is a link state protocol like OSPF, but it runs directly over Layer 2, not IP. Level 1 routes inside an area and Level 2 is the backbone. Its AD is 115. It runs many service provider cores and the hidden underlays of Cisco ACI (Application Centric Infrastructure), SD-Access and FabricPath.
OSPF is the open, multi vendor default inside an enterprise. EIGRP converges fast and is easy to run, but it is mainly found on Cisco estates. IS-IS scales well and is common in large fabrics. BGP joins all of them at the edges.
BGP (Border Gateway Protocol) is a path vector protocol. It runs over TCP (Transmission Control Protocol) port 179, carries a list of autonomous systems (AS) with every route to stop loops, and lets me apply policy to every path. It carries the internet's table, about a million IPv4 routes, and the routes between my sites, providers and clouds.
Swipe sideways to see the whole diagram.
The session is not trying to connect: it is shut down, it exceeded its maximum prefix limit, it has no route to the neighbour, or it is waiting after an error. show ip bgp summary shows Idle (Admin) or Idle (PfxCt) for the first two.
BGP opens a TCP connection to port 179. Neighbours are configured by hand on both sides, and the source address must match what the other side expects.
Despite the name, Active means the TCP connection failed and BGP is retrying. A session stuck in Active almost always means TCP 179 is blocked or the addressing is wrong. telnet 192.0.2.1 179 /source-interface Loopback0 proves it quickly.
TCP is up, and each side sends an OPEN message with its AS number, hold time, router ID and capabilities, such as the address families it supports. A bad OPEN, such as the wrong AS number, gets a NOTIFICATION and the session closes.
The OPEN was accepted. Each side sends a KEEPALIVE, and the first KEEPALIVE received moves the session to Established.
The routers exchange UPDATE messages carrying routes and their attributes. Keepalives go every 60 seconds and the hold time is 180 seconds by default on Cisco, so BFD is needed for fast failure detection.
Swipe sideways to see the whole diagram.
Full mesh. An iBGP router never passes a route learned from one iBGP peer to another. That is iBGP split horizon, and it prevents loops inside an AS. So every iBGP router must peer with every other: n(n−1)/2 sessions, which grows fast.
Route reflectors. A route reflector (RFC 4456) may pass routes to its clients. Each client peers with two reflectors for resilience. Two attributes, ORIGINATOR_ID and CLUSTER_LIST, stop reflected routes looping. A reflector passes on only its own best path, which can hide alternatives.
Confederation. The AS is split into sub ASes with private numbers. Inside each one, iBGP rules apply. Between them, sessions behave like eBGP for loop prevention but keep the next hop, local preference and MED (multi exit discriminator). From outside, the whole thing looks like one AS. It is common after mergers and in large providers.
eBGP runs between ASes, usually between directly connected addresses, with a TTL (time to live) of 1 (ebgp-multihop changes that). iBGP runs inside an AS between loopbacks, with update-source Loopback0. eBGP sets itself as next hop. iBGP leaves the next hop alone, so I use next-hop-self or carry the edge subnets in the IGP.
Every eBGP session is filtered in both directions with prefix lists and route maps, and AS path filters such as ^$ so I never become a transit for a provider. neighbor 192.0.2.1 maximum-prefix 500 80 restart 15 drops a session that suddenly sends far too many routes.
A community is a tag on a route, written AA:NN, that policy can match later. Well known ones include NO_EXPORT, NO_ADVERTISE, GRACEFUL_SHUTDOWN (RFC 8326), which drains traffic before maintenance, and BLACKHOLE (RFC 7999), which asks a provider to drop traffic to an address under attack.
A 180 second hold time is far too slow for a WAN. BFD (RFC 5880) runs alongside BGP, OSPF or a static route and detects a failed path in under a second, for example 300 ms with a multiplier of 3. Single hop BFD uses UDP (User Datagram Protocol) port 3784.
After changing inbound policy, clear ip bgp 192.0.2.1 soft in asks the neighbour to resend its routes using the route refresh capability, so the session stays up. The older neighbor 192.0.2.1 soft-reconfiguration inbound keeps an unfiltered copy of every received route in memory. I use it only where a peer lacks route refresh, or when I need received-routes.
aggregate-address 10.1.0.0 255.255.0.0 summary-only advertises one summary once at least one component route is in the BGP table, and hides the rest. Route flap dampening adds a penalty of 1,000 each time a route flaps and suppresses it above 2,000 until the penalty decays below 750, with a 15 minute half life.
RPKI (Resource Public Key Infrastructure) lets an edge router check whether an AS may originate a prefix. How I use it is on the security page.
BGP compares paths to the same prefix one attribute at a time, in a fixed order, and stops at the first difference. Step through three paths to 203.0.113.0/24 to see where each one drops out.
Swipe sideways to see the whole diagram.
Only paths with a reachable next hop, and without my own AS in the AS path, are considered. All three qualify.
Weight is Cisco only and never leaves the router. Higher wins. It is 0 for learned routes, so they tie.
Local preference is shared across my AS, and higher wins. I set 200 on routes from both providers, so the iBGP path at the default of 100 drops out. This is how I choose the exit for outbound traffic.
A route I originate myself would win here. Next comes the shortest AS path: both providers show two ASes, so they tie.
IGP beats EGP (exterior gateway protocol, the predecessor of BGP), which beats incomplete. Routes redistributed into BGP show as incomplete. Both are IGP.
MED (multi exit discriminator), lower wins, is only compared between paths from the same neighbouring AS by default. These come from different providers, so it is skipped.
eBGP beats iBGP, then the lowest IGP cost to the next hop. Both are eBGP over direct links, so they still tie. With maximum-paths 2 and bgp bestpath as-path multipath-relax, needed because the two AS paths are different, BGP could now use both.
The oldest eBGP path wins, which keeps traffic stable. Then the lowest router ID, the shortest cluster list and the lowest neighbour address. Provider A has been up for 40 days and is chosen.
Swipe sideways to see the whole diagram.
Outbound: I decide. A route map on the session to provider A sets local preference 200 on every route received, so every router in my AS sends internet traffic out through A. Provider B stays as the backup at the default of 100. Weight does the same on a single router only.
Inbound: I can only ask. Other networks choose how to reach me. Prepending my AS several times towards B makes that path look longer, so most traffic returns through A, though more than three or four prepends rarely helps. Other tools: MED towards one provider with two links, communities the provider publishes, and more specific prefixes, down to /24 for IPv4 on the internet.
Well known mandatory attributes are in every update: ORIGIN, AS_PATH and NEXT_HOP. Well known discretionary ones every router understands but need not send: LOCAL_PREF, used only inside an AS, and ATOMIC_AGGREGATE. Optional transitive ones pass on even if a router does not understand them: COMMUNITY and AGGREGATOR. Optional non transitive ones are dropped by a router that does not understand them, and never travel far: MED, ORIGINATOR_ID and CLUSTER_LIST.
The address to forward traffic to for the route. An eBGP speaker sets it to itself. An iBGP speaker passes it on unchanged, so every router in the AS must be able to reach it, through the IGP or with next-hop-self at the edge. An unreachable next hop means the route is never used.
When a router summarises routes and drops the AS paths of the parts, it adds ATOMIC_AGGREGATE to warn that path detail is lost. AGGREGATOR records the AS and router ID that made the summary. aggregate-address ... as-set keeps the AS numbers instead, which avoids loops but makes the summary change whenever a part changes.
Redistribution copies routes from one protocol into another. It is common during migrations and mergers, and it is where many routing loops come from, especially when two routers redistribute in both directions.
Swipe sideways to see the whole diagram.
A site router puts a static route into EIGRP. Inside EIGRP it is an external route, with administrative distance 170.
R1 redistributes EIGRP into OSPF, so the route enters OSPF as a type 5, E2 route. Every OSPF router learns it, including R2.
R2 hears the route two ways: EIGRP external at 170 and OSPF at 110. It prefers OSPF, and redistributes that route back into EIGRP as a new external route. If EIGRP routers now see a better metric through R2, traffic circles through R2, the OSPF core and R1. Traceroute bounces until the TTL runs out.
On every redistributing router I tag routes on the way in, with set tag 100 in a route map, and deny that tag on the way back with match tag 100. Summarising at the boundary, or changing the distance of external routes, helps too.
Each protocol needs a starting metric for redistributed routes. OSPF uses 20 as E2, and 1 for routes from BGP. EIGRP needs a metric set, or the routes are silently left out, apart from connected and static. Older IOS also needs the subnets keyword into OSPF.
Redistributing OSPF into BGP takes internal routes only, unless I match externals too. I never redistribute a full BGP table into an IGP. I advertise a default or a few summaries instead.
show ip route 192.168.50.0 repeated a few seconds apart shows the source changing. The route tag is in the output, and show ip ospf database external shows who is advertising it.
A VRF (virtual routing and forwarding instance) gives a router a separate routing table for each segment, so tenants or zones stay apart even if their addresses overlap. Policy based routing (PBR) overrides the routing table for chosen traffic.
Swipe sideways to see the whole diagram.
Separate tables. Each interface belongs to one VRF, and each VRF has its own routing table, so the same 10.0.0.0/24 can exist twice. Without MPLS this is VRF Lite: one subinterface and one routing process or address family per VRF on each link. On IOS XE: vrf definition RED with address-family ipv4, then vrf forwarding RED on the interface, then the IP address again, because adding the VRF removes it.
Route leaking. Shared services such as DNS (domain name system) and NTP (network time protocol) can be reached by importing their routes into each VRF. Route targets with MP-BGP (multiprotocol BGP) do this, or a static route that points into the other table. It works, but it bypasses inspection, so I treat every leak as a design decision with an owner, and the return route must leak too.
Through a firewall. My preferred design joins VRFs through a firewall, which checks and logs every flow between zones. A VLAN, or its equivalent in a data centre fabric, is one segment. A VRF is a zone that holds many segments. Zones meet at the firewall.
An ACL (access control list) matches the traffic, a route map sets the next hop with set ip next-hop, and ip policy route-map applies it on the incoming interface. Traffic that does not match routes normally. PBR does not show in the routing table, so I document it carefully.
show vrf, show ip route vrf RED and ping vrf RED 10.0.0.1. On NX-OS, management traffic lives in VRF management, so ping 10.9.9.1 vrf management.
In an MPLS VPN (virtual private network) each customer is a VRF on the provider edge. In a VXLAN EVPN fabric (Virtual Extensible LAN with an Ethernet VPN control plane) each tenant VRF maps to a Layer 3 VNI (VXLAN network identifier). Both are covered on the WAN page and the VXLAN EVPN page.
Multicast sends one copy of a stream, such as market data, video or a building system update, to every receiver that asks for it. Hosts join with IGMP (Internet Group Management Protocol), and routers build the delivery tree with PIM (Protocol Independent Multicast). Group addresses come from 224.0.0.0/4.
Swipe sideways to see the whole diagram.
The receiver sends an IGMP report for group 239.1.1.1 to its router. Switches with IGMP snooping note the port, so the stream reaches only that port.
The last hop router sends a PIM join towards the RP (rendezvous point), building a shared tree for the group, written (*,G). Each router sends the join out of the interface its unicast table uses to reach the RP. Multicast is then accepted only on that interface, the RPF (reverse path forwarding) check, which stops loops.
The source's first hop router wraps the first packets in unicast register messages to the RP. The RP joins back towards the source, and the stream flows down the shared tree to the receiver.
Once the stream arrives, the last hop router joins the source directly, a source tree written (S,G), and traffic moves off the RP. Cisco does this after the first packet by default. Source specific multicast, in 232.0.0.0/8, skips the RP altogether.
ip pim rp-address 10.0.0.1 on every router. It is simple and predictable, but every router needs changing if the RP moves, so I pair it with Anycast RP for resilience.
Cisco's own method. Candidate RPs announce themselves to 224.0.1.39, and a mapping agent picks one per group and sends the result to 224.0.1.40. In a sparse mode network those two groups need ip pim autorp listener to reach every router.
The bootstrap router (BSR) method, the open standard in PIM version 2 (RFC 5059). Candidate RPs report to the elected BSR, which floods the RP set hop by hop to every PIM router. It works across vendors.
Two or more RPs share one loopback address, so each router joins the nearest and a failed RP simply drops out of routing. The RPs tell each other about active sources with MSDP (Multicast Source Discovery Protocol), as in RFC 3446, or with PIM itself, as in RFC 4610.
IPv6 routing follows the same rules: longest match, distance, metric. What changes is how neighbours and gateways are found, and some commands. Addressing itself is on the IP addressing page.
On IOS, routers do not forward IPv6 until ipv6 unicast-routing is set. Hosts find their gateway through router advertisements, part of NDP (Neighbor Discovery Protocol), which also replaces ARP.
Routing protocols use the neighbour's link local address, in fe80::/10, as the next hop. That is why a static route over Ethernet often needs both the interface and the link local address.
OSPFv3 runs per interface and carries IPv6, and with address families IPv4 too. EIGRP and IS-IS support IPv6 directly. BGP carries IPv6 as a separate address family, so the neighbour must be activated under address-family ipv6.
Queues form where a fast link meets a slower one, which is usually the WAN edge. QoS (quality of service) decides which traffic goes first, which gets a guaranteed share, and what is dropped. It only helps during congestion. It never creates bandwidth.
Swipe sideways to see the whole diagram.
The LAN runs at 1 Gb/s but the circuit is 100 Mb/s. When traffic bursts, packets must wait. Without QoS, the queue is first in, first out, and voice waits behind a file copy.
The router sorts each packet into a class by its DSCP (differentiated services code point) marking, set at the campus edge. Classification and marking happen as close to the source as possible.
LLQ (low latency queuing) gives voice, marked EF (expedited forwarding), a strict priority queue, policed to its rate so it cannot starve the rest. CBWFQ (class based weighted fair queuing) guarantees each other class a share. WRED (weighted random early detection) drops a little best effort traffic early, so TCP slows down smoothly.
When a 1 Gb/s port connects to a 100 Mb/s service, the router would send faster than the provider accepts and the provider would drop at random. A parent shaper holds the port to 100 Mb/s, and the queuing policy runs inside it. This is hierarchical QoS.
A policer drops or remarks traffic above a rate at once, which makes TCP back off hard. A shaper buffers the excess and sends it later, adding delay but fewer drops. Providers police, so I shape before their policer.
show policy-map interface Gi0/0/0 shows matches, queue depth and drops for each class. Drops in the priority class mean voice is over its budget. Drops in best effort during busy hours are expected.
The markings and classes in the campus, on the WAN and in the data centre must agree. A provider that remarks traffic, or a tunnel that hides the marking, breaks it.
Most routing faults are a neighbour that will not form, a route that never arrives, or traffic that returns a different way. Pick a symptom to see what I check, with commands for IOS and IOS XE, and NX-OS where they differ.
show ip ospf neighbor, because parameters differ and hellos are dropped, or it shows INIT on one side only, because hellos are lost in one direction.show ip ospf interface Gi0/0/1 on both ends side by side, show ip ospf interface brief, show ip protocols, show logging | include OSPF. NX-OS: show ip ospf neighbors and show ip ospf interface brief.show interfaces Gi0/0/1 | include MTU on both ends, ping 172.16.0.2 size 1500 df-bit stepping down, debug ip ospf adj briefly, then undebug all.mtu and ip mtu with what the circuit really carries, then confirm FULL and traffic back on the primary.show ip route 10.1.20.15 (the longest match it really uses), show ip ospf database | include 10.1.20.0, show ip protocols for filters and distances, show ip ospf border-routers.show ip bgp summary shows Active for the neighbour, often after a firewall change.ebgp-multihop. An ACL, control plane policing (CoPP) or a firewall blocking TCP 179 in either direction. MD5 (message digest 5) authentication set on one side only. NAT changing the source.ping 192.0.2.1 source Loopback0, telnet 192.0.2.1 179 /source-interface Loopback0 (open means a BGP setting, refused means the far end is not configured, timeout means dropped), show tcp brief | include 179, show logging | include BGP|BADAUTH. NX-OS: show bgp ipv4 unicast summary.show ip bgp neighbors 192.0.2.1 | include BGP state|Last reset, show ip route 192.0.2.1, show logging | include NOTIFICATION|MAXPFX.no neighbor 192.0.2.1 shutdown, fix the prefix source or raise the limit under change, restore the route to the peer, or correct the AS.network statement that does not exactly match a route in the table. An outbound or inbound filter, with its implicit deny. The address family not activated. iBGP split horizon. A next hop that is not reachable. RIB failure, where a static or OSPF route wins on distance.show ip bgp neighbors 192.0.2.1 advertised-routes and received-routes (which needs soft-reconfiguration inbound), show ip bgp 203.0.113.0, show ip bgp rib-failure, show route-map, show ip prefix-list.traceroute in both directions, show standby brief, show ip route for the user subnet on the upstream router, and the firewall's drop reasons, such as show asp drop on an ASA (no SYN seen).show ip route 192.168.50.0 a few times (source and tag), show ip ospf database external | include Tag|Link State ID, show ip protocols.ip ospf cost. Intra area preferred over inter area regardless of cost. E2 ignoring internal cost.traceroute 10.1.20.15 source Loopback0, show ip ospf interface brief on each hop for cost, show ip cef exact-route 10.9.1.1 10.1.20.15 for which ECMP member is used.My general method, from defining the problem to closing the record, is on the troubleshooting page.
OSPF, BGP and MPLS (Multiprotocol Label Switching) run through my career, from enterprise WANs to provider and peering edges, and my home lab runs an MPLS core on physical routers.
My home lab runs an MPLS core, provider edge routers with customer VRFs, DMVPN (Dynamic Multipoint VPN) backup across a simulated internet built from VRFs, and a default route out to the real internet, all on physical routers.
Defined routing security policy for the peering and transit edge: RPKI and ROA (route origin authorisation) origin validation, and RIPE NCC (the European regional internet registry) resource governance.
Managed provider IP space and AS numbers with RIPE NCC, including RPKI ROAs, for a Tier 3 data centre serving over 300 customer sites.
Secured the consolidated internet edge with RPKI based BGP origin validation and cleaned up the RIR (regional internet registry) records across the merged address space.
High and low level designs for enterprise customers including Rolls Royce, Virgin Money, Rio Tinto and Primark.
Cisco based telecoms infrastructure for over 4,000 clients across East Africa and South America.
Backed by CCNP Enterprise, CCNP Routing and Switching, CCDP, the Cisco Enterprise Advanced Infrastructure specialist certification and the CCIE Routing and Switching written exam. See all credentials
Every page follows the same idea: how it works, the choices that matter, and where I have used it. The foundations pages explain the basics the others build on.