I design and build Cisco SD-Access fabrics with Catalyst Center and ISE (Identity Services Engine). This page walks through each plane, every role and the traffic between them, from the first switch powering on to the fault finding when something goes wrong.
Live view · endpoints registered, traffic tagged by group
Overview
A campus fabric where identity sets the policy
SD-Access is Cisco's campus fabric. It splits the network into four planes, each doing one job, so that a user or device gets the same access wherever it connects. I design it from the underlay up. The SD-WAN (software defined wide area network) page has a shorter summary of how it fits with the WAN (wide area network).
Swipe sideways to see the whole diagram.
Underlay. A plain routed network: point to point links, a loopback on every switch, and IS-IS (Intermediate System to Intermediate System) as the routing protocol. There is no spanning tree between fabric switches. Catalyst Center builds and checks it, and nothing clever lives here on purpose.
Control plane. LISP (Locator/ID Separation Protocol) keeps a live directory of every endpoint: which switch it sits behind. Edges register what they learn and ask the control plane node when they need to reach something, instead of flooding.
Data plane. Traffic between edges travels in VXLAN (virtual extensible LAN) between the switches' loopbacks. Any subnet can live anywhere without stretching VLANs (virtual local area networks), and every edge acts as the default gateway.
Policy plane. Cisco ISE (Identity Services Engine) gives each user or device a scalable group tag (SGT) when it connects. The tag travels inside the VXLAN header, and the destination edge allows or blocks traffic by group pair, not by IP address.
Animation
Three things with similar names
Catalyst SD-WAN joins sites across the WAN. SD-Access is the campus fabric. Catalyst Center, formerly DNA Center (Digital Network Architecture Center), is the controller that builds, runs and assures SD-Access. They meet at the border, where each fabric virtual network becomes an SD-WAN service VPN (virtual private network).
Two levels of segmentation
A virtual network (VN) is a VRF (virtual routing and forwarding table): a separate routing table, used for large groups such as staff, guests, IoT (Internet of Things) and OT (operational technology). Inside a VN, scalable group tags give fine grained, identity based rules. Traffic between VNs leaves the fabric through a firewall.
When I recommend it
Large campuses with many device types, a need for identity based policy, frequent moves and a team ready to run ISE and Catalyst Center. For a small site, a well designed routed access campus is often enough.
How it compares with a traditional campus
Traditional VLAN campus
SD-Access fabric
Segmentation
VLANs, ACLs (access control lists) and firewall rules written per subnet
Virtual networks for large groups, scalable group tags inside them
Where a device lives
Its subnet is tied to a closet or a stretched VLAN
Any subnet on any edge, with the same gateway everywhere
Loops and failure domains
Spanning tree across the access and distribution layers
Routed underlay, no spanning tree between fabric switches
Wired and wireless
Separate policy, wireless often tunnelled to a controller
One policy, wireless traffic switched at the edge
Change
Switch by switch configuration
Intent set once in Catalyst Center and pushed to every switch
Costs and dependencies
Familiar and simple to licence
Advantage tier Catalyst licences, a dependency on ISE, and pool and policy design up front
Underlay
An underlay that is boring on purpose
The underlay is just a routed network that carries the fabric. Catalyst Center can build it for me with LAN (local area network) Automation, so every switch is addressed and configured the same way. I can also build it by hand with OSPF (Open Shortest Path First), as long as the rules are the same.
Swipe sideways to see the whole diagram.
Step 1
I configure one or two seed switches by hand, normally the distribution or border pair, and make them reachable from Catalyst Center. Everything below them will be built automatically.
New switches arrive with no configuration. They get a temporary address from a DHCP (Dynamic Host Configuration Protocol) pool on the seed, which also tells them where Catalyst Center is. Their Plug and Play agent calls home, and Catalyst Center can upgrade the software before anything else.
Catalyst Center takes addresses from the underlay pool I defined: a /31 for every point to point link and a /32 Loopback0 on every switch. The loopback becomes the switch's address in the fabric, its routing locator (RLOC).
LAN Automation turns on IS-IS on every link. The access layer is now routed, so there is no spanning tree between fabric switches and every uplink carries traffic with equal cost paths.
The system MTU (maximum transmission unit) is 9100 bytes everywhere, so VXLAN's 50 extra bytes never cause trouble. If I need Layer 2 flooding or native multicast later, I add underlay multicast now.
Rules I never bend
Routed point to point links, a /32 loopback per switch reachable everywhere, equal cost paths, MTU 9100 end to end, and no spanning tree between fabric nodes.
Size the pools for the end state
Underlay, endpoint and wireless pools are planned for the final size of the campus, not today's. A /24 that looks generous today runs out when the IoT devices arrive.
Check it like any routed network
show isis neighbors, show ip route for every loopback, and a ping with a large size and the do not fragment bit between loopbacks to prove the MTU.
Fabric roles
Every switch has one clear job
A fabric is easier to run when every device has a defined role. I pick the roles for each site by size and resilience: two control plane nodes and two borders as standard, and as many edges as there are closets.
Swipe sideways to see the whole diagram.
Edge node. The access switch. It is the default gateway for every VN, with the same anycast address on every edge. It authenticates users and devices with 802.1X or MAB (MAC Authentication Bypass), applies the group tag, registers endpoints with the control plane and enforces group policy. Typically a Catalyst 9200, 9300 or 9400.
Border node. The way in and out. An internal border connects to known networks, such as the data centre. An external border is the default exit to everything else, such as the internet through a firewall, and an internal and external border does both. A Layer 2 border joins a fabric pool to an old VLAN during migration. Each VN is handed off separately, usually as VRF lite with BGP (Border Gateway Protocol) to a fusion device or firewall.
Control plane node. The LISP map server and map resolver: a database of every endpoint and the edge it sits behind. I always deploy two per fabric site, on Catalyst 9500 or 9600 switches, or on Catalyst 8000V virtual routers for large sites.
Fabric wireless. Access points keep CAPWAP (Control and Provisioning of Wireless Access Points) control traffic to the wireless LAN controller, but client data goes straight into the fabric at the edge in VXLAN. The controller registers each wireless client with the control plane node, so wired and wireless share one policy.
Extended node. A Catalyst IE (Industrial Ethernet) industrial switch or a compact switch in a plant room or ceiling, onboarded and managed by Catalyst Center. Its ports join VNs like edge ports. A policy extended node can also enforce group policy; a plain one relies on its parent edge.
Animation
Role
What it does
Typical platform
Control plane node
LISP map server and resolver: where every endpoint lives
Catalyst 9500 or 9600, or Catalyst 8000V. Two per site
Border node
Connects the fabric to everything else, one handoff per VN. Internal, external, or both
Routes underlay traffic only, knows nothing about the fabric
Any routed switch
Fabric wireless controller
CAPWAP control for access points, registers wireless clients
Catalyst 9800 series
Extended node
Extends fabric ports to places an edge cannot reach, in REP (Resilient Ethernet Protocol) rings where needed
Catalyst IE 3300 or 3400, compact Catalyst switches
Fabric terms in one line each
Term
What it means
INFRA_VN
The virtual network for access points and extended nodes. It lives in the global routing table, not a VRF.
DEFAULT_VN
The user virtual network Catalyst Center creates for you. I add my own VNs for each large group.
Anycast gateway
The same gateway address and MAC (media access control) address on every edge, so a device keeps its gateway wherever it connects.
Critical VLAN
Where a port puts devices when ISE cannot be reached, so they keep a known, limited level of access.
Device tracking
SISF (switch integrated security features) learns each endpoint's addresses from DHCP, ARP (Address Resolution Protocol) and neighbour discovery, so the edge can register it with LISP.
pxGrid
ISE's publish and subscribe service, which shares who and what is on the network with firewalls and other tools.
SXP (SGT Exchange Protocol)
It carries IP address to group bindings over TCP (Transmission Control Protocol) to devices that cannot read tags inline.
Plug and Play
How a new switch finds Catalyst Center and gets its configuration, with LAN Automation building the underlay.
Licences
SD-Access needs the Advantage tier of the switch and Catalyst Center licences. Essentials is not enough.
LISP
A directory instead of flooding the campus
The control plane is what makes the fabric scale. Instead of every switch learning every address by flooding, each edge registers what it has and asks for what it needs. The instance ID in these commands is an example: each VN has its own.
Swipe sideways to see the whole diagram.
Step 1
The laptop connects to edge A and gets an address. Device tracking on the switch (SISF) learns its IP and MAC address from DHCP and ARP. In LISP terms the laptop is an endpoint identifier (EID), and edge A's loopback is its routing locator (RLOC).
Edge A sends a map register to the control plane node: 10.10.1.20 sits behind 10.0.250.11. Each VN has its own LISP instance, so the same address can exist in two VNs without clashing.
When the laptop sends to the printer, edge A checks its map cache. It has no entry, so it sends a map request to the control plane node instead of flooding the whole campus.
The control plane node answers with a map reply: 10.10.1.50 is behind 10.0.250.12. Edge A caches the answer, so later packets go straight out without asking again. On current releases the borders subscribe to the control plane node, so they learn every registration as it happens.
Edge A wraps the packet in VXLAN with edge B's loopback as the outer destination. The underlay only routes between loopbacks and never learns endpoint addresses.
An address the control plane node does not know, such as an internet address, gets a negative reply. Edge A sends that traffic to the border, which acts as the proxy for everything outside the fabric.
When the laptop roams to edge B, edge B registers it again. The control plane node tells edge A, so any edge still sending to the old place is corrected. The laptop keeps its address and its gateway.
Checking the control plane
What I want to know
Command on IOS XE
Did the edge learn the endpoint?
show device-tracking database
Is the edge talking to the control plane node?
show lisp session
What has this edge registered?
show lisp instance-id 4099 ipv4 database
What does the control plane node know?
show lisp instance-id 4099 ipv4 server
What has this edge resolved?
show lisp instance-id 4099 ipv4 map-cache
Data plane
VXLAN carries the packet and the identity
Inside the fabric every packet travels in VXLAN between edge loopbacks. The header is small, but it carries two things the old campus could not: which virtual network the packet belongs to, and which group sent it.
Swipe sideways to see the whole diagram.
An ordinary frame. What the laptop sends: an Ethernet header, an IP header, a TCP header and up to 1,460 bytes of data.
In the fabric. The edge adds 50 bytes: outer Ethernet, outer IP between the two edges' loopbacks, UDP (User Datagram Protocol) on port 4789 and the VXLAN header. SD-Access uses the VXLAN group policy option (VXLAN-GPO), which carries the 24 bit network identifier (VNI) and the 16 bit group tag, so identity travels with every packet.
Why MTU matters. With 50 extra bytes, a full size packet becomes 1,550 bytes. On an underlay link left at 1,500 it is dropped, and the symptom is confusing: pings and small requests work, file copies and TLS (Transport Layer Security) sessions stall. That is why the underlay runs at 9,100.
Animation
A packet walk inside one virtual network
Swipe sideways to see the whole diagram.
Step 1
The laptop sends to its default gateway, 10.10.1.1. Every edge answers for that address with the same MAC address, so the gateway is always one hop away.
Edge A finds the printer in its LISP map cache: it sits behind edge B, whose loopback is 10.0.250.12. If it were not cached, edge A would ask the control plane node first.
Edge A wraps the packet in VXLAN: outer source 10.0.250.11, outer destination 10.0.250.12, the network identifier for VN CORP and the laptop's group tag 10. The intermediate switches route on the outer header only, and spread flows over both paths.
Edge B removes the VXLAN header and reads the source tag. It checks the group policy matrix for staff (10) to printers (30), finds a permit, and delivers the packet. A denied pair would be dropped here, at the destination.
Anycast gateway
Every edge holds the same gateway IP and MAC address for each pool. A device that moves, or a phone that roams between access points, always has its gateway on the switch it is connected to: no first hop redundancy protocol, no hairpin and no new ARP.
Layer 2 and Layer 3 networks
Each pool gets a Layer 2 network identifier for traffic inside the subnet, and each VN gets a Layer 3 identifier for routed traffic. ARP is answered using LISP rather than flooded.
When something needs broadcast
Broadcast is dropped by default. Devices that discover by broadcast, such as BACnet (building automation and control networks) controllers or some printers, need Layer 2 flooding on their pool, which uses underlay multicast. A protocol based fix, such as a BACnet broadcast management device, also works.
Multicast in the fabric
Multicast in the fabric runs one of two ways. With head end replication, the source edge copies each packet to every receiving edge as unicast VXLAN, so the underlay needs nothing extra. With native multicast, the underlay runs Protocol Independent Multicast in source specific mode (PIM SSM) and replicates, which scales far better when many edges have receivers.
Group policy
Rules written by who, not where
In SD-Access the policy is a matrix of groups, not a list of IP addresses. A camera is a camera wherever it is plugged in, and the rules follow it.
Swipe sideways to see the whole diagram.
Step 1
When the laptop authenticates, ISE returns a scalable group tag (SGT) with the access result: 10 for staff. Servers and fixed devices can be tagged statically by subnet, VLAN or port.
The ingress edge writes the tag into the VXLAN header of every packet, so the destination knows who sent it without trusting the source IP address.
Outside the fabric, SXP (SGT Exchange Protocol) shares IP to tag mappings with firewalls and older switches, or links carry the tag inline. The firewall can also learn tags from ISE over pxGrid and write rules by group.
Each edge downloads only the rows of the matrix it needs and enforces them as group ACLs (SGACLs) on traffic for its own endpoints. The ingress edge only classifies, which keeps policy in one place.
Try the group policy matrix
Each row is a source group and each column a destination. Select a cell to see the rule and why I set it that way. This is an example design, not a real customer's policy.
From \ to
Finance app
Video recorder
Building systems
Print server
Internet
Staff
Contractors
Cameras
Building sensors
Printers
Guests
Default rule for any pair not listed: deny. I start from deny and open only what each group needs.
Macro and micro segmentation
Virtual networks keep large groups apart completely: staff, guests, IoT and OT never share a routing table. Group tags then control which groups may talk inside a VN. I use as few VNs as I can, because traffic between VNs has to leave the fabric.
A short matrix
A handful of groups per VN and a matrix that fits on one page. Complexity is what stops a fabric from scaling, not hardware limits. Every cell has an owner and a reason.
Tags are a security control
Whoever can change a device's group can change its access. I restrict who can edit authorisation policy in ISE, and the tag assignments are reviewed like firewall rules.
Onboarding
Getting every device into the right group
Onboarding decides everything that follows: which VN a device joins and which group it carries. I design it with Cisco ISE, and roll it out in phases so nothing important is locked out on day one.
Swipe sideways to see the whole diagram.
Step 1
When the link comes up, the edge switch, as authenticator, asks the device for an identity using EAP (Extensible Authentication Protocol) over LAN. Until the device is authorised, the port only allows what its authentication template permits.
A managed laptop answers with a certificate using EAP-TLS, the strongest method. The switch relays EAP inside RADIUS (Remote Authentication Dial In User Service) to an ISE policy service node. PEAP (Protected EAP) with a password still works, but it is weaker and newer Windows protections break it.
ISE checks who and what the device is, and returns the pool or VLAN for its VN and its group tag. It can add a downloadable ACL or session timers.
A camera or printer cannot do 802.1X. After it times out, MAC authentication bypass (MAB) sends its MAC address, and ISE profiles it from DHCP, LLDP (Link Layer Discovery Protocol), CDP (Cisco Discovery Protocol) and other clues. The result is always tightly scoped, because a MAC address is easy to copy.
When posture or a profile changes, or a security tool flags the device, ISE sends a change of authorisation (CoA) and the switch reauthorises the session: into quarantine, or into a better group.
Existing sessions stay up until they reauthenticate. New devices would fail, so I set critical authentication on the templates: ports fall into a known, limited state rather than going dark. Two policy service nodes per region make this rare.
Authentication templates in Catalyst Center
Authentication template
What it does
When I use it
Closed
802.1X with MAB fallback. Nothing passes until the device is authorised
The target state for every user and device port
Open (monitor mode)
Traffic passes whatever the result, while ISE logs what it would have done
The first weeks, to find every device before enforcing
Low impact
A small set of traffic, such as DHCP and DNS (Domain Name System), passes before authorisation
Sites moving from open to closed
No authentication
A static assignment with no checks
Only for documented exceptions: APs (access points), extended nodes and devices that cannot authenticate
Placing ISE
ISE persona
Job
How I place it
Policy Administration Node (PAN)
Where policy is written and stored
A primary and secondary in different data centres
Monitoring and Troubleshooting (MnT)
Logs, live sessions and reports
A primary and secondary in different data centres
Policy Service Node (PSN)
Answers RADIUS, runs portals and profiling
Close to users, at least two per region. Switches try the local PSN (policy service node) first, then a remote one
pxGrid
Shares context such as user, group and posture with other tools
Enabled where firewalls or security tools consume tags
The same ideas in Aruba ClearPass
Cisco ISE
Aruba ClearPass equivalent
Policy sets and authorisation rules
Services, role mapping and enforcement policies
Authorisation profile: VLAN, dACL, SGT (scalable group tag)
Enforcement profile: VLAN, user role, dACL
Guest portal
Guest module
BYOD (bring your own device) onboarding
OnBoard, with its own certificate authority
Posture
OnGuard
PAN, MnT and PSN nodes
Publisher, standby publisher and subscribers
IoT in phases
I start every IoT device class in open mode with profiling on, build the endpoint groups and the policy, then move one class at a time to closed with MAB. Unknown devices go to a quarantine group with DHCP and DNS only, and raise an alert.
Certificates where possible
Cameras, gateways and industrial switches that support it get a device certificate, so they can use EAP-TLS instead of MAB. The certificate comes from the factory (IEEE 802.1AR) or is enrolled with SCEP (Simple Certificate Enrollment Protocol) or EST (Enrollment over Secure Transport).
Test the failure
Every quarter I test what happens when a PSN, or all of ISE, is unreachable, and check the reconnect storm when it comes back. Sleepy IoT devices need longer timers.
Guests
Guests get a fabric SSID (service set identifier) mapped to a guest VN, with central web authentication on ISE. The detail is on the wireless page.
Between VNs
Crossing between networks, only through a firewall
Virtual networks keep large groups completely apart. When traffic must cross between them, it leaves the fabric at the border and passes a fusion device, usually a firewall, where I can inspect and log it.
Swipe sideways to see the whole diagram.
Step 1
The camera joins VN IOT with group tag 40. Inside that VN it can only reach what the group policy matrix allows. Other VNs do not exist as far as it is concerned.
The recorder is not in VN IOT, so the edge has no LISP entry for it and sends the traffic to the border, still in VXLAN and still tagged.
The border hands each VN to the fusion device on its own VLAN and its own BGP session: VRF lite. Keeping the VNs apart all the way to the firewall is what makes this a security boundary.
The fusion firewall checks the flow. With tags from ISE over pxGrid, the rule can say cameras may reach the recorder on its streaming port and nothing else. It then routes the traffic into the shared services network.
DHCP, DNS, ISE and the recorder must be reachable from every VN. The fusion device leaks their routes into each VN, so every device can get an address while the VNs still never reach each other directly.
Fusion router or firewall?
A fusion router only leaks routes. A fusion firewall leaks routes and enforces policy between VNs. I use a firewall wherever the VNs differ in trust, such as IoT or OT against corporate.
DHCP needs care
The DHCP server sits in shared services, so the relay on each edge's anycast gateway and the route back into the VN must both work. SD-Access also relies on DHCP option 82 so the offer returns to the right edge, and the server must hand it back unchanged.
Keep VNs few
Every new VN is another handoff, another BGP session and another firewall interface. I add a VN only when a group needs a separate routing table, and use group tags for everything else.
Sites and transits
One policy across many buildings
A fabric site is one set of control plane nodes, borders and edges, usually one building or campus, and a failure domain I am happy to accept. Sites are joined by a transit, and managed together as a fabric domain with shared VNs and policy.
Swipe sideways to see the whole diagram.
IP transit. Each border hands every VN off as VRF lite with BGP, and the WAN carries a VRF per VN. The group tag travels inline where the WAN supports it, or by SXP. It works over any WAN, which is why it is the most common choice.
SD-Access transit. LISP and VXLAN run end to end between sites, with transit control plane nodes holding the summary of every site. Policy and tags stay native, but the WAN must be private, fast and able to carry the larger MTU, such as campus fibre or metro Ethernet.
With SD-WAN. At the border, each VN becomes an SD-WAN service VPN, so segmentation runs from the campus across the WAN to other sites and the cloud. The group tag can be carried in the SD-WAN overlay too. More on the SD-WAN page.
Animation
Size each site
I keep each site inside the scale limits published for the release: endpoints per control plane node, edges per site, and VNs. Several smaller sites joined by a transit are easier to run than one huge one.
Plan the pools
Separate pools for each device class, sized for the end state, often a /22 or /21 where a /24 looks enough today, with IPv6 (Internet Protocol version 6) on the roadmap. Pools are hard to change once devices live in them.
Size the services too
Catalyst Center runs as a three node cluster for resilience. ISE policy service nodes are sized for the RADIUS rate at peak, not the device count, because sleepy IoT devices send a new MAB request every time they reconnect.
Wireless in the fabric
Fabric SSIDs map to pools and groups, access points tunnel control to the controller, and client data joins the fabric at the edge. I plan 2.4 GHz capacity for IoT separately from staff Wi-Fi, with identity PSK (iPSK) for devices that cannot do 802.1X.
Operations
Running the fabric from one place
Catalyst Center is where the fabric is designed, built and watched. I follow its four stages in order, and I plan the migration from the old campus as carefully as the fabric itself.
Stage
What I set up in Catalyst Center
Design
Site hierarchy of areas, buildings and floors. Network settings for AAA (authentication, authorisation and accounting) with ISE, DHCP, DNS, NTP (Network Time Protocol), syslog and SNMP (Simple Network Management Protocol). Credentials, IP pools, wireless profiles and templates.
Policy
Virtual networks, scalable groups from ISE, the group based access matrix and application QoS (quality of service) policy.
Provision
Discover or Plug and Play the switches, assign them to sites, run LAN Automation, create the fabric site, give each switch its role and set host onboarding per port.
Assurance
Health scores for devices, clients and applications from streaming telemetry, Client 360 with the onboarding timeline of each client, path trace across the fabric, Endpoint Analytics feeding device profiles to ISE, and issues raised against learned baselines.
How I migrate a live campus onto SD-Access
Find out how devices really talk. Before moving anything I list each device class and whether it needs broadcast, multicast, a static address or a hard coded gateway.
Build the fabric alongside. The new fabric runs in parallel, joined to the old network at the border with a Layer 3 handoff, plus a Layer 2 border for VLANs that must stay stretched during the move.
Move by device class and closet. Not building by building. Each move has a rollback, which for a closet can be as simple as patching back.
Open first, enforce later. Authentication starts in monitor mode and moves to closed mode once the profiling and policy are proven.
Cut the old gateway last. The old gateway stays until the last device in that subnet has moved, then the Layer 2 handoff is removed and recorded as closed.
Agree windows with OT. Plant and building systems move only in agreed windows, with device by device checks for anything that must not fail.
When a device misbehaves
Detect. An Assurance anomaly, a firewall or IDS (intrusion detection system) alert, Cyber Vision or Secure Network Analytics. IoT networks are quiet, so unexpected traffic out is a strong signal.
Contain. ISE Adaptive Network Control quarantines the endpoint, and a CoA moves it into a quarantine group at once. A firewall or SIEM (security information and event management) can trigger this automatically over pxGrid.
Investigate. Path trace, flow records, and the device's firmware version and certificate.
Recover. Rebuild or replace the device, rotate its credentials or certificate, then release it from quarantine. If the traffic was legitimate, I fix the profile or the policy instead.
Learn. Update the runbook, and tighten the group policy or move the device class behind a gateway.
Troubleshooting
Finding a fabric fault plane by plane
I troubleshoot a fabric the same way every time: is the underlay healthy, has the edge learned and registered the device, does the policy allow it, and does the path out of the fabric work? Choose a symptom to see where it usually fails. Catalyst Center Assurance shows each client's onboarding timeline, which often shows the cause before I log in to a switch.
Swipe sideways to see the whole diagram.
Authorised, but no IP address. The session is up, so I check the pool: the returned VLAN name must exist on this edge for this VN. Then DHCP: the relay on the anycast gateway, the route from the DHCP server back into the VN through the fusion device, and option 82 handling on the server. show device-tracking database shows whether the switch saw any address at all.
Wrong access.show authentication sessions interface Gi1/0/12 details shows the method, the result and the group. In ISE live logs I check which rule matched. Usually the device was profiled as something else, or matched a rule above the one intended. Fix the profile or rule, then send a CoA.
After an ISE outage. Existing sessions survived, but every port that reauthenticated failed. The fix is design: two policy service nodes per region with dead server detection, critical authentication on the templates, and timers long enough that a short outage is invisible. show aaa servers shows the RADIUS state.
Discovery fails. Some building and IoT devices find each other by broadcast, which the fabric drops by default, and devices in different pools are in different broadcast domains anyway. I enable Layer 2 flooding on that pool, which needs underlay multicast, or use a protocol fix such as a BACnet broadcast management device.
Reaches nothing. The edge learned the device but the control plane node does not know it. I check show lisp session on the edge, then show lisp instance-id 4099 ipv4 server on the control plane node. No registrations usually means the edge cannot reach the control plane node across the underlay.
One pair blocked.show cts role-based counters shows drops for that source and destination group. I check the matrix cell in ISE and that the edge has downloaded the current policy with show cts role-based permissions. Catalyst Center path trace shows the hit too.
Cannot reach another VN. Traffic between VNs leaves through the border to the fusion device. I check the border's BGP session for that VN, then the routes on the fusion device: show ip route vrf IOT. A missing leak or a firewall rule is the usual cause.
Animation
Commands by check
What I want to know
Command on the edge or border
Is the device authenticated, and with what result?
show authentication sessions interface Gi1/0/12 details, or show access-session on IBNS (Identity Based Networking Services) 2.0
Did the switch learn its address?
show device-tracking database
Is the edge registered with the control plane node?
show lisp session, show lisp instance-id 4099 ipv4 database
Which policy is downloaded, and what is hitting it?
show cts environment-data, show cts role-based permissions, show cts role-based counters
Is the underlay healthy?
show isis neighbors, ping 10.0.250.12 source Loopback0 size 1550 df-bit
Is the border handing off this VN?
show ip bgp vpnv4 vrf IOT summary, show ip route vrf IOT
Is power the problem?
show power inline, show interfaces status
Engagement
How I run an SD-Access programme, from discovery to steady running
A campus fabric changes how identity, policy and operations work, not just the switches. I run it as a programme with the security, desktop and facilities teams, because they own most of the devices that will join it.
Swipe sideways to see the whole diagram.
Step 1
I start with what the business needs from segmentation: containing threats, onboarding IoT safely, or meeting an audit finding. I agree who owns ISE, who owns the device groups and who decides on policy (a RACI: responsible, accountable, consulted, informed).
Functional requirements become virtual networks, groups and a draft policy matrix. Non functional ones set the scale per site, the recovery time when a node fails, what happens when ISE is unreachable, and how quickly a new device must be onboarded.
I build an inventory of every device class from switch tables, ISE profiling in monitor mode and walk rounds. For each class I record how it gets an address, whether it needs broadcast or multicast, and who owns it. Surprises here are what derail fabric rollouts.
The high level design (HLD) covers fabric sites, transits, roles, VNs and the fusion design. The low level design (LLD) covers pools, handoff VLANs, BGP, ISE personas, templates and the full matrix. Both go to the design authority.
I prove the design in a lab, including node, ISE and transit failures, then pilot one closet in open authentication. Device owners sign off each class before enforcement, and the rollout moves closet by closet with a rollback for each.
Operations get Catalyst Center Assurance, runbooks for the common onboarding faults and a clear owner for policy changes. Service desk staff learn to read the onboarding timeline, which answers most "my device will not connect" calls on its own.
Catalyst Center, switch software and ISE must stay on versions that are supported together. I plan upgrades in waves against the compatibility matrix, review the policy matrix every quarter and retire groups nobody uses.
Functional requirements
What it must do.
Authenticate every user and device port, with 802.1X where possible and MAB with profiling where not.
Keep staff, guests, IoT, building systems and OT in separate virtual networks.
Control access between groups inside each network with a policy matrix.
Inspect every flow between virtual networks at a firewall.
Give wired and wireless users the same policy wherever they connect.
Onboard new switches and access points automatically.
Non functional requirements
How well it must do it, with a number wherever one can be agreed.
Availability
No single node failure takes down a site: two control plane nodes and two borders per site.
Recovery
Traffic moves to the surviving border or path within seconds of a failure.
Resilience to ISE
Ports fall into a known critical state if ISE is unreachable, with two policy service nodes per region.
Scale
Each site within the published limits for endpoints, edges and VNs, with 30% headroom.
Onboarding
A new device class can be profiled, grouped and enforced within an agreed number of days.
Operability
Health, onboarding and policy hits visible in Catalyst Center, and every policy change logged.
These are typical starting targets for a fabric site. Each organisation sets its own, and each number is traced to a failover or onboarding test in the pilot.
Who I talk to
Network, security and identity teams
Desktop, print and telephony teams
Facilities, building management and OT owners
Application owners for systems that devices talk to
The service desk, who will take the calls
How I find out
ISE in monitor mode with profiling on, to see every device before enforcing
Walk rounds of closets, plant rooms and ceilings
Reading switch MAC, ARP and DHCP tables per VLAN
Reviewing current ACLs and firewall rules between VLANs
Workshops per device class with its owner
Discovery tools
Tool or source
What it tells me
Cisco ISE profiling and endpoint analytics
What each device is, and how sure the profile is.
Catalyst Center discovery and inventory
Switch models, software and licences, and whether they can join the fabric.
show mac address-table, show ip arp, show ip dhcp snooping binding
Which devices sit where, and how they get addresses.
NetFlow or Secure Network Analytics
Who really talks to whom, to seed the policy matrix.
Cyber Vision or another passive OT sensor
Industrial devices and protocols, without probing them.
Packet captures on a mirror port
Broadcast and multicast discovery that the fabric would drop.
I troubleshoot a fabric plane by plane, and I always start with the onboarding timeline in Catalyst Center, because most faults are authentication, addressing or policy rather than hardware.
My approach
Scope. One device, one class, one closet or a whole site? A whole site points at the underlay, ISE or a border.
Port and authentication. Link, power, the authentication session and the ISE live log for that MAC address.
Address. The pool and VLAN returned, DHCP relay and the route back from the DHCP server.
Control plane. Device tracking, LISP registration on the edge and on the control plane node.
Policy and path. SGACL (scalable group ACL) counters, the downloaded matrix, and the path out through the border and fusion device.
Fix and record. Fix through change, CoA the session if needed, and add the cause to the known errors so the service desk recognises it next time.
Faults I see most
A device profiled as the wrong class, so it lands in the wrong group
No DHCP offer because the route back into the VN is missing
Broadcast discovery that the fabric drops by default
An edge that cannot reach the control plane node after an underlay change
A missing route leak at the fusion device
Underlay MTU left at 1,500 on one link
Layer by layer
Layer
What I check
1 · Physical
Link, optics and PoE (Power over Ethernet): show interfaces status, show power inline.
2 · Data link
Authentication and the VLAN returned: show authentication sessions interface Gi1/0/12 details.
3 · Network
Underlay and overlay: show isis neighbors, show lisp session, show ip route vrf IOT.
4 · Transport
VXLAN MTU between loopbacks: ping 10.0.250.12 source Loopback0 size 1550 df-bit.
5 to 7 · Application
Group policy and the application: show cts role-based counters, DHCP and DNS responses.
A fabric is run by more than the network team, so I agree the support model with identity, desktop and facilities teams before go live.
Service model
Who owns the fabric, ISE, the policy matrix and each device class, with escalation routes between them.
Assurance
Catalyst Center health scores, onboarding timelines and issues, with alerts tuned during hypercare.
Policy changes
A light change process for matrix and group changes, with the owner of each group approving.
Runbooks
Onboarding a new device class, replacing an edge, an ISE outage, and a device stuck in quarantine.
Hypercare
Extra cover after each wave, ending when unknown devices are near zero and the alert noise is under control.
Problem management
Repeat onboarding failures traced to a profiling, template or DHCP root cause and fixed once.
Catalyst Center, switch software and ISE move on their own release cycles, and they must stay compatible. I plan their life together.
Release plan. Target releases for Catalyst Center, switches and ISE, checked against Cisco's compatibility matrix.
Upgrade waves. Lab first, then a pilot site, then waves by site, with Catalyst Center's image management doing the work.
Patch. Cisco PSIRT (Product Security Incident Response Team) advisories for IOS XE, Catalyst Center and ISE triaged by CVSS (Common Vulnerability Scoring System) score and exposure. Critical fixes go through emergency change, the rest into the next upgrade wave.
Policy review. A quarterly review of the matrix and groups, removing rules and groups nobody uses.
Capacity. Endpoints, pools and node limits checked each quarter, so a site never reaches its ceiling unplanned.
Hardware. An end of life register for switches, controllers and appliances, with replacements planned before support ends.
Projects
Where I have designed and built it
SD-Access has been part of my campus work for years, usually alongside Catalyst SD-WAN, with ISE behind both.
AVEVA · 2020 to 2021
A global LAN and WAN rebuild
Designed and built LAN and WAN on Catalyst SD-WAN, SD-Access and Catalyst Center through the reverse merger, modernising every site worldwide.
Vodafone · 2024
Fabric planning at scale
Shaped SD-WAN, SD-Access and Catalyst Center infrastructure planning so backbone, data centre and edge environments could carry high throughput, low latency workloads.
XIDA · 2021 to 2024
Fabric designs for clients
Produced multi vendor SD-WAN and SD-Access designs for enterprise clients.
XIDA · 2025 to now
Discovery to provisioning
Led network and security transformation from discovery to provisioning across Cisco Nexus, Catalyst SD-WAN, SD-Access and AWS (Amazon Web Services), Azure and Google Cloud environments.
iSystems Integration · three engagements, 2015 to 2019
Architecture and delivery
Directed architecture and delivery across cloud, disaster recovery, Catalyst SD-WAN, SD-Access and security, and led and mentored a team of 4 engineers.
Home lab
Cisco ISE and wireless
My lab runs Cisco ISE for network access control and a virtual wireless controller on its ESXi hosts, which I use to test authentication and onboarding policy. See the lab.
Backed by CCNP (Cisco Certified Network Professional) Enterprise and the Cisco Certified Specialist exams in Enterprise Core, Enterprise Design and Enterprise Advanced Infrastructure Implementation. See all credentials
Explore
More areas I design and deliver
Every page follows the same idea: how it works, the choices that matter, and where I have used it. The foundations pages explain the basics the others build on.