Why I put one in front
More capacity by adding servers, no outage when one fails, and zero downtime patching: drain a member, patch it, return it. The same box can also end TLS and apply a web application firewall.
I design application delivery on F5 BIG-IP and the public cloud load balancers. This page follows a request from the DNS (Domain Name System) answer to the server and back. On the way it covers methods, persistence, address translation, TLS (Transport Layer Security), security and failover, then shows how I find the fault when it breaks.
A load balancer gives an application one stable address and spreads users across a pool of servers, removing any that fail their health checks. The examples use F5 BIG-IP, and the same ideas apply to cloud load balancers.
Swipe sideways to see the whole diagram.
Users connect to a virtual server, often called the VIP (virtual IP address): an address and port, here 203.0.113.10 on 443, that DNS points the application name at. Nobody needs to know how many servers sit behind it.
The virtual server hands each new connection to its pool. The pool picks a member, an IP address and port, using its load balancing method. Round robin is the default: one each in turn.
A health monitor tests every member on a schedule. I use an HTTP (Hypertext Transfer Protocol) monitor that requests a real health page and expects a specific answer, not just an open port, so a broken application is caught too.
When web2 stops giving the right answer for the timeout, 16 seconds with F5's HTTP defaults, it is marked down. New connections go only to web1 and web3. Existing connections to web2 stay until they close or time out, unless I set the pool's action on service down to reject, so clients reconnect to a healthy member at once.
When web2 passes its monitor again it rejoins the pool. For a slow starting application I add a slow ramp time, so it is not flooded the moment it returns.
More capacity by adding servers, no outage when one fails, and zero downtime patching: drain a member, patch it, return it. The same box can also end TLS and apply a web application firewall.
A virtual server (the address users hit) uses profiles (TCP, HTTP, TLS) and sends to a pool. The pool holds members, each a node address and a port, and monitors decide which members are up.
A monitor should prove the application works, for example by requesting a health page that checks the database, and the receive string should match something only a healthy page returns.
| Monitor | What it proves | Note |
|---|---|---|
| Gateway ICMP (Internet Control Message Protocol), or a node monitor | The host answers | Says nothing about the application |
| TCP (Transmission Control Protocol) | The port accepts a connection | A hung application still passes |
| HTTP or HTTPS (HTTP over TLS) with send and receive strings | The page and what it depends on work | The timeout is three times the interval plus one: 5 and 16 seconds by default |
| Receive disable string | The server asks to be drained | The member stops taking new users but keeps persisted ones |
| External or inband | Custom checks, or failures in real traffic | Inband reacts to live errors with no extra probes |
A pool can also require at least two of several monitors to pass before a member counts as up.
The first design choice is how deep the load balancer looks. At Layer 4 it only sees addresses and ports. At Layer 7 it understands the application request and can make far smarter decisions.
Swipe sideways to see the whole diagram.
Layer 4. The load balancer decides on the IP address and port alone, then forwards packets, rewriting addresses on the way. On F5 this is a performance virtual server with a FastL4 profile. It is fast and works for any TCP or UDP (User Datagram Protocol) application, but it cannot see URLs (web addresses), cookies or anything inside TLS.
Layer 7. The load balancer is a full proxy: it ends the client's connection, reads the HTTP request, and opens its own connection to a member. That lets it route by host name or URL, insert cookies, rewrite headers, offload TLS and run a web application firewall. It costs more processing per connection.
The method decides which member gets each new connection. The right one depends on how similar the servers are, how long requests take, and whether the application keeps state.
Swipe sideways to see the whole diagram.
Round robin. The default: each new connection goes to the next member. Simple and even when members are the same size and requests take about the same time.
Ratio, or weighted. Each member has a weight, so a bigger server gets a bigger share. I use it during migrations to send a small slice of traffic to new servers first.
Least connections. New work goes to the member with the fewest open connections. It adapts when one server is slow or requests vary in length, which is why it is my usual choice for applications.
Source address hash. The client's address picks the member, so a client keeps reaching the same server without any state on the load balancer. Behind a large NAT (network address translation), a whole office looks like one client and lands on one server.
Server 2 can be made slow, so its connections last three times longer. Requests come from five offices, and the first office sits behind one NAT address and sends half of all traffic.
Round robin shares requests evenly, but the slow server ends up holding the most open connections.
| Method | How it chooses | Where I use it |
|---|---|---|
| Round robin | Next member in turn | Identical servers with short, similar requests |
| Ratio | In proportion to a weight per member | Mixed server sizes, or sending a slice of traffic to new servers |
| Least connections | Fewest open connections | Most applications, especially with long or uneven requests |
| Fastest or observed | Quickest response, or connections weighted over time | Members whose performance varies |
| Source address hash | A hash of the client address | Simple stickiness without state. On BIG-IP I get it from source address affinity or hash persistence, not a pool method |
| Global availability | First healthy site in a list | Active and standby data centres, in global DNS balancing |
Some applications keep a user's session on the server that started it. Persistence, sometimes called stickiness, sends that user back to the same member. The best fix is a stateless application with shared session storage, but persistence is how most estates run today.
Swipe sideways to see the whole diagram.
No persistence. Each request is balanced on its own. That is fine for stateless applications, but if the server keeps the session in memory, such as a shopping basket, the second request lands elsewhere and the user is logged out.
Source address affinity. Requests from the same client address go to the same member until a timeout expires. It works for any protocol, but everyone behind one NAT address, such as a whole office, sticks to one server.
Cookie insert. On the first reply the load balancer adds a cookie that encodes the chosen member (on F5 it is named after the pool). The browser sends it back, so the user returns to the same server whatever their address. It needs HTTP, so with HTTPS the load balancer must decrypt.
SSL (Secure Sockets Layer) session ID persistence for TLS 1.2 that is not decrypted, universal persistence on any value an iRule extracts, such as a header, and destination address affinity for caches.
Once a user is persisted, the load balancing method is skipped for them. Long persistence timeouts can leave servers uneven, especially after one comes back from maintenance.
Disabling a member stops new users but still accepts persisted ones. Forcing it offline lets existing connections finish but refuses every new one, even from persisted users. I disable, wait for sessions to end, then patch.
A load balancer changes addresses in both directions, so the reply has to pass back through it. When it does not, the connection breaks in a way that looks like an application fault. This is the first thing I check in any new design.
Swipe sideways to see the whole diagram.
Two arm, routed. The load balancer sits inline and its floating self IP is the servers' default gateway. It only changes the destination address, so servers see the real client address and replies naturally return through it.
One arm without SNAT. The load balancer hangs off one network and only changes the destination. The server sees the client's real address and replies through its own gateway, straight to the client. The client gets a reply from 10.1.1.11 for a connection it opened to 10.1.0.10, and drops it. This is the classic "works direct, fails through the VIP" fault.
One arm with SNAT. Source address translation replaces the client address with one of the load balancer's, so the server always replies to the load balancer. On F5, SNAT (source network address translation) Automap uses the self IP, the floating one in a pair. The server loses the client's address, so I pass it in the X-Forwarded-For header and log it there.
One source address gives about 64,000 ports to each server. Very busy virtual servers need a SNAT pool with several addresses, or port exhaustion appears as random connection failures.
With SNAT, I insert X-Forwarded-For on HTTP and teach the application and its logs to read it. For non HTTP traffic, the routed two arm design keeps the real address.
If a stateful firewall sits between the load balancer and the servers, asymmetric return paths are dropped there too. Return symmetry is a design rule for both.
The load balancer forwards only the requests. Servers hold the VIP on a loopback and reply straight to clients. It suits very heavy outbound traffic such as video, but the load balancer sees half the flow, so there is no Layer 7 routing, TLS offload or SNAT.
Most applications are HTTPS, so the load balancer either ends TLS, re encrypts it, or passes it through. The choice decides what the load balancer can see, and therefore what it can do.
Swipe sideways to see the whole diagram.
TLS offload. The certificate and key live on the load balancer, which decrypts and sends plain HTTP to the servers. It saves server CPU (central processing unit) and lets the load balancer route by URL, insert cookies and run a web application firewall. Only acceptable where the server network is trusted.
Re encryption, also called TLS bridging. The load balancer decrypts with a client side profile, inspects, then encrypts again with a server side profile. Every hop is encrypted, which most security policies now ask for. This is my default.
Pass through. The load balancer forwards TLS untouched at Layer 4. The servers hold the certificates and nothing in the middle can see the traffic, so no URL routing, cookie persistence or WAF (web application firewall). Persistence falls back to source address. SSL session ID persistence only works with TLS 1.2 and earlier, because TLS 1.3 does not reuse session IDs.
TLS 1.2 and 1.3 only, strong cipher suites, the full certificate chain on the profile, and renewal dates in monitoring. An expired certificate on a busy virtual server is an outage with a very public symptom.
One virtual server can hold several certificates and pick one by the server name the browser asks for, called SNI (Server Name Indication). I keep a default certificate for clients that do not send it.
An iRule is a small script, in the Tcl language, that runs on events such as each HTTP request. It can route, rewrite or block. I prefer built in policies where they can do the job, because every iRule is code to maintain.
A small iRule that routes API (application programming interface) calls to their own pool
when HTTP_REQUEST {
# send API calls to their own pool, everything else to the web pool
if { [HTTP::uri] starts_with "/api/" } {
pool pool_api
} else {
pool pool_web
}
}Local load balancing spreads users across servers in one site. Global server load balancing (GSLB) chooses the site, using DNS. LTM (Local Traffic Manager) balances servers, BIG-IP DNS balances sites.
Swipe sideways to see the whole diagram.
The user's device asks its DNS resolver, at the internet provider or the company, for app.example.com.
The resolver asks BIG-IP DNS, formerly GTM (Global Traffic Manager), which is authoritative for that name, called a wide IP. It answers with an address, not a fixed record.
BIG-IP DNS learns the health of each site's virtual servers from the local load balancers over iQuery. With both healthy, the method, here topology, sends a user near London to London.
The user connects to the London virtual server, and the local load balancer spreads the work across its servers. The global layer only chooses the site.
When London is down, the next DNS answer is Manchester. Users whose resolver still holds the old answer wait for the time to live (TTL) to expire, so I keep it short, around 30 seconds, and test that clients honour it.
| Method | What it does | When I use it |
|---|---|---|
| Global availability | Always the first healthy site in an ordered list | Active and standby data centres |
| Round robin or ratio | Shares answers across sites | Active and active sites of similar size |
| Topology | Answers by where the user or resolver is | Sending users to their nearest region |
| Round trip time | Answers with the site closest in network terms | Global applications sensitive to latency |
Because the load balancer already decrypts and reads every request, it is a natural place for a web application firewall. On F5 this is BIG-IP Advanced WAF, which grew out of ASM (Application Security Manager).
Swipe sideways to see the whole diagram.
Clean traffic. The web application firewall (WAF) policy sits on the virtual server. It checks each request against attack signatures and against what the application should accept: its URLs, parameters, file types and lengths. A clean request passes to the pool as normal.
An attack. A parameter carries SQL (Structured Query Language) injection, which matches a signature, so the WAF blocks it and returns a page with a support ID. I start a new policy in transparent mode, which logs without blocking, then tune out false positives before I switch it to blocking.
A bot. A script sends requests far faster than a person and cannot run JavaScript. Bot defence challenges it, then blocks or rate limits it. Layer 7 denial of service protection works the same way, watching transactions per second and server latency.
The policy covers the web attacks in the OWASP (Open Worldwide Application Security Project) Top 10: injection (which now includes cross site scripting), broken access control and the rest, with signatures kept up to date.
The policy builder learns the application from real traffic and suggests what to allow. I review each suggestion, stage new signatures before enforcing them, and log every exception with a reason.
An API gets its own policy built from its OpenAPI specification, so only the documented paths, methods and fields are accepted. Anything else is blocked and logged.
The same protection comes as a service: AWS (Amazon Web Services) WAF, Azure Web Application Firewall on Application Gateway or Front Door, Google Cloud Armor and F5 Distributed Cloud. I choose by where the application lives and who runs it.
A load balancer in front of every application must not become the single point of failure. BIG-IP runs as a pair, or a larger cluster, and moves the addresses between units when one fails.
Swipe sideways to see the whole diagram.
Two units trust each other and form a sync failover device group. Traffic group 1 holds the virtual addresses and the floating self IP. It is active on unit A, so A carries the traffic.
The units exchange network failover heartbeats on UDP port 1026, over more than one path. Config sync keeps their configuration identical, and connection mirroring can copy live connections to the standby.
Unit A loses power. Unit B stops hearing heartbeats and, after a short timeout, decides its peer has gone. A failed VLAN (virtual local area network) check, a lost gateway or an HA (high availability) group score can trigger the same move.
Unit B makes traffic group 1 active and sends gratuitous ARP (Address Resolution Protocol) for each address, so the router and the servers update their ARP caches. MAC (media access control) masquerade keeps the MAC address the same and removes the wait.
Traffic now flows through unit B. Mirrored connections carry on, and the rest reconnect. When A returns it stays standby unless I fail back on purpose, which avoids a second interruption.
Each traffic group can be active on a different unit, so both units carry live traffic. I size each unit to carry everything alone, otherwise a failover becomes an outage.
I send heartbeats over the HA VLAN and the management network. One failed link then cannot make both units go active, the split brain that takes everything down.
VLAN and gateway failsafes catch failures that heartbeats cannot see. An HA group scores each unit on its trunks and pools, so the healthier unit takes the traffic.
Connection mirroring keeps long sessions alive through a failover, at a cost in CPU and HA link traffic. I mirror only the virtual servers that need it, such as long running database connections.
Every public cloud splits its load balancers the same way: Layer 7 for HTTP, Layer 4 for everything else, and DNS or anycast for global entry. This is how they line up when I move an application. More on each platform is on the cloud platforms page.
| Need | AWS | Microsoft Azure | Google Cloud |
|---|---|---|---|
| HTTP and HTTPS at Layer 7 | Application Load Balancer (ALB) | Application Gateway | Application Load Balancer |
| TCP and UDP at Layer 4 | Network Load Balancer (NLB) | Load Balancer | Network Load Balancer, proxy or passthrough |
| Global HTTP entry with edge security | CloudFront with AWS WAF | Front Door | global external Application Load Balancer with Cloud Armor |
| Global choice by DNS | Route 53 routing policies | Traffic Manager | Cloud DNS routing policies |
| Global anycast entry for TCP and UDP | Global Accelerator | Load Balancer, global tier | global external proxy Network Load Balancer |
| Firewalls inserted inline | Gateway Load Balancer (GWLB) | Gateway Load Balancer | internal passthrough Network Load Balancer as the next hop |
| Private, inside the network | internal ALB or NLB | internal Load Balancer or Application Gateway | internal Application or Network Load Balancer |
For a new cloud application I start with the native load balancer. It scales on its own, is billed by use, and is built from the same code as the rest of the platform. I add BIG-IP where a requirement needs it.
BIG-IP Virtual Edition runs in all three clouds when an application needs iRules or the same WAF policy as on premises. Cloud networks do not honour gratuitous ARP, so failover moves addresses and routes through the cloud API instead.
Cloud health checks come from known sources, such as Azure's 168.63.129.16 or Google Cloud's published ranges. If a security group or firewall rule blocks them, every member shows as down.
In Kubernetes, an ingress controller or the Gateway API does the Layer 7 work inside the cluster, with a cloud or BIG-IP load balancer in front of it.
When an application behind a load balancer fails, the load balancer is blamed first. Often it is right, sometimes it is the application. I follow the connection through each stage in order and let the evidence decide. Choose a symptom to see where it fails and what I run. My general method is on the troubleshooting page.
Swipe sideways to see the whole diagram.
Some users reach the wrong site. The name resolves to a site that is down or far away. I ask BIG-IP DNS directly and compare with what the user's resolver returns. A resolver holding an old answer past its TTL, a topology record that places users in the wrong region, or a pool member marked up when it is not are the usual causes.
Nothing answers on the VIP. If the counters stay at zero, traffic is not reaching the box: a missing route upstream, a firewall in front, ARP turned off on the virtual address, or the virtual server enabled on the wrong VLAN. If the counters rise but clients get a reset, the virtual server is answering and the fault is further along.
The handshake fails. I check the certificate chain and dates the client sees, whether the client sent SNI and got the right certificate, and whether the client SSL profile allows its TLS version and ciphers. An expired certificate or a missing intermediate is the usual cause. With re encryption, I run the same test against a member, because the server side can fail too.
Every connection is reset. All pool members are marked down, so the virtual server has nowhere to send traffic. I run the monitor's request by hand from the BIG-IP to a member, for example with curl, and compare the reply with the receive string. A changed health page, a server firewall blocking the self IP, or no route back to the self IP are the usual causes.
One server does most of the work. Usually persistence: a long timeout, or source address affinity with a whole office behind one NAT address. Otherwise a ratio left over from a migration, or a member back from maintenance with no slow ramp time. I check the method, ratios and persistence records before blaming the servers.
Connections hang after the SYN. The member gets the SYN (synchronise, the first packet of a TCP connection), but its reply goes to its default gateway and around the load balancer, so the client drops it. In a one arm design this is the classic fault. I turn on SNAT with Automap, so replies come back to the self IP, or make the floating self IP the servers' gateway.
Users are logged out at random. Requests from one user land on different members. I check that a persistence profile is attached, that its timeout outlasts the application session, and for cookies, that the load balancer can see HTTP, which means TLS must end there.
| What I want to know | Command |
|---|---|
| Is the virtual server up, and are its counters moving? | tmsh show ltm virtual vs_app |
| Which members are up, and why not? | tmsh show ltm pool pool_web members |
| What is the traffic doing on the wire? | tcpdump -ni 0.0:nnn host 10.1.1.11, where :nnn adds the flow detail F5 support asks for |
| Who is persisted where? | tmsh show ltm persistence persist-records |
| Which connections does one client have? | tmsh show sys connection cs-client-addr 198.51.100.7 |
| Which unit is active, and are they in sync? | tmsh show sys failover, tmsh show cm sync-status |
| What happened, and when? | tail -f /var/log/ltm |
| Is the certificate right? | openssl s_client -connect app.example.com:443 -servername app.example.com |
A load balancer sits in front of the applications the business notices first. I treat each change as a small programme with the application owners, because they know how their service really behaves.
Swipe sideways to see the whole diagram.
I start with why the change is happening: a platform at end of life, a move to cloud, a new security requirement or an outage. I agree who owns each application and who decides on risk (a RACI: responsible, accountable, consulted, informed).
Functional requirements become virtual servers, pools, monitors, persistence and TLS settings. Non functional ones set availability, failover time, connections per second, certificate standards and how quickly a member must be removed when it fails.
I export the configuration and build an inventory of every virtual server, pool, monitor, iRule and certificate, then match each one to an owner. Unowned virtual servers with no traffic are retired, not migrated. Statistics show the real peaks.
The high level design (HLD) covers the platform, placement, HA and global balancing. The low level design (LLD) sets standard patterns per application type, so most services fit a template and only a few need an iRule.
I build the patterns in a lab, then fail each unit, member and link on purpose. Applications move one at a time, usually by changing DNS to the new VIP with a short TTL, so rollback is a DNS change too.
Operations get dashboards for pool health, connections and certificates, and runbooks for the faults on this page. Each application has a named owner who approves changes to its virtual server, policy and certificates.
Load balancer software, signatures, certificates and licences all have their own cycles. I plan them together, upgrade the standby unit first, and review capacity each quarter against connections, throughput and TLS transactions.
What it must do.
How well it must do it, with a number wherever one can be agreed.
These are typical starting targets for a load balanced service. Each application owner agrees the numbers for their own service, and each one is traced to a failover or load test.
| Tool or source | What it tells me |
|---|---|
tmsh list ltm virtual and a UCS (user configuration set) archive | The full configuration, read safely offline. |
tmsh show ltm virtual and the dashboard | Live connections and throughput per virtual server. |
| iHealth | F5's analysis of a diagnostics file, with known issues and advisories. |
openssl s_client and certificate scanners | What clients really see: chains, dates and ciphers. |
| Flow data and packet captures | Who uses each service, and how long their sessions last. |
| Application monitoring | Response times seen by users, to compare before and after. |
I troubleshoot an application behind a load balancer stage by stage, and I always test a member directly first, because the load balancer is often blamed for an application fault.
| Layer | What I check |
|---|---|
| 1 · Physical | Interfaces, trunks and optics: tmsh show net interface. |
| 2 · Data link | VLANs, ARP and gratuitous ARP after failover: tmsh show net arp. |
| 3 · Network | Self IPs, routes and SNAT: tmsh show net route, tmsh list ltm snatpool. |
| 4 · Transport | Connections and resets: tmsh show sys connection and a capture. |
| 5 to 7 · Application | TLS, HTTP and persistence: openssl s_client, HTTP status codes and persistence records. |
The fault finder is further up this page, and my general method is on the troubleshooting page.
A load balancer is shared by many applications, so I agree the support model with every owner before go live.
Who owns the platform, each virtual server, each certificate and each WAF policy, with escalation routes between them.
Pool and member health, connections, throughput, failover events and certificate expiry, with alerts tuned during hypercare.
Standard changes for adding a pool member or renewing a certificate, and full review for new iRules and policies.
A member down, a failover, an expired certificate, a WAF false positive and a site failover.
Extra cover after each migration wave, ending when incidents and WAF exceptions are back to normal.
Repeat faults traced to a root cause, such as a monitor design or a session timeout, and fixed once.
Platform software, attack signatures, certificates and hardware all move on their own cycles. I plan their life together.
Load balancing has come with the data centre, security and disaster recovery work in my career. F5 is named in my iSystems Integration architecture work. The other programmes below are data centre moves and refreshes where the applications had to keep running throughout.
Cloud, software defined networking (SDN), disaster recovery and security architecture on Nexus, Viptela, Palo Alto, F5 and Aruba for the firm's enterprise clients. I returned for cloud and security architecture, then network transformation.
Safeguarded over £200M of transactional services through data centre transitions, holding 99.99% uptime throughout, while leading network and security transformation across every site.
Audit, design and implementation across LAN (local area network), WAN (wide area network), SD-WAN (software defined wide area network) and data centre infrastructure, replacing end of life equipment with minimal impact on global operations.
Lead architect on network, cloud and data centre modernisation programmes, from discovery to operational handover, across several client environments.
F5 sits in my core security skills alongside Palo Alto, Check Point and Cisco Firepower. Backed by AWS Certified Solutions Architect Professional, CCNP (Cisco Certified Network Professional) Enterprise and the CCIE (Cisco Certified Internetwork Expert) Security written exam. See all credentials
Every page follows the same idea: how it works, the choices that matter, and where I have used it. The foundations pages explain the basics the others build on.