What to Do During a DDoS Attack: A Runbook for Server Teams
What to do during a DDoS attack, in order: confirm it is an attack and not a traffic spike or a bug, identify the layer being hit, protect what you can with rate limits and caching, contact your provider with the target IP, start time and samples, fail over if needed, then write it up.
This page is that DDoS runbook in three parts: before, during and after. Keep a copy somewhere other than the server it protects.
What to prepare before a DDoS attack
A DDoS incident response plan for a small team fits on one page. These five items are the core of any DDoS attack checklist.
- An incident contact list. Who decides, who can change DNS, who holds the registrar and client area logins, how to reach support. Store it, with this runbook, outside the systems that might be down.
- Monitoring and a status page hosted off site. A check running on the server under attack goes quiet exactly when you need it. Put both in another city, as in monitoring your own server from another city.
- A baseline of normal traffic. At your busiest hour, record bandwidth, requests per second, connection states, CPU and database connections with the commands you will run during an attack, so you compare numbers, not impressions.
- A tested second location and low DNS TTLs. A standby with current code and data, and a TTL of 300 seconds or less on the records you would move. Lowering a TTL mid attack does not help resolvers that cached the old value. See pairing two VPS locations for failover.
- Your tier and the over cap rule. Know which DDoS protection tier each server has and what happens when an attack is bigger. The provider section below has the numbers.
Out of band here means watching and failing over from another city, not from the server under attack.
What to do during a DDoS attack: the first 15 minutes
When a server under DDoS attack stops answering, the instinct is to start blocking things. Answer two questions first: is this an attack at all, and which layer is it hitting.
How to tell if you are being DDoSed or just busy
Many incidents that look like a DDoS are a newsletter, a release, a crawler or a slow query. Rule those out first, because blocking real customers is its own outage.
| What you see | Likely cause | Check first |
|---|---|---|
| Traffic up on normal pages, real browsers, a campaign referrer | A marketing email or a mention somewhere | What went out today, the referrers |
| CPU or database pinned, traffic near baseline, started around a deploy | A release or a slow query | Deploy log, slow query log |
| One user agent walking every URL | An aggressive crawler | User agent in the access log |
| Bandwidth or packet rate many times baseline, ports you do not serve | Volumetric L3/L4 flood | iftop and a tcpdump sample |
| Thousands of half open TCP connections | SYN flood or connection exhaustion | ss state counts |
| Many addresses hitting one URL or random query strings, no images loaded | Layer 7 HTTP flood | Top addresses and URLs in the access log |
Flood traffic does not follow your daily curve. It often skips the assets a browser would load, and it concentrates on whatever is most expensive for you to serve.
Identify the layer with a few commands
Run these on the server and save each output with the UTC time. Replace eth0 with your interface from ip -br link.
# connection summary, then counts by TCP state
ss -s
ss -Htan | awk '{print $1}' | sort | uniq -c | sort -rn
# live bandwidth in and out
iftop -ni eth0
# a 2000 packet sample for support, then its top sources
tcpdump -ni eth0 -c 2000 -w /root/sample.pcap
tcpdump -nr /root/sample.pcap | awk '{print $3}' | cut -d. -f1-4 | sort | uniq -c | sort -rn | head
# top client addresses and top URLs in recent requests
tail -n 50000 /var/log/nginx/access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head -20
tail -n 50000 /var/log/nginx/access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20
# CPU and database load
vmstat 5 5
psql -c "SELECT state, count(*) FROM pg_stat_activity GROUP BY state;"
mysqladmin processlist
Read the results against your baseline:
- Bandwidth or packet rate far above normal: a volumetric flood or a full port. VPS ports are shaped by plan at 100 to 300 Mbps and dedicated servers start at a guaranteed 1 Gbps, so check which direction is full. A full outbound port means your server is sending: a large download, a backup or a compromised process. Our post on bandwidth and data transfer limits explains the ceilings.
- A large, growing count in
SYN-RECV: a SYN flood or connection exhaustion. - Normal bandwidth, a few URLs or addresses dominating the log: Layer 7.
- Normal traffic, busy CPU, a database full of waiting queries: an application problem until proven otherwise.
What to do in the next hour
If bandwidth looks normal but the app is down
Every voxa.host KVM VPS and dedicated server includes 100 Gbps of always on DDoS protection at Layer 3 and Layer 4, filtered upstream before traffic reaches your port. So if bandwidth and packet counts sit near baseline while the site is failing, you are most likely facing a Layer 7 flood or an application problem, not a full pipe.
Layer 7 filtering exists only on the 10 Tbps tier. On every other tier, HTTP floods are handled in your own stack, as our guide to Layer 7 DDoS protection against HTTP floods explains.
Rate limit the expensive routes
An HTTP flood hurts most where each request is costly: search, login, password reset, checkout, exports and API endpoints that go straight to the database. Put a per address limit on those first. In nginx:
limit_req_zone $binary_remote_addr zone=expensive:10m rate=5r/s;
location /search {
limit_req zone=expensive burst=10 nodelay;
limit_req_status 429;
proxy_pass http://app;
}
Then cache what you can, even for seconds, and shorten client timeouts so slow connections cannot hold workers. If the top addresses list shows a handful of sources, block them in nftables (our firewall guide covers the basics). If it shows thousands, a block list will not keep up, and limits plus caching will do more.
For APIs, limit per key as well as per address. Our post on DDoS protection for APIs and SaaS covers the patterns.
Change one thing at a time, and log it
Do not change five things at once. If the site recovers you will not know which change helped, and if it gets worse you will not know which one to undo. Make one change, watch, then decide.
Keep a running log with a UTC timestamp for every observation and change. An example:
14:02Z external check fails, pages timing out
14:06Z bandwidth at baseline, requests to /search about 30x normal
14:11Z limit_req 5r/s per address on /search, nginx reloaded
14:15Z error rate falling, CPU at 60 percent
That log becomes your support message now and your timeline later.
When to fail over
Fail over when the targeted IP has been null routed, or when the application cannot keep up and you have a tested standby elsewhere. Point DNS at the second location and keep working on the first. Resolvers that honour your TTL move within that window.
An attacker who follows DNS may follow you, so the standby needs protection too. Every VPS and dedicated server includes the same 100 Gbps L3/L4 tier.
What to tell your hosting provider during a DDoS attack
Contact support through the voxa.host contact page once you have the first facts. A useful message contains:
- The target IP address and which server it belongs to.
- The start time in UTC, and whether it is still going.
- What you see: bandwidth in and out against baseline, connection states, which services fail.
- Samples: the packet capture, the top addresses and URLs, a few raw log lines.
- What you have changed so far, copied from your log.
A message that only says the site is down starts a round of questions.
What happens when an attack is bigger than your tier
The included tier covers 100 Gbps. The first 5 attacks a month that exceed 100 Gbps are still mitigated at no charge. From the 6th in the same month, the targeted IP address is null routed until the attack ends or the server is upgraded to a larger tier. Only that address goes dark, not the whole server, as null routing and blackholing explained sets out.
You can change the tier from the client area, and billing is a flat monthly price per server with no per attack fee. Prices are per server, per month, excluding VAT, and cover every IP address on that server.
| Tier | Layers | Per month |
|---|---|---|
| 100 Gbps | L3/L4 | Included |
| 500 Gbps | L3/L4 | €156.45 |
| 1 Tbps | L3/L4 | €523.95 |
| 2 Tbps | L3/L4 | €1,048.95 |
| 5 Tbps | L3/L4 | €2,623.95 |
| 10 Tbps and Layer 7 | L3/L4/L7 | €4,723.95 |
Nobody can tell you in advance how long an attack will last. Decide before an incident which servers justify a larger tier and which can sit out a null route behind a failover.
What to do about a ransom DDoS email
A ransom DDoS demand threatens an attack unless you pay by a deadline, usually in cryptocurrency. Some come with a short demonstration attack, some with nothing behind them. Respond the same way.
- Do not pay. Payment buys no guarantee, and it tells the sender you are worth coming back to.
- Keep the email with its full headers and any payment address. Do not reply.
- Report it to the police through your national cybercrime reporting channel, and send a copy to your provider's support.
- Harden and upgrade instead. Work through the preparation list, rate limit the expensive routes now, and if the threat is credible, raise the tier of the servers that matter.
If a demonstration attack arrives, start the runbook from the top.
What to do after a DDoS attack
- Write the timeline from your UTC log while it is fresh: first signal, first action, each change, recovery. Note where you waited for information you could have prepared.
- Record what worked and what did not, including which commands told you the most.
- Look for origin IP leaks. If an address that should never be public was hit, such as a backend behind your reverse proxy, find out how it was found: an old DNS record, a forgotten subdomain, a test hostname, or a mail server on the same machine putting its address in outgoing headers.
- Review your tier. If you reached the over cap rule, or came close, weigh the cost of a null routed address against the price of the next tier. Our guide on how much DDoS protection you need works through that arithmetic.
- Rehearse the failover, especially if you meant to use it and did not. Then update the contact list and this runbook.
Next step: check what your servers already include
Every voxa.host KVM VPS and dedicated server starts with 100 Gbps of always on L3/L4 protection, active from delivery with nothing to switch on. The DDoS protection page lists every tier, the over cap rule, and how to add a tier at checkout or change it later from the client area.
Deploy what you just read about
Deploy your first server now.
No minimum term. Intel Xeon builds carry a delivery estimate of under an hour; AMD EPYC builds are made to order. Every configuration has full root, ECC memory and a guaranteed 1 Gbps unmetered port.
Dedicated billed monthly · No setup fee · Full root on every server
- IPv4 address
- One included
- Unmetered port
- 1 Gbps guaranteed
- DDoS protection
- 100 Gbps · L3/L4
- Root access
- Included
- Setup fee
- €0.00
- Minimum term
- None