If your GA4 reports show sudden spikes from unknown regions, identical session lengths, or landing pages that nobody shares, you are often looking at bots that still fire the analytics beacon. The goal is not to block every crawler. The goal is to cut the junk that pollutes decisions while leaving Googlebot, Bingbot, and real visitors alone.
This walkthrough is for WordPress on a DigitalOcean droplet (or similar VPS) with nginx, optional fail2ban, and GA4. It treats robots.txt, server firewalls, WordPress plugins, and CDN bot tools as different layers with different jobs. Learn more about SEO and GEO optimization services at WPPoland when you need crawl and citation work beyond traffic hygiene.
Why bad bots inflate Google Analytics
GA4 counts a hit when a browser (or a headless browser that behaves like one) runs your tag and sends events. That means three different populations show up in different places:
- JS-capable scrapers and click fraud tools can appear in GA4 as sessions, page views, and even conversions if they hit the right URLs.
- Simple HTTP scrapers often never execute JavaScript, so they miss GA4 entirely and still load PHP, MySQL, and any uncached templates on the origin.
- Named search crawlers should appear in server logs and Search Console, not as a large share of GA4 “Users.”
When the JS-capable group grows, acquisition and engagement reports drift. Bounce rate and average engagement time look weird. Paid and organic reports get noisy. Teams start optimizing for traffic that never buys anything.
Common shapes of bad traffic (without inventing specific IPs):
- High request volume from a small set of cloud ASNs, with no referrer and odd User-Agents
- Repeated hits on
/wp-login.php,xmlrpc.php, author-enum query strings, or old plugin paths - Sessions that land only on thin URLs and bounce in under a second, yet still create GA4 page_view events
- “Organic” traffic that never matches Search Console query data for the same period
Cloud providers such as DigitalOcean and AWS host plenty of legitimate sites. They also host scrapers. That overlap is why blanket “deny this entire /16” rules are risky. Prefer confirmed sources from your own logs over gossip lists copied from old blog posts.
How to detect bot traffic in GA4 and server logs
Change without a baseline is storytelling. Capture a baseline for at least seven days if you can, or a full business week if traffic is seasonal.
How to spot bot traffic in GA4 reports
Build a simple comparison view before you edit nginx or fail2ban:
- Note Users, Sessions, Engaged sessions, and Views for the last 7 and 28 days.
- Open Traffic acquisition and scan for sources with absurd engagement (near-zero engagement time, near-100% bounce on every landing page).
- Check Tech > Browser and Device for empty, “Unknown,” or obviously scripted clients.
- Look at Landing page for paths that only bots love (
/?author=1, old query spam, non-existent product IDs). - Export or screenshot the period so you can compare like-for-like later (same length, same weekday mix).
GA4 already removes known bots on its own (next section), but that list does not catch every abusive client that still runs JavaScript. Treat GA4 as one signal, not the only one.
How to find bot IPs in nginx access logs
On DigitalOcean you usually have nginx access logs and, if enabled, fail2ban jails. Useful checks:
# Top remote addresses in the last access log (adjust path)
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head
# Paths that look like probes
awk '{print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -40
# User-Agents that dominate
awk -F\" '{print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | headCorrelate GA4 spikes with timestamps in those logs. If GA4 jumps at 03:00 UTC and the access log shows the same IPs hammering xmlrpc.php, you have a server-side problem that analytics filters alone will not fix.
Also note origin CPU and PHP-FPM queue length during the spike. Bot noise that never reaches GA4 can still exhaust a small droplet.
Does GA4 filter bot traffic automatically
Partly. Google’s help article on known bot traffic says GA4 excludes traffic from known bots and spiders automatically. The list behind it combines Google’s own research with the International Spiders and Bots List maintained by the Interactive Advertising Bureau (IAB).
Two details on that page matter here. You cannot switch the exclusion off, and you cannot see how much traffic it removed. So when GA4 still shows junk sessions, those clients were not on the known list: typically headless browsers on rented servers that run your tag like an ordinary visitor. The automatic exclusion also does nothing for requests that never execute JavaScript, because GA4 never receives them.
That gap is what the rest of this guide covers. GA4 handles declared crawlers; your server and your edge have to deal with clients that hide what they are.
How to exclude bot IP ranges with GA4 data filters
GA4 data filters come in three types: developer traffic (debug mode), internal traffic (by IP address or range) and unwanted web hostnames. There is no bot type, so the only IP-based tool is the internal traffic filter.
It works in two steps, described in Google’s guide to defining internal traffic. First, in Admin under Data collection and modification, open Data streams, pick the web stream, go to Configure tag settings, then Define internal traffic, and create a rule. Matching events receive a traffic_type parameter. Its default value is internal, but you can set your own, for example bot_do, so a bot rule never mixes with your office rule. IP conditions accept exact match, begins with, ends with, contains, a CIDR range or a regular expression. Second, under Data filters you create an Internal Traffic filter that excludes that traffic_type value.
Limits to know before you rely on it:
- A property can hold up to 10 data filters.
- Each filter is Testing, Active or Inactive. In Testing, matching events are labelled with the
Test data filter namedimension, which you can inspect in Explore before anything is removed. - An active filter changes incoming data permanently and does not apply to historical data. Google says a filter can take 24 to 36 hours to start working.
- For a temporary view without losing data, Google points to report filters instead.
A safer sequence: keep a bot IP rule in Testing for a full week, compare the tagged sessions with what the nginx log shows for the same addresses, and switch to Active only when the tagged traffic is clearly junk. Keep the rule list short and note the date each range was added. And keep in mind what this filter is: a reporting fix. The requests still hit nginx, PHP and MySQL; GA4 just stops counting them.
How to check who owns an IP before blocking it
One address in a log tells you little until you know who announces it. whois gives you the registry record. The RIPEstat data API gives the routing view, meaning the prefix and the autonomous system (AS) announcing it right now:
whois 5.101.96.1 | grep -iE "orgname|org-name|netname"
curl -s "https://stat.ripe.net/data/network-info/data.json?resource=5.101.96.1"
curl -s "https://stat.ripe.net/data/as-overview/data.json?resource=AS14061"When I ran the two API calls on 14 October 2026, RIPEstat returned the prefix 5.101.96.0/21 announced by AS14061, and named the holder of AS14061 as DIGITALOCEAN-ASN, DigitalOcean, LLC. The network-info endpoint is free and answers for addresses from every regional registry, not only RIPE.
Read the CIDR suffix before you paste a range anywhere. The number after the slash counts the fixed prefix bits, so a /24 holds 256 addresses, a /16 holds 65,536 and a /14 holds 262,144. One old line such as deny from 104.236.0.0/16 shuts out 65,536 addresses, nearly all of them unrelated to the client that bothered you.
Where to find the current DigitalOcean IP ranges
DigitalOcean publishes its address space as a CSV feed, linked as DigitalOcean IP Ranges from its platform documentation, and states that the file is updated automatically. Each row holds the prefix, the country code, the region, the city and the postal code.
On 14 October 2026 the file listed 1,234 prefixes, 148 of them IPv6. By country, 654 rows were in the United States, 224 in the Netherlands, 111 in Germany, 90 in Singapore and 67 in the United Kingdom. That spread is the practical case against blanket blocking: the same prefixes carry Droplets for agencies, SaaS tools, uptime monitors, CI runners and ordinary businesses, next to the scrapers.
Use the feed as a lookup, not as a deny list. When one address dominates your access log, check whether it sits in DigitalOcean space:
curl -s -o do-ranges.csv https://www.digitalocean.com/geo/google.csv
python3 -c 'import csv,ipaddress,sys; nets=[ipaddress.ip_network(r[0]) for r in csv.reader(open("do-ranges.csv")) if r]; ip=ipaddress.ip_address(sys.argv[1]); print(any(ip in n for n in nets))' 5.101.96.1A True tells you the client runs on a DigitalOcean server. It does not tell you the client is malicious; the log behaviour decides that.
Can robots.txt block bad bots
robots.txt is a voluntary protocol. Cooperative crawlers read it and skip disallowed paths. Abusive clients ignore it.
Use robots.txt to guide good bots:
- Disallow staging paths, search result URLs, and admin endpoints you do not want indexed
- Keep crawl-delay and wild blanket Disallow rules modest; over-blocking can hide pages you still want in search
- Put sensitive areas behind authentication, not only behind a Disallow line
Do not expect a Disallow rule to stop credential stuffing on /wp-login.php. That request never consults robots.txt in a meaningful way. Firewall, rate limit, and auth belong on the server or at the edge.
A practical split:
| Layer | What it does | What it does not do |
|---|---|---|
| robots.txt | Polite crawl guidance | Stop malicious clients |
| nginx / fail2ban | Rate limit and ban by behavior | Replace a product WAF on its own |
| WordPress firewall plugin | App-aware rules inside PHP | Stop traffic before WordPress boots |
| Cloudflare Bot Management / WAF | Filter at the edge | Fix a misconfigured origin by itself |
How to verify Googlebot before blocking an IP
Blocking Googlebot by User-Agent string is a classic own-goal. Spoofed Googlebot strings are common; real Googlebot verifies through reverse DNS and forward confirmation. Google documents the process in Verifying Googlebot.
Before you deny an IP that claims to be Googlebot:
- Run reverse DNS on the IP.
- Confirm the hostname ends in a Google domain pattern Google documents.
- Resolve that hostname forward and confirm it matches the original IP.
- Only then treat the client as Googlebot (or not).
Never add a rule like “deny if User-Agent contains Googlebot” as a security control. That blocks the real crawler when someone spoofs the string elsewhere, and it fails to stop spoofers who use a different string.
Same caution applies to Bingbot and other named crawlers you care about. Prefer allowlists for verified crawler ranges only when you maintain them carefully; prefer behavioral rate limits for everything else.
How to rate limit bots on DigitalOcean with nginx and fail2ban
On a typical LEMP droplet, the cheapest place to shed abuse is nginx, before PHP-FPM and WordPress autoload.
nginx limit_req for wp-login.php and xmlrpc.php
The ngx_http_limit_req_module limits request rate by key (usually IP). A conservative pattern for login and XML-RPC looks like this:
# http context
limit_req_zone $binary_remote_addr zone=wp_login:10m rate=1r/s;
limit_req_zone $binary_remote_addr zone=wp_xmlrpc:10m rate=1r/s;
# server / location examples
location = /wp-login.php {
limit_req zone=wp_login burst=5 nodelay;
include fastcgi_params;
# ... existing PHP handler ...
}
location = /xmlrpc.php {
limit_req zone=wp_xmlrpc burst=2 nodelay;
# Many sites return 403 here if they do not need XML-RPC at all
}Tune rates from your baseline. A newsroom with many editors sharing a NAT will need a higher burst than a brochure site. Log limit_req rejects and watch for false positives from office egress IPs.
Other nginx habits that help on DO droplets:
- Disable or tightly restrict
xmlrpc.phpif you do not use it - Rate-limit POST to
admin-ajax.phpfor anonymous clients when plugins allow it - Terminate TLS at nginx (or at Cloudflare) and keep only needed ports open in the DigitalOcean cloud firewall
fail2ban jails for WordPress login and probe paths
fail2ban watches logs and adds temporary firewall bans when a filter matches. Typical WordPress-oriented jails watch nginx logs for repeated 401/403/404 on login and probe paths. Keep jails narrow:
- Ban for minutes or hours, not forever, until you trust the filter
- Exclude your office VPN and monitoring checkers
- Recheck after WordPress or plugin updates change log formats
fail2ban is excellent against noisy scanners. It is a poor substitute for fixing an exposed plugin endpoint that returns 200 on every probe.
Which ports to open in the DigitalOcean cloud firewall
Use the DO firewall (or equivalent) to allow 22/80/443 from the world only if you must, and prefer key-based SSH from known IPs. Network blocks stop packets before nginx. They still should not be “block every DigitalOcean /16 on Earth,” which will catch customers, agencies, and CI runners hosted on the same provider.
Do WordPress firewall plugins stop bot traffic
Plugins such as Wordfence, Sucuri, and solid security suites add login protection, malware scans, and application rules. They help. They also run after nginx has already accepted the connection and PHP has started.
Caveats that matter for analytics cleanup:
- Cost in CPU: every blocked request still paid for a PHP bootstrap unless something earlier rejected it.
- False positives: aggressive “block unknown countries” or “block all cloud ASNs” rules can lock out freelancers, agencies, and payment providers.
- Overlap with edge WAF: duplicate rules at Cloudflare and in the plugin make debugging harder when a legitimate user is challenged twice.
- Log noise: plugin logs and GA4 will disagree because they see different layers.
Hardening guidance from the WordPress hardening handbook still applies: keep core and plugins updated, use least privilege for database users, and restrict file permissions. A firewall plugin does not replace those basics.
Practical order of operations:
- Fix exposed endpoints and weak credentials.
- Add nginx rate limits and fail2ban for noisy paths.
- Add a WordPress firewall for app-aware rules and alerts.
- Add Cloudflare bot tools if you already use the CDN.
How to block bots with Cloudflare before they reach WordPress
If the site sits behind Cloudflare, use their bot documentation rather than inventing User-Agent deny lists at the origin. Edge rules can:
- Challenge or block obvious automated traffic before it reaches DigitalOcean
- Rate-limit specific paths (
/wp-login.php,/xmlrpc.php) - Keep verified search engine bots in an allow path when you configure that carefully
Origin nginx rules still matter as defense in depth if someone bypasses the proxy (grey-cloud DNS, direct origin IP). Lock the droplet firewall so only Cloudflare IP ranges (or your admin IPs) can speak to port 80/443 when you run full proxy mode.
Cloudflare does not make GA4 magically accurate by itself. It reduces abusive hits that would have executed tags or exhausted PHP. Measure both edge analytics and GA4 after you change Bot Fight Mode or WAF rules, because challenges can also change how real users complete a session.
How to challenge DigitalOcean traffic with Cloudflare WAF custom rules
Cloudflare WAF custom rules can match the network a request comes from. The field is ip.src.asnum, the AS number of the client IP, and DigitalOcean’s network is AS14061. One rule aimed at the AS replaces maintaining 1,234 prefixes by hand.
Keep the scope narrow. A rule that only touches DigitalOcean traffic on the paths bots abuse looks like this:
(ip.src.asnum eq 14061 and http.request.uri.path in {"/wp-login.php" "/xmlrpc.php"})Choose Managed Challenge as the action, not Block. A person working through a VPN that runs on a Droplet can pass a challenge; a simple script usually cannot. Block ends the request, and because Cloudflare evaluates custom rules in order, it also stops every later custom rule from running for that request.
Plan limits decide how many such rules you can afford. Cloudflare lists 5 custom rules on Free, 20 on Pro, 100 on Business and 1,000 on Enterprise, and the regex operator only on Business and Enterprise. On a free zone, one rule for the ASN challenge on login paths leaves four for problems you cannot solve at the origin.
Should you block DigitalOcean and AWS IP ranges
Older advice pasted long deny from lists for whole DigitalOcean or AWS ranges into .htaccess. That approach ages poorly:
- Provider ranges change
- Legitimate traffic shares those ranges
- Apache
denysyntax does nothing useful if you serve WordPress through nginx without Apache
If you still use Apache, prefer confirmed single addresses or small CIDRs from your logs, not a copied mega-list:
# Example shape only - replace with IPs you confirmed in your own logs
# RequireAll / Require directives depend on Apache 2.4 authz syntax
<RequireAll>
Require all granted
# Require not ip 203.0.113.10
</RequireAll>On nginx, use deny inside a geo/map block or fail2ban, and keep a short comment with the date and why the address was banned. Revisit bans quarterly.
Never publish or rely on “known bot IP” tables from random posts without verification. They go stale and they invite collateral damage.
Read the Docs hit the limits of IP blocking in June 2026, in an attack David Fischer described on 8 September 2026. At peak it sent over 5.5 million requests per minute, about 100 times normal traffic, for nearly ten days, from millions of unique IP addresses across hundreds of networks (ASNs). The attackers targeted uncached 404 paths and 302 redirects; the team moved the redirects to the Cloudflare edge within about half an hour, and Fischer says a “penalty box” rule that rate-limits clients collecting many 404 or redirect responses probably made the biggest difference. His first takeaway is “IP blocking is obsolete for distributed attacks”, and the harshest response Read the Docs uses is a JavaScript challenge. For a WordPress site the same logic applies: cache 404s and redirects at the edge and rate-limit suspicious clients instead of blocking whole cloud ranges.
How to check that bot traffic dropped in GA4
After you ship rate limits or bans, wait through the same window length you used for the baseline (7 or 28 days). Then compare:
- Users / Sessions / Engaged sessions - look for junk volume dropping while engaged sessions stay flat or rise.
- Landing pages - probe URLs should shrink in GA4 if those clients executed JS; they should shrink in nginx logs either way.
- Traffic acquisition - mystery sources with zero engagement should fall.
- Server metrics - PHP-FPM busy workers and 5xx rates during off-hours should ease if origin abuse was real.
- Search Console - crawl stats and coverage should not collapse. If they do, you likely blocked a real crawler; roll back and verify Googlebot properly.
Annotate the change date in GA4 (Admin > Annotations, or your team’s change log) so later you remember what moved the line.
If GA4 barely moves but nginx rejects climb, the bots were never executing your tag. That is still a win for server health, even when the analytics chart looks calm.
If GA4 drops hard and Search Console crawl errors spike, undo the last deny rule first. Analytics cleanliness is worthless if indexing breaks.
Checklist to stop bot traffic on WordPress
Work through this in order:
- Baseline GA4 (7/28 days) and sample nginx access logs.
- Confirm whether the noise is JS-capable (GA4) or origin-only (logs).
- Tighten
xmlrpc.phpand login rate limits in nginx. - Add fail2ban jails for repeated probes; exclude your own IPs.
- Keep robots.txt focused on crawl guidance, not security theater.
- Add or tune a WordPress firewall without blocking entire cloud providers.
- Optionally enable Cloudflare bot / WAF controls if the site already uses Cloudflare.
- Re-measure GA4 and logs over a matching period; watch Search Console.
If you want help reviewing crawl noise, server rules, and analytics hygiene together with broader SEO and GEO work, that is a better fit than pasting eternal deny lists into the WordPress root.





