Google treats 404 and 410 the same, and the code that changes its behavior is 429

A status code, the three-digit number a server sends back with every page request, is the only thing telling a crawler, the automated program a search engine uses to fetch and read pages, what actually happened. Google’s own current documentation says a 410, the code for “this page is gone for good,” and a 404, the code for “this page was not found,” get handled identically.¹ So the common advice that switching a dead page from 404 to 410 removes it from Google’s index, the search engine’s stored list of pages it can show searchers, any faster has no support in Google’s own words. The code that actually changes what Google does next is 429, and it is not the one most people are watching.

“All 4xx errors, except 429, are treated the same: Google crawlers inform the next processing system that the content doesn’t exist.”

What does an HTTP status code actually tell a crawler?

An HTTP status code, sent under the protocol browsers and crawlers use to request web pages, is a three-digit number a server attaches to every response. Before it reads anything else on the page, a crawler reads that number. Status codes sort into four ranges. The 200s mean success. The 300s mean the page moved. The 400s mean the request was the problem, and the 500s mean the server broke.

In the response headers, the metadata lines a server sends alongside the page content, some of that message travels, and that is also where a redirect’s destination lives. A 301 tells a crawler a page moved permanently, and a 307 tells it the move is temporary, a real difference in meaning. Google still folds 307 into the same treatment as 302 and 308 into the same treatment as 301, adding only that “they’re semantically different” even while handling them alike.¹ Google has recently folded this guidance into a new /crawling/ documentation section, moved from its older crawling-indexing path, without changing the substance of the policy.¹ Google makes an almost identical move with 4xx codes, and untangling it is the rest of this piece.

Does a 410 really remove a page faster than a 404?

A 410 does not remove a page from the index any faster than a 404 does. Google’s own documentation states plainly that the two codes get treated exactly the same way once a crawler reports either one.¹ The advice that switching a dead page to 410 speeds up its removal is common, and it has no support in that documentation.

RFC 9110, the technical standard defining what HTTP status codes mean, disagrees with that treatment on paper. It says a 404 “does not indicate whether this lack of representation is temporary or permanent,” while a 410 tells the requester the resource is “no longer available at the origin server and that this condition is likely to be permanent,” adding that 410 “is preferred over 404 if the origin server knows, presumably through some configurable means, that the condition is likely to be permanent.”² The standard draws a real distinction between the two codes, and Google’s crawler does not act on it. That gap between what the standard prefers and what the crawler actually does is exactly where the popular advice came from, standards language borrowed and applied to the wrong system.

Why does Google treat every 4xx the same?

Google treats every 4xx code the same because its indexing pipeline, the internal process deciding what stays in the index, only checks whether the content exists. Its documentation states that for any 4xx status except 429, “Google crawlers inform the next processing system that the content doesn’t exist,” and a previously indexed URL then gets removed.¹ Even though it means forbidden access, not a missing page, a 403 reports the identical verdict as a 404.

Google’s documentation makes the same point about crawl rate directly: “Don’t use 401 and 403 status codes for limiting the crawl rate. The 4xx status codes, except 429, have no effect on crawl rate.”¹ A 404 gets treated as the same hard verdict as a permissions error or an outright block, and none of them change how often Google comes back to check again.

Which code actually changes how often Google crawls you?

Only 429 and the 5xx codes, server errors such as 500 and 503, change how often Google crawls a site. Google’s documentation states that “5xx and 429 server errors prompt Google’s crawlers to temporarily slow down with crawling,” while every other 4xx code has no effect on crawl rate at all.¹ A site’s crawl frequency moves only when the server itself signals trouble, never when a single page turns up missing.

When a site recovers, that slowdown reverses on its own. Google’s documentation says: “Once the server starts responding with a 2xx status code, Google gradually increases the crawl rate for the site.”¹ The reverse runs the same shape as the drop. A plain 500 alone “decreases the crawl rate for the site,” with no stated speed for either direction.¹

What does a 503 buy you, and for how long?

A 503, the code meaning the server cannot handle a request, buys time with no stated expiration. RFC 9110 says a 503 response can carry a “Retry-After” header, a line telling the requester when to check back, that itself “indicates how long the service is expected to be unavailable to the client.”² Neither source gives a number of hours or days, only a direction: slow crawling, then speed back up once normal responses return.

Since neither Google nor the standard ever published a duration, a lot of practitioner advice fills that gap with a specific day count of its own. RFC 9110 adds one more limit. The standard states plainly that “the 503 status code does not imply that a server has to use it when becoming overloaded,”² so a server can choose to fail some other way instead. Observable behavior ends the slowdown. Google’s documentation says the crawl rate “gradually increases” the moment 2xx responses start coming back, with the same missing timeframe as the slowdown itself.¹

Why is 429 filed with the server errors?

Google files 429, the status code named “too many requests,” as a server error rather than a client error, even though its number starts with a 4. Its documentation states: “429 (too many requests) Google’s crawlers treat the 429 status code as a signal that the server is overloaded, and it’s considered a server error.”¹

429’s reclassification as a server error carries the whole payload. Call it a digit-blind bucket: a code’s handling category is not decided by its first digit, and 429 is the proof. While 429 sits with 500 and 503 in come-back-later, every other 4xx code sits with 404 in the treat-as-gone bucket, sorted by what caused the code rather than by which range its number falls in. A team that reads 429 as “just another client error, ignore it” is reading the number and missing the policy attached to it.

Think of the first digit of a status code as a shelf label at a library, a rough sort rather than a rule: everything on the 4 shelf is supposed to be the reader’s problem, and everything on the 5 shelf is supposed to be the librarian’s. The analogy breaks exactly at 429, the one title Google pulls off the 4 shelf and reshelves under 5, because what decides its home is the reason it showed up.

What does this site return for a URL that does not exist?

brandonlazovic.dev returns a real 404, not a soft 200 dressed up as a normal page, whenever someone requests a URL that never existed. A direct check against the live site confirms it: curl -I https://brandonlazovic.dev/articles/this-slug-does-not-exist/ comes back HTTP 404, and the site’s configuration sets not_found_handling to 404-page, which serves a real 404 status alongside the visible error page.³ That routing config applies globally, so the same 404 behavior holds for every URL on the site.

A second kind of 404 shows up on this same site with no explicit error at all. An article scheduled for a future publish date is excluded from the build entirely, so its URL 404s by absence rather than by any code returned on purpose, and three of this week’s own articles sat in exactly that state until their reveal dates arrived.³ With nothing in the response headers marking one as deliberate and the other as a scheduling artifact, Google’s crawler cannot tell those two cases apart, since both simply produce a 404 response.

Which code should you reach for, and when?

Reach for 404 or 410 when a page is gone for good and it is never coming back, since Google’s own documentation treats the two identically, and the choice mostly affects which other tools or teammates on your team read your intent.

Reach for 301 when a page moved permanently and 307 when the move is temporary. A canonical tag, the on-page signal naming which URL among several duplicates should count as the real one, complements either redirect when the content still lives somewhere else on the same site. Reach for 429 or 503 only when the server itself needs Google to slow down, never as a way to hide a page from it.

None of these choices happen on a promised clock. Google’s documentation states the direction it moves in, never a day count, and RFC 9110 does the same for how a 410 signals other systems reading a response. This week’s other pieces make the same case from a different angle: the longest matching rule in a robots.txt file wins, and a disallow line is not a noindex tag either, both cases where the documented behavior overrides what a rule’s name seems to imply. A status code is the same kind of trap. What both sources agree on is simpler than any clock. Pick the code that actually describes what happened, and Google will read it exactly as documented.