Websites blocking AI agents: Yelp and two banks say they chose to, while Walmart told TechCrunch its blocks were not intentional

Websites are blocking AI agents, the assistants that browse and buy on a person’s behalf, for two different reasons. Some decided to. Yelp only admits agents that pay for its data, and two banks bar them from accounts for security reasons.¹ Others did not mean to. TechCrunch reported that a Walmart spokesperson “told us these weren’t intentional” about the blocks its shoppers’ agents hit, and Zillow said its anti-scraping measures can occasionally block agents it allows.² For a store, the second kind matters more, because it means customers can be turned away by defenses the store never meant to aim at them.

A Walmart spokesperson told TechCrunch the blocks its shoppers’ AI agents hit were not intentional, even though Walmart is a partner of the agent being blocked.

Why are websites blocking AI agents?

Websites block AI agents either as a stated business or security policy, or as a side effect of defenses built to stop scraping and fraud. TechCrunch described the split in October 2026. “Some of these are intentional, like Amazon’s, while others are the result of websites’ traditional anti-bot measures designed to protect against spam and malicious activity.”²

The Verge reported that users of Meta’s Muse, xAI’s Grok, OpenAI’s Dots and Instinct had hit blocks “across top websites,” and that some users said sites which used to work with their agents, including Cars.com and Amtrak, no longer did.¹ The table below sorts the companies that spoke on the record by what they said decided the block.

Site	What the company said	What decided the block	Source
Yelp	No non-human traffic unless the agent pays for its data licensing program	Business policy	TechCrunch2
JPMorgan Chase, Charles Schwab	Confirmed they block agents from accounts over security concerns	Security policy	The Verge1
eBay	Restricts unauthorized agents and actions such as scraping and model training, not all shopping agents	Business policy	TechCrunch2
Delta Air Lines	No partnership yet for third-party agents, and steps against unauthorized automated activity; asked whether it blocks agents, it did not say	Not stated	TechCrunch2
Walmart	The blocks were not intentional; Walmart is a Muse partner	Not intended, per Walmart	TechCrunch2
Zillow	Allows consumer agents, but anti-scraping measures can occasionally block some	Its anti-bot measures, per Zillow	TechCrunch2

Amazon’s block on Meta’s Muse is the best-known case of the first kind, and it is covered in the article on Shopify’s signed-agent checkout.

Did Walmart mean to block AI shopping agents?

Walmart did not mean to block AI shopping agents, according to a Walmart spokesperson quoted by TechCrunch. Users of Meta’s Muse had reported trouble checking out on Walmart, and TechCrunch wrote that the spokesperson “told us these weren’t intentional.” Walmart is a partner of Muse, announced at Meta’s Connect conference in September, so the agent its own partner built was being turned away at its door.²

TechCrunch’s reading of the user reports, which is TechCrunch’s and not Walmart’s, is that the site shows a button a visitor has to click to prove they are human. “If that experience is interrupted, the check could fail, and the agent gets booted out.”² The Verge reported the same checkout trouble from user posts, without a statement from Walmart.¹

Zillow described the same tension in its own terms. After TechCrunch’s story ran, Zillow emailed to say it allows AI agents acting on behalf of consumers, but that its anti-bot measures against scraping can occasionally block certain agents.²

How can a website block an AI agent it never meant to block?

A website can block an AI agent it never meant to block when its defenses against scrapers and fraud catch the agent too. Those defenses judge how a visitor behaves, and an agent clicking through a checkout behaves a lot like the bots they were built for. In my reading, which no source applies to Walmart or Zillow, the layer worth checking first is the network edge, the service every request passes through before it reaches the store.

Chris Canal, chief executive of EquiStamp, told The Verge that many defenses are “set up not even to block agents, but to block in general bots or programs that are trying to scam or hack,” and “These AI agents trigger those same safeguards.”¹

Think of a shop in a mall where the building’s security desk, run by property management, decides who gets through the front entrance. The shop can want every customer it can get, and still lose the ones the desk turns away under a rule written for someone else.

Cloudflare’s own documentation shows how such a rule can arrive without anyone at the store writing it. In July 2026 Cloudflare let customers manage AI bots by behavior, as Search, Agent or Training, and set “New defaults, in which Training and Agent are blocked on pages that display ads while Search remains allowed,” for new domains from September 15, 2026.³

Cloudflare’s definition of an agent is “automated activity acting in real time on a person’s behalf, such as chat fetch bots and browser-use agents,” and a block covers verified bots in that class “plus additional unverified bots that fall under these classifications.”⁴ Whether a page shows ads is decided by “Cloudflare automated detection” for that zone.⁴

Cloudflare’s free bot product is harder to make exceptions in. Bot Fight Mode “Cannot be customized, adjusted, or reconfigured via WAF custom rules,” the firewall rules a site owner writes, and Cloudflare warns it “may challenge API or mobile app traffic.”⁵ Neither a custom rule nor a page rule can skip it, and Cloudflare points owners who need exceptions to its Super Bot Fight Mode instead.⁵

No source ties Walmart’s or Zillow’s block to Cloudflare or to this default. Asked by TechCrunch about suspicions that Cloudflare and other content delivery networks were disrupting Muse traffic, Cloudflare “said it didn’t have any specific data to share at this time.”²

TechCrunch relayed that some attribute the disruption to the September 15 change, and wrote that existing sites set to block AI bots for security reasons “were then shifted to block AI agents on any pages that had ads.” That account is TechCrunch’s, while Cloudflare’s own pages state the new default for new domains only.²⁴

Can a site’s own logs show AI agents being blocked?

A site’s crawler log can show that AI agents asked for its pages, but it shows a block only if it records what the site answered, and many logs do not. On this site, the log of bot requests drawn from Cloudflare’s analytics stores the date, the user agent (the label a program sends to say what it is) and the page address, with no response status.⁶

So the log separates agents from crawlers clearly, and still cannot say whether my own edge served them or turned them away. The log holds 16,729 rows from September 20 to October 25.⁶ Of those, 551 came from user agents that name themselves as fetching for a person.⁶ The table below lists them.

Agent	Rows	Days seen, of 36	Distinct pages	Article pages
ChatGPT-User	296	36	109	54
Claude-User	216	28	40	9
Meta-ExternalFetcher	21	2	21	0
Perplexity-User	14	3	14	0
Keenable-User	3	3	3	1
OpenCode-User	1	1	1	1

ChatGPT’s fetcher showed up every day of the 36. The Claude count is not reader demand, because 37 of the 39 Claude-User labels belong to Claude Code, and I fetch my own pages with it. Meta-ExternalFetcher is Meta’s fetcher for shared links, and nothing in the log says it was shopping for anyone.⁶

The agents most like a shopper are the ones this log cannot see at all. A browser-use agent that presents an ordinary browser label is dropped before the log is written, because the log keeps only requests whose label marks them as a bot.⁶ What my own edge does to agents is a question for the zone’s bot settings, which the log does not record.

How did I count the agent requests?

I counted agent requests in this site’s crawler log for September 20 to October 25, 2026, which holds one row per date, user agent and page, built from Cloudflare’s analytics.⁶ An agent here means a user agent carrying a “-User” product name, the naming OpenAI, Anthropic, Perplexity and Mistral use for fetches a person triggers, or Meta-ExternalFetcher. Everything else counts as a crawler or other bot.

That definition follows Cloudflare’s own agent class, fetches made in real time for a person.⁴ Requesters were grouped by the product name in their label, days are distinct dates with a row, and pages are distinct addresses. The count had no pass or fail bar, because it only measures what the log can see.

What should a store do about AI agents it may be blocking?

A store that wants AI agents to shop with it should read its own bot settings at the network edge beside its commerce policy, and check that the two agree. When they disagree, the edge setting is the one a request meets first. Cloudflare’s split into Search, Agent and Training means the agent decision can be made on its own, instead of riding along with a block meant for training crawlers.³

Three checks cover most of it. Find out whether the zone was added after September 15, 2026, when the ad-page default took effect for new domains.³ Find out whether Bot Fight Mode is on, since no custom rule or page rule can skip it.⁵ And log the response status beside each bot request, so a block shows up in your own data before a partner’s customers report it.

Meta framed the cost from the agent maker’s side: “Turning away a personal agent means turning away the customer behind it.”¹ A store that decides to block agents can still make that call on purpose. The blocks worth finding are the ones the store did not intend, as Walmart’s were by its own account. For the route where an agent proves who it is before checkout, see the article on signed agent requests, and for another setting that ships switched on, the article on catalogs shared by default.