Brandon Lazovic

Pulse · October 31, 2026

Confirmed

Google And OpenAI Commit To Honoring A Shared robots.txt Signal For AI Training

On October 30, Google and OpenAI each announced plans to honor a shared robots.txt policy token for AI training, drawn from the IETF's AIPREF working group. Google says a site using the token to opt out, without naming Google-Extended, will be read as if it disallowed Google-Extended. Both say existing crawler-specific controls keep working. Google aims to support it in 2027, and OpenAI says in the coming months.

In plain termsToday a site that wants to opt out of AI training has to list each company's crawler by name; a shared policy token is one agreed phrase in robots.txt that every participating company promises to read the same way.

Why it mattersOne line in robots.txt could soon cover several AI providers' training opt-outs, but it governs training only and does not change whether a page is fetched for search or retrieval.

Our takeI read the citation study as showing that publishers blocked the bots that collect training data and left the retrieval bot open, so the citations kept coming. A shared training signal makes the first half of that easier to do, and it leaves the second half exactly where it was.

Confirmed

Bing Splits AI Training And Grounding Into Separate Publisher Controls, Retiring NoArchive And NoCache

Microsoft's October 30 post adds two independent preferences. Training uses the shared robots.txt signal, set with a __usage_ai-training user agent, a meta tag or a header. Grounding is page-level only, with Allow, DisallowExtended or Disallow. New settings take effect February 1, 2027. After March 1, 2027, NoArchive and NoCache no longer count as a training or grounding preference. Normal Bing search eligibility is unchanged.

In plain termsTraining means a company uses your pages to build its model, while grounding means its assistant reads your page at answer time to support a response; Bing now lets a page say yes to one and no to the other.

Why it mattersA site that relies on NoArchive or NoCache to limit Copilot and Bing AI answers has until March 1, 2027 to publish page-level grounding settings, and choosing Disallow reduces its visibility in those answers.

Our takeGrounding is the control that decides whether a page can be cited in a Copilot answer, so it is the setting with a visibility cost. Page-level only means a large site will need a template-level decision, not a robots.txt line.

Confirmed

Discord To Publish Every Discoverable Server As A Public Web Page, With Chat Messages In Search From Mid-December

Discord announced on October 27 that from November 16 every Discoverable Server, more than 63,000 of them, gets a public Server Web Page with automated channel summaries. From mid-December, some public-channel messages may also appear on those pages and in web search. Admins can change channel settings before November 16. The member message setting is on by default except in the EU, UK, Canada, Brazil, Australia and South Korea.

In plain termsUntil now Discord chats lived behind the app; this makes the public ones readable by anyone with a link, which also means a search engine can read and list them.

Why it mattersA large body of community conversation becomes indexable text on discord.com, which creates a new source of search results and AI answer citations for brand and product questions.

Our takeForum threads already earn visibility in AI answers because they read like first-hand experience, and Discord messages will now compete for the same slot. Brands with an active Discord community should read what it says about them before a search result does.

Confirmed

Shopify's AI Agent Catalog Adds Promoted Placements That Pay Agent Developers A Commission

Shopify's developer documentation describes promoted placements in its catalog for AI shopping agents, in invite-only Developer Preview. When a shopper buys a promoted product, the agent's developer earns a 0.3% base commission. A merchant's Shop Campaigns can add more, with 1.5% shown as an example, but only when the purchase completes in Shop or on shop.com. Shopify says details are not final.

In plain termsThink of paid shelf placement in a store, except the shopper is an AI agent, and the agent's maker, not the store, collects a cut when the promoted item sells.

Why it mattersA merchant's catalog position in agent results may soon depend on a funded campaign, and the higher commission pays only on purchases that complete in Shop, not on the merchant's own site.

Our takeI argued that syndicating through Shopify Catalog makes Shopify the data gatekeeper for a merchant's AI-driven sales. Promoted placements add a price to that gate, and the documentation does not say whether a higher commission ranks a product higher.