Soft tokens: how single embeddings change item representation

A wave of 2026 papers converges on the same architectural bet: compress everything you know about an item into a small set of differentiable embeddings, drop them directly into the recommendation model’s token space, and stop fighting the prompt-length ceiling. The bet is paying off in fresh-content reach. The deeper question, for anyone watching organic and AI-surface performance, is what happens when that approach migrates from the recommendation pipeline into the retrieval systems that feed search-side ranking.

The clearest result comes from Token Factory (arXiv:2606.19635), a production-scale system from a team working on large-scale industrial recommendation. It does not name the production platform, but it reports a number that is hard to ignore.

+67.1% fresh-content impressions and coverage, with satisfaction flat-to-positive. This is not a satisfaction or revenue lift.

What is a soft token, and how does it differ from a text item description?

A text item description is what it sounds like: you take a product title, a description, a set of attributes, and serialize them into natural language that the model reads as text. The model already knows how to process text, so the approach requires almost no new machinery. The cost is length. A rich item description runs long, and recommendation models process sequences, so the length multiplies across millions of items and billions of requests.

A soft token is not text at all. It is a differentiable vector sitting directly in the model’s embedding space, produced by a learned mapping function called a Token Maker in the Token Factory paper. The Token Maker is MLP-based: it takes heterogeneous input signals, which can be dense features like engagement rates, sparse features like category codes, or interaction sequences like click histories, and compresses them into a compact N-token representation. That representation lives in the same space as the model’s own vocabulary embeddings, so the model treats it as a native token without needing to parse language.

The practical difference is that you do not textualize the signal at all. No serialization step, no natural-language intermediary, no prompt-length explosion. The compression happens in the Token Maker, not in the token budget. The Token Factory paper frames this as the core motivation: conventional approaches cause excessively long prompts, large memory footprints, and high computational overhead. Soft tokens prevent that.

Why does the token format matter for cold-start?

Cold-start is the hardest problem in recommendation. A new item has no interaction history, so collaborative filtering has nothing to work with. Discrete semantic identifiers, the sequences of 2-8 tokens derived by hierarchically quantizing item content embeddings, handle this reasonably well when the identifiers are built from content signals like titles and descriptions.⁹ Spotify’s GLIDE system (arXiv:2603.17540) uses exactly this approach for podcast discovery: episode titles and descriptions generate semantic IDs for newly published shows immediately, before any streams have accumulated. Offline, it delivered +29.9% Recall@30 and +31.2% NDCG@30 versus semantic-ID-only baselines⁴, with +35.4% NDCG for unfamiliar content specifically.⁴

The problem is that discrete identifiers draw a hard line. A 2026 reproducibility study (arXiv:2603.29845) evaluated eight generative models across three datasets and found that item cold-start is substantially harder than user cold-start⁶, that semantic codes maintain warm-start strength but fail on unseen items⁶, and that increasing model size yields only limited improvement under cold-start conditions. The warm/cold performance gap remains large regardless of scale.

Soft tokens approach the problem differently. Because they are continuous and differentiable, you can train the Token Maker end-to-end: the mapping from raw signals to the embedding space is learned jointly with the recommendation objective, rather than fixed by a quantization step. Fresh content that lacks interaction history can still contribute content-based signals through the Token Maker. The 67.1% impression lift for one-day-fresh content in Token Factory is the evidence that this matters in practice; the mechanism is that new items become representable in the model’s space from day one, without waiting for interaction volume to accumulate.

UniGRec (arXiv:2601.17438) takes a related approach, using continuous probability distributions over codebook codewords rather than hard discrete assignment. It outperforms all baselines on Upwork, Beauty, and Pet datasets.⁵ The paper does not include explicit cold-start experiments, so that specific claim cannot be extended to UniGRec.

What does the +67.1% figure actually measure?

The number needs its full context. Token Factory reports a 67.1% increase in unique impressions for one-day-fresh content in generative retrieval, alongside training speedups of up to 200% compared to baseline approaches.² These statistics come from a secondary review of the paper at themoonlight.io; the paper’s HTML page returned a 404 during research, so they could not be confirmed directly from the paper’s own text. The paper’s abstract, which is accessible, validates the system in a production-scale recommendation environment without naming the platform.¹

What the number measures is coverage and reach for new content, not engagement quality, not revenue, not ranking quality downstream of retrieval. Satisfaction is described as flat-to-positive. That framing matters: a retrieval system that surfaces dramatically more fresh content is valuable because fresh items were previously undersurfaced, not because every fresh item is high-quality. The gain is distributional, not per-item quality. For any commerce application, that distinction changes the analysis.

Is there a path from recommendation retrieval to OPG and AI shopping surfaces?

My read is yes, and I label this as inference: the architecture that makes soft tokens effective for recommendation retrieval is the same architecture that makes items representable to generative retrieval systems more broadly. When a retailer’s product catalog contains thousands of new SKUs per week, the cold-start problem is not academic. Items that fail to accumulate interaction history fast enough never develop strong semantic IDs through collaborative filtering, which means they sit below the retrieval threshold for surfaces that use those identifiers. Soft token approaches, by grounding item representation in content signals from the first day, address the coverage gap directly.

ByteDance’s Next-User Retrieval framework (arXiv:2506.15267) is another data point: it generates lookalike users for cold-start items via prefix prompt embeddings using item ID and category, deployed on Douyin at 600 million plus daily users.⁸ The interaction gains are modest at the aggregate level, +0.0142% DAU, +0.1144% publications, but at that scale modest percentages represent enormous absolute counts. The pattern is consistent: soft or continuous item representations improve new-content reach measurably.

The connection to OPG and AI Mode citation pipelines requires a further inference step, and that step is where I want to be careful.

How does the recommendation pipeline differ from the AI Mode citation pipeline?

The recommendation pipeline operates on a closed candidate set: the platform knows its own inventory, scores items against user context, and retrieves from its own index. Token Factory and GLIDE both operate here. The retrieval problem is matching items to users within a walled garden, and the system controls both the item representations and the ranking signals.

The AI Mode citation pipeline, or any search-side generative surface, operates differently. The system has to decide which external URLs, which product pages, which content, to cite in response to a query. That decision is influenced by how well the page’s content signals translate into the model’s representational space, but the page does not control its own embedding in Google’s index. A retailer cannot push a soft token into Google’s recommendation model the way a platform engineer can configure a Token Maker for its own catalog.

What retailers can influence is the content signal quality that feeds into whatever representation Google derives from a crawl: structured data, attribute completeness, freshness signals, schema markup. The inference, and it is an inference, is that the architectural work happening in recommendation retrieval will eventually shape what signals the search-side systems weight when they build item representations for AI-surface citation. That connection is worth watching. It is not confirmed by any paper cited here.

What should practitioners take from the production deployments?

Three things are clear from the papers that are actually confirmed. First, the cold-start gap is real and documented: the 2026 reproducibility study found it is not solved by model scale alone, and content-based representations close it more reliably than interaction-volume-dependent approaches. For any catalog with high SKU churn, this matters now, in the recommendation surfaces where these systems are live.

Second, compute efficiency is not a secondary benefit. TokenMinds (arXiv:2606.25147), the YouTube-associated system, cuts training compute by 50% and serving compute by 31% through cross-scenario modeling³, and reduces storage for user tokens by 72% versus dense embeddings.³ Snapchat’s semantic ID deployment across ads ranking, dynamic product ads, friend recommendations, search, and short-form video retrieval shows the same pattern: the architectural gain is not just in retrieval quality but in the cost of running the system at scale.⁷

Third, the gains in fresh-content coverage compound with catalog size. A 67.1% impression increase for one-day-fresh content is most valuable when the catalog turns over fast and when the delta between fresh-item reach and established-item reach is largest. For e-commerce, that describes most large catalogs. The measurement gap to close is not whether soft tokens improve fresh-content reach; the production evidence for that is solid. The gap is whether the content signals feeding those Token Makers are rich enough per item to do the compression justice, and that is a catalog quality and structured data problem, not a modeling one.