Conversational Analytics in BigQuery just went generally available, and GA is a support milestone, not an accuracy guarantee

Google made Conversational Analytics in BigQuery generally available on June 30, 2026, letting a user type a question in plain English and get back an answer, a reasoning trace, and the SQL the tool generated, querying native BigQuery tables, Iceberg lakehouse tables, and cross-cloud sources including Databricks Unity Catalog, AWS Glue, SAP, and Salesforce.¹ General availability marks a support and production-readiness milestone. Google’s own documentation for this exact product admits its accuracy is not guaranteed, naming “reduced reasoning accuracy” as a known behavior once queries get large.³ Natural-language-to-SQL is also a benchmarked, measurably hard problem in its own right. The question a GA badge cannot answer for a practitioner is how often this tool’s generated SQL will diverge from the query they would have written by hand, on their own data.

GA marks Google’s support and stability commitment. The top system on the BIRD text-to-SQL benchmark still executes only 81.95% of queries correctly, against a 92.96% human baseline.⁴

What does it mean that Conversational Analytics is “GA”?

Conversational Analytics in BigQuery is a Google Cloud feature that answers a natural-language analytical question by generating SQL against a dataset, running it, and returning the result alongside its reasoning trace. Google’s announcement describes it as “an agent that behaves like an analyst who knows your business, thinks before it answers, and stands behind its work,” and it reached general availability on June 30, 2026.¹

The tool queries native BigQuery tables, Iceberg lakehouse tables managed outside BigQuery’s own storage, and cross-cloud sources including Databricks Unity Catalog, AWS Glue, SAP, and Salesforce, so the natural-language layer sits over data that never had to move into BigQuery first.¹ It can also call BigQuery’s built-in AI functions directly from a chat prompt: AI.KEY_DRIVERS for root-cause analysis, AI.FORECAST to project a trend, and AI.DETECT_ANOMALIES to flag an outlier.¹ Google ships a scheduled version too, an autonomous agent that runs on a cadence such as a Monday-morning business report and posts its findings without a person triggering the run.¹

That is a genuine capability jump, and it is also exactly why the GA label is worth interrogating rather than taking as a green light. A tool a non-analyst can point at a live metric, unattended, on a schedule, is only as trustworthy as the SQL it silently writes on each run.

Does GA say anything about whether its answers are correct?

General Availability says nothing about output correctness. Google Cloud’s own launch-stage definitions describe GA strictly in terms of customer access and support: products “are open to all customers, ready for production use, and covered by a Google Cloud SLA, where applicable,” and are typically supported “through APIs, CLIs, and the Google Cloud Console.”² Nothing in that definition addresses whether a given output, or a given generated query, is accurate.

Preview, the stage before GA, gets a similar kind of treatment. Offerings there are “ready for testing by customers,” “not necessarily feature-complete,” carry no SLA or technical-support commitment, and are “intended for use in test environments only,” typically for about six months.² Read the two definitions side by side and the axis they describe is support and stability: whether Google will keep the feature running, answer a support ticket, and honor an uptime commitment. Correctness of any individual answer sits on a different axis entirely, one these definitions never touch.

Google’s own documentation for Conversational Analytics confirms the gap directly rather than leaving it to inference. The product’s known-limitations page states that “querying large amounts of data can cause reduced reasoning accuracy in data agents,” and separately notes that the API “may automatically retry the operation by generating a corrected query” after a failed validation or execution.³ A vendor whose own documentation names an accuracy-degradation mode, and describes retrying with a regenerated query after a failure, is not claiming its SQL comes out right every time. My read: GA and correctness are two independent axes here. A feature can be fully supported, covered by an SLA, and still generate a wrong query on a given run, and nothing in Google’s own materials claims otherwise.

How accurate is natural-language-to-SQL, really?

Text-to-SQL, converting a natural-language question into a working database query, is a distinct, long-studied machine learning problem, and BIRD is the benchmark researchers built specifically to stress-test it under realistic conditions: large, messy real-world databases, external domain knowledge a question assumes but never states, and SQL efficient enough to run against 33.4 GB of data spread across 95 databases.⁵

When BIRD was introduced in 2023, the strongest system tested at the time, ChatGPT, reached only 40.08% execution accuracy (the share of questions where the query’s results matched the correct answer), against a human baseline of 92.96% on the same questions.⁵ Progress since has been real. Checking BIRD’s live leaderboard on July 24, 2026, the top-ranked entry, “AskData + GPT-4o” from AT&T’s CDO-DSAIR team, dated September 25, 2025, reaches 81.95% execution accuracy, against the same 92.96% human baseline recorded on the leaderboard.⁴ Two years of dedicated research roughly doubled accuracy on this benchmark, and the best system on record still misses close to one in five questions and still trails human performance by about eleven points.

None of this is a measurement of Conversational Analytics itself. Google has not published, and I could not find, a BIRD or Spider score for its own product, and BIRD’s leaderboard is built from systems entered for that specific competition rather than from arbitrary commercial tools.⁴ What BIRD establishes is the honest ceiling on the general problem: turning an open-ended natural-language question into exactly the right SQL, against a real, messy schema, remains unsolved even for systems built for no other purpose than winning this exact benchmark. A general-purpose conversational agent built for a much broader job than a text-to-SQL leaderboard has no documented reason to clear a bar the specialists have not cleared yet.

How would you test Conversational Analytics on your own data?

Test Conversational Analytics with a divergence test: a fixed protocol that compares the tool’s generated SQL against SQL you already know is correct, on a dataset you already understand, rather than leaning on the GA label or a general benchmark. Pick a real dataset, write a set of analytical questions you already know the right answer to, and hand-write correct SQL for each question first.

Run every question through Conversational Analytics and capture both its answer and the SQL it generated. What actually matters is execution: does running that generated query return the same result set as the hand-written query, row for row? Whether the generated query text merely looks reasonable is beside the point. Count the share of questions where the two result sets disagree. That share is the divergence rate, the single number this exercise exists to produce.

The count alone will not tell you much. Each divergence is worth tagging by failure type, because the categories point to different fixes. A wrong join grain silently duplicates or collapses rows by joining at the wrong level of detail. A silently dropped filter runs cleanly and simply omits a condition the question implied, so the query executes without error and the number is still wrong. A wrong date window shifts a reporting period by a day, a week, or a fiscal boundary, which is easy to miss because the result still looks like a plausible number. Logging the failure type alongside the failure count turns a pass or fail grade into a map of exactly where the tool still needs a human check.

Should you let it near a client-facing metric yet?

Conversational Analytics is not yet safe to run unattended on a number a client will see without a human checking it first. It suits exploratory, analyst-assisted work well, where a person reads the generated SQL before repeating the question elsewhere, better than it suits an unattended, scheduled agent writing straight into a client-facing report.

The decision rule below is my own synthesis, built from the evidence in this piece:

If this is true	Then the move is
GA means Google supports and stands behind the product’s continued availability	Treat the GA badge as clearance to adopt the tool, and keep verifying individual outputs regardless
Text-to-SQL tops out around 82% execution accuracy on the hardest public benchmark, even for purpose-built systems	Expect a real, nonzero error rate from any natural-language-to-SQL tool, including this one, until you measure your own
Google’s own documentation names reduced reasoning accuracy as a known behavior on larger queries	Keep the scheduled, unattended agent mode away from anything a client sees directly until a human review step sits between the agent and the report
The divergence rate on your own data is still unmeasured	Run the divergence test above before the tool touches a metric anyone outside your team relies on

This is the same discipline the site’s read on Google’s TabFM applied to a different BigQuery AI launch: a vendor’s benchmark chart or a launch-stage label is a claim worth checking yourself, before it becomes the basis for a client-facing number. Conversational Analytics is a genuinely useful front door onto a warehouse for people who do not write SQL, and GA means Google will keep that door open and staffed. Whether what walks through it is correct stays the practitioner’s question to answer, one dataset at a time.