AtScale, a leading provider of the universal semantic layer for enterprise AI and analytics, has released research showing that adding its semantic layer doubled AI accuracy on business questions on average 1.7x to 2.4x, with gains on all seven AI models tested. The findings demonstrate how shared business definitions and governed computation can help enterprises improve AI reliability as they expand adoption across finance, sales, and operations.
AtScale research found that adding its semantic layer doubled AI accuracy on business questions on average 1.7x to 2.4x, with gains on all seven models tested.
Leading models answered fewer than four in ten business questions correctly when querying the database directly, even with schema, column documentation and a business glossary provided.
The research used BIRD-Interact, an independently developed benchmark with 600 tasks across 22 databases, including 410 business questions.
With the AtScale semantic layer, Kimi K2.5 scored within 1.2 points of OpenAI's GPT-5 on business questions.
On the 317 business questions with a verifiable answer, Claude Sonnet 5's accuracy rose from 46% to 93%.
Google's Gemini 3.1 Pro improved from the lowest pass rate on follow-up questions at 10.2% to leading every model at 45.6%, a 345% lift.
As businesses race to adopt the latest AI models, many overlook a more fundamental question: does the AI understand what their business data actually means? Without that understanding, even the strongest model has to guess at definitions like revenue, margin, and active customer. In AtScale's research, leading models answered fewer than four in ten business questions correctly when they queried the database directly, even with the schema, column documentation and business glossary provided. That gap is what keeps AI out of the financial and operational workflows where the numbers have to be right.
AtScale conducted the research using BIRD-Interact, an independently developed benchmark with 600 tasks across 22 databases. 410 of the tasks are business questions, and 317 of those have a verifiable correct answer. Its ambiguous questions, purposefully messy data, and simulated-user conversations reflect challenges encountered in enterprise AI workflows.
"Large language models are probabilistic by design. Finance and operations demand consistent, auditable answers. That mismatch is one of the biggest barriers to putting enterprise AI into production. A semantic layer bridges the gap: the model reasons about the question, while governed business definitions and deterministic computation produce the answer. That's how AI agents move from experimentation into the workflows that run the business, and how model providers unlock the next wave of enterprise adoption," said Jay Schuren, President, AtScale.
The benchmark reveals a path to 100% accuracy. BIRD-Interact tests the four places where AI guesses on real business questions. The results point to a framework that addresses each one: disambiguating prompts, resolving vague semantics, handling dirty data, and calculating complex multi-pass metrics.
For customers, the value extends beyond a benchmark score: a shared definition of revenue across finance and sales, consistent calculations across agents and dashboards over time, and the freedom to adopt new AI models without rebuilding business logic. AtScale gives enterprises a governed foundation for moving AI into everyday decisions, with a repeatable way to measure and improve accuracy as their business evolves.
The research supports AtScale for AI, announced separately today, which brings together semantic modeling, question clarification, governed computation, and accuracy evaluation. Powered by AtScale's AI Computation Engine, the suite helps enterprises extend trusted business metrics into AI workflows and continuously evaluate the answers those workflows produce.
The preliminary, AtScale-reported findings use the best of three runs per model with the AtScale semantic layer, following the BIRD-Interact leaderboard convention, compared with one run per model without it. Results on the 317 answerable questions apply the same grading corrections to both approaches, compare against the average of three runs without the semantic layer, and are not comparable to the leaderboard's success rate. The full methodology is available in the Benchmark eBook.
About AtScale
AtScale helps enterprises turn data into trusted answers for AI and analytics through a universal semantic layer that delivers consistent business metrics across models, agents, applications, and dashboards.
