EvoOntology Automates Semantic Layers for Data Agents

EvoOntology replaces manual schema mapping with a three-tier architecture that manages how data agents interpret relational databases. The schema layer stores raw metadata, the content layer maps natural language intent to specific columns, and the tool layer defines functions for data retrieval. Instead of agents parsing raw SQL schemas or relying on static mappings, they query the EvoOntology server to resolve natural language intents into actionable database queries. This offloads the work of keeping metadata current when the database structure shifts, as the system autonomously discovers and corrects mappings.

The Mechanism of Continuous Refinement

The system operates through an attribution-guided typed edit process. When a discrepancy appears between user intent and the database structure, the builder agent analyzes the SQL execution failure or the low confidence score returned by the LLM during query generation. This attribution signal pinpoints the specific semantic gap, such as an unrecognized term in the prompt. The agent then proposes an edit that is typed against the schema. It maps natural language tokens to database constraints found in the JSON lookup table, ensuring the proposed change adheres to existing data types. This prevents the agent from guessing arbitrary synonyms and forces alignment with the actual table schema.

Updates are validated through a backbone-conditional paired evaluation. The builder agent executes the query across four distinct LLM backbones to determine if the proposed mapping improves retrieval accuracy. The system compares the SQL output generated before and after the mapping change by running both against a sandbox database instance containing historical query-result pairs. An update is committed to the server only if the evaluation shows a reduction in execution errors on the target subset. This prevents any single model from introducing faulty logic into the production layer.

Performance and Scaling Constraints

In benchmarks across the Spider, BIRD, and WikiSQL datasets, EvoOntology demonstrated improved execution accuracy compared to static semantic layers. By moving the mapping logic to the server, the agent reduces the amount of metadata sent per query, as it pulls only relevant schema details rather than the entire database structure.

Consider a user querying a product inventory database. If the agent fails to link the user's term 'popular items' to the 'sales_rank' column, the system identifies the failure by observing a SQL syntax error or a null result set. It then performs a lookup in the schema metadata to find columns related to frequency or ranking. Once it identifies 'sales_rank' as the best candidate for the user's intent, it updates the content layer mapping. Subsequent queries then succeed without repeating the initial discovery phase.

The primary technical constraint is the complexity of the four-model consensus loop. Each validation cycle requires concurrent calls to multiple LLMs and sandbox executions, creating a computational bottleneck. This overhead means that while the system excels at refining specific, recurring queries, it is not optimized for environments requiring instantaneous, high-volume schema updates across massive, rapidly shifting databases. It remains unclear how this architecture performs when the underlying database contains high-cardinality values that shift faster than the agent can trigger validation cycles. The reliance on a sandbox of curated historical pairs also limits its effectiveness for novel schemas where such ground-truth data does not yet exist.