Answers from the documentation; checking which model is on duty
Hi - I answer from the OpenSmartRoute documentation: routing, the API, plans and quotas, self-hosting. Ask away, or open a support ticket if you need a person.
Grounded in the docs - follow a source before acting on it.
AI Models Often Cite Web Pages They Never Opened - OpenSmartRoute
Researchers found Large Language Models frequently answer using search snippets without reading the actual web pages. This behavior leads to outdated answers and misleading citations across multiple model types.
Key points
Models answered 2.6% to 76% of questions without opening any web pages.
Snippet-only answers were outdated 6.4% of the time versus 1.3% when pages were opened.
Eight model configurations were tested against facts changed since February 2026.
GPT-5.6 Luna answered 53.6% of episodes from search snippets alone.
Why it matters: Running agents that rely on search snippets risks outdated facts and broken citations.
By OpenSmartRoute editorial · written through the router by writer-small
From Unite.AI - “AI Often Cites Web Pages It Has Never Even Read”
Researchers found that AI models often ignore the actual web pages they are supposed to read. They answer questions using short snippets shown in search results instead. This behavior happens even when the models have access to the full text. The study shows this leads to outdated and misleading answers.
Large Language Models (LLMs) are AI systems that generate text based on patterns. They are trained on huge amounts of data up to a specific cutoff date. After that date, their knowledge does not automatically update unless they use tools. This study looked at how these models handle new information found on the web.
The authors came from Imperial College London in the United Kingdom. They published their findings in a new paper titled "Retrieved but Not Read". The paper investigates why search-augmented LLMs fail to use available evidence properly. Search-augmented means the model uses a search engine to find relevant information before answering.
The researchers wanted to understand why models cite URLs without reading the content. They found that models often throw supporting links at users without opening them. This happens with both proprietary and open-weight models. Proprietary models are closed-source systems like GPT-5.6 Luna. Open-weight models have public weights that anyone can download and run locally.
The team tested one proprietary model and six open-weight configurations. They used hundreds of facts that changed after the models were trained. These facts covered areas like FDA drug information, finance, and sports. They wanted to see if models relied on snippets or actual pages.
Tests showed that models frequently answered questions without opening any pages. This behavior was linked to higher rates of outdated answers. It also led to misleading citations across multiple model types. The gap between available evidence and its use was significant.
Even when retrieved evidence influenced internal predictions, models favored outdated info. This happened in their final answers despite newer facts being supplied. The controlled experiments revealed a specific gap in how models process evidence.
The authors measured answer accuracy and reliance on search snippets. They also checked how often citations contradicted the model's claims. In some cases, more than half of searches ended without a page being opened.
OpenAI released 719 math manuscripts from an unreleased frontier model. Mistral and Reflection AI launched new open-weight models to compete with Chinese systems.
The first row of data used live web search. The other rows used standardized search results. All figures in the study were percentages. The results highlighted a major issue with current AI search systems.
Supplying the correct evidence directly could reduce these failures. However, some models continued to favor outdated information. This happened even when newer facts were placed directly in front of them.
The two most obvious reasons for this behavior are cost and speed. Downloading and reading actual material costs bandwidth and time. Serving a wrong answer quickly is often better than delaying a better one.
The authors were more interested in the base mechanics of this shortfall. They wanted to understand the technology rather than the initial design. The economics of gaming citation behavior make sense to some observers.
The actual technologies are still unmapped and in a state of flux. It is difficult to prove motives at the moment due to these factors. The study focused on establishing the mechanics of the quality shortfall.
To test how well models use web evidence, researchers assembled 225 facts. These facts changed since February 16, 2026. This was the most recent reported training cutoff among the models tested.
The facts were drawn from various domains including FDA drug information. There were 62 facts about drugs, 52 about finance, and 47 about sports. People-related facts made up 43 of the total set.
SEC filings and chemical classifications made up the remaining facts. Changes that had already taken effect by the cutoff were excluded. Medical claims answerable from general knowledge were also excluded.
Unchanged facts were used as controls to distinguish failures. This allowed researchers to see if new information caused the errors. It helped separate new information failures from general errors.
To establish a baseline, GPT-5.6 Luna was tested without web search access. It correctly answered just 23.3% of 103 questions about the changed facts. This baseline showed how much search access improves performance.
An unspecified AI model rewrote FDA labeling clauses into testable statements. Each statement was then manually reviewed to ensure accuracy. This process ensured the facts were clear and verifiable.
The open-weight models tested included GLM-5.3-Flash from Z.ai. OpenAI's gpt-oss-120b and gpt-oss-20b were also tested. Google DeepMind's gemma-4-31b was tested with and without reasoning.
Qwen's qwen3-8b was another model included in the testing group. All models received the same web search results for fair comparison. Luna was also tested separately using its own built-in web search.
For 223 of the 225 facts, ten search results were collected in advance. Brave Search was used to gather these results. Every model received the same results and could open pages using identical tools.
The page containing the correct answer was always made available. If it appeared naturally among the top ten results, its position stayed unchanged. Otherwise, it was inserted at a position determined in advance.
This allowed researchers to examine how models used available evidence under comparable conditions. Consistency was key to making the tests fair and reliable.
To make the tests more consistent, temperature was set to zero. Temperature controls how random or creative the model's output is. Each model was run through the same provider throughout the experiments.
Of the two prompts used, the first asked an open question. Examples included asking which club a footballer plays for. The second asked the model to verify a statement containing current or outdated information.
This allowed researchers to compare how each version of the statement was treated. Four additional prompt types were used in the combined results. These included open questions with cues and requests for quotations.
Requests to provide a citation supporting a supplied claim were also tested. Fact-sheet checks asked the model to identify one outdated statement among several correct ones.
Every prompt specified a date and required answers in a fixed format. Most required answers in a fixed format that permitted a 'CANNOT VERIFY' response. This allowed results to be graded without human or LLM-based adjudication.
Metrics used were reading depth, answer accuracy, and citation faithfulness. Reading depth measured whether source pages were opened and cited. Answer accuracy was the percentage of correct answers excluding abstentions.
Citation faithfulness checked whether cited pages contradicted the model's claims. Grading was automated where possible with an unspecified LLM. Negative judgments requiring interpretation were assigned to human reviewers.
Unchanged control facts were used to establish each model's baseline citation error rate. Initial tests showed that access to web search improved the models' ability to answer questions correctly.
When checking whether factual statements were accurate, their scores rose by 27–42 percentage points. When answering questions directly, they gave correct answers in 85.8–95.4% of cases.
However, many answers were produced without models visiting the websites. They relied on the short extracts of text displayed alongside search results. Even when supplied with the same search results, the proportion of searches in which no page was opened varied from 2.6% to 76%, depending on the model.
GPT-5.6 Luna turned in a particularly low-effort performance in these tests. Given the same search results, models answer 2.6–76.0% of episodes without opening any page. Under its native search, Luna answers 53.6% of episodes from snippets alone.
Test results compared outdated answers and misleading citations according to how thoroughly models examined search results. Snippet means no web page was opened. Opened means a page was opened but not cited. Integr. means an opened page was cited.
All figures were percentages pooled across seven model configurations and all prompt types. Errors were generally more frequent when models relied solely on search snippets.
Across seven model configurations, 6.4% of snippet-only answers were outdated. This compared with 1.3% when opened pages were cited. Misleading citations were also more frequent at 6.4% versus 2.3%.
For sports questions, outdated answers among open-weight models reached 20.2% versus 2.1%. When answers were outdated, 81.8% of citations supporting those claims contradicted them.
Just 0.9% of up-to-date answers had contradictory citations. Similar patterns appeared in fact-sheet tests. This means AI models are more likely to give wrong answers when they rely on short descriptions.
Worse, they sometimes link to pages that prove their answers are wrong. Unless the selection used for these tests are 'collective outliers', it seems reasonable to assume other models are affected.
In fact, of nine types of failure observed in GPT-5.6 Luna's live web searches, seven were significantly associated with failures when other models were tested using the same fixed search results. This suggests the problems were not limited to Luna.
Providing the correct Wikipedia page directly reduced outdated answers. Failures fell from 14 to zero for gemma-4-31b. They fell from 11 to one for gpt-oss-20b and from 16 to eight for qwen3-8b.
Repeating the original question after each opened page also corrected 26.1% of previously failed fact-sheet checks. This shows that direct access to correct information is highly effective.
Finally, the researchers tested three open-weight models on footballers who had changed clubs. They used gpt-oss-20b, gemma-4-31b, and qwen3-8b for this specific test.
Each was given outdated information about a player's club, sometimes alongside the correct Wikipedia page. They examined to see which answer it favored during processing.
The results showed that models could recognize the correct club but still give the wrong answer. In qwen3-8b, the correct club sometimes took the lead internally before the outdated club overtook it just before the answer was produced.
A similar pattern appeared in gpt-oss-20b, while gemma-4-31b generally favored the 'outdated' club throughout. This indicates a specific failure in how models weigh conflicting information internally.
Therefore, it seems that models can process correct information but still output outdated answers. This happens even when the correct information is available and visible. The internal processing stages determine the final output.
This behavior matters for teams running agents today because it affects trust and accuracy. Agents rely on accurate data to make decisions for users. If an agent cites a wrong page, users may lose trust in the system.
Teams need to know when their search-augmented systems are failing silently. Silent failures happen when the model looks correct but the data is wrong. This is dangerous in fields like healthcare or finance.
Safety is a major concern when agents provide medical or legal advice. Outdated information can lead to harmful recommendations for patients or investors. Teams must ensure their systems verify information before presenting it.
Quality is another key factor that teams must monitor closely. Low-quality answers waste time and resources for both users and developers. Teams need clear metrics to track the quality of their AI outputs.
New capabilities in search are needed to fix these issues. Teams should look for tools that force models to read full pages. Some tools allow teams to set rules about which pages must be read.
OpenSmartRoute helps teams manage these risks by routing requests to the best-fit model. It sends each request to the best-fit model, agent, tool or skill from a catalogue the team defines.
The team scores every candidate on quality, cost, speed and safety. They set the weights per request to prioritize what matters most. Hard rules pin a request to specific constraints like data privacy or cost caps.
Text with personal data stays on an on-premises model to protect privacy. A region or a cost cap is never crossed by the routing logic. It learns from outcomes so a model that answers well gets more traffic.
One model that fails gets less traffic automatically. A new model is one catalogue entry and competes on the next request. Nothing else changes in the app when a new model is added.
It works with any OpenAI-compatible provider, with open-weight models served locally. It supports MCP tools and A2A agents for complex workflows. A savings ledger shows what each routed request cost next to what the most expensive model would have cost.
An input guard spots prompt injection and personal data before a request leaves the system. 'osr eval' measures routing accuracy on the team's own prompts and can fail a build when it drops.
The hosted platform keeps a models catalogue with prices and public rankings built from real traffic. This gives teams visibility into the performance and cost of every model they use.
Teams can check the catalogue to see which models perform best for their specific needs. They can compare prices and rankings to optimize their spending.
What to do when building search-augmented systems starts with defining clear evaluation criteria. Teams should test their models on facts that change frequently after training. They need to measure reading depth and citation faithfulness regularly.
Check if models open pages or just read snippets. If most answers come from snippets, the system needs improvement. Teams should consider adding rules that force page reading for critical queries.
Implementing hard rules can prevent outdated information from being used. Teams can set rules that require a page to be opened before an answer is generated. This ensures the data is current and accurate.
Use OpenSmartRoute to manage the complexity of routing requests. The router can enforce rules about which models handle sensitive data. It can also optimize costs by choosing the right model for each task.
Monitor the savings ledger to see how routing decisions impact costs. Look for patterns where cheaper models fail more often than expensive ones. Adjust the weights to balance cost and quality effectively.
Run 'osr eval' tests to ensure the routing system works as intended. If the evaluation drops, the build should fail to catch the issue early. This prevents bad routing from reaching production environments.
Teams should also review the public rankings in the models catalogue. This helps them understand the general performance of different models. They can pick models that have proven reliable in similar tasks.
By following these steps, teams can build more reliable search-augmented systems. They can reduce the risk of outdated answers and misleading citations. The goal is to create systems that users can trust completely.
How OpenSmartRoute helps
Teams using OpenSmartRoute avoid outdated answers by routing requests through an input guard. This guard spots prompt injection and personal data before a request leaves the system. It ensures sensitive information stays on an on-premises model where it belongs.
The router scores every candidate on quality, cost, speed and safety. It sets the weights per request so the best-fit model handles each task. A new model is just one catalogue entry that competes on the next request. Nothing else changes in the app when a model improves or fails.
The savings ledger shows what each routed request cost next to the most expensive option. This helps teams track spending while maintaining high accuracy. The platform keeps a models catalogue with prices and public rankings built from real traffic.
SpaceXAI released Grok Voice 3.0, which scored 86.9 on the Artificial Analysis Speech-to-Speech Index. It is the top-ranked model for speech-to-speech tasks.
