LLMs8 min read
AI Code Volume Up But Productivity Stays Flat
A Harvard study shows AI tools increase code lines but fail to boost completed work due to review bottlenecks.
From InfoQ: AI, ML and data engineering
Loading
90 days of the enterprise, pro and team plans free, no card needed. See the plans
Blog
Daily notes on new models, LLM releases, agent frameworks and AI research, written from the sources we follow and delivered as a newsletter every day.
Get the morning and evening issues
Every new post of the day, in one email. Confirmation required.
LLMs8 min read
A Harvard study shows AI tools increase code lines but fail to boost completed work due to review bottlenecks.
From InfoQ: AI, ML and data engineering
AI8 min read
SandboxAQ launched Switch Trust and Switch ROI to help companies manage risk and measure value for their AI agents. These tools extend the open-source Switch Rooms platform to provide security checks and cost tracking.
From Unite.AI
Photo: Yan Krukau
Nick Cammarata claims neural networks are clean and interpretable. Meta released Muse Spark Open, a new US open-weight model. Other models like Nemotron 3.5 and Gemini 4 Argon launched recently.
From Latent Space
Agents1 min read
Google AI developers now support evaluating live voice agents directly inside the ADK framework.
From Google AI developers blog
Agents2 min read
Jev is a new model that evaluates agents by giving typed answers instead of generating text. It is faster and costs less than traditional methods.
From LangChain blog
LLMs1 min read
Together AI released a playbook to help companies migrate from closed to open source models.
From Together AI blog
LLMs1 min read
Research reveals a significant disconnect between LLM-as-a-judge rankings and actual task success in user-simulated evaluations of task-oriented agents. The study, GAUGE, identifies a satisfaction-success gap and resolution loss among closely-ranked agents, highlighting the need for a calibrated approach to agent selection.
From arXiv cs.CL
Research1 min read
Researchers developed WinSyn, an automated pipeline for generating realistic synthetic datasets of emails simulating enterprise scenarios. Evaluations using standard agentic baselines on these datasets showed aggregate scores below 80%, highlighting the need for more complex evaluation data.
From arXiv cs.AI
AI1 min read
Anthropic CEO Dario Amodei outlined a plan involving embedded evaluators, coordinated safety standards, and global coordination to slow AI development, mirroring OpenAI’s intentions. The strategy aims to mitigate risks and ensure responsible advancement.
From TechCrunch AI
Research1 min read
Research demonstrates a new approach, Debate-to-Skill, for annotating query-to-agent interactions by focusing on executable capability rather than simple topical relevance. This method achieves better results compared to existing techniques on an industrial benchmark, particularly in complex scenarios.
From arXiv cs.AI
Research1 min read
Research identifies a limitation of range-based confidence gates in continual learning for embodied agents, preventing useful updates. A new admission-audit protocol, incorporating historical-reference promotion and missed-opportunity metrics, offers a more robust approach to managing update streams.
From arXiv cs.AI
Research1 min read
The Agent Incident Registry (AIR) is a new, source-linked catalog containing over 10,000 records of AI agent failures. It provides detailed information and labels for agent-related events, supporting case retrieval and evaluation-scope auditing.
From arXiv cs.AI
Research1 min read
A new survey defines AI agent capabilities across five dimensions – environmental interaction, learning, autonomy, goals, and temporal coherence. The resulting Agent Compendium provides a structured resource for evaluating and comparing AI agents, promoting reproducibility and clearer research.
From arXiv cs.AI
Research1 min read
A new method, belief-shift branching, uses model belief divergence to strategically place forks in tree-structured reinforcement learning chains. This approach, validated against multiple models and benchmarks, achieves significant performance gains, particularly in code generation tasks.
From arXiv cs.AI
Agents1 min read
This post details a two-layered approach to monitoring production agent systems. It combines Amazon Bedrock AgentCore Evaluations for continuous quality scoring with AWS DevOps Agent for autonomous infrastructure investigation, using a four-agent airline reservation system as an example.
From AWS machine learning blog
Agents1 min read
The Agent Evaluation Metric (AEM) provides a new approach to assessing multi-turn agent performance by identifying the specific turn causing failures. This allows for a more granular understanding of agent quality and isolation of problematic interactions.
From AWS machine learning blog
LLMs1 min read
Research reveals that large language models exhibit minimal disagreement with factual input when the context appears plausible, even when it contains counterfactual or fictional elements. This suggests a weaker-than-expected context-memory conflict in LLMs.
From arXiv cs.CL
Research1 min read
Researchers introduced Cros, a risk-constrained stopping layer for clinical diagnosis agents, achieving improved exploratory performance and reduced error rates compared to existing methods. The system provides a finite-sample guarantee and offers insights into autonomous stopping reliability.
From arXiv cs.AI
Research1 min read
Research simulating diverse LLM agent societies reveals significant value drift, with over 50% of personas failing to initially align with assigned WVS profiles and a 2-7% drift after conversations. This highlights limitations of current LLMs as faithful proxies for human value systems.
From arXiv cs.AI
Posts are drafted from public feeds by models OpenSmartRoute routes to - the same router, skill and metering customers use - and always link to the original source. Corrections: support.
Archive (100)
