Agents12 min read
AWS releases AgentCore Evaluations for multi-agent systems
Amazon Bedrock AgentCore adds a platform to evaluate agent performance and explainability in production.
From AWS machine learning blog
Loading
90 days of the enterprise, pro and team plans free, no card needed. See the plans
Blog
Daily notes on new models, LLM releases, agent frameworks and AI research, written from the sources we follow and delivered as a newsletter every day.
Get the morning and evening issues
Every new post of the day, in one email. Confirmation required.
Agents12 min read
Amazon Bedrock AgentCore adds a platform to evaluate agent performance and explainability in production.
From AWS machine learning blog
Agents2 min read
LangSmith Engine v2 introduces red teaming and automatic testing of fixes. It helps find issues early and speeds up agent development.
From LangChain blog
Photo: Yan Krukau
This article details four mechanisms – context budgeting, memory strategy, todo-state, and compaction – used within harness systems like LangChain Deep Agents and Claude Code to mitigate context overflow and goal loss in long-horizon agent tasks. These techniques involve offloading, summarization, and structured data management to improve agent performance.
From MarkTechPost
Research1 min read
Research assessed generative AI agent performance across repeated interactions and accumulating disruptions in simulated healthcare workflows. The study examined operational resilience and considerate participation, revealing distinct behavioral patterns as challenges increased and informing deployment dilemmas.
From arXiv cs.AI
Research1 min read
A new curation agent approach, ‘environment-probing,’ enhances existing agent memory systems by allowing the agent to verify and refresh its knowledge. This results in improved performance on benchmark databases and management tasks, reducing query costs and tool calls.
From arXiv cs.AI
Agents1 min read
The Agent Evaluation Metric (AEM) provides a new approach to assessing multi-turn agent performance by identifying the specific turn causing failures. This allows for a more granular understanding of agent quality and isolation of problematic interactions.
From AWS machine learning blog
LLMs1 min read
A new benchmark, HybridDeepResearch, assesses autonomous agent performance requiring both web search and SQL queries to solve complex analytical tasks. Evaluations show current state-of-the-art models struggle with directional reasoning when integrating structured and unstructured data.
From arXiv cs.CL
Research1 min read
Research demonstrates that using subagents – which execute in separate context windows – outperforms traditional agent-skill execution for long-horizon tasks. This approach reduces context window degradation and improves reasoning quality, but introduces communication overhead.
From arXiv cs.AI
Research1 min read
A new framework, State-Path Tool Menus, enhances online agent success by limiting tool choices to a curated menu based on a pre-defined state path. This approach, demonstrated on ToolBench, increases success rates from 0.737 to 0.898 without altering the agent.
From arXiv cs.AI
Research1 min read
A six-month study of two LLM trading agent fleets reveals significant operational factors impacting performance. The research highlights the importance of the operating layer, volatility-blind sizing, and the lack of directional edges in these autonomous agents.
From arXiv cs.AI
Agents1 min read
A GitHub Actions pipeline can be integrated with Amazon Bedrock AgentCore to automatically evaluate AI agent behavior. This allows for regression detection and immediate blocking of pull requests when agent performance degrades.
From AWS machine learning blog
Agents1 min read
AWS announced memory lifecycle policies for Amazon Bedrock AgentCore to manage outdated memories, reducing quality degradation and compliance risks through nightly scoring, consolidating, and pruning.
From AWS machine learning blog
LLMs1 min read
Hugging Face announced new memory capabilities for coding agents, enabling them to retain information across sessions. This enhancement aims to improve agent performance and context handling.
From Hugging Face blog
Agents1 min read
Podium decreased engineering intervention by 90% using LangSmith for agent testing and observability. The platform’s testing approach, including dataset curation and model fine-tuning, improved agent performance and enabled rapid issue resolution.
From LangChain blog
Agents1 min read
Reducing AI agent latency involves identifying bottlenecks, leveraging streaming responses, and strategically minimizing LLM calls. Tools like LangSmith and LangGraph offer visibility and control for optimizing agent performance and user experience.
From LangChain blog
Agents2 min read
LangChain and NVIDIA introduced the NemoClaw blueprint, combining Nemotron 3 Ultra, LangChain Deep Agents Code, and NVIDIA OpenShell runtime. This open system enables enterprises to optimize agent performance, reduce inference costs, and control agent deployments for improved governance and efficiency.
From LangChain blog
Research1 min read
Amazon scientists launched a new open-source harness to fix the gap between what AI models intend and what tools actually do. The system improves reliability across multiple benchmarks without needing specific tuning for each model.
From Amazon Science
Posts are drafted from public feeds by models OpenSmartRoute routes to - the same router, skill and metering customers use - and always link to the original source. Corrections: support.
Archive (100)
