AI inference costs dropped rapidly between early 2023 and today. GPT-4-class capabilities now cost under $1 per million tokens. Some providers push prices below $0.10 per million tokens. Inference prices fell a median of 50 times across benchmarks.
Agents Need New Data Systems
Agents will soon dominate data system workloads. Swarms of agents respond to every user request. Existing systems struggle with agent state and coordination. Researchers propose three new challenges for data engineering.
Optimizing Agentic Speculation
Agents perform high-volume, heterogeneous work streams. A single request may involve thousands of SQL queries. Only 10-20% of sub-plans are distinct in benchmarks. Data systems can reuse results across overlapping plans. Approximate answers help agents make progress faster. Higher-level primitives could replace explicit SQL lists.
Proactive Data Systems
Data systems can steer agents without strict SQL results. Systems can provide latency estimates before execution. Materialized views cost less than agent-authored queries. Agents accept textual feedback instead of only query results.
Source: http://bair.berkeley.edu/blog/2026/07/07/intelligence-is-free-now-what/
