Addressing PR Backlogs with Automated Systems
Leading open-source projects are adapting to a surge in contributions by adopting a new approach to managing pull requests. Projects like Vercel’s AI SDK, Astro, and tldraw are moving away from accepting external pull requests, opting instead to utilize internal agent systems for development and maintenance. This shift is driven by the volume of requests and the desire for greater control over the project’s development lifecycle.
Vercel’s AI SDK project, which receives over 20 million npm downloads weekly, implemented a ‘software factory’ to address a backlog of over 1,000 open issues and nearly 800 pull requests. This system employs a team of agents that handle various stages, including bug reproduction, fix implementation, and code review. The system’s architecture includes a UI, web app, API, execution space, and sandboxes, synchronized with GitHub to trigger automated actions. Data indicates a 25-35% contribution rate and a 70-80% issue closure rate within four weeks of implementation.
Similar strategies are being adopted by Astro, a web framework with 62,000 stars on GitHub. Creator Fred Schott implemented a ‘software factory’ to manage issues, reducing the number of open issues and shifting the team’s approach to issue handling. The Astro system utilizes ‘auto-triage’ to prioritize issues and automate the initial stages of the workflow. This has led to the creation of a new agent framework, Flue.
Flue, developed by Fred Schott, takes a more radical approach, automatically closing external PRs and converting them into issues or discussions. The project leverages state-of-the-art LLMs alongside the team’s expertise to guide decisions and prioritize work. This approach treats incoming requests as leads, rather than tasks for a maintainer to review. tldraw, a React drawing tool with 50,000 stars, has also adopted this strategy, closing external PRs and prioritizing internal agent workflows.
