The LangChain team identified a significant bottleneck in their support workflow: engineers spent considerable time resolving technical questions. This stemmed from a prototype chatbot that wasn’t actively used, highlighting a gap in how they approached troubleshooting. The team decided to address this by replicating the internal support process within the LangChain platform.
The core of the solution involved a three-step ritual: searching documentation, checking the knowledge base, and examining the codebase. This approach, combined with a Deep Agent, dramatically improved debugging efficiency. The agent autonomously performed these steps, synthesizing information and delivering detailed answers, citing relevant documentation, knowledge base articles, and code snippets. This internal agent system significantly reduced the time engineers spent on complex debugging tasks.
To bring this approach to the public Chat LangChain, the team adopted a strategy focused on combining two distinct architectures. Most questions could be answered using documentation and knowledge base resources. However, a subset of queries required analysis of the underlying code. The CreateAgent abstraction was chosen for speed, enabling rapid tool calls and immediate answers. This architecture leverages Claude Haiku 4.5, which demonstrates exceptional speed and accuracy when executing tool calls, achieving sub-15-second response times for documentation queries.
LangSmith played a crucial role in optimizing the Chat LangChain. By tracing conversations, the team identified unnecessary tool calls and refined prompts. A/B testing different prompting strategies, combined with LangSmith’s evaluation suite, led to improvements in both speed and accuracy. The system now delivers answers in 3-6 tool calls, primarily using Claude Haiku 4.5, and the entire process takes less than 30 seconds.
For tasks requiring code analysis, a Deep Agent with specialized subgraphs – one for documentation, one for the knowledge base, and one for codebase search – was implemented. This architecture allows for precise verification of implementation details and provides engineers with the necessary context to resolve complex issues. Source: https://www.langchain.com/blog/rebuilding-chat-langchain
