Look at and label your data, build and evaluate your LLM-evaluator, and optimize it against your labels.
Read the original at Eugene Yan: AlignEval: Building an App to Make Evals Easy, Fun, and Automated
Loading
90 days of the enterprise, pro and team plans free, no card needed. See the plans
LLMs1 min readarchive
Look at and label your data, build and evaluate your LLM-evaluator, and optimize it against your labels.
By OpenSmartRoute editorial · attributed excerpt
From Eugene Yan
This post is in the archive: it was published on Oct 27, 2024 and the models, prices or APIs it describes may have changed since. See the latest posts.
Look at and label your data, build and evaluate your LLM-evaluator, and optimize it against your labels.
Read the original at Eugene Yan: AlignEval: Building an App to Make Evals Easy, Fun, and Automated
Keep reading
LLMs1 min read
Microsoft will scale its Maia accelerator family to multi-gigawatt deployments from 2027, making it the primary inference system for its own products. Maia 300 collapses scale-up and scale-out onto one Ethernet fabric and moves KV state ...
LLMs1 min read
Imagine building an AI agent that has to decide whether a customer complaint needs human intervention, which department should handle a query, or whether a submitted delivery photo is valid. Getting these decisions right matters. A false...
LLMs8 min read
A Harvard study shows AI tools increase code lines but fail to boost completed work due to review bottlenecks.