Cheapest LLM for the Job Guide
When choosing a large language model (LLM) for a specific task, cost is often a key factor. This guide explains how to identify the most affordable LLM options suitable for different use cases, based on available models and their costs.
Understanding Cost Factors in LLM Selection
The cost of an LLM depends on several factors, including token pricing, model complexity, latency, and quality prior. Models with lower token costs are generally more economical, especially for high-volume tasks.
Available Models and Their Costs
Within the platform, different models are available with varying capabilities and costs. For example, some models are designed for quick, inexpensive responses, while others prioritize quality.
Example of a Cost-Effective Model
- HR helpdesk model: costs approximately 0.0002 USD per 1,000 tokens and offers basic capabilities like QA, classify, and summarize.
- People-data model: more complex, costs about 0.001 USD per 1,000 tokens.
How to Choose the Cheapest LLM for Your Task
To select the most affordable model:
- Define the task requirements: actions needed, complexity, and latency.
- Compare models based on their cost per 1,000 tokens and capabilities.
- Consider constraints like data boundary and PII allowance.
- Test models with sample prompts to evaluate quality versus cost.
Example Configuration for Cost Optimization
targets:
- id: hr-helpdesk-fast
kind: llm
name: HR helpdesk model
capabilities:
domains: [hr, general, general_chat]
actions: [qa, classify, summarize]
constraints: { pii_allowed: false }
cost: { usd_per_1k_tokens: 0.0002 }
latency_ms: 300
quality_prior: 0.55
Comparing Cost and Capabilities
| Model Name | Cost per 1k tokens | Capabilities | Latency (ms) |
|---|---|---|---|
| HR helpdesk model | $0.0002 | QA, classify, summarize | 300 |
| People-data model |
