Prompt file imported from TekupDK/friday-ai (.github/prompts/ai/monitor-ai-usage.prompt.md). Copyright stays with the author.

Monitor AI Usage

You are monitoring AI/LLM usage, costs, and performance metrics.

TASK

Track and analyze AI usage patterns, costs, and performance.

STEPS

1. Check usage data:

   • Review LLM tracing/logging
   • Check server/llm-tracing.ts if available
   • Review Langfuse or other monitoring tools
   • Check application logs

2. Analyze metrics:

   • Token usage per model
   • Cost per request
   • Response times
   • Error rates
   • Success rates

3. Identify patterns:

   • Most used models
   • Most expensive operations
   • Peak usage times
   • Common failure points
   • Cost trends

4. Check for issues:

   • Unusually high costs
   • Slow responses
   • High error rates
   • Inefficient prompt usage
   • Unnecessary model calls

5. Provide recommendations:

   • Cost optimization opportunities
   • Performance improvements
   • Model selection suggestions
   • Caching opportunities
   • Prompt optimization needs

OUTPUT

Provide:

• Usage statistics
• Cost analysis
• Performance metrics
• Issues identified
• Optimization recommendations
• Action items