Instruction file imported from krizzo101/arxiv-py-enhanced (.cursor/rules/3225-llm-output-validation.mdc). Copyright stays with the author.

llm-output-validation

1.1.0

Metadata

{ "rule_id": "3225-llm-output-validation", "taxonomy": { "category": "LLM Integration", "parent": "LLM IntegrationRule", "ancestors": [ "Rule", "LLM IntegrationRule" ], "children": [ "3226-llm-output-validation-advanced", "3227-llm-output-validation-basic" ] }, "tags": [ "output-validation", "llm", "integration", "quality-assurance" ], "priority": "75", "inherits": [ "000", "020", "030", "1000", "1005" ] }

Overview

{ "purpose": "MUST ensure that the output generated by LLMs is validated TO confirm its accuracy and reliability BEFORE being presented to users.", "application": "SHOULD be applied WHEN an LLM generates responses, particularly in contexts WHERE accuracy is critical, such as code generation or natural language processing tasks.", "importance": "This rule MATTERS because inaccurate LLM outputs can lead to user confusion, misuse of information, and potential errors in application functionality, ultimately affecting user trust and system integrity." }

input_validation

{ "description": "MUST validate the input received by the LLM TO ensure it is appropriate for the expected output.", "requirements": [ "Input data MUST be checked for format correctness, including type and structure.", "Any input containing sensitive data MUST be filtered or anonymized BEFORE processing.", "Input MUST NOT exceed predefined size limits TO prevent performance issues." ] }

output_verification

{ "description": "MUST implement mechanisms TO verify the accuracy and reliability of the output generated by the LLM.", "requirements": [ "Output generated by the LLM MUST undergo automated checks against known valid responses.", "Confidence scores provided by the LLM MUST be analyzed TO determine if the output meets a defined threshold.", "Any output that fails validation MUST be logged and flagged for review by a human operator." ] }

error_handling

{ "description": "MUST define clear procedures for handling validation errors and providing user feedback.", "requirements": [ "When validation fails, the system MUST inform the user of the failure clearly and provide suggestions for correction.", "A fallback mechanism MUST be in place TO allow for alternative outputs or explanations in case of validation failures.", "All errors MUST be documented for further analysis TO improve LLM performance over time." ] }

import openai
import logging
import time

# Configure logging for error tracking
logging.basicConfig(level=logging.INFO)

# Constants for API configuration
API_KEY = 'your_openai_api_key'
MODEL = 'gpt-3.5-turbo'
VALIDATION_THRESHOLD = 0.85

# Function to validate input before sending to LLM
def validate_input(prompt: str) -> bool:
    if not isinstance(prompt, str) or len(prompt) == 0:
        logging.error('Invalid input: Prompt should be a non-empty string.')
        return False
    return True

# Function to validate the output from LLM
def validate_output(output: str, confidence: float) -> bool:
    # Example of a simple validation check
    if len(output) < 5 or confidence < VALIDATION_THRESHOLD:
        logging.error('Output validation failed: low confidence or invalid length.')
        return False
    return True

# Function to call the OpenAI API
def call_openai_api(prompt: str) -> str:
    try:
        response = openai.ChatCompletion.create(
            model=MODEL,
            messages=[{'role': 'user', 'content': prompt}],
            api_key=API_KEY
        )
        return response.choices[0].message['content'], response.choices[0].finish_reason
    except Exception as e:
        logging.error(f'API call failed: {e}')
        return None, None

# Main function to generate response and validate it
def generate_and_validate_response(user_prompt: str) -> str:
    if not validate_input(user_prompt):
        return 'Invalid input provided. Please try again.'

    output, finish_reason = call_openai_api(user_prompt)
    if output is None:
        return 'Error occurred while generating response. Please try again later.'

    # Simulate a confidence score for validation (normally would come from LLM)
    confidence_score = 0.9  # Example fixed value for demonstration purposes

    if not validate_output(output, confidence_score):
        return 'The generated output is not valid. Please try again or provide more context.'

    return output

# Example usage
if __name__ == '__main__':
    # User prompt
    prompt = 'What are the key benefits of using Python for data science?'  
    response = generate_and_validate_response(prompt)
    print('Response:', response)

This Python example demonstrates how to implement LLM output validation according to the specified rule ID 3225. The program starts by defining a logging configuration for error tracking, ensuring that any issues during execution are captured for analysis.

1. Input Validation: The validate_input function checks if the input prompt is a non-empty string, which is essential to prevent sending invalid requests to the LLM. This is crucial for maintaining the integrity of the inputs sent to the model.

2. API Call with Error Handling: The call_openai_api function interacts with the OpenAI API. It wraps the API call in a try-except block TO handle potential exceptions gracefully, logging any errors encountered and returning None to indicate failure.

3. Output Validation: The validate_output function checks the generated output for validity based on length and a simulated confidence score. This mimics a scenario WHERE the LLM may provide a confidence score, which is then used TO determine whether the output is reliable. If validation fails, the function logs the issue.

4. Main Logic: The generate_and_validate_response function integrates these components, first validating the input, then making the API call, and finally validating the output. If any step fails, it returns a user-friendly message.

5. Usage Example: In the main section, a sample prompt is defined, and the response is generated and printed. This demonstrates how to utilize the validation framework in a real-world scenario, ensuring that all outputs are vetted for accuracy and reliability before reaching the user.

Overall, this example illustrates best practices in LLM integration by emphasizing input and output validation, robust error handling, and logging, all of which are critical for maintaining user trust and system performance.