Imported from Atharva150/RAG-clone (AGENTS.md). Install upstream with npx skills add Atharva150/RAG-clone. Copyright stays with the author.

AGENTS.md

Project

Movie Retrieval-Augmented Generation (RAG) Search Engine

Purpose

This repository is an educational and production-oriented implementation of Retrieval-Augmented Generation built from first principles.

The primary goal is to understand Information Retrieval, search algorithms, embeddings, ranking, retrieval pipelines, and RAG architecture.

This repository should remain educational while gradually evolving into a production-ready AI application.

Development Priorities

Always work in this order.

1. Keep existing functionality working.
2. Build missing features.
3. Refactor duplicated code.
4. Improve architecture.
5. Improve retrieval quality.
6. Optimize performance.

Never sacrifice correctness for optimization.

Existing Features

The repository already contains implementations of:

• Inverted Index
• Tokenization
• Stopword Removal
• Stemming
• TF
• IDF
• TF-IDF
• BM25
• Semantic Search
• Sentence Transformers
• Embedding Generation
• Embedding Cache
• Cosine Similarity
• Chunking
• Semantic Chunking
• Hybrid Search
• Reciprocal Rank Fusion
• Query Enhancement
• Re-ranking
• Multimodal Search
• LLM Integration

Preserve these implementations unless explicitly instructed otherwise.

Coding Principles

Prefer improving existing code instead of rewriting it.

Avoid unnecessary abstractions.

Avoid duplicate logic.

Write modular code.

Keep functions focused.

Separate business logic from interfaces.

Keep type hints.

Follow PEP8.

Use descriptive variable names.

Architecture

CLI should only:

• parse arguments
• call services
• print results

Business logic should never print.

Business logic should return structured objects.

FastAPI routes should only call services.

Frontend should never directly implement retrieval logic.

Before Writing Code

Always:

1. Read relevant files.
2. Understand dependencies.
3. Produce a short implementation plan.
4. Explain architectural decisions.

Do not make major architectural changes without approval.

Testing

After every implementation:

Run relevant tests.

Verify affected CLI commands.

Verify FastAPI routes if applicable.

Do not mark work complete if tests fail.

Refactoring

When refactoring:

Preserve behavior.

Explain every significant change.

Avoid changing public APIs unless requested.

Remove dead code only after confirming it is unused.

Search Quality

When improving retrieval:

Prefer measurable improvements.

Explain why retrieval quality should improve.

Avoid changing retrieval algorithms without justification.

Performance

Avoid premature optimization.

Measure before optimizing.

Cache expensive computations.

Reuse loaded models.

Avoid repeated disk I/O where practical.

Future Roadmap

Preferred implementation order:

1. Refactor services
2. FastAPI backend
3. React frontend
4. Chatbot
5. Document upload
6. Vector database
7. Authentication
8. Streaming responses
9. Evaluation dashboard
10. Production deployment

Communication

When completing work:

Explain:

• files modified
• why they changed
• trade-offs
• possible improvements

Do not continue to the next phase automatically.

Wait for approval.

Important

This project exists primarily for learning.

Readable code is preferred over clever code.

Whenever introducing a new concept, include concise comments explaining its purpose if it is not already obvious.