zotero-rag

Imported from zotero-rag/zotero-rag (AGENTS.md). Install upstream with npx skills add zotero-rag/zotero-rag. Copyright stays with the author.

Information for Coding Agents (Claude Code, Codex, Pi, etc.)

This is a Rust-based Zotero RAG QA System for answering questions from academic libraries.

Project Structure

• zqa-pdftools: PDF parsing and text extraction utilities. Highly focused on zero-cost abstractions and zero-copy parsing.
• zqa-rag: Core RAG implementation with vector database (LanceDB) and AI provider clients (LLMs, Embeddings, Reranking).
• zqa: Command-line interface and query processing.

Development Commands

Building and Testing

• Build (release): cargo build --release
• Run CLI: cargo run --bin zqa
• Tests (workspace): cargo test --workspace
• Tests (per crate): cargo test -p zqa-rag (or -p zqa, -p zqa-pdftools)
• Lint: cargo clippy --all-targets --all-features -- -D warnings
• Format: cargo fmt --all
• Bench (zqa-pdftools): cargo bench -p zqa-pdftools
• Faster Linux linking: uses mold via .cargo/config.toml (install or remove the flag).

Coding Standards

Rust Conventions

• Use standard Rust formatting: cargo fmt
• Follow Rust naming conventions (snake_case for functions/variables, PascalCase for types)
• Use 4-space indentation (Rust standard)
• Prefer explicit error handling with Result<T, E>. The anyhow crate is strictly disallowed.
• As much as possible, use idiomatic Rust patterns. cargo clippy is helpful here

Version Control

• The owner/maintainer of this repo uses jj, not git. If you're unsure, check for a .jj directory before running version control commands. In general, you should not commit unless the user explicitly requests you to. In that case, follow the Conventional Commits format. If you're unsure of what scope to use, it is okay to omit it, but you must indicate breaking changes if you introduce one, using the ! syntax.

Project-Specific Patterns

• Error types defined in respective modules (e.g., zqa-rag/src/llm/errors.rs)
  • Prefer to inline error definitions if there are only a few, but if there are many kinds of errors, use a dedicated file like the above. Define domain-specific error enums via the thiserror crate.
  • In general, prefer to handle errors explicitly as in idiomatic Rust. Errors from external sources should be wrapped appropriately and propagated until they are handled.
• Factory pattern for LLM clients (zqa-rag/src/llm/factory.rs)
• Trait-based design for extensibility (base traits in zqa-rag/src/llm/base.rs)
• In general, functions should have documentation above them. This does not need to be done for trait implementations, if the trait is standard in Rust (e.g., From<...>, Copy, etc.).
• In general, the library crates zqa-pdftools and zqa-rag should not have side-effects such as printing to stdout, unless that side-effect provides useful information to the user (e.g., warnings, specific error messages, etc.).
• Although cargo clippy is automatically run and will block PR merging, you should also perform checks for idiomatic Rust, especially for code that reimplements functions that are built-in. However, if the user notes, or you believe, that Clippy marked that instance as okay, this is fine, and Clippy's ruling should be followed.

• Documentation comments for functions follow this pattern:

/// Description
///
/// # Arguments
///
/// * `some_arg` - Description
///
/// # Returns
///
/// Description

• For API functions or abstractions such as traits that are expected to be the idiomatic way to do something (such as the Tool trait in zqa-rag), a good docstring includes a code example that ends up as a doctest.
• See STYLE.md for the style guide. In particular, there are specific instructions in a separate section in that file for coding agents.

Agent Skills

To maintain consistency and speed up development, leverage the available agent skills located in .agents/skills/. To invoke a skill, load the SKILL.md file in that directory.

• new-provider: Use this when scaffolding a new AI provider (LLM, Embedding, or Reranking) in zqa-rag. It provides a step-by-step checklist of all required trait implementations, factory registrations, and configuration touch-points.
• pdf-diagnostics: Use this when debugging PDF parsing issues in zqa-pdftools. It outlines a specific diagnostic workflow for introspecting PDF byte streams, font dictionaries, and text matrices without introducing regressions.

Debugging

A few tests are provided specifically to help debugging when working with PDFs:

• test_pdf_content shows the raw PDF content stream for the first page of a specified file.
• test_font_properties prints out information about a font as obtained from the page's font dictionary. If available, it will also print out the font's CMap, but you can disable this by commenting out those lines.
• test_get_content_around_object: Extracts a byte slice surrounding a specific piece of text.

Run them via: cargo test -p zqa-pdftools <test_name> -- --ignored --nocapture

PR Review

• Assess that the PR code follows idiomatic Rust and the coding standards set above.
• In general, bias heavily for performance, particularly in zqa-pdftools. Avoid heap allocations (String, Vec) where borrowed slices (&str, &[u8]) can be used. However, there may be cases where some efficiency is traded off for readability or better UX; but this should be limited. Performance (perf) PRs MUST include benchmark results. (Note: AI agents should generally avoid creating perf PRs for zqa-pdftools).
• PRs should, generally speaking, contain tests for the code they add. This should be exempted in very limited situations where there is a good reason.
• Minimize the use of emojis unless you need to strongly emphasize something; use standard Markdown instead.
• Do not leave inline comments unless you have specific recommendations for improvements.
• Do not leave inline comments to state that something has improved or is better than before.
• Keep your overall comment concise. In a paragraph or two, describe the overall PR quality and the recommendations in your comments.
• If an inline comment you leave is pedantic or otherwise minor, prefix it with "nit: ", and keep it short, about one sentence. This is not to discourage pedantry, but nits should be non-blocking. The sentence immediately following "nit: " should not start with a capital letter. Example: "nit: prefer is_some_and(..) over is_some(..) && ..".
• In general, the repo favors using tests to ensure that things that can go out of sync don't. See test_all_reasoning_effort_values_mapped in crates/zqa-rag/src/llm/base.rs or the various tests in crates/zqa-rag/src/capabilities.rs. This isn't strictly necessary, or blocking, however.
• When two files have code that is shared, similar, or otherwise would need to be updated together, if it would not be obvious to a maintainer that the two locations should be kept in sync, the repo uses NOTE: comments in both places. Such a comment must use the word "maintainers".
• When reviewing a revision of a PR, you should check not only whether the previous comments were addressed, but also whether the changed code itself follows the style guides, introduces new bugs, has inefficiencies, etc.

Important Files

• crates/zqa-rag/src/llm/: LLM client implementations
• crates/zqa-rag/src/embedding/: Embedding client implementations
• crates/zqa-rag/src/vector/: Vector database operations
• crates/zqa-pdftools/src/: PDF parsing utilities
• crates/zqa/src/: CLI interface (work in progress)

Testing Notes

• Integration tests in crates/zqa/tests/--currently disabled
• Use cargo test to run available tests. If you are debugging, use RUST_BACKTRACE=1.

Commit & Pull Request Guidelines

• Commits follow Conventional Commits, e.g.: feat(rag): add checkhealth, fix(ci): ..., perf(pdftools): ..., lint: cargo clippy.
• PRs should include: concise description, rationale, linked issues, and test notes/output (use --log-level debug when relevant).
• Ensure cargo fmt, cargo clippy, and cargo test --workspace pass.

Security & Configuration Tips

• Secrets: configure via .env (see .env.tmpl); never commit real keys.
• Recommended defaults: Anthropic for generation, Voyage AI for embeddings.
• Data location: LanceDB under data/lancedb-table/. Remove the folder to reset the index.