Instruction file imported from neilbhammar/ocr-auto-label (.cursor/rules/product-requirements-doc.mdc). Copyright stays with the author.

Lead‑Photo Auto‑Tagger – Updated PRD (Actual Implementation)

1 · Purpose & Context

Field teams shoot thousands of lead‑testing photos in Malawi and Kenya. Each object must be renamed to its handwritten sample code (MWI.… or KEN.…). Manual renaming is slow and error‑prone. We built a browser‑based app (runs locally, no Docker required) that:

• Auto‑extracts the code with Gemini 2.0 Flash Vision API.
• Extracts the top‑3 object colours server-side via Gemini (not client-side k-means).
• Renames all matching images and groups those missing a visible label using advanced time/visual/description similarity.
• Lets a human correct unknowns or mistakes in a virtualized table with comprehensive keyboard shortcuts.

2 · Success Metrics

3 · Personas

Tammy (Field Researcher) – drags camera folders/ZIP archives into the browser, sees real-time processing, and exports renamed sets.

QA Reviewer – bulk‑fixes with Excel-like keyboard shortcuts (arrow keys, bulk select, inline editing).

4 · Scope (Implemented v1)

MUST Have ✅

• Drag‑n‑drop upload of files/folders/ZIP archives (JPEG, PNG, HEIC).
• Immediate Table View with virtualization (chronologically sorted):
  • Columns: ☐, Preview, New Name, Original Name, Group, Status (comprehensive status display).
  • Real-time updates via Server-Sent Events (SSE).
• Parallel processing – Up to 40 Gemini calls concurrent (rate-limited).
  • Gemini 2.0 Flash with enhanced prompt:

    {
      "code": "MWI… or KEN… with strict validation",
      "codeConfidence": 0.85,
      "otherText": "Other visible text", 
      "objectDesc": "3-word object description",
      "objectColors": [
        {"color": "#RRGGBB", "name": "color name"},
        {"color": "#RRGGBB", "name": "color name"},
        {"color": "#RRGGBB", "name": "color name"}
      ]
    }

  • Advanced sample code validation with specific regex patterns for MWI/KEN codes.
• Intelligent auto‑grouping for unlabeled images:
  • Window: ±2 minutes (refined from ±30s).
  • Multi-factor scoring: object colors + description similarity + time proximity.
  • Confidence scoring for grouping decisions.
• Excel-like editing – Full keyboard navigation, inline editing, bulk operations.
• Smart export – ZIP generation with intelligent filename collision handling.

Additional Features Implemented ✅

• ZIP file extraction – Handles large photo archives (up to 5GB).
• Advanced status tracking – Separate gemini/grouping status + comprehensive UI status.
• Confidence scores – Visual indicators for AI decision quality.
• Keyboard shortcuts panel – F1 for help, comprehensive hotkey system.
• How It Works educational section for new users.
• OS temp directory storage – Files stored in system temp, auto-cleanup.
• EXIF timestamp extraction – Accurate photo capture dates.
• Real-time processing updates – Live progress indicators.

NICE to Have (Future)

• One‑click "Ask again with Gemini 1.5 Pro" on selections.
• Multi‑user share link (Supabase integration).

5 · Non‑Functional Requirements

• Zero Docker ✅ – npm install:all && npm start (Node 18+).
• Workspace Architecture – Separate frontend/backend packages with concurrently.
• Local‑first ✅ – Files processed locally, only base64 previews sent to Gemini.
• Performance ✅ – 40 concurrent Gemini calls, table virtualization, optimized thumbnail generation.
• Extensibility ✅ – Clean service interfaces for swapping AI providers.

6 · Technical Architecture (Actual)

┌──────────────┐  HTTP/SSE  ┌───────────────┐  HTTPS  ┌─────────────┐
│ React (Vite) │◀─────────▶│  Local NodeJS │────────▶│ Gemini 2.0  │
│ + Zustand    │            │  (Express)    │         │ Flash API   │
│ + Radix UI   │            │  + Prisma     │         └─────────────┘
│ + shadcn/ui  │            │  + SQLite     │
└──────────────┘            │  + Sharp      │
      ▲                     │  + p-limit    │
      │ File System API     └───────────────┘
      └─────────────────────────────────────┘

Tech Stack:

• Frontend: React 18, Vite, TypeScript, Zustand, Radix UI, shadcn/ui, Tailwind, @tanstack/react-virtual
• Backend: Node.js, Express, TypeScript, Prisma, SQLite, Sharp, p-limit
• AI: Google Generative AI SDK, Gemini 2.0 Flash
• Storage: OS temp directory with automatic cleanup

7 · Detailed Functional Flow (Implemented)

1. Upload → Multiple methods supported:

   • Drag & drop individual files
   • Drag & drop entire folders (recursive)
   • ZIP archive extraction (up to 5GB)
   • Immediate file listing with metadata extraction

2. Processing Pipeline (40 concurrent Gemini calls):

   • Phase 1: File upload, thumbnail generation, EXIF extraction
   • Phase 2: Gemini 2.0 Flash analysis with enhanced prompt
   • Phase 3: Sample code validation with strict regex patterns
   • Phase 4: Auto-grouping with multi-factor similarity scoring
   • Real-time SSE updates → UI status indicators update live

3. Smart Grouping Algorithm:

   • Text similarity: Levenshtein distance on object descriptions
   • Color similarity: RGB distance calculation on top-3 object colors
   • Time proximity: ±2 minute window with decay scoring
   • Confidence calculation: Weighted combination of all factors

4. User Interface:

   • Virtualized table handles thousands of rows efficiently
   • Comprehensive keyboard shortcuts: Arrow navigation, bulk selection, inline editing
   • Real-time status display: Processing → Extracted → Grouped states
   • Smart filename generation: Handles duplicates (code_2, code_3, etc.)

5. Export System:

   • Creates ZIP with renamed files
   • Intelligent collision handling
   • Progress tracking with size estimates

8 · UI Implementation Details

• Landing Page: "How It Works" section + drag-drop upload area + keyboard shortcuts panel
• Main Interface:
  • Toolbar: Filters (All/Complete/Pending/Needs Attention), Search, Export controls
  • Virtualized Table: Checkbox selection, thumbnails, inline editing, status displays
  • Sidebar: Large preview, metadata, confidence scores, color palette display
• Status System:
  • Processing states: pending → extracting → extracted → grouping → complete
  • Visual indicators: Color-coded status icons, confidence percentage bars
• Keyboard Navigation: Full Excel-like experience with F1 help panel

9 · Data Model (Prisma Schema)

model Image {
  id                 String   @id @default(cuid())
  originalName       String
  newName           String?
  filePath          String
  thumbnailPath     String?
  fileSize          Int
  timestamp         DateTime
  group             String?
  
  // Dual-phase processing status
  geminiStatus      String   @default("pending")    // Gemini API processing
  groupingStatus    String   @default("pending")    // Auto-grouping processing
  status            String   @default("pending")    // Overall UI status
  
  // Extracted data from Gemini
  code              String?   // Sample code (MWI.x.x.x...)
  otherText         String?   // Other visible text
  objectDesc        String?   // 3-word object description
  objectColors      String?   // JSON: Array<{color: string, name: string}>
  
  // AI confidence scores
  geminiConfidence    Float?   // 0-1 confidence for code detection
  groupingConfidence  Float?   // 0-1 confidence for auto-grouping
  
  timestamps...
}

10 · Advanced Features Implemented

Sample Code Validation

• Pattern 1: MWI.1.[1-3].[1-24].[1-10][A-D].[1-30].[1-12]
• Pattern 2: MWI.0.[1-3].[1-6].[1-13].[1-27].[1-12]
• Pattern 3: KEN.0.[1-2].[1-9].[1-8].[1-11].[1-12]
• Strict regex validation prevents false positives

Keyboard Shortcuts System

• Navigation: Arrow keys, Page Up/Down, Home/End
• Selection: Ctrl+A (select all), Shift+Click (range select)
• Editing: Enter (inline edit), G (set group), N (rename)
• Operations: Delete (remove), Ctrl+Z (undo)
• Help: F1 (show shortcuts panel)

Intelligent Grouping Algorithm

// Multi-factor scoring system
function calculateGroupingScore(target, candidate) {
  let score = 0;
  
  // Object description similarity (highest weight)
  if (textSimilarity >= 0.8) score += 0.8;
  
  // Color similarity (medium weight)  
  if (colorSimilarity >= 0.5) score += colorSimilarity * 0.6;
  
  // Time proximity (low weight)
  score += timeProximityScore * 0.2;
  
  return score;
}

11 · Performance Optimizations

• Rate Limiting: p-limit with 40 concurrent Gemini calls
• Table Virtualization: @tanstack/react-virtual for thousands of rows
• Thumbnail Generation: Sharp with quality optimization
• Memory Management: Streaming for large ZIP files
• Caching: OS temp directory with intelligent cleanup

12 · Risks & Mitigations (Updated)

13 · Acceptance Criteria ✅

Primary Flow: npm start → localhost:3000 → drop 2,000 photos → ~8 min processing → ZIP export with ~85% auto-tagged correctly.

Advanced Scenarios:

• ✅ ZIP archive extraction (5GB+ files)
• ✅ Real-time processing updates via SSE
• ✅ Keyboard navigation for 1000+ images
• ✅ Accurate timestamp extraction from EXIF
• ✅ Intelligent grouping with confidence scores
• ✅ Export with smart filename handling

14 · Future Enhancements

• AI Provider Flexibility: Easy swap between Gemini/GPT-4o/Claude
• Batch Processing: Queue multiple folders for overnight processing
• Cloud Sync: Optional Supabase integration for team collaboration
• Advanced Filters: Date ranges, confidence thresholds, custom groups
• Export Formats: CSV metadata export, custom naming patterns

This PRD reflects the actual implemented system as of the current codebase, capturing all architectural decisions, feature implementations, and technical choices made during development.