Perplexity releases pplx-embed-v2-late with a 0.6B edge model and a 9B model scoring 92.4%

Performance details including MADQA scores and benchmark comparisons

The 9B model achieved a score of 92.4% on the MADQA benchmark. This is the strongest result reported by Perplexity so far. It leads all tested models by 1.6 percentage points in domain-specific text tasks. The 0.6B model trails the gemini-embedding-2 by 0.3 percentage points. It stays within 1.2 percentage points of the nemotron-colembed-v2-8b model.

Mixedbread's retriever scored 88.9% on similar tasks. Perplexity's 9B model beats that score significantly. The 0.6B model scores 61.2% on the ViDoRe v3 Markdown benchmark. This is the weakest result reported in their announcement. It remains the second-best score on that specific benchmark. Tencent's EVIE model scores higher than Perplexity here.

Technical specifics on parameter counts, vector dimensions, and training

The 0.6B model uses about 340 million active parameters for images. This number stays close to rivals with 8 billion parameters. The 9B index can be searched using the 0.6B queries. It recovers about half of the quality gap found in text tasks. Token vectors are 128 dimensions long. Rivals typically use 2,048 or 4,096 dimensions for their vectors.

Perplexity distilled both models from an 18B teacher model. They used LEAF-style token-level training for this process. This training method creates the shared embedding space between sizes. The published checkpoints are stored in F32 format. This doubles the download size compared to FP16 weights. Engineers need sentence-transformers version 6.0.0 or higher. They also require transformers version 5.4.0 or higher.

Why it matters for cost, speed, and new capabilities in RAG

RAG stands for retrieval-augmented generation. It uses embeddings to find relevant documents for an AI model. These models matter because they change the cost structure of search. The 0.6B queries can run on cheaper hardware than before. Teams can index large PDFs with the 9B model efficiently. They can query that index with the fast 0.6B model locally.

Speed improves when you use the smaller model for user requests. Latency drops because edge devices handle the heavy lifting. Cost drops because you do not need to run a massive model for every query. Quality stays high because the 9B model builds the initial index. This setup works well for visual document search over PDFs. It also works for slides and scanned reports.

How OpenSmartRoute helps teams deploy these models efficiently

OpenSmartRoute is an open-source router for AI requests. Teams define a catalogue of models, agents, tools, or skills in their app. The router sends each request to the best-fit candidate from that list. It scores every option on quality, cost, speed, and safety. The team sets specific weights for these scores per request type.

Hard rules pin certain requests to specific models. For example, text with personal data stays on an on-premises model. A region or a cost cap is never crossed by the router. It learns from outcomes over time in production. A model that answers well gets more traffic automatically. One that fails gets less traffic without human intervention.

A new model becomes one catalogue entry in this system. It competes for requests on the next call immediately. Nothing else changes in the application when a model updates. OpenSmartRoute works with any OpenAI-compatible provider. It supports open-weight models served locally by your team. It integrates with MCP tools and A2A agents easily.

How it compares

Perplexity released pplx-embed-v2-late as a new pair of models. They use ColBERT-style architecture for embeddings. The 0.6B model runs fast on edge devices. The 9B model provides maximum quality for indexing. Both models share one single embedding space. This design allows them to work together seamlessly.

Before this release, teams often chose between speed and quality. Dense models compressed documents into single vectors. Perplexity's new approach keeps a vector for every token. It stores 128-dimensional tokens for each input token. The index size grows linearly with document length. This differs from standard dense retrieval methods.

The 0.6B model rivals the 8B Nemotron ColBERT VL model. It uses about 340 million active parameters for images. Its performance on text is close to its 8B rival. The 9B model leads all tested models by 1.6 percentage points. It beats Mixedbread's retriever significantly in domain-specific tasks.

Some features remain unchanged from previous versions. The MIT license allows commercial use freely. Both models require specific Python package versions. They need sentence-transformers version 6.0.0 or higher. Teams must also install transformers version 5.4.0 or newer. These dependencies ensure compatibility with the new weights.

The main difference lies in how they handle data. Perplexity encodes pages as images directly. This removes the need for an OCR step during processing. The 0.6B model can query a 9B index effectively. It recovers about half the quality gap at lower cost. Mixing text and images in one input is not supported yet.

Questions this leaves open

Perplexity has not published the full technical report yet. All performance scores are self-reported by the company. Readers cannot verify these numbers independently right now. The team invites others to check the benchmarks themselves. Independent verification builds trust in the model's capabilities.

The image retrieval scores remain lower than some competitors. Tencent's EVIE model scores higher on ViDoRe v3. Gemini Embedding 2 beats the 9B model on MIRACL-Vision. The gap is about two percentage points on PPLX-Q2I. Image search performance is a known limitation for this pair.

Storage requirements grow with every document added to the index. A single vector exists per token in the system. Long documents create larger index files quickly. Teams must calculate storage needs before scaling up. The F32 checkpoint format doubles the download size significantly. This impacts bandwidth and initial setup time.

The hosted API endpoint is planned but not live yet. Teams cannot access Perplexity's official cloud service immediately. They must host the models themselves on their infrastructure. Hugging Face hosts the weights under an MIT license. Users can download the checkpoints directly from there.

Evaluation results depend heavily on the specific test suite used. MADQA scores are strong at 92.4% for the 9B model. ViDoRe v3 Markdown scores drop to 61.2% for the 0.6B model. These numbers vary based on the benchmark chosen. Teams should run their own tests for accuracy.

The training method uses an 18B teacher model. Perplexity applied LEAF-style token-level training techniques. This process created the shared embedding space. Distillation from a larger teacher improves efficiency. The specific loss function details remain proprietary information.

Security considerations regarding prompt injection are not fully addressed. An input guard can spot injections before requests leave. However, the router does not block all potential attacks automatically. Teams must implement their own security layers. Production environments need extra caution with user inputs.

The 128-dimensional vector size is narrower than typical rivals. Competitors use dimensions between 2048 and 4096 for vectors. This narrowness affects similarity search precision in some cases. It might impact results for complex multi-modal queries. Teams should test their specific query patterns carefully.

The lack of a public technical report limits transparency. Researchers cannot reproduce the exact training setup easily. The team credits the researcher but does not share code. Open-source communities often prefer full reproducibility details. This gap hinders independent research and validation efforts.

The cost structure for self-hosting is estimated by Perplexity. Memory usage figures are approximations based on two bytes per parameter. Actual costs depend on your specific hardware configuration. Cloud providers charge differently for GPU memory access. Teams must calculate their own total cost of ownership.

The model cards show CUDA GPU usage explicitly. This confirms the models run on NVIDIA hardware primarily. Other GPU vendors might face compatibility issues. Apple Silicon or AMD GPUs are not officially supported yet. Teams using non-NVIDIA hardware need to verify support first.

The shared embedding space simplifies retrieval logic significantly. Queries and documents map to the same vector space. This reduces complexity in building the search pipeline. However, it also means all modalities share the same constraints. Text and image quality trade-offs exist within this design.

The 0.6B model fits edge devices well for latency-sensitive tasks. The 9B model requires more power for indexing operations. Running both simultaneously increases overall system resource usage. Teams must balance capacity against performance needs carefully.

The MIT license permits modification of the model weights. Users can fine-tune the models for specific domains. This flexibility is a major advantage for custom applications. However, it also means quality control becomes the user's responsibility. No one else guarantees the output accuracy after changes.

The technical report delay suggests ongoing internal testing. Perplexity likely validates results before public release. The team wants to ensure no critical bugs exist. Rushing the report might hide important performance issues. Patience allows for a more stable final product eventually.

Readers should check the Hugging Face page for updates. The model weights are available there currently. Look for any changelog entries or release notes. Monitor the repository for new information about the report. Community discussions on Reddit or Twitter might offer insights.

The 150k+ ML SubReddit mentioned in the source is a community hub. Joining it provides access to broader AI discussions. Other users share experiences with similar embedding models. Peer reviews can highlight potential pitfalls not seen yet.

The sponsored section for TinyFish MCP server is unrelated to Perplexity. It promotes a different tool for web search and fetch. Teams should evaluate all tools against their specific requirements. Do not assume one tool fits every use case perfectly.

Asif Razzaq leads Marktechpost, the platform covering this news. His team provides context on machine learning trends. Their content aims to be technically sound and accessible. Over two million monthly views indicate high reader interest.

The comparison between Perplexity's models and others is still evolving. New benchmarks will emerge as the industry matures. Stay updated on future releases and evaluation results. The landscape of embedding models changes rapidly every month.

What to do next with the open weights

You can check the model weights on Hugging Face directly. The technical report is not out yet, so verify numbers yourself. Compare these scores against other benchmarks like ViDoRe or MIRACL-Vision. Image search remains a gap compared to Gemini Embedding 2 in some areas. Storage growth is a caveat because index size grows with document length.

Try running the 0.6B model on a local edge device first. Test it with your own PDF collection to see real performance. Measure if the 128-dim vectors fit your memory constraints well. Check if sentence-transformers and transformers meet your version requirements. Ensure your deployment handles the F32 checkpoint size correctly.

OpenSmartRoute can show you a savings ledger for these models. It compares what each routed request cost next to the most expensive option. An input guard spots prompt injection before requests leave your system. The 'osr eval' tool measures routing accuracy on your own prompts. It can fail a build if the routing accuracy drops too low.

The hosted platform keeps a catalogue with prices and public rankings. These rankings are built from real traffic data over time. You do not need to guess which model fits your use case. The router finds the right balance between speed and quality automatically. Start by downloading the weights and testing them in your environment.

Source: https://www.marktechpost.com/2026/10/07/perplexity-ai-releases-pplx-embed-v2-late-a-0-6b-edge-model-and-a-9b-model-scoring-92-4-on-madqa/

Reflection's Beam beats Chinese models with less compute

Reflection released Beam, an open-weight model that matches GLM 5.2 and Qwen 3.8 on benchmarks while using three to four times less compute.