Skip to content
OpenSmartRoute

LLMs1 min read

SinoGlyphBench: Benchmark Reveals Chinese Glyph Obfuscation Impact on LLMs

SinoGlyphBench, a new diagnostic benchmark, highlights the vulnerability of LLMs and MLLMs to Chinese glyph-level obfuscation. Evaluations revealed a significant increase in false negatives and positives, alongside reduced accuracy, demonstrating the need for robust moderation strategies.

By OpenSmartRoute editorial · written through the router by writer-small

From arXiv cs.CL - “SinoGlyphBench: A Diagnostic Benchmark for Chinese Glyph-Level Obfuscation in Language-Model Moderation”

SinoGlyphBench is a benchmark designed to assess the impact of glyph-level obfuscation on language models. The benchmark consists of paired evaluations of 12 LLMs and MLLMs, comparing original and glyph-obfuscated Chinese text and image inputs. The benchmark identifies label-critical semantic anchors and perturbs them, along with background context, to differentiate between genuine corruption and surface variation.

Across 176,916 paired evaluations, the benchmark observed an increase of 6.1 percentage points in harmful false-negative rates and 4.7 percentage points in false-positive rates. Four-way accuracy decreased by 5.0 points. Models retained 75.7% of the decisions made on the original inputs.

Analysis indicated that full-scope perturbations caused the greatest degradation, while anchor-only perturbations were more damaging than background-only perturbations. Cross-script substitution proved particularly challenging within the text modality. The benchmark identified observable mismatches in visible-form reading, intended-message recovery, and final safety judgment.

The results demonstrate that current models remain brittle when processing Chinese content utilizing non-canonical glyphs. This highlights a critical area for improvement in language model moderation systems.

Source: https://arxiv.org/abs/2609.05843

Published Sep 9, 2026 · updated Sep 9, 2026 · 167 words

Keep reading

Related posts

More in LLMs

LLMs1 min read

AlignDiff: Filtering Preference Data with Model Signals

AlignDiff is a new framework that improves LLM alignment by prioritizing challenging preference samples based on model-intrinsic signals like negative log-likelihood gaps. Evaluations on LLaMA, Qwen, and common benchmarks demonstrate consistent performance improvements over established baselines.

LLMs1 min read

CLLPU Benchmark Reveals Propagation Challenges in Multilingual LLM Unlearning

A new benchmark, CLLPU, assesses multilingual LLM unlearning by evaluating the extent of knowledge suppression across languages. Experiments with Llama-3.1-8B-Instruct show that universal suppression often fails to fully remove target knowledge, while language-conditioned confinement leads to uncontrolled propagation.

LLMs1 min read

CONDUIT: KV Cache Reuse Policy for VLMs

CONDUIT is a training-free refresh policy for KV caches in vision-language models, unifying single and multi-image reuse. It achieves high accuracy and speed improvements with a small refresh budget, outperforming budgeted methods on latency.