The paper "Text Embeddings Reveal As Much as Text" (EMNLP 2023) investigates whether embedding vectors allow for the recovery of original input text. This addresses concerns regarding the security of storing sensitive information in vector databases used by Retrieval Augmented Generation (RAG) systems.
Information Loss in Neural Networks
Text embeddings are outputs of neural networks involving matrix multiplications and nonlinear functions. The data processing inequality states that functions cannot add information to an input, only sustain or decrease it. Furthermore, ubiquitous nonlinear layers like ReLU destroy specific input information by setting negative values to zero.
Image Inversion Precedents
Similar inversion capabilities exist in computer vision. Deep representations from image models can reconstruct input images with fidelity. An early result (Dosovitskiy, 2016) showed that CNN feature outputs could be inverted to produce blurry-but-similar versions of original images.
Text Recovery Feasibility
Researchers developed a toy setting to test text inversion, restricting inputs to 32 tokens (approximately 25 words) and embedding them into vectors of 768 floating-point numbers. The study confirms that input text can be recovered from these output embeddings, indicating that embedding vectors do not securely hide the information they represent.
