The UnEmbedding Matrix as a Feature Lens: Why LLM Embeddings Get Distracted by Common Words

Introduction

Large language models are excellent generalists. They translate, summarize, reason, and code with little task-specific tuning. Yet when we ask them to perform one of the most fundamental operations in modern retrieval systems, producing a good text embedding, they consistently underperform dedicated encoders. This is a familiar frustration for anyone building retrieval-augmented generation (RAG) pipelines: the same model that can write a coherent essay about a document struggles to place that document near semantically similar ones in vector space.

The paper "Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings" by Wu, Chen, Liu, and colleagues offers a mechanistic explanation for this gap, and it is more satisfying than the usual heuristic fixes. Rather than proposing yet another prompt template, the authors trace the deficiency back to a specific geometric artifact in the model and then remove it with a single linear transformation they call EmbedFilter. What I find compelling is that the diagnosis and the cure both fall out of the same observation, and the cure is essentially free.

The Core Observation: Embeddings That Speak in Common Words

The starting point is a diagnostic trick borrowed from interpretability research: the Logit Lens. The idea is to take an intermediate hidden representation and project it through the model's unembedding matrix (the output projection that maps hidden states to vocabulary logits) to see which tokens that representation "wants" to predict. Applied to text embeddings extracted from an LLM, the result is revealing and slightly absurd: the embeddings overwhelmingly light up high-frequency but semantically empty tokens. Function words, punctuation, and other corpus-common tokens dominate the top decoding probabilities, largely regardless of what the input text actually says.

This is what the authors call a representation collapse. The embedding is not primarily encoding the meaning of the input; a large component of it is pointing toward a generic, frequency-weighted region of the space. The phenomenon shows up across multiple model families, which suggests it is not a quirk of one training recipe but a structural property of how these models are built.

The authors connect this to two well-documented facts about language model representations:

• Anisotropy. Embeddings from transformers are not spread uniformly across the space; they occupy a narrow cone. This has been studied for years in the context of BERT and GPT-style models, and it degrades the usefulness of cosine similarity because everything is already somewhat similar to everything else.
• The "average token." Prior work characterizes the centroid of that narrow cone as a frequency-weighted average embedding over the training corpus. In other words, the geometric center of the representation space corresponds to the statistically "average" token, which is dominated by the most common vocabulary items.

Putting these together gives a clean mechanistic story. Raw LLM embeddings get pulled toward this commonality region. The pull is strong enough to overshadow the unique semantic signal in any given input. If you could subtract the component of the embedding that points toward the "average" token, you would unmask the genuine semantic content underneath.

Finding and Filtering the Edge Spectrum

The genuinely clever part of the paper is identifying where in the model this average-token pull is being written into the embedding space. The authors reverse-engineer an "average" token representation and then apply Logit Spectroscopy to it, a technique that decomposes the unembedding matrix using its singular value decomposition and examines how different singular directions contribute to the output logits.

What they find is that the directions responsible for amplifying frequent tokens are not in the "middle" of the spectrum where you might expect the dominant semantic content to live. Instead they sit at the two extremes: the right singular vectors associated with the smallest and largest singular values. The authors name this the edge spectrum. When they truncate the projection of the average token onto this edge-spectrum subspace, the logits for those frequent, uninformative tokens are sharply disrupted. That is strong evidence that this subspace is the active writer of the frequency bias.

From here, EmbedFilter is almost trivial to describe. It is a linear transformation, derived directly from the parameters of the unembedding matrix, that projects out the edge-spectrum subspace from any text embedding the model produces. Three properties make this attractive:

• No training. The filter is read off from weights the model already has. There is no fine-tuning, no contrastive objective, no labeled data.
• Post-processing. It operates on embeddings after the fact, so it slots into existing pipelines as a wrapper around whatever extraction method you already use.
• Free dimensionality reduction. Because the filter removes a subspace, the refined embeddings live in a lower-dimensional space. The paper reports that this lowers index storage and speeds up retrieval while preserving the quality of the refined embeddings. You get a smaller, better embedding at the same time.

The experiments span multiple LLM backbones and a range of downstream tasks, with the headline result being that LLMs equipped with EmbedFilter achieve better zero-shot retrieval performance even at reduced dimensionality. The improvement is described as steady and incremental on top of existing extraction methods, which is the right framing: this is a correction to a known artifact, not a wholesale new embedding paradigm.

My Take: A Geometric Artifact We Have Been Paying to Undo

What strikes me most about this work is the reframing it implies for the entire practice of fine-tuning LLMs into embedding models. A large amount of effort in the embedding community goes into contrastive fine-tuning, where you take a base LLM and train it on pairs and triplets to produce well-separated representations. If a meaningful fraction of what that fine-tuning accomplishes is simply suppressing the frequency-driven anisotropy, then we have been spending substantial compute to learn away a defect we could characterize analytically.

The edge-spectrum finding also fits a pattern emerging across interpretability research: the extremes of the singular spectrum often behave differently from the bulk. The smallest singular directions in particular are easy to dismiss as noise, but here they are doing real work, encoding the frequency prior. This is a useful reminder that low-energy directions in a weight matrix are not necessarily unimportant directions.

I want to be honest about limitations, though.

• Frequency is not always noise. In some retrieval settings, function words and surface forms carry real signal, for example in stylistic or authorship tasks. A filter that universally strips the frequency subspace could hurt these cases. The paper's framing assumes that frequent tokens are uninformative, which is true on average but not always.
• The "average token" is an approximation. Reverse-engineering a single average token and removing one subspace is a linear, global correction. The true frequency bias may be input-dependent in ways a fixed projection cannot capture.
• Comparison ceiling. EmbedFilter is positioned as a post-processing enhancement. The interesting open question is how much of the gap to a fully fine-tuned encoder it closes, and whether the remaining gap is semantic content the base model simply never learned.

Conclusion and Open Questions

As I noted in my earlier short summary, my prediction is that frequency-debiasing of the unembedding space becomes a default preprocessing step for LLM-derived embeddings within the next few years. The cost is near zero, the operation is interpretable, and it composes with existing extraction methods. The natural next step is to bake this correction directly into model checkpoints so that practitioners stop fine-tuning encoders just to undo a geometric artifact that the base weights already describe.

Several questions remain open. Can the edge-spectrum analysis be turned into a training-time regularizer that prevents the collapse from forming in the first place, rather than removing it afterward? Does the same subspace structure govern multilingual or code embeddings, where token frequency distributions differ sharply? And can we generalize from a single average-token correction to a small, learned set of nuisance directions without sliding back into heavy fine-tuning?

The broader lesson is methodological. By treating the unembedding matrix as a lens rather than just an output layer, the authors extracted both a diagnosis and a fix from parameters the model already carried. That is the kind of result that makes interpretability feel less like archaeology and more like engineering.