Geometry and Hierarchies in LLM Representations
How do large language models encode conceptual hierarchies? Two recent lines of work give complementary answers: one describes the geometry of categorical concepts in autoregressive LLMs, the other reconstructs concept lattices from the conditional statistics of masked language models.
Geometry of concepts in LLMs
The Geometry of Categorical and Hierarchical Concepts in Large Language Models (Park, Choe, Jiang, Veitch) asks how a hierarchy of categorical notions is represented inside an LLM. The starting point is the linear representation hypothesis, made precise by the same authors in The Linear Representation Hypothesis and the Geometry of Large Language Models. That earlier work shows that binary semantic features with natural opposites—male/female, true/false—correspond to directions in representation space. Such features are defined by counterfactual pairs of words, for example (man, woman) or (king, queen). Under a suitably chosen inner product, causally separable concepts become orthogonal.
The 2024 paper extends the same hypothesis beyond this binary, contrastive setting. Many natural concepts have no counterfactual pair: is_animal is a feature, not a swap between two tokens, and {mammal, bird, reptile, fish} is a categorical concept rather than a single axis. In that regime it is no longer enough to work with directions. The authors introduce a vector representation of a feature and, with it, a metric structure on the unified embedding–unembedding space. The required inner product is the causal inner product of the earlier paper: after a whitening transformation of the unembedding matrix, the Euclidean geometry of the transformed space respects semantic manipulations.
Once features are vectors rather than rays, a categorical concept can be represented as a polytope whose vertices are the vector representations of its elements—elephant, tiger, dog for mammal. For a “natural” category the polytope is a simplex. Hierarchical relations then receive a geometric counterpart. If changing the parent concept (animal) should not alter the relative probabilities of its children (mammal versus bird), the parent vector must be orthogonal to the child-to-child contrast. The paper proves that concept vectors at different levels of a semantic hierarchy live in orthogonal subspaces.
The theoretical claims are tested on WordNet hyponymy, using Gemma-2B and LLaMA-3-8B. After filtering synsets to those with at least 50 in-vocabulary words, the authors estimate representations for more than 900 hierarchically related noun and verb concepts. Projections of held-out tokens onto the estimated feature vectors concentrate near 1 for members of the concept and near 0 for random tokens. Cosine similarities among the vectors recover the WordNet graph, and child-minus-parent residuals are approximately orthogonal to the parent. Shuffled-unembedding controls indicate that the orthogonality is not an artefact of high dimension or of set inclusion alone.
Formal concepts and lattices in BERT
A complementary picture comes from Formal Concept Analysis (FCA), a classical framework in computer science and AI for representing knowledge through object–attribute relations. A formal context is a triple (G, M, I): objects, attributes, and an incidence relation. A formal concept is a closed pair (extent, intent)—a set of objects together with exactly the attributes they share. Inclusion of extents and intents induces a partial order, and the resulting concept lattice is usually drawn as a Hasse diagram, a directed acyclic graph of covering relations. FCA has a direct topological counterpart: the nerve of a covering reconstructs the same incidence data; see Topology of nerves and formal concepts.
From Tokens to Lattices: Emergent Lattice Structures in Language Models (Xiong, Staab; ICLR 2025) argues that a pretrained masked language model organizes world knowledge into such a lattice. The claim is not that BERT stores a hand-built ontology, but that the MLM objective implicitly learns a formal context: objects and attributes appear as masked tokens, and the conditional probabilities between them, given a linguistic pattern, approximate the incidence relation. From that probabilistic context one can reconstruct a concept lattice by FCA.
Concretely, the model is probed with templates of the form “[MASK] is an animal that can [MASK]”, or analogous object–attribute patterns in other domains. The resulting conditional distribution is treated as a (possibly triadic) formal context, binarized, and closed under the usual FCA operators. The reconstructed lattice contains both named taxonomic nodes and latent concepts: closed object–attribute sets that the model has learned but that have no conventional lexical label—for example, birds that both swim and hunt. This is the distributional, rather than definitional, account of conceptualization: concepts arise from shared attributes in context, not from explicit handbook definitions.
The method, BertLattice, is evaluated on three gold-standard formal contexts: region–language, animal–behavior, and disease–symptom. Ranking the recovered incidences against these standards substantially outperforms a baseline that only compares last-layer embeddings of object and attribute tokens. Domain-adapted models matter: BioBERT and PharmBERT reconstruct medical contexts much more faithfully than generic BERT, consistent with the claim that the lattice is learned from observed co-occurrence rather than from an a priori taxonomy. Concept classification from the reconstructed context likewise improves over embedding distance, and visualizations of the region–language lattice recover interpretable paths (for instance, German-speaking versus French-speaking regions).
Two geometries of hierarchy
The two papers describe different models and different mathematical objects—vector polytopes in a causal inner product versus lattices of formal concepts—but they converge on the same thesis: hierarchical conceptual structure is not an external annotation imposed on language models. It is already present in the representation, either as an orthogonal direct-sum decomposition of feature vectors or as a lattice reconstructed from masked-token statistics. One account is geometric and causal; the other is order-theoretic and distributional. Together they suggest that interpretability work on concepts can move from isolated binary probes toward the algebraic structure of the taxonomy itself.