Revisiting the Platonic Representation Hypothesis: An Aristotelian View

URL

Notes

Abstract

Platonic Representation Hypothesis suggests that representations from neural networks are converging to a common statistical model of reality ()

we introduce a permutation-based null-calibration framework that transforms any representational similarity metric into a calibrated score with statistical guarantees ()

the apparent convergence reported by global spectral measures largely disappears after calibration, while local neighborhood similarity, but not local distances, retains significant agreement across different modalities ()

Aristotelian Representation Hypothesis: representations in neural networks are converging to shared local neighborhood relationships ()

1. Introduction

To measure representational similarity across models, different metrics have been proposed ()

Centered Kernel Alignment ()

Canonical Correlation Analysis ()

Representational Similarity Analysis ()

mutual k-Nearest Neighbors ()

two pervasive confounders that distort representational similarity measurements ()

model width: when the embedding dimension increases relative to the sample size, interaction-matrix-based similarity metrics exhibit a systematic positive baseline even when representations are independent ()

model depth ()

Taking a maximum over many comparisons inflates the reported score even if there is no similarity, since the expected maximum of independent draws exceeds the mean (2)

null-calibration for representational similarity, a general permutation-based framework that transforms any similarity metric into a calibrated score with a principled null reference, here defined as no relationship (2)

We find that, after calibration, the previously reported convergence in global metrics (Huh et al., 2024; Maniparambil et al., 2024; Tjandrasuwita et al., 2025) largely disappears, suggesting it was driven primarily by width and depth confounders (2)

local neighborhood-based metrics retain significant cross-modal alignment (2)

Aristotelian Representation Hypothesis1: Neural networks, trained with different objectives on different data and modalities, converge to shared local neighborhood relationships (2)