Beyond Fixed Representations: The Vocabulary and Verifier Gaps in Open-Ended AI

URL

Notes

Abstract

open-ended innovation requires additional classes of operations: the creation, stabilization, and reuse of new representational primitives, which alter the space being searched rather than simply searching within it ()

distance between current AI systems and genuinely open-ended intelligence through two gaps ()

vocabulary gap, the difficulty of inventing and stabilizing new representational primitives ()

verifier gap, the difficulty of judging the value of a new primitive when its full payoff may be visible only after future reuse ()

cognitive discrepancy reduction ()

intelligent behaviors as a sequence of cognitive transformations ()

intra-space transformations ()

generative transformations ()

1 Introduction

framed problems are useful for measuring progress, but they also impose an important limitation: our headline metrics are, by construction, weak tests of the second that require stronger capability (2)

LLM-generated research ideas are narrow and highly-concentrated on the pattern of recombining and synthesizing multiple existing ideas (2)

“vocabulary gap”, the difficulty of inventing and stabilizing new representational primitives (2)

“verifier gap”, the difficulty of judging whether a newly introduced primitive is worth retaining when its value is not yet validated (2)

innovation gaps through the lens of cognitive discrepancy reduction (2)

"Cognitive discrepancy" is the idea for the objective function here.

pressure to reduce gaps between current states and unresolved demand states by prediction, explanation and creation (2)

"Innovation" is like the backprop here.   The 3 processes involved while doing "cognitive discrepancy reduction":

1. Prediction
2. Explanation
3. Creation

Concept invention becomes necessary when existing representations cannot sufficiently reduce such discrepancies (2)

At lower levels, systems perform intra-space transformation (2)

At higher levels, systems must perform representational transformation that enables the invention of new primitives (2)

"Representation Transformation" is the thing these open-ended AI should be able to do.

Moving upward requires capability augmentation beyond stronger pattern matching and completion (2)

next-token prediction objective can be interpreted as a specific instantiation of discrepancy reduction, yet this objective does not by itself reward the autonomous expansion of the system’s representational space (3)

Progress toward open-ended intelligence therefore requires broader objectives and architectural designs that explicitly support representational growth (3)

2 The Distance from Open-ended Intelligence

two central gaps that must be narrowed for AI systems to acquire such open-ended capabilities: the vocabulary gap and the verifier gap (3)

2.1 The Vocabulary Gap

minimum-description-length (MDL) view of abstraction, where a primitive justifies its value by compressing a family of observations, reducing the representational budget required to express them (3)

Representational Budget.

amortization: a primitive must justify its representational value across a family of problems (3)

LL(·) denote description length (3)

language L (3)

LB L (f ) denote the length of the shortest solution to task f discoverable within the search budget B under L (3)

task family F (3)

primitive π to obtain L′ = L ∪ {π} is generative (3)

(i) Amortized compression (3)

(ii) Feasibility extension (3)

without such an autonomous vocabulary expansion, open-ended innovation is out of reach (4)

2.2 The Verifier Gap

verification for open-ended intelligence is much harder (4)

two distinct versions (4)

first is a delayed and expensive verification (4)

second difficulty is deeper. In representation-expanding innovation, the value of a new primitive may not be measurable by the current evaluator at all (4)

DreamCoder’s wake–sleep cycl (4)

open challenge is representation expansion in general environments, across a broader range of tasks (4)

closing the verifier gap requires a further capability: deciding which primitives are worth keeping when the current evaluator cannot yet see their full value (5)

Human science partly solves this problem through distributed curiosity-driven exploration: many researchers pursue directions that are not locally optimal under current objectives, and only later does the community discover which concepts become generative (5)

analogous mechanism: not merely agents that optimize fixed objectives, but exploratory populations (5)

3 Cognitive Discrepancy Reduction

view intelligence itself as the minimization of cognitive discrepancies (5)

3.1 The Framework

cognitive discrepancy we mean the mismatch between a system’s current representational state and a desired goal state (5)

Rt denote the system’s internal representational state at time t (5)

Gt denote a target condition (5)

general schema for intelligence as trajectory-level discrepancy reduction, carried out through a sequence of K cognitive transformations (5)

Dt measures the discrepancy (6)

discrepancy may take the form of prediction error, lack of a usable concept, an incomplete explanation, a control error, or an inconsistency (6)

transformations Tt include a set of fundamental cognitive operations (6)

• Association: links related concepts and form relational structures (6)

• Abstraction: extracts invariants across cases and representing them at a higher level (6)

• Analogy: maps structures with relational alignment from a known domain onto a new one (6)

• Composition: combines existing primitives into larger conceptual, functional, or procedural structures (6)

• Concept creation: stabilizes invariants as a named, reusable primitive to make abstraction persistent (6)

operations have to be guided and regulated by cognitive priors (6)

Without priors, the space of possible transformations is intractably large: infinitely many abstractions, analogies, and compositions can be proposed (6)

Useful cognition therefore requires regularizing assumptions (6)

simplicity, symmetry, spatial and temporal continuity, causal coherence, compositionality, and preference for reusable structure (6)

priors as a cost or preference term Ω(Tt, Rt) (6)

intelligent behavior can be described as a sequence of locally constrained transformations that reduce the mismatch between representational status and task demand (6)

3.2 Connection with Existing Frameworks

Free-energy minimization formalizes prediction and belief updating (7)

minimum description length (MDL) and algorithmic information theory (AIT) formalize compression and abstraction (7)

structure-mapping theory (SMT) formalizes analogy (7)

reinforcement learning (RL) formalizes action selection under goals and rewards (7)

identify for each framework the corresponding cognitive operation T and the prior Ω (7)

Compression and abstraction (MDL, AIT)

MDL

minimizing L(M ) + L(O | M ) (7)

L(M ) is the cost of describing the model itself (7)

residual L(O | M ) is the cost of describing the observation (7)

cognitive operation is abstraction (7)

cognitive prior is simplicity, Ω = L(M ) (7)

AIT

K(·) for Kolmogorov complexity (7)

Length of the shortest program that can output something.

Ex:

for x = AAAAAAAAAAAAAAAAAA then, K(x) << x

for x = AHBBFIUWOHDKBICDWW then, K(x) ~ x

h for a hypothesis (7)

The representation.

Ex:

0, 4, 9, 16, 25

then, h* = x^2 K(h) = a K(O|h) = 0

preferred hypothesis is h∗ ∈ arg minh[ K(h) + K(O | h) ] (7)

K(h) the program length of the hypothesis (7)

K(O | h) the residual information in the data once h is supplied (7)

prior of simplicity Ω = K(h) (7)

Its computable, resource-bounded form is exactly the LB L (7)

Bounded intelligence.

The shortest description an agent can realistically find (might not be the absolute argmin)

both fix the description basis, and neither explains how a bounded system invents the new primitives that enlarge that basis and make future descriptions shorter (7)

Both MDL and AIT

missing step is precisely the vocabulary gap (7)

Analogy (SMT)

analogy as the alignment of relational structure between a source S and a target U (8)

mapping m (8)

scored by systematicity (8)

Cost(m) is a parsimony penalty (8)

cognitive operation T is analogy (8)

cognitive prior Ω is systematicity (8)

Inference and belief update (free energy, Bayes)

free-energy principle

q(z) be an approximate posterior over latent states z (8)

pR(O, z) be the generative model (8)

representational state R (8)

first term measures the prediction error (8)

second term is a complexity measure which discourages posterior beliefs q(z) from moving from the prior pR(z) (8)

cognitive operators are perceptual inference (8)

cognitive prior is parsimony and coherence in explanation (8)

Bayesian inference

first term measures prediction error (8)

second term reflects the prior plausibility of the hypothesis (8)

operation is belief revision (8)

prior preference (8)

Curiosity and compression progress

trajectory-level extension of the compression frameworks (8)

h≤t = (o1, . . . , ot) denote an agent’s experience up to time t (8)

θt parameterize its current predictive model (8)

L(h≤t; θt) be the description length (8)

Updating θt−1 to θt so that the same history becomes more compressible (9)

improvement in compression (9)

cognitive operation T therefore includes two layers: abstraction for experience compression, and exploration for learning experience maximization (9)

cognitive prior is not simplicity alone, but learnable progress (9)

Curiosity is therefore the framework that comes closest to valuing representational change (9)

curiosity still does not close the verifier gap (9)

it cannot judge whether a primitive will matter for tasks or standards of success that are not yet expressible in the present representation (9)

We don't know what we don't know.

3.3 What the Comparison Establishes

every operation they formalize searches within a fixed representation R (9)

4 Levels of Innovation Autonomy

ladder is organized in three dimensions: (10)

1. Search pattern: is the system searching within a fixed representational space, or can it modify the space being searched? (10)

2. Vocabulary autonomy: can it create and reuse new conceptual primitives? (10)

3. Verifier autonomy: does it own the verifier and evolve it as needed, or is the success criteria supplied and fixed from outside? (10)

L0: Direct intra-space search

Standard LLM assistants (10)

L1: Intra-space search loop with fixed verifier

FunSearch (10)

AlphaEvolve-style (10)

L2: Scaffolded representation-space augmentation

DreamCoder (11)

Stitch-style (11)

L3: Open-ended innovation autonomy

5 What Would Have to Change

path to open-ended AI cannot be reduced to scaling models, extending context windows, adding tools, or increasing inference-time search (12)

Objectives that reward useful representation change

Next-token prediction (NTP) can be interpreted as a specific instance of the discrepancy-reduction schema (12)

R is the model’s representation of the token context (12)

G is the observed next token (12)

D is the cross-entropy loss (12)

transformation T is identity T = I (12)

system should receive credit for triggering an operation T that produces a more useful representation R′ = T (R) for solving a problem, including abstraction, compression, relational mapping, composition (13)

Data that elucidates invention, not only its outcome

Data for open-ended intelligence should instead expose trajectories of representational revision (14)

different data format from the ordinary NTP over finished text (14)

better equip models to recognize and perform framechanging actions, rather than merely absorbing the surface forms of already stabilized knowledge (14)

data should instantiate diverse choices of R, T , and D, including representations that fail in different ways (14)

Persistent primitive stores and consolidation

Open-ended intelligent systems therefore need selective stores to keep candidate primitives (14)

consolidation could be realized through fine-tuning or continual learning (14)

Surrogate verifiers

surrogate verifier, a cheap and computable estimate of a candidate’s value, to identify promising search directions (15)

A system rewarded for satisfying its surrogate will find ways to satisfy it without maximizing the underlying value, as Eurisko’s self-crediting heuristics did (15)

Self-extending evaluation

Representational expansion becomes feasible only when what was previously unexpressible or untestable is made verifiable. This is the capability most current systems lack (15)

6 Related Work

6.1 Open-Endedness: Necessity and Limits

6.2 Cognitive Foundations of Innovation

Abstraction

abstract concepts are essential for sophisticated situated action, as they allow humans to integrate complex situational elements to comprehend events and predict outcomes (16)

Analogy

Composition

“next-primitive prediction” as a stronger inference paradigm (17)

combinatorial creativity is a distinct form of generalization that fixed-target benchmarks miss, and should be scored by degrees of novelty and utility rather than against a fixed answer (17)

Limits of current systems

LLMs struggle with abductive “jumps” of the kind required for major scientific reframing, such as the conceptual leap from Newtonian mechanics to general relativity (17)

open-ended intelligence requires more than interpolation inside an inherited conceptual vocabulary (17)

6.3 Self-Modifying Discovery Systems

Lenat’s Automated Mathematician (AM) (17)

mutating small Lisp programs under a body of heuristics (17)

Eurisko (17)

applying heuristics to revise its own heuristics (17)

representation expansion was partly a property of the representation (17)

verifier gap requires an operational, ablationbased criterion rather than an observer’s impression that “something new emerged.” (18)

PowerPlay (18)

searches for the simplest still-unsolved task together with a solver modification that solves it while preserving prior skills (18)

AI-GAs framework (18)

learn not only solutions but also architectures, learning algorithms, and environments (18)

POET [59] and Enhanced POET (18)

ecosystem of paired problems and solutions, allowing new challenges and capabilities to co-evolve (18)

progress can arise from a growing population of tasks, agents, and stepping stones rather than from optimization toward one predefined goal (18)

they make progress toward open-ended search (18)

6.4 Recursive Self-Improvement and Automated Research

Recursive self-improvement converts search experience into better search machinery (18)

Gödel Machine (GM) is the classical statement: a self-referential solver that rewrites any part of its own code, including the proof searcher and the self-improvement mechanism itself, once it proves the rewrite beneficial under an encoded utility function (18)

Darwin GM (18)

Huxley GM (18)

Red Queen GM (18)

couples foundation models with search and external evaluation (18)

FunSearch (18)

LLM-generated programs, when embedded in an evolutionary loop (18)

AlphaEvolve (18)

EvoX (18)

aspects of the search strategy (18)

AI Scientist and AI co-scientist systems (18)

AutoResearch AI (18)

designer supplies a reliable evaluator, a clear objective, and a structured search loop, AI systems can make nontrivial discoveries (18)

they do not yet autonomously decide when the representation frame or evaluation frame should change (18)

6.5 Conceptual Spaces and Representation Transformation

cognitive operations discussed in this paper can be viewed as transformations over conceptual space (19)

central claim is therefore not simply that intelligence searches a conceptual space, but that open-ended intelligence modifies the space in which future search occurs (19)

7 Limitations and Open Problems

formal definitions and architectural implications require further development (19)

sharpen the proposed notions of vocabulary and verifier gaps, primitive usefulness, and innovation autonomy through more precise formalization and controlled experiments (19)

This paper does not claim to provide a complete theory of human innovation or scientific discovery. Rather, it identifies several computationally relevant mechanisms that are underdeveloped in current AI systems (19)

determine which aspects of human innovation are computationally tractable, which can be modeled algorithmically, and which may require fundamentally different modeling assumptions (19)

self-extending evaluation can raise significant safety concerns. A system that can revise its own evaluators may also drift from its original objectives, game its surrogate signals, or create harmful primitives (19)

Capability improvement and safety analysis therefore must proceed together, not in sequence. The same mechanism that would make an L3 system open-ended is also what would make it difficult to constrain (19)

8 Conclusion

Civilization is, in large part, an unbounded recursion in what can be represented, constructed, and transformed (19)

This capacity, to reshape environments, generate new concepts, and expand the space of what can be known and built, is largely absent from today’s AI systems (19)

deriving training objectives directly from the discrepancy-reduction schema rather than from next-token prediction alone (20)

data formatting and curricula that make representational change necessary (20)

architectures with persistent memory for invented primitives (20)

verification protocols that can extend themselves (20)

Open-ended intelligence, as a higher form of intelligence, subsumes the closed-ended systems we have today, and the one that will require most of the effort ahead (20)