Conjecture 6.10: Self-Reference Improvement

EXPLORED Meta-beliefs (beliefs about one's own belief accuracy) are conjectured to be informative. The runs to date measure no benefit.

Statement

An agent that maintains a meta-belief \( b^{(2)} \) about the accuracy of its own belief \( b^{(1)} \) will achieve better calibration than an agent with only first-order beliefs, because the meta-belief provides a self-correcting signal.

Status: Explored

Explored in the exploratory anima runs. Agents with self-referential beliefs perform the same as or slightly worse than agents with only first-order beliefs in those runs; the self-correcting signal the conjecture predicts is not seen there. The conjecture is not supported by the runs to date and remains open for other belief models.

Evidence Summary

The experiment compares three agent types across multiple environments:

First-order only (belief \( b^{(1)} \)): baseline calibration
With meta-belief (belief \( b^{(1)} \) + meta-belief \( b^{(2)} \)): calibration equal to or 2–5% worse than baseline
With second-order meta-belief (\( b^{(1)} + b^{(2)} + b^{(3)} \)): 5–10% worse — additional self-reference degrades further

A candidate mechanism: the meta-belief \( b^{(2)} \) is computed from the same evidence that determines \( b^{(1)} \), so it carries no independent information. The additional computation introduces noise from finite-sample estimation of the meta-belief, and this noise propagates back into the first-order belief update.

The pattern is the same across the stationary, changing and adversarial environments in the run. In none of them do meta-beliefs give a measurable benefit.

Relevant Experiments

exp_memory_dynamics.sx — meta-belief dynamics and self-reference tests
exp_anima_deep.sx — deep belief hierarchies including meta-levels

What This Means

In the belief model tested, the information needed for calibration comes from the evidence stream and a meta-belief computed from the same stream adds estimation noise. Whether a meta-belief with access to a different signal behaves differently is untested. Meta-gradients, which operate on a different signal, are a separate construction (see Conjecture 6.6).