Experiment: Foundational Invariants

Hypothesis

The foundational constraints of the adaptive system are preserved at every step, even under strong gradient perturbations. Specifically, if \( C(\theta) \le 0 \) defines the constraint set, then:

\[ C(\theta_t) \le 0 \implies C(\theta_{t+1}) \le 0 \quad \forall\, t \]

This invariant holds for single constraints, multiple simultaneous constraints, and under gradient magnitudes up to 2200.

Method

Setup: Three test conditions of increasing difficulty.

Parameters:

Steps per trial: 5000
Parameter dimension: 4
Single constraint: norm bound \( \|\theta\| \le R \)
Multiple constraints: norm bound + positivity + box constraints
Strong gradient: adversarial gradient with magnitude 2200

Procedure: Run the system forward and check constraint satisfaction at every step. The constraint projection mechanism should correct any violation before the next step.

Results

Condition	Total Steps	Status
Single constraint	5000	Pass
Multiple constraints	5000	Pass
Strong gradient (mag 2200)	5000	Pass

Zero violations across all 15,000 total steps.

Analysis

The constraint preservation mechanism works via projection after each update step. Even when the gradient magnitude reaches 2200 (far exceeding typical training gradients of order 1-10), the projection correctly clips the parameter back to the feasible set.

The multiple-constraint case is noteworthy: with three simultaneous constraints, the projection must satisfy all of them jointly. Alternating projection (Dykstra's algorithm) achieves this in at most 3 inner iterations per step.

This result establishes that the invariant set is forward-invariant under the composed dynamics, which is a prerequisite for the Lyapunov analysis in Theorem 3.

Conclusion

Pass — All foundational constraints preserved with zero violations across 15,000 steps including adversarial conditions. Proposition 3.5 is validated.

Reproducibility

../simplex/build/sxc exp_invariants.sx -o build/exp_invariants.ll
clang -O2 build/exp_invariants.ll ../simplex/runtime/standalone_runtime.c \
  -o build/exp_invariants -lm -lssl -lcrypto -L$(brew --prefix openssl)/lib
./build/exp_invariants