Experiments › Diagnostics

Experiment: I-ratio identity checks

Source: exp_iratio_proof.sx · Checks the balance identity used by the I-ratio diagnostic

The identity

For \( K \ge 2 \) gradient vectors \( g_1, \ldots, g_K \), write \( D = \sum_i \|g_i\|^2 \) and define the interaction ratio

\[ I(\theta) = \frac{\displaystyle\sum_{i < j} g_i \cdot g_j}{D}. \]

Expanding the squared norm of the summed gradient gives the algebraic identity

\[ \left\|\sum_i g_i\right\|^2 = D\,(1 + 2I), \]

so \( I = -\frac{1}{2} \) exactly when \( \sum_i g_i = 0 \), and in general \( I \in [-\tfrac{1}{2}, \tfrac{K-1}{2}] \). The identity holds for every \( K \) and every parameter dimension. This experiment checks that the implementation computes it to double-precision rounding; the checks test the code, not anything beyond the identity.

Method

Setup: Three test suites: systematic K sweep, off-balance control, and random problem instances.

Parameters:

  • K sweep: \( K = 2, 3, \ldots, 20 \), with 3 balanced configurations (\( \sum g_i = 0 \)) per K
  • Off-balance control: 10 cases where \( \sum g_i \neq 0 \)
  • Random problems: 70 randomly generated multi-objective instances
  • Tolerance: \( |I + 0.5| < 10^{-10} \)

Procedure: For each test, construct gradient vectors with (or without) \( \sum g_i = 0 \), compute \( I(\theta) \), and check whether it equals \( -0.5 \) within tolerance.

Results

Test SuitePassTotalRateStatus
K=2..20 balanced sweep575757/57Pass
Off-balance (negative control)101010/10 give \(I \neq -0.5\)Pass
Random problems707070/70Pass

Precision

MetricValue
Maximum absolute error \(|I + 0.5|\)\(2.22 \times 10^{-16}\)
Mean absolute error\(< 10^{-16}\)

The maximum error of \( 2.22 \times 10^{-16} \) is one unit of least precision for IEEE 754 double arithmetic, which is what an exact algebraic identity evaluated in floating point is expected to give.

Analysis

The result follows from

\[ \left\|\sum_i g_i\right\|^2 = \sum_i \|g_i\|^2 + 2\sum_{iwhich is the identity \( \|\sum g\|^2 = D(1 + 2I) \). When \( \sum_i g_i = 0 \) the left side vanishes and \( I = -\frac{1}{2} \); when it does not, \( I > -\frac{1}{2} \).

Observations:

  • The identity is dimension-free and holds for any number of objectives \( K \ge 2 \).
  • The off-balance controls show the implementation returns \( I \neq -0.5 \) when \( \sum g_i \neq 0 \), as the identity requires.
  • The size of the error (one ULP) shows the computation is an exact identity in floating point rather than an approximation.

As a diagnostic, \( |I + 0.5| \) measures how far the summed gradient is from zero relative to \( D \). It is a scale-free readout of gradient cancellation, and nothing more is claimed for it here.

Summary

Pass — All 137 checks pass (57 balanced + 10 negative control + 70 random). Maximum error is 1 ULP. The implementation computes the balance identity correctly.

Reproducibility

../simplex/build/sxc exp_iratio_proof.sx -o build/exp_iratio_proof.ll
clang -O2 build/exp_iratio_proof.ll ../simplex/runtime/standalone_runtime.c \
  -o build/exp_iratio_proof -lm -lssl -lcrypto -L$(brew --prefix openssl)/lib
./build/exp_iratio_proof

Related