B-Flow
Statement
For \( K \) objectives with gradient vectors \( g_1, \ldots, g_K \), write \( G = \sum_i g_i \) and \( D = \sum_i \|g_i\|^2 \). Define the balance residual
\[ B(\theta) = \frac{\|G\|^2}{D} = 1 + 2I(\theta), \]where \( I \) is the interaction ratio. Then \( 0 \le B \le K \), and \( B = 0 \) exactly when \( G = 0 \). At a point \( \theta^* \) with \( G(\theta^*) = 0 \) and \( D(\theta^*) > 0 \), \( \theta^* \) is a critical point of \( B \) and
\[ \nabla^2 B(\theta^*) = \frac{2\,J^{\mathsf T} J}{D}, \qquad J = \frac{\partial G}{\partial \theta}\Big|_{\theta^*} = \sum_i \nabla^2 L_i(\theta^*). \]Derivation
Differentiating the quotient, \( \nabla B = \dfrac{2J^{\mathsf T}G}{D} - \dfrac{\|G\|^2\,\nabla D}{D^2} \). Both terms vanish at \( G = 0 \). Differentiating once more, every term carrying a factor of \( G \) vanishes at \( \theta^* \) and the only surviving term is \( 2J^{\mathsf T}J/D \). The Hessian is positive semi-definite, with null space equal to the null space of \( J \). \( \blacksquare \)
Gradient descent on \( B \)
"B-flow" is gradient descent on \( B \) rather than on the summed loss. Because \( B \) measures only \( \|G\| \), B-flow moves toward the nearest point where the gradients cancel; that point need not be a minimiser of any objective, and where \( J \) is singular the Hessian is degenerate and the local rate is not controlled. In the runs on the balance-residual experiment page, the two flows are compared on the same test problems and the final values of \( B \) are reported as data.
Two-phase use
- Loss-flow to reduce the individual losses and reach a neighbourhood of a balanced point.
- B-flow in that neighbourhood, where the local Hessian above governs the behaviour.
The switch is triggered by monitoring \( B \) (equivalently \( |I + \tfrac12| \)) against a chosen tolerance.
Experiment Files
exp_balance_residual.sx — B-flow and loss-flow on the same test problems; final residuals reported
exp_equilibrium_mapping.sx — location of balanced points across problem classes