E.2 Gaussian directions and coefficient measures🔗

Experiment 1 uses Q=\operatorname{diag}(j^{-2}), truncated to d=10,100,1000,10000 coordinates. Its 200 test inputs have coordinates x_j=u_j/j with independent u_j uniform on [-1,1]. Gaussian directions with covariance Q and zero bias give the ReLU networks. Errors are averaged over 20 trials.

Experiment 2 uses \alpha=1, P=Q=\operatorname{diag}(j^{-2}), W=I and \rho(t)=\mathrm{He}_4(t)e^{-t^2/2}. Its coefficient is \gamma_g(a,b)=D^{-1/2}\sqrt{2\pi}\,\phi_{1+\kappa_W(a)}^{(4)}(b), where \phi_u is the centered Gaussian density of variance u and D=\det(I+Q^{1/2}WQ^{1/2}). Quadrature evaluates the total variation and the target 6f_g. Conditional biases are drawn by rejection from a mixture of Gaussian moment densities. Sampling–importance resampling uses 200000 Gaussian-mixture proposals, with effective sample sizes between 2800 and 9700. Ten trials at each of d=10,100,1000 use 100 test inputs drawn by the same rule as Experiment 1. Directions are drawn with replacement from the weighted pool, approximating the continuous direction law. The pool is reused across trials at each resolution, so trial variation is conditional on that pool and does not measure its approximation error. The conditional bias law is exact: with u=1+\kappa_W(a) and b=\sqrt u\,z, its density is |\mathrm{He}_4(z)|\phi_1(z)/m_4. The proposal density is (z^4+6z^2+3)\phi_1(z)/12, accepted with probability |\mathrm{He}_4(z)|/(z^4+6z^2+3). The code uses c=-b; symmetry of the bias density and its sign factor preserves the synthesis integral.