8.1. Experiment 1: ReLU with Gaussian random directions🔗

This experiment illustrates Corollary 7.5. On \ell^2, set Q=\operatorname{diag}(j^{-2}) and truncate to d=10,100,1000,10000 coordinates. The test set consists of 200 points with coordinates x_j=u_j/j, where the u_j are uniform on [-1,1]. Compare f_{\mathrm{ReLU},Q}(x)=\sqrt{\langle Qx,x\rangle/(2\pi)} with N^{-1}\sum_{j=1}^N\operatorname{ReLU}(\langle a_j,x\rangle), where a_j\sim\mu_Q, over 20 trials.

The observed mean supremum error decays approximately as N^{-1/2} at every tested resolution. The bound at N=4096 is 0.24, about twenty times the observed error, so this comparison illustrates the rate rather than sharpness of its constant.