4. Tempered synthesis activations and ReLU
This chapter is Section 5 of the manuscript together with Appendix C. The analysis filter
\rho is a Schwartz function, but the activation that synthesizes a network may be unbounded;
a real \beta\in\mathcal S'(\mathbb R) is paired with the band-pass filter through the
distributional constant
C_{\beta,\rho}^{(\alpha)}=\frac1{2\pi}\langle\widehat\beta,\widehat\rho(-\,\cdot\,)|\cdot|^{-\alpha}\rangle
of Definition 3.1.4, which is well defined because \widehat\rho
vanishes near the origin. The weighted Sobolev activation spaces
\mathcal A_{s,t}=\langle\cdot\rangle^tH^s(\mathbb R), in which this pairing is continuous,
are recalled from Appendix C.