Let \(\rho \ll \mu \) be probability measures with \(\log \frac{d\rho }{d\mu } \in L^1(\rho )\), and let \(g \in L^1(\rho )\) with \(e^g \in L^1(\mu )\). Then \(\int g \, d\rho \le \int \log \frac{d\rho }{d\mu } \, d\rho + \log \int e^g \, d\mu \).

Let \(\rho , \mu \) be probability measures with \(\mathrm{KL}(\rho \| \mu ) {\lt} \infty \), and let \(g \in L^1(\rho )\) with \(\int e^{g} \, d\mu {\lt} \infty \). Then \(\int g \, d\rho \le \mathrm{KL}(\rho \| \mu ) + \log \int e^{g} \, d\mu \).

Let \(\rho , \mu \) be probability measures with \(\mathrm{KL}(\rho \| \mu ) {\lt} \infty \), and let \(g \ge 0\) be measurable with \(\int e^{g} \, d\mu {\lt} \infty \) (Lebesgue integral). Then \(\int g \, d\rho \le \mathrm{KL}(\rho \| \mu ) + \log \int e^{g} \, d\mu \), where the left-hand side is the Bochner integral (equal to \(0\) if \(g \notin L^1(\rho )\)).

Let \(\rho , \mu \) be probability measures with \(\mathrm{KL}(\rho \| \mu ) {\lt} \infty \), and let \(g\) be measurable with \(|g| \le C\). Then \(\int g \, d\rho \le \mathrm{KL}(\rho \| \mu ) + \log \int e^{g} \, d\mu \).

Let \(\rho ,\mu \) be probability measures with \(\operatorname {KL}(\rho \| \mu ){\lt}\infty \) and let \(g\ge 0\) be measurable with \(\int e^g\, \mathrm d\mu {\lt}\infty \). Then \(\int g\, \mathrm d\rho \le \operatorname {KL}(\rho \| \mu )+\log \int e^g\, \mathrm d\mu \) in \([0,\infty ]\), where the left-hand side is the Lebesgue integral of \(g\); in particular \(g\in L^1(\rho )\). The proof applies the bounded Donsker–Varadhan inequality to \(g\wedge n\) and lets \(n\to \infty \) by monotone convergence.

Let \(\rho ,\mu \) be probability measures with \(\operatorname {KL}(\rho \| \mu ){\lt}\infty \) and let \(g\ge 0\) be measurable with \(\int e^g\, \mathrm d\mu {\lt}\infty \). Then \(g\in L^1(\rho )\).

For \(v \ge 0\) and \(2 s v {\lt} 1\), \(x \mapsto e^{s x^2}\) is integrable with respect to \(\mathcal{N}(0, v)\).

For \(v \ge 0\) and \(2 s v {\lt} 1\), \(\int e^{s x^2} \, \mathcal{N}(0, v)(dx) = (1 - 2 s v)^{-1/2}\).

For \(v \ge 0\), \(\int |x| \, \mathcal{N}(0, v)(dx) = \sqrt{2 v / \pi }\).

For \(\mu \in \mathbb {R}\) and \(v \ge 0\), \(\int |x| \, \mathcal{N}(\mu , v)(dx) \le |\mu | + \sqrt{2 v / \pi }\).

For \(\mu \in \mathbb {R}\) and \(v \ge 0\), \(\int x^2 \, \mathcal{N}(\mu , v)(dx) = \mu ^2 + v\).

For \(\lambda {\gt} 0\), \(\beta {\gt} 0\) and \(s \in \mathbb {R}\), \(\int _{\mathbb {R}} e^{-(a s + \lambda a^2/2)/\beta } \, da = \sqrt{2\pi \beta /\lambda } \, e^{s^2/(2\lambda \beta )}\).

For \(\lambda {\gt} 0\), \(\beta {\gt} 0\) and \(s \in \mathbb {R}\), as measures on \(\mathbb {R}\), \(e^{-(a s + \lambda a^2/2)/\beta } \, da = \sqrt{2\pi \beta /\lambda } \, e^{s^2/(2\lambda \beta )} \, \mathcal{N}(-s/\lambda , \beta /\lambda )(da)\).

For \(\lambda ,\beta {\gt}0\) and \(c\in {\mathbb R}\), as measures on \({\mathbb R}\), \(e^{-ac/\beta }\, {\mathcal N}(0,\beta /\lambda )(\, \mathrm da) =e^{c^2/(2\lambda \beta )}\, {\mathcal N}(-c/\lambda ,\beta /\lambda )(\, \mathrm da)\).

For \(v{\gt}0\) and \(m,m'\in {\mathbb R}\), as measures on \({\mathbb R}\), \({\mathcal N}(m,v)(\, \mathrm da)=\exp \Bigl(\frac{a(m-m')-(m^2-m'{}^2)/2}{v}\Bigr){\mathcal N}(m',v)(\, \mathrm da)\).

Let \(\mu , \nu \) be finite measures and \(g\) a measurable function with \(|g| \le C\). Then \(\left|\int g \, d\mu - \int g \, d\nu \right| \le 2 C \, \| \mu - \nu \| _{\mathrm{TV}}\).

For finite measures \(\mu ,\nu \) and a measurable map \(g\), \(\| g_*\mu -g_*\nu \| _{\mathrm{TV}}\le \| \mu -\nu \| _{\mathrm{TV}}\), since \(g_*\mu (A)-g_*\nu (A) =\mu (g^{-1}A)-\nu (g^{-1}A)\).

If \(\mu (A)-\mu (A^c)\le 2C\) for every measurable \(A\) then \(\| \mu \| _{\mathrm{TV}}\le C\): on a Hahn set \(S\) of \(\mu \), \(|\mu |(Z)=\mu (S^c)-\mu (S)\).

For a finite measure \(\mu \) and \(\varphi \in L^1(\mu )\) measurable, the total variation of the signed measure \(\varphi \, \mu \) is \(\int |\varphi |\, \mathrm d\mu \) (its Jordan decomposition is \((\varphi ^+\mu ,\varphi ^-\mu )\)).

With \(w_i=\, \mathrm d\nu _i/\, \mathrm d(\nu _1+\nu _2)\), \(\int |w_1-w_2|\, \mathrm d(\nu _1+\nu _2)\le 2\| \nu _1-\nu _2\| _{\mathrm{TV}}\) (split on \(\{ w_1\ge w_2\} \)).

For \(p \in [0,1]\) and \(q \in (0,1)\), \(p \log \frac{p}{q} + (1-p) \log \frac{1-p}{1-q} \ge 2 (p - q)^2\).

Let \(\rho , \mu \) be probability measures with \(\mathrm{KL}(\rho \| \mu ) {\lt} \infty \). Then for every measurable set \(A\), \(|\rho (A) - \mu (A)| \le \sqrt{\mathrm{KL}(\rho \| \mu ) / 2}\).

Let \(\rho , \mu \) be probability measures with \(\mathrm{KL}(\rho \| \mu ) {\lt} \infty \). Then \(\| \rho - \mu \| _{\mathrm{TV}} \le \sqrt{\mathrm{KL}(\rho \| \mu ) / 2}\).

Let \(\rho , \mu \) be probability measures with \(\mathrm{KL}(\rho \| \mu ) {\lt} \infty \) and \(g\) a measurable function with \(|g| \le C\). Then \(\left|\int g \, d\rho - \int g \, d\mu \right| \le 2 C \sqrt{\mathrm{KL}(\rho \| \mu ) / 2}\).

For probability measures \(\rho ,\rho '\) with \(\operatorname {KL}(\rho \| \rho ')\) and \(\operatorname {KL}(\rho '\| \rho )\) finite, \(\| \rho -\rho '\| _{\mathrm{TV}}^2 \le \frac12\min \{ \operatorname {KL}(\rho \| \rho '),\operatorname {KL}(\rho '\| \rho )\} \), hence \(2\| \rho -\rho '\| _{\mathrm{TV}}\le \sqrt{\operatorname {KL}(\rho \| \rho ')+\operatorname {KL}(\rho '\| \rho )}\).

If \(\gamma =\mu _1-\mu _2\) with \(\mu _1,\mu _2\) finite measures and \(f\) is integrable for \(\mu _1\) and \(\mu _2\), then \(\int f\, \, \mathrm d\gamma =\int f\, \, \mathrm d\mu _1-\int f\, \, \mathrm d\mu _2\).

For \(h\in L^1(\nu )\) and \(g\) measurable with \(gh\in L^1(\nu )\), \(\int g\, \, \mathrm d(h\nu )=\int gh\, \, \mathrm d\nu \).

On a metrizable space, two finite signed measures with the same integrals \(\int g\, \, \mathrm d\gamma =\int g\, \, \mathrm d\gamma '\) for every \(g\in C_b\) are equal.

Let \(\rho ,\mu \) be probability measures with \(\operatorname {KL}(\rho \| \mu ){\lt}\infty \), let \(A\) be measurable and \(t\ge 0\). Then \(t\, \rho (A)\le \operatorname {KL}(\rho \| \mu )+\log \bigl(1+(e^t-1)\mu (A)\bigr)\). (Donsker–Varadhan with \(g=t\, \mathbf1_A\), for which \(\int e^g\, \mathrm d\mu =1+(e^t-1)\mu (A)\).) With \(t=\log (1+1/\mu (A))\) this is \(\rho (A)\le [\operatorname {KL}(\rho \| \mu )+\log 2]/\log (1+1/\mu (A))\).

Let \(\mu \) be a probability measure on a Hausdorff space such that \(\{ \mu \} \) is tight (e.g. any probability measure on a Polish space), and let \(C\in {\mathbb R}\). Then the sublevel set \(\{ \rho \in {\mathcal P}:\operatorname {KL}(\rho \| \mu )\le C\} \) is tight. Indeed, given \({\varepsilon }{\gt}0\) let \(t:=(\max (C,0)+\log 2)/{\varepsilon }\) and let \(K\) be compact with \(\mu (K^c)\le e^{-t}\); by Lemma 858, \(t\rho (K^c)\le C+\log (1+(e^t-1)e^{-t})\le C+\log 2\), so \(\rho (K^c)\le {\varepsilon }\).

Let \(\mu \) be a probability measure on a Polish space and \(C\in {\mathbb R}\). Then \(\{ \rho \in {\mathcal P}:\operatorname {KL}(\rho \| \mu )\le C\} \) is tight (Lemma 859, since \(\{ \mu \} \) is tight on a Polish space).

For probability measures \(\rho ,\mu \) and bounded measurable \(g\), \(\mathrm{DV}_{\rho ,\mu }(g)\le \operatorname {KL}(\rho \| \mu )\) in \([0,\infty ]\) (Lemma 829).

For probability measures \(\rho ,\mu \) on a topological space with its Borel \(\sigma \)-algebra and \(g\in C_b\), \(\mathrm{DV}_{\rho ,\mu }(g)\le \operatorname {KL}(\rho \| \mu )\).

Let \(\rho \ll \mu \) be finite measures. Then \((\log \frac{d\rho }{d\mu })^-\) is \(\rho \)-integrable: \(\int (\log \frac{d\rho }{d\mu })^-\, \mathrm d\rho =\int r\, (\log r)^-\, \mathrm d\mu \le \mu (\alpha )\) with \(r=\frac{d\rho }{d\mu }\), since \(r(\log r)^-\le 1-r\le 1\) on \((0,1)\).

Let \(\rho ,\mu \) be probability measures and \(K\in [0,\infty ]\) with \(\mathrm{DV}_{\rho ,\mu }(g)\le K\) for every bounded measurable \(g\). Then \(\operatorname {KL}(\rho \| \mu )\le K\). Proof: if \(\rho \not\ll \mu \), the functions \(t\mathbf1_A\) with \(\mu (A)=0{\lt}\rho (A)\) give \(\mathrm{DV}=t\rho (A)\to \infty \). Otherwise let \(r=d\rho /d\mu \) and \(g_n:=\min (n,\log \max (r,e^{-n}))\), so that \(|g_n|\le n\) and \(\int e^{g_n}\, \mathrm d\mu \le \int (r+e^{-n})\, \mathrm d\mu =1+e^{-n}\); hence \(\int g_n\, \mathrm d\rho \le K+\log (1+e^{-n})\). Since \(\min (n,(\log r)^+)\le g_n+(\log r)^-\) and \((\log r)^-\in L^1(\rho )\) (Lemma 864), monotone convergence gives \((\log r)^+\in L^1(\rho )\), so \(\operatorname {KL}(\rho \| \mu )=\int \log r\, \mathrm d\rho =\lim \int g_n\, \mathrm d\rho \le K\) by dominated convergence.

Let \(\rho ,\mu \) be probability measures on a metrizable space with its Borel \(\sigma \)-algebra, and \(K\in [0,\infty ]\) with \(\mathrm{DV}_{\rho ,\mu }(g)\le K\) for every \(g\in C_b\). Then \(\mathrm{DV}_{\rho ,\mu }(g)\le K\) for every bounded measurable \(g\). Proof: given \(|g|\le C\) and \({\varepsilon }{\gt}0\), choose \(g_1\in C_b\) with \(\int |g-g_1|\, \mathrm d(\rho +\mu )\le {\varepsilon }\) (density of \(C_b\) in \(L^1\) of a finite measure on a metrizable space) and clamp it to \(g_2:=\max (-C,\min (C,g_1))\in C_b\), which still satisfies \(\int |g-g_2|\, \mathrm d(\rho +\mu )\le {\varepsilon }\). Then \(|\int g\, \mathrm d\rho -\int g_2\, \mathrm d\rho |\le {\varepsilon }\) and \(|\int e^g\, \mathrm d\mu -\int e^{g_2}\, \mathrm d\mu |\le e^C{\varepsilon }\), so \(\mathrm{DV}(g)\le \mathrm{DV}(g_2)+{\varepsilon }+\log (\int e^g\, \mathrm d\mu +e^C{\varepsilon })-\log \int e^g\, \mathrm d\mu \), and the error tends to \(0\) as \({\varepsilon }\to 0\).

Let \(\rho ,\mu \) be probability measures on a metrizable space with its Borel \(\sigma \)-algebra and \(K\in [0,\infty ]\) with \(\mathrm{DV}_{\rho ,\mu }(g)\le K\) for every \(g\in C_b\). Then \(\operatorname {KL}(\rho \| \mu )\le K\) (Lemmas 866 and 865).

Let \(\rho ,\mu \) be probability measures on a metrizable space with its Borel \(\sigma \)-algebra. Then \(\operatorname {KL}(\rho \| \mu )=\sup _{g\in C_b}\Bigl[\int g\, \mathrm d\rho -\log \int e^g\, \mathrm d\mu \Bigr]\) in \([0,\infty ]\) (the supremum of the nonnegative parts; the inequality \(\ge \) is the Donsker–Varadhan inequality, and \(\le \) is Lemma 867).

Let \(\mu \) be a probability measure on a metrizable space with its Borel \(\sigma \)-algebra, and let \(\rho _i\to \rho \) weakly (along a filter) in \({\mathcal P}\). Then \(\operatorname {KL}(\rho \| \mu )\le \liminf _i\operatorname {KL}(\rho _i\| \mu )\). Proof: for \(g\in C_b\), \(\mathrm{DV}_{\rho ,\mu }(g)=\lim _i\mathrm{DV}_{\rho _i,\mu }(g)\le \liminf _i\operatorname {KL}(\rho _i\| \mu )\) by the Donsker–Varadhan inequality and the weak convergence, and the supremum over \(g\) is \(\operatorname {KL}(\rho \| \mu )\) by Lemma 868.

Let \(\mu _i\) be \(\sigma \)-finite measures and \(f_i\ge 0\) be \(\mu _i\)-integrable. Then \(\bigotimes _i(f_i\mu _i)=\bigl(\prod _if_i(x_i)\bigr)\bigotimes _i\mu _i\). (Both sides agree on measurable boxes by Fubini.)

For a \(\sigma \)-finite \(\mu \) and a permutation \(\sigma \) of the finite index set, the image of \(\mu ^{\otimes \iota }\) under \(\theta \mapsto \theta \circ \sigma \) is \(\mu ^{\otimes \iota }\).

If \(T\) is measurable with \(T_*\nu =\nu \) and \(D\circ T=D\) for a measurable \(D\ge 0\), then \(T_*(D\nu )=D\nu \).

For a \(\sigma \)-finite \(\rho \), measurable \(Y\) and \(t\in {\mathbb R}\), \(\int e^{t\sum _iY(x_i)}\, \rho ^{\otimes \iota }(\, \mathrm dx)=\bigl(\int e^{tY}\, \mathrm d\rho \bigr)^{|\iota |}\) (Fubini; no integrability is needed, both sides being \(0\) when the right-hand factor is not integrable).

For all \(x\in {\mathbb R}\), \(e^x\le 1+x+x^2e^{|x|}\). (For \(|x|\le 1\) this is the Taylor bound \(|e^x-1-x|\le x^2\); for \(x\ge 1\), \(e^x\le x^2e^x\); for \(x\le -1\), \(e^x\le 1\le 1+x+x^2\).)

Let \(\rho \in {\mathcal P}(\Omega )\) and \(Y\) be measurable with \({\mathbb E}_\rho Y=0\) and \(K:={\mathbb E}_\rho [Y^2e^{|Y|}]{\lt}\infty \). Then for \(|u|\le 1\), \({\mathbb E}_\rho e^{uY}\le 1+Ku^2\), hence \(\log {\mathbb E}_\rho e^{uY}\le Ku^2\). (Integrate \(e^{uY}\le 1+uY+u^2Y^2e^{|Y|}\).)

For \(\sigma \)-finite \(\mu ,\nu \) and a measurable isomorphism \(e\), \(\operatorname {KL}(e_*\mu \| e_*\nu )=\operatorname {KL}(\mu \| \nu )\).

Let \(\rho \) be a finite measure on \(\alpha \times \Omega \) with \(\Omega \) standard Borel, \(\nu _1\) finite and \(\nu _2\) a probability measure. Then \(\operatorname {KL}(\rho ^{(1)}\| \nu _1)\le \operatorname {KL}(\rho \| \nu _1\otimes \nu _2)\), where \(\rho ^{(1)}\) is the first marginal. (Chain rule: \(\operatorname {KL}(\rho \| \nu _1\otimes \nu _2)=\operatorname {KL}(\rho ^{(1)}\| \nu _1) +\operatorname {KL}(\rho \| \rho ^{(1)}\otimes \nu _2)\) with \(\rho =\rho ^{(1)}\otimes _{\mathrm m}\kappa \) its disintegration.)

Under the hypotheses of Lemma 877 with the roles of the factors exchanged (\(\alpha \) standard Borel, \(\nu _1\) a probability measure), \(\operatorname {KL}(\rho ^{(2)}\| \nu _2)\le \operatorname {KL}(\rho \| \nu _1\otimes \nu _2)\).

Let \(\Omega \) be standard Borel, \(\nu \in {\mathcal P}(\Omega )\) and \(\mu \in {\mathcal P}(\Omega ^n)\) with marginals \(\mu ^{(i)}\). Then \(\sum _{i=1}^n\operatorname {KL}(\mu ^{(i)}\| \nu )\le \operatorname {KL}(\mu \| \nu ^{\otimes n})\). (Induction on \(n\) with the chain rule and Lemmas 877, 878.)

Let \(E\) be a real inner product space of dimension \(d\) and let \(n\ge d+2\). For all points \((x_i,s_i)\in E\times {\mathbb R}\), \(i=1,\dots ,n\), there is a pattern \(T\subseteq [n]\) such that no \((w,b)\in E\times {\mathbb R}\) satisfies \(s_i\le \langle w,x_i\rangle -b\iff i\in T\) for all \(i\). In other words, the affine class \(\{ x\mapsto \langle w,x\rangle -b\} \) has pseudo-dimension at most \(d+1\).

Let \(E\) be a real inner product space of dimension \(d\) and let \(S\subset E\) have \(d+2\) points. There is \(T\subseteq S\) such that for no \((w,c)\in E\times {\mathbb R}\) one has \(\{ v\in S:\langle w,v\rangle +c{\gt}0\} =T\). Hence the VC dimension of the class of affine halfspaces of \(E\) is at most \(d+1\).

\(\mu \) satisfies \(\mathrm{LSI}(\alpha )\) if and only if \(\operatorname {Ent}_\mu (g^2)\le \frac2\alpha \int |\nabla g|^2\, \mathrm d\mu \) for all \(g\in C^1_c\).

For \(u\ge 0\) and \(v\in {\mathbb R}\), \(uv\le u\log u-u+e^v\).

For \(u\ge 0\) and \(t{\gt}0\), \(u\log u-u\log t-u+t\ge 0\).

For \(0\le u\le v\), \(|u\log u|\le 1+|v\log v|\).

For \(f\ge 0\) and every \(t{\gt}0\), \(\operatorname {Ent}_\mu (f)\le \int f\log f\, \mathrm d\mu -\bigl(\int f\, \mathrm d\mu \bigr)\log t-\int f\, \mathrm d\mu +t\); for a probability measure this is \(\operatorname {Ent}_\mu (f)\le \int \bigl(f\log \tfrac ft-f+t\bigr)\, \mathrm d\mu \), with equality for \(t=\int f\, \mathrm d\mu \).

For a probability measure \(\mu \), \(f\ge 0\) and \(h\) with \(f,\ f\log f,\ fh,\ e^h \in L^1(\mu )\), \(\int fh\, \mathrm d\mu \le \operatorname {Ent}_\mu (f)+\bigl(\int f\, \mathrm d\mu \bigr)\bigl(\int e^h\, \mathrm d\mu -1\bigr)\).

For a probability measure \(\mu \) and \(f\ge 0\) with \(f,\ f\log f\in L^1(\mu )\), \(\operatorname {Ent}_\mu (f)\ge 0\).

Let \(\mu ,\nu \) be probability measures with \(\, \mathrm d\nu =h\, \mathrm d\mu \), \(0\le h\le K\). Then \(\operatorname {Ent}_\nu (f)\le K\, \operatorname {Ent}_\mu (f)\) for every \(f\ge 0\) with \(f,\ f\log f\in L^1(\mu )\cap L^1(\nu )\).

Let \(\, \mathrm d\nu =h\, \mathrm d\mu \) with \(0{\lt}c\le h\le K\). Then \(\int \psi \, \mathrm d\mu \le c^{-1}\int \psi \, \mathrm d\nu \) for every integrable \(\psi \ge 0\).

(F3, Holley–Stroock.) If \(\mu \) satisfies \(\mathrm{LSI}(\alpha )\), \(\psi \) is measurable with \(m\le \psi \le M\) and \(\nu :=Z^{-1}e^{-\psi }\mu \), then \(\nu \) satisfies \(\mathrm{LSI}(\alpha e^{-(M-m)})\).

If \(g\) and \(T\) are differentiable and \(\| DT(x)\| \le L\) for all \(x\), then \(|\nabla (g\circ T)(x)|\le L\, |\nabla g(T(x))|\).

(F4, transport.) If \(\mu \) satisfies \(\mathrm{LSI}(\alpha )\) and \(T\) is a proper \(C^1\) map with \(\| DT\| \le L\), then \(T_\# \mu \) satisfies \(\mathrm{LSI}(\alpha /L^2)\).

(F4, transport.) If \(\mu \) satisfies \(\mathrm{LSI}(\alpha )\) and \(T\) is a proper \(C^1\) map which is Lipschitz with constant \(L\), then \(T_\# \mu \) satisfies \(\mathrm{LSI}(\alpha /L^2)\).

For probability measures \(\mu ,\nu \) and a bounded measurable \(f\ge 0\) on \(X\times Y\),

For a differentiable \(g\) on the Euclidean product \(E_1\times E_2\), \(|\nabla g(x,y)|^2=|\nabla _xg(\cdot ,y)(x)|^2+|\nabla _yg(x,\cdot )(y)|^2\).

(F2, tensorization.) If \(\mu _1\) satisfies \(\mathrm{LSI}(\alpha _1)\) on \(E_1\) and \(\mu _2\) satisfies \(\mathrm{LSI}(\alpha _2)\) on \(E_2\), then \(\mu _1\otimes \mu _2\) satisfies \(\mathrm{LSI}(\min \{ \alpha _1,\alpha _2\} )\) on the Euclidean product \(E_1\times E_2\).

(F2, tensorization.) If \(\mu _1\) satisfies \(\mathrm{LSI}(\alpha _1)\) and \(\mu _2\) satisfies \(\mathrm{LSI}(\alpha _2)\), then \(\mu _1\otimes \mu _2\) satisfies \(\mathrm{LSI}(\min \{ \alpha _1,\alpha _2\} )\) on \(\Theta ={\mathbb R}\times Z\).

There is \(C\ge 0\) such that \(|\nabla \chi _R(x)|\le C/R\) for all \(R{\gt}0\) and \(x\).

Let \(\mu \) satisfy \(\mathrm{LSI}(\alpha )\) on a finite-dimensional space, and let \(\rho =p\, \mu \) be a probability measure with \(p{\gt}0\) of class \(C^1\), \(\nabla \log p\in L^2(\rho )\) and \(\operatorname {KL}(\rho \| \mu ){\lt}\infty \). Then \(\operatorname {KL}(\rho \| \mu )\le \frac1{2\alpha }\int |\nabla \log p|^2\, \mathrm d\rho \).

(L1.) If \(\mu \) satisfies \(\mathrm{LSI}(\alpha )\) on a finite-dimensional space and \(\rho \ll \mu \) is a probability measure with a smooth density and \(\operatorname {KL}(\rho \| \mu ){\lt}\infty \), then \(\operatorname {KL}(\rho \| \mu )\le \frac1{2\alpha }I(\rho |\mu )\).

Let \(\rho ,\rho ',\mu \) be probability measures with \(\operatorname {KL}(\rho \| \mu ){\lt}\infty \) and \(\operatorname {KL}(\rho '\| \mu ){\lt}\infty \), and let \({\varepsilon }\in [0,1]\). Then \(\operatorname {KL}((1-{\varepsilon })\rho +{\varepsilon }\rho '\| \mu )\le (1-{\varepsilon })\operatorname {KL}(\rho \| \mu )+{\varepsilon }\operatorname {KL}(\rho '\| \mu ){\lt}\infty \). (Proof: \(\frac{\, \mathrm d\rho _{\varepsilon }}{\, \mathrm d\mu }=(1-{\varepsilon })\frac{\, \mathrm d\rho }{\, \mathrm d\mu }+{\varepsilon }\frac{\, \mathrm d\rho '}{\, \mathrm d\mu }\) and \(\operatorname {KL}(\rho \| \mu )=\int \phi (\frac{\, \mathrm d\rho }{\, \mathrm d\mu })\, \mathrm d\mu \) with \(\phi (x)=x\log x+1-x\) convex.)

For \(\nu =w\, \nu _0\) with \(w{\gt}0\), \(\operatorname {KL}(\nu \| \nu _0)=\int (w\log w+1-w)\, \mathrm d\nu _0\le \int (w-1)^2\, \mathrm d\nu _0=\chi ^2(\nu \| \nu _0)\), since \(\log w\le w-1\).

For \(v{\gt}0\) and \(t\ge 0\), \({\mathcal N}(0,v)(\{ |x|{\gt}t\} )=2\bar\Phi (t/\sqrt v) \le e^{-t^2/(2v)}\). (For \(x\ge t\ge 0\), \(x^2\ge t^2+(x-t)^2\), so \(\phi _v(x)\le e^{-t^2/(2v)}\phi _v(x-t)\); integrate over \(x{\gt}t\) and use \({\mathcal N}(0,v)(0,\infty )\le \tfrac 12\).)

For \(v{\gt}0\) and \(t\ge 0\), \(\int _t^\infty x\, {\mathcal N}(0,v)(\, \mathrm dx)=\sqrt{v/(2\pi )}\, e^{-t^2/(2v)}=v\, \phi _v(t)\).

Let \(v{\gt}0\), \(|m|\le m_0\le T\) and \(X\sim {\mathcal N}(m,v)\). Then \({\mathbb E}\bigl[|X|\, 1_{\{ |X|{\gt}T\} }\bigr]\le \bigl(m_0+\sqrt{2v/\pi }\bigr)e^{-(T-m_0)^2/(2v)}\). (Write \(X=m+W\) with \(W\sim {\mathcal N}(0,v)\): \(|X|{\gt}T\) forces \(|W|{\gt}t:=T-m_0\) and \(|X|\le m_0+|W|\), so the left side is at most \(m_0{\mathbb P}(|W|{\gt}t)+{\mathbb E}[|W|1_{\{ |W|{\gt}t\} }]\), and Lemmas 906 and 907 bound the two terms.)

Let \(K\ge 0\) and \(\Phi \colon [0,\infty )\to {\mathbb R}\) with \(\Phi (t)\, (1+K(t-s))\le \Phi (s)\) for all \(0\le s\le t\). Then \(\Phi (t)\le e^{-Kt}\Phi (0)\) for all \(t\ge 0\). (Iterating on the partition \(kt/n\) gives \(\Phi (t)(1+Kt/n)^n\le \Phi (0)\); let \(n\to \infty \).)

For \(\rho ^2+c^2=1\) and every \(x\in {\mathbb R}\), \(\int He_n(\rho x+cz)\, \gamma (\, \mathrm dz)=\rho ^nHe_n(x)\): the Hermite polynomials are the eigenfunctions of the Ornstein–Uhlenbeck semigroup. (Two-step induction with the recursion \(He_{n+2}=XHe_{n+1}-(n+1)He_n\) and Gaussian integration by parts.)

Both marginals of the correlated standard Gaussian pair are \(\gamma \).

For \(|\rho |\le 1\) and standard Gaussians \((X,Y)\) with correlation \(\rho \), \({\mathbb E}[He_m(X)He_n(Y)]=\rho ^n\, n!\, \delta _{mn}\). (Proof: \({\mathbb E}[He_n(\rho x+\sqrt{1-\rho ^2}Z)] =\rho ^nHe_n(x)\) by the three-term recursion and Gaussian integration by parts, then orthogonality.)

If \(g\in L^2(\gamma )\) and \(\int gHe_n\, \, \mathrm d\gamma =0\) for all \(n\ge 0\), then \(g=0\) \(\gamma \)-a.e.: the Hermite polynomials are complete in \(L^2(\gamma )\). (Every monomial is a combination of Hermite polynomials, so all moments of \(g\gamma \) vanish; by dominated convergence its Fourier transform vanishes, and the positive and negative parts of \(g\gamma \) have the same characteristic function, hence coincide.)

For \(f\in L^2(\gamma )\), \(\| f\| _{L^2(\gamma )}^2=\sum _{n\ge 0}a_n(f)^2\), the series converging.

For \(F,G\in L^2(\gamma )\) and \(|\rho |\le 1\), \({\mathbb E}[F(X)G(Y)]=\sum _{n\ge 0}\langle He_n/\sqrt{n!},F\rangle \langle He_n/\sqrt{n!},G\rangle \rho ^n\). (Expand \(F\) and \(G\) in the Hermite basis, use \({\mathbb E}[He_m(X)He_n(Y)]=\rho ^n\, n!\, \delta _{mn}\) and the continuity of \((F,G)\mapsto {\mathbb E}[F(X)G(Y)]\) on \(L^2(\gamma )\times L^2(\gamma )\).)

For \(f,g\in L^2(\gamma )\), \(|\rho |\le 1\) and standard Gaussians \((X,Y)\) with correlation \(\rho \), \({\mathbb E}[f(X)g(Y)]=\sum _{n\ge 0}a_n(f)\, a_n(g)\, \rho ^n\), the series converging (absolutely, by Parseval and \(|\rho |\le 1\)).