Proof. The mean of the univariate Gaussian distribution is given by: \begin{align} \mathbb{E}\left[x\right] &= \,\,\,\int_{-\infty}^{\infty} \left(\frac{1}{2\pi\sigma^2}\right)^{1/2}\exp\left\lbrace-\frac{1}{2\sigma^2}\left(x-\mu\right)^2\right\rbrace x\,dx\\ \end{align} Changing the variables using $y = x - \mu$, the above expression becomes \begin{align} \mathbb{E}\left[x\right] &= \,\,\,\int_{-\infty}^{\infty} \left(\frac{1}{2\pi\sigma^2}\right)^{1/2}\exp\left(-\frac{1}{2\sigma^2}y^2\right)\left(y+\mu\right)dy\\ &=\,\,\,\int_{-\infty}^{\infty} \left(\frac{1}{2\pi\sigma^2}\right)^{1/2}\exp\left(-\frac{1}{2\sigma^2}y^2\right)y\,dy \notag\\\ &\,\,\,\,\,\,\,\,+ \int_{-\infty}^{\infty} \left(\frac{1}{2\pi\sigma^2}\right)^{1/2}\exp\left(-\frac{1}{2\sigma^2}y^2\right)\mu\,dy\\ &= \,\,\,\mu \end{align} The first term in the sum is an odd integrand and hence it integrates to 0. In the second term, we can pull the constant $\mu$ outside the integral and we are left with the normalized Gaussian distribution which integrates to 1. □

Note on odd functions:
A function $f$ defined from $\mathbb{R}$ to $\mathbb{R}$ is an odd function if $f(-x) = -f(x)$ for all $x$. Odd functions have the property that for any $a$, $$\int_{-a}^{a}f(x)\,dx = 0,$$ assuming that the integral exists.This is because the area under the function from $-a$ to 0 cancels the area under the function from 0 to $a$. To show this explicitly, let us integrate the first term in the sum from $-\infty$ to 0 and from 0 to $\infty$. This term should integrate to 0. \begin{align*} & \,\,\,\int_{-\infty}^{\infty} \left(\frac{1}{2\pi\sigma^2}\right)^{1/2}\exp\left(-\frac{1}{2\sigma^2}y^2\right)y\,dy\\ &=\,\,\,\int_{-\infty}^{0} \left(\frac{1}{2\pi\sigma^2}\right)^{1/2}\exp\left(-\frac{1}{2\sigma^2}y^2\right)y\,dy\\ &\,\,\,\,\,\,\,\,+ \int_{0}^{\infty} \left(\frac{1}{2\pi\sigma^2}\right)^{1/2}\exp\left(-\frac{1}{2\sigma^2}y^2\right)y\,dy\\ &=\,\,\,\int_{\infty}^{0} \left(\frac{1}{2\pi\sigma^2}\right)^{1/2}\left(\frac{1}{2}\right)\exp\left(-\frac{1}{2\sigma^2}u\right)du\\ &\,\,\,\,\,\,\,\,+ \int_{0}^{\infty} \left(\frac{1}{2\pi\sigma^2}\right)^{1/2}\left(\frac{1}{2}\right)\exp\left(-\frac{1}{2\sigma^2}u\right)du \tag{i}\label{result}\\ &=\,\,\,\left(\frac{1}{2\pi\sigma^2}\right)^{1/2}\left(\frac{1}{2}\right)\left(-{2\sigma^2}\right)\\ &\,\,\,\,\,\,\,\,\,\,\left\lbrace\left[\exp\left(-\frac{1}{2\sigma^2}u\right)\right]_{\infty}^{0} + \left[\exp\left(-\frac{1}{2\sigma^2}u\right)\right]_{0}^{\infty}\right\rbrace\\ &=\,\,\,\left(\frac{1}{2\pi\sigma^2}\right)^{1/2}\left(\frac{1}{2}\right)\left(-{2\sigma^2}\right)\left\lbrace\left[\frac{1}{e^0}-\frac{1}{e^\infty}\right] + \left[\frac{1}{e^\infty}-\frac{1}{e^0}\right]\right\rbrace\\ &=\,\,\,0 \end{align*} where we have used the change of variables $y^2=u$ and hence $2y\,dy=du$. Also from \eqref{result} we can immediately conclude that the expression integrates to 0 since $$\int_{a}^{b} f(x)\, dx + \int_{b}^{a} f(x)\, dx = \int_{a}^{b} f(x)\, dx - \int_{a}^{b} f(x)\, dx = 0$$