Abstract

We will prove Weierstrass’ approximation theorem by coin flips

Bernstein polynomials

Let $$f_n:D \to \mathbb{R}$$ be a sequence of functions. Recall that $$f_n \to f$$ uniformly in $$D$$ if $\lim_{n \to \infty}\sup_{x \in D} |f_n(x) - f(x)| =0.$

Theorem (Weierstrass):

Let $$f:[0,1] \to \mathbb{R}$$ be a continuous function. There exists a sequence of polynomials $$f_n$$ that converge uniformly to $$f$$.

Exercise: Show that Weierstrass’ theorem holds on any bounded interval $$[a,b]$$.

The proof we present here, uses probability is due to Bernstein.

Proof (Bernstein): Let $$f:[0,1] \to \mathbb{R}$$ be continuous. Let $$p\in (0,1)$$. Let $$S_n \sim Bern(n,p)$$. Consider the Bernstein polynomial given by $f_n(p) = \mathbb{E}_p(f(S_n/n)) = \sum_{k=0} ^n {n \choose k}f(k/n)p^k (1-p)^{n-k};$ notice that $$f_n(1)= f(1)$$ and $$f_n(0) = f(0)$$.

Let $$\epsilon>0$$. Note that since $$f$$ is continuous on the compact interval $$[0,1]$$, it is uniformly continuous on $$[0,1]$$; so that there is a $$\delta >0$$ (which depends only on $$\epsilon$$ and is independent of $$x$$ and $$y$$) such that if $$|x - y| < \delta$$, then $$|f(x) - f(y)| < \epsilon$$. Also, let $$M= \sup_{x \in [0,1]} |f(x)|$$. Thus for all $$p \in [0,1]$$, we have that

$\begin{eqnarray*} |f_n(p) - f(p)| & = & | \mathbb{E}_p f(S_n/n) - \mathbb{E}_p f(p) | \\ &\leq & \mathbb{E}_p | f(S_n/n) - f(p)| \\ &\leq& \epsilon + 2M\mathbb{P}_p( |S_n/n - p| > \delta) \\ &\leq& \epsilon + 2M\delta^{-2} \mathbb{E}_p|S_n/n -p|^2 \\ &=& \epsilon + 2M \delta^{-2} \mathrm{var}_p(S_n/n) \\ &=& \epsilon + 2M \delta^{-2} p(1-p)/n \\ &\leq& \epsilon + M\delta^{-2} /2n \end{eqnarray*}$ Thus $$\limsup_{n \to \infty} \sup_{p \in [0,1]} |f_n(p) - f(p)| \leq \epsilon$$.

Exercise: Let $$f:[0,1] \to \mathbb{R}$$ be a continuous function. Assume that for all nonnegative integers $$n$$ we have,
$\int_0 ^1 x^n f(x)dx = 0$ Show that $$f=0$$. Hint: you may want to use the fact that if $$\int_0^1 |f(x)|^2 dx = 0$$, then $$f =0$$.

Exercise: Let $$X$$ and $$Y$$ be continuous random variables with continuous pdfs that are supported in $$[0,1]$$. Show that if $$\mathbb{E} X^n = \mathbb{E} Y^n$$ for all nonnegative intergers $$n$$, then $$X$$ and $$Y$$ have the same law.