Exam P — Expectation, Variance & MGFs Flashcards

For a discrete $X$ with pmf $p(x)$: $E[X]=\sum_{x} x\,p(x)$. For a continuous $X$ with pdf $f(x)$: $E[X]=\int_{-\infty}^{\infty} x\,f(x)\,dx$. $E[X]$ exists only if the sum/integral converges absolutely.

You do **not** need the distribution of $Y=g(X)$ to find its mean. Use the distribution of $X$ directly: Discrete: $E[g(X)]=\sum_{x} g(x)\,p(x)$. Continuous: $E[g(X)]=\int_{-\infty}^{\infty} g(x)\,f(x)\,dx$.

No. Equality holds when $g$ is affine, $g(x)=ax+b$ — more generally when $g$ is affine on the support of $X$, or when $X$ is degenerate (constant). For a convex $g$, **Jensen's inequality** gives $E[g(X)]\geq g(E[X])$ (e.g. $E[X^{2}]\geq (E[X])^{2}$), strict when $g$ is strictly convex and $X$ is nondegenerate; for concave $g$ the inequality reverses.

$E[X]=\int_{0}^{\infty} S(x)\,dx=\int_{0}^{\infty}\big(1-F(x)\big)\,dx$. For a nonnegative integer-valued $X$: $E[X]=\sum_{k=1}^{\infty}\Pr(X\geq k)$. This 'tail formula' is very efficient for deductible/limit problems.

$E[X]=0(0.4)+1(0.3)+2(0.2)+3(0.1)=0+0.3+0.4+0.3=1.0$. $E[X^{2}]=0(0.4)+1(0.3)+4(0.2)+9(0.1)=0+0.3+0.8+0.9=2.0$. So $E[X]=1.0$ and $E[X^{2}]=2.0$.

$E[X]=\int_{0}^{1} x\cdot 2x\,dx=\int_{0}^{1}2x^{2}\,dx=\frac{2}{3}$. $E[1/X]=\int_{0}^{1}\frac{1}{x}\cdot 2x\,dx=\int_{0}^{1}2\,dx=2$. Note $E[1/X]=2\neq 1/E[X]=3/2$ — averaging a nonlinear $g$ is not $g$ of the average.

$\operatorname{Var}(X)=E[X^{2}]-(E[X])^{2}$. It avoids re-centering each value about the mean: you only need the first two raw moments. The definitional form is $\operatorname{Var}(X)=E\big[(X-\mu)^{2}\big]$ with $\mu=E[X]$.

pdf $f(x)=\frac{1}{100}$ on $[0,100]$. $E[X]=\int_{0}^{100}\frac{x}{100}\,dx=\frac{1}{100}\cdot\frac{100^{2}}{2}=50$. $E[X^{2}]=\int_{0}^{100}\frac{x^{2}}{100}\,dx=\frac{1}{100}\cdot\frac{100^{3}}{3}=\frac{10000}{3}$. $\operatorname{Var}(X)=\frac{10000}{3}-50^{2}=\frac{10000}{3}-2500=\frac{2500}{3}\approx 833.3$ $\big(=\frac{(b-a)^{2}}{12}\big)$.

$E[X]=1\cdot p+0\cdot(1-p)=p=0.25$. $E[X^{2}]=1^{2}\cdot p+0=p=0.25$ (since $X^{2}=X$ for a $0/1$ variable). $\operatorname{Var}(X)=E[X^{2}]-(E[X])^{2}=p-p^{2}=p(1-p)=0.25\times0.75=0.1875$.

Mean $E[X]=np=20(0.3)=6$. Variance $\operatorname{Var}(X)=np(1-p)=20(0.3)(0.7)=4.2$. $\operatorname{SD}(X)=\sqrt{4.2}\approx 2.049$.

$E[X]=1(0.5)+2(0.3)+4(0.2)=0.5+0.6+0.8=1.9$. $E[X^{2}]=1(0.5)+4(0.3)+16(0.2)=0.5+1.2+3.2=4.9$. $\operatorname{Var}(X)=4.9-(1.9)^{2}=4.9-3.61=1.29$.

Expand: $E[(X-c)^{2}]=E[X^{2}]-2cE[X]+c^{2}$. Differentiate in $c$: $-2E[X]+2c=0\Rightarrow c=E[X]$ (second derivative $2>0$, a minimum). The minimum value is $E[X^{2}]-(E[X])^{2}=\operatorname{Var}(X)$. So the mean is the least-squares center, and the variance is that minimal mean-squared deviation.

$E[aX+b]=a\,E[X]+b$. $\operatorname{Var}(aX+b)=a^{2}\operatorname{Var}(X)$, so $\operatorname{SD}(aX+b)=|a|\,\sigma$. A shift $b$ moves the mean but never changes the spread; only the scale $a$ does, and it enters squared.

$E[Y]=3E[X]-5=3(20)-5=55$. $\operatorname{Var}(Y)=3^{2}\operatorname{Var}(X)=9(9)=81$. $\operatorname{SD}(Y)=\sqrt{81}=9=|3|\cdot\sqrt{9}$.

Only the multiplicative factor $\frac{5}{9}$ affects spread; the shift $-32\cdot\frac{5}{9}$ does not. $\operatorname{SD}(C)=\big|\frac{5}{9}\big|\cdot\operatorname{SD}(F)=\frac{5}{9}\times 18=10$. So the standard deviation of the Celsius reading is $10$ degrees.

The constant $+5$ does not affect variance. $\operatorname{Var}(2X-Y)=2^{2}\operatorname{Var}(X)+(-1)^{2}\operatorname{Var}(Y)=4(4)+1(9)=16+9=25$. So $\operatorname{Var}(2X-Y+5)=25$. (Independence: variances add for both sum and difference.)

$E[\bar{X}]=\mu=8$ (the sample mean is unbiased for the mean). $\operatorname{Var}(\bar{X})=\frac{\sigma^{2}}{n}=\frac{4}{25}=0.16$, so $\operatorname{SD}(\bar{X})=\frac{\sigma}{\sqrt{n}}=\frac{2}{5}=0.4$. Averaging shrinks variance by a factor of $n$.

Raw (about the origin): $\mu_{n}'=E[X^{n}]$. Central (about the mean $\mu=E[X]$): $\mu_{n}=E\big[(X-\mu)^{n}\big]$. Note $\mu_{1}'=\mu$, $\mu_{1}=0$, and $\mu_{2}=\operatorname{Var}(X)$.

$E[X^{2}]=\operatorname{Var}(X)+(E[X])^{2}=9+25=34$. Let $Y=2X-3$: $E[Y]=2(5)-3=7$, $\operatorname{Var}(Y)=2^{2}(9)=36$. $E[Y^{2}]=\operatorname{Var}(Y)+(E[Y])^{2}=36+49=85$. So $E[(2X-3)^{2}]=85$.

$E[Y]=E[X^{2}]=\int_{0}^{1}x^{2}\cdot 3x^{2}\,dx=\int_{0}^{1}3x^{4}\,dx=\frac{3}{5}$. $E[Y^{2}]=E[X^{4}]=\int_{0}^{1}x^{4}\cdot 3x^{2}\,dx=\int_{0}^{1}3x^{6}\,dx=\frac{3}{7}$. $\operatorname{Var}(Y)=\frac{3}{7}-\big(\frac{3}{5}\big)^{2}=\frac{3}{7}-\frac{9}{25}=\frac{75-63}{175}=\frac{12}{175}\approx 0.0686$.

$E[X]=\int_{1}^{\infty}x\cdot\frac{2}{x^{3}}dx=\int_{1}^{\infty}2x^{-2}dx=\big[-2x^{-1}\big]_{1}^{\infty}=2$. $E[X^{2}]=\int_{1}^{\infty}x^{2}\cdot\frac{2}{x^{3}}dx=\int_{1}^{\infty}2x^{-1}dx=\big[2\ln x\big]_{1}^{\infty}=\infty$. Since $E[X^{2}]$ diverges, $\operatorname{Var}(X)$ is **infinite** — a hallmark of heavy tails (and no MGF).

Let $U\sim\operatorname{Uniform}(0,1000)$: $E[U]=500$, $E[U^{2}]=\frac{1000^{2}}{3}=333{,}333.33$. $E[X]=0.9(0)+0.1(500)=50$. $E[X^{2}]=0.9(0)+0.1(333{,}333.33)=33{,}333.33$. $\operatorname{Var}(X)=33{,}333.33-50^{2}=33{,}333.33-2500=30{,}833.33$.

Skewness $\gamma_{1}=\frac{E[(X-\mu)^{3}]}{\sigma^{3}}=\frac{\mu_{3}}{\sigma^{3}}$. $\gamma_{1}>0$: right-skewed (long right tail, mean above median — e.g. the exponential). $\gamma_{1}<0$: left-skewed. $\gamma_{1}=0$ for any symmetric distribution.

Kurtosis $\frac{E[(X-\mu)^{4}]}{\sigma^{4}}=\frac{\mu_{4}}{\sigma^{4}}$. Excess kurtosis subtracts $3$. A normal distribution has kurtosis $3$ (excess $0$). Excess $>0$ means heavier tails / sharper peak (leptokurtic); excess $<0$ means lighter tails (platykurtic).

$\operatorname{CV}=\frac{\operatorname{SD}(X)}{E[X]}=\frac{\sigma}{\mu}$ (often as a percentage), defined for $\mu\neq 0$, usually for positive $X$. It is a **scale-free** measure of relative dispersion, letting you compare variability across variables with different units or magnitudes.

$\operatorname{SD}(X)=\sqrt{2{,}560{,}000}=1{,}600$. $\operatorname{CV}=\frac{\operatorname{SD}(X)}{E[X]}=\frac{1{,}600}{2{,}000}=0.8$ (i.e. $80\%$).

For an exponential with mean $\theta$: $\operatorname{SD}=\theta$ and $E[X]=\theta$, so $\operatorname{CV}=\theta/\theta=1$. The CV is scale-free, and changing $\theta$ is just a rescaling $X\to cX$, which leaves $\operatorname{SD}/\operatorname{mean}$ unchanged. So every exponential has $\operatorname{CV}=1$.

$E[S]=3\times100=300$. $\operatorname{Var}(S)=3\times400=1200$ (independence: variances add). $\operatorname{SD}(S)=\sqrt{1200}\approx 34.64$, so $\operatorname{CV}(S)=\frac{34.64}{300}\approx 0.115$. The CV of the sum ($\approx0.115$) is smaller than each component's CV ($20/100=0.2$) — pooling diversifies.

$M_{X}(t)=E\big[e^{tX}\big]$. Discrete: $\sum_{x} e^{tx}p(x)$; continuous: $\int_{-\infty}^{\infty} e^{tx}f(x)\,dx$. It must be finite for $t$ in some open interval containing $0$ for the MGF to be useful. Always $M_{X}(0)=1$.

Differentiate $n$ times and evaluate at $0$: $E[X^{n}]=M_{X}^{(n)}(0)=\frac{d^{n}}{dt^{n}}M_{X}(t)\Big|_{t=0}$. So $E[X]=M_{X}'(0)$ and $E[X^{2}]=M_{X}''(0)$; this is why it 'generates' moments.

This is the normal MGF $\exp(\mu t+\frac{1}{2}\sigma^{2}t^{2})$. Matching exponents: $\mu=3$ and $\frac{1}{2}\sigma^{2}=8\Rightarrow\sigma^{2}=16$. So $E[X]=3$ and $\operatorname{Var}(X)=16$ (with $\operatorname{SD}=4$).

$M_{X}'(t)=6(1-2t)^{-4}\Rightarrow E[X]=M_{X}'(0)=6$. $M_{X}''(t)=48(1-2t)^{-5}\Rightarrow E[X^{2}]=M_{X}''(0)=48$. $\operatorname{Var}(X)=48-6^{2}=48-36=12$. (This is a gamma with $\alpha=3,\theta=2$: mean $\alpha\theta=6$, variance $\alpha\theta^{2}=12$.)

Rule: $M_{aX+b}(t)=e^{bt}\,M_{X}(at)$ (the shift gives $e^{bt}$, the scale rescales the argument). With $a=3,b=1$: $M_{Y}(t)=e^{t}\,M_{X}(3t)=e^{t}\cdot\frac{1}{1-2(3t)}=\frac{e^{t}}{1-6t}$, valid for $t<\frac{1}{6}$.

If two random variables have MGFs that are equal (and finite) on an open interval around $0$, they have the **same distribution**. So an MGF determines the distribution uniquely. This lets you 'recognize the distribution from its MGF' and read off its mean/variance from the standard form.

For independent sums MGFs multiply, $M_{X+Y}(t)=M_{X}(t)M_{Y}(t)$. Poisson MGF: $\exp[\lambda(e^{t}-1)]$. $M_{X+Y}(t)=e^{2(e^{t}-1)}\,e^{3(e^{t}-1)}=e^{5(e^{t}-1)}$. By uniqueness this is $\operatorname{Poisson}(5)$. Independent Poissons add: $\lambda=2+3=5$.

This matches the geometric MGF $\frac{pe^{t}}{1-qe^{t}}$ with $p=0.4,\,q=0.6$ on support $\{1,2,\dots\}$. By uniqueness $X$ is geometric (trials to first success): $E[X]=\frac{1}{p}=\frac{1}{0.4}=2.5$ and $\operatorname{Var}(X)=\frac{q}{p^{2}}=\frac{0.6}{0.16}=3.75$.

For a **nonnegative integer-valued** $X$: $G_{X}(s)=E\big[s^{X}\big]=\sum_{k=0}^{\infty} s^{k}\Pr(X=k)$. It converges at least for $|s|\leq 1$, with $G_{X}(1)=1$. It relates to the MGF by $G_{X}(s)=M_{X}(\ln s)$.

$\Pr(X=k)=\frac{G_{X}^{(k)}(0)}{k!}$ (the $k$-th Taylor coefficient). Factorial moments at $s=1$: $G_{X}'(1)=E[X]$ and $G_{X}''(1)=E[X(X-1)]$, so $\operatorname{Var}(X)=G''_{X}(1)+G'_{X}(1)-[G'_{X}(1)]^{2}$.

$\Pr(X=0)=G_{X}(0)=e^{4(0-1)}=e^{-4}$. $G_{X}'(s)=4e^{4(s-1)}\Rightarrow E[X]=G_{X}'(1)=4$. $G_{X}''(s)=16e^{4(s-1)}\Rightarrow E[X(X-1)]=G_{X}''(1)=16$. $\operatorname{Var}(X)=16+4-4^{2}=4$. (This is $\operatorname{Poisson}(4)$: mean $=$ variance $=4$.)

Markov (requires $X\geq 0$): $\Pr(X\geq a)\leq \frac{E[X]}{a}$ for $a>0$. With $a=2000$: $\Pr(X\geq 2000)\leq \frac{400}{2000}=0.2$. At most $20\%$ of claims can be $2000$ or larger. The bound uses only the mean and is usually loose.

Chebyshev: $\Pr\big(|X-\mu|\geq k\sigma\big)\leq \frac{1}{k^{2}}$ (no shape assumption). The interval is $\mu\pm 10=\mu\pm 2\sigma$, so $k=2$: $\Pr(|X-50|\geq 10)\leq \frac{1}{2^{2}}=0.25$. Thus $\Pr(40<X<60)=1-\Pr(|X-50|\geq 10)\geq 0.75$.

Markov: $\Pr(X\geq1200)\leq 400/1200=0.333$. Chebyshev ($\sigma=400$, $1200=\mu+2\sigma$): the two-sided bound $\Pr(|X-400|\geq 800)\leq 1/4=0.25$, and since $\{X\geq 1200\}\subseteq\{|X-400|\geq 800\}$ this gives $\Pr(X\geq1200)\leq 0.25$. Exact: $\Pr(X\geq1200)=e^{-1200/400}=e^{-3}\approx 0.0498$. The inequalities are valid but loose; exact distributional knowledge is far tighter.