Exam SRM · practice
Exam SRM — Model Selection & Diagnostics Practice Flashcards
Thirty exam-realistic multiple-choice problems on SOA Exam SRM model selection and diagnostics — standardized and deleted residuals, leverage $h_{ii}$ and Cook's distance, the variance inflation factor, best-subset and forward/backward stepwise selection, Mallows' $C_p$, adjusted $R^{2}$, AIC and BIC, the LOOCV leverage shortcut, and ridge-vs-lasso regularization — each with a fully worked solution.
8 free sample30 total · in appFree · fact-checked · LaTeX math
Tap card or press Space to flip
Answer
Unlock the full set
You're studying a free 8-problem sample. All 30 Model Selection & Diagnostics practice problems — plus every other Exam SRM subject and spaced-repetition scheduling — are built into the Willys AI Flashcards & Quizzes app. 14-day free trial, then $14.99.
Every deck is built into the Willys app
All of these decks — including the full practice problem banks — come built into Willys AI Flashcards & Quizzes for iPhone & iPad (Mac version coming soon), with FSRS + SM-2 spaced repetition, streaks, and exam-date cram mode. 14-day free trial, then $14.99. To load a deck in the app: Settings → Library → Browse, then pick your exam and deck.
More Exam SRM decks:
Clustering & KNN Clustering & KNN Practice Decision Trees & Ensembles Decision Trees & Ensembles Practice Generalized Linear Models Generalized Linear Models Practice
Browse all 30 problems as a list
- Residual diagnosticsA fitted regression has $\hat\sigma=3.0$. Observation $i$ has raw residual $e_i=7.5$ and leverage $h_{ii}=0.36$. Calculate its standardized (internally studentized) residual. (A) $2.50$ (B) $3.13$ (C) $3.91$ (D) $4.69$ (E) $7.50$**Answer: (B).** The standardized residual is $r_i=\dfrac{e_i}{\hat\sigma\sqrt{1-h_{ii}}}$. $\sqrt{1-h_{ii}}=\sqrt{1-0.36}=\sqrt{0.64}=0.8$, so the denominator is $3.0(0.8)=2.4$. $r_i=\dfrac{7.5}{2.4}=3.125\approx 3.13$. Since $|r_i|>2$ (indeed $>3$), observation $i$ is a clear outlier in the $y$-direction. Dividing by $\hat\sigma$ alone (forgetting the $\sqrt{1-h_{ii}}$ factor) gives $7.5/3.0=2.50$ — distractor (A); dividing by $1-h_{ii}$ instead of its square root gives $7.5/(3.0(0.64))\approx 3.91$ — distractor (C).
- Residual diagnosticsWhich statement about a normal Q-Q plot of regression residuals is correct? (A) Points hugging the $45^{\circ}$ line indicate heteroscedasticity. (B) A funnel (fan) shape is the classic Q-Q signature of non-normal errors. (C) Points bending below the line at the left end and above it at the right end indicate heavy-tailed errors. (D) The Q-Q plot is used to detect non-constant variance, not non-normality. (E) Curvature in the Q-Q plot indicates a missing nonlinear predictor term.**Answer: (C).** A normal Q-Q plot orders the standardized residuals and plots them against standard-normal quantiles; under normality the points fall near the $45^{\circ}$ reference line. Heavy-tailed errors produce more extreme values than a normal at both ends: the low residuals fall **below** the line on the left and the high residuals rise **above** it on the right (an S/sideways pattern). That is exactly choice (C). Choice (D) describes the residual-vs-fitted plot, which diagnoses non-constant variance (the funnel of choice B) and curvature/nonlinearity (choice E) — those belong to a different plot, not the Q-Q plot. Points on the line (A) indicate normality, the opposite of trouble.
- Residual diagnosticsTwo observations have the same raw residual $e=6.0$ but different leverages: point P has $h_{ii}=0.10$ and point Q has $h_{ii}=0.50$. Using $\hat\sigma=4.0$, which point has the larger standardized residual, and why? (A) P, because lower leverage shrinks the residual's variance (B) Q, because higher leverage inflates the standardizing denominator (C) Q, because $\operatorname{Var}(e_i)=\sigma^2(1-h_{ii})$ is smaller, so the same raw residual is more extreme (D) They are equal, since the raw residuals are equal (E) P, because $\operatorname{Var}(e_i)=\sigma^2 h_{ii}$ is smaller for P**Answer: (C).** The variance of a fitted residual is $\operatorname{Var}(e_i)=\sigma^2(1-h_{ii})$, so higher leverage means a **smaller** residual variance. Standardizing divides by $\hat\sigma\sqrt{1-h_{ii}}$. For P: $r_P=\dfrac{6.0}{4.0\sqrt{0.90}}=\dfrac{6.0}{3.7947}\approx 1.581$. For Q: $r_Q=\dfrac{6.0}{4.0\sqrt{0.50}}=\dfrac{6.0}{2.8284}\approx 2.121$. Q is larger. Because Q's residual has the smaller true variance, the same raw deviation of $6.0$ is more standard deviations from zero — choice (C). Choice (B) has the right answer but the wrong reason: higher leverage **shrinks** the denominator $\sqrt{1-h_{ii}}$, it does not inflate it.
- Residual diagnosticsA regression residual plot shows a clear funnel: the spread of the residuals widens as the fitted values increase. Which conclusion and remedy are appropriate? (A) The errors are non-normal; refit with robust regression. (B) A predictor enters nonlinearly; add a quadratic term. (C) The variance is non-constant; the OLS coefficients are biased and must be re-estimated. (D) The variance is non-constant; OLS estimates stay unbiased but the usual standard errors are invalid, so transform $y$ (e.g. $\ln y$) or use weighted least squares. (E) An influential point is present; delete the highest-leverage observation.**Answer: (D).** A funnel/fan shape in the residual-vs-fitted plot is the signature of **heteroscedasticity** (non-constant error variance), not non-normality (B/A) or nonlinearity (B) or a single influential point (E). Under heteroscedasticity, OLS coefficient estimates remain **unbiased**, so they need not be re-estimated — choice (C) is wrong on that point. What breaks is the standard-error formula, so $t$-tests, $F$-tests, and confidence intervals are no longer valid. The standard fixes are a variance-stabilizing transformation of the response (such as $\ln y$ or $\sqrt{y}$), weighted least squares, or heteroscedasticity-robust standard errors — choice (D).
- Leverage & influenceA multiple regression uses $p=4$ predictors fit to $n=80$ observations. Using the common rule that flags a point when $h_{ii}>2\cdot\frac{p+1}{n}$, what is the leverage threshold? (A) $0.050$ (B) $0.0625$ (C) $0.100$ (D) $0.125$ (E) $0.250$**Answer: (D).** The average leverage is $\dfrac{p+1}{n}=\dfrac{4+1}{80}=\dfrac{5}{80}=0.0625$ — distractor (B), the average itself, not the threshold. The high-leverage rule of thumb is **twice** the average: $2\cdot\dfrac{p+1}{n}=2(0.0625)=0.125$. Any observation with $h_{ii}>0.125$ is flagged as having unusually extreme predictor values. Using $p$ instead of $p+1$ in the numerator (forgetting the intercept) gives $2(4/80)=0.10$ — distractor (C).
- Leverage & influenceA simple linear regression has $n=16$, $\bar x=20$, and $S_{xx}=\sum(x_i-\bar x)^2=400$. Calculate the leverage $h_{ii}$ of a point with $x_i=30$, and state whether it exceeds the $2\cdot\frac{p+1}{n}$ threshold. (A) $h_{ii}=0.25$; not flagged (B) $h_{ii}=0.3125$; flagged (C) $h_{ii}=0.25$; flagged (D) $h_{ii}=0.0625$; not flagged (E) $h_{ii}=0.5625$; flagged**Answer: (B).** For simple linear regression, $h_{ii}=\dfrac{1}{n}+\dfrac{(x_i-\bar x)^2}{S_{xx}}$. $(x_i-\bar x)^2=(30-20)^2=100$. $h_{ii}=\dfrac{1}{16}+\dfrac{100}{400}=0.0625+0.25=0.3125$. With $p=1$ predictor, the average leverage is $\dfrac{p+1}{n}=\dfrac{2}{16}=0.125$, so the threshold is $2(0.125)=0.25$. Since $0.3125>0.25$, the point is **flagged** as high leverage — choice (B). Dropping the $\frac{1}{n}$ term gives $0.25$ (distractor A/C).
- Leverage & influenceAn observation in a model with $p=3$ predictors has standardized residual $r_i=2.8$ and leverage $h_{ii}=0.36$. Calculate Cook's distance $D_i$. (A) $0.27$ (B) $0.55$ (C) $0.88$ (D) $1.10$ (E) $1.96$**Answer: (D).** Cook's distance is $D_i=\dfrac{r_i^{2}}{p+1}\cdot\dfrac{h_{ii}}{1-h_{ii}}$, with $p+1=4$. $\dfrac{r_i^{2}}{p+1}=\dfrac{2.8^{2}}{4}=\dfrac{7.84}{4}=1.96$. $\dfrac{h_{ii}}{1-h_{ii}}=\dfrac{0.36}{0.64}=0.5625$. $D_i=1.96(0.5625)\approx 1.1025\approx 1.10$. Since $D_i>1$, the point is flagged as influential. Forgetting the $\frac{h}{1-h}$ leverage factor and just using $r_i^2/(p+1)=1.96$ gives distractor (E); using $h_{ii}$ alone instead of $\frac{h_{ii}}{1-h_{ii}}$ gives $1.96(0.36)\approx 0.71$.
- Leverage & influenceWhich description correctly distinguishes a high-leverage point from an influential point? (A) A high-leverage point always changes the fitted coefficients; an influential point never does. (B) A high-leverage point has an unusual response $y$; an influential point has unusual predictor values. (C) Leverage depends only on the $X$-values; a point is most influential when it has both high leverage and a large residual. (D) Influence depends only on the residual; leverage depends only on the residual too. (E) Every high-leverage point is influential, and every influential point is high-leverage.**Answer: (C).** Leverage $h_{ii}$ is the $i$-th diagonal of the hat matrix and depends **only on the predictor values** $X$, not on $y$ — it measures how unusual a point's $X$-position is. A point is **influential** when its removal substantially shifts the fitted coefficients, which Cook's distance captures by blending outlyingness ($r_i^2$) with leverage ($\frac{h_{ii}}{1-h_{ii}}$). A point is most influential when it has **both** high leverage **and** a large residual — choice (C). A high-leverage point with a tiny residual sits right on the trend line and is **not** influential, so (A) and (E) are false. Choice (B) swaps the definitions of leverage and outlier.