{
  "deckName": "Exam MAS-I — Markov Chains",
  "examCode": "Exam MAS-I",
  "cards": [
    {
      "front": "State the **Markov property** for a discrete-time stochastic process $\\{X_n\\}$.",
      "back": "The future depends on the past only through the present:\n$\\Pr(X_{n+1}=j \\mid X_n=i, X_{n-1}=i_{n-1},\\dots,X_0=i_0) = \\Pr(X_{n+1}=j \\mid X_n=i)$.\nGiven the current state, the next state is conditionally independent of the entire earlier history. A chain is **time-homogeneous** when this one-step probability does not depend on $n$.",
      "tag": "Transition matrices"
    },
    {
      "front": "Define the **one-step transition matrix** $P$ and the constraints its entries satisfy.",
      "back": "$P$ is the matrix with entries $P_{ij}=\\Pr(X_{n+1}=j \\mid X_n=i)$, the probability of moving from state $i$ to state $j$ in one step.\nIt is a **stochastic matrix**: every entry $P_{ij}\\ge 0$, and **each row sums to $1$**, $\\sum_j P_{ij}=1$, because from any state the chain must go somewhere.",
      "tag": "Transition matrices"
    },
    {
      "front": "Given $P=\\begin{pmatrix} 0.7 & 0.3 \\\\ 0.4 & 0.6 \\end{pmatrix}$ on states $\\{1,2\\}$, identify $\\Pr(X_1=2\\mid X_0=1)$ and verify $P$ is stochastic.",
      "back": "$\\Pr(X_1=2\\mid X_0=1)=P_{12}=0.3$.\nRow 1 sums to $0.7+0.3=1$ and row 2 sums to $0.4+0.6=1$, all entries are $\\ge 0$, so $P$ is a valid stochastic matrix.\nState $1$ stays put with probability $0.7$ and jumps to state $2$ with probability $0.3$.",
      "tag": "Transition matrices"
    },
    {
      "front": "State the **Chapman–Kolmogorov equations** and the resulting form of the $n$-step transition matrix.",
      "back": "$\\Pr(X_{m+n}=j \\mid X_0=i) = \\sum_{k} (P^{m})_{ik}\\,(P^{n})_{kj}$, i.e. $P^{(m+n)}=P^{(m)}P^{(n)}$.\nConsequently the $n$-step matrix is just the matrix power $P^{(n)}=P^{n}$, and\n$\\Pr(X_n=j \\mid X_0=i)=(P^{n})_{ij}$.\nGoing from $i$ to $j$ in $m+n$ steps means passing through some intermediate state $k$ at time $m$ and summing over all such $k$.",
      "tag": "Chapman-Kolmogorov"
    },
    {
      "front": "For $P=\\begin{pmatrix} 0.7 & 0.3 \\\\ 0.4 & 0.6 \\end{pmatrix}$, compute the **two-step** matrix $P^{2}$.",
      "back": "$P^{2}=P\\cdot P$. Entry by entry:\n$(P^2)_{11}=0.7(0.7)+0.3(0.4)=0.49+0.12=0.61$.\n$(P^2)_{12}=0.7(0.3)+0.3(0.6)=0.21+0.18=0.39$.\n$(P^2)_{21}=0.4(0.7)+0.6(0.4)=0.28+0.24=0.52$.\n$(P^2)_{22}=0.4(0.3)+0.6(0.6)=0.12+0.36=0.48$.\nSo $P^{2}=\\begin{pmatrix} 0.61 & 0.39 \\\\ 0.52 & 0.48 \\end{pmatrix}$; each row still sums to $1$.",
      "tag": "Chapman-Kolmogorov"
    },
    {
      "front": "Using $P=\\begin{pmatrix} 0.7 & 0.3 \\\\ 0.4 & 0.6 \\end{pmatrix}$, find $\\Pr(X_2=1 \\mid X_0=1)$ two ways and confirm they agree.",
      "back": "**Matrix power:** $(P^2)_{11}=0.61$ (from $P^2$ above).\n**Conditioning on the intermediate state** $X_1$:\n$\\Pr(X_2=1\\mid X_0=1)=P_{11}P_{11}+P_{12}P_{21}=0.7(0.7)+0.3(0.4)=0.49+0.12=0.61$.\nBoth give $0.61$ — this is exactly Chapman–Kolmogorov, summing the path $1\\to 1\\to 1$ and the path $1\\to 2\\to 1$.",
      "tag": "Chapman-Kolmogorov"
    },
    {
      "front": "How do you compute the **unconditional distribution** of $X_n$ from an initial distribution row vector $\\alpha$?",
      "back": "If $\\alpha=(\\alpha_1,\\alpha_2,\\dots)$ is the row vector of starting probabilities ($\\Pr(X_0=i)=\\alpha_i$), then the distribution after $n$ steps is the row vector\n$\\alpha P^{n}$,\nwhose $j$-th entry is $\\Pr(X_n=j)=\\sum_i \\alpha_i (P^n)_{ij}$.\nLeft-multiplying by a row vector mixes the rows of $P^n$ according to the initial state probabilities.",
      "tag": "Chapman-Kolmogorov"
    },
    {
      "front": "With $P=\\begin{pmatrix} 0.7 & 0.3 \\\\ 0.4 & 0.6 \\end{pmatrix}$ and initial distribution $\\alpha=(0.5,\\,0.5)$, find the distribution of $X_2$.",
      "back": "First the one-step distribution $\\alpha P = (0.5,0.5)\\begin{pmatrix} 0.7 & 0.3 \\\\ 0.4 & 0.6 \\end{pmatrix} = (0.55,\\,0.45)$.\nThen $X_2$: $(0.55,0.45)P = \\big(0.55(0.7)+0.45(0.4),\\ 0.55(0.3)+0.45(0.6)\\big)$\n$=(0.385+0.180,\\ 0.165+0.270)=(0.565,\\,0.435)$.\nEquivalently $\\alpha P^2 = (0.5,0.5)\\begin{pmatrix} 0.61 & 0.39 \\\\ 0.52 & 0.48 \\end{pmatrix}=(0.565,0.435)$.",
      "tag": "Chapman-Kolmogorov"
    },
    {
      "front": "A weather chain has states Sunny ($S$) and Rainy ($R$) with $P=\\begin{pmatrix} 0.8 & 0.2 \\\\ 0.5 & 0.5 \\end{pmatrix}$. Today is Sunny. Find $\\Pr(X_3=S)$.",
      "back": "Start from $\\alpha=(1,0)$ (Sunny today is row 1).\n$\\alpha P = (0.8,0.2)$.\n$\\alpha P^2 = (0.8,0.2)\\begin{pmatrix} 0.8 & 0.2 \\\\ 0.5 & 0.5 \\end{pmatrix}=(0.64+0.10,\\ 0.16+0.10)=(0.74,0.26)$.\n$\\alpha P^3 = (0.74,0.26)\\begin{pmatrix} 0.8 & 0.2 \\\\ 0.5 & 0.5 \\end{pmatrix}=(0.592+0.130,\\ 0.148+0.130)=(0.722,0.278)$.\nSo $\\Pr(X_3=S)=0.722$.",
      "tag": "Chapman-Kolmogorov"
    },
    {
      "front": "An NCD auto-insurance chain has discount levels $\\{0\\%,20\\%,40\\%\\}$ with $P=\\begin{pmatrix} 0.2 & 0.8 & 0 \\\\ 0.1 & 0 & 0.9 \\\\ 0 & 0.1 & 0.9 \\end{pmatrix}$. A new policyholder starts at $0\\%$. Find the level distribution after $2$ years.",
      "back": "Start $\\alpha=(1,0,0)$.\nYear 1: $\\alpha P = (0.2,\\,0.8,\\,0)$ — row 1 of $P$.\nYear 2: $(0.2,0.8,0)P$:\nLevel $0\\%$: $0.2(0.2)+0.8(0.1)+0(0)=0.04+0.08=0.12$.\nLevel $20\\%$: $0.2(0.8)+0.8(0)+0(0.1)=0.16$.\nLevel $40\\%$: $0.2(0)+0.8(0.9)+0(0.9)=0.72$.\nDistribution after $2$ years: $(0.12,\\,0.16,\\,0.72)$, summing to $1$.",
      "tag": "Chapman-Kolmogorov"
    },
    {
      "front": "Define a **path probability**: for a chain starting in state $i$, what is the probability of a specific sequence of states?",
      "back": "By the Markov property the joint probability factors into one-step transitions:\n$\\Pr(X_0=i_0,X_1=i_1,\\dots,X_n=i_n)=\\alpha_{i_0}\\,P_{i_0 i_1}\\,P_{i_1 i_2}\\cdots P_{i_{n-1} i_n}$.\nMultiply the initial probability by each successive one-step transition along the path. For example, with $\\alpha_{i_0}=1$, $\\Pr(\\text{path } 1\\to 2\\to 2 \\mid X_0=1)=P_{12}\\,P_{22}$.",
      "tag": "Transition matrices"
    },
    {
      "front": "In the weather chain $P=\\begin{pmatrix} 0.8 & 0.2 \\\\ 0.5 & 0.5 \\end{pmatrix}$ (states $S,R$), starting Sunny, find $\\Pr(\\text{Sunny, Rainy, Rainy})$ over days $0,1,2$.",
      "back": "The path is $S\\to R\\to R$ with $X_0=S$ given (probability $1$).\n$\\Pr = P_{SR}\\cdot P_{RR}=0.2 \\times 0.5 = 0.10$.\nMultiply the one-step probabilities along the path; there is no sum because we fixed the entire sequence rather than just the endpoint.",
      "tag": "Transition matrices"
    },
    {
      "front": "Define **accessibility** ($i\\to j$) and **communication** ($i\\leftrightarrow j$) of states.",
      "back": "State $j$ is **accessible** from $i$ (written $i\\to j$) if $(P^n)_{ij}>0$ for some $n\\ge 0$ — the chain can reach $j$ from $i$ in finitely many steps.\nStates $i$ and $j$ **communicate** ($i\\leftrightarrow j$) if each is accessible from the other. Communication is an **equivalence relation** (reflexive, symmetric, transitive), so it partitions the state space into **communicating classes**.",
      "tag": "State classification"
    },
    {
      "front": "Define an **irreducible** chain and a **closed** (absorbing) communicating class.",
      "back": "A chain is **irreducible** if all states communicate — there is a single communicating class, so every state can reach every other.\nA communicating class $C$ is **closed** if no state outside $C$ is accessible from inside $C$ (once the chain enters $C$ it never leaves). A single state $i$ with $P_{ii}=1$ is an **absorbing** state — a closed class by itself.",
      "tag": "State classification"
    },
    {
      "front": "Distinguish **recurrent** and **transient** states.",
      "back": "Let $f_i$ be the probability that, starting in $i$, the chain ever returns to $i$.\n**Recurrent:** $f_i=1$ — return is certain, so $i$ is visited infinitely often and the expected number of visits is infinite.\n**Transient:** $f_i<1$ — there is positive probability of never returning, the number of visits is geometric with finite mean $\\frac{1}{1-f_i}$.\nIn a **finite** chain, recurrence/transience is a class property: every state in a class is of the same type, and at least one class is recurrent.",
      "tag": "State classification"
    },
    {
      "front": "How can you tell, from the class structure of a **finite** chain, which states are recurrent and which are transient?",
      "back": "In a finite Markov chain:\n- States in a **closed** communicating class are **recurrent**.\n- States in a class that is **not closed** (the chain can leave and not come back) are **transient**.\nEvery finite chain has at least one closed class, so recurrent states always exist. Absorbing states (closed singletons) are recurrent; states that can reach an absorbing state but cannot be re-entered are transient.",
      "tag": "State classification"
    },
    {
      "front": "Classify the states of $P=\\begin{pmatrix} 0.5 & 0.5 & 0 & 0 \\\\ 0.5 & 0.5 & 0 & 0 \\\\ 0 & 0 & 0 & 1 \\\\ 0.25 & 0 & 0.25 & 0.5 \\end{pmatrix}$ on states $\\{1,2,3,4\\}$.",
      "back": "From the nonzero entries:\n- $1\\leftrightarrow 2$ and $\\{1,2\\}$ is **closed** (rows 1,2 have no mass on $3,4$) → class $\\{1,2\\}$ is **recurrent**.\n- State $4$ reaches $1$ (and $3$); state $3$ reaches $4$; so $3\\leftrightarrow 4$. But from $4$ the chain can leave to $\\{1,2\\}$ and never return, so class $\\{3,4\\}$ is **not closed** → **transient**.\nThus $\\{1,2\\}$ recurrent, $\\{3,4\\}$ transient.",
      "tag": "State classification"
    },
    {
      "front": "Define the **period** $d(i)$ of a state and what **aperiodic** means.",
      "back": "$d(i)=\\gcd\\{\\,n\\ge 1 : (P^n)_{ii}>0\\,\\}$ — the greatest common divisor of the step-counts at which return to $i$ is possible.\nA state is **aperiodic** if $d(i)=1$ (e.g. any state with $P_{ii}>0$ is automatically aperiodic). Period is a class property: all communicating states share the same period. A chain is aperiodic if every state has period $1$.",
      "tag": "State classification"
    },
    {
      "front": "Find the period of every state of the cyclic chain $P=\\begin{pmatrix} 0 & 1 & 0 \\\\ 0 & 0 & 1 \\\\ 1 & 0 & 0 \\end{pmatrix}$.",
      "back": "From state $1$ the chain goes $1\\to 2\\to 3\\to 1$, returning only after $3,6,9,\\dots$ steps, so $(P^n)_{11}>0$ exactly when $n$ is a multiple of $3$.\n$d(1)=\\gcd\\{3,6,9,\\dots\\}=3$. By symmetry $d(2)=d(3)=3$.\nThe chain is irreducible with **period $3$** (periodic, not aperiodic), so a single limiting distribution does not exist even though a stationary distribution does.",
      "tag": "State classification"
    },
    {
      "front": "Define a **stationary distribution** $\\pi$ and the equations it must satisfy.",
      "back": "A row vector $\\pi=(\\pi_1,\\pi_2,\\dots)$ is **stationary** if it is unchanged by one transition:\n$\\pi = \\pi P$, with $\\pi_j\\ge 0$ and the normalization $\\sum_i \\pi_i = 1$.\nIf the chain starts with distribution $\\pi$, then $X_n\\sim\\pi$ for every $n$. The component equations are $\\pi_j=\\sum_i \\pi_i P_{ij}$ for each $j$, plus the normalizing constraint.",
      "tag": "Stationary distribution"
    },
    {
      "front": "State the **limiting-distribution theorem** linking $\\pi$ to $\\lim_n (P^n)_{ij}$ and to mean recurrence time.",
      "back": "For an **irreducible, aperiodic** (ergodic) finite chain there is a unique stationary $\\pi$ and\n$\\lim_{n\\to\\infty}(P^n)_{ij}=\\pi_j$ for every starting state $i$ — the long-run fraction of time spent in $j$, independent of where you start.\nThe **mean recurrence time** (expected number of steps to return to $j$) is $m_{jj}=\\frac{1}{\\pi_j}$.\nIf the chain is periodic, $\\pi$ still exists as the long-run time-average but $(P^n)_{ij}$ need not converge.",
      "tag": "Stationary distribution"
    },
    {
      "front": "Solve for the stationary distribution of $P=\\begin{pmatrix} 0.7 & 0.3 \\\\ 0.4 & 0.6 \\end{pmatrix}$.",
      "back": "Write $\\pi=(\\pi_1,\\pi_2)$ with $\\pi=\\pi P$ and $\\pi_1+\\pi_2=1$.\nFirst component: $\\pi_1 = 0.7\\pi_1 + 0.4\\pi_2 \\Rightarrow 0.3\\pi_1 = 0.4\\pi_2 \\Rightarrow \\pi_1=\\tfrac{4}{3}\\pi_2$.\nNormalize: $\\tfrac{4}{3}\\pi_2 + \\pi_2 = 1 \\Rightarrow \\tfrac{7}{3}\\pi_2=1 \\Rightarrow \\pi_2=\\tfrac{3}{7}$, $\\pi_1=\\tfrac{4}{7}$.\nSo $\\pi=\\left(\\tfrac{4}{7},\\tfrac{3}{7}\\right)\\approx(0.5714,\\,0.4286)$.",
      "tag": "Stationary distribution"
    },
    {
      "front": "For the stationary $\\pi=\\left(\\tfrac{4}{7},\\tfrac{3}{7}\\right)$ of the two-state chain, find the **mean recurrence time** of each state.",
      "back": "The expected number of steps to return to a state is the reciprocal of its stationary probability:\n$m_{11}=\\frac{1}{\\pi_1}=\\frac{1}{4/7}=\\frac{7}{4}=1.75$ steps.\n$m_{22}=\\frac{1}{\\pi_2}=\\frac{1}{3/7}=\\frac{7}{3}\\approx 2.33$ steps.\nState $1$, being visited more often in the long run, returns to itself sooner on average.",
      "tag": "Stationary distribution"
    },
    {
      "front": "Find the stationary distribution of the weather chain $P=\\begin{pmatrix} 0.8 & 0.2 \\\\ 0.5 & 0.5 \\end{pmatrix}$ (states $S,R$). What fraction of days are rainy long-run?",
      "back": "$\\pi=\\pi P$ with $\\pi_S+\\pi_R=1$.\n$\\pi_S = 0.8\\pi_S + 0.5\\pi_R \\Rightarrow 0.2\\pi_S=0.5\\pi_R \\Rightarrow \\pi_S=2.5\\pi_R$.\nNormalize: $2.5\\pi_R+\\pi_R=1 \\Rightarrow 3.5\\pi_R=1 \\Rightarrow \\pi_R=\\frac{1}{3.5}=\\frac{2}{7}\\approx 0.2857$, $\\pi_S=\\frac{5}{7}\\approx 0.7143$.\nLong-run, about $28.6\\%$ of days are rainy.",
      "tag": "Stationary distribution"
    },
    {
      "front": "Solve for the stationary distribution of the $3$-state chain $P=\\begin{pmatrix} 0.5 & 0.3 & 0.2 \\\\ 0.1 & 0.6 & 0.3 \\\\ 0.2 & 0.3 & 0.5 \\end{pmatrix}$.",
      "back": "Solve $\\pi=\\pi P$ (column equations) with $\\pi_3=1-\\pi_1-\\pi_2$.\nState $1$: $\\pi_1=0.5\\pi_1+0.1\\pi_2+0.2\\pi_3$. Substitute $\\pi_3=1-\\pi_1-\\pi_2$: $0.5\\pi_1-0.1\\pi_2-0.2(1-\\pi_1-\\pi_2)=0 \\Rightarrow 0.7\\pi_1+0.1\\pi_2=0.2$.\nState $2$: $\\pi_2=0.3\\pi_1+0.6\\pi_2+0.3\\pi_3$. Substitute: $-0.3\\pi_1+0.4\\pi_2-0.3(1-\\pi_1-\\pi_2)=0 \\Rightarrow 0.7\\pi_2=0.3 \\Rightarrow \\pi_2=\\tfrac{3}{7}\\approx 0.4286$.\nBack into the first: $0.7\\pi_1=0.2-0.1(0.4286)=0.15714 \\Rightarrow \\pi_1\\approx 0.2245$.\n$\\pi_3=1-0.2245-0.4286\\approx 0.3469$.\nSo $\\pi\\approx(0.2245,\\,0.4286,\\,0.3469)$.",
      "tag": "Stationary distribution"
    },
    {
      "front": "Find the stationary distribution of the NCD chain $P=\\begin{pmatrix} 0.2 & 0.8 & 0 \\\\ 0.1 & 0 & 0.9 \\\\ 0 & 0.1 & 0.9 \\end{pmatrix}$ on levels $\\{0\\%,20\\%,40\\%\\}$.",
      "back": "Solve $\\pi=\\pi P$, $\\pi_1+\\pi_2+\\pi_3=1$ (states $1,2,3$ = $0\\%,20\\%,40\\%$).\nCol 1: $\\pi_1=0.2\\pi_1+0.1\\pi_2 \\Rightarrow 0.8\\pi_1=0.1\\pi_2 \\Rightarrow \\pi_2=8\\pi_1$.\nCol 3: $\\pi_3=0.9\\pi_2+0.9\\pi_3 \\Rightarrow 0.1\\pi_3=0.9\\pi_2 \\Rightarrow \\pi_3=9\\pi_2=72\\pi_1$.\nNormalize: $\\pi_1(1+8+72)=1 \\Rightarrow 81\\pi_1=1 \\Rightarrow \\pi_1=\\tfrac{1}{81}\\approx 0.0123$.\n$\\pi_2=\\tfrac{8}{81}\\approx 0.0988$, $\\pi_3=\\tfrac{72}{81}=\\tfrac{8}{9}\\approx 0.8889$.\nLong-run, about $88.9\\%$ of policyholders sit at the $40\\%$ discount.",
      "tag": "Stationary distribution"
    },
    {
      "front": "Use the stationary distribution of the NCD chain ($\\pi=(\\tfrac{1}{81},\\tfrac{8}{81},\\tfrac{8}{9})$) to find the long-run **average premium discount**, given discounts $0\\%,20\\%,40\\%$.",
      "back": "Take the $\\pi$-weighted average of the discount levels:\n$E[\\text{discount}]=\\tfrac{1}{81}(0\\%)+\\tfrac{8}{81}(20\\%)+\\tfrac{8}{9}(40\\%)$.\n$=0 + \\tfrac{8}{81}(20)+\\tfrac{72}{81}(40)=\\tfrac{160}{81}+\\tfrac{2880}{81}=\\tfrac{3040}{81}\\approx 37.5\\%$.\nSo in steady state the average policyholder enjoys about a $37.5\\%$ no-claims discount.",
      "tag": "Stationary distribution"
    },
    {
      "front": "Describe the **canonical (block) form** of an absorbing chain and name each block.",
      "back": "Reorder states so absorbing states come first and transient states last:\n$P = \\begin{pmatrix} I & 0 \\\\ R & Q \\end{pmatrix}$.\n- $I$ — identity block: absorbing states stay put.\n- $0$ — absorbing states never move to transient states.\n- $Q$ — transient-to-transient transition probabilities.\n- $R$ — transient-to-absorbing transition probabilities.\nThe interesting dynamics live in $Q$ and $R$.",
      "tag": "Absorbing chains"
    },
    {
      "front": "Define the **fundamental matrix** $N$ of an absorbing chain and the meaning of its entries.",
      "back": "$N=(I-Q)^{-1}=I+Q+Q^2+\\cdots$.\nThe entry $N_{ij}$ is the **expected number of visits** to transient state $j$, starting from transient state $i$, before absorption (counting the initial occupation when $i=j$).\nThe series converges because $Q^n\\to 0$ — from any transient state absorption is eventually certain.",
      "tag": "Absorbing chains"
    },
    {
      "front": "Give the formulas for **expected time to absorption** and **absorption probabilities** in terms of $N$ and $R$.",
      "back": "**Expected time to absorption** from transient state $i$: the $i$-th row sum of $N$,\n$t_i=\\sum_j N_{ij}$, i.e. $t=N\\mathbf{1}$ where $\\mathbf{1}$ is a column of ones.\n**Absorption probabilities:** $B=NR$, where $B_{ik}$ is the probability of being absorbed in absorbing state $k$ starting from transient state $i$. Each row of $B$ sums to $1$.",
      "tag": "Absorbing chains"
    },
    {
      "front": "A chain on $\\{1,2,3\\}$ has state $3$ absorbing, with transient block $Q=\\begin{pmatrix} 0.4 & 0.3 \\\\ 0.2 & 0.5 \\end{pmatrix}$ (states $1,2$). Compute the fundamental matrix $N=(I-Q)^{-1}$.",
      "back": "$I-Q=\\begin{pmatrix} 0.6 & -0.3 \\\\ -0.2 & 0.5 \\end{pmatrix}$.\nDeterminant $=0.6(0.5)-(-0.3)(-0.2)=0.30-0.06=0.24$.\nInverse $=\\frac{1}{0.24}\\begin{pmatrix} 0.5 & 0.3 \\\\ 0.2 & 0.6 \\end{pmatrix}$.\n$N=\\begin{pmatrix} 0.5/0.24 & 0.3/0.24 \\\\ 0.2/0.24 & 0.6/0.24 \\end{pmatrix}=\\begin{pmatrix} 2.0833 & 1.25 \\\\ 0.8333 & 2.5 \\end{pmatrix}$.",
      "tag": "Absorbing chains"
    },
    {
      "front": "Using $N=\\begin{pmatrix} 2.0833 & 1.25 \\\\ 0.8333 & 2.5 \\end{pmatrix}$ from the previous chain, find the **expected time to absorption** from each transient state.",
      "back": "Expected time to absorption is the row sum of $N$:\nFrom state $1$: $t_1=2.0833+1.25=3.3333$ steps.\nFrom state $2$: $t_2=0.8333+2.5=3.3333$ steps.\nSo regardless of whether you start in transient state $1$ or $2$, the expected number of steps until absorption in state $3$ is about $3.33$.",
      "tag": "Expected times"
    },
    {
      "front": "In the same chain, the transient states $1,2$ reach absorbing state $3$ with $R=\\begin{pmatrix} 0.3 \\\\ 0.3 \\end{pmatrix}$. Confirm the absorption probabilities $B=NR$.",
      "back": "With one absorbing state, every transient start is absorbed there with probability $1$, so $B$ should be a column of ones.\n$B=NR=\\begin{pmatrix} 2.0833 & 1.25 \\\\ 0.8333 & 2.5 \\end{pmatrix}\\begin{pmatrix} 0.3 \\\\ 0.3 \\end{pmatrix}$.\nRow 1: $2.0833(0.3)+1.25(0.3)=0.625+0.375=1.0$.\nRow 2: $0.8333(0.3)+2.5(0.3)=0.25+0.75=1.0$.\n$B=\\begin{pmatrix} 1 \\\\ 1 \\end{pmatrix}$ — absorption is certain, as expected.",
      "tag": "Absorbing chains"
    },
    {
      "front": "A chain has **two** absorbing states $\\{0,3\\}$ and transient states $\\{1,2\\}$, with $Q=\\begin{pmatrix} 0 & 0.5 \\\\ 0.5 & 0 \\end{pmatrix}$ and $R=\\begin{pmatrix} 0.5 & 0 \\\\ 0 & 0.5 \\end{pmatrix}$ (columns = absorb at $0$, at $3$). Find $N$.",
      "back": "$I-Q=\\begin{pmatrix} 1 & -0.5 \\\\ -0.5 & 1 \\end{pmatrix}$.\nDeterminant $=1(1)-(-0.5)(-0.5)=1-0.25=0.75$.\n$N=(I-Q)^{-1}=\\frac{1}{0.75}\\begin{pmatrix} 1 & 0.5 \\\\ 0.5 & 1 \\end{pmatrix}=\\begin{pmatrix} 1.3333 & 0.6667 \\\\ 0.6667 & 1.3333 \\end{pmatrix}$.\nThis is the expected-visits matrix for the symmetric random walk on $\\{1,2\\}$ with absorbing barriers at $0$ and $3$.",
      "tag": "Absorbing chains"
    },
    {
      "front": "Using $N=\\begin{pmatrix} 1.3333 & 0.6667 \\\\ 0.6667 & 1.3333 \\end{pmatrix}$ and $R=\\begin{pmatrix} 0.5 & 0 \\\\ 0 & 0.5 \\end{pmatrix}$, find the **absorption probabilities** $B=NR$.",
      "back": "$B=NR=\\begin{pmatrix} 1.3333 & 0.6667 \\\\ 0.6667 & 1.3333 \\end{pmatrix}\\begin{pmatrix} 0.5 & 0 \\\\ 0 & 0.5 \\end{pmatrix}=\\begin{pmatrix} 0.6667 & 0.3333 \\\\ 0.3333 & 0.6667 \\end{pmatrix}$.\nStarting in state $1$: absorbed at $0$ with prob $\\tfrac{2}{3}$, at $3$ with prob $\\tfrac{1}{3}$.\nStarting in state $2$: at $0$ with prob $\\tfrac{1}{3}$, at $3$ with prob $\\tfrac{2}{3}$.\nEach row sums to $1$ — you reach the nearer barrier more often, the classic gambler's-ruin pattern.",
      "tag": "Absorbing chains"
    },
    {
      "front": "State the **gambler's ruin** absorption-probability formula for a fair game and apply it to a gambler with $\\$2$ targeting $\\$5$.",
      "back": "For a symmetric random walk on $\\{0,1,\\dots,N\\}$ with absorbing barriers at $0$ and $N$ ($p=q=\\tfrac12$), the probability of reaching $N$ (winning) starting from $i$ is\n$\\Pr(\\text{reach }N \\mid i)=\\frac{i}{N}$.\nGambler with $\\$2$, target $N=5$: $\\Pr(\\text{win})=\\frac{2}{5}=0.4$, and $\\Pr(\\text{ruin})=1-0.4=0.6$.\nThe linear formula matches the $NR$ computation for the absorbing chain.",
      "tag": "Absorbing chains"
    },
    {
      "front": "State the **expected duration** of a fair gambler's-ruin game and compute it for a gambler with $\\$2$ and target $\\$5$.",
      "back": "For the symmetric walk on $\\{0,\\dots,N\\}$ ($p=q=\\tfrac12$), the expected number of steps to absorption starting from $i$ is\n$E[\\text{duration} \\mid i]=i(N-i)$.\nWith $i=2$, $N=5$: $E=2(5-2)=2\\cdot 3 = 6$ steps.\nThis equals the row sum of the fundamental matrix $N$ for the corresponding $4\\times 4$ transient block.",
      "tag": "Expected times"
    },
    {
      "front": "Set up the absorbing-chain blocks for a fair gambler's ruin on $\\{0,1,2,3,4\\}$ (barriers $0,4$; transient $1,2,3$). Write $Q$ and $R$.",
      "back": "From each interior state the walk moves $\\pm 1$ with probability $\\tfrac12$. Order transient states $1,2,3$; absorbing $0$ and $4$.\n$Q=\\begin{pmatrix} 0 & 0.5 & 0 \\\\ 0.5 & 0 & 0.5 \\\\ 0 & 0.5 & 0 \\end{pmatrix}$ (transient $\\to$ transient).\n$R=\\begin{pmatrix} 0.5 & 0 \\\\ 0 & 0 \\\\ 0 & 0.5 \\end{pmatrix}$ (columns = absorb at $0$, at $4$); only states $1$ and $3$ can step directly to a barrier.",
      "tag": "Absorbing chains"
    },
    {
      "front": "For the fair gambler's ruin on $\\{0,\\dots,4\\}$ with $Q=\\begin{pmatrix} 0 & 0.5 & 0 \\\\ 0.5 & 0 & 0.5 \\\\ 0 & 0.5 & 0 \\end{pmatrix}$, find the fundamental matrix $N=(I-Q)^{-1}$.",
      "back": "$I-Q=\\begin{pmatrix} 1 & -0.5 & 0 \\\\ -0.5 & 1 & -0.5 \\\\ 0 & -0.5 & 1 \\end{pmatrix}$, $\\det = 1(1-0.25)-(-0.5)(-0.5)= 0.75-0.25=0.5$.\nInverting (tridiagonal) gives\n$N=\\begin{pmatrix} 1.5 & 1.0 & 0.5 \\\\ 1.0 & 2.0 & 1.0 \\\\ 0.5 & 1.0 & 1.5 \\end{pmatrix}$.\nCheck: row sums $=3,4,3$, matching expected durations $i(N-i)=1\\cdot 3,\\,2\\cdot 2,\\,3\\cdot 1 = 3,4,3$ from states $1,2,3$.",
      "tag": "Expected times"
    },
    {
      "front": "Using $N=\\begin{pmatrix} 1.5 & 1.0 & 0.5 \\\\ 1.0 & 2.0 & 1.0 \\\\ 0.5 & 1.0 & 1.5 \\end{pmatrix}$ and $R=\\begin{pmatrix} 0.5 & 0 \\\\ 0 & 0 \\\\ 0 & 0.5 \\end{pmatrix}$, find the probability of ruin (absorption at $0$) from each interior state.",
      "back": "$B=NR$; the first column gives absorption at $0$.\nFrom state $1$: $1.5(0.5)+1.0(0)+0.5(0)=0.75$.\nFrom state $2$: $1.0(0.5)+2.0(0)+1.0(0)=0.50$.\nFrom state $3$: $0.5(0.5)+1.0(0)+1.5(0)=0.25$.\nRuin probabilities $\\left(\\tfrac34,\\tfrac12,\\tfrac14\\right)$ match $\\frac{N-i}{N}=\\frac{4-i}{4}$ for $i=1,2,3$.",
      "tag": "Absorbing chains"
    },
    {
      "front": "A bonus-malus claim chain has transient states $\\{A,B\\}$ and absorbing 'lapse' state $L$, with $Q=\\begin{pmatrix} 0.5 & 0.3 \\\\ 0.2 & 0.6 \\end{pmatrix}$. Find the expected number of years a policy starting in $A$ stays active before lapsing.",
      "back": "$I-Q=\\begin{pmatrix} 0.5 & -0.3 \\\\ -0.2 & 0.4 \\end{pmatrix}$, $\\det=0.5(0.4)-(-0.3)(-0.2)=0.20-0.06=0.14$.\n$N=\\frac{1}{0.14}\\begin{pmatrix} 0.4 & 0.3 \\\\ 0.2 & 0.5 \\end{pmatrix}=\\begin{pmatrix} 2.857 & 2.143 \\\\ 1.429 & 3.571 \\end{pmatrix}$.\nExpected active years from $A$ = row-1 sum $=2.857+2.143=5.0$ years.\n(From $B$ it is $1.429+3.571=5.0$ years as well.)",
      "tag": "Expected times"
    },
    {
      "front": "In the bonus-malus chain with $N=\\begin{pmatrix} 2.857 & 2.143 \\\\ 1.429 & 3.571 \\end{pmatrix}$, how many years is a policy starting in $A$ expected to spend **in state $B$** before lapsing?",
      "back": "The expected number of visits to transient state $B$ starting from $A$ is the entry $N_{AB}$.\n$N_{AB}=2.143$ years.\nThe diagonal entry $N_{AA}=2.857$ is the expected time spent in $A$ itself, so of the $5.0$ total active years, about $2.86$ are spent in $A$ and $2.14$ in $B$.",
      "tag": "Expected times"
    },
    {
      "front": "Why does the geometric series $N=I+Q+Q^2+\\cdots$ converge, and what does each term contribute to expected visits?",
      "back": "For an absorbing chain every transient state reaches absorption with positive probability, so the spectral radius of $Q$ is $<1$ and $Q^n\\to 0$; the Neumann series therefore converges to $(I-Q)^{-1}$.\nInterpretation: $(Q^n)_{ij}$ is the probability of being in transient state $j$ exactly $n$ steps after starting in $i$ (still un-absorbed). Summing over $n=0,1,2,\\dots$ gives the **expected number of visits** to $j$, which is precisely $N_{ij}$.",
      "tag": "Absorbing chains"
    },
    {
      "front": "Contrast a **stationary distribution** (recurrent/ergodic chain) with the **fundamental matrix** analysis (absorbing chain) — when do you use each?",
      "back": "**Stationary distribution $\\pi=\\pi P$:** used for an irreducible chain that keeps moving forever; it gives long-run time fractions, limiting probabilities (if aperiodic), and mean recurrence times $1/\\pi_j$.\n**Fundamental matrix $N=(I-Q)^{-1}$:** used when the chain has absorbing states; it answers \"how long until absorption\" (row sums of $N$), \"expected visits before absorption\" ($N_{ij}$), and \"which absorbing state\" ($B=NR$).\nA chain with absorbing states has no positive stationary distribution over the transient states — eventually all probability mass collapses onto the absorbing states.",
      "tag": "Stationary distribution"
    }
  ]
}