Alexander
Professor Abuabara
DAEN 427
College Station, 13 Dec 2025

Practice questions

Rubric: Full credit may be given for any reasonable Bayesian analysis.

1) Probability of being sunny

From the following expressions, which one corresponds to the sentence: The probability of being sunny given that it is June 14th of 1976?

(a)

$p(\text{sunny})$

(b)

$p(\text{sunny}\mid \text{June})$

(c)

$p(\text{sunny}\mid \text{June 14th of 1976})$

(d)

$p(\text{June 14th of 1976}\mid \text{sunny})$

(e)

$p(\text{sunny},\text{June 14th of 1976})∕p(\text{June 14th of 1976})$

2) Picking a Pope

Show that the probability of choosing a human at random and picking the Pope is not the same as the probability of the Pope being human. In the animated series Futurama, the (Space) Pope is a reptile. How does this change your previous calculations?

3) Broken telephone

Consider the following simplified version of the broken telephone game. A first player secretly communicates a message $m_1\in \{0,1,2\}$ to a second player with probabilities

The second player then secretly communicates a message $m_2=m_1+n$ to a third (final) player, where

(a)

What is the optimal Bayes classification of the message received by the last player?

(b)

What is the expected error probability?

4) Binary classification

Consider the following binary classification problem with losses $L(C,C_{\mathrm{pred}})$:

Your prior is

(a)

What is the optimal classifier (and expected loss) based only on the prior distribution?

(b)

Suppose you can run an experiment with binary outcomes $x\in \{0,1\}$ with likelihoods

$$P(x=1\mid C=+1)=\frac{3}{4},P(x=1\mid C=-1)=\frac{1}{20}.$$

Find the classifier that minimizes expected loss. Compute the optimal expected loss and compare with part (a).

Answers

1) Probability of being sunny

The correct answers are (c) and (e).

The last one follows from the definition of conditional probability.

2) Picking a Pope

Let’s assume there are $7.8\cdot 10^8$ humans and there is only one Pope (Pope Leo XIV at the time of this writing). If a human is picked at random from the entire human population, the chances of that human being the pope are $1$ in $7.8\cdot 10^8$.

Because Pope is selected among humans. We can assert that given someone is the Pope, there is 100% chance he is a human. So, $p(\text{human}\mid \text{Pope})=1$.

Regarding the animated series Futurama and the space Pope is a reptile. We then have that $p(\text{Pope}\mid \text{human})=0$ and $p(\text{human}\mid \text{Pope})=0$.

3) Broken telephone

(a)

Note that $m_2\in \{0,1,2,3\}$ and

$$P(m_2=m_1\mid m_1)=\frac{2}{3}P(m_2=m_1+1\mid m_1)=\frac{1}{3}$$

Hence the posterior distribution is:

$$P(m_1=0\mid m_2)=\{\begin{array}{c}\frac{\frac{2}{3}\cdot \frac{1}{3}}{\frac{2}{3}\cdot \frac{1}{3}+0\cdot \frac{1}{3}+0\cdot \frac{1}{3}}=1,m_2=0\hfill \\ \frac{\frac{1}{3}\cdot \frac{1}{3}}{\frac{1}{3}\cdot \frac{1}{3}+\frac{2}{3}\cdot \frac{1}{3}+0\cdot \frac{1}{3}}=\frac{1}{3},m_2=1\hfill \\ \frac{0\cdot \frac{1}{3}}{0\cdot \frac{1}{3}+\frac{1}{3}\cdot \frac{1}{3}+\frac{2}{3}\cdot \frac{1}{3}}=0,m_2=2\hfill \\ \frac{0\cdot \frac{1}{3}}{0\cdot \frac{1}{3}+0\cdot \frac{1}{3}+\frac{1}{3}\cdot \frac{1}{3}}=0,m_2=3.\hfill \end{array}$$

$$P(m_1=1\mid m_2)=\{\begin{array}{c}\frac{0\cdot \frac{1}{3}}{0\cdot \frac{1}{3}+\frac{1}{3}\cdot \frac{1}{3}+0\cdot \frac{1}{3}}=0,m_2=0\hfill \\ \frac{\frac{2}{3}\cdot \frac{1}{3}}{\frac{2}{3}\cdot \frac{1}{3}+\frac{1}{3}\cdot \frac{1}{3}+0\cdot \frac{1}{3}}=\frac{2}{3},m_2=1\hfill \\ \frac{\frac{1}{3}\cdot \frac{1}{3}}{\frac{1}{3}\cdot \frac{1}{3}+\frac{2}{3}\cdot \frac{1}{3}+0\cdot \frac{1}{3}}=\frac{1}{3},m_2=2\hfill \\ \frac{0\cdot \frac{1}{3}}{0\cdot \frac{1}{2}+0\cdot \frac{1}{3}+\frac{1}{3}\cdot \frac{1}{3}}=0,m_2=3.\hfill \end{array}$$

and $P(m_1=2\mid m_2)=1-P(m_1=0\mid m_2)-P(m_1=1\mid m_2)$.

Hence,

$$m_{1_{\mathrm{𝑝𝑟𝑒𝑑}}}(m_2)=\{\begin{array}{c}0,m_2=0\hfill \\ 1,m_2=1\hfill \\ 2,m_2=2\hfill \\ 2,m_2=3.\hfill \end{array}$$

(b)

$P(m_1=0\mid m_2=1)P(m_2=1)+P(m_1=1\mid m_2=2)P(m_2=2)=\frac{1}{3}\cdot \frac{1}{3}+\frac{1}{3}\cdot \frac{1}{3}=\frac{2}{9}$

4) Binary classification

(a)

The a priori Bayes classifier is $C_{{\mathrm{pred}}_{\mathit{𝑀𝐴𝑃}}}=-1$ because $P(C=+1)<P(C=-1)$. However, this is not the classifier the minimizes expected loss since: $$\begin{array}{llll}\hfill 𝔼[L(C,C_{\mathrm{pred}}& =+1)]=10P(C=-1)=9& \hfill & \\ \hfill & <𝔼[L(C,C_{\mathrm{pred}}=-1)]=100P(C=+1)=10& \hfill & \end{array}$$

The optimal expected loss is $𝔼[L(C,C_{\mathrm{pred}}=+1)]=10P(C=-1)=9$.

(b)

The posterior distribution is:

$$P(C=+1\mid x)=\{\begin{array}{ccc}\frac{\frac{3}{4}\cdot \frac{1}{10}}{\frac{3}{4}\cdot \frac{1}{10}+\frac{1}{20}\cdot \frac{9}{10}}=0.625\hfill & \hfill & x=1\hfill \\ \hfill & \hfill & \hfill \\ \frac{\frac{1}{4}\cdot \frac{1}{10}}{\frac{1}{4}\cdot \frac{1}{10}+\frac{19}{20}\cdot \frac{9}{10}}=0.0284\hfill & \hfill & x=0\hfill \end{array}$$

Hence,

$$\begin{array}{llll}\hfill 𝔼[L(C,& C_{\mathrm{pred}}=+1)\mid x=1]=10P(C=-1\mid x=1)=3.75& \hfill & \\ \hfill & <𝔼[L(C,C_{\mathrm{pred}}=-1)\mid x=1]=100P(C=+1\mid x=1)=62.5& \hfill & \end{array}$$

and

$$\begin{array}{llll}\hfill 𝔼[L(C,& C_{\mathrm{pred}}=+1)\mid x=0]=10P(C=-1\mid x=0)=9.71& \hfill & \\ \hfill & >𝔼[L(C,C_{\mathrm{pred}}=-1)\mid x=0]=100P(C=+1\mid x=0)=2.84& \hfill & \end{array}$$

Hence, the optimal classifier is:

$$C_{\mathrm{pred}}(x)=\{\begin{array}{ccc}\hfill +1& \hfill & \hfill x=1\\ \hfill -1& \hfill & \hfill x=0\end{array}$$

The expected loss under optimal classifier is:

$$\begin{array}{llll}\hfill 𝔼[L(C,C_{\mathrm{pred}}(x))]& =10P(C=-1\mid x=1)P(x=1)+& \hfill & \\ \hfill & 100P(C=+1\mid x=0)P(x=0)& \hfill & \\ \hfill & =10(0.375)P(x=1)+100(0.0284)P(x=0)& \hfill & \\ \hfill & =10(0.375)(0.12)+100(0.0284)(0.88)=2.93<9& \hfill & \end{array}$$

Return DAEN 427 home