Probability Axioms

In the stochastic truth table below, you can change the probabilities assigned to each row. Recall that the sum of the numbers assigned to each row must be 1 and each number must be greater than or equal to 0. The probabilities of the formulas are updated when the stochastic truth table changes. When you hover over a formula with your mouse, the rows of the truth table where that formula is true is highlighted. As you change the probabilities assigned to each row, answer the following questions:

• What is the minimum and maximum value of a probability assigned to a formula?
• What is the relationship between $Pr(\neg P\vee (P\wedge Q))$ and the sum $Pr(\neg P) + Pr(P\wedge Q)$?
• What is the relationship between $Pr(P), Pr(Q)$ and $Pr(P \vee Q)$?
• What is the relationship between $Pr(P)$ and $Pr(\neg P)$ (and between $Pr(Q)$ and $Pr(\neg Q)$)?

	$P$	$Q$
	$\mathsf{T}$	$\mathsf{T}$
	$\mathsf{T}$	$\mathsf{F}$
	$\mathsf{F}$	$\mathsf{T}$
	$\mathsf{F}$	$\mathsf{F}$

$Pr(P)$ $\ =\$0.5000$Pr(Q)$ $\ =\$ 0.5000

$Pr(\neg P)$ $\ =\$ 0.5000$Pr(\neg Q)$ $\ =\$ 0.5000

$Pr(P \wedge Q)$ $\ =\$ 0.2500

$Pr(\neg P \vee (P\wedge Q))$ $\ =\$ 0.7500

$Pr(P \vee Q)$ $\ =\$ 0.7500

Returning to the first two questions asked above:

• What is the minimum and maximum value of a probability assigned to a formula? In any stochastic truth table, the probabilities assigned to a formula must be greater than or equal to 0 and less than or equal to 1. This means that in any stochastic truth table, for any formula $X$, if $X$ is a tautology (so, $X$ is true in every row), then $Pr(X)=1$.

• What is the relationship between $Pr(\neg P\vee (P\wedge Q))$ and the sum $Pr(\neg P) + Pr(P\wedge Q)$? Note that $\neg P$ and $P\wedge Q$ are mutually exclusive: There is no row of the truth table that makes both formulas true. That is, $\neg P$ is true in rows 3 and 4, while $P\wedge Q$ is true in row 1. Since the disjunction of these two formulas $\neg P\vee (P\wedge Q)$ is true in rows 1, 3, and 4, in any stochastic truth table, $Pr(\neg P\vee (P\wedge Q)) = Pr(\neg P) + Pr(P\wedge Q)$.

These two observations motivate the following three core principles of probability, which are known as the Kolmogorov Axioms (for a further discussion, see Section 1 of Interpretations of Probability, Stanford Encyclopedia of Philosophy by Alan Hajek).

Kolmogorov Axioms

Given any stochastic truth table, the following is true:

1. For all $X$, $Pr(X)\ge 0$.
2. For all $X$, if $X$ is a tautology, then $Pr(X) = 1$.
3. For all $X$ and $Y$, if $X$ and $Y$ are mutually exclusive, then $Pr(X\vee Y) = Pr(X)+ Pr(Y)$.

Since $X$ and $\neg X$ are contradictory and hence mutually exclusive, the Kolmogorov axiom 3 implies that

$$Pr(X\vee\neg X)=Pr(X) + Pr(\neg X).$$

Furthermore, since $X\vee\neg X$ is a tautology, Kolmogorov axiom 3 implies that $Pr(X\vee\neg X)=1$. Putting these two equations together, we have $1=Pr(X) + Pr(\neg X)$. This justifies the following:

Complement Law

Given any stochastic truth table, for all formulas $X$,

$$Pr(\neg X)=1-Pr(X).$$

Another important property is that equivalent formulas are always assigned the same probability. If $X$ and $Y$ a tautologically equivalent (that is, in every row of the truth table $X$ and $Y$ have the same truth vale), then the probabilities assigned to $X$ must be the same as the probability assigned to $Y$. For instance, since $X$ is equivalent to $(X\wedge Y)\vee (X\wedge\neg Y)$, they must have the same probability. These two observations are gathered below:

Observation

Given any stochastic truth table, for all formulas $X$ and $Y$,

1. if $X$ and $Y$ are tautologically equivalent (i.e., $X\leftrightarrow Y$ is a tautology), then $Pr(X)=Pr(Y)$; and
2. $Pr(X)=Pr((X\wedge Y)\vee (X\wedge\neg Y))$.

Using the observations above, we can derive the following further properties of probability.

Observation

Given any stochastic truth table, the following are true:

1. For all formulas $X$, if $X$ is a contradiction, then $Pr(X)=0$.
2. For all formulas $X$ and $Y$, $Pr(X)=Pr(X\wedge Y) + Pr(X\wedge\neg Y)$.
3. For all formulas $X$ and $Y$,
   if $X\rightarrow Y$ is a tautology, then $Pr(X)\le Pr(Y)$.
4. For all formulas $X$ and $Y$,
   $Pr(X\vee Y)= Pr(X) + Pr(Y) - Pr(X\wedge Y)$

Principles of Conditional Probability

As in the example above, in the stochastic truth table below, you can change the probabilities assigned to each row. The conditional probabilities of the formulas are updated when the stochastic truth table changes. As you change the probabilities assigned to each row, answer the following questions:

• What is the relationship between $Pr(P\wedge Q)$, $Pr(Q)$ and $Pr(P\mid Q)$?
• Can you find a stochastic truth table such that $Pr(P)\ne Pr(Q) Pr(P\mid Q) + Pr(\neg Q) Pr(P\mid \neg Q)$?

	$P$	$Q$
	$\mathsf{T}$	$\mathsf{T}$
	$\mathsf{T}$	$\mathsf{F}$
	$\mathsf{F}$	$\mathsf{T}$
	$\mathsf{F}$	$\mathsf{F}$

$Pr(P)$ $\ =\$ 0.5000$Pr(Q)$ $\ =\$ 0.5000

$Pr(\neg P)$ $\ =\$ 0.5000$Pr(\neg Q)$ $\ =\$ 0.5000

$Pr(P \mid Q)$$\ =\$ $Pr(P\wedge Q)$ $/$ $Pr(Q)$ $\ =\$ 0.2500 / 0.5000$\ =\$ 0.5000

$Pr(P \mid \neg Q)$ $\ =\$ $Pr(P\wedge \neg Q)$ $/$ $Pr(\neg Q)$ $\ =\$ 0.2500 / 0.5000$\ =\$ 0.5000

$Pr(Q)Pr(P\mid Q) + Pr(\neg Q)Pr(P \mid \neg Q)$ $\ =\$
0.5000 $\times$ 0.5000 $+$ 0.5000 $\times$ 0.5000 $\ =\$ 0.5000

The following observation follows from the definition of conditional probability:

Observation

Given any stochastic truth table, the following are true:

1. For all formulas $X$ and $Y$,
   $Pr(X\wedge Y)=Pr(Y)\times Pr(X\mid Y)$.
2. For all formulas $X$, $Y$ and $Z$,
   $Pr(X\wedge Y\mid Z)=Pr(Y\mid Z)\times Pr(X\mid Y\wedge Z)$.

The next observation follows from the following two facts:

1. $Pr(X) = Pr(X\wedge Y) + Pr(X\wedge\neg Y)$; and
2. $Pr(Y) Pr(X\mid Y) + Pr(\neg Y) Pr(X\mid \neg Y) =$
   $Pr(Y)\frac{Pr(X\wedge Y)}{Pr(Y)} + Pr(\neg Y)\frac{Pr(X\wedge\neg Y)}{Pr(\neg Y)} =$
   $Pr(X\wedge Y)+Pr(X\wedge\neg Y)$

Law of Total Probability

Given any stochastic truth table, for all formulas $X$ and $Y$,
$Pr(X) = Pr(Y) Pr(X\mid Y) + Pr(\neg Y) Pr(X\mid \neg Y)$.

Practice Questions

1. True or False: For any formulas $X$ and $Y$, for any stochastic truth table, $Pr(X\ |\ Y) = Pr(Y\ |\ X)$

   True False

   ​

   Let $X$ be the atomic proposition $A$ and $Y$ be the atomic proposition $B$. Then, note that in the following stochastic truth table, we have $Pr(A\mid B)\ne Pr(B\mid A)$:

   	$A$	$B$	$(A\wedge B)$	$(B\wedge A)$
   0.1	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{T}$
   0.1	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$
   0.4	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$
   0.4	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$

   $Pr(A\mid B)\ =\$$Pr(A\wedge B)$$/$$Pr(B)$ $\ =\$$0.10$$/$$0.50$$\ =\ 0.200$

   $Pr(B\mid A)\ =\$$Pr(B\wedge A)$$/$$Pr(A)$ $\ =\$$0.10$$/$$0.20$$\ =\ 0.500$

2. True or False: For any formulas $X$ and $Y$, in any stochastic truth table, if $Pr(X\leftrightarrow Y)=1$, then $Pr(X)=Pr(Y)$.

   True False

   ​

   Since $X\leftrightarrow Y \approx (X\wedge Y)\vee (\neg X\wedge \neg Y)$, $(\neg X\wedge\neg Y) \approx \neg (X\vee Y)$, and $X\wedge Y$ and $\neg X\wedge\neg Y$ are mutually exclusive, we have that:

   $$Pr(X\leftrightarrow Y) = Pr(X\wedge Y) + Pr(\neg X\wedge\neg Y) = Pr(X\wedge Y) + Pr(\neg (X\vee Y))$$

   Since $Pr(X\leftrightarrow Y) = 1$ and $Pr(\neg (X\vee Y)) = 1 - Pr(X\vee Y)$, we have that:

   $$1 = Pr(X\wedge Y) + 1 - Pr(X\vee Y) = Pr(X\wedge Y) + 1 - (Pr(X) + Pr(Y) - Pr(X\wedge Y))$$

   So, $Pr(X) + Pr(Y) = 2Pr(X\wedge Y)$.

   Recall that $Pr(X)=Pr(X\wedge Y) + Pr(X\wedge\neg Y)$ and $Pr(Y)=Pr(Y\wedge X) + Pr(Y\wedge \neg X)$. This means that:

   $$2Pr(X\wedge Y) = Pr(X) + Pr(Y) = Pr(X\wedge Y) + Pr(X\wedge\neg Y) + Pr(Y\wedge X) + Pr(Y\wedge \neg X)$$

   Hence, $0=Pr(X\wedge\neg Y) + Pr(Y\wedge\neg X)$, and so, $Pr(X\wedge\neg Y)= Pr(Y\wedge\neg X)=0$.

   Then, $Pr(X)= Pr(X\wedge Y)+Pr(X\wedge\neg Y)=Pr(X\wedge Y)$ and $Pr(Y)= Pr(X\wedge Y) + Pr(\neg X\wedge Y)=Pr(X\wedge Y)$. Thus, $Pr(X)=Pr(Y)$.

3. True or False: For any formulas $X$ and $Y$, in any stochastic truth table, if $X\rightarrow Y$ is a tautology, then $Pr(Y\mid X)=1$.

   True False

   ​

   If $X\rightarrow Y$ is a tautology, then $X\wedge Y\approx X$. This means that

   $$Pr(Y\mid X)=\frac{Pr(Y\wedge X)}{Pr(X)} = \frac{Pr(X)}{Pr(X)} = 1$$

4. Suppose that $Pr(P\wedge B)=0.75$, $Pr(P\wedge \neg Q)= 0.1$, find $Pr(P)$.

5. Suppose that $Pr(P)=0.35$, $Pr(Q\mid P)=0.7$, $Pr(Q\mid\neg P)= 0.1$, find $Pr(Q)$.

6. Suppose that $Pr(Q)=0.85$, $Pr(P\mid Q)=0.2$, $Pr(P\mid\neg Q)= 0.3$, find $Pr(P)$.