Confirmation Measures

When an argument is inducitvely strong, the premsises are said to confirm the conclusion. For example, suppose that you are interested in whether Ann (whom you do not know very well) is a student at the UMD. If $H$ is the proposition "Ann is a student at UMD", then your probability of $H$ is around $0.5$. Now supppose that you observe, Ann in the Skinner Building carrying a backpack. This evidence confirms $H$ in the sense that it boosts your probability of $H$, even though it does not guarantee that $H$ is true. On the other hand, observing Ann carrying a backpack in the National Mall would probably not raise your probability of $H$, and so does not confirm $H$.

As we have noted when discussing the definition of inductively strong arguments, it is important to distinguish between absolute confirmation and incremental confirmation:

1. $X$ evidentially supports $Y$ ($Pr(Y\mid X)>0.5$) represents aboslute confirmation where learning that $X$ is true raises the probability of the hypothesis $Y$ above some fixed threshold (typically the threshold is $0.5$).

2. $X$ is positively relevant to $Y$ ($Pr(Y\mid X)>Pr(Y)$) represents incremental confirmation. This is the "boost" that the conclusion would receive after learning the premises are true.

So far in this book, we have focused on evaluating a single argument. To evaluate an argument $X\Rightarrow Y$, we first ask:

1. Is the argument valid? That is, is $Y$ true in all situations that $X$ is true?

If the answer is no to the above question, then we ask the following:

2. Should you believe that the conclusion $Y$ is true under the supposition that the premise $X$ is true?
3. Is the premise $X$ relevant to the truth of the conclusion $Y$?

Of course, it is also important to determine wether the premise $X$ is true to fully evaluate the above argument.

In this section, we are going to introduce ways to compare two different arguments in terms of both abolute and incremental confirmation. For example, consider the following stochastic truth table:

	$P$	$Q$	$R$	$(Q\wedge P)$	$(Q\wedge R)$
0.2	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{T}$
0.2	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$
0.1	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$
0.1	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$
0.15	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{T}$
0.05	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$
0.1	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$
0.1	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$

$Pr(Q\mid P)\ =\$$Pr(Q\wedge P)$$/$$Pr(P)$ $\ =\$$0.40$$/$$0.60$$\ =\ 0.667$

$Pr(Q\mid R)\ =\$$Pr(Q\wedge R)$$/$$Pr(R)$ $\ =\$$0.35$$/$$0.55$$\ =\ 0.636$

In both of the following arguments, the premise evidentially supports the conclusion:

1. $P\Rightarrow Q$
2. $R\Rightarrow Q$

But, there is more we can say about these two arguments. Since $Pr(Q\mid P)>Pr(Q\mid R)$, $P$ provides stronger support for the conclusion $Q$ than $R$ does. So, in the absolute sense of confirmation, $P$ confirms $Q$ more than $R$ confirms $Q$

Note that $Pr(Q)=0.6$. This means that $P$ is positively relevant to $Q$ and $R$ is positively relevant to $Q$. So, both arguments are inductively strong. Is there some way to measure which premise is more positively relevant to $Q$? There are different ways to measure how positively relevant a premise is to a conclusion. One natural thought is to measure positive relevance in terms of difference between the conditional and unconditional probabilities.

Confirmation Measure: Difference

In any stochastic truth table, for any formulas $X$ and $Y$, let $d(X, Y)$ be defined as follows:

$$d(X, Y) = Pr(Y\mid X) - Pr(Y).$$

In the above stochastic truth table, we have:

• $d(P,Q)=Pr(Q\mid P)-Pr(Q)=0.667 -0.6 = 0.067$; and
• $d(R,Q)=Pr(Q\mid R)-Pr(Q)=0.636 -0.6 = 0.036$.

For any stochastic truth table, and any formulas $X$ and $Y$, the number $d(X, Y)$ is a measure of the boost in probability that $X$ receives from $Y$. There are many other ways to measure how much $X$ confirms $Y$. It is beyond the scope of this course to survey all the different confirmation measures. One problem with $d(X, Y)$ as a measure of incremental confirmation is that if $Y$ is already very probability (i.e., the probability is close to $1$), then $d(X, Y)$ will be very small regardless of how strong the evidence $X$ is for $Y$. The next way of measuring positive relevance does not suffer from this problem:

Confirmation Measure: Likelihood Ratios

In any stochastic truth table, for any formulas $X$ and $Y$, let $\ell(X, Y)$ be defined as follows:

$$\ell(X, Y) = \frac{(Pr(X\mid Y) - Pr(X\mid \neg Y))}{(Pr(X\mid Y) + Pr(X\mid \neg Y))}$$

Both $d(X, Y)$ and $\ell(X, Y)$ measures the incremental confirmation that $X$ gives $Y$. One potential issue that may lead to confusion with the definition of $\ell$ is that the equation of $\ell(X, Y)$ uses $Pr(X\mid Y)$---the probability of $X$ conditional on $Y$---but $d(X, Y)$ uses $Pr(Y\mid X)$. If $X$ is the evidence and $Y$ is the hypothesis, then $Pr(X\mid Y)$ is the likelihood of observing $X$ supposing that the hypothesis $Y$ is true and $Pr(X\mid\neg Y)$ is the likelihood of observing $X$ supposing that the hypothesis $Y$ is false. So, the numerator $Pr(X\mid Y)-Pr(X\mid\neg Y)$ is the differece between observing $X$ assuming $Y$ is true and observing $X$ assuming that $Y$ is false. To calculate $\ell(P,Q)$ and $\ell(R,Q)$ in the above stochastic truth table we need to calculate the following probabilities:

	$P$	$Q$	$R$	$(P\wedge Q)$	$(P\wedge \neg Q)$	$\neg Q$	$(R\wedge Q)$	$(R\wedge \neg Q)$	$\neg Q$
0.2	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$
0.2	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$
0.1	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{T}$
0.1	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{T}$
0.15	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$
0.05	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$
0.1	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{T}$
0.1	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{T}$

$Pr(P\mid Q)\ =\$$Pr(P\wedge Q)$$/$$Pr(Q)$ $\ =\$$0.40$$/$$0.60$$\ =\ 0.667$

$Pr(P\mid \neg Q)\ =\$$Pr(P\wedge \neg Q)$$/$$Pr(\neg Q)$ $\ =\$$0.20$$/$$0.40$$\ =\ 0.500$

$Pr(R\mid Q)\ =\$$Pr(R\wedge Q)$$/$$Pr(Q)$ $\ =\$$0.35$$/$$0.60$$\ =\ 0.583$

$Pr(R\mid \neg Q)\ =\$$Pr(R\wedge \neg Q)$$/$$Pr(\neg Q)$ $\ =\$$0.20$$/$$0.40$$\ =\ 0.500$

In the above stochastic truth table, we then have:

• $\ell(P,Q)=\frac{Pr(Q\mid P)-Pr(Q\mid \neg P)}{Pr(Q\mid P)+Pr(Q\mid \neg P)}=\frac{0.667 - 0.5}{0.667 + 0.5}\approx 0.143$; and

• $\ell(R,Q)=\frac{Pr(R\mid P)-Pr(R\mid \neg P)}{Pr(R\mid P)+Pr(R\mid \neg P)}=\frac{0.583 - 0.5}{0.583 + 0.5}\approx 0.076$.

Although $d(X, Y)$ and $\ell(X, Y)$ are often different numbers (as in the above example), they are related in the following way:

Observation

In any stochastic truth table, for all formulas $X$ and $Y$:

1. $X$ is positively relevant to $Y$ if and only if $d(X, Y) > 0$ if and only if $\ell(X, Y)>0$
2. $X$ and $Y$ are independent if and only if $d(X, Y) = 0$ if and only if $\ell(X, Y) = 0$
3. $X$ is negatively relevant to $Y$ if and only if $d(X, Y) < 0$ if and only if $\ell(X, Y) < 0$

There are many ways to compare and contrast the above two different ways to measure incremental confirmation. One reason to prefer $\ell$ is illustrated by computing $d(P, P\vee Q)$ and $\ell(P,P\vee Q)$ in the following stochastic truth table:

Example

	$P$	$Q$
0.2	$\mathsf{T}$	$\mathsf{T}$
0.1	$\mathsf{T}$	$\mathsf{F}$
0.3	$\mathsf{F}$	$\mathsf{T}$
0.4	$\mathsf{F}$	$\mathsf{F}$

The evidence is $P$
The hypothesis is $(P\vee Q)$

$Pr((P\vee Q)\mid P)\ = \$$1.000$

$Pr((P\vee Q))\ = \$$0.600$

$d(P, (P\vee Q))\ = \$$Pr((P\vee Q)\mid P) - Pr((P\vee Q))\ = \$$0.400$

$Pr(P\mid (P\vee Q))\ = \$$0.500$

$Pr(P\mid \neg (P\vee Q))\ = \$$0.000$

$\ell(P, (P\vee Q))\ = \$$(Pr(P\mid (P\vee Q)) - Pr(P\mid \neg (P\vee Q)))/(Pr(P\mid (P\vee Q)) + Pr(P\mid \neg (P\vee Q)))\ = \$$1.000$

Intuitively, since $P\models P\vee Q$ non-trivially (in the sense that the premise is not a contradiction and the conclusion is not a tautology), the premise $P$ should give the greatest confirmation to the conclusion $P\vee Q$. While $\ell(P, P\vee Q)$ is assigned the maximum value of 1, the measure $d(P, P\vee Q)$ is only 0.4. In fact we have the following key observation:

Observation

In any stochastic truth table, For any formulas $X$ and $Y$, in any stochastic truth table:

$$d(X, Y)=\begin{cases} Pr(\neg Y) & \text{ if } P\models Y, Pr(X)>0, 0< Pr(Y) < 1 \\-Pr(Y) & \text{ if } P\models \neg Y, Pr(X)>0, 0< Pr(Y) < 1\\\end{cases}$$

$$\ell(X, Y)=\begin{cases} 1 & \text{ if } P\models Y, Pr(X)>0, 0< Pr(Y) < 1 \\-1 & \text{ if } P\models \neg Y, Pr(X)>0, 0< Pr(Y) < 1\\\end{cases}$$

Practice Questions

1. Find $d(E,H)$ and $\ell(E,H)$ in the following stochastic truth table.

   	$E$	$H$
   0.25	$\mathsf{T}$	$\mathsf{T}$
   0.25	$\mathsf{T}$	$\mathsf{F}$
   0.25	$\mathsf{F}$	$\mathsf{T}$
   0.25	$\mathsf{F}$	$\mathsf{F}$

2. Find $d(E,H)$ and $\ell(E,H)$ in the following stochastic truth table.

   	$E$	$H$
   0	$\mathsf{T}$	$\mathsf{T}$
   0.5	$\mathsf{T}$	$\mathsf{F}$
   0.5	$\mathsf{F}$	$\mathsf{T}$
   0	$\mathsf{F}$	$\mathsf{F}$

3. Find $d(P,Q)$, $d(P,R)$, $\ell(P,Q)$ and $\ell(P,R)$ in the following stochastic truth table.

   	$P$	$Q$	$R$
   0.1	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{T}$
   0.2	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{F}$
   0.1	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{T}$
   0.2	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$
   0.2	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{T}$
   0.1	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$
   0.05	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{T}$
   0.05	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$

4. Find $d(P,Q)$, $d(P,R)$, $\ell(P,Q)$ and $\ell(P,R)$ in the following stochastic truth table.

   	$P$	$Q$	$R$
   0.2	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{T}$
   0	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{F}$
   0.1	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{T}$
   0.3	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$
   0.2	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{T}$
   0.1	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$
   0	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{T}$
   0.1	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$

5. Find $d(P,(Q\vee R))$ and $\ell(P,(Q\vee R))$ in the following stochastic truth table.

   	$P$	$Q$	$R$
   0.2	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{T}$
   0	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{F}$
   0.1	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{T}$
   0.3	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$
   0.2	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{T}$
   0.1	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$
   0	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{T}$
   0.1	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$

6. Find $d((Q\vee R), Q)$ and $\ell((Q\vee R), Q)$ in the following stochastic truth table.

   	$Q$	$R$
   0.3	$\mathsf{T}$	$\mathsf{T}$
   0.1	$\mathsf{T}$	$\mathsf{F}$
   0.3	$\mathsf{F}$	$\mathsf{T}$
   0.3	$\mathsf{F}$	$\mathsf{F}$

7. Find $d((Q\vee R), P)$ and $\ell((Q\vee R), P)$ in the following stochastic truth table.

   	$P$	$Q$	$R$
   0.2	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{T}$
   0	$\mathsf{T}$	$\mathsf{T}$	$\mathsf{F}$
   0.1	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{T}$
   0.3	$\mathsf{T}$	$\mathsf{F}$	$\mathsf{F}$
   0.2	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{T}$
   0.1	$\mathsf{F}$	$\mathsf{T}$	$\mathsf{F}$
   0	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{T}$
   0.1	$\mathsf{F}$	$\mathsf{F}$	$\mathsf{F}$

8. Find $d(P \rightarrow Q, Q)$ and $\ell(P \rightarrow Q, Q)$ in the following stochastic truth table.

   	$P$	$Q$
   0.1	$\mathsf{T}$	$\mathsf{T}$
   0.2	$\mathsf{T}$	$\mathsf{F}$
   0.4	$\mathsf{F}$	$\mathsf{T}$
   0.3	$\mathsf{F}$	$\mathsf{F}$

9. Find $d(Q, P \rightarrow Q)$ and $\ell( Q, P \rightarrow Q)$ in the following stochastic truth table.

   	$P$	$Q$
   0.1	$\mathsf{T}$	$\mathsf{T}$
   0.2	$\mathsf{T}$	$\mathsf{F}$
   0.4	$\mathsf{F}$	$\mathsf{T}$
   0.3	$\mathsf{F}$	$\mathsf{F}$

10. Find $d(P, P \rightarrow Q)$ and $\ell( P, P \rightarrow Q)$ in the following stochastic truth table.

    	$P$	$Q$
    0.1	$\mathsf{T}$	$\mathsf{T}$
    0.2	$\mathsf{T}$	$\mathsf{F}$
    0.4	$\mathsf{F}$	$\mathsf{T}$
    0.3	$\mathsf{F}$	$\mathsf{F}$