# Marginalization

## Alexei Gilchrist

### 1 Mutually exclusive and exhaustive

The extended sum-rule is

### 2 Variables

Up till now we have been considering single propositions that can be either true or false. In fact, we can consider the proposition \(A\) to be a shorthand for \((A\!=\!\text{True})\) and \(\bar{A}\) a shorthand for \((A\!=\!\text{False})\). It’s often convenient to talk about a set of related propositions, so instead of having an endless list \((A_1\!=\!\text{True})\), \((A_2\!=\!\text{True})\), \((A_3\!=\!\text{True})\) etc. We can group them all together in a single proposition \(A=A_1+A_2+A_3 \ldots\) (where `\(+\)' signifies a logical *or*). Another way of looking at this set is to think of \(A\) as a *variable* that can take on values from the set \(\{A_1,A_2,A_3,\ldots\}\) and we are asking a sequence of questions “is \(A\) equal to \(A_1\)” etc. Some examples,

- \(\text{Coin}\in\{H,T\}\),
- \(\text{Die}\in\{1,2,3,4,5,6\}\),
- \(\text{Weather}\in\{\text{Raining},\text{Sunny},\text{Other}\}\),
- \(\text{Height}\in\{(h<1.6),(1.6\le h \le 1.8),(h>1.8)\}\).

The set of possible values of the variable is the *domain* of the variable, and the values are the *states*, *options*, *instances*, or *possibilities*. I’ll try and stick with *possibilities* even though this is a non-standard term, as it is not overloaded with prior meaning. The possibilities can also form a countably infinite or continuous set, though taking this limit needs to be done with care. It is particularly useful if the possibilities are mutually exclusive (only one can be true at one time) and exhaustive (one of them must be true), but the way we have set up variables this need not be the case.

Where the context is clear, we’ll abbreviate the notation by just giving the possibility, e.g. \(P(A_1)\equiv P(A\!=\!A_1)\), or use the variable name to stand for a whole set of relations, one for each combination of possibilities. So for example \(P(A|B) = P(A)P(B|A)/P(B)\) stands for

Say \(A=\{A_1,A_2\}\), \(B=\{B_1,B_2\}\), and \(C=\{C_1,C_2,C_3\}\), then we might have:

### 3 Marginalization

There are two ways Eq. (\ref{eq:sum1}) is typically used. First we can use it to remove *nuisance* variables—say we have a probability that depends on a number of variables e.g. \(P(ABC)\), but we are interested in the dependence on only one of the variables (so the other dependancies are a nuisance). We can simply sum over the other variables:

This trick to eliminate variables from a joint probability also works with conditional probabilities, for example

The other main way in which Eq. (\ref{eq:sum1}) is commonly used is to expand a probability out as a sum over conditional probabilities. For instance

Say \(A\in\{A_T,A_F\}\) represents the result of a test for a disease, and \(B\in\{B_T,B_F\}\) is whether you have the disease or not. Then

Note that since the variables have only true/false possibilities we could have written this example treating them as propositions instead of variables (\(A=\) test indicates the disease, \(B=\) have the disease):