Date: August 29, 2025

Topic: Foundations in Statistics

Recall

Random variables hold probabilities for what their actual value is. When querying the random variable with a particular value, we get the probability of that particular value appearing.

Notes

Random Variables

• Place-holder for something we care about
• Random variables can hold particular values (e.g., int can hold numbers, etc)
  • beard: <True, False>
• However rand. vars change their value all the time
  • We don’t know the value of the rand. var. but we know the probability of seeing each possible value
• They come with probability distributions over values
  • beard: <0.6, 0.4> → 60% of the time have beard (True), 40% do not have (False)

Querying Random Variables

• What is the probability that I am in a world in which $X$ has a particular value?
  • $P(\text{BEARD}=\text{TRUE})=0.6$
  • $P(\text{BEARD}=\text{FALSE})=0.4$
• If a value is not specified (e.g., True or False), then we get a distribution over all values
  • $P(\text{BEARD}) = <0.6, 0.4>$

We can have complex queries like in the case of OR, where we combine multiple variables together to get their probability.

Complex Queries

Can use logical operators like OR and AND to modify queries

• Always True: $P(\text{BEARD}=\text{TRUE} \lor \text{BEARD}=\text{FALSE}) = P(\text{TRUE})=1$
  • Probability of True is always 1
• Always False: $P(\text{BEARD}=\text{TRUE} \land \text{BEARD}=\text{FALSE}) = P(\text{FALSE})=0$
  • Probability of False is always 0

Combining Multiple Variables

Using OR

• $P(\text{BEARD} = \text{TRUE} \lor \text{CAR} = \text{pickup}) = \newline P(\text{BEARD}=\text{TRUE})+P(\text{CAR}=\text{pickup}) - P(\text{BEARD}=\text{TRUE} \land \text{CAR}=\text{pickup})$
  • When we have an OR, we double count the intersection, so we need to take 1 of the counts away
• Generally, $P(A=x \lor B=y)= \newline P(A=x)+P(B=y)-P(A=x \land B=y)$

From the Full Joint Distribution table we can derive many probabilities as it shows the probabilities from all worlds.

This allows us to compute the probability of any combination of values for a set of random variables.

Full Joint Distribution

• When more than one rand. var., the full joint distribution describes the possible worlds

• $P(\text{NINJAS},\text{SHADOWS},\text{NOISES})$ is a full joint distribution

  

  • E.g., ninjas may create noises and cast shadows, but in the world, not all noises and shadows are from ninjas
  • These discrete variables can be represented as a giant lookup table (3D table above)
  • Every possible world ( $2^3 = 8$ possible worlds) is some combination of the 3 variables
    • Probability of all the worlds sum to 1

Using the Table

• With the full joint distribution, we can start asking more questions
• E.g., $P(\text{SHADOWS}=\text{TRUE} \land \text{NINJAS}=\text{TRUE} \land \text{NOISES}=\text{FALSE}) = 0.012$
• Furthermore, we may just be interested in 1 variable (are there ninjas in the room?)
• Or if we are look at an OR case

<aside> 📌 SUMMARY: Random vars have probability distributions over their possible values; with multiple variables, the full joint distribution lists probabilities for all “worlds” where we get marginals and OR/AND queries from by summing the appropriate entries. Bayes Rule allows us to invert the conditional probabilities, which is useful as usually we want $P(\text{CAUSE}\mid\text{EFFECT})$ but we have data of $P(\text{EFFECT} \mid \text{CAUSE})$. Using Naive Bayes, we can assume that the effect variables are always independent, allowing for simple calculation.

</aside>