The Probability Theory

Let the probability of rain tomorrow be \( P(R) = 0.6 \).
This satisfies Axiom 1 because \( 0.6 \geq 0 \).

Suppose a fair die is rolled. The sample space is \[ S = \{1, 2, 3, 4, 5, 6\} \] Each outcome has probability \( \frac{1}{6} \). Then, \[ P(S) = \sum_{i=1}^{6} P(i) = 6 \times \frac{1}{6} = 1 \]

Let event \( A \) = getting an even number when a die is rolled, i.e., \( A = \{2, 4, 6\} \)
Let event \( B \) = getting an odd number, i.e., \( B = \{1, 3, 5\} \)

These are disjoint, so: \[ P(A \cup B) = P(A) + P(B) = \frac{3}{6} + \frac{3}{6} = 1 \]

The three axioms of probability form the foundation of probability theory:

1. Probabilities are always non-negative.

2. The total probability of all possible outcomes is 1.

3. For disjoint events, the probability of their union is the sum of their individual probabilities.

Understanding and applying these axioms is essential for solving any probability problem.

For any event \( A \), the probability is a number between 0 and 1: \[ 0 \leq P(A) \leq 1 \] Example 1: Tossing a coin. Let event \( A \) be getting heads. Then \( P(A) = 0.5 \in [0, 1] \)

Example 2: Rolling a die. Let event \( B \) be getting number 3. \( P(B) = \frac{1}{6} \approx 0.167 \in [0,1] \)

The sum of probabilities of all possible elementary events in a sample space is always 1. \[ \sum_{i=1}^{n} P(E_i) = 1 \]

Example 1: Tossing a coin. \[ P(H) + P(T) = 0.5 + 0.5 = 1 \]

Example 2: Rolling a die. \[ P(1)+P(2)+P(3)+P(4)+P(5)+P(6) = 6 \times \frac{1}{6} = 1 \]

The probability of the empty set (an impossible event) is 0. \[ P(\emptyset) = 0 \]

Example 1: Rolling a 7 on a standard die: \( P(7) = 0 \)

Example 2: Drawing a purple card from a standard deck: \( P(\text{purple}) = 0 \)

The probability of the sample space \( S \), representing a certain event, is 1. \[ P(S) = 1 \]

Example 1: Drawing a card from a 52-card deck. Event: "drawing a card from the deck" — always true.

Example 2: Tossing a coin. Event: getting either head or tail. \[ P(H \cup T) = 1 \]

The probability of the complement of an event \( A \) is: \[ P(A^c) = 1 - P(A) \]

Example 1: Rolling a die. Let \( A \) be the event of getting an even number: \( P(A) = \frac{3}{6} = 0.5 \)

Then \( P(A^c) = 1 - 0.5 = 0.5 \)

Example 2: Tossing a coin. Let \( A \) be getting heads, \( P(A) = 0.5 \Rightarrow P(A^c) = 0.5 \)

If \( A \) and \( B \) are mutually exclusive events, then: \[ P(A \cup B) = P(A) + P(B) \]

Example 1: Tossing a coin: \( A = \text{Head}, B = \text{Tail} \). Then: \[ P(A \cup B) = 0.5 + 0.5 = 1 \]

Example 2: Rolling a die: \( A = \text{2}, B = \text{5} \Rightarrow P(A \cup B) = \frac{1}{6} + \frac{1}{6} = \frac{1}{3} \)

If \( A \) and \( B \) are any two events: \[ P(A \cup B) = P(A) + P(B) - P(A \cap B) \]

Example 1: Drawing from a deck, \( A = \text{red card}, B = \text{king} \)

\[ P(A) = \frac{26}{52},\quad P(B) = \frac{4}{52},\quad P(A \cap B) = \frac{2}{52} \] \[ P(A \cup B) = \frac{26 + 4 - 2}{52} = \frac{28}{52} \]

Example 2: In a class: - 40 play football (\( A \)), 30 play cricket (\( B \)), 10 play both. - Total = 100 students.

\[ P(A \cup B) = \frac{40 + 30 - 10}{100} = \frac{60}{100} = 0.6 \]

If \( A \) and \( B \) are independent: \[ P(A \cap B) = P(A) \cdot P(B) \]

Example 1: Tossing a coin and rolling a die. - \( A = \text{Heads}, P(A) = 0.5 \) - \( B = \text{3 on die}, P(B) = \frac{1}{6} \)

\[ P(A \cap B) = 0.5 \cdot \frac{1}{6} = \frac{1}{12} \]

Example 2: Two separate bulbs with failure probabilities of 0.1 and 0.2: \[ P(\text{both fail}) = 0.1 \cdot 0.2 = 0.02 \]

Two events \( A \) and \( B \) are said to be mutually exclusive (or disjoint) if they cannot occur at the same time. That is, \[ P(A \cap B) = 0 \] This means that the occurrence of one event excludes the possibility of the other.

Key Identification Tip: If two events describe outcomes that cannot happen together in a single trial of the experiment, they are mutually exclusive.

Two events \( A \) and \( B \) are said to be independent if the occurrence of one does not affect the probability of the other occurring. That is, \[ P(A \cap B) = P(A) \cdot P(B) \]

Key Identification Tip: If you can say “the probability of one event is not influenced by the outcome of the other,” they are likely independent.

• Mutually Exclusive: Check if events can happen together. If not, they are mutually exclusive.

• Independent: Compute \( P(A \cap B) \) and compare it to \( P(A) \cdot P(B) \). If equal, the events are independent.

• Important: Mutually exclusive events are never independent (unless one event has probability 0).

Conditional probability is the probability of one event occurring with some relationship to one or more other events. If $A$ and $B$ are two events and $P(B) > 0$, then the conditional probability of $A$ given $B$ is defined as:

\[ P(A \mid B) = \frac{P(A \cap B)}{P(B)} \]

This formula is read as "the probability of A given B equals the probability of A and B occurring together divided by the probability of B."

• Multiplication Rule: $P(A \cap B) = P(A \mid B) \cdot P(B) = P(B \mid A) \cdot P(A)$

• If $A$ and $B$ are independent, then $P(A \mid B) = P(A)$ and $P(B \mid A) = P(B)$.

• Bayes' Theorem: $P(B \mid A) = \dfrac{P(A \mid B)P(B)}{P(A)}$

A box contains 5 red balls and 3 green balls. One ball is drawn at random. Given that it is not green, what is the probability that it is red?

Solution: Total outcomes: 8 (5 red + 3 green).
Let event A: ball is red, event B: ball is not green = red.
Then $P(A|B) = \frac{P(A \cap B)}{P(B)}$
Since red $\cap$ not green = red: $P(A \cap B) = \frac{5}{8},\ P(B) = \frac{5}{8}$
Thus, $P(A|B) = \frac{\frac{5}{8}}{\frac{5}{8}} = 1$

A card is drawn from a deck of 52 cards. Given that it is a face card, what is the probability it is a king?

Solution: There are 12 face cards (Jack, Queen, King from each suit), and 4 Kings.
$P(King | Face) = \frac{P(King \cap Face)}{P(Face)} = \frac{4/52}{12/52} = \frac{1}{3}$

Two dice are thrown. What is the probability that the sum is 9, given that the sum is odd?

Solution: Total odd sums: 3,5,7,9,11 → Count = 18.
Sum = 9 possible pairs: (3,6), (4,5), (5,4), (6,3) → 4 ways.
Only those with odd sum: All 4 valid.
So, $P(9|odd) = \frac{4}{18} = \frac{2}{9}$

A family has two children. Given that one is a girl, what is the probability that both are girls?

Solution: Possible combinations: BB, BG, GB, GG → remove BB → 3 cases.
Only one case is GG, so $P(GG|at\ least\ one\ G) = \frac{1}{3}$

A student passes a subject with 0.6 probability. What is the probability he passes both subjects if passing second subject given first is 0.5?

Solution: $P(A) = 0.6,\ P(B|A) = 0.5$
$P(A \cap B) = P(A) \cdot P(B|A) = 0.6 \times 0.5 = 0.3$

A test is positive in 95% of people with disease, and 2% of people without disease. 1% have disease. What is the probability that a person has the disease if the test is positive?

Solution: Let D = has disease, T = test positive
$P(D) = 0.01,\ P(T|D) = 0.95,\ P(T|\neg D) = 0.02,\ P(\neg D) = 0.99$
$P(T) = 0.01 \cdot 0.95 + 0.99 \cdot 0.02 = 0.0095 + 0.0198 = 0.0293$
$P(D|T) = \frac{0.0095}{0.0293} \approx 0.324$

A machine has 3 parts: A, B, and C. Failure probability of each is 0.1. Given that one failed, what is the probability it was part A?

Solution: Failures equally likely: $P(A|one\ fails) = \frac{1}{3}$

Two cards drawn without replacement. Find probability that second card is king given first is king.

Solution: $P(K_2 | K_1) = \frac{3}{51}$ (Only 3 kings left out of 51)

A number is chosen from 1 to 100. Given it is divisible by 5, what is probability it is also divisible by 10?

Solution: Divisible by 5: 20 numbers.
Divisible by 10: 10 numbers → All divisible by 5 too.
So $P(10|5) = \frac{10}{20} = \frac{1}{2}$

A box contains 2 white and 3 black balls. Two balls are drawn one after another without replacement. Find the probability that the second is black given that the first is white.

Solution: Total: 5 balls. If first is white (2/5), then left: 1 white, 3 black = 4 balls.
$P(B_2 | W_1) = \frac{3}{4}$

Bayes' theorem allows us to update the probability of an event based on new information. It relates the conditional and marginal probabilities of random events.

Formula: \[ P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)} \]

Where:

• $P(A|B)$ is the probability of event $A$ given that event $B$ has occurred.

• $P(B|A)$ is the probability of event $B$ given that event $A$ has occurred.

• $P(A)$ is the prior probability of event $A$.

• $P(B)$ is the marginal probability of event $B$.

Let $E_1, E_2, \dots, E_n$ be a partition of the sample space $S$: \[ E_i \cap E_j = \varnothing \ (i\neq j), \quad \bigcup_{i=1}^n E_i = S, \quad \mathbb{P}(E_i) > 0. \]

Then for any event $A$, \[ \mathbb{P}(A) = \sum_{k=1}^n \mathbb{P}(A \mid E_k)\,\mathbb{P}(E_k). \]

Given $E_1,\dots,E_n$ a partition with $\mathbb{P}(E_k)>0$ and $\mathbb{P}(A)>0$, Bayes’ theorem states: \[ \mathbb{P}(E_i \mid A) = \frac{\mathbb{P}(A \mid E_i)\,\mathbb{P}(E_i)} {\displaystyle \sum_{k=1}^n \mathbb{P}(A \mid E_k)\,\mathbb{P}(E_k)}, \quad i=1,\dots,n. \]

A disease affects 1% of the population. A test for the disease is 95% accurate, meaning:

• If someone has the disease, the test is positive with probability 0.95.

• If someone does not have the disease, the test is negative with probability 0.95.

What is the probability that someone who tests positive actually has the disease?

Solution:

Let:

• $D$ = person has disease $\Rightarrow P(D) = 0.01$

• $\neg D$ = person doesn't have disease $\Rightarrow P(\neg D) = 0.99$

• $T$ = test is positive

• $P(T|D) = 0.95$ (true positive)

• $P(T|\neg D) = 0.05$ (false positive)

Use Bayes' Theorem: \begin{align*} P(D|T) &= \frac{P(T|D) \cdot P(D)}{P(T|D) \cdot P(D) + P(T|\neg D) \cdot P(\neg D)} \\ &= \frac{0.95 \cdot 0.01}{0.95 \cdot 0.01 + 0.05 \cdot 0.99} \\ &= \frac{0.0095}{0.0095 + 0.0495} \\ &= \frac{0.0095}{0.059} \\ &\approx 0.161 \end{align*}

So, the probability is approximately 16.1%.

In a certain city, 60% of taxis are green and 40% are blue. A witness identifies a blue taxi involved in an accident. The witness is correct 80% of the time. What is the probability that the taxi was actually blue?

Solution:

Let:

• $B$ = taxi is blue $\Rightarrow P(B) = 0.4$

• $G$ = taxi is green $\Rightarrow P(G) = 0.6$

• $W$ = witness says blue

• $P(W|B) = 0.8$, $P(W|G) = 0.2$

Using Bayes’ Theorem: \begin{align*} P(B|W) &= \frac{P(W|B) \cdot P(B)}{P(W|B) \cdot P(B) + P(W|G) \cdot P(G)} \\ &= \frac{0.8 \cdot 0.4}{0.8 \cdot 0.4 + 0.2 \cdot 0.6} \\ &= \frac{0.32}{0.32 + 0.12} \\ &= \frac{0.32}{0.44} \\ &\approx 0.727 \end{align*}

So, the probability that the taxi was actually blue is approximately 72.7%.

A factory has 3 machines:

• Machine A produces 30% of items and has a 2% defect rate.

• Machine B produces 45% of items and has a 3% defect rate.

• Machine C produces 25% of items and has a 1% defect rate.

If an item is defective, what is the probability it was produced by Machine B?

Solution:

Let $D$ = item is defective, and $B$ = produced by Machine B.

\[ P(D) = (0.3 \cdot 0.02) + (0.45 \cdot 0.03) + (0.25 \cdot 0.01) = 0.006 + 0.0135 + 0.0025 = 0.022 \] \[ P(B|D) = \frac{P(D|B) \cdot P(B)}{P(D)} = \frac{0.03 \cdot 0.45}{0.022} = \frac{0.0135}{0.022} \approx 0.614 \]

So, the probability that a defective item came from Machine B is approximately 61.4%.