Axiomatic Approach and Laws of Probability

Axiomatic Approach to Probability (Axioms of Probability)

Need for a General Approach

The classical definition of probability, $P(E) = n(E)/n(S)$, is intuitive but has limitations. It requires that all outcomes in the sample space be equally likely and that the sample space be finite. This definition cannot be applied directly to situations involving:

• Experiments where outcomes are not equally likely (e.g., a biased coin, the probability of rain vs. sunshine).
• Experiments with infinite sample spaces (e.g., the exact time a phone call arrives, the number of coin tosses until the first head).

The experimental (empirical) approach provides estimates based on observed frequencies, but it doesn't provide theoretical properties or allow for rigorous mathematical deduction.

To provide a more general and mathematically rigorous foundation for probability theory, regardless of the specific nature of the experiment or how probabilities are assigned, the **axiomatic approach** was developed. The most widely accepted axiomatic system was proposed by Russian mathematician **Andrey Kolmogorov** in 1933.

This approach defines probability as a measure that satisfies a set of fundamental rules or axioms. Any function that adheres to these axioms is considered a valid probability measure.

Setup of the Axiomatic Framework

The axiomatic approach to probability involves the following components:

1. Sample Space ($S$ or $\Omega$):

   This is the set of all possible outcomes of a random experiment. It can be finite, countably infinite, or uncountably infinite.

2. Events:

   An event is defined as a subset of the sample space $S$. The collection of all events for which probabilities can be defined must satisfy certain closure properties (it forms a $\sigma$-algebra). For introductory purposes, it's sufficient to think of events as the subsets of $S$ that are of interest, including the sample space $S$ itself (the sure event) and the empty set $\phi$ (the impossible event).

3. Probability Measure ($P$):

   This is a function that assigns a real number, called the probability of event $E$ and denoted by $P(E)$, to each event $E$ in the collection of events. This function $P$ must satisfy the following set of axioms.

Kolmogorov's Axioms of Probability

For any random experiment with sample space $S$ and a defined collection of events, a probability function $P$ must satisfy the following three axioms:

1. Axiom 1: Non-negativity

   The probability of any event $E$ is a non-negative real number.

   $$P(E) \ge 0 \quad \text{for every event } E$$

   ... (1)

   This means probabilities cannot be negative.

2. Axiom 2: Normalization (Probability of the Sample Space)

   The probability of the entire sample space $S$ (the sure event) is equal to 1.

   $$P(S) = 1$$

   ... (2)

   This means the total probability of all possible outcomes is 1, representing certainty that one of the outcomes in the sample space will occur.

3. Axiom 3: Countable Additivity (for Mutually Exclusive Events)

   If $E_1, E_2, E_3, \dots$ is a sequence of **mutually exclusive** events (i.e., no two events in the sequence can occur simultaneously, which means their pairwise intersections are empty: $E_i \cap E_j = \phi$ for all $i \neq j$), then the probability of the union of these events is the sum of their individual probabilities.

   $$P(E_1 \cup E_2 \cup E_3 \cup \dots) = P(E_1) + P(E_2) + P(E_3) + \dots$$

   ... (3)

   This axiom is particularly important for infinite sample spaces. For a finite number of mutually exclusive events, say A and B such that $A \cap B = \phi$, the axiom implies:

   $$P(A \cup B) = P(A) + P(B) \quad \text{if } A \cap B = \phi$$

   ... (4)

These three axioms are the fundamental rules from which all other properties and theorems of probability theory can be logically derived. Any probability assignment must satisfy these axioms to be mathematically consistent.

Laws of Probability (Addition Law, Probability of Complementary Event)

Derived Properties and Theorems

Based on Kolmogorov's three axioms of probability, several fundamental rules and theorems can be logically derived. These laws help in calculating probabilities of complex events from the probabilities of simpler ones.

1. Probability of the Impossible Event

The probability of the impossible event (the empty set, $\phi$) is always 0.

Proof:

Consider the sample space $S$. We know that $S = S \cup \phi$. Also, $S$ and $\phi$ are mutually exclusive events, since their intersection is empty ($S \cap \phi = \phi$).

By Axiom 3 (Additivity for mutually exclusive events):

Since $S \cup \phi = S$, the left side is $P(S)$. So the equation becomes:

By Axiom 2 (Normalization), $P(S) = 1$. Substituting this into equation (ii):

Subtracting 1 from both sides gives $P(\phi) = 0$.

2. Probability of the Complementary Event

For any event $A$, the probability that the event $A$ does **not** occur, denoted by $P(A')$, is equal to 1 minus the probability that $A$ does occur.

This implies $P(A) + P(A') = 1$.

Proof:

Let $A$ be any event in the sample space $S$. The complementary event $A'$ consists of all outcomes in $S$ that are not in $A$.

By definition of a complementary event, $A$ and $A'$ together make up the entire sample space, and they have no outcomes in common:

• $A \cup A' = S$ (Their union is the sample space)
• $A \cap A' = \phi$ (They are mutually exclusive)

Since $A$ and $A'$ are mutually exclusive, we can apply Axiom 3 (Additivity):

Since $A \cup A' = S$, the left side is $P(S)$. So the equation becomes:

By Axiom 2 (Normalization), $P(S) = 1$. Substituting this into equation (v):

Rearranging this equation gives the desired result: $P(A') = 1 - P(A)$.

3. Range of Probability (Revisited)

From Axiom 1, we know $P(E) \ge 0$. Using the property of complementary events, we can show that $P(E) \le 1$.

Proof that $P(E) \le 1$:

For any event $E$, its complement $E'$ exists, and $P(E') \ge 0$ by Axiom 1. Also, we derived that $P(E) + P(E') = 1$.

So, $P(E) = 1 - P(E')$. Since $P(E') \ge 0$, subtracting a non-negative number from 1 means $P(E)$ cannot exceed 1. Thus, $P(E) \le 1$.

Combining with Axiom 1, we reaffirm the range of probability:

4. Addition Law of Probability (for any two events)

For any two events A and B associated with a random experiment (whether they are mutually exclusive or not), the probability of the occurrence of at least one of the events (meaning A occurs, or B occurs, or both occur) is given by the formula:

Here, $A \cup B$ represents the union of events A and B (A or B or both occurring), and $A \cap B$ represents the intersection of events A and B (both A and B occurring).

Interpretation: When we simply add the probabilities of A and B, $P(A) + P(B)$, any outcome that is in both A and B (i.e., in the intersection $A \cap B$) is counted twice (once as part of A, once as part of B). To correct for this double counting, we subtract the probability of the intersection $P(A \cap B)$ once.

Special Case (Addition Law for Mutually Exclusive Events):

If events A and B are mutually exclusive, then their intersection is the empty set, $A \cap B = \phi$. The probability of the impossible event is $P(\phi) = 0$. Substituting this into the general Addition Law:

This is consistent with Axiom 3 for a finite number of mutually exclusive events and is often referred to as the special Addition Rule.

5. Addition Law for Three Events

The Addition Law can be extended to three events A, B, and C:

This formula follows the principle of inclusion-exclusion.

Example

Answer: