Chapter 6 Probability Distribution (Concepts)

Probability Distribution

In probability theory, we often deal with numerical outcomes of random experiments. A random variable is a variable whose value is a numerical outcome of a random phenomenon. It is a function that maps the outcomes of a random experiment to real numbers. Random variables allow us to apply mathematical analysis to the results of random events.

Random variables are primarily classified into two types:

• Discrete Random Variable: A variable that can take only a finite or countably infinite number of distinct values. These values are typically integers or other discrete units. Examples include the number of heads in coin tosses, the number of defective items in a sample, or the number of calls received by a call centre in an hour.
• Continuous Random Variable: A variable that can take any value within a given range or interval. These values are typically measurements. Examples include the height or weight of a person, the temperature of a place, or the time taken to complete a task.

A probability distribution describes how the total probability of 1 is distributed among the possible values of a random variable. It is a function that links each possible value (or range of values) of the random variable to its probability of occurrence. Understanding the probability distribution is crucial for characterizing the behavior of a random variable and for making predictions or inferences about the random phenomenon it represents.

Discrete Probability Distribution

A discrete probability distribution describes the probabilities for a discrete random variable. It is defined by a list of the possible values the variable can take and the probability associated with each value.

Probability Mass Function (PMF)

For a discrete random variable $X$, the Probability Mass Function (PMF), often denoted by $P(X=x)$ or $f(x)$, gives the probability that the random variable $X$ takes on a specific value $x$. The PMF essentially lists out all possible outcomes and their corresponding probabilities.

The properties of a valid PMF $P(X=x)$ for a discrete random variable $X$ are:

1. The probability of each specific value must be non-negative and not greater than 1:

   $0 \le P(X=x) \le 1$

   (For all possible values of $x$)

2. The sum of the probabilities for all possible values must be exactly 1:

   $\sum_x P(X=x) = 1$

   (Sum over all possible values of $x$)

   ... (1)

A discrete probability distribution can be represented in various ways:

• Table: Listing the values $x$ and their probabilities $P(X=x)$ in columns.
• Graph: Using a bar chart or histogram where the height of the bar represents the probability for each value of $x$.
• Formula: A mathematical expression that gives $P(X=x)$ for any valid value of $x$.

Example: Consider the random experiment of tossing two fair coins. Let $X$ be the discrete random variable representing the number of heads obtained. The possible outcomes of the experiment are {HH, HT, TH, TT}. The corresponding values of the random variable $X$ are:

• For HH, $X=2$
• For HT, $X=1$
• For TH, $X=1$
• For TT, $X=0$

Now, we calculate the probability for each possible value of $X$:

• $P(X=0) = P(\text{TT}) = \frac{1}{4}$ (Probability of getting 0 heads)
• $P(X=1) = P(\text{HT or TH}) = P(\text{HT}) + P(\text{TH}) = \frac{1}{4} + \frac{1}{4} = \frac{2}{4} = \frac{1}{2}$ (Probability of getting 1 head)
• $P(X=2) = P(\text{HH}) = \frac{1}{4}$ (Probability of getting 2 heads)

The probability distribution of $X$ can be shown in a table:

Let's check the properties of PMF:

• All probabilities ($1/4, 1/2, 1/4$) are between 0 and 1. Property 1 is satisfied.
• The sum of probabilities is:

  $\sum P(X=x) = P(X=0) + P(X=1) + P(X=2)$

  $= \frac{1}{4} + \frac{1}{2} + \frac{1}{4} = \frac{1}{4} + \frac{2}{4} + \frac{1}{4} = \frac{1+2+1}{4} = \frac{4}{4} = 1$

  Property 2 is satisfied.

Cumulative Distribution Function (CDF)

For a discrete random variable $X$, the Cumulative Distribution Function (CDF), denoted by $F(x)$, gives the probability that the random variable $X$ takes on a value less than or equal to a specific value $x$. It accumulates the probabilities up to that value.

where the sum is over all possible values $t$ of $X$ such that $t \le x$.

For the coin toss example ($X$=number of heads):

• $F(0) = P(X \le 0) = P(X=0) = \frac{1}{4}$
• $F(1) = P(X \le 1) = P(X=0) + P(X=1) = \frac{1}{4} + \frac{1}{2} = \frac{1}{4} + \frac{2}{4} = \frac{3}{4}$
• $F(2) = P(X \le 2) = P(X=0) + P(X=1) + P(X=2) = \frac{1}{4} + \frac{1}{2} + \frac{1}{4} = 1$

The CDF is defined for all real numbers $x$. For values of $x$ not in the set of possible outcomes, $F(x)$ remains constant between consecutive possible values and jumps up at each possible value. For $x < 0$, $F(x) = 0$. For $x \ge 2$, $F(x) = 1$. For instance:

• $F(0.5) = P(X \le 0.5) = P(X=0) = 1/4$ (since the only possible value $\le 0.5$ is 0)
• $F(1.5) = P(X \le 1.5) = P(X=0) + P(X=1) = 1/4 + 1/2 = 3/4$

Properties of a discrete CDF $F(x)$:

• $F(x)$ is non-decreasing: If $a < b$, then $F(a) \le F(b)$.
• $F(x)$ is a step function, jumping at the possible values of $X$.
• $\lim\limits_{x \to -\infty} F(x) = 0$
• $\lim\limits_{x \to \infty} F(x) = 1$
• $P(a < X \le b) = F(b) - F(a)$
• $P(X = x) = F(x) - \lim\limits_{t \to x^-} F(t)$ (the jump size at $x$)

Continuous Probability Distribution

A continuous probability distribution describes the probabilities for a continuous random variable. Since a continuous random variable can take an infinite number of values in an interval, the probability of it taking any single specific value is zero ($P(X=x) = 0$ for any $x$). Instead, probabilities are defined for intervals of values.

Probability Density Function (PDF)

For a continuous random variable $X$, the Probability Density Function (PDF), denoted by $f(x)$, is a function such that the area under the curve of $f(x)$ between two points $a$ and $b$ gives the probability that $X$ falls within the interval $[a, b]$. The height of the PDF curve at a point $x$ indicates the relative likelihood of the variable being close to $x$.

The properties of a valid PDF $f(x)$ for a continuous random variable $X$ (defined over its domain) are:

1. The function must be non-negative for all values in its domain:

   $f(x) \ge 0$

   (For all $x$ in the domain of $X$)

2. The total area under the curve $y = f(x)$ over the entire domain of $X$ is equal to 1. If the variable is defined over $(-\infty, \infty)$, this means:

   $\int\limits_{-\infty}^{\infty} f(x) dx = 1$

   ... (3)

   If the domain is a specific interval $[c, d]$, then $\int\limits_c^d f(x) dx = 1$, and $f(x) = 0$ outside this interval.

The probability that the random variable $X$ takes a value within a specific interval $[a, b]$ is given by the definite integral of the PDF over that interval:

For a continuous random variable, $P(X=a) = \int_a^a f(x) dx = 0$. Therefore, for continuous distributions, $P(a \le X \le b) = P(a < X \le b) = P(a \le X < b) = P(a < X < b)$. The inclusion or exclusion of endpoints does not change the probability.

Cumulative Distribution Function (CDF)

For a continuous random variable $X$, the Cumulative Distribution Function (CDF), $F(x)$, gives the probability that $X$ takes on a value less than or equal to $x$. It is defined as the integral of the PDF from the lower limit of the domain (often $-\infty$) up to $x$.

From the Fundamental Theorem of Calculus, if $F(x)$ is differentiable, its derivative is the PDF:

Properties of a continuous CDF $F(x)$:

• $F(x)$ is non-decreasing: If $a < b$, then $F(a) \le F(b)$.
• $F(x)$ is continuous for all $x$.
• $\lim\limits_{x \to -\infty} F(x) = 0$
• $\lim\limits_{x \to \infty} F(x) = 1$
• $P(a \le X \le b) = F(b) - F(a) = \int_a^b f(x) dx$

Examples

Answer:

Answer:

Mathematical Expectation

The mathematical expectation or expected value of a random variable is a crucial concept in probability and statistics. It represents the long-run average value of the random variable if the random experiment is repeated many times. It is a measure of the central location or centre of gravity of the probability distribution. The expected value of a random variable $X$ is denoted by $E(X)$ or $\mu$ (mu).

Think of it this way: If you performed a random experiment a very large number of times and calculated the average of the outcomes, that average would be very close to the expected value.

Expected Value for Discrete Random Variables

For a discrete random variable $X$ that can take on values $x_1, x_2, x_3, \dots, x_n$ (or infinitely many values in a countable set), with corresponding probabilities $P(X=x_1), P(X=x_2), P(X=x_3), \dots, P(X=x_n)$, the expected value $E(X)$ is calculated as the weighted average of the possible values, where the weights are the corresponding probabilities.

The formula for the expected value of a discrete random variable $X$ is:

where the sum is taken over all possible values $x_i$ that the random variable $X$ can take, and $P(X=x_i)$ is the probability mass function (PMF) at $x_i$.

Expected Value for Continuous Random Variables

For a continuous random variable $X$ with a probability density function (PDF) $f(x)$, defined over the range $(-\infty, \infty)$, the expected value is calculated using integration instead of summation, because the variable can take on any value within a range.

The formula for the expected value of a continuous random variable $X$ is:

If the PDF $f(x)$ is zero outside a specific interval $[a, b]$, then the limits of integration can be restricted to that interval:

The integral sums up the product of each possible value $x$ and its "probability weight" $f(x) dx$ over the entire range.

Properties of Expectation

Mathematical expectation has several useful properties that simplify calculations. Let $X$ and $Y$ be random variables, and let $a$ and $b$ be constants.

1. Expected value of a constant: The expected value of a constant is the constant itself.

   $E(c) = c$

   ... (4)

2. Expected value of a constant multiplied by a random variable: A constant factor can be pulled out of the expectation.

   $E(aX) = a E(X)$

   ... (5)

3. Expected value of a sum of random variables: The expected value of the sum of random variables is the sum of their expected values. This property holds regardless of whether the variables are independent or not.

   $E(X + Y) = E(X) + E(Y)$

   ... (6)

   This property extends to any finite number of random variables: $E(X_1 + X_2 + \dots + X_n) = E(X_1) + E(X_2) + \dots + E(X_n)$.
4. Expected value of a linear transformation: Combining properties 1, 2, and 3.

   $E(aX + b) = a E(X) + E(b) = a E(X) + b$

   ... (7)

5. Expected value of a product of independent random variables: If $X$ and $Y$ are independent random variables, then the expected value of their product is the product of their expected values.

   $E(XY) = E(X) E(Y)$

   ... (8)

   Note: This property is ONLY true if $X$ and $Y$ are independent. In general, $E(XY) \neq E(X)E(Y)$.
6. Expected value of a function of a random variable: If $g(X)$ is a function of a random variable $X$, the expected value of $g(X)$ is calculated as follows:
   • For discrete $X$ with PMF $P(X=x_i)$:

     $E(g(X)) = \sum_i g(x_i) P(X=x_i)$

     ... (9)

   • For continuous $X$ with PDF $f(x)$:

     $E(g(X)) = \int\limits_{-\infty}^{\infty} g(x) f(x) dx$

     ... (10)

   A common application is finding $E(X^2)$ by setting $g(X) = X^2$.
   • For discrete $X$: $E(X^2) = \sum_i x_i^2 P(X=x_i)$
   • For continuous $X$: $E(X^2) = \int\limits_{-\infty}^{\infty} x^2 f(x) dx$

Examples

Answer:

Answer:

Variance

While the expected value gives us a measure of the central tendency of a random variable, it doesn't tell us anything about the dispersion or spread of the possible values around this centre. The variance and standard deviation are measures that quantify this spread.

The variance of a random variable $X$ is a measure of how far the values of the random variable are spread out from its expected value (mean). A high variance indicates that the possible values are widely spread, while a low variance indicates that they are clustered closely around the mean. Variance is denoted by $Var(X)$ or $\sigma^2$ (sigma squared).

The variance is formally defined as the expected value of the squared difference between the random variable $X$ and its mean $E(X)$ (denoted by $\mu$):

Or, using the notation $\mu = E(X)$:

We square the difference $(X - \mu)$ because squaring makes all differences positive (so that deviations in either direction from the mean contribute to the spread) and also emphasizes larger deviations. Taking the expected value of this squared difference gives the average squared deviation from the mean.

Alternative Formula for Variance

The definition formula $Var(X) = E[(X - \mu)^2]$ is useful for understanding what variance measures, but it can be tedious to compute directly, especially when dealing with the squared terms. An alternative formula is often easier for calculations:

This formula states that the variance is the expected value of the square of the random variable minus the square of the expected value of the random variable.

Derivation of the Alternative Formula:
Starting with the definition $Var(X) = E[(X - \mu)^2]$ where $\mu = E(X)$:

Expand the squared term $(X - \mu)^2$:

Using the linearity property of expectation $E(aY + bZ) = aE(Y) + bE(Z)$:

Using the properties $E(cY) = cE(Y)$ and $E(c) = c$, where $c$ is a constant. Here, $-2\mu$ and $\mu^2$ are constants (since $\mu = E(X)$ is a single value):

Now, substitute $\mu = E(X)$ back into equation (A):

Combine the terms with $[E(X)]^2$:

This confirms the alternative formula for variance. To use this formula, you first need to calculate $E(X)$ and $E(X^2)$.

Expected Value of $X^2$, $E(X^2)$

The term $E(X^2)$ is the expected value of the random variable $X^2$. Using the property $E(g(X)) = \sum g(x_i) P(X=x_i)$ for discrete variables and $E(g(X)) = \int g(x) f(x) dx$ for continuous variables (as discussed in the previous section on Properties of Expectation, point 6), we set $g(X) = X^2$.

• For a discrete random variable $X$ with possible values $x_i$ and probabilities $P(X=x_i)$:

  $\mathbf{E(X^2) = \sum_i x_i^2 P(X=x_i)}$

  ... (4)

• For a continuous random variable $X$ with PDF $f(x)$:

  $\mathbf{E(X^2) = \int\limits_{-\infty}^{\infty} x^2 f(x) dx}$

  ... (5)

  If the PDF is defined over $[a, b]$, the integral becomes $\int\limits_{a}^{b} x^2 f(x) dx$.

Variance for Discrete Random Variables

For a discrete random variable $X$ with possible values $x_1, x_2, \dots, x_n$ and corresponding probabilities $P(X=x_1), P(X=x_2), \dots, P(X=x_n)$, and mean $\mu = E(X)$:

Using the definition:

Using the alternative formula:

where $E(X) = \sum x_i P(X=x_i)$ and $E(X^2) = \sum x_i^2 P(X=x_i)$.

Variance for Continuous Random Variables

For a continuous random variable $X$ with probability density function $f(x)$ and mean $\mu = E(X)$:

Using the definition:

Using the alternative formula:

where $E(X) = \int x f(x) dx$ and $E(X^2) = \int x^2 f(x) dx$, with integrals taken over the appropriate range of $x$.

Standard Deviation

The standard deviation of a random variable $X$, denoted by $\sigma$, is defined as the positive square root of the variance:

The standard deviation is often preferred over variance as a measure of spread for a couple of reasons:

• It has the same units as the random variable $X$ itself, making it easier to interpret in the context of the problem.
• It provides a more direct measure of the typical distance of a value from the mean.

Properties of Variance

Like expectation, variance also has several useful properties. Let $X$ and $Y$ be random variables, and let $a$ and $b$ be constants.

1. Variance of a constant: A constant has no variability.

   $Var(c) = 0$

   ... (11)

2. Variance of a constant times a random variable: The constant comes out squared.

   $Var(aX) = a^2 Var(X)$

   ... (12)

   Derivation: $Var(aX) = E[(aX - E(aX))^2] = E[(aX - aE(X))^2] = E[a^2 (X - E(X))^2] = a^2 E[(X - E(X))^2] = a^2 Var(X)$.
3. Variance of a linear transformation: Adding a constant shifts the distribution but does not change its spread.

   $Var(aX + b) = Var(aX) + Var(b) = a^2 Var(X) + 0 = a^2 Var(X)$

   ... (13)

   This shows that adding or subtracting a constant ($b$) does not affect the variance, while multiplying by a constant ($a$) multiplies the variance by the square of the constant.
4. Variance of a sum of independent random variables: If $X$ and $Y$ are independent random variables, the variance of their sum (or difference) is the sum of their variances.

   If $X, Y$ are independent, then $Var(X + Y) = Var(X) + Var(Y)$

   ... (14)

   If $X, Y$ are independent, then $Var(X - Y) = Var(X) + Var(Y)$

   ... (15)

   Note: This property holds only if $X$ and $Y$ are independent. For dependent variables, a covariance term is involved: $Var(X+Y) = Var(X) + Var(Y) + 2Cov(X,Y)$.

Examples

Answer:

Answer:

Binomial Distribution

Among the various discrete probability distributions, the Binomial Distribution is one of the most important and frequently used. It is applied to situations where a fixed number of independent trials are conducted, and each trial has only two possible outcomes. The binomial distribution helps us calculate the probability of getting a specific number of 'successes' in these trials.

Bernoulli Trial

The foundation of the Binomial Distribution is the Bernoulli Trial. A Bernoulli trial is a single random experiment that has only two possible outcomes. These outcomes are typically labelled as:

• Success: The desired outcome. The probability of success is denoted by $p$.
• Failure: The other outcome. The probability of failure is denoted by $q$.

Examples of Bernoulli trials include:

• Tossing a coin (Head or Tail)
• Checking if a manufactured item is defective (Defective or Not Defective)
• Answering a multiple-choice question with one correct option (Correct or Incorrect)
• Drawing a card from a deck and noting its colour (Red or Black)

Binomial Experiment (or Sequence of Bernoulli Trials)

A Binomial Experiment is a sequence of independent Bernoulli trials that satisfy four specific conditions:

1. There must be a fixed number of trials, denoted by $n$. The experiment consists of repeating the same Bernoulli trial $n$ times.
2. Each trial must have only two possible outcomes, designated as "success" (S) and "failure" (F).
3. The probability of success, $p$, is the same for each trial. Consequently, the probability of failure, $q = 1 - p$, is also the same for each trial.
4. The trials must be independent. The outcome of one trial does not influence the outcome of any other trial.

In a binomial experiment, the random variable of interest is the number of successes obtained in the $n$ trials. Let this random variable be $X$. Since there are $n$ trials, the number of successes $X$ can take any integer value from 0 (no successes) up to $n$ (all trials are successes).

If a random variable $X$ follows a binomial distribution with $n$ trials and probability of success $p$, we denote this by $X \sim B(n, p)$.

Probability Mass Function (PMF) of the Binomial Distribution

The Probability Mass Function (PMF) of a binomial random variable $X$ gives the probability of obtaining exactly $k$ successes in $n$ independent Bernoulli trials, where the probability of success in a single trial is $p$.

Consider a sequence of $n$ trials. If we want exactly $k$ successes, these $k$ successes can occur in any combination of the $n$ trials. The number of ways to choose $k$ positions for the successes out of $n$ trials is given by the binomial coefficient $\binom{n}{k}$.

For any specific sequence with exactly $k$ successes and $n-k$ failures (e.g., S S ... S F F ... F), the probability of this specific sequence occurring is $p^k q^{n-k}$ due to the independence of trials ($p$ for each success, $q$ for each failure, multiplied together).

Since there are $\binom{n}{k}$ such distinct sequences, and each has a probability of $p^k q^{n-k}$, the total probability of getting exactly $k$ successes is the sum of the probabilities of all these sequences.

The PMF of a binomial random variable $X \sim B(n, p)$ for obtaining exactly $k$ successes is:

where:

• $k$ is the number of successes we are interested in ($0 \le k \le n$).
• $\binom{n}{k} = \frac{n!}{k!(n-k)!}$ is the binomial coefficient, representing the number of combinations of $n$ items taken $k$ at a time.
• $p$ is the probability of success in a single trial.
• $q = 1 - p$ is the probability of failure in a single trial.
• $n-k$ is the number of failures in $n$ trials.

The sum of probabilities for all possible values of $k$ must be 1, i.e., $\sum_{k=0}^n P(X=k) = \sum_{k=0}^n \binom{n}{k} p^k q^{n-k}$. By the Binomial Theorem, this sum is equal to $(p+q)^n = (p + (1-p))^n = 1^n = 1$. This confirms that it is a valid probability distribution.

Mean and Variance of the Binomial Distribution

For a binomial random variable $X \sim B(n, p)$, the mean, variance, and standard deviation have simple formulas.

Mean ($E(X)$ or $\mu$)

The expected number of successes in $n$ trials is given by the product of the number of trials and the probability of success in a single trial.

The derivation of this formula can be done using the definition of expectation for a discrete variable $E(X) = \sum_{k=0}^n k P(X=k) = \sum_{k=0}^n k \binom{n}{k} p^k q^{n-k}$. This involves algebraic manipulation of summations and binomial coefficients.

Variance ($Var(X)$ or $\sigma^2$)

The variance measures the spread of the distribution around the mean. For a binomial distribution, the variance is given by the product of the number of trials, the probability of success, and the probability of failure.

The derivation of the variance formula is more involved than that for the mean and typically uses $Var(X) = E(X^2) - [E(X)]^2$, requiring the calculation of $E(X^2) = \sum_{k=0}^n k^2 P(X=k)$.

Standard Deviation ($\sigma$)

The standard deviation is the positive square root of the variance:

Examples

Answer:

Answer:

Poisson Distribution

The Poisson Distribution, named after the French mathematician Siméon Denis Poisson, is a discrete probability distribution that is used to model the probability of a given number of events occurring in a fixed interval of time or space, or in a fixed number of trials, provided these events occur with a known constant mean rate and independently of the time or space since the last event. It is particularly useful for describing the distribution of rare events.

For example, the number of traffic accidents at a specific intersection in a week, the number of typos on a page of a book, the number of customers arriving at a service counter in an hour, or the number of radioactive decays per second are often modeled using a Poisson distribution.

Poisson Process Conditions

A random phenomenon is said to follow a Poisson process if the occurrence of events satisfies the following conditions over a given interval (time, space, etc.):

1. Events are independent: The number of events occurring in any fixed interval is independent of the number of events occurring in any other disjoint interval. The occurrence of an event does not affect the probability of another event occurring in a separate interval.
2. Events occur singly: Events occur one at a time. It is not possible for two or more events to occur at exactly the same point in the interval.
3. Constant average rate: The average rate at which events occur is constant over the interval. This constant average rate is the sole parameter of the Poisson distribution and is denoted by $\lambda$ (lambda). $\lambda$ represents the mean number of events in the specified interval.

The random variable $X$ in a Poisson distribution counts the number of events that occur in a fixed interval. The possible values of $X$ are non-negative integers: $0, 1, 2, 3, \dots$. Theoretically, there is no upper limit to the number of events, although the probability of a very large number of events becomes very small.

A random variable $X$ following a Poisson distribution with parameter $\lambda$ is denoted by $X \sim P(\lambda)$ or $X \sim \text{Poisson}(\lambda)$. The parameter $\lambda$ must be positive ($\lambda > 0$).

Probability Mass Function (PMF) of the Poisson Distribution

The Probability Mass Function (PMF) of a Poisson random variable $X$ with parameter $\lambda$ gives the probability of observing exactly $k$ events in the given interval.

The PMF is given by the formula:

where:

• $k$ is the number of events we are interested in (a non-negative integer).
• $\lambda$ is the average rate of events in the fixed interval ($\lambda > 0$).
• $e$ is the base of the natural logarithm, approximately 2.71828.
• $k!$ is the factorial of $k$, defined as $k! = k \times (k-1) \times \dots \times 2 \times 1$ for $k > 0$, and $0! = 1$.

The sum of probabilities for all possible values of $k$ must be 1, i.e., $\sum_{k=0}^{\infty} P(X=k) = \sum_{k=0}^{\infty} \frac{e^{-\lambda} \lambda^k}{k!}$. This sum is equal to $e^{-\lambda} \sum_{k=0}^{\infty} \frac{\lambda^k}{k!}$. The series $\sum_{k=0}^{\infty} \frac{\lambda^k}{k!}$ is the Taylor series expansion of $e^{\lambda}$. So, the sum is $e^{-\lambda} \cdot e^{\lambda} = e^0 = 1$. This confirms that the PMF is valid.

Mean and Variance of the Poisson Distribution

A distinguishing characteristic of the Poisson distribution is that its mean and variance are equal to its parameter $\lambda$.

Mean ($E(X)$ or $\mu$)

The expected number of events in the fixed interval is simply the average rate $\lambda$.

Note: The derivation of $E(X)=\lambda$ from $E(X) = \sum_{k=0}^\infty k P(X=k) = \sum_{k=0}^\infty k \frac{e^{-\lambda} \lambda^k}{k!}$ involves manipulating infinite series.

Variance ($Var(X)$ or $\sigma^2$)

The variance of a Poisson distributed variable is also equal to $\lambda$.

Note: The derivation of $Var(X)=\lambda$ from $Var(X) = E(X^2) - [E(X)]^2$ is also involved and uses properties of infinite series.

Standard Deviation ($\sigma$)

The standard deviation is the positive square root of the variance:

Poisson Distribution as a Limiting Case of Binomial Distribution

The Poisson distribution can be viewed as a special or limiting case of the binomial distribution. Consider a binomial distribution $X \sim B(n, p)$ where $n$ is the number of trials and $p$ is the probability of success. If we have a situation where the number of trials $n$ is very large, but the probability of success $p$ in any single trial is very small, such that the product $np$ remains constant and finite, say $np = \lambda$, then the binomial distribution approaches the Poisson distribution with parameter $\lambda$.

Formally, if $n \to \infty$ and $p \to 0$ such that $np \to \lambda$, then for any fixed non-negative integer $k$, the binomial probability $P(X=k) = \binom{n}{k} p^k (1-p)^{n-k}$ converges to the Poisson probability $P(X=k) = \frac{e^{-\lambda} \lambda^k}{k!}$.

This relationship is practically useful because calculating binomial probabilities $\binom{n}{k} p^k (1-p)^{n-k}$ can be computationally intensive for large $n$. If $n$ is large and $p$ is small, we can approximate the binomial probability using the Poisson formula with $\lambda = np$. A common rule of thumb for this approximation to be reasonably good is when $n \ge 50$ and $np \le 5$.

Examples

Answer:

Given:

To Find:

Solution:

Answer:

Given:

To Find:

Solution:

Normal Distribution

The Normal Distribution is arguably the most significant probability distribution in the fields of statistics and probability theory. It is a continuous probability distribution characterized by its symmetrical, bell-shaped curve. Many natural phenomena and measurements (such as heights, weights, blood pressure, measurement errors, test scores) tend to follow a normal distribution. Furthermore, its theoretical importance is underscored by the Central Limit Theorem, which states that the distribution of sample means approaches a normal distribution as the sample size increases, regardless of the shape of the original population distribution.

A normal distribution is completely specified by its mean ($\mu$) and its variance ($\sigma^2$). We denote a normal random variable $X$ with mean $\mu$ and variance $\sigma^2$ as $X \sim N(\mu, \sigma^2)$. The standard deviation is $\sigma = \sqrt{\sigma^2}$.

Properties of the Normal Distribution

The normal distribution has several distinctive properties:

1. Shape and Symmetry: The graph of the normal distribution is a symmetric, unimodal (single peak), and bell-shaped curve. It is perfectly symmetrical about its mean $\mu$.
2. Central Tendency: For a normal distribution, the mean ($\mu$), median, and mode are all equal and are located at the center of the distribution.
3. Asymptotic Nature: The normal curve extends infinitely in both directions along the horizontal axis. It approaches the x-axis but never touches it.
4. Total Area Under the Curve: The total area under the probability density curve of a normal distribution is always equal to 1. This reflects that the sum of probabilities for all possible outcomes is 1.
5. Parameters: The shape and position of the normal curve are determined solely by its two parameters: the mean ($\mu$) and the standard deviation ($\sigma$).
   • The mean ($\mu$) determines the location of the center of the distribution on the horizontal axis.
   • The standard deviation ($\sigma$) determines the spread or dispersion of the distribution. A larger $\sigma$ leads to a wider, flatter curve, indicating more variability. A smaller $\sigma$ leads to a narrower, taller curve, indicating less variability.
6. Inflection Points: The points where the curve changes its curvature (from concave down to concave up) are called inflection points. For a normal distribution, these points occur exactly at one standard deviation away from the mean, i.e., at $x = \mu - \sigma$ and $x = \mu + \sigma$.
7. Empirical Rule (68-95-99.7 Rule): This rule provides a quick estimate of the proportion of data that falls within certain standard deviation ranges from the mean in a normal distribution.
   • Approximately 68% of the data falls within one standard deviation of the mean: $P(\mu - \sigma \le X \le \mu + \sigma) \approx 0.68$.
   • Approximately 95% of the data falls within two standard deviations of the mean: $P(\mu - 2\sigma \le X \le \mu + 2\sigma) \approx 0.95$.
   • Approximately 99.7% of the data falls within three standard deviations of the mean: $P(\mu - 3\sigma \le X \le \mu + 3\sigma) \approx 0.997$.
   This rule highlights how concentrated the data is around the mean in a normal distribution.

Probability Density Function (PDF)

As the normal distribution is continuous, its probability distribution is described by a Probability Density Function (PDF), not a PMF. The PDF of a normal random variable $X$ with mean $\mu$ and standard deviation $\sigma$ is given by the formula:

where:

• $x$ is any real number, representing a possible value of the random variable.
• $\mu$ is the mean of the distribution.
• $\sigma$ is the standard deviation of the distribution ($\sigma > 0$).
• $\pi \approx 3.14159$ and $e \approx 2.71828$ are mathematical constants.

The height of the curve at any point $x$ is given by $f(x)$. The probability of $X$ falling within an interval $[a, b]$ is the area under the curve between $a$ and $b$, calculated by the definite integral:

However, this integral does not have a simple closed-form solution and must be evaluated using numerical methods or by converting to the standard normal distribution.

Standard Normal Distribution

Since the integral of the normal PDF is difficult to compute directly for arbitrary $\mu$ and $\sigma$, probabilities for normal distributions are typically found by converting the problem to an equivalent one involving the Standard Normal Distribution.

The standard normal distribution is a special case of the normal distribution where the mean is 0 and the standard deviation is 1.

A random variable following the standard normal distribution is usually denoted by $Z$. $Z \sim N(\mu=0, \sigma^2=1)$.

The PDF of the standard normal distribution is obtained by setting $\mu=0$ and $\sigma=1$ in the general PDF formula:

Probabilities for the standard normal distribution, $P(Z \le z)$, have been extensively calculated and are available in standard normal tables (often called Z-tables). These tables typically provide the cumulative probability, i.e., the area under the standard normal curve to the left of a given Z-value.

Z-Score (Standardization)

Any value $x$ from a general normal distribution $X \sim N(\mu, \sigma^2)$ can be converted into a corresponding value $z$ from the standard normal distribution $Z \sim N(0, 1)$ using the Z-score formula:

The Z-score tells us how many standard deviations a particular value $x$ is away from the mean $\mu$. A positive Z-score means $x$ is above the mean, a negative Z-score means $x$ is below the mean, and a Z-score of 0 means $x$ is equal to the mean.

Using the Z-score transformation, probabilities for a general normal distribution can be found using the Z-tables:

Let $z_1 = \frac{a - \mu}{\sigma}$ and $z_2 = \frac{b - \mu}{\sigma}$. Then $P(a \le X \le b) = P(z_1 \le Z \le z_2)$. This probability can be calculated using Z-tables as $P(Z \le z_2) - P(Z \le z_1)$.

Due to the symmetry of the standard normal distribution around 0:

• $P(Z > z) = 1 - P(Z \le z)$
• $P(Z < -z) = P(Z > z) = 1 - P(Z \le z)$
• $P(-z \le Z \le z) = P(Z \le z) - P(Z \le -z) = P(Z \le z) - (1 - P(Z \le z)) = 2 P(Z \le z) - 1$

Examples

Answer:

Given:

To Find:

Solution:

Answer:

Given:

To Find:

Solution: