Figure 1: Examples of univariate Gaussian

Figure 1: Examples of univariate Gaussian

Univariate Gaussian distributions are fundamental in statistics, illustrating how continuous random variables behave. The document provides visual examples of probability density functions (PDFs) for various mean and variance parameters, showcasing the symmetric nature of the Gaussian curve. It serves as an essential resource for students and professionals studying statistical methods and probability theory. The content includes mathematical formulations and graphical representations, making it suitable for those preparing for exams in statistics or data analysis.

Key Points

  • Illustrates univariate Gaussian probability density functions with different parameters.
  • Explains the significance of mean and variance in Gaussian distributions.
  • Includes visual representations to aid understanding of statistical concepts.
  • Serves as a resource for students studying statistics and probability theory.
  • newtopiccyclegrowin
337
/ 6
−4 −3 −2 −1 0 1 2 3 4
0
0.2
0.4
0.6
0.8
x
µ = 0, σ = 1
µ = 1, σ =
1
/2
µ = 0, σ = 2
Figure 1: Examples of univariate Gaussian pdfs N(x; µ, σ
2
).
The Gaussian distribution
Probably the most-important distribution in all of statistics is the Gaussian distribution, also called
the normal distribution. The Gaussian distribution arises in many contexts and is widely used for
modeling continuous random variables.
The probability density function of the univariate (one-dimensional) Gaussian distribution is
p(x | µ, σ
2
) = N(x; µ, σ
2
) =
1
Z
exp

−
(x − µ)
2
2σ
2

.
The normalization constant Z is
Z =
√
2πσ
2
.
The parameters µ and σ
2
specify the mean and variance of the distribution, respectively:
µ = E[x]; σ
2
= var[x].
Figure 1 plots the probability density function for several sets of parameters
(µ, σ
2
)
. The distribution
is symmetric around the mean and most of the density (
≈ 99.7%
) is contained within
±3σ
of the
mean.
We may extend the univariate Gaussian distribution to a distribution over
d
-dimensional vectors,
producing a multivariate analog. The probablity density function of the multivariate Gaussian
distribution is
p(x | µ, Σ) = N(x; µ, Σ) =
1
Z
exp

−
1
2
(x − µ)
>
Σ
−1
(x − µ)

.
The normalization constant Z is
Z =
p
det(2πΣ) = (2π)
d
/2
(det Σ)
1
/2
.
1
−4 −2 0 2 4
−4
−2
0
2
4
x
1
x
2
(a) Σ =

1 0
0 1

−4 −2 0 2 4
−4
−2
0
2
4
x
1
x
2
(b) Σ =

1
1
/2
1
/2 1

−4 −2 0 2 4
−4
−2
0
2
4
x
1
x
2
(c) Σ =

1 −1
−1 3

Figure 2: Contour plots for example bivariate Gaussian distributions. Here
µ = 0
for all examples.
Examining these equations, we can see that the multivariate density coincides with the univariate
density in the special case when Σ is the scalar σ
2
.
Again, the vector
µ
specifies the mean of the multivariate Gaussian distribution. The matrix
Σ
specifies the covariance between each pair of variables in x:
Σ = cov(x, x) = E

(x − µ)(x − µ)
>

.
Covariance matrices are necessarily symmetric and positive semidefinite, which means their eigen-
values are nonnegative. Note that the density function above requires that
Σ
be positive definite, or
have strictly positive eigenvalues. A zero eigenvalue would result in a determinant of zero, making
the normalization impossible.
The dependence of the multivariate Gaussian density on
x
is entirely through the value of the
quadratic form
∆
2
= (x − µ)
>
Σ
−1
(x − µ).
The value
∆
(obtained via a square root) is called the Mahalanobis distance, and can be seen as a
generalization of the Z score
(x−µ)
σ
, often encountered in statistics.
To understand the behavior of the density geometrically, we can set the Mahalanobis distance to a
constant. The set of points in
R
d
satisfying
∆ = c
for any given value
c > 0
is an ellipsoid with
the eigenvectors of Σ defining the directions of the principal axes.
Figure 2 shows contour plots of the density of three bivariate (two-dimensional) Gaussian distribu-
tions. The elliptical shape of the contours is clear.
The Gaussian distribution has a number of convenient analytic properties, some of which we
describe below.
Marginalization
Often we will have a set of variables
x
with a joint multivariate Gaussian distribution, but only be
interested in reasoning about a subset of these variables. Suppose
x
has a multivariate Gaussian
distribution:
p(x | µ, Σ) = N(x, µ, Σ).
2
−4 −2 0 2 4
−4
−2
0
2
4
x
1
x
2
(a) p(x | µ, Σ)
−4 −2 0 2 4
0
0.1
0.2
0.3
0.4
x
1
p(x
1
)
(b) p(x
1
| µ
1
, Σ
11
) = N (x
1
; 0, 1)
Figure 3: Marginalization example. (a) shows the joint density over
x = [x
1
, x
2
]
>
; this is the same
density as in Figure 2(c). (b) shows the marginal density of x
1
.
Let us partition the vector into two components:
x =

x
1
x
2

.
We partition the mean vector and covariance matrix in the same way:
µ =

µ
1
µ
2

Σ =

Σ
11
Σ
12
Σ
21
Σ
22

.
Now the marginal distribution of the subvector x
1
has a simple form:
p(x
1
| µ, Σ) = N(x
1
, µ
1
, Σ
11
),
so we simply pick out the entries of µ and Σ corresponding to x
1
.
Figure 3 illustrates the marginal distribution of x
1
for the joint distribution shown in Figure 2(c).
Conditioning
Another common scenario will be when we have a set of variables
x
with a joint multivariate
Gaussian prior distribution, and are then told the value of a subset of these variables. We may
then condition our prior distribution on this observation, giving a posterior distribution over the
remaining variables.
Suppose again that x has a multivariate Gaussian distribution:
p(x | µ, Σ) = N(x, µ, Σ),
and that we have partitioned as before:
x = [x
1
, x
2
]
>
. Suppose now that we learn the exact value
of the subvector x
2
. Remarkably, the posterior distribution
p(x
1
| x
2
, µ, Σ)
3
/ 6
End of Document
337
You May Also Like

FAQs of Figure 1: Examples of univariate Gaussian

What are the key characteristics of univariate Gaussian distributions?
Univariate Gaussian distributions are defined by their mean (µ) and variance (σ²), which determine the shape and spread of the distribution. The probability density function (PDF) is symmetric around the mean, with approximately 99.7% of the data falling within three standard deviations from the mean. This property makes the Gaussian distribution crucial for statistical analysis, as it often describes the behavior of real-valued random variables in various fields.
How does the variance affect the shape of a Gaussian distribution?
Variance in a Gaussian distribution determines the width of the curve. A smaller variance results in a steeper curve, indicating that the data points are closely clustered around the mean. Conversely, a larger variance produces a flatter curve, suggesting that the data points are more spread out. Understanding this relationship is essential for interpreting statistical data and making predictions based on Gaussian models.
What is the significance of the normalization constant in Gaussian distributions?
The normalization constant in Gaussian distributions ensures that the total area under the probability density function equals one, making it a valid probability distribution. For univariate Gaussian distributions, this constant is derived from the variance and is crucial for accurately calculating probabilities. Without this normalization, the distribution would not properly represent the likelihood of different outcomes.
What applications do univariate Gaussian distributions have in statistics?
Univariate Gaussian distributions are widely used in various statistical applications, including hypothesis testing, confidence intervals, and regression analysis. They serve as the foundation for many statistical methods, allowing researchers to model and analyze data effectively. Additionally, they are used in machine learning algorithms, where assumptions of normality can simplify complex calculations and improve model performance.

Related of Figure 1: Examples of univariate Gaussian