%% ================================================================
\newpage
\section{Probability and measure theory}

\begin{center}
\emph{
Probability theory is measure theory with a soul.
} \\
--- Mark Kac.
\end{center}

Modern probability is a special case of measure theory, but this course avoids
the latter.  Here we draw the connections for the reader with a
measure-theoretic background.  (Full information may be found in \cite{bib:fg},
but I like to have a brief, handy reference.  See also \cite{bib:folland},
\cite{bib:rudin}, or \cite{bib:royden}.)

%% ----------------------------------------------------------------
\begin{tabular}{p{3.0in}|p{3.0in}}
\textbf{Measure theory / analysis} & \textbf{Probability} \\
\hline

The \bfidx{sample space} $\Omega$ is simply a set.

&

Same.

\\
\end{tabular}
%% ----------------------------------------------------------------

%% ----------------------------------------------------------------
\begin{tabular}{p{3.0in}|p{3.0in}}
\textbf{Measure theory / analysis} & \textbf{Probability} \\
\hline

A \bfidx{$\sigma$-field} $\mcF$ on $\Omega$ is a subset of $2^\Omega$
satisfying the axioms of definition \ref{defn:sigma_field}; $\mcF$ must contain
at least $\emptyset$ and $\Omega$, and it must be closed under complements,
countable unions, and countable intersections.

&

Same.

\\
\end{tabular}
%% ----------------------------------------------------------------

Note that even if $\Omega$ is uncountable (for example, $\Omega=\R$),
$2^\Omega$ still satisfies the axioms for a $\sigma$-field.  If $\Omega$ is a
topological space (e.g. $\R^d$), the standard $\sigma$-field is the
\bfidx{Borel $\sigma$-field} which is the one generated by all the open sets of
$\Omega$.

%% ----------------------------------------------------------------
\begin{tabular}{p{3.0in}|p{3.0in}}
\textbf{Measure theory / analysis} & \textbf{Probability} \\
\hline

The pair $(\Omega, \mcF)$ is called a \bfidx{measurable space}.  This is an
unfortunate misnomer since it may not be possible to put a measure on it --- in
which case we would certainly think of it as ``unmeasurable''!

&

Same.

\\
\end{tabular}
%% ----------------------------------------------------------------

%% ----------------------------------------------------------------
\begin{tabular}{p{3.0in}|p{3.0in}}
\textbf{Measure theory / analysis} & \textbf{Probability} \\
\hline

Elements of $\mcF$ are called \bfidx{measurable
sets}.  This is also a misnomer because we haven't defined measures yet!

&

An \bfidx{event} is nothing more than a measurable set.

\\
\end{tabular}
%% ----------------------------------------------------------------

%% ----------------------------------------------------------------
\begin{tabular}{p{3.0in}|p{3.0in}}
\textbf{Measure theory / analysis} & \textbf{Probability} \\
\hline

A \bfidx{measure} is a function $\mu:\mcF \to
[0,+\infty]$ with the following properties:
\begin{itemize}
\item $\mu(\emptyset)=0$.
\item For all $A \in \mcF$, $\mu(A)\ge 0$.
\item If $A_1, A_2, \ldots,$ is a finite or countable subset of $\mcF$,
	with the $A_i$'s all (pairwise) disjoint, then
	$P(\cup_{i} A_i) = \sum_i P(A_i)$.
	This is the \bfidx{countable additivity} property of the probability
	measure $\mu$.
\end{itemize}

&

A \bfidx{probability measure} is a measure with the additional requirement that
$\mu(\Omega)=1$.  Thus, we have $\mu:\mcF \to [0,1]$.

\\
\end{tabular}
%% ----------------------------------------------------------------

Note that if $\Omega$ is finite or countably infinite, it is possible to define
a measure on the biggest $\sigma$-field on $\Omega$, namely, $\mcF=2^\Omega$.
If $\Omega=\R$, then it is not possible to define a measure on $\mcF=2^\Omega$.
See \cite{bib:folland} for a proof.

%% ----------------------------------------------------------------
\begin{tabular}{p{3.0in}|p{3.0in}}
\textbf{Measure theory / analysis} & \textbf{Probability} \\
\hline

A \bfidx{measure space} is a triple $(\Omega, \mcF, \mu)$ where $\mu$ is a
measure on $\mcF$.

&

A \bfidx{probability space} is a triple $(\Omega, \mcF, P)$ where $P$ is a
probability measure on $\mcF$.

\\
\end{tabular}
%% ----------------------------------------------------------------

%% ----------------------------------------------------------------
\begin{tabular}{p{3.0in}|p{3.0in}}
\textbf{Measure theory / analysis} & \textbf{Probability} \\
\hline

A \bfidx{measurable function} is a function $f$ from one measurable space
$(\Omega,\mcF)$ to another measurable space $(\Psi,\mcG)$ such that the
preimage under $f$ of each measurable set in $\Psi$ is a measurable set in
$\Omega$.  That is, for all $B \in \mcG$, $f^{-1}(B) \in \mcF$.

&

A \bfidx{random variable} is a measurable function $X$ from a probability space
$(\Omega,\mcF,P)$ to a measurable space $(\Psi,\mcG)$.  For this course, that
measurable space has been $\Psi=\R$, with $\mcG$ being the Borel sets in $\R$.

\\
\end{tabular}
%% ----------------------------------------------------------------


%% ----------------------------------------------------------------
\begin{tabular}{p{3.0in}|p{3.0in}}
\textbf{Measure theory / analysis} & \textbf{Probability} \\
\hline

Expectation: $\int X(\omega)\,dP(\omega)$.

&

Expectation: $E[X]$.

\\
\end{tabular}
%% ----------------------------------------------------------------

\begin{rem}
In an undergraduate context, we use the terms \emphidx{probability density
function}\index{PDF} and \emph{cumulative distribution function}.
In a graduate context, we use the following:
\begin{itemize}

\item We have a real-valued random variable $X$.  That is, we have
$X:\Omega\to\R$ where $(\Omega,\mcF,P)$ is a probability space and the
measurable space is $(\R,\mcB)$, namely, the reals with the Borel
$\sigma$-algebra.

\item This gives us a \plainidx{probability measure} $\mu_X$ on $\R$ by
$\mu_X(B) = P(X^{-1}(B))$.  This is called simply the \bfidx{distribution}
of $X$.

\item We can then measure the particular Borel sets $(-\infty, x]$.  This
defines a function $F_X(x)=\mu_X((-\infty,x])$.  This (the CDF) is called
the \bfidx{distribution function} of $X$.

\item If $d\mu_X$ is absolutely continuous with respect to Lebesgue measure,
i.e. $d\mu_X=f_X(x)\,dx$ (where $f_X(x)$ is the familiar PDF), we say $f_X(x)$
is the \bfidx{density} of $X$.

\end{itemize}
\end{rem}

%% ----------------------------------------------------------------
\begin{tabular}{p{3.0in}|p{3.0in}}
\textbf{Measure theory / analysis} & \textbf{Probability} \\
\hline

The\index{Laplace transform}\index{Fourier transform}
Laplace and Fourier transforms, respectively, of $f:\R\to\R$
are
$$
	(\mcL f)(t) = \int_\R e^{tx}\,f(x)\,dt
$$
and
$$
	(\mcF f)(t) = \int_\R e^{itx}\,f(x)\,dt.
$$

&

The\index{moment-generating function}\index{characteristic function}
moment-generating and characteristic functions, respectively, of a
real-valued random variable $X$ are
$$
	M_X(t) = E[e^{tX}]
\qand
	\beta_X(t) = E[e^{itX}]
.
$$
Let $\mu$ be the distribution of $X$.  We say that the moment-generating
and characteristic functions are the \bfidx{Laplace transform} and
\bfidx{Fourier transform}, respectively, of the measure $\mu$:
$$
	\mcL\mu = \int_\R e^{tx}\,d\mu
\qand
	\mcF\mu = \int_\R e^{itx}\,d\mu.
$$
If the distribution of $X$ has a density function $f(x)$, i.e. $d\mu = f(x) dx$
where $d\mu$ is absolutely continuous with respect to Lebesgue measure $dx$,
then these are
$$
	M_X(t) = E[e^{tX}] = \int_\R e^{tx}\,f(x)\,dx = (\mcL f)(t)
$$
and
$$
	\beta_X(t) = E[e^{itX}] = \int_\R e^{itx}\,f(x)\,dx = (\mcF f)(t).
$$

\\
\end{tabular}
%% ----------------------------------------------------------------

%% ----------------------------------------------------------------

xxx 8-29 cty of finite measures.

xxx three kinds of measures: discrete, cts, singular.

xxx 9-5 and 9-24: MCTx2, DCTx2, Fub, Fatou

xxx cvgce: a.s./w.p.1; i.p.; i.m.s.; in dist (10-29 \& 10-31) flavors.

xxx 9-14 Borel-Cantelli.

xxx 9-26 $\sigma$-field indep.

xxx 9-28 prod meas

xxx 10-19 Kolm 0-1 and tail field?
