%% ================================================================
\newpage
\section{Probability and measure theory}

Modern probability is a special case of measure theory, but this course avoids
the latter.  Here we draw the connections for the reader with a
measure-theoretic background.  (Full information may be found in \cite{bib:fg},
but I like to have a handy one-page reference.  See also \cite{bib:folland},
\cite{bib:rudin}, or \cite{bib:royden}.)

\textbf{Measure theory and probability:}  The \bfidx{sample space} $\Omega$ is
simply a set.

\textbf{Measure theory:}  A \bfidx{$\sigma$-field} $\mcF$ on $\Omega$ is a
subset of $2^\Omega$ satisfying the axioms of definition
\ref{defn:sigma_field}; $\mcF$ must contain at least $\emptyset$ and $\Omega$,
and it must be closed under complements, countable unions, and countable
intersections.

Note that even if $\Omega$ is uncountable (for example, $\Omega=\R$), $2^\Omega$
still satisfies the axioms for a $\sigma$-field.

\textbf{Measure theory:}  If $\Omega$ is a topological space (e.g. $\R^d$), the
standard $\sigma$-field is the \bfidx{Borel $\sigma$-field} which is the one
generated by all the open sets of $\Omega$.

\textbf{Measure theory:}  The pair $(\Omega, \mcF)$ is called a
\bfidx{measurable space}.  This is an unfortunate misnomer since it may not be
possible to put a measure on it --- in which case we would certainly think of
it as ``unmeasurable''!

\textbf{Measure theory:}  Elements of $\mcF$ are called \bfidx{measurable
sets}.  This is also a misnomer because we haven't defined measures yet!

\textbf{Probability:}  An \bfidx{event} is nothing more than a measurable set.

\textbf{Measure theory:}  A \bfidx{measure} is a function $\mu:\mcF \to
[0,+\infty]$ with the following properties:
\begin{itemize}
\item $\mu(\emptyset)=0$.
\item For all $A \in \mcF$, $\mu(A)\ge 0$.
\item If $A_1, A_2, \ldots,$ is a finite or countable subset of $\mcF$,
	with the $A_i$'s all (pairwise) disjoint, then
	$$
		P(\cup_{i} A_i) = \sum_i P(A_i).
	$$
	This is the \bfidx{countable additivity} property of the probability
	measure $\mu$.
\end{itemize}

\textbf{Probability:}  A \bfidx{probability measure} is a measure with the additional requirement that
$\mu(\Omega)=1$.  Thus, we have $\mu:\mcF \to [0,1]$.

Note that if $\Omega$ is finite or countably infinite, it is possible to define
a measure on the biggest $\sigma$-field on $\Omega$, namely, $\mcF=2^\Omega$.
If $\Omega=\R$, then it is not possible to define a measure on $\mcF=2^\Omega$.
See \cite{bib:folland} for a proof.

\textbf{Measure theory:}  A \bfidx{measure space} is a triple $(\Omega, \mcF, \mu)$ where $\mu$ is a
measure on $\mcF$.

\textbf{Probability:}  A \bfidx{probability space} is a triple $(\Omega, \mcF, P)$ where $P$ is a
probability measure on $\mcF$.

\textbf{Measure theory:}  A \bfidx{measurable function} is a function $f$ from
one measurable space $(\Omega,\mcF)$ to another measurable space $(\Psi,\mcG)$
such that the preimage under $f$ of each measurable set in $\Psi$ is a
measurable set in $\Omega$.  That is, for all $B \in \mcG$, $f^{-1}(B) \in
\mcF$.

\textbf{Probability:}  A \bfidx{random variable} is a measurable function $X$
from a probability space $(\Omega,\mcF,P)$ to a measurable space $(\Psi,\mcG)$.
For this course, that measurable space has been $\Psi=\R$, with $\mcG$ being
the Borel sets in $\R$.

\textbf{Measure theory:} xxx $\int X(\omega)\,dP(\omega)$.

\textbf{Probability:}  xxx $E[X]$.

\textbf{Measure theory:} xxx  Lapl/Four xform of measure.  Also, traditional
forms with densities.

\textbf{Probability:}  xxx 

%\begin{tabular}{p{3.0in}|p{3.0in}}
%\textbf{Measure theory / analysis} & \textbf{Probability} \\
%
%\hline
%
%The quick brown fox jumped over the lazy dogs.
%The quick brown fox jumped over the lazy dogs.
%The quick brown fox jumped over the lazy dogs.
%The quick brown fox jumped over the lazy dogs.
%The quick brown fox jumped over the lazy dogs.
%The asdfasdf asd fasd fasdf quick brown fox jumped over the lazy dogs.
%The quick brown fox jumped over the lazy dogs.
%The quick brown fox jumped over the lazy dogs.
%The quick brown fox jumped over the lazy dogs.
%The quick brown fox jumped over the lazy dogs.
%The quick brown fox jumped over the lazy dogs.
%The quick brown fox jumped over the lazy dogs.
%The quick brown fox jumped over the lazy dogs.
%The quick brown fox jumped over the lazy dogs.
%
%&
%
%sdfaadsf
%
%\\
%
%\end{tabular}

xxx (make footnotes in main body) ``distribution'' is measure; ``distribution
function'' is CDF; ``density'' is PDF.  Abs. cty and Rad-Nyk.

xxx 8-29 cty of finite measures.

xxx three kinds of measures: discrete, cts, singular.

xxx 9-5 and 9-24: MCTx2, DCTx2, Fub, Fatou

xxx cvgce: a.s./w.p.1; i.p.; i.m.s.; in dist (10-29 \& 10-31) flavors.

xxx 9-14 bo-can.

xxx 9-26 $\sigma$-field indep.

xxx 9-28 prod meas

xxx 10-19 Kolm 0-1 and tail field?
