\documentclass{article}
\usepackage{psfig,amsmath,amscd}
\input{preamble.tex}
\newcommand{\tm}{{\rm TM}}
\newcommand{\prob}[2][]{\underset{#1}{\rm Prob}\left[{#2}\right]}
\newcommand{\func}{\rightarrow}
\newcommand{\Space}[1]{\rm SPACE(#1)}

\begin{document}

\lecture{14}{April 1, 1999}{Daniel A. Spielman}{Prahladh Harsha}

\section{Pseudo-Random Generators}

In today's lecture we shall introduce the notion of a {\it
Pseudo-Random Generator} (PRG). Roughly speaking, a PRG is a
deterministic device that takes a fairly small (random) seed, blows it
up in size and fools a randomised $\tm$ into believing that this
blown-up string was a purely random string. In more formal terms, we have

\begin{definition}
Let $M$ be a randomised $\tm$ that on input $w$ requires $f(|w|)$
random bits. The family $\{g_n\}_{n=1}^{\infty}$ of functions
($g_n:\{0,1\}^{s(n)} \func \{0,1\}^{f(n)}$) is an $\epsilon-$generator
for $M$ if
\begin{eqnarray*}
\forall w\qquad  \left| \prob[r \in \{0,1\}^{f(|w|)}]{M(w,r) \mathrm{
accepts}} - \prob[z \in \{0,1\}^{s(|w|)}]{M(w,g_{|w|}(z)) \mathrm{
accepts}} \right| < \epsilon
\end{eqnarray*}
\end{definition}
Instead of a family of functions, it is more convenient to view $g$,
the $\epsilon-$generator for $M$ as a $\tm$ that on input a string of length
$s(n)$ outputs a string of length $f(n)$. $s(n)$ is the length of the
seed of the PRG.

\section{Randomised Logspace $\tm$}
In this lecture, we shall present a PRG that fools all randomised
logspace $\tm$s. In the next lecture, we shall see a PRG for
polynomial time $\tm$s based on some unproven hardness
assumptions. Before proceeding any further, we have to clarify a point
regarding how the random bits are accessed in a randomised space
bounded computation (logspace in our case). There are 2 varying
definitions of randomised space bounded $\tm$s based on the manner
the random bits are accessed
\begin{itemize}
\item The $\tm$ obtains the random bits as and when required by it.
\item The string of random bits is fed as an auxiliary off-line input
(on a separate tape) in addition to the regular input to the $\tm$. In
this case, the head accessing the random bits on this tape can move
back and forth on the tape.
\end{itemize}
It is to be noted that in the case of randomised time bounded
computation it is immaterial which convention we observe. For
randomised space bounded computation, we shall consider only $\tm$s of
the first kind or equivalently consider $\tm$s of the second kind in
which the head on the random tape is restricted in the sense that it
can only move right along the tape (ie., the random tape is one-way
read-only tape) We shall also assume that all randomised space $S$
$\tm$s halt in time less than $2^S$.

\section{Nissan's Generator}

The PRG for randomised logspace $\tm$s is as follows:
 
\begin{theorem}\label{nissan}
There exists a $\Space{n}$ $\tm$ $ G$, $G:\{0,1\}^{O(\log^2m)} \func
\{0,1\}^m$ such that for all randomised logspace $\tm$ $M$, $G$ is a
$\frac{1}{n}-$generator for M.\footnote{We shall infact prove a more
general result that for all randomised space $S$ $\tm$ $M$, $G$ is a
$\frac{1}{2^S}-$generator for M.}
\end{theorem}

Before going into proving the existence of a PRG as mentioned in
Theorem \ref{nissan}, we shall first study the structure of the
computation tableau of a randomised space $S$ $\tm$ and find how this
structure can be exploited to reduce the randomness. The computation
tableau for a randomised space $S$ $\tm$ is as shown in Figure
1.

\begin{figure}[h]\label{comptable}
\begin{center}
\mbox{\psfig{file=computation.ps}}
\caption{The Computation Tableau of a space $S$ TM}
\end{center}
\end{figure} 
    
In the original definition of the randomised $\tm$, the computation
requires atmost $R(=2^S)$ random bits. Consider the tableau being divided into
2 halves with each half requiring random strings $r_1$ and $r_2$
respectively each of length $r (=R/2)$. If $r_1$ and $r_2$ are chosen
independently, then by definition the randomised $\tm$ works to give
the right results. We would like to choose $r_1$ and $r_2$ in such a
manner that the probability of going from state $s_1$ to state $s_2$
(ie., $\prob{s_1 \func s_2}$) and $\prob{s_2 \func s_3}$ are not
significantly different from the case when $r_1$ and $r_2$ are chosen
independently. We would now use the fact that the computation tableau
is very thin (width $S$ compared to length $2^S$) to let us choose
$r_1$ and $r_2$ in a manner better than independently.

To put things more formally, we have 2 algorithms $A_1$ and $A_2$ such that

\begin{equation*}
\begin{CD}
A_1 @<<< x_1\\
@VV{b_1}V\\
A_2 @<<< x_2\\
@VVV\\
b_2
\end{CD}
\end{equation*} 
\begin{center}{{\bf Figure 2:} $A_1,A_2$ with random inputs}\end{center}

\begin{itemize}
\item $A_1$ takes an input $x_1$ of length $r$ and outputs a string
$b_1$ of length $c$.  
\item $A_2$ takes as input the output $b_1$ of $A_1$ and another
string $x_2$ of length $r$ and outputs a string $b_2$ of length
$c$.
\end{itemize}


We can consider $A_1$ and $A_2$ to be the upper and lower halves
respectively of the computation of the randomised space $S$ $\tm$ and
$x_1,x_2$ to be the random strings $r_1,r_2$. What we are in search of
is a generator that supplies strings $x_1$ and $x_2$ in a fashion
better than choosing them independently. For notational brevity, given
a function $g$, define functions $g^l$ and $g^r$ such that $g^l(z)$
and $g^r(z)$ denote the left half and the right half of the string
$g(z)$ (ie., $g(z) = g^l(z)\circ g^r(z)$ and $|g^l(z)| = |g^r(z)|$)\footnote{$\circ$ denotes the concatenation operator}

\begin{definition} 
A function $g$ ($g:\{0,1\}^t \func \{0,1\}^r \times\{0,1\}^r$) is
defined to be a $\epsilon-$ generator for communication $c$ if for all
functions $A_1$ and $A_2$ such that $A_1:\{0,1\}^r \func \{0,1\}^c$
and $A_2:\{0,1\}^c\times\{0,1\}^r \func \{0,1\}^c$, we have that
\begin{eqnarray*} 
\forall b \qquad \left| \prob[x_1,x_2 \in
\{0,1\}^r]{A_2(A_1(x_1),x_2) = b} - \prob[z \in
\{0,1\}^t]{A_2(A_1(g^l(z)),g^r(z)) = b} \right| <
\epsilon
\end{eqnarray*}
\end{definition}
\begin{equation*}
\begin{CD}
A_1 @<<< g^l(z) \\
@VV{b_1}V\\
A_2 @<<< g^r(z) \\
@VVV\\
b_2
\end{CD}
\end{equation*}
\begin{center}{{\bf Figure 3:} $A_1,A_2$ with inputs from
generator}\end{center}

For notational convenience, we shall call a $2^{-c}-$generator for
communication $c$ a $c-$generator.

\section{Proof of Theorem \ref{nissan}}

For the present we shall assume the following lemma and give a sketch
of its proof in the end of the lecture.
\begin{lemma}\label{genexist}
There exists a constant $k>0$ such that for all $r,c$, there exists a
polynomial time computable $c-$generator $g$ where g is such that
$g:\{0,1\}^{r+kc} \func \{0,1\}^r\times \{0,1\}^r$.
\end{lemma}
Let $g_i:\{0,1\}^{(i+1)kS} \func \{0,1\}^{ikS} \times \{0,1\}^{ikS}$
be a $2S-$generator for $i = 1,2,\ldots ,S$ as guaranteed by lemma
\ref{genexist}. Define functions $G_i:\{0,1\}^{(i+1)kS} \func
\{0,1\}^{2^ikS}$ for $i = 0,1,\ldots S$ inductively as follows
\begin{eqnarray*}
G_0(z) & = & z\\
G_i(z) & = & G_{i-1}(g_i^l(z))\circ G_{i-1}(g_i^r(z))
\end{eqnarray*}

We shall show that the existence of $G_S$ implies Theorem
\ref{nissan}. Clearly, by definition of $G_S$, $G_S$ is a $\Space{n}$
$\tm$ (ie., it runs in space $O(kS^2)$). We only have to show that
$G_S$ fools all randomised space $S$ $\tm$s, which is implied by the
following theorem.

\begin{theorem}
For all $\tm$s A that run in space $S$ and in time $2^ikS$, $G_i$ is
an $(2^{i+1}-1)2^{-2S}-$generator for A
\end{theorem}
\begin{proof}
We shall prove by induction. Assume the statement of the theorem to be
true for all $i<j$. Consider any $\tm$ A that runs in space $S$ and
time $2^jkS$ and divide its computation into 2 halves $A_1$ and
$A_2$. By induction hypothesis we have that $G_{j-1}$ is a
$(2^j-1)2^{-2S}-$generator for both $A_1$ and $A_2$ (since both $A_1$
and $A_2$ run in space $S$ and time $2^{j-1}kS$). Hence we have that
for all $b_1, b_2$:
\begin{eqnarray}
\left| \prob[x_1]{A_1(x_1) =b_1} - \prob[y_1]{A_1(G_{j-1}(y_1))
=b_1}\right| < (2^j-1)2^{-2S}\\
\left| \prob[x_2]{A_2(b_1,x_2) =b_2} -
\prob[y_2]{A_2(b_1,G_{j-1}(y_2)) =b_2}\right| <
(2^j-1)2^{-2S}
\end{eqnarray}
where $x_1,x_2 \in \{0,1\}^{2^{j-1}kS}$ and $y_1,y_2 \in
\{0,1\}^{jkS}$. Combining (1) and (2) we have,
\begin{eqnarray}
\left| \prob[x_1,x_2]{A_2(A_1(x_1),x_2) =b_2} -
\prob[y_1,y_2]{A_2(A_1(G_{j-1}(y_1)),G_{j-1}(y_2)) = b_2} \right| <
(2^{j+1}-2)2^{-2S}
\end{eqnarray}
As $g_j$ is a $2S-$generator, we have
\begin{eqnarray}
\left| \prob[y_1,y_2]{A_2(A_1(G_{j-1}(y_1)),G_{j-1}(y_2)) = b_2}
-\prob[z]{A_2(A_1(G_{j-1}(g_j^l(z))),G_{j-1}(g_j^r(z))) = b_2} \right|
< 2^{-2S}
\end{eqnarray}
Combining (3) and (4) we have,
\begin{eqnarray*}
\left| \prob[x_1,x_2]{A_2(A_1(x_1),x_2) =b_2} -
\prob[z]{A_2(A_1(G_{j-1}(g_j^l(z))),G_{j-1}(g_j^r(z))) = b_2} \right|
< (2^{j+1}-1)2^{-2S}
\end{eqnarray*}
which proves that $G_j$ is a $(2^{j+1}-1)2^{-2S}-$generator for A. To
complete the proof, we have to anchor the induction. As $G_0(z)=z$,
the statement of the theorem is trivially true for $i=0$.
\end{proof}

\section{Sketch of Proof of Lemma \ref{genexist}}

Lemma \ref{genexist} basically tells us that Figure 2 can be replaced
by Figure 3. In other words, the $\epsilon-$generator for
communication $r+kc$, $g$, should construct strings $g^l(z)$ and $g^r(z)$
such that functions $A_1$ and $A_2$ are fooled into believing that
these strings were random ones. For a given $b_1,b_2$, the ``good''
$x_1,x_2$ are given by $$R_{b_1,b_2} = \{(x_1,x_2)|A_1(x_1)=b_1,
A_2(b_1,x_2)=b_2;x_1,x_2 \in \{0,1\}^r\}$$ The generator $g$ should
satisfy the following requirement in terms of $R_{b_1,b_2}$ $$\forall
b_1,b_2 \qquad \left|\prob[z\in \{0,1\}^{r+kc}]{(g^l(z),g^r(z)) \in
R_{b_1,b_2}} -\prob[x_1,x_2\in \{0,1\}^r]{x_1,x_2\in
R_{b_1,b_2}}\right| < \epsilon$$ Let us look at the structure of
$R_{b_1,b_2}$. let us consider a graph on vertices $V\times V$ where
$V=\{0,1\}^r$ (ie., the vertices correspond to all possible pairs
$(x_1,x_2)$. Also for the present we shall assume there are no edges
in the graph.)
\addtocounter{figure}{2}
\begin{figure}[h]\label{R_b1_b2}
\begin{center}
\mbox{\psfig{file=R_b1_b2.ps}}
\caption{Structure of $R_{b_1,b_2}$}
\end{center}
\end{figure} 
The graph on $V\times V$ can be divided into rows as shown in Figure 4
where each row corresponds to an element $b_1^{(i)}$ and contains all
$x$ such that $A_1(x) = b_1^{(i)}$. Now, each of these rows is further
divided into blocks corresponding to $b_2^{(j)}$ with each block
containing all elements $x$ such that $A_2(b_1^{(i)},x) =
b_2^{(j)}$. Viewing $V \times V$ in this manner, $R_{b_1,b_2}$ looks
like a combinatorial rectangle of the form $S_1 \times S_2; S_1,S_2
\subseteq V$, where $S_1 =\{x_1|A_1(x_1)=b_1\}, S_2 =\{x_2|A_2(b_1,x_2)=b_2\}$
(refer figure). Thus the probability that a random $(x_1,x_2)$ is
in $R_{b_1,b_2}$ is equal to the probability that a random pair of
vertices in $V \times V$ is from $S_1 \times S_2$. (ie.,
$\prob{(x_1,x_2) \in R_{b_1,b_2}} = \frac{|S_1||S_2|}{|V|^2}$) Now if
we imagine an expander $G$ on the set of vertices $V\times V$, then
probability that a random edge in $E(G)$ lies in
$E(S_1,S_2)$\footnote{$E(V_1,V_2) =\{e| e \mbox{ has one endpoint in
}V_1 \mbox{ and another in } V_2$} is very close to the probability
that a pair of random vertices lies in $S_1\times S_2$ as guaranteed
by the following lemma. And that does the job for us as then the
generator could choose a random edge in $G$ and output the 2 endpoints of
that edge.

\begin{lemma}
If $G$ be an expander graph of degree $d$ on vertices $V$ and $\lambda$
is an upper bound on all eigen values but for the largest, then $$ \forall S_1,S_2 \subseteq V \qquad \left|\frac{E(S_1,S_2)}{|E|}-\frac{|S_1||S_2|}{|V|^2}\right| < \frac{\lambda}{d}$$
\end{lemma}

\end{document}