\documentclass{article}
%
%\usepackage{pdfsync}           % Used in Mac OSX
\usepackage[mathcal,mathbf]{euler}
\usepackage{theorem,amsmath,enumerate,fancyhdr,amssymb,amsfonts}
\usepackage{graphicx}           %Used in Mac OSX
%\usepackage[pdftex]{graphics}  %Might be needed in non Mac OSX systems
\usepackage{myDefs}
%In order to be consistent use the following notation in your notes:

\title{
Network Design and Game Theory\\
Spring 2008\\
Lecture 1 %1 or 2 or 3 etc
}
\author{
{\bf Instructor:} Mohammad T. Hajiaghayi\\
{\bf Scribe:} Ricardo A. Collado
}
\date{January 28, 2008} %put the date of the lecture here NOT the date the note was written

\begin{document}

\pagestyle{fancy}
\lhead{{\bf scribe:}{ Ricardo A. Collado}
\\{\bf Lecture 1} } %insert lecture number
\rhead{{\bf Date: }01/28/2008} %enter lecture date NOT today's date

\maketitle
%

\medskip
\section{Overview} %give a one paragraph short description of the topics
We define and explore three classical problems: {\sc Set Cover}, {\sc Unique Coverage}, and {\sc Maximum Coverage} problems.

\section{Definitions}
(1) {\bf SET COVER}\\[.5ex]
\noindent
{\sc Input :}
A universe set $U=\{e_1,\ldots,e_n\}$, a collection $\mathcal{S} = \{S_1\ldots S_k\}$ of subsets of $U$ such that $U=\bigcup\{S_1,\ldots,S_k\}$, and a cost function $cost: \mathcal{S}\to\mathbb{Q}^+$.
\\[1ex]
\noindent
{\sc Goal:} Find a minimum cost $\mathcal{S}' \subseteq \mathcal{S}$ such that $S'$ covers $U$, i.e. $U=\bigcup_{S\in\mathcal{S}'}S$.
\\[2ex]

\noindent
(2) {\bf UNIQUE COVERAGE}\\[.5ex]
\noindent
{\sc Input :}
A universe set $U=\{e_1,\ldots,e_n\}$ and a collection $\mathcal{S} = \{S_1\ldots S_k\}$ of subsets of $U$ such that $U=\bigcup\{S_1,\ldots,S_k\}$.
\\[1ex]
\noindent
{\sc Goal:} Find $\mathcal{S}' \subseteq \mathcal{S}$ that maximizes the number of uniquely covered elements.
\\[2ex]

\noindent
(3) {\bf MAXIMUM COVERAGE}\\[.5ex]
\noindent
{\sc Input :}
A universe set $U=\{e_1,\ldots,e_n\}$ and a collection $\mathcal{S} = \{S_1\ldots S_k\}$ of subsets of $U$ such that $U=\bigcup\{S_1,\ldots,S_k\}$. Also,  cost and weight functions $cost: \mathcal{S}\to\mathbb{Q}^+$ and $w: U\to\mathbb{Q}^+$ and a bound $L\in\mathbb{Q}^+$.
\\[1ex]
\noindent
{\sc Goal:} Find $\mathcal{S}' \subseteq \mathcal{S}$ such that the total cost of its sets is at most $L$ while maximizing the total weight of covered elements.
\\[3ex]
These three problems are NP-complete and can be polynomialy reduced to one another. For example, any instance of {\sc Set Cover} can be solved with polynomialy many calls to {\sc Maximum Coverage} by assigning a weight of 1 to each element of $U$ and solving {\sc Maximum Coverage} for increasing  $L$'s until the minimum $L$ that gives a cover of $U$ is found.

Wireless networking has many applications of these problems. For example,
suppose that there are different options of where to build  transmission towers and each location has its range and cost. {\sc Set cover} can be used to find a network of towers that cover everything and has minimum cost. If a limiting budget is added then {\sc Maximum Coverage} could be used to find a network that covers the maximum within the budget's constraints. Also, the {\sc Unique Coverage} could be used to minimize interference among the transmission towers.

\section{Set Cover}
Consider an instance of {\sc Set Cover} with universe $U$ and collection $\mathcal{S}$.
\begin{definition}
The \emph{frequency} of $e_i\in U$ is the number of sets from $\mathcal{S}$ that contain $e_i$. Let $F$ be the maximum frequency of all elements of $U$.
\end{definition}

Next we will see three approximation algorithms to {\sc Set Cover} and analyze their approximation ratio.\\


The next algorithm assumes that the cost of all sets is 1.

\noindent
{\bf Greedy $F$} Approximation Algotithm:
\begin{enumerate}
\item[1:] Set $A \leftarrow \emptyset$, $U' \leftarrow \emptyset$.
\item[2:] While $U$ has an element $e$ not covered by $A$ add all sets containing $e$ to $A$ and add $e$ to $U'$.
\end{enumerate}
Output: Cover $A$.\\

\noindent
{\bf Analysis}~~
The number of sets chosen is at most $F\,|U'|$ and no set can cover two elements of $U'$. Therefore $$OPT \ge |U'| \ge \frac{1}{F}|A|$$ and we conclude that {\bf Greedy $F$} is an $F$ approximation algorithm.\\


The next algorithm uses linear programming to find and approximation to {\sc Set Cover}.


\noindent
{\bf LP-Rouding} Approximation Algorithm:\\
\begin{enumerate}

\item Set $A \leftarrow \emptyset$
\item Solve the following linear program in polynomial time
\begin{equation}
\label{LP}
\begin{array}{rrl}
\text{minimum:} &  \sum_{j=1}^{|\mathcal{S}|} cost(S_j)\cdot x_j\\[2ex]
\text{subject to}& \sum\{x_j \mid e_i \in S_j \}\ge 1, &i=1,\ldots,|U|\\
&x_i \ge 0, &i=1,\ldots,|\mathcal{S}|
\end{array}
\end{equation}
\item In the optimal solution to (\ref{LP}) pick all variables $x_i$'s with value at least $\frac{1}{F}$ and  add the corresponding $S_i$'s to $A$.
\end{enumerate}
Otput: Cover $A$.\\


\noindent
{\bf Remark} ~~The solution of (\ref{LP}) plus the constraints $x_i\in\{0,1\}, i=1,\ldots,|\mathcal{S}|$ is exactly the solution of {\sc Set Cover}. Therefore (\ref{LP}) is a linear programming relaxation of {\sc Set Cover}.\\

\noindent
{\bf Analysis}\\
\noindent
Let $x^*$ be the optimal solution to the $LP$ relaxation obtained in step 2 of the algorithm. \\[1ex]
(i) Suppose that $e\in U$ is not covered by $A$. Then
$$ \sum\{x^*_j \mid e \in S_j \} < \sum\{\frac{1}{F} \mid e \in S_j \} \le F \frac{1}{F}= 1
$$
which contradicts the feasibility of $x^*$.  Therefore $A$ is a cover of $U$.\\[1ex]
\noindent
(ii) Let $cost(OPT), cost(LP)$ be the cost of the optimal solution and the cost of the solution to the $LP$ relaxation obtained in step 2, respectively. Then
\begin{align*}
cost(OPT) &\ge cost(LP)  \\
&=\sum_{j=1}^{|\mathcal{S}|} cost(S_j)\cdot x_j^*\\
&=\sum_{S\in A}cost(S)\cdot x^*_S + \sum_{Z\not\in A}cost(Z)\cdot x^*_Z\\
&\ge \sum_{S\in A}cost(S)\cdot x^*_S\\
&\ge \frac{1}{F}\sum_{S\in A}cost(S).
\end{align*}
Therefore $cost(OPT) \ge \frac{1}{F} cost(A)$, where $A$ is the cover given by the algorithm, so {\bf LP-Rounding} is an $F$ approximation algorithm.
\\[1ex]

\noindent
{\bf Greedy $O(\lg n)$} Approximation Algorithm:\\
\begin{enumerate}
\item Set $A\leftarrow \emptyset, C\leftarrow \emptyset$.
\item While $C \not= U$ do:
\begin{enumerate}
\item Find a set $S$ which is most cost effective: i.e. $C$ minimizes $$\alpha = \frac{cost(S)}{S-C},$$
\item add $S$ to $A$ and for each $e\in S-C$ set $price(e)=\alpha$, and
\item set $C\leftarrow C\cup \{S\}$.
\end{enumerate}
\end{enumerate}
Output: Cover $A$.\\

\noindent
{\bf Analysis}\\
\noindent
(i) Let $\{S_1,\dots,S_k\}$ be an optimal solution to {\sc Set Cover} at iteration $t$. Then
\begin{align*}
cost(opt_t) &= \sum_{i=1}^k cost(S_i)\\
&= \sum_{i=1}^k \frac{cost(S_i)}{|S_i|}|S_i|\\
&\ge  \sum_{i=1}^k \alpha_t |S_i|\\
&\ge \alpha_t|U-C|.
\end{align*}
Therefore at iteration $t$ $$\alpha_t \le \frac{cost(opt_t)}{|U-C|}.$$
\noindent
(ii)\begin{align*}
cost(ALG) &= \sum_{e\in U}price(e)\\
&= \sum_{i=1}^n price(e_i)\\
&\le \sum_{i=1}^n \frac{cost(opt_i)}{n-i+1}\\
&\le cost(opt) \sum_{i=0}^n\frac{1}{i}\\
&= cost(opt)\lg(n).
\end{align*}
Therefore $$cost(opt)\ge\frac{1}{\lg (n)} cost(ALG)$$ and {\bf Greedy $O(\lg n)$} is an $O(\lg n)$ approximation algorithm .
\\[1ex]


Important theorem to remember:
\begin{theorem}[Ferge]
There is no $(1-\epsilon)\lg n$ approximation algorithm to {\sc set cover} unless $NP\subseteq DTIME\left( n^{\lg\lg n} \right)$.
\end{theorem}
















\end{document}
























In order to discuss the complexity of methods used to solve semidefinite programing problems one should take care to avoid problems where the solution is exponentially large in the size of the input. The following example illustrates this:
%
\begin{equation}
\label{expo_orig}
\begin{array}{rll}
\text{min:}& \vx_n&\\
\text{s.t.}& \vx_i^2\le\vx_{i+1}, &i=0,\ldots,n-1\\
&\vx_i\ge0, &i=0,\ldots,n-1\\
&\vx_0=2.&
\end{array}
\end{equation}
%
Clearly problem (\ref{expo_orig}) is equivalent to the following semidefinite formulation.
\begin{equation}
\label{expo_semi}
\begin{array}{rll}
\text{min:}& \vx_n&\\
\text{s.t.}&\vx_0=2&\\[.8ex]
& \left(\begin{array}{cc}\vx_{i+1} & \vx_i \\\vx_i & 1\end{array}\right) \psd\vzero&\\[2ex]
&\text{for each }i=0,\ldots,n-1.
\end{array}
\end{equation}
%
By noting that $\vx_i^2\le\vx_{i+1}$ if and only if
$$\left(\frac{\vx_i}{\sqrt{2}}\right)^2 + \left(\frac{\vx_{i+1}-1}{2}\right)\le\left(\frac{\vx_{i+1}+1}{2}\right),$$
%
we obtain the following second order cone programming formulation of problem (\ref{expo_orig}):
%
\begin{equation}
\label{expo_socp}
\begin{array}{rll}
\text{min:}& \vx_n&\\
\text{s.t.}&\vx_0=2&\\[.8ex]
& \left(\begin{array}{c}\frac{\vx_{i+1}+1}{2} \\\frac{\vx_{i+1}-1}{2} \\\frac{\vx_i}{\sqrt{2}}\end{array}\right) \Qgeq\vzero\\[2ex]
&\text{for each }i=0,\ldots,n-1.
\end{array}
\end{equation}
%
It is easy to see from formulation (\ref{expo_orig}) that the solution to this problem is $2^{2^n}$ (which is of size $O(2^n)$) but the input to all of the formulations can be done in $O(n)$ bits.

Another similar problem were we are concerned with very small numbers instead of very large ones is
%
\begin{equation}
\label{expo_small}
\begin{array}{rll}
\text{max:}& \vx_0&\\
\text{s.t.}& \vx_i^2\le\vx_{i+1}, &i=0,\ldots,n-1\\
&\vx_i\ge0, &i=0,\ldots,n-1\\
&\vx_n=1/2.&
\end{array}
\end{equation}
%
The solution to this problem $2^{-2^n}$ which is also of size $O(2^n)$.


It is important to see that any feasible solution to any of these problems is of size $O(2^n)$ and thus exponential in the size of the input. This is dire contrast to the linear programming problem where an application of Cramer's rule shows that the size of  the solution is polynomial in the size of the input always.  Another difference between LP and SDP is that in LP we stop the ellipsoid method when the ellipsoid gets "too small" (in polynomial number of steps) for the solution to be represented with few bits but in SDP these feasible points could be "too large" or "too small".
%Thus in SDP we need a different stopping criterion for the ellipsoid method.
It is also important to note that an LP with rational numbers will have a rational solution whereas an SDP with rational numbers could yield only irrational solutions.

In order to avoid the aforementioned pitfalls we will make the following assumptions on all the problems we consider.

\noindent
We will solve:
%
\begin{equation*}
\begin{array}{lrl}
\text{(}P\text{)}~~~~&\text{min:}& \vc^T\vx\\
&\text{s.t.}& \vx\in C\\
&&C\text{ is a convex set.}
\end{array}
\end{equation*}
%
We assume:
\begin{enumerate}
\item Exist $r>0, R>0$ and $\va\in C$ such that $B(\va,r)\subseteq C \subseteq B(\vzero,R)$.\\[1ex] Here $r$ guarantees that the points are not "too small" and $R$ that the points are not "too large".

\item There is $\epsilon>0$ such that if $\vx^*$ is the solution of problem ($P$) we are actually looking for a feasible solution $\vx_\epsilon$ such that $\|\vx_\epsilon-\vx^*\|<\epsilon$.
\\[1ex]
Here we agree that a "good enough" approximation will suffice for our purposes.
\end{enumerate}

\begin{definition}
An algorithm is {\bf polynomial} if it runs in polynomial time on the size of $r$, $R$, $\epsilon$, and the data structures needed to describe $C$.
\end{definition}
The constants $r$, $R$, and $\epsilon$ are usually used to devise stopping conditions for the algorithms.

In general we will see two types of algorithms:
\begin{enumerate}
\item Ellipsoid Method Algorithms: Developed in the 70's. Solves feasibility and starts with a large ellipsoid.
\item Interior Point Methods: Developed in the 50's and 60's. Originally discarded because impracticability and later found to be both doable and practical.
\end{enumerate}
Both of these methods can be used to solve SDP and SOCD problems efficiently.

\section{Barrier Functions}
Let $\mathbb{R}^*$ denote the extended real numbers, i.e. $\mathbb{R}^*=\mathbb{R}\cup\{-\infty,\infty\}$. Let $C$ be a convex cclosed set.
\begin{definition}
A function $b:C\to\mathbb{R}^*$ is a {\bf barrier function} if
\begin{enumerate}
\item $b\left({Int(C)}\right) \subseteq \mathbb{R}$,
\item $b$ is convex, and
\item if $\{\vx_i\} _{i=0}^\infty \subseteq Int(C)$ is a sequence converging to $\vx^*\in bd(C)$, implies that $\lim_{i\to\infty}b(\vx_i)=\infty$.
\end{enumerate}
\end{definition}

\subsection{Use of Barrier Functions}
Let $C$ be closed convex set, $b$ a barrier function, and $\mu>0$. Let $\vx_\mu$ be the optimal solution to the problem
%
\begin{equation*}
\begin{array}{lrl}
\text{(}P_\mu\text{)}~~~&\text{min:}& \vc^T\vx + \mu b(\vx).
\end{array}
\end{equation*}
%
Problem ($P_\mu$) is easier to solve than ($P$) because the barrier function $b$ will not let $\vx_\mu$ be too close to $bd(C)$ and thus unconstrained optimization techniques can be used to approximate $\vx_\mu$. Following this idea we show a general scheme  for the interior point methods:
\begin{center}
{\sc A Template for Interior Point Approach}
\end{center}
\begin{enumerate}
\item Start with a feasible point $\vx:=\vx_1\in Int(C)$ and $\mu:=\mu_1>0$.
\item Use some algorithm (e.g. Newton's Method) to approximate the solution to $(P_\mu)$. Call this point $\vx_{\mu_1}$.
\item Let $0<\alpha<1$ and $\mu_{i+1} =\alpha\mu_i$.
\item Go back to step 2  and iterate with $\mu = \mu_{i+1}$ and $\vx=\vx_{\mu_{i+1}}$.
\end{enumerate}
%
As $\mu\to 0$ the points $\vx_\mu$ traverse continuos a path in the interior of $C$. It is a theorem that as $\mu_i\to0$ the points $\vx_\mu$ converge to $\vx^*$, the optimal solution of ($P$), see Figure \ref{CentralPath}. This path is known as the {\bf central path}.
%
\begin{figure}[htbp]
\centering
\includegraphics[scale=.60]{CentralPath}
\caption{Continuos path delineated by $\vx_{\mu_i}$ as $\mu_i\to0$.}
\label{CentralPath}
\end{figure}

We will approximate the central path with a polygonal path on the points $\vx_{\mu_i}$, see Figure \ref{PolyPath}, and will stop the algorithm when the next polygonal path step is less than $r$.

To prove polynomial complexity of the algorithm it has to be shown that there are not "too many" breaks in the line. A necessary condition for this is that the path is not too twisted, i.e. the curvature of the path is not too much. This is called the {\bf self corcondance theory}.
%
\begin{figure}[htbp]
\centering
\includegraphics[scale=.60]{PolyPath}
\caption{Approximation of the central path of $C$ by a polygonal path with vertices $\vx_{\mu_i}$.}
\label{PolyPath}
\end{figure}


\subsection{Examples of Barrier Functions}
\subsubsection{Barriers on the Nonnegative Orthant}
Let $A$ be the nonnegative orthant, i.e. $A=\left\{\vx\in\mathbb{R}^n ~\middle|~ \vx_i\ge0, i=1,\ldots,n\right\}$. Define $b:A\to\mathbb{R}^*$ by
%
\begin{equation*}
b(\vx)=\begin{cases}
\sum_{i=1}^n-\ln\vx_i& \text{if $\vx\in Int(A)$},\\
\infty& \text{if $\vx\in bd(A)$}.
\end{cases}
\end{equation*}
%
Clearly $b$ is a continuos function and it satisfies the first two properties of a barrier function.
\begin{claim}
$b$ is a convex function.
\end{claim}
\begin{proof}
Since $b$ is twice continuos we check that the Hessian of $b$ is positive semidefinite. The gradient of $b$ is
$$\Delta_\vx b(\vx) = \left( -\frac{1}{\vx_1},\ldots, -\frac{1}{\vx_n}\right).$$ Therefore the Hessian of $b$ is:
$$\Delta_\vx^2 b(\vx) = \left(\begin{array}{ccc}\frac{1}{\vx_1^2} & 0 & 0 \\0 & \ddots & 0 \\0 & 0 & \frac{1}{\vx_n^2}\end{array}\right) \psd \vzero$$
%
\end{proof}
Therefore $b$ is a barrier function for the positive orthant. With an almost identical proof it can be showed that the following is also a barrier function for the nonnegative orthant $A$.
%
\begin{equation*}
b_1(\vx)=\begin{cases}
\sum_{i=1}^n\frac{1}{\vx_i}& \text{if $\vx\in Int(A)$},\\
\infty& \text{if $\vx\in bd(A)$}.
\end{cases}
\end{equation*}
%
\subsubsection{Barriers on the SCOP Cone}
The SCOP cone is $$Q = \left\{\vx\in\mathbb{R}^{n+1} ~\middle|~ \vx_0\ge\|{\bvx}\|\right\} = \left\{\vx\in\mathbb{R}^{n+1} ~\middle|~ x_0^2 \ge \|{\bvx}\|^2\right\}.$$ Define $b: Q\to\mathbb{R}^*$ by
%
\begin{equation*}
b(\vx)=\begin{cases}
-\ln(\vx_0^2-\|{\bvx}\|^2)& \text{if $\vx\in Int(Q)$},\\
\infty& \text{if $\vx\in bd(Q)$}.
\end{cases}
\end{equation*}

 Remember that $bd(Q)=\left\{\vx\in\mathbb{R}^{n+1} ~\middle|~ \vx_0^2=\|{\bvx}\|^2\right\}$. Clearly $b$ is a continuos function and it satisfies the first two properties of a barrier function.
%
\begin{claim}
$b$ is a convex function.
\end{claim}
\begin{proof}
Since $b$ is twice continuos we check that the Hessian of $b$ is positive semidefinite. The gradient of $b$ is
$$\Delta_\vx b(\vx)= \frac{-1}{\vx_0^2-\|{\bvx}\|^2} \left(2\vx_0,-2\vx_1,\ldots,-2\vx_n\right).$$
Therefore the Hessian of $b$ is
%
\begin{align*}
\Delta_\vx^2 b(\vx) &= \frac{2}{\left(\vx_0^2 - \|\bvx\|^2\right)^2}
\left(\begin{array}{c|c}\vx_0^2 + \|\bvx\|^2 & -2\vx_0\bvx^T \\\hline &  \\-2\vx_0\bvx & \left(\vx_0^2 - \|\bvx\|^2\right) I + 2\bvx\bvx^T \\ & \end{array}\right)\\[1ex]
%
&=  \frac{2}{\left(\vx_0^2 - \|\bvx\|^2\right)^2} \left(2\vx\vx^T + \left(\|\bvx\|^2-\vx_0^2\right)R\right)\\[1.3ex]
&\psd \vzero
\end{align*}
%
Where $$R = \left(\begin{array}{rccc}-1 & 0 & \cdots & 0 \\0 & 1 & 0 & 0 \\\vdots & 0 & \ddots & 0 \\0 & 0 & 0 & 1\end{array}\right).$$
%
\end{proof}
Therefore $b$ is a barrier function for the SCOP cone. It can be shown that the following is also a barrier function for the SCOP cone.
\begin{equation*}
b(\vx)=\begin{cases}
-\ln(\vx_0-\|{\bvx}\|)& \text{if $\vx\in Int(Q)$},\\
\infty& \text{if $\vx\in bd(Q)$}.
\end{cases}
\end{equation*}

\subsection{Kronecker Products of Matrices}
Let $A\in\mathbb{R}^{m\times n}$ and $B\in \mathbb{R}^{p\times q}$.
\begin{definition}
The {\bf  Kronecker product} of $A$ and $B$ is the matrix $A\otimes B \in \mathbb{R}^{mp\times nq}$ defined by
$$A\otimes B = \left(\begin{array}{ccc}a_{11}B & \cdots & a_{1n}B \\\vdots &  & \vdots \\a_{m1}B & \cdots & a_{mn}B\end{array}\right).$$
\end{definition}


\subsubsection{Main Properties of Kronecker Products}
Let $A$, $B$, $C$, and $D$ be matrices.
It is assumed that the dimension of the matrices is such that all the following operations are well defined. The main properties of the Kronoecker product are:
\begin{enumerate}
\item $(A\otimes B)(C\otimes D) = (AC)\otimes(BD)$.

\item Let $A_i$ denote the $i$-th column of $A$ and
define $Vec(A) := \left(\begin{smallmatrix} A_1\\ \vdots \\ A_n\end{smallmatrix}\right)$. Then $Vec(ABC) = (C^T\otimes A) Vec(B).$

\item $(A\otimes B)^{-1} = A^{-1}\otimes B^{-1}$.

\item Let $Ap=\lambda p$ and $Bq=\omega q$. Then
\begin{align*}
(A\otimes B)(p\otimes q) &= (Ap)\otimes(Bq)\\
&= (\lambda p)\otimes(\omega q)\\
&= (\lambda\omega)(p\otimes q),
\end{align*}
i.e. $\lambda\omega$ is an eigenvalue with corresponding eigenvector $p\otimes q$.

\item Let $E_{Val}(X)$ denote the set of eigenvalues of $X$. Suppose that $E_{Val}(A)=\{\lambda_1,\ldots,\lambda_n\}$ and $E_{Val}(B)=\{\omega_1,\ldots\omega_m\}$. Then $$E_{Val}(A\otimes B) = \{\lambda_i\omega_j \mid i=1,\ldots,n,~ j=1,\ldots,m\}.$$

\item If $A,B$ are symmetric then $A\otimes B$ is symmetric.

\item If $A\psd\vzero,  B\psd \vzero$ then $(A\otimes B) \psd\vzero$.
\item
Let $A,B$ be symmetric. The {\bf Kronocker sum} of $A$ and $B$ is the matrix $A\boxplus B$ given by
$A\boxplus B := \left(A\otimes I_m\right) + \left(I_n\otimes B\right)$.

Let $E_{Val}(A)=\{\lambda_1,\ldots,\lambda_n\}$ and $E_{Val}(B)=\{\omega_1,\ldots\omega_m\}$. Then $$E_{Val}(A\boxplus B) = \left\{\lambda_i+\omega_j \mid i=1,\ldots,n,~ j=1,\ldots,m\right\}.$$
\end{enumerate}


\subsection{More Barrier Functions}
\subsubsection{Barriers on the SDP Cone}
Let $S$ denote the SDP cone. Define $b:S\to\mathbb{R}^*$ by
%
\begin{equation*}
b(X)=\begin{cases}
-\sum_{i=1}^n\ln(\lambda_i)& \text{if $\vx\in Int(S)$}\\
\infty& \text{if $\vx\in bd(S)$}
\end{cases},
\end{equation*}
where $E_{Val}(X)=\{\lambda_1,\ldots,\lambda_n\}$. Clearly for $X\in Int(S)$
$$b(X) = -\sum_{i=1}^n\ln(\lambda_i)
= -\ln\left( \prod_{i=1} \lambda_i \right)
= -\ln\left(\det\left(X\right)\right).$$

The gradient of $b$ is
%
\begin{align*}
\Delta_X b(X) &= \frac{-1}{\det(X)} \frac{d\det(X)}{d(X)}\\
&= \left(\begin{array}{c} \\\frac{-1}{\det(X)} \frac{\partial\det(X)}{\partial X_{ij}} \\ \\ \end{array}\right) \\
&= \left(\begin{array}{c} \\\frac{-1}{\det(X)} (-1)^{i+j}\delta_{ij} \\ \\ \end{array}\right)\\[1ex]
&= -\left(X^T\right)^{-1}\\[1ex]
&= -2X^{-1}
\end{align*}
where $\delta_{ij}$ is the $ij$-minor of $X$.

Considering  the Hessian  as an operator on matrices we obtain that $\Delta_X^2 ( \,\cdot \,)\to 2X^{-1} ( \,\cdot \,) X^{-1}$. In terms of Kronecker products we see that $\Delta_X^2 = 2X^{-1}\otimes X^{-1}$. Cleary $X\psd\vzero$ implies that $X^{-1}\psd\vzero$. Then the previous work on Kronecker products imply that $\Delta_X^2 \psd \vzero$, thus $b$ is convex  which implies that it is a barrier function.

\section{Barrier Method for LP}
Consider the following LP problem and its dual
%
\begin{align*}
LP:&&Dual:\\
\min&~~~\vc^T\vx &max&~~~\vb^T\vy\\
s.t.&~~~A\vx=\vb &s.t.&~~~A^T\vy+\vs=\vc\\
&~~~~~~\vx\ge\vzero & &~~~\vs\ge\vzero
\end{align*}
%
and for every $\mu>0$ define
\begin{align*}
LP_\mu:&\\
\min&~~~ \vc^T\vx-\mu\sum_{i=1}^n\ln\vx_i\\
s.t.&~~~A\vx=\vb.
\end{align*}
Let $\mathcal{L}(\vx,\vy) = \vc^T\vx-\mu\sum_{i=1}^n\ln\vx_i + \vy^T(\vb-A\vx)$.
By the Lagrange multipliers method we know that
%
\begin{align*}
\Delta_\vx\mathcal{L}(\vx,\vy)&=\vc-\mu\left(1/\vx_1,\ldots,1/\vx_n\right)-\vy^TA = \vzero, \text{ and}\\
\Delta_\vy\mathcal{L}(\vx,\vy)&=(\vb-A\vx)^T = \vzero.
\end{align*}
%
Letting $\vs =  \mu\left(1/\vx_1,\ldots,1/\vx_n\right)$  we obtain
%
\begin{align*}
D_\mu:& \\
&A\vx= \vb,\\
 &A^T\vy+\vs=\vc,\\
&\vx_i\vs_i=\mu.
\end{align*}
Therefore we need to solve the previous system with relaxed complementarity (i.e $D_\mu$), and then let $\mu \to 0$. We will use Newton's method to solve $D_\mu$.

\subsubsection{Newton's Method}
Let $F:\mathbb{R}^n \to \mathbb{R}^n$ be twice differentiable. The Newton's method approximates a root of $F$, i.e. looks for an $\vx\in \mathbb{R}^n$ such that $F(\vx)=\vzero$.
%
\begin{center}
\sc{Newton's Method}
\end{center}
\begin{enumerate}
\item Let $\vx$ be your current guess.
\item We are looking for a $\delta\vx$ such that $F(\vx+\delta\vx)=\vzero$. Since $F$ is differentiable $0=F(\vx) + \Delta_\vx F(\vx)\delta\vx + \psi(\delta\vx)$, where $\lim_{\delta\to 0}\frac{\psi(\delta\vx)}{\delta\vx}=0$. For our purposes we take the linear approximation $\Delta_\vx F(\vx) \delta\vx = -F(\vx)$, therefore we solve:
$$\delta\vx = -\left(\Delta F(\vx)\right)^{-1}F(\vx).$$
\item Let $\vx := \vx+\delta\vx$ and iterate to step 2.
\end{enumerate}

\subsubsection{Newton's Method Applied to LP}
When applying the N-M to the system $D_\mu$ the $(\vx,\vy,\vs)$ is unknown and at any iteration we have an approximation to it. Therefore:
%
\begin{align*}
A(\vx+\delta\vx)&= \vb\\
A^T(\vy+\delta\vy)+(\vs+\delta \vs) &=\vc\\
(\vx_i+\delta\vx_i)(\vs_i+\delta\vs_i)&=\mu
\end{align*}
%
which gives
%
\begin{align*}
A\delta\vx &= \vb-A\vx\\
A^T\delta\vy + \delta\vs &= \vc-A^T\vy-\vs\\
\vx_i\delta\vs_i + \vs_i\delta\vx_i + \delta\vx_i\delta\vs_i &= \mu-\vx_i\vs_i
\end{align*}
%
Using  N-M we drop the term $\delta\vx_i\delta\vs_i$ from the last equality to get an equality linear in the $\delta$'s. Ana so we get:
$$\left(\begin{array}{ccc}A & 0 & 0 \\0 & A^T & I \\S & 0 & X\end{array}\right) \left(\begin{array}{c}\delta\vx \\\delta\vy \\\delta\vs\end{array}\right) = \left(\begin{array}{c}r_p \\r_d \\r_c\end{array}\right)$$ where the $r$'s are th right hand side of the previous system, $S = \left(\begin{smallmatrix}\vs_1 & 0 & 0 \\0 & \ddots& 0 \\0 & 0 & \vs_n\end{smallmatrix}\right)$, and $X=\left(\begin{smallmatrix}\vx_1 & 0 & 0 \\0 & \ddots & 0 \\0 & 0 & \vx_n\end{smallmatrix}\right)$.
From this we obtain the following:
\begin{align*}
\delta\vy &= (AS^{-1}XA^T)^{-1}\left[r_p+AS^{-1}(\vx r_d-r_c)\right]\\
\delta\vs &= r_d - A^T\delta\vy\\
\delta\vx &=  -S^{-1}(X\delta S-r_c)
\end{align*}
where $AS^{-1}XA^T \psd \vzero$.









\end{document}