A numerical implementation of an interior point method for semidefinite programming

A numerical implementation of an interior point method for semidefinite programming

Djamel Benterki^I; Jean-Pierre CrouzeixII, ^*; Bachir Merikhi^III

^IDépartement de Mathématiques Faculté des Sciences Université Ferhat Abbas Sétif – Algérie

IILIMOS – Université Blaise Pascal Aubière Cedex – France. [e-mail]

^IIIDépartement de Mathématiques Faculté des Sciences Université Ferhat Abbas Sétif – Algérie

ABSTRACT

This paper is concerned with an algorithm proposed by Alizadeh for linear semidefinite programming. The proof of convergence given by Alizadeh relies on a wrong inequality, we correct the proof. At each step, the algorithm uses a line search. To be efficient, such a line search needs the value of the derivative, we provide this value. Finally, a few numerical examples are treated.

Keywords: semidefinite programming; interior point methods.

RESUMO

Este artigo considera um algoritmo proposto por Alizadeh para programação semidefinida linear. A prova de convergência apresentada por Alizadeh baseia-se numa inequação errada, corrigimos a demonstração. Em cada passo, o algoritmo utiliza uma busca linear. Para ser eficiente, esta busca linear precisa do valor da derivada, apresentamos este valor. Finalmente, alguns exemplos numéricos são tratados.

Palavras-chave: programação semidefinida, métodos de pontos interiores.

1. Introduction

The object of this paper is to present an implementation of an interior point method. Alizadeh (1995) has proposed an attractive extension of an interior point algorithm by Ye (1991) for linear programming to linear semidefinite programming. Actually, the proof of convergence given by Alizadeh relies on a wrong inequality, we provide a counterexample. We amend the proof to make it correct. At step k, the algorithm makes use of a line search, Goldstein-Price's methods are efficient to perform such a line search, but they need the knowledge of the derivative. We show how to obtain it, allowing an efficient implementation of the algorithm. Finally, we treat some numerical examples for solving the so-called linear semidefinite program:

where b Î R^m, C and A_i, belong to E, the linear space of n x n symmetric matrices. The inner product on E of two matrices A and B is the trace of their product, i.e., áA, Bñ = tr(AB) = S_ijA_ij B_ij Finally, K denotes the cone of positive semidefinite matrices of E and the interior of K denoted by int(K) is the cone of positive definite matrices of E.

Max-cut problems in a graph, graph bisection problems, the problem of finding a largest clique in a graph, min-max eigenvalue problems can be formulated as linear semidefinite programs, see for instance Helmberg et al. (1996).

Linear semidefinite programming presents a great similitarity with linear programming. The objective function is linear, the constraint functions are affine, the difference consists of the positivity cone. Testing the positive semidefinitess of a n x n matrix X requires to check áXh, hñ > 0 for all h with norm 1, an infinity of linear constraints, while testing that a vector belongs to the nonnegative orthant in an euclidean space involves, by definition, a finite number of linear constraints. The duality scheme in linear semidefinite programming is about the same than the one in linear programming; there are also complementarity slackness conditions. It turns out that interior point methods used in linear programming can be easily extended to linear semidefinite programming.

Next, we precise the notation used in the paper. We have already defined the sets E, K,int(K) and the scalar product á , ñ in E. The identity matrix of E is denoted by I. Given A Î E its Frobenius norm is

where l_i = i = 1,2,...,n are the eigenvalues of A. Besides this norm, we will also use the spectral norm on E:||A||_{s =} max_i(|l_i|). Given a rectangular m x n matrix B, vecB is the vector of Â^mn defined by (vecB) _i+(j-1)m= B_ijfor i = 1,2...,m, j = 1,2,...,n. For instance the set of relations áA_i , Xñ = b_i for i = 1,..,m may be written as vec X = b where is a m x n² matrix, that is row i of is vec^tA_i. F Ä G is the Kronecker product of matrices: if F is a m X n matrix and G a p x q matrix then F Ä G is the m x n block matrix whose the (i,j) block is the p x q matrix F_ijG.

Let u, v Î Â^r we denote by u^tv their inner product and by ||u||₂, ||u||_¥ the euclidean and the max-norm of u, respectively. Given u Î Â^r,Diag(u) is the r x r diagonal matrix with the components of u as the diagonal entries. Conversely, given X an r x r matrix (not necessarily diagonal), diag(X) is the vector in  ^r obtained from the diagonal elements of X.

2. Duality in SDP

Let us consider the problems:

and

X is said to be a strictly feasible solution of (SDP) if X Î int(K)and áA_i, X ñ = b_ifor i = 1...m. y Î Â ^m is said to be a strictly feasible solution of (DSDP) if (C - y_iA_i) Î int(K). A weak duality result says that m_d< m_p. A strong duality result says that if both (SDP) and (DSDP) have strictly feasible solutions, then m_d = m_p and both problems have optimal solutions. Furthermore, for such optimal solutions X and y, the following complementarity slackness condition holds

More information on semidefinite programming can be found in Alizadeh (1995), Helmberg et al. (1996) and Vanderbeghe & Boyd (1996).

3. A potential reduction method for (SDP)

The method proposed by Alizadeh (1995) is a direct extension of a method given by Ye (1991) for linear programming. It is based on the reduction of a potential function and a projective transform. Let us consider the (SDP) program and his dual (DSDP). Given z < m_p and a fixed integer r > 0, the algorithm uses the two following functions:

and

It is clear that f is well defined for all X strictly feasible solution of (SDP), while y is well defined for all X positive definite and y strictly feasible solution of (DSDP).

The function f (the function y) is called the primal (the primal-dual) potential function. The convergence of the algorithm is obtained from the decrease of the sequence {y (X_k, y_k)}

This method needs the knowledge of initial strictly feasible solutions X₀ and y₀.

3.1 The projective transform

At step k, the current X_k Î int(K). The Cholesky factorization of X_k gives a lower triangular matrix L_k with positive diagonal entries such that L_kL^t_k = X_k. Then, we consider the projective transform

where

Here e_ris the vector of ones in Â^r Notice that X = (, ). Via this projective transform, (SDP) is equivalent to the problem

where

Remark: The potential function associated to (TSDP) is

It is easily proved (see Alizadeh, 1995) that

4. The algorithm iteration

In a similar way than in the Karmarkar's method for classical linear programming, we relax the problem (TSDP) in the problem

It is easily seen that, for b Î ]0,1[, the constraint || - I||² + || - e_r||< b₂ implies that Î int(k)and > 0. Using a matricial formulation, we have

where

is a (m + 1)x (n² + r) matrix.

It results, from a classical optimality condition, that the optimal solution (, ) of (R) is given by

where p(z_k) is the projection of the vector on the affine subspace corresponding to the equality, i.e.,

where P_n(z_k) is an n x n matrix and p_ris a vector in Â^r.

Let

Then y(z) is a solution for the linear system

This system can be efficiently solved thanks to a Cholesky factorization.

Remark: b may stay fixed during the different iterations, but it is better to adjust b at each iteration in order to accelerate the speed of convergence. This can be done thanks to a Goldstein-Price line search. For that, it is necessary to know the derivative of the potential function at 0. This is the object of the following lemma.

Lemma 1. Let P Î E and

Then

Proof. We have

Set

H(b) = det(X - b LPL^t) = det(LL^t - b LPL^t) = det(X) x det(I - bP).

P being symmetric, there exist an orthogonal matrix U and a diagonal matrix D such that P = U^t DU. Hence,

Then

Finally,

n

4.1 Convergence of the algorithm

The proof of convergence given by Alizadeh for his algorithm is based on the following inequality.

Actually, this inequality is false as shown by the following counter-example:

In fact, we have:

Lemma 2. Let Î K and Î Â^r.Then > 0

Proof. Reporting to Alizadeh (1995), the following inequality holds

But

and

Hence

Finally (1) gives

If we replace in the proof of Alizadeh d(b) = ab by d(b) = ab , then his proof becomes valid. In particular, we have

and

where

Furthermore, Alizadeh proves the polynomial convergence:

after a number of iteration of order 0 ( |log e|) for e > 0.

n

5. Implementation of the Alizadeh algorithm

In the algorithm, a Î [0,1] and e > 0 are fixed.

The Alizadeh algorithm

a) Initialization: Start with X₀,y₀, a Î [0,1], z₀ = b^ty0 and k=0,

b) If (áC, X_kñ - b^ty_k) < e STOP: (X_k, y_k) is an e – approximate solution

If not:

– Compute p (z_k).

– If ||p(z_k)||₂> a then

* Find b^*= Argmin _{0 <b<1} (Y (X_k - bL_kP_n(z_k) L^tk,y (z_k))).

* Take = - b^*

* Take X_k+1 = (, )and go to c).

– If not:

* Find z^* = Argmin_z<zk (Y(X_k,y(z))).

* Take y_k+1 = y(z^*), z_{k+1 =} b ^ty(z^*)and go to c).

c) Do k = k + 1 and go to back b).

6. Numerical examples

In the treated examples, the initial primal-dual feasible solutions have been given .

6.1 Example 1

where:

and b = (0,1)^t.

After 4 iterations we obtain the optimal dual solution y^* = (1,0)^tand the optimal primal solution

Example 2

In this example n=2m.

where C = - I. For k = 1,...,m, b_k = 2 and the n x n matrices A_k are defined by

with k+m+1 < n.

The following array gives the number of iterations needed to obtain the primal and the dual solutions for different sizes.

7. Conclusion

The present paper corrects a small point in the proof of the convergence given by Alizadeh (1995) and gives the value of the derivative of the potential function at 0 needed to perform an efficient line search.

Recebido em 02/2002, aceito em 11/2002 após 1 revisão

• (1) Alizadeh, F. (1995). Interior point methods in semidefinite programming with applications to combinatorial optimization. SIAM Journal on Optimization, 5, 13-51.
• (2) Helmberg, C.; Rendl, F.; Vanderbei, R.J. & Wolkowicz, H. (1996). An interior point method for semidefinite programming. SIAM Journal on Optimization, 6, 342-361.
• (3) Jansen, B.; Roos, C. & Terlaky, T. (1997). A family of polynomial affine scaling algorithms for semidefinite linear complementarity problems. SIAM Journal on Optimization, 7, 126-140.
• (4) Keraghel, A. (1989). Etude adaptative et comparative des principales variantes dans l'algorithme de Karmarkar. Dissertation thesis, Universitè Joseph Fourier, Grenoble, France.
• (5) Kojima, M.; Shindoh, S. & Hara, S. (1997). Interior point methods for the monotone semidefinite linear complementarity problem in symmetric matrices. SIAM Journal on Optimization, 7, 86-125.
• (6) Marcus, M. & Ming, H. (1964). A survey of matrix theory and matrixinequalities University of California, Santa Barbara, Allyn and Bacon, ING, Boston.
• (7) Monteiro, R.D.C. (1997). Primal-dual path following algorithms for semidefinite programming. SIAM Journal on Optimization, 7, 663-678.
• (8) Overton, M. & Wolkowicz, H. (1997). Semidefinite programming. Mathematical Programming, Series B, 77, 105-109.
• (9) Rama, M.V.; Tuncel, L. & Wolkowicz, H. (1997). Strong duality for semidefinite programming. SIAM Journal on Optimization, 7, 641-662.
• (10) Vanderbeghe, L. & Boyd, S. (1996). Positive definite programming. SIAM Review, 38, 49-95.
• (11) Wright, S.J. (1997). Primal-dual interior point methods SIAM, Philadelphia, PA.
• (12) Ye, Y. (1991). An O(n³L)- potential reduction algorithm for linear programming. Mathematical Programming, 50, 239-258.

*

Corresponding author / autor para quem as correspondências devem ser encaminhadas

Publication Dates

• Publication in this collection
  26 May 2003
• Date of issue
  Jan 2003

History

• Accepted
  Nov 2002
• Received
  Feb 2002