A General Nonconvex Multiduality Principle

We introduce a (possibly infinite) collection of mutually dual nonconvex optimization problems, which share a common optimal value, and give a characterization of their global optimal solutions. As immediate consequences of our general multiduality principle, we obtain Toland–Singer duality theorem as well as an analogous result involving generalized perspective functions. Based on our duality theory, we propose an extension of an existing algorithm for the minimization of d.c. functions, which exploits Toland–Singer duality, to a more general class of nonconvex optimization problems.


Introduction
One of the main tools in optimization is duality theory, which associates to a given (primal) problem (P) another (dual) problem (D), in such a way that the relation between the two problems provides useful information about (P). In the case of convex optimization problems, under suitable assumptions, the optimal values of (P) and (D) coincide and the primal optimal solutions can be recovered from the dual optimal solutions; this is particularly useful when (D) happens to be easier to solve than (P). The essential tools of convex duality theory are convex conjugation and the notion of subgradient; we refer to the classical book by Rockafellar [1] for a detailed treatment of classical convex duality. In the last decades, there has also been a very active research in nonconvex programming, motivated by the fact that many real life optimization problems are nonconvex. In this paper, we propose a generalization of one of the most classical nonconvex duality theories, namely, Toland-Singer duality for d.c. (difference of convex) functions [2][3][4], consisting in a multiduality principle involving a (possibly infinite) collection of mutually dual problems. Our simple multiduality result does not require any convexity assumption, and one of its consequences is a new duality theorem involving just two mutually dual problems, expressed in terms of classical Fenchel conjugates. This theorem generalizes the well-known Toland-Singer duality theorem, and we use it to characterize (approximate) global optimal solutions, thus generalizing a well-known necessary and sufficient global optimality condition due to Hiriart-Urruty [5]. Another application of the new theorem yields a duality result involving the generalized perspective functions introduced by Maréchal in [6].
There are some numerical algorithms based on Toland-Singer duality, such as the DC algorithm [7] and branch-and-bound/cutting-plane type algorithms [8,9]. Following the approach by Tao and El Bernoussi [7], we propose an algorithm for searching local optimal solutions of nonconvex problems having the format considered in our duality theorem.
The paper consists of three more sections. In Sect. 2, we state the new multiduality principle, which is naturally formulated in the framework of generalized conjugation theory. Section 3 considers the special case when the collection of mutually dual problems consists of just two problems; for simplicity, this section is presented in the setting of classical convex conjugation. In Sect. 4, we propose an extension of the DC algorithm to the broader class of nonconvex problems considered in Sect. 3.

Multiduality
Let (X i ) i∈I be a family of nonempty sets, indexed by I = ∅. We denote X := i∈I X i and X −i := j∈I \{i} X j for i ∈ I. For y ∈ X −i and z ∈ X i , we define x (−i,y),(i,z) ∈ X by Given a function c : X → R, for each i ∈ I we consider the coupling function Notice that, for x ∈ X, one has c i (x −i , x i ) = c(x); here and throughout we denote by x −i the projection of x ∈ X onto X −i , that is, Following the generalized conjugation scheme of [10], we define a new coupling function c i : and consider the conjugation operators associated with c i and c i , namely the c iconjugate of g : and, analogously, the c i -conjugate of h : here and in the sequel we adopt the conventions For h : X i → R, z ∈ h −1 (R) and ≥ 0, we set One can easily check that Given f : X → R, for i ∈ I and z ∈ X i we define f i,z : Notice that, for x ∈ X, one has f i, Hence, the optimal value of problem does not depend on i. Then Proof Apply Theorem 2.1 with f := δ G .

Corollary 2.2 Let G i ⊆ X i for every i ∈ I. Then
Proof Apply Corollary 2.1 with G := i∈I X i .
Then, for every i ∈ I, one has

Theorem 2.2
Assume that the optimal value of (P i ) is finite, and let , η ≥ 0. Ifz is and j ∈ I \{i}, the pointȳ j is an ( + η)-optimal solution to P j .
Proof We have the last equality being an immediate consequence of Theorem 2.1.

A Generalization of Toland-Singer Duality
Let (X 1 , X 2 , ·, · ) be a dual pair of locally convex spaces and f : In this section, we will apply the general results of Sect. 2 to the special case when I := {1, 2} and c is the duality pairing ·, · . We recall that the Fenchel conjugates of g : X 2 → R and h : X 1 → R are the functions g * : X 1 → R and h * : X 2 → R, respectively, defined by It is easy to see that, for the coupling function c considered in this section, one has g c 1 = g * and h c 2 = h * . Thus, Theorem 2.1 and Corollaries 2.1 and 2.2 yield: Proof Apply Corollary 2.1 with G := Graph T.
Similarly, from Corollary 2.3 or, alternatively, Corollary 3.1 (by setting f (z, y) := h (z) + g (y)), one obtains the following version of the classical Toland-Singer duality theorem [2,4]: From the preceding corollary, by taking g := k * , with k : X 1 → R such that k * * = k, one immediately obtains the standard Toland-Singer formula. We recall that, for η ≥ 0, the η-subdifferential of h : From Theorem 2.2, one gets the following result on approximate optimal solutions of the pair of dual problems Corollary 3.5 Assume that the optimal value of (P) is finite, and let , η ≥ 0. Ifz is an -optimal solution to (P) such that f (z, ·) is convex and l.s.c. then, for everȳ y ∈ ∂ η f (z, ·) * (z), the pointȳ is an ( + η)-optimal solution to (D).
The following characterization of approximate global optimal solutions generalizes a well-known necessary and sufficient global optimality condition due to Hiriart-Urruty [5].
Theorem 3.1 Assume that the optimal value of (P) is finite, and letz ∈ X 1 and ≥ 0. Thenz is an -optimal solution to (P) if and only if for every η ≥ 0 and every Proof Letz be an -optimal solution to (P), and η ≥ 0 and y ∈ f (z, ·) −1 (R) be such thatz ∈ ∂ η f (z, ·)(y). Using Corollary 3.1, we obtain which proves (2).
To apply the duality theory presented in this section to a given optimization problem, one has to be able to recognize whether the objective function of the problem under consideration can be written in the form f (z, ·) * (z) for some function f : X 1 × X 2 → R. The following theorem provides a useful criterion to make this recognition possible. Theorem 3.2 For every f : X 1 × X 2 → R, the function ϕ : X 2 1 → R defined by ϕ(z, z ) := f (z, ·) * z is convex, proper (accepting as proper the constant functions +∞ and −∞) and l.s.c. in its second argument. Conversely, for every function ϕ : X 2 1 → R with these properties there exists f : Proof The first part of the statement is an immediate consequence of well-known properties of conjugate functions. To prove the converse, define f : X 1 × X 2 → R by f (z, y) := ϕ (z, ·) * (y) ; then, since ϕ (z, ·) is convex, proper and l.s.c, from the equality f (z, ·) := ϕ (z, ·) * it follows that ϕ (z, ·) := f (z, ·) * , and therefore ϕ (z, z) := f (z, ·) * (z).
According to the preceding theorem, the class of problems to which our duality theory applies coincides with that consisting of problems with format the function ϕ : X 2 1 → R being as in the statement. Since, in view of the proof, the objective function of these two problems are linked by the relation f (z, ·) := ϕ (z, ·) * , a straightforward computation yields the dual objective function in terms of ϕ : Hence, if the primal problem is stated as (3), the formulation of the dual problem is To illustrate that solving the dual problem may be advantageous, we present the following academic example, in which the primal problem consists in maximizing a nonconcave function and, on the contrary, the dual objective function is concave.

Example 3.1 Let us consider the primal problem
where α is a nonconcave function defined on a normed space X 1 . Let us define ϕ : here B is the closed unit ball in X 1 and N > 0. Clearly, ϕ(z, z) = α(z), and therefore (P) can be rewritten as (3). Since according to (4) we get the following expression for the dual objective function: which, in view of [11, p. 200], shows that it is the smallest N -Lipschitz majorant of α. This N -Lipschitz envelope is concave in some cases; for instance, let us consider the case when X 1 := R, α(z) := −2z 6 + 15z 4 − 36z 2 and N := 10. Then one can easily prove that with r being the smallest positive real number satisfying α (−r ) = 10. Figure 1 depicts the graphs of both α and y → f (·, y) * (y) and shows that the latter function is concave.

Example: Generalized Perspective Functions
Let g : X 2 → ]0, +∞[ and h : X 1 → ]0, +∞[ , and define f : . For this particular function f, the objective functions of problems (P) and (D) can be expressed by means of the operation introduced in [6], which constitutes a generalization of the so called perspective function of Convex Analysis. Indeed, for z ∈ X 1 one has f (z, ·) = h(z)g, and hence Similarly, for y ∈ X 2 one has f (·, y) * (y) = (g(y)h) * (y) = g(y)h * y g(y) = h * g (y, y) .

An Algorithm
In the setting of the preceding section, in this one we will assume that X 1 and X 2 are the Euclidean space R n and ·, · is the standard Euclidean product. Let f : R n ×R n → R, and consider the sets According to Theorem 3.1, the set P l contains all the optimal solutions to (P) and the set D l contains all the optimal solutions to (D).
Forz ∈ R n andȳ ∈ R n , we consider the auxiliary problems and We will denote by O S(z) and O T (ȳ) the sets of optimal solutions to problems S(z) and T (ȳ), respectively.
This algorithm requires the auxiliary problems occurring at each iteration to be solvable, and from now on we will assume that this is the case. The following proposition states sufficient conditions for solvability of these problems. We recall that a function is called co-finite if its conjugate is finite valued everywhere. Theorem 4.1 Let f : R n × R n → R ∪ {+∞} and let the sequences z k and y k be constructed according to (8). Then f z k+1 , · * z k+1 ≥ f ·, y k * y k ≥ f z k , · * z k .
The first inequality holds with the equal sign if and only if z k+1 ∈ ∂ f (z k+1 , ·)(y k ), in which case, assuming that the function f (·, y k ) is convex, proper and l.s.c., we have y k ∈ D l . The second inequality holds with the equal sign if and only if y k ∈ ∂ f (·, y k )(z k ), in which case, assuming that the function f (z k , ·) is convex, proper and l.s.c., we have z k ∈ P l .
The 'if and only if" assertions follow from the above chain of inequalities, combined with the well-known characterization of subgradients as those elements that satisfy the Fenchel inequality with the equal sign. Let us assume that the first inequality holds with the equal sign, and let z ∈ R n be such that y k ∈ ∂ f (·, y k )(z). Since we have z k+1 ∈ O T (y k ) and z k+1 ∈ ∂ f (z k+1 , ·)(y k ), using that f (·, y k ) is convex, proper and l.s.c., we obtain the chain of inequalities at the beginning of the proof of Theorem 4.1, the continuity