On Minimal Subspaces in Tensor Representations

In this paper we introduce and develop the notion of minimal subspaces in the framework of algebraic and topological tensor product spaces. This mathematical structure arises in a natural way in the study of tensor representations. We use minimal subspaces to prove the existence of a best approximation, for any element in a Banach tensor space, by means of a tensor given in a typical representation format (Tucker, hierarchical, or tensor train). We show that this result holds in a tensor Banach space with a norm stronger than the injective norm and in an intersection of finitely many Banach tensor spaces satisfying some additional conditions. Examples using topological tensor products of standard Sobolev spaces are given.


Introduction
Recently, there has been an increased interest in numerical methods which make use of tensors. In particular, for high spatial dimensions one must take care that the numerical cost (in time and storage) is linear in the space dimension and does not increase exponentially. For three spatial dimensions, these methods can be applied with great success.
A first family of applications using tensor decompositions concerns the extraction of information from complex data. It has been used in many areas such as psychometrics [6,26], chemometrics [2], analysis of turbulent flows [3], image analysis and pattern recognition [28], and data mining. Another family of applications concerns the compression of complex data (for storage or transmission), also introduced in many areas such as signal processing [19] or computer vision [30]. A survey of tensor decompositions in multilinear algebra and an overview of possible applications can be found in the review paper [18]. In these applications, the aim is to compress the information as much as possible or to extract a few modes representing some features to be analysed. The use of tensor product approximations is also of growing interest in numerical analysis for the solution of problems defined in high-dimensional tensor spaces, such as partial differential equations (PDEs) arising in stochastic calculus [1,5,11] (e.g., the Fokker-Planck equation), stochastic parametric PDEs arising in uncertainty quantification with spectral approaches [9,22,23], and quantum chemistry (cf., e.g., [29]). For details, we refer to [14].
Let d vector spaces V j be given (assume, e.g., that V j = R n j ). The generated tensor space is denoted by V = a d j =1 V j , where a d j =1 V j = span{ d j =1 v j : v j ∈ V j and 1 ≤ j ≤ d } (assume, e.g., that represents the Kronecker product). A typical representation format is the tensor subspace or Tucker format where I = I 1 × · · · × I d is a multi-index set with I j = {1, . . . , r j }, r j ≤ dim(V j ), b (j ) i j ∈ V j (i j ∈ I j ) are basis vectors, and a i ∈ R. Here, i j are the components of i = (i 1 , . . . , i d ). The data size is determined by the numbers r j collected in the tuple r := (r 1 , . . . , r d ). The set of all tensors representable by (1.1) with fixed r is there are subspaces U j ⊂ V j such that dim(U j ) = r j and v ∈ U := a d j =1 U j . (1.2) Here, it is important that the description (1.1) with the vectors b (j ) i can be replaced by the generated subspace U j = span{b (j ) i : i ∈ I j }. Note that T r is neither a subspace of V nor a convex set.
A question about minimal subspaces arises naturally from (1.2): Given a tensor v ∈ V, what are the subspaces U j ⊂ V j with minimal dimension r j such that v ∈ d j =1 U j ?
Another natural question is the approximation of some v ∈ V by u ∈ T r for a fixed r: Find u best ∈ T r such that v − u best equals inf v − u : u ∈ T r (1.3) for a suitable norm. In the finite-dimensional case, compactness arguments show the existence of a best approximation. In this paper we discuss this question in the infinite-dimensional case (i.e., dim(V j ) = ∞, while still dim(U j ) = r j < ∞).
Here, one should note that tensors have properties which are unexpected compared with matrix theory. For instance, one can define another tensor format (r-term or canonical format) as follows. Fix an integer r ∈ N 0 and set For d = 2, R r corresponds to matrices of rank ≤ r. Seeking a solution of inf{ v−u : u ∈ R r }, one finds examples of v ∈ V even for finite-dimensional V, but d ≥ 3, such that there is no minimiser u best ∈ R r (cf. [7]).
There are other formats with even better properties than (1.2) (cf. [15,24]), which are again related to subspaces. In these cases, further subspaces like, e.g., U 12 ⊂ U 1 ⊗ U 2 appear. The representation using the hierarchical format from [15] uses subspaces U 12 with dimension not exceeding a given bound. For these formats, the results of this paper also apply, e.g., they ensure the existence of best approximations.
There are practical reasons for the interest in the existence of a best approximation. Truncation of a tensor v to a certain format tries to minimise v − u . If a best approximation does not exist, one has to expect a numerical instability as v − u approaches the infimum. Even if V is finite dimensional, it is often a discrete version of a function space. If the infinite-dimensional function space allows a best approximation, one can expect uniform stability (i.e., independent of the discretisation parameters).
The hierarchical format of [15] is connected with a certain dimension partition tree. In particular, the approach in [24] using a linear tree corresponding to the matrix product systems is applied in quantum chemistry (cf. [29]). There are approaches using a general graph structure (cf. [17]); however, as soon as loops are contained in a graph, the parameters of its representation cannot be described by dimensions of certain subspaces, and the results of this paper do not apply.
In the sequel, we define minimal subspaces U min j (v) for algebraic tensors v ∈ a d j =1 V j (cf. Theorem 2.17) as well as for topological tensors v ∈ · d j =1 V j (cf. Definition 3.11). The main result is given in Theorem 3.15, where we show that for weakly convergent sequences v n v (see Definition 3.12), the dimension of the limiting minimal subspace is bounded by This is the key property which allows us to derive the desired properties.
Finally, we discuss the nature of the closed subspace · In the algebraic case, we have by definition that v ∈ a d j =1 U min j (v). This property does not seem obvious for a general topological tensor v ∈ · d j =1 V j , but we give sufficient conditions for this property. In particular, it holds for Hilbert tensor spaces.
The paper is organised as follows. In Sect. 2, we introduce the concept of minimal subspaces of an algebraic tensor and describe a characterisation. In Sect. 3, minimal subspaces are defined and characterised for Banach tensor spaces. Finally, Sect. 4 is devoted to the proof of the existence of best approximation tensors in T r in a Banach tensor space.

Minimal Subspaces in an Algebraic Tensor Space
In the following, X is a Banach space with norm · = · X . While X denotes the algebraic dual, X * is the dual space of functionals with bounded dual norm · * = · X * : (2.1) This implies that we recover the · X norm from the dual norm via By L(X, Y ) we denote the space of continuous linear mapping from X into Y . The corresponding operator norm is written as · Y ←X . L(X, Y ) is a subspace of the space L(X, Y ) of all linear mappings (without topology).
The following result is known as the Lemma of Auerbach and is proved, e.g., in Meise-Vogt [21, Lemma 10.5].

Lemma 2.2
For any n-dimensional subspace of a Banach space X, there exists a basis {x ν : 1 ≤ ν ≤ n} and a corresponding dual basis

Definitions and Elementary Facts
Concerning the definition of the algebraic tensor space a d j =1 V j generated from vector spaces V j (1 ≤ j ≤ d), we refer to Greub [12]. As the underlying field we choose R, but the results hold also for C. The suffix 'a' in a d j =1 V j refers to the 'algebraic' nature. By definition, all elements of Then The following well-known result is formulated for d = 2.

Lemma 2.4 For any tensor
Proof Take any representation v = n i=1 v i ⊗ w i . If, e.g., the {v i : 1 ≤ i ≤ n} are not linearly independent, one v i can be expressed by the others. Without loss of generality shows that x possesses a representation with only n − 1 terms: Since each reduction step decreases the number of terms by one, this process terminates; i.e., we obtain a representation with linearly independent v i and w i .
In accordance with the usual matrix rank we introduce the following definition.

Definition 2.5
The number r appearing in Lemma 2.4 will be called the rank of the tensor v and denoted by rank(v).
The following notation and definitions will be useful. We recall that L(V , W ) is the space of linear maps from V into W , while V = L(V , R) is the algebraic dual. For metric spaces, L(V , W ) denotes the continuous linear maps, while V * is the topological dual.
Let I := {1, . . . , d} be the index set of the 'spatial directions'. In the sequel, the index sets I\{j } will appear. Here, we use the abbreviations (2.6) Note that (2.6) extends uniquely to a linear mapping A : V → W.

Remark 2.6
(a) Let V := a d j =1 V j and W := a d j =1 W j . Then the linear combinations of tensor products of linear mappings A = d j =1 A j defined by means of (2.6) form a subspace of L(V, W): Often, mappings A = d j =1 A j will appear, where most of the A k are the identity (and therefore V k = W k ). If A j ∈ L(V j , W j ) for one j , we use the following notation: and since id •A j = A j •id, the following identity 1 holds for j = k: (in the first line we assume j < k). Proceeding inductively with this argument over all indices, we obtain .

Matricisation
Definition 2.7 For j ∈ I = {1, . . . , d}, the map M j is defined as the isomorphism In the finite-dimensional case of V k = R n k , the tensor space V j ⊗ a V [j ] of order 2 may be considered as a matrix from R n j ×n [j ] , where n [j ] = k =j n k . Then, M j maps a tensor entry v[i 1 , . . . , i j , . . . , i d ] into the matrix entry (M j (v))[i j , (i 1 , . . . , i j −1 , i j +1 , . . . , i d )]. As long as we do not consider matrix properties which depend on the ordering of the index set, we need not introduce an ordering of the (d −1)-tuple (i 1 , . . . , i j −1 , i j +1 , . . . , i d ).
The lexicographical ordering of (i, k) leads to the matrix Next, we restrict the considerations to finite-dimensional V k . Since tensor products of two vectors can be interpreted as matrices, the mapping M j is named 'matricisation' (or 'unfolding'). The interpretation of tensors v as matrices enables us to transfer the matrix terminology to v. In particular, we may define the rank of M j (v) as a property of v. (2.8) Hitchcock [16, p. 170] (1927) introduced rank j (v) as 'the rank on the j th index'. For infinite-dimensional vector spaces V j , the generalisation is given by rank j (v) := dim U min j (v), where the minimal subspaces U min j (v) will be defined in Sect. 2.2. The next result extends Lemma 2.4.
are linearly independent in a k =j V k .
Proof Consider, without loss of generality, the case j = 1. If the tensors {v [1] ν : 1 ≤ ν ≤ r} are linearly dependent, we also may assume, without loss of generality, that v [1] r may be expressed as v [1] [1] ν implies that rank(v) < r in contradiction to the minimality of r.

Case d = 2
The matrix case d = 2 will serve as the start of an induction. To ensure the existence of minimal subspaces U 1 , U 2 with v ∈ U 1 ⊗ U 2 , we need a lattice structure, which is the subject of the next lemma.

Lemma 2.11
Assume that X i and Y i are subspaces of V i for i = 1, 2. Then By assumption, v has the two representations to the first representation yields (id 1 1 , while the second representation leads to n y ν=1 ξ (2) For each v, we introduce the family F (v) as the set of pairs Hereby, the existence and uniqueness of minimal subspaces U min j (v) are guaranteed.
Then these vectors span the minimal spaces: Proof Apply the proof of Lemma 2.11 to X 1 = span{v Since a strict inclusion is excluded, X j = U min j (v) proves the assertion.

Corollary 2.15
The following statements hold.
Proof For statement (a) consider the representation of v by (2.4) with bases Then holds, proving the assertion.

Definition in the General Case
In the following, we assume that d ≥ 3, and generalise some of the features of tensors of second order. By Lemma 2.3, we may assume v ∈ U := a d j =1 U j with finite-dimensional subspaces U j ⊂ V j . The lattice structure from Lemma 2.11 generalises to higher order.

Lemma 2.16 Assume that
Proof For the start of the induction at d = 2 use Lemma 2.11. Assume that the assertion holds for d − 1 and write a d j =1 X j as X 1 ⊗ a X [1] with X [1] [1] ). By the induction hypothesis, To this end, we introduce the following two subspaces (recall (2.7e)): In the case of a normed space V j , we may consider the subspace

Theorem 2.17 For any
, whose algebraic characterisation is given by Proof Since the mappings id j ⊗ ϕ [j ] is a finite-dimensional subspace of the normed space a k =j V k , by the Hahn-Banach theorem, the algebraic functional ϕ [j ] can be extended to a k =j V k such that ϕ [ We remark that for d ≥ 3, in general, the dimensions of U min j (v) may be different.

Hierarchies of Minimal Subspaces
We have introduced the minimal subspace U min j (v) ⊂ V j for a singleton {j } ⊂ D := {1, 2, . . . , d}. Instead we may consider general disjoint and non-empty subsets of , 4}, and α 3 = {5, 6, 7}. Then we can conclude that there are minimal sub- The relation between U min j (v) and U min α ν (v) is as follows.
An obvious generalisation of the previous results is given below.
The algebraic characterisation of U min α (v) is analogous to that given in Theorem 2.17. (k) . The analogues of (2.10c), (2.10d) apply as soon as norms are defined on V j and a j ∈α c V j .

Minimal Subspaces in a Banach Tensor Space
In this section we assume the existence of a norm, namely · , defined on a tensor space V. More precisely, we introduce the following class of Banach spaces. Definition 3. 1 We say that V · is a Banach tensor space if there exist an algebraic tensor space V and a norm · on V such that V · is the completion of V with respect to a given norm · , i.e., If V · is a Hilbert space, we will say that V · is a Hilbert tensor space.
Next, we give some examples of Banach and Hilbert tensor spaces.
is a Banach tensor space. Examples of Hilbert tensor spaces are We recall that for the set of norms over a given vector space V , we can define a partial ordering Given a vector space V , its completion with respect to a norm · yields a Banach space which we denote by V · := V · . Note that · 1 · 2 implies that V · 2 ⊂ V · 1 .

Tensor Product of Banach Spaces
Let · j , 1 ≤ j ≤ d, be the norms of the vector spaces V j appearing in V = a d j =1 V j . By · we denote the norm on the tensor space V. Note that · is not determined by · j , but there are relations which are 'reasonable'. Any is called a cross norm. As usual, the dual norm to · is denoted by · * . If · is a cross norm and · * is also a cross norm on a

3)
· is called a reasonable cross norm.

Remark 3.3 Equation (3.2) implies the inequality
j , which is equivalent to the continuity of the tensor product mapping given by ⊗ ((v 1 , . . . , v d ) By standard arguments, continuity of the tensor product implies the following result.

Lemma 3.4 Let
Example 3.5 It is well known that the norm · 0,2 is a reasonable cross norm on a d j =1 L 2 (I j ), whereas · N,2 for N ≥ 1 is not a reasonable cross norm on Note that any functional . If · is a reasonable cross norm, then by (3.3) the map [13] named the following norm · ∨ the injective norm.
It is well known that the injective norm is a reasonable cross norm (see Lemma 1.6 in [20]). Further properties are given by the next proposition.

Proposition 3.7
The following statements hold.

(a) The injective norm is the weakest reasonable cross norm on
Proof Statement (a) is a classical result (cf. [20], [14]). To prove (b), we use the fact that · ∨ · implies · * ∨ · * (see again [20], [14]). Then for some C > 0, and the proof ends using the fact that · * ∨ is also a cross norm.

Minimal Subspaces in a Banach Tensor Space
Let V be a tensor product of Banach spaces  10d)). Assume that the norm · on V satisfies · · ∨ (3.6) 4 We recall that the definition of U IV j (v) requires the definition of a norm on V [j ] . The following arguments will be based on U III j (v).
(cf. Proposition 3.7a). This assumption ensures that the Banach tensor space V · is always a Banach subspace of the Banach tensor space V · ∨ . This fact allows us to extend the definition of minimal subspaces to a Banach tensor space V · with a norm · satisfying (3.6). To this end, the following lemma will be useful.
The last inequality holds for any norm on V satisfying · ≥ (1/C) · ∨ and proves (3.7). The statement about the extension id j ⊗ ϕ [j ] is standard.
An immediate consequence of Lemma 3.8 and Theorem 2.17 is the following.
be a Banach space and assume that · is a norm on V satisfying (3.6). Then for each algebraic tensor v ∈ V the representation For the hierarchical format from [15] we need to extend the results to a minimal subspace in the tensor space V α := k∈α V k , where α ⊂ D := {1, . . . , d} contains more than one index. Then the splitting V j ⊗ V [j ] from above becomes V α ⊗ V α c , where V α c := k∈D\α V k . The definition of, e.g., U I j (v) in (2.10a) becomes Remark 3.10 By arguments analogous to those above, we can show that

Definitions
So far, U min j (v) has been defined for algebraic tensors only. From id j ⊗ ϕ [j ] ∈ L(V · , V j ), we can extend the definition of U j min (v) in Corollary 3.9 even to topological tensors v ∈ V · \ V as follows.

Definition 3.11
For a given Banach tensor space V · with a norm · satisfying (3.6) we define the set Observe that (id j ⊗ ϕ [j ] )(v) is well defined, because id j ⊗ ϕ [j ] is continuous and coincides with the standard definition when v ∈ V. Thus, for each v ∈ V · we can define its 'minimal subspace' by If we take into account the topological properties of V · , we may consider its closure with respect to the norm · : The second identity is a consequence of Lemma 3.4. If v ∈ V, the set U min j (v) is a finite-dimensional subspace in V j and therefore closed, i.e., U min j (v) · j = U min j (v). In the general case of v ∈ V · , the subspace U min j (v) may be not closed.
Before we discuss the Banach subspace U · (v) in Sect. 3.3.3, we first analyse the properties of the subspace U min j (v). To this end we use the following definition.
Definition 3. 12 We say that a sequence (x n ) n∈N in a Banach space X converges weakly to x ∈ X, if lim ϕ(x n ) = ϕ(x) for all ϕ ∈ X * . In this case, we write x n x.

Dependence of U min j (v) on v
The properties of the maps id j ⊗ ϕ [j ] involved in the definition of U min j (v) are discussed in Lemma 3.13. As a consequence, we shall establish our main result in Theorem 3.15 about the dimensions of U min j (v n ) and U min j (v) for a weakly convergent sequence v n v.

Lemma 3.13 Assume that the norm of the Banach tensor space
k be an elementary tensor. We have to show that holds for all ϕ j ∈ V * j . By Lemma 3.8, id j ⊗ ϕ [j ] : V · → V j is continuous. Therefore, the composition ϕ j • (id j ⊗ ϕ [j ] ) : V · → R is a continuous functional belonging to V * · , and hence v n v implies This proves the lemma for an elementary tensor ϕ [j ] . The result extends immediately to finite linear combinations ϕ [j ] ∈ a k =j V * k .

Lemma 3.14 Assume
Then there is an n 0 such that for all n ≥ n 0 the N -tuples (x (i) . Continuity of the determinant proves n → ∞ := det((δ ij ) N i,j =1 ) = 1. Hence, there is an n 0 such that n > 0 for all n ≥ n 0 , proving linear independence of {x

Theorem 3.15 Assume that the norm of the Banach tensor space
, the dimensions are identical in the sense of footnote 5. We can select a subsequence (again denoted by v n ) such that dim U min j (v n ) is weakly increasing. If dim U min j (v n ) → ∞ holds, nothing is to be proved. Therefore, assume that lim dim U min j (v n ) = N < ∞. For an indirect proof assume that dim U min j (v) > N. Then, there are N + 1 linearly independent vectors By Lemma 3.13, the sequence b (i) [j ] )(v n ) b (i) converges weakly. By Lemma 3.14, for large enough n, also {b For the hierarchical format from [15], id j ⊗ ϕ [j ] must be replaced by id α ⊗ ϕ α c (cf. Corollary 2.19 and Remark 3.10). Similar methods as above show the following generalisations: Here, we equip the tensor space V α = a j ∈α V j with the injective norm · ∨ from (3.5).

dim(U min j (v)) < ∞
Consider U(v) and U · (v) from (3.8a,b). For algebraic tensors v we know that v ∈ U(v). However, the corresponding conjecture v ∈ U · (v), in the general case, turns out to be not quite obvious.
with v = lim v n . We do not have a proof that this holds in general. A positive result holds for the Hilbert case (see Sect. 3.4) and if the subspaces U min j (v) are finite dimensional (see Theorem 3.16). In the general Banach case, we give a proof for v = lim v n , provided that the convergence is fast enough.
For practical applications, the finite-dimensional case is the most important one, since it follows from Theorem 3.15 with bounded lim inf n→∞ dim U min j (v n ).

Theorem 3.16
Assume that V · is a Banach tensor space with · satisfying (3.6).
For v ∈ V · and all 1 ≤ j ≤ d assume that dim(U min j (v)) < ∞. Then v belongs to the (algebraic) tensor space a We set Thus, the theorem follows, if we prove that Thus we need to show the following.

dim(U min
Thus, under the assumption dim(U min j (v)) = ∞ for some j ∈ {1, 2, . . . , d} we have v ∈ V · \ V, and then v is defined as the limit of some Cauchy sequence in V.
For the next theorem we need a further assumption on the norm · . A sufficient condition is that · is a uniform cross norm; i.e., it is a cross norm (cf. (3.2)) and satisfies The uniform cross norm property implies that · is a reasonable cross norm (cf. [25]). Hence, condition (3.6) is ensured (cf. Proposition 3.7a). A further consequence will be needed.

Lemma 3.17 Let · be a uniform cross norm on V. Note that
The map defined by does not depend on the choice of v d . Therefore, it defines a norm on V [d] .
, the following estimates hold: (3.14) Proof (a) Let ϕ d ∈ V * d be the functional with ϕ d * , the estimate can be continued by Together with the opposite inequality from above, we have proved the second equation in (3.13).
(c) Any proves the first inequality in (3.14). The second one can be proved analogously.

Lemma 3.18
Let Y ⊂ X be a subspace of a Banach space X with dim(Y ) ≤ n. Then there exists a projection Φ ∈ L(X, X) onto Y such that The bound is sharp for general Banach spaces, but can be improved to Φ X←X ≤ n 1/2−1/p for X = L p .
Before we state the next theorem we recall the following definition. In Sect. 1, we introduced the set R r for the tensor space V. Since V ∼ = V [d] ⊗ a V d we now introduce the notation

Theorem 3.19
Assume that V · is a Banach tensor space with a uniform cross norm with r ≤ n, and a convergence rate given by According to Corollary 2.15b, we can fix any basis {v The triangle inequality yields Note that On the other hand, according to Lemma 2.2, we can choose a basis {v (v n ) and its corresponding dual basis {χ An analogous proof shows that satisfies the properties The uniform cross norm property (3.12) where the latter bound is given by Lemma 3.18. Therefore, Altogether, we get the estimate The assumption v − v n ≤ o(n −3/2 ) implies u n − v → 0.

Minimal Closed Subspaces in a Hilbert Tensor Space
Let ·, · j be a scalar product defined on V j (1 ≤ j ≤ d), i.e., V j is a pre-Hilbert space. Then V = a d j =1 V j is again a pre-Hilbert space with a scalar product which This bilinear form has a unique extension ·, · : V × V → R. One verifies that ·, · is a scalar product, called the induced scalar product. Let V be equipped with the norm · corresponding to the induced scalar product ·, · . As usual, the Hilbert tensor space V · = · d j =1 V j is the completion of V with respect to · . Since the norm · is derived via (3.17), it is easy to see that · is a reasonable and even uniform cross norm.
We recall that orthogonal projections P ∈ L(V , V ) (V Hilbert space) are self-adjoint projections. P is an orthogonal projection onto the closed subspace U := range(P ) ⊂ V , which leads to the direct sum V = U ⊕ U ⊥ , where U ⊥ = range(id − P ). Vice versa, each closed subspace U ⊂ V defines an orthogonal projection P with U = range(P ).

Lemma 3.20 Let V j be Hilbert spaces with subspaces
Proof We consider the case d = 2 only (d ≥ 3 can be obtained by induction). Then the assertion to be proved is The analogous statement for the algebraic tensor spaces holds by Lemma 2.11. The general rule X ∩ Y ⊂ X ∩ Y (· is the closure with respect to · ) implies that The lemma is proved, if the opposite inclusion holds: (3.18) Let v ∈ U 1 ⊗ · V 2 . By definition, there is a sequence v n ∈ U 1 ⊗ a V 2 with v n → v. Let P 1 be the orthogonal projection onto U 1 . Then (P 1 ⊗ id 2 )v n = v n proves P 1 v = v for the extension P 1 := P 1 ⊗ id 2 . Similarly, P 2 v = v follows with P 2 := id 1 ⊗ P 2 , where P 2 is the orthogonal projection onto U 2 . Since P 1 ⊗ id 2 and id 1 ⊗ P 2 commute, the product P 1 ⊗ P 2 is also an orthogonal projection. Its range is U 1 ⊗ a U 2 , while U 1 ⊗ · U 2 is the range of its extension P := P 1 ⊗ P 2 = P 1 P 2 = P 2 P 1 . Hence, This ends the proof of (3.18).

Lemma 3.21
Let V i (i = 1, 2) be Hilbert spaces, and U 1 ⊂ V 1 a closed subspace. Then the direct sum Proof Consider the ranges of P 1 = P 1 ⊗ id 2 and id − P 1 , where P 1 is the orthogonal projection onto U 1 .
Unlike Theorem 3.19 for the Banach tensor space setting, we need no assumption on the speed of the convergence v n → v to obtain the result v ∈ U · (v).

Theorem 3.22
Assume that V j are Hilbert spaces and that V is equipped with the norm · corresponding to the induced scalar product. Then for all v ∈ V · it follows that v ∈ U · (v).
Proof (1) In order to simplify the notation, we set U j : For an indirect proof we assume v ⊥ = 0. Then there are v j ∈ U ⊥ j and v [j ] ∈ V [j ] with v j ⊗ v [j ] , v ⊥ = v j ⊗ v [j ] , v = 0 (otherwise there are no algebraic tensors converging to v ⊥ ). For ϕ [j ] So far, we have assumed that the norm · of the Hilbert space V corresponds to the induced scalar product. In principle, we may also define another scalar product ·, · V on V together with another norm · V . In this case, we have to assume that · V is a uniform cross norm (at least, . This ensures that the projections P j (as defined in the proof of Lemma 3.20) belong to L(V, V). Furthermore, (3.6) holds. Scalar products like v j ⊗ v [j ] , v in the proof above are to be replaced with (ϕ j ⊗ ϕ [j ] )(v), where, as usual, ϕ j ∈ V * j is defined via ϕ j (·) = v j , · j . Then we can state again that v ∈ U · (v).

Main Statement
Theorem 4.1 Let V · be a reflexive Banach tensor space with a norm satisfying (3.6). Then for each v ∈ V · there exists w ∈ T r such that (4.1) Proof Combine Theorem 4.2 and Proposition 4.3 given below.
A subset M ⊂ X is called weakly closed, if x n ∈ M and x n x implies x ∈ M. Note that 'weakly closed' is stronger than 'closed', i.e., M weakly closed ⇒ M closed.
Theorem 4.2 [4] Let (X, · ) be a reflexive Banach space with a weakly closed subset ∅ = M ⊂ X. Then the following minimisation problem has a solution: For any

Proposition 4.3 Let V · be a Banach tensor space with a norm satisfying (3.6).
Then the set T r is weakly closed.
Proof Let {v n } ⊂ T r be such that v n v. Then there are subspaces U j,n ⊂ V j such that v n ∈ U j,n with dim U j,n = r j . Since U min j (v n ) ⊂ U j,n , dim U min j (v n ) ≤ r j holds for all n ∈ N. Consequently, by Theorem 3.15, dim U min j (v) ≤ r j . Thus, U min j (v) is finite dimensional. From Theorem 3.16 we conclude that v ∈ a d j =1 U min j (v) and, thereby, v ∈ T r . Corollary 4.4 A statement analogous to Theorem 4.1 also holds for the set H r appearing for the hierarchical format from [15] and the format from [24]. The proof uses the fact that H r is weakly closed.
Since the assumption of reflexivity excludes important spaces, we add some remarks on this subject. The existence of a minimiser or 'nearest point' w in a certain set A ⊂ X to some v ∈ V \A is a well-studied subject. A set A is called 'proximinal' if v − w = min u∈A v − u has at least one solution w ∈ A. Without the assumption of reflexivity, there are statements ensuring under certain conditions that the set of points in V \A possessing nearest points in A is dense (e.g., Edelstein [10]). A smaller class than the weakly closed subsets A are the closed and convex subsets. However, even for closed and convex subsets one cannot avoid reflexivity, in general, because of the following result (cf. [4,Proposition 4]). Note that the sets T r and H r are not convex, but weakly closed, as we will show with the help of minimal subspaces.

Generalisation to the Intersection of Finitely Many Banach Tensor Spaces
We recall that the assumption (3.6) implies a Proposition 3.7b). For certain Banach tensor spaces this property does not hold. Therefore, we have to check whether some of the results given in the previous section can be extended to this case. Thus, in this section we introduce the intersection tensor spaces. We also study sequences of minimal subspaces in this framework in order to prove the existence of a best T r approximation. To illustrate this situation we give the following example.
Recall that f C 1 (I ) = max x∈I {|f (x)|, |f (x)|} is the norm of continuously differentiable functions in one variable x ∈ I ⊂ R. The naming · 1,mix of the following norm is derived from the mixed derivative involved. · C 1 (I ) ) and W = (C 1 (J ), · C 1 (J ) ). For the tensor space V ⊗ a W we introduce the norm ϕ(x, y) .
(4.2) It can be shown that · 1,mix is a reasonable cross norm. However, the standard norm of C 1 (I × J ) given by is not a reasonable cross norm.
We have seen that the space C 1 (I × J ) is not the straightforward result of the tensor product C 1 (I ) ⊗ C 1 (J ). The norm · 1,mix from (4.2) turns out to be a reasonable cross norm, but then the resulting space C 1 mix (I × J ) is a smaller space than C 1 (I × J ). Vice versa, the dual norm · * C 1 (I ×J ) of C 1 (I × J ) is not bounded for v * ⊗ w * ∈ V * ⊗ W * . Therefore, it is not a reasonable cross norm.
The family of Sobolev spaces H m,p (I j ) for m = 0, 1, . . . , N is an example of a scale of Banach spaces which we introduce below. From now on, we fix integers N j and denote the j -th scale by For each n in an admissible index set N , we define the tensor space (4.7) All spaces V (n) are subspaces of . Assume that the following conditions hold: (a) For each admissible n ∈ N d 0 , a norm · n on V (n) exists satisfying · n ≤ · m for n ≤ m ∈ N , and (b) the norm · 0 on V (0) = a d j =1 V j satisfies (3.6). Now, we introduce the Banach tensor space (4.8) Note that for each n ∈ N , if n ≤ m ∈ N , then V (m) · . From Lemma 2.16 one derives the following result.
Proof From Lemma 2.16 we have Let N ⊂ N d 0 be an admissible index set. From Lemma 4.8b it follows that the intersection of the set of tensor spaces {V (n) : n ∈ N } is the tensor space V (N 1 ,...,N . Observe that the index (N 1 , . . . , N d ) does not necessarily belong to the index set N . Also, by Lemma 4.8a, we obtain the following minimal representation: (4.9) Next, we introduce the Banach space induced by intersection of the set of Banach tensor spaces {V or an equivalent one.
Next, we consider elementary tensors from the tensor space V (0) .

Proposition 4.10
Let N ⊂ N d 0 be an admissible index set. Then and a minimal number r of terms.
Proof By definition (4.10), · is an elementary tensor, it belongs to V (0) ∩ V (n) = V (n) . From Lemma 2.16 it follows that v ∈ n∈N V (n) = a d j =1 ( n∈N V (n j ) j ). By condition (4.6c), one of the n j equals N j , which implies v ∈ a Moreover, it is easy to see that the · mix -norm is generated by the induced scalar product (3.17) (I j ).
Thus, a natural question arising in this example is whether Theorem 4.1 holds for the Hilbert tensor space H N (I) characterised by (4.14).
From Proposition 4.10, there are different equivalent versions of how to define the minimal subspace U min Here, we can state the following.  (N 1 ,...,N d ) , Corollary 4.14 cannot be extended as Corollary 3.9 was for the Banach space case. A simple counterexample is f ∈ C 1 (I × J ) with f (x, y) = F (x + y) and F / ∈ C 2 . Choose ϕ ∈ C 1 (J ) * as ϕ = δ η . Then ϕ(f )(x) = −F (x + η) ∈ C 0 (I ), but ϕ(f ) is not in C 1 (I ) in contrast to Corollary 4.14. While, in Corollary 4.14, we could take functionals from ( a k =j V (n k ) k ) for any n bounded by n k ≤ N k , we now have to restrict the functionals to n = 0. Because of the notation V (0) k = V k , the definition coincides with the usual one: where the completion is performed with respect to the norm · j,0 of V (0) j . In the following we show that the same results can be derived as in the standard case. Condition (3.6) used before must be adapted to the situation of the intersection space. Consider the tuples N j = (0, . . . , 0, N j , 0, . . . , 0) ∈ N from (4.6c) and the corresponding tensor space ⊗ a V j +1 ⊗ · · · ⊗ a V d endowed with the norm · N j . From now on, we denote by · ∨(N j ) the injective norm defined from the Banach spaces V 1 , . . . , V j −1 , V (N j ) j , V j +1 , . . . , V d . · n : n ∈ N } satisfying assumption (4.17). Let ϕ [j ] ∈ a k =j V * k and v n , v ∈ V · N with v n v. Then weak convergence Proof Repeat the proof of Lemma 3.13 and note that ϕ j ∈ (V (N j ) j ) * composed with an elementary tensor ϕ [j ] = k =j ϕ k (ϕ k ∈ V * k ) yields ϕ = d k=1 ϕ k ∈ a d k=1 (V (n k ) k ) * with n k = 0 for k = j and n j = N j . By (4.17) and Proposition 3.7b, ϕ belongs to (V (N j ) ) * . Proof Let v m ∈ V (N 1 ,...,N d ) be a sequence with v m → v ∈ V · N By definition (4.10) of the intersection norm, v m − v N j → 0 holds for all j . Then (3.7) shows that (id j ⊗ ϕ [j ] )(v − v m ) j,N j → 0. Since (id j ⊗ ϕ [j ] )(v m ) ∈ V (N j ) j by Proposition 4.10, the limit of (id j ⊗ ϕ [j ] )(v) also belongs to V (N j ) j .

Lemma 4.18 Assume that V · N is a Banach space induced by the intersection of the set of Banach tensor spaces {V
(n) · n : n ∈ N } satisfying assumption (4.17).
Proof We can repeat the proof from Theorem 3.15.
Finally, in a similar way as in Proposition 4.3, we can also obtain the following statement. Example 4.20 Now, we return to H N (I) characterised as an intersection of Hilbert tensor spaces by (4.14). Recall that · N j is a reasonable cross norm in V (N j ) = L 2 (I 1 ) ⊗ a · · · ⊗ a L 2 (I j −1 ) ⊗ a H N,2 (I j ) ⊗ a L 2 (I j +1 ) ⊗ · · · ⊗ a L 2 (I d )

Proposition 4.19 Assume that V · N is a Banach space induced by the intersection of the set of Banach tensor spaces {V
for 1 ≤ j ≤ d. Then condition (4.17) holds, and we obtain the existence of a best T r approximation in this space.
For Hilbert spaces, Uschmajew [27] has proved the existence of minimisers of (1.3) using particular properties of Hilbert spaces.

Some Consequences of the Best T r Approximation in a Hilbert Tensor Space
In this section we assume that the Hilbert space V · := · d j =1 V j , which is the completion of V := a d j =1 V j with respect to · , has the property that the set T r is weakly closed. Then for each r ∈ N d 0 with r ≥ 1, we can define a map from V · to [0, ∞), using u r := max v, u : v ∈ T r , v = 1 . (4.18) Observe that if · is induced by the scalar products of V j , then · 1 = · ∨ . For general r ≥ 1 we obtain the following result.

Theorem 4.21
Assume that in the Hilbert space V · the set T r is weakly closed. Then for each r ∈ N d 0 with r ≥ 1, the following statements hold. Note that u − λw 2 = u 2 − 2λ u, w + λ 2 . The minimum of u − λw 2 2 for w ∈ D is obtained for λ = u, w > 0 and equals u − λw 2 2 = u 2 2 − | u, w | 2 . Thus, To prove (c) note that the norm axiom λu r = |λ| u r and the triangle inequality are standard. To prove that u = 0 implies u r > 0, note that if u r = 0 we have u, v = 0 for all v ∈ T r . Since span T r is dense in V · , we obtain that u = 0.
Let V 1 = R n 1 and V 2 = R n 2 be equipped with the usual Euclidean norm. Then V 1 ⊗ a V 2 is isomorphic to matrices from R n 1 ×n 2 with the Frobenius norm · . It is not difficult to see that u (1,1) coincides with σ 1 , the first singular value of the singular value decomposition of u.