Continuous functions on closed and bounded subsets of infinite dimensional vector spaces

/ Oscar

Hello and welcome to my first post! I recently revisited a question from an analysis qualifying exam that explored the following:

Do continuous functions defined on closed and bounded subsets of infinite dimensional vector spaces achieve their extrema?

What I think makes this question interesting is that it probes one of the most central results in a first year analysis course regarding the notion of compactness and shows that, in the infinite dimensional setting, characterizing compactness gets a bit tricky (or at least trickier than in Euclidean space). Before we answer this question, let’s gather some important results regarding compactness in \mathbb{R}^n and Banach spaces.

Let’s first introduce the most general definition of compactness which is presented through the lens of open covers. A subset A \subset \mathbb{R}^n is compact if and only if every open cover \{U_{\alpha_n}\}_{\alpha_n \in \mathcal{I}} of A has a finite subcover that contains A, i.e. there exists a finite subset of indices \{\alpha_{n_1},\dots,\alpha_{n_k}\} \subset \mathcal{I}: A \subset U_{\alpha_{n_1}} \cup \dots \cup U_{\alpha_{n_k}}. One can think of compact sets as the continuous analogue of finite sets. While most compact sets we will face in the wild are infinite in cardinality, they are finite in the sense that we can cover them with finitely many open balls. These open balls are the continuous analogue of discrete points. In essence, compact sets can be described using only a finite amount of information and this property can be exploited to give some nice results.

While this definition is important, it will not be the most useful for our purposes. Instead, we will use what is known as sequential compactness: a subset A \subset \mathbb{R}^n is (sequentially) compact if and only if every infinite sequence in A has a convergent subsequence. Note that this definition gives a different flavor of compactness. In a nutshell, it states that a compact set does not allow any infinite sequence to escape from its limit points. Of course, there are settings in which these definitions are not equivalent, but we will be restricting ourselves to Banach spaces in this discussion.

So why would we care whether a set is compact? Well as alluded to earlier, compact sets behave as nearly finite objects and exploiting this structure can be extremely useful. A central result regarding compact sets is the following: any continuous function defined on a compact domain achieves its extrema. That is, if f: A \subset \mathbb{R}^n \rightarrow \mathbb{R} is continuous and A is compact, then there exists an x_0,y_0 \in A:

f(x_0) \leq f(x) \leq f(y_0)\ \forall\ x\in A.

Hence, characterizing compact sets is an important task. With this in mind, we present the simplest characterization of compact sets in \mathbb{R}^n (or any finite dimensional vector space over the field \mathbb{R}).

Heine-Borel Theorem:
A subset A \subset \mathbb{R}^n is compact if and only if it is closed and bounded.

Ex 1: We will prove that the unit sphere S^{n-1} = \{x \in \mathbb{R}^n\ |\ \|x\|_2 = 1\} is compact using the Heine-Borel Theorem. Let f : \mathbb{R}^n \rightarrow \mathbb{R} be defined by f(x) = \|x\|_2. Note that S^{n-1} = f^{-1}(\{1\}), i.e. the unit sphere is the pre-image of the closed set \{1\} under the continuous map x \mapsto \|x\|_2. Hence S^{n-1} is closed. It is also clearly bounded, so by the Heine-Borel Theorem, it is compact.

It is important to note that, even in the more general Banach space setting, any compact set is closed and bounded. Hence the power in Heine-Borel lies in the backwards implication in which any closed and bounded set is compact in \mathbb{R}^n. With this in mind, a natural inquiry is whether the backwards implication holds true for infinite dimensional vector spaces. Unfortunately, we will see that this is not the case.

Ex 2: Consider the vector space of absolutely summable sequences endowed with the \ell_1-norm. That is, consider the space

\ell_1 = \left\{x = (x_1,x_2,\dots)\ |\ \|x\|_1 = \sum_{i=1}^{\infty} |x_i| < \infty\right\}.

We just saw that the unit sphere in \mathbb{R}^n is compact using Heine-Borel. Let’s consider the “unit \ell_1-sphere”, i.e. the set S = \{x \in \ell_1\ |\ \|x\|_1 = 1\}. This set is closed and bounded, but we will show it is not sequentially compact by exhibiting an infinite sequence with no convergent subsequence. For all n \in \mathbb{N}, let e_n be a sequence whose i-th entry is 1 if i = n and 0 otherwise and consider (e_n)_{n \in \mathbb{N}}. Then for any n \neq m, \|e_n - e_m\|_1 = 2 so no convergent subsequence of (e_n)_{n \in \mathbb{N}} exists since any two distinct elements of this sequence will always be a distance 2 apart from one another.

This example showcases the fact that closed and boundedness is no longer equivalent to compactness in the infinite dimensional setting. If that is the case, what are some examples of compact sets in \ell_1?

Ex 3: Define the set

A_1 = \left\{(\alpha,0,0,\dots)\ |\ \alpha \in [0,1]\right\}

and let (x_n)_{n \in \mathbb{N}} be an infinite sequence in A_1. Then any element of the sequence is of the form (\alpha_n,0,0,\dots) where each \alpha_n \in [0,1]. Since [0,1] is compact in \mathbb{R}, the sequence (\alpha_n)_{n \in \mathbb{N}} has a convergent subsequence, say (\alpha_{n_k})_{n_k \in \mathbb{N}}. This generates a convergent subsequence (x_{n_k})_{n_k \in \mathbb{N}} of (x_n)_{n \in \mathbb{N}} so A_1 is compact.

Ex 4: Define e_n in the same way as in example 2. Let

A_2 = \left\{(0,0,0,\dots),e_1,\frac{e_2}{2},\dots,\frac{e_n}{n},\dots\right\}.

Note that if we take any infinite sequence (x_n)_{n \in \mathbb{N}} in A_2, there are two possibilities:

  • If (x_n)_{n \in \mathbb{N}} contains only finitely many elements of A_2, there must be a constant subsequence of (x_n)_{n \in \mathbb{N}} since it is an infinite sequence.
  • Otherwise, x_n \rightarrow (0,0,0,\dots).

In either case, (x_n)_{n \in \mathbb{N}} must have a convergent subsequence so A_2 is compact.

In this example, our compact set had two main ingredients: a convergent sequence (e_n/n)_{n \in \mathbb{N}} and its limit (0,0,0,\dots). While we could have simplified this example, the type of sequence we generated will be a good starting point to answer our original question: do continuous functions defined on closed and bounded subsets of infinite dimensional vector spaces achieve their extrema?

Define the function f : \ell_1 \rightarrow \mathbb{R} by f(x) = \sum_{i=1}^{\infty}x_i. It is a straightforward exercise to show that f is a well-defined bounded linear operator and, hence, continuous. For all n \in \mathbb{N}, define the sequence \tilde{e}_n = (0,\dots,0,1 + 1/n,0,\dots). That is, the i-th entry of \tilde{e}_n is 0 if i \neq n and 1+1/n if i = n. Then set A_3 = \{\tilde{e}_1,\tilde{e}_2,\dots\}; A_3 is clearly bounded, but is it closed? What are its limit points and does it contain them? It turns out A_3 is vacuously closed as it does not have any limit points. We can rigorously show A_3 is closed by showing that its complement \ell_1 \setminus A_3 is open. First, note that for any x,y \in A_3 with x \neq y, we have \|x-y\|_1 \geq 2. Fix x_0 \in \ell_1 \setminus A_3. There are two possibilities:

  • If B(x_0,1) \subset \ell_1 \setminus A_3, then we are done.
  • Otherwise, there exists an x \in A_3 such that \|x - x_0\|_1 < 1. Set \epsilon = \|x-x_0\|_1. Then B(x_0,\epsilon/2) \subset \ell_1 \setminus A_3 since any distinct element y \in A_3 with x \neq y has \|x - y\|_1 \geq 2.

In either case, \ell_1 \setminus A_3 is open so A_3 is closed. Finally, observe that

\{f(x)\ |\ x \in A_3\} = \{1 + 1/n\ |\ n \in \mathbb{N}\}

so \inf_{x \in A_3} f(x) = 1 but this infimum is never achieved! From this, one can easily construct an example in which the supremum is never achieved as well.

This concludes my first post! Thank you for reading. In my next post, I will discuss epsilon nets and their relation to compactness along with how they are utilized as a tool in signal recovery theory.

Leave a comment