Infinite horizon differential games for abstract evolution equations

Infinite horizon differential games for abstract evolution equations

A.J. Shaiju^*

Department of Mathematics, Indian Institute of Science, Bangalore, 560 012, India, E-mail: [e-mail]

ABSTRACT

Berkovitz's notion of strategy and payoff for differential games is extended to study two player zero-sum infinite dimensional differential games on the infinite horizon with discounted payoff. After proving dynamic programming inequalities in this framework, we establish the existence and characterization of value. We also construct a saddle point for the game.

Mathematical subject classification: 91A23, 49N70, 49L20, 49L25.

Key words: differential game, strategy, value, viscosity solution, saddle point.

1 Introduction

In [1], Berkovitz has introduced a novel approach to differential games of fixed duration. He has extended this framework to cover games of generalized pursuit-evasion [2] and games of survival [3] in finite dimensional spaces. Motivated by these developments we define strategies and payoff for infinite horizon discounted problems whose state is governed by a controlled semi-linear evolution equation in a Hilbert space. In this setup, we show the existence of value and then characterize it as the unique viscosity solution of the associated Hamilton-Jacobi-Isaacs (HJI for short) equation. To achieve this, we follow a dynamic programming method and hence we differ from Berkovitz's approach for finite horizon problems [4]. We also establish the existence of a saddle point for the game by constructing it in a feedback form.

The rest of this paper is organized as follows. The description of the game and some important preliminary results are given in Section 2. In Section 3, we deal with dynamic programming and characterization of the value function. Section 4 contains the construction of saddle point equilibrium. We conclude the paper with some remarks in Section 5.

2 Preliminaries

Let the compact metric spaces U and V be the control sets for players 1 and 2 respectively. For 0 < s < t , let

The sets [s, t] and [s, t] are called the control spaces on the time interval [s, t] for players 1 and 2 respectively. The functions u(·) Î [s, t] and v(·) Î [s, t] are referred to as the precise or usual controls (or simply 'controls') on the time interval [s, t] for players 1 and 2 respectively. We denote [0, ¥) and [0, ¥) by and respectively.

Let , a real Hilbert space, be the state space. Let x(t) Î denote the state at time t. The state x(·) with initial point x₀Î is governed by the following controlled semi-linear evolution equation:

where f: × U × V ® , u(·) Î , v(·) Î and –A : É D(A) ® is the generator of a contraction semigroup {S(t)} on . We assume that

(A1) The function f is continuous and, for all x, y Î and (u, v) Î U × V,

Under the assumption (A1), for each u(·) Î , v(·) Î and x₀Î , (2.1) has a unique global mild solution (see e.g., Proposition 5.3, p. 66 in [10]) which is denoted by f(·, x₀, u(·), v(·)) and is referred to as the trajectory corresponding to the pair of controls (u(·), v(·)) with initial point x₀.

Following Warga [13], we now describe relaxed controls and relaxed trajectories. Let

where (U) and (V) are the spaces of probability measures on U and V respectively with the topology of weak convergence. The sets [s, t] and [s, t] are called the relaxed control spaces on the time interval [s, t] for players 1 and 2 respectively. These relaxed control spaces, equipped with weak^* topology, are compact metric spaces. The relaxed control spaces [0, ¥) and [0,¥) are denoted by and respectively. Note that by identifying u(·) and v(·) by d_u(·) and d_v(·) respectively, precise controls can be treated as relaxed controls.

For m(·) Î and n(·) Î , the state equation in the relaxed control framework is given by

Since f satisfies (A1), it follows that also satisfies (A1) with u Î U , v Î V replaced respectively by m Î (U) , n Î (V). Therefore for each x₀Î , m(·) Î and n(·) Î , the existence and uniqueness of a global mild solution to (2.2) follows analogously. This solution is called a relaxed trajectory and is denoted by y(·, x₀, m(·), n(·)).

We now begin the description of the game by defining the strategies of players. A strategy for player 1 is a sequence P = {P_n} of partitions of [0, ¥), with ||P_n|| ® 0 , and a sequence G = {G_n} of instructions described as follows:

Let P_n = {0 = t₀ < t₁ < ¼}. The nth stage instruction G_n is given by a sequence , where G_n,1Î [t₀, t₁) and for j > 2,

Similarly, a strategy for player 2 is a sequence = {_n} of partitions of [0, ¥), with ||_n|| ® 0 , and a sequence D = {D_n} of instructions described as follows:

Let

_n = {0 = s₀ < s₁ < ¼}. The nth stage instruction D_n is given by a sequence , where D_n,1Î [s₀, s₁) and for j > 2,

We suppress the dependence of the sequence of partitions on a strategy and, by an abuse of notation, denote a strategy by G or D. In what follows G stands for a strategy for player 1 and D stands for a strategy for player 2.

Note that a pair (G_n, D_n) of nth stage instructions uniquely determines a pair (u_n(·),v_n(·)) Î × as follows. Let _n = {0 = r₀ < r₁ < ¼} be the common refinement of P_n and _n. The control functions u_n(·) and v_n(·) are given by the sequences (u_n,1(·), u_n,2(·),¼) and (v_n,1(·), v_n,2(·),¼) respecively, where u_n,j(·) Î [r_j–1, r_j) and v_n,j(·) Î [r_j–1, r_j). Let u(·), v(·) denote respectively the restrictions of u_n(·), v_n(·) to the interval [r₀, r_j).

On [r₀, r₁), set u_n,1(·) = G_n,1 and v_n,1(·) = D_n,1.

Let j > 1. If r_j = t_i, then on [r_j, r_j+1) we take u_n,j+1(·) = G_n,i+1(u(·),v(·)) and v_n,j+1(·) = D_n,l+1(u(·), v(·)) , where l is the greatest integer such that s_l < r_j and s_l = r_j^¢.

If r_j = s_m, then on [r_j, r_j+1) take u_n,j+1(·) = G_n,k+1(u(·),v(·)) and v_n,j+1(·) = D_n,m+1(u(·),v(·)) , where k is the greatest integer such that t_k < r_j and t_k = r_m^¢.

The pair (u_n(·),v_n(·)) determined this way is called the nth stage outcome of the pair (G, D) of strategies.

Let c: × U × V ® be the running payoff function and let l > 0 be the discount factor. We assume that

(A2) The function c is bounded, continuous and, for all x, y Î and (u, v) Î U × V

|c(x, u, v) – c(y, u, v)|< K||x – y||.

Without any loss of generality, we take c to be nonnegative. For x₀Î and (u(·), v(·)) Î × , let f⁰(·, x₀, u(·), v(·)) denote the solution of

Let (·, x₀, u(·), v(·)) denote . The running cost in the relaxed framework is defined by

Note that satisfies (A2) with (u, v) Î U × V replaced by (m, n) Î (U) × (V). Now the relaxed trajectory (·, x₀, m(·), n(·)) is interpreted in an analogous way. Let

The next result is helpful in defining the concept of motion in the game. To achieve this, we make the following assumption.

(A3) The semigroup {S(t)} is compact.

Lemma 2.1. Assume (A1)-(A3). Let {(u_n(·), v_n(·)} be the sequence of nth stage outcomes corresponding to a pair (G, D) of strategies and {x_0n}, a sequence converging to x₀. Then the sequence {(·, x_0n, u_n(·), v_n(·))} of nth stage trajectories is relatively compact in C([0, ¥);

).

Proof. Let _n(·) = (·, x_0n, u_n(·), v_n(·)), h_n(·) = (·, _n(·), u_n(·), v_n(·)), _n(·) =

It is enough to show that for each T > 0, the sequence of nth stage trajectories, when restricted to [0, T], is relatively compact in C([0, T]; ). Fix T > 0. Let Q : L²([0, T]; ) ® C([0, T]; ) be the operator defined by

Then

_n (·) = _n (·) + Q(h_n(·))(·).

Since the sequence

_n(·) converges to (·) uniformly, it is sufficient to prove the relative compactness of {Q(h_n(·))(·) }. To achieve this we show that the operator Q is compact. This will imply the desired result since, by (A1) and (A2), {h_n(·)} is bounded in L²([0, T]; ). Let {h_k(·)} be a sequence in the unit ball of L²([0, T]; ). We need to show that {Qh_k(·)(·)} is relatively compact. By Arzela-Ascoli theorem, the proof will be complete if we establish the pointwise relative compactness and equicontinuity of the sequence {Qh_k(·)(·)}.

Let t Î [0, T]. We first prove the relative compactness of {Qh_k(·)(t)}. This is trivial if t = 0. So we assume that t > 0. Let > 0 be given. Since {h_k(·)} is in the unit ball, there exists d Î (0, t) such that for all k

Note that

where

Since (d) is compact and {y_k} is bounded in , there exist y₁,¼, y_m Î such that . Therefore {Q(h_k(·))(t)} Ì B(y_i, ). Thus we have established the relative compactness of {Q(h_k(·))(t)}.

Next we prove the equicontinuity of {Q(h_k(·))(t)}. Let t, s Î [0, T] and s < t. The case when s = 0 is trivial. Assume that 0 < s. Now for d small enough,

Q(h_k(·))(t) – Q(h_k(·))(s) = I₁ + I₂ + I₃ ;

where

By Cauchy-Schwartz inequality, we get

where C is a constant independent of k. The map t (t) is continuous in the uniform operator topology on (0, ¥) because {S(t)} is a compact semigroup. Thus from the above, we get the equicontinuity of {Q(h_k(·))(·)}.

We now define the concept of motion. Let {x_0n} be a sequence converging to x₀ and let {(u_n(·), v_n(·))} be the sequence of nth stage outcomes corresponding to the pair of strategies (G, D). By Lemma 2.1, the sequence of nth stage trajectories {(·, x_0n, u_n(·), v_n(·))} is relatively compact in C([0, ¥); ). We define a motion to be the local uniform limit of a subsequence of a sequence of nth stage trajectories.

A motion is denoted by [·, x₀, G, D]. Let [·, x₀, G, D] denote the set of all motions corresponding to (G, D) which start from x₀. A motion [·, x₀, G, D] can be written as

Let F⁰[·, x₀, G, D], F[·,x₀, G, D] respectively denote the set of all f⁰[·, x₀, G, D] , f[·, x₀, G, D]. The set of all [t, x₀, G, D] where [·, x₀, G, D] runs over [·, x₀, G, D] is denoted by [t, x₀, G, D]. Similarly, the sets F⁰[t, x₀, G, D] and F[t, x₀, G, D] are defined. Since c > 0, for any f⁰[·] = f⁰[·, x₀, G, D], limf⁰[t] exists and is denoted by f⁰[¥, x₀, G, D]. As above, F⁰[¥, x₀, G, D] is the set of all f⁰[¥, x₀, G, D]. If the initial point of the augmented component f⁰(·) of the trajectory is not zero, then the corresponding extended trajectory is denoted by

By (·, t₀, ₀, u(·), v(·)), we mean the trajectory

Similarly, the relaxed trajectories (·, ₀, m(·), n(·)) and (·, t₀, ₀, m(·), n(·)) are defined. To complete the description of the game, we need to define the payoff. The payoff associated with the pair of strategies (G, D) is set valued and is given by

P(x₀,G, D) = F⁰[¥, x₀, G, D].

The player 1 tries to choose G so as to maximize all elements of P(x₀, G, D) and player 2 tries to choose D so as to minimize all elements of P(x₀, G, D). This gives rise to the upper and lower value functions which are respectively given by

(If {D_a} is a collection of subsets of , then sup_aD_a : = supÈ_aD_a and inf_aD_a : = infÈ_aD_a.) Therefore the upper and lower value functions are real valued functions. Clearly W⁺> W^–. If W⁺ = W^– = W, then we say that the game has a value and W is referred to as the value function.

A pair of strategies (G^*, D^*) is said to constitute a saddle point for the game starting from x₀, if for all (G, D),

P(x₀, G, D^*) <P(x₀, G^*, D^*) < P(x₀,G^*, D).

(By D₁< D₂ we mean r₁< r₂ for all (r₁, r₂) Î D₁ × D₂.) Note that if (G^*, D^*) is a saddle point, then P(x₀, G^*, D^*) is singleton and

W⁺(x₀) = W–(x₀) = P(x₀, G^*, D^*).

By a constant component strategy G^c for player 1 corresponding to the sequence {u_n(·)} of controls, we mean a strategy where, for each n, the player 1 chooses the open loop control u_n(·) at the nth stage. If u_n(·) º u(·) for all n, then this strategy is referred to as a constant strategy corresponding to the open loop control u(·). Constant component strategies and constant strategies for player 2 are defind in a similar fashion.

In view of Lemma 2.1, the following result may be obtained by modifying the arguments in [1]. Hence we omit the proof.

Lemma 2.2. Assume (A1)-(A3).

(i) Let x₀Î ,

be a constant strategy corresponding to

(·) Î and let D be any strategy for player 2. Then for any motion [·, x₀,

, D], there exists a relaxed control n(·) Î such that

Conversely, given any relaxed trajectory (·, x₀, (·), n(·)), there exists a motion [·, x₀,

, D] such that (2.4) holds.

(ii) For any 0 < t < ¥ and constant strategy

, the set È_D[t, x₀,

, D] is compact.

Analogous results hold with

, D replaced respectively by , G.

3 Dynamic programming and viscosity solution

Before proving the dynamic programming inequalities, we show the continuity properties of W⁺ and W^–. To this end, we first compare the trajectories with different initial points.

Lemma 3.1. Assume (A1) and (A2). For any a Î (0, 1] Ç(0, ), there exists C_a > 0 such that

|f⁰(t, x₀, u(·), v(·)) – f⁰(t, y₀, u(·), v(·))|< C_a||x₀ – y₀ ||^a,

for all t > 0, x₀, y₀Î and (u(·),v(·)) Î ×

.

Proof. Let ₁(·) = (·, x₀, u(·), v(·)) and ₂(·) = (·, y₀, u(·), v(·)). Obviously,

From this, it follows by using Gronwall inequality that

||f₁(t) – f₂(t)||<||x_{0 –} y₀||e^Kt.

We have

Therefore for any a Î (0,1] Ç (0, ), we obtain

Henceforth we take a Î (0,1] Ç (0, ) and C_a = .

Lemma 3.2. Assume (A1)-(A3). Let x₀, y₀Î and (G, D) a pair of strategies. Then for any motion [·, x₀, G, D], there is a motion [·, y₀, G, D] with the property that

|f⁰[¥, x₀, G, D] – f⁰[¥, y₀, G, D]| < C_a||x₀ – y₀||^a.

Proof. Consider a motion [·, x₀, G, D] and without any loss of generality let it be the local uniform limit of a sequence {(·, x_0n, u_n(·), v_n(·))} of nth stage trajectories. Let [·, y₀, G, D] be the local uniform limit of a subsequence {(·, y₀, un_k(·), vn_k(·))} of the sequence {(·, y₀, u_n(·), v_n(·))} of nth stage trajectories. From Lemma 3.1, it follows that for t > 0,

|f⁰(t, x0n_k, un_k(·), vn_k(·)) – f⁰(t, y₀, un_k(·), vn_k(·))| < C_a|| x₀n_k – y₀||^a.

Letting k ® ¥, we get

|f⁰[t, x₀,G, D] – f⁰[t, y₀, G, D]| < C_a||x₀ – y₀||^a.

The required result now follows by lettiong t ¥ in the above inequality.

Lemma 3.3. Assume (A1)-(A3). The upper and lower value functions are bounded and Holder continuous on

with exponent a Î (0,1] Ç (0, ) .

Proof. The boundedness of c gives the boundedness of W⁺ and W^–. The Holder continuity of W⁺ and W^– follow immediately from Lemma 3.2.

Having established the continuity of W⁺ and W^–, we now prove dynamic programming inequalities.

Lemma 3.4. Assume (A1)-(A3). For x₀Î and 0 < t < ¥,

Proof. Take an arbitrary (·) Î and keep it fixed. It is enough to show that

Let E₀ = { : = (t, x₀, (·), n(·)) for some n(·) Î } and > 0. For any

Therefore for each Î E₀, there exists a strategy G() such that for all D,

Let d() > 0 be such that whenever || – || < d() ,

Now E₀ is compact (by Lemma 2.2 (ii)) and the collection {B(, d()) : Î E₀} is an open cover for E₀. Let ₁, ₂, ¼, _k Î E₀ be such that E₀Ì B(_i, d(_i)) .

In order to prove (3.3), it is sufficient to construct a strategy with the property that for all D and all motions [·, x₀, , D],

We first define . Let P^¢(i) = {} be the sequence of partitions associated with G(_i); i = 1, 2,¼, k. Let be the refinement of . Let P_n be the partition of [0, ¥) such that t is a partition point, [0, t] is partitioned into n equal intervals and the interval [t, ¥) is partitioned by the translation of to the right by t. We take P = {P_n} to be the sequence of partitions associated with = {_n}. Let P_n = {0 = t₀ < t₁ < ¼}.

We now define

_n = (_n,1, _{n, 2}, ¼).

For i = 1,¼, n, define _n,i to be the map which always selects (·) on [t_i–1, t_i).

For i > n + 1 , we define _n,i as follows. Let u(·) Î [t₀, t_i-1), v(·) Î [t₀, t_i-1) and (·) = (·, x₀, u(·), v(·)). If (t) Ï B(_i, d(_i)), then we define _n,i(u(·), v(·)) to be a fixed element u₀Î U.

Let (t) Î B(_i, d(_i)) and j the least integer such that (t) Î B(_j, d(_j)). We then take _n,i to be G_n(_j) in the following sense.

Let = {0 = < < ¼}. Let (·) be the control that G_n,1(_j) selects on [, ). For any i such that n < i < n+i₁, the map _n,i selects (·) on [t_i–1 , t_i). If (·) denotes the control that G_n,2(_j) selects on [, ), then for i with n + i₁ < i < n + i₂, the map _n,i selects (·) on [t_i–1 , t_i). Now the definition of is complete.

It remains to prove (3.5). To this end, let D be any strategy for player 2, [·, x₀, , D] a motion and {(u_n(·), v_n(·))} the sequence of nth stage outcomes corresponding to (, D). Without any loss of generality, we assume that this motion is the uniform limit of a sequence {(·, x_0n, u_n(·), v_n(·))} of nth stage trajectories. Since u_n(·) = (·) on [t₀, t_n] = [0, t], := [t, x₀, , D)] Î E₀ . Let j be the smallest integer such that Î B(_j, d(_j)) and _n := (t, x_0n, u_n(·), v_n(·)). Since _n ® , for n large enough, _n Î B(_j, d(_j)).

Let D^c = {D} be the constant component strategy corresponding to {v_n(·)} with the associated sequence of partitions same as that of D restricted to [t, ¥) and translated back to [0, ¥). Therefore for large n, the pair (u_n(·), v_n(·)) is the outcome of (G_n(_j), D).

Hence by (3.4), we obtain

By arguing analogously, we can prove the next result.

Lemma 3.5 Assume (A1)-(A3). For x₀Î and 0 < t < ¥,

The strict inequality can hold in (3.2) and (3.6). The following example illustrates this fact for (3.6).

Example: = , U = V = [–1, 1], A = 0, f(x, u, v) = u + v, c(x, u, v) = c(x), a bounded, nonnegative and Lipschitz continuous function.

Note that W(x) = c(x), since H⁺(x, p) = H^–(x, p) = –c(x). Furthermore take c such that c(x) = |x| on [–2,2]. Let x₀ = 0. For 0 < t < 1, we get

Using dynamic programming inequalities, we next show that the upper (resp. lower) value function is a viscosity sub- (resp. super) solution of the HJI lower (resp. upper) equation. The HJI lower and upper equations are, respectively, the following:

where for x, p Î ,

We take the definition of viscosity solution given by Crandall and Lions in [5] and [6]. We first recall their definition of viscosity solution. To this end, let

S₀ : = {Y Î C¹() | Y is weakly sequentially lower semi-continuous and A^*DY Î C()}

₀ : = {g Î C¹() | g(x) = r(||x||) for some r Î C¹() with r^¢ > 0 }.

Definition 3.6. An upper (resp. lower) semi-continuous function W:® is called a viscosity sub-(reps. super) solution of (3.7) (resp. (3.8)) if whenever W – Y – g ( Y Î S₀, g Î ₀) has a local maximum (resp. minimum) at x Î , we have

If W Î C(

) is both a viscosity subsolution and a viscosity supersolution of an equation, then we call it a viscosity solution.

Lemma 3.7. Assume (A1)-(A3). The upper value function W⁺ is a viscosity subsolution of (3.7) and the lower value function W^– is a viscosity supersolution of (3.8).

Proof. We prove that W⁺ is a viscosity subsolution of (3.7). The other part can be proved in a similar fashion. Let Y Î S₀, g Î ₀ and let x₀ be a local maximum of W⁺ – Y – g. Without any loss of generality we assume that W⁺(x₀) = Y(x₀) and g(x₀) = 0.

We need to show that

Fix an arbitrary Î V. It is enough to show that

Let > 0. By Lemma 3.5, for each t > 0, there exists m_t(·) Î such that

We denote (·, x₀, m_t(·), ) by _t(·) =

Hence for small enough t,

This implies that for t small enough,

It can be shown that (see e.g., Lemmas 3.3, 3.4 in pp. 240-241, [10]) for t small enough,

Combining (3.10), (3.11), (3.12), (3.13) and letting t ® 0, we get

Since is arbitrary, we get the required inequality (3.9).

We next show the existence of value and characterize it as the unique viscosity solution of the associated HJI equation. To achieve this we make the following assumption.

(A0) There exists a positive symmetric linear operator B : ® and a constant c₀ such that R(B) Ì D(A^*) and (A^*+c₀I)B > I.

Let = áBx, xñ and , the class of all bounded functions W: ® with the property that for all x, y Î , |W(x) – W(y) | < w(|x – y|_B) for some modulus w. We shall prove the characterization in this class . Note that the class F is contained in the class of bounded uniformly continuous functions.

We also require the so called Isaacs min-max condition. By 'local game' at (, ) Î × , we mean the zero-sum static game, in which player 1 is the minimizer and player 2 the maximizer, with payoff x⁰ + ᖠp ,f(x, u, v)ñ – p⁰c(x, u, v). The Isaacs condition is that for each (, ) Î × , the associated local game has a saddle point. In other words, we assume that

(A4) For all (, ) Î × ,

Remark 3.8. In (A4) it is enough to take p⁰ = ±1. For proving the existence of value, we only need (A4) with p⁰ = +1. But in the next Section we want p⁰ = ±1.

Theorem 3.9. Assume (A0)-(A4). The differential game has a value and this value function is the unique viscosity solution of (3.7) (or (3.8)) in the class

.

Proof. We first show that W⁺ and W^– belong to . Boundedness of W⁺ and W^– has been proved in Lemma 3.3.

Let x₀, y₀, u(·), v(·), f₁(·), f₂(·), (·), (·) be as in Lemma 3.1. Let > 0. Since c is bounded, there exists T = T() large enough such that

It can be shown that (see e.g., Lemma 2.5, p. 233 in [10])

for some constant C. Therefore, we obtain

This implies that we can choose d = d() > 0 such that |(¥) – (¥)| < whenever |x₀ – y₀|_B < d. Hence there is a modulus w with the property that |(¥) – (¥)|< w(|x₀ – y₀|_B). Now we can mimic the arguments in Lemmas 3.2 and 3.3 to get the fact that the upper and lower value functions are in the class .

Under (A4), both (3.7) and (3.8) coincide. Therefore W⁺ and W^– are respectively sub- and super solutions of this equation (HJI equation). Now, we have the comparison result for the HJI equation in the class (see [6] and Chapter 6 in [10]). Therefore W⁺< W^–. But we always have W⁺> W^–. Hence W⁺ = W^–. The uniqueness follows from the same comparison result.

4 Saddle point

Under the Isaacs condition (A4), we prove the existence of a saddle point for the game. To achieve this we use only the dynamic programming inequalities in Section 3. We don't use the fact that the game has a value.

Fix an arbitrary x₀Î . Let r₀ = W^–(x₀) and r⁰ = W⁺(x₀). Consider the sets

Clearly (0,) Î C(r₀) Ç C(r⁰) and, by the continuity of W⁺ and W^–, the sets C(r₀) and C(r⁰) are closed.

The next two results are very crucial in constructing the optimal strategies. These results follow respectively from Lemmas 3.4 and 3.5.

Lemma 4.1. Assume (A1)-(A3). Let (t, ) Î C(r₀) and d > 0. Then for any u(·) Î [t, t + d], there exists n(·) Î [t, t + d] such that (t + d,

(t + d, t,

, u(·), n(·))) Î C(r₀).

Proof. Suppose that the result is not true. Then there exist (t, ) Î C(r₀), d > 0 and u(·) Î [t, t + d] such that for all n(·) Î [t, t + d], (t + d, (t + d, t, , u(·), n(·))) Ï C(r₀).

That is,

This together with Lemma 2.2 (ii) implies that

Applying Lemma 3.4, we obtain

x⁰ + e^–ltW^–(x) > r₀.

This contradicts the fact that (t, ) Î C(r₀).

In an analogous manner, by using Lemmas 2.2 (ii) and 3.5, we can establish the next result.

Lemma 4.2. Assume (A1)-(A3). Let (t, ) Î C(r⁰) and d > 0. Then for any v(·) Î [t, t + d], there exists m(·) Î [t, t + d] such that (t + d, (t + d, t,

, m(·), v(·))) Î C(r⁰).

We now define extremal strategies and, using Lemmas 4.1 and 4.2, show that they constitute a saddle point. Any sequence F = {F_n}, F_n : [0, ¥) × [0, ¥) × ® U, defines a strategy G = G(F) for player 1 in the following way. We take the nth stage partition P_n = {0 = t₀ < t₁ < ¼} to be the one which divides [0, ¥) into subintervals of length . G_n,1º F_n(0, ). Let j > 2, (u(·), v(·)) Î [t₀, t_j–1) × [t₀, t_j–1), and (·) = (·, x₀, u(·), v(·)). We define G_n,j(u(·), v(·)) = F_n(t_j–1, (t_j–1)).

For any sequence G = {G_n}, where G_n : [0, ¥) × [0, ¥) × ® V, a strategy D = D(G) for player 2 is defined in an analogous manner. The strategies G(F) and D(G) are referred to as feedback strategies associated with F and G respectively. The optimal strategies G_e and D_e which we define now are of this feedback form. That is, G_e = G(F_e) and D_e = D(G_e). We define the sequence G_e = {G_en} and the definition of F_e = {F_en} is similar.

Let (t, ) Î [0, ¥) × [0, ¥) × . If (t, ) Î C(r₀), then we define G_en(t, ) to be a fixed element v₀Î V. Let (t, ) Ï C(r₀) . Let C_t(r₀) = { : (t, ) Î C(r₀) } and Î C_t(r₀) be such that || – || < ( , C_t(r₀)) . We then define G_en(t, ) to be v_*, where (u_*, v_*) is a saddle point for the local game at (, – ).

The next result compares trajectories governed by two special pairs controls. The proof may be obtained by modifying the proof of the analogous finite dimensional result in [9].

Lemma 4.3. Assume (A1)-(A4). Let

,

belong to a bounded subset M of [0, ¥) ×

, t Î [0,T), m(·) Î [t,T] and n(·) Î [t,T]. Let (u_*,v_*) be a saddle point for the local game at (,

– ). Let (·) = (·, t,

, m(·), v_*), (·) = (·, t,

, u_*, n(·)), (·) = (·) – (·). Then there exists a modulus

and b > 0, depending only on M and T, such that for 0 < d < T – t,

Using Lemmas 4.1, 4.2 and 4.3, we try to establish the optimality of (G_e, D_e).

Lemma 4.4. Assume (A1)-(A4). Let G be any strategy for player 1 and let [·] = [·, x₀, G, D_e] be a motion corresponding to (G, D_e). Then for all t > 0, (t, [t]) Î C(r₀).

Proof. Let (t) = dist((t, [t]),C(r₀)) . Without any loss of generality, let [·] be the local uniform limit of the sequence {(·, x_on, u_n(·), v_n(·))} of nth stage trajectories. Let _n(·) = (·, ·, x_on, u_n(·), v_n(·)) and _n(t) = dist((t, _n(t)), C(r₀)). Clearly for each t, _n(t) ® (t) as n ® ¥. Therefore it suffices to show that for all t > 0, lim_{n ® ¥}

_n(t) = 0. Fix t > 0 and an integer N > t. We now estimate _n(t).

Let

_n = {0 = t_n,0 < t_n,1 < ¼ < t_n,Nn = N < ¼} be the nth stage partition associated with D_e. Let t Î (t_n,j, t_n,j+1], 0 < j < Nn–1. Choose Î (r₀) such that ||_n(t_n,j) – || <

_n(t_n,j). Let (u_*, v_*) be a saddle point for the local game at (_n(t_n,j), _n(t_n,j) – ). Now by Lemma 4.1, there exists n(·) Î [t_n,j, t] such that the relaxed trajectory (·) = (·, t_n,j, , u_*, n(·)) has the property that (t, (t)) Î C(r₀). Therefore

Applying Lemma 4.3, we get

Therefore

Letting n ® ¥, we get the desired result.

Similarly, we can prove the next result.

Lemma 4.5. Assume (A1)-(A4). Let D be any strategy for player 2 and let [·] = [·, x₀, G_e, D] be a motion corresponding to (G_e, D). Then for all t > 0, (t, [t]) Î C(r⁰).

Now we can show that the pair of strategies (G_e, D_e) constitute a saddle point equilibrium for the game.

Theorem 4.6. Assume (A1)-(A4). The pair (G_e, D_e) is a saddle point for the game with initial point x₀.

Proof. From Lemmas 4.4 and 4.5, it follows that for any (G, D) and motions [·, x₀, G, D _e], [·, x₀, G_e, D], we have

This holds for all t > 0. Letting t ¥, we get

Hence we obtain

The required result now follows.

5 Conclusions

We have extended the Berkovitz's framework to study infinite horizon discounted problems. In this setup, following a dynamic programming approach, we have shown that the two player zero-sum infinite dimensional differential game on the infinite horizon with discounted payoff has a value. This value function is then characterized as the unique viscosity solution of the associated HJI equation. This has been achieved by using the notion of viscosity solution proposed by Crandall-Lions in [5] and [6]. By using our dynamic programming inequalities and mimicking the arguments in [8], without using (A0), we can also characterize the value function in the class of bounded uniformly continuous functions by taking the definition of viscosity solution as in [7] which is a refinement of Tataru's notion (see [11] and [12]). In the Elliott-Kalton framework, this has been established by Kocan et. al. [8] under more general assumptions on A.

6 Acknowledgements

The author wishes to thank M.K. Ghosh for suggesting the problem and for useful discussions. The author is grateful to an anonymous referee for important comments.

Received: 04/II/03.

Accepted: 09/VI/03.

#563/03.

• [1] L.D. Berkovitz, The existence of value and saddle point in games of fixed duration, SIAM J. Control Optim., 23 (1985), 173-196. Errata and addendum, ibid, 26 (1988), 740-742.
• [2] L.D. Berkovitz, Differential games of generalized pursuit and evasion, SIAM J. Con. Optim., 24 (1986), 361-373.
• [3] L.D. Berkovitz, Differential games of survival, J. Math. Anal. Appl., 129 (1988), 493-504.
• [4] L.D. Berkovitz, Characterizations of the values of differential games, Appl. Math. Optim., 17 (1988), 177-183.
• [5] M.G. Crandall and P.L. Lions, Viscosity solutions of Hamilton-Jacobi equations in infinite dimensions, Part IV, J. Func. Anal., 90 (1990), 237-283.
• [6] M.G. Crandall and P.L. Lions, Viscosity solutions of Hamilton-Jacobi equations in infinite dimensions, Part V, J. Func. Anal., 97 (1991), 417-465.
• [7] M.G. Crandall and P.L. Lions, Viscosity solutions of Hamilton-Jacobi equations in infinite dimensions, Part VI, 'Evolution Equations, Control Theory and Biomathematics', Lecture Notes in Pure and Appl. Math., 155 (1994), Dekker, New York, 51-89.
• [8] M. Kocan, P. Soravia and A. Swiech, On differential games for infinite-dimensional systems with nonlinear unbounded operators, J. Math. Anal. Appl., 211 (1997), 395-423.
• [9] N.N. Krasovskii and A.I. Subbotin, Game-Theoretical Control Problems, Springer-Verlag, (1988).
• [10] X. Li and J. Yong, Optimal Control Theory for Infinite Dimensional Systems, Birkhauser, (1995).
• [11] D. Tataru, Viscosity solutions for Hamilton-Jacobi equations J. Math. Anal. Appl., 163 (1992), 345-392.
• [12] D. Tataru, Viscosity solutions for Hamilton-Jacobi equations with unbounded nonlinear terms: A simplified approach, J. Differential Equations, 111 (1994), 123-146.
• [13] J. Warga, Optimal Control of Differential and Functional Equations, Academic Press, (1972).

*

The author is a CSIR research fellow and the financial support from CSIR is gratefully acknowledged.

Publication Dates

• Publication in this collection
  20 July 2004
• Date of issue
  2003

History

• Accepted
  09 June 2003
• Received
  04 Feb 2003