## Reinforcement Learning and Adaptive Optimisation of Complex Dynamic Systems and Industrial Applications

View this Special IssueResearch Article | Open Access

Kai Wan, "Convergence Analysis of Iterative Learning Control for Two Classes of 2-D Linear Discrete Fornasini–Marchesini Model", *Complexity*, vol. 2020, Article ID 6843730, 14 pages, 2020. https://doi.org/10.1155/2020/6843730

# Convergence Analysis of Iterative Learning Control for Two Classes of 2-D Linear Discrete Fornasini–Marchesini Model

**Academic Editor:**Guangchen Zhang

#### Abstract

This paper first investigates convergent property of two iterative learning control (ILC) laws for two kinds of two-dimensional linear discrete systems described by the first Fornasini–Marchesini model (2-D LDFFM with a direct transmission from inputs to outputs and 2-D LDFFM with input delay). Different from existing ILC results for 2-D LDFFM, this paper provides convergence analysis in a three-dimensional (3-D) framework. By using row scanning approach (RSA) or column scanning approach (CSA), it is theoretically proved no matter which method is adopted, perfect tracking on the desired reference surface is accomplished. In addition, linear matrix inequality (LMI) technique is utilized to computer the learning gain of the ILC controller. The effectiveness and feasibility of the designed ILC law are illustrated through numerical simulation on a practical thermal process.

#### 1. Introduction

In practical industrial applications for two-dimensional (2-D) dynamical systems, for example, in form-closure grasps, the immobilized manipulation of 2-D serial chains could be regarded as a repetitive control problem [1]. Also, mold processing and material manufacturing usually require repeated operations of the controlled processing units to obtain a high-precision reference surface [2]. With regard to this kind of repetitive tracking problem, the traditional control methods in infinite coordinate domain, such as control [3], stochastic stability analysis [4], and sliding mode control [5], are difficult to be used because the precise model information is known in advance. Clearly, for the repetitively tracking cases mentioned above, the traditional tracking control approaches for 2-D dynamical systems in infinite coordinate domain are difficult to be applicable. Clearly, for the repetitively tracking cases mentioned above, the traditional tracking control approaches for 2-D dynamical systems in infinite coordinate domain are difficult to be applicable.

To the best of our knowledge, iterative learning control (ILC), as a data-driven and unsupervised control approach, does not require accurate knowledge of the controlled system, which makes ILC be widely prevalent in practical applications. A large number of ILC research results reported in the past few decades have fundamentally designed for one-dimensional (1-D) dynamical systems [6–14], only very few results involved in 2-D dynamical systems [15–22], which concentrate on mainly 2-D linear discrete first Fornasini–Marchesini model (2-D LDFFM). An optimal ILC algorithm was proposed in [16], such that the ILC tracking error converges to zero monotonically. Unfortunately, the system parameters of 2-D LDFFM are known. In [15], five ILC algorithms were proposed, the effectiveness and feasibility of which is verified through only numerical example. Afterwards, by using the column scanning approach (CSA), the iterative process of 2-D LDFFM is transformed into a iterative process of 1D systems; rigorously, theoretical proof on convergence and robustness of a two-gain ILC law is provided in [17]. Also, using the CSA as [18], to track a class of nonrepetitive reference surface described by a high-order internal model operator (HOIM), two HOIM-based ILC laws were, respectively, investigated in [23] for 2-D LDFFM by using 2-D HOIM-based linear inequality theory, but the ultimate ILC tracking error can only converge to a bounded range. To this end, adaptive ILC approach was proposed in [21, 22] to identify all unknown system parameters of 2-D LDFFM, and the ILC result of perfect tracking on iteration-varying reference surface can be obtained. Unfortunately, the gain matrix in 2-D LDFFM is required to be positive definite, such that the proposed adaptive ILC algorithm, in practical applications, is greatly restricted. Notably, the aforementioned ILC results have concerned mainly on 2-D LDFFM without input delay.

Time delay, recognized as one of the main sources for poor performance or even instability of control systems, is frequently encountered in the signal transmission process of engineering and biological systems [24]. Over the past few years, with respect to 2-D systems with time delay on horizontal direction and vertical direction, some initial results have been reported on the states estimation [25], control and filtering [26, 27], and nonfragile robust optimal guaranteed cost control [28]. For instance, the authors in [23] have investigated the robust state estimation problem for 2-D linear discrete systems with state delay, and a robust filter for 2-D discrete systems with time delays has been designed in [23]. A delay-dependent controller for 2-D discrete state delay systems was designed in [27]. Nonfragile robust optimal guaranteed cost control subject to both state delay and input delay is discussed in [28]. However, no works have considered the ILC tracking problem for 2-D linear discrete systems with input delay, which motivates our current study.

This paper first investigates the convergence property of two ILC laws for 2-D LDFFM with a direct transmission from inputs to outputs and 2-D LDFFM with input delay to track the desired reference surface, respectively. The 2-D LDFFM is very typical, and many practical 2-D engineering systems can be represented as the form of 2-D LDFFM [29, 30]. Under a 3-D framework, a 3-D dynamical process is converted into a 2-D dynamical process by using CSA or row scanning approach (RSA), and it is theoretically proved no matter which method is used; perfect tracking on the desired reference surface is fulfilled. Additionally, this paper theoretically provides a linear matrix inequality (LMI) technique to computer the learning gain matrix of the ILC law. The main contributions of this paper are summarized as follows:(1)All of the existing ILC results for 2-D dynamical systems have not considered input delay. To the best of our knowledge, it is the first time to investigate ILC algorithms for 2-D LDFFM with a direct transmission from inputs to outputs and for 2-D LDFFM with input delay.(2)Compared with the adaptive ILC algorithm for 2-D LDFFM in [21, 22], two ILC algorithms proposed in this paper have no restriction on the numbers of system inputs and outputs.(3)Different from the existing ILC work for 2-D LDFFM [17, 18], a 3-D framework learning mechanism is presented in this paper and can reveal the dynamical behavior of the 2-D LDFFM in the horizontal dynamical direction, vertical dynamical direction, and iteration direction.(4)It is theoretically proved that RSA or CSA can guarantee perfect tracking on the desired reference surface.(5)Compared with the literature [20], this paper has addressed the ILC issue on nonidentical boundary states. In addition, this paper successfully avoids strong Assumption 2.2 on desired reference surface and Assumption 2.3 on the system matrix, and LMI technique is utilized to computer the learning gain of the ILC controller.

The rest of this paper is arranged as follows: Section 2 presents problem formulation and some preliminaries. Sections 3 and 4 provide convergence analysis of a symmetrical P-type ILC law and extend to 2-D LDFFM with input delay, respectively. Two simulation examples are introduced in Section 5. Finally, a conclusion on this article is drawn in Section 6. Notations: () denotes positive-definite (negative-definite) matrix. represents with positive integer , and represents . denotes identity matrix with dimension . represents spectral radius of matrix.

#### 2. Problem Formulation and Some Preliminaries

Consider the following 2-D LDFFM with a direct transmission from inputs to outputs [23], executing designated tracking tasks repetitively over dynamical region and :where , , and represent, respectively, control input, system state, and control output; , , , , , and are real matrices with appropriate dimensions. denotes the -th repetitive number (or iteration number) of 2-D LDFFM (1); and are, respectively, discrete indexes along the horizontal dynamical direction and vertical dynamical direction. The independent indexes and in practical 2-D LDFFM, i.e., chemical reactors, heater exchangers, and pipe furnaces, usually represent space locations and time instants, respectively [31]. As , 2-D LDFFM (1) has been investigated in [17, 18].

For and , let the desired trajectory surface and the corresponding tracking error at th iteration, respectively, be denoted as and , and the control objective of ILC for 2-D LDFFM is to update the control input iteratively, such that the system output converges to desired reference surface asymptotically, i.e.,

For the convenience of discussing the ILC problem for 2-D LDFFM, the property on a block matrix, assumptions, and Lemma is provided as follows.

*Property 1. *(see [32]). Give the following block matrix with a nonsingular matrix :and the inverse matrix of which is given aswithIn particular, as , becomes

*Property 2. *(see [32]). For in Property 1, according to matrix theory, there is , which is irrelevant to , , …, and .

*Remark 1. *Properties 1 and 2 are often used in the ILC convergence analysis for 2-D linear discrete dynamical systems to track reference surface described by a high-order internal model or varying uniformly between -th iteration and -th iteration [18]. Certainly, Properties 1 and 2 are also suitable to track repetitive reference surface considered in this paper.

*Assumption 1. *The matrix is full-row rank.

*Assumption 2. *The boundary states and of 2-D LDFFM (1) are assumed to satisfy , and , , where and are iteration-invariant functions with respective to and .

Lemma 1. *Consider the following 3-D linear discrete system for , , and :where and , respectively, are state and control input; and denote real matrices. Suppose the boundary state , where is a bounded vector function. When , if is satisfied, then the 3-D system is asymptotically stable along the dynamical direction , i.e.,**In particular, when , for and , can still be obtained for and . The proof process of Lemma 1 is shown in Appendix.*

#### 3. Convergence Analysis of a Symmetrical P-Type ILC Law

The following symmetrical P-type ILC law is applied to 2-D LDFFM (1) for and :where . The learning gain is to be designed.

*Remark 2. *It is worth noting that 2-D LDFFM (1) is a zero relative degree system; the proposed ILC law (9) is extended on the existing ILC work for 1D linear discrete dynamical systems with zero relative degree [33]. Additionally, 2-D LDFFM (1) and the ILC law (9) are symmetrical on the discrete indexes and ; therefore, row scanning approach (RSA) or column scanning approach (CSA) can guarantee the same ILC tracking result, which is demonstrated in Theorem 1 and Appendix.

Theorem 1. *Consider the 2-D LDFFM with a direct transmission from inputs to outputs (1) under Assumptions 1 and 2 and use the symmetrical P-type ILC law (9). If the learning gain satisfiesthen the tracking error converges to zero, i.e.,*

*Proof 1. *For and , letUsing (1), for and , there isApplying the ILC law (9) into (13), it generatesOn the contrary, according to and (1), we havewhere and . Substituting the ILC law (9) into (15), it yieldsRearranging (16), there isLetwhere . According to (18) and (19), using the CSA on horizontal dynamical direction , (14) can be rewritten aswhereThen, since is a nonsingular matrix, premultiplying on both sides of (20), we getwhere and . Similarly, (17) can also rewritten aswhereFrom Assumption 2, we obtain for . Thus, (22) and (23) can be reformulated aswhere . In (25), based on Assumption 2, we deduceand is bounded for . Using Lemma 4 in [18], if (equivalently, , see Property 2), there isAdditionally, taking in (17), we havewhere . From (27) and , we obtain , . Using Lemma 1, if is satisfied, there isFrom (27) and (29) and the definition on in (19), it yieldsTheorem 1 is completed.

*Remark 3. *The convergence condition in Theorem 1 only depends on the matrix and is irrelevant to , , , , and , which is in accordance with the ILC convergence condition of the following 1D linear discrete iterative system to track repetitive reference trajectory [33]:under a P-type ILC law and initial iterative state .

*Remark 4. *Taking advantage of the estimated information on , the learning gain matrix of can be determined by using LMI technique. For the following 1D linear discrete dynamical system described by ,Asymptotically stable condition is satisfied if and only if there exists a such thatUsing Schur complement theorem [34], it yieldsPre- and postmultiplying by on both sides of (34), we haveTherefore, is equivalent to solving the following LMI: there exists positive-definite matrix and matrix such thatwith . Therefore, the selection problem of the learning gain matrix is transformed into the feasibility problem on solving LMI (36).

*Remark 5. *Theorem 1 employs the CSA on dynamical direction to obtain ILC result under Assumption 2. As the RSA on dynamical direction is applied to 2-D LDFFM (1) and the ILC law (9), we can still acquire the same ILC result with Theorem 1 due to the symmetrical characteristics on and in 2-D LDFFM (1) and the ILC law (9), and see Appendix.

#### 4. Extension to 2-D LDFFM with Input Delay

In this section, under Assumption 2, we investigate the ILC issues for 2-D LDFFM with input delay [23], which is described in the following for and :where and are two positive integers. , , and have been described in (1).

*Remark 6. *It is worthwhile noting that ILC tracking issues for 2-D LDFFM with and in (37) have been investigated in [17, 18]. It is the first time that ILC issues for 2-D LDFFM with input delay are investigated in this paper.

*Assumption 3. *The matrix is full-row rank.

Next, we discuss the convergence property of the ILC law for 2-D LDFFM with input delay (37) under Assumptions 2 and 3, and there is the following Theorem 2.

Theorem 2. *Consider the 2-D LDFFM with input delay (37) under Assumptions 2 and 3 and use the following ILC law for and :**If the learning gain satisfiesthen the tracking error converges to zero, i.e.,*

*Proof 2. *For and , letFrom (37), there iswhere and . Inserting the ILC law (38) into (42), it generatesOn the contrary, according to and using (37), we havewhere and . Letwhere . From (45) and (46), (43) can be rewritten aswhereand and have been described in (20). Similarly, (44) can also be reformulated aswhereThen, since is a nonsingular matrix, premultiplying on both sides of (47), we getwhere and . Substituting (51) into (49), it yieldswhere . From Assumption 2, we obtain for . Thus, for , (51) and (52) can be represented asIn (53), based on Assumption 2, we obtain . Furthermore,and is bounded for due to the boundedness property of . Using Lemma 4 in [18], if holds (equivalently, , see Remark 7), there isFrom (46), we obtainTheorem 2 is completed.

*Remark 7. *The convergence condition in (53) depends on a complicated matrix ; thus, there is a need to simplify it. From the definition of and using Property 1, the simplifying process is shown as follows:From (57) and Property 2, we can know . Consequently, can be replaced by a more convenient condition without using any knowledge on the system matrix and the interval length in the 2-D LDFFM with input delay (37).

*Remark 8. *In Remark 7, using the estimated information on and , the learning gain matrix of can be decided by using the LMI technique. Therefore, a similar Remark 4 can be obtained.

*Remark 9. *When the RSA on dynamical direction is applied to 2-D LDFFM with input delay (37) and the ILC law (38), the same ILC result with Theorem 2 can be obtained, the proof process of which is shown in Appendix 7.3.

*Remark 10. *It is noted that 2-D LDFFM (1) and (37) contain 2-D Attasi model as a special case, i.e., . And the 2-D Roesser model under some specified coefficient matrix requirements can also be converted into 2-D LDFFM [29]. Consequently, depending on the ILC results obtained from 2-D LDFFM (1) and (37), it is easy to extend 2-D Roesser model and 2-D Attasi model with input delay.

#### 5. Illustrative Examples

To show the effectiveness and feasibility of the proposed ILC algorithms (9) and (38) for the 2-D LDFFM with a direct transmission from inputs to outputs (1) and 2-D LDFFM with input delay (37), two simulation examples are introduced in this section.

*Example 1. *Consider the following 2-D LDFFM with a direct transmission from inputs to outputs over dynamical region and :which is required to repetitively track a desired reference surface for and shown in Figure 1. Under Assumption 2, the boundary states of 2-D LDFFM (58) are given as , , and , . Using the proposed ILC law (9) with the initial control input for and , the gain matrix is selected as , which satisfies the convergence condition in Theorem 1. In order to evaluate the accuracy of ILC tracking, the following maximum absolute tracking error indexis used. As a result, the ILC tracking error surfaces at , and at are depicted in Figures 2 and 3, respectively. Also, Figure 4 presents the profiles of the ILC tracking error indexes and in iteration domain by utilizing the ILC law (9). Obviously, it is observed from Figures 2–4 that the convergent effect of the ILC law (9) against the iteration-invariant boundary states is illustrated.

**(a)**

**(b)**

**(a)**

**(b)**

**(c)**

**(d)**

**(a)**

**(b)**

**(c)**

**(d)**

**(a)**

**(b)**

*Example 2. *Some practical thermal processes can be represented by the following partial differential equation (PDE) [31]:where and , respectively, denote space delay and time delay; , , , and are real coefficients. The repetitive index implies that the ILC issue is investigated in the example. Through the discretization approximation on (60), system parameters and delay factors of 2-D LDFFM with input delay (37) are given as follows:where is shown in Figure 5. And the boundary states satisfy , , and , . Under the ILC law (38) with initial control input for and , let the learning gain be selected as and compute , which satisfies the convergent condition in Theorem 2. The maximum absolute tracking error index is adopted to evaluate the ILC tracking performance, which is given as follows:And Figure 6 displays the profile of with iteration number by using the ILC law (38). The tracking error surface at is shown in Figure 7. Apparently, perfect tracking on repetitive reference surface can be observed from Figures 6 and 7.

**(a)**

**(b)**

**(c)**

**(d)**

#### 6. Conclusions

To date, ILC tracking issues for 2-D LDFFM with a direct transmission from inputs to outputs and with input delay are first concerned in this paper. Different from existing ILC results for 2-D LDFFM, a 3-D analysis approach is first applied to the convergent property of the proposed ILC law, which supplies a new tool for follow-up ILC development for 2-D systems. Additionally, the LMI technique is used to compute the learning gain. In our future work, reinforcement learning (RL) [35–38] will be incorporated into the ILC designs for 2-D dynamical systems such that more ILC results are made.

#### Appendix

*Proof of Lemma 1. *The solution to is given as follows:Then, taking the limit on both sides of (A.1), there isUnder in Lemma 1, if holds, we obtainLemma 1 is completed.

*Proof of Theorem 1 by using the RSA. *LetFrom (A.4) and (A.5), using the RSA on vertical direction , for , (14) can be rewritten aswhere