Recurrent neural network approaches for biped walking robot based on zero-moment point criterion

Recurrent neural network approaches for biped walking robot based on zero-moment point criterion

J. B. Gonçalves^I; D. E. Zampieri^II

^IDept. of Electrical Engineering (DEE), University of Taubaté (UNITAU), Rua Daniel Danelli, s/n, 12060-440 Taubaté, SP. Brazil. E-mail: [e-mail] ^IISchool of Mechanical Engineering, Dept. of Computational Mechanics (DMC), State University of Campinas (UNICAMP), Postal Number: 6122, 13083-970 Campinas, SP. Brazil. E-mail: [e-mail]

ABSTRACT

The main objective of this paper is to use a recurrent neural network (RNN) to determine the trunk motion for a biped-walking machine, based on the zero-moment point (ZMP) criterion. ZMP criterion can be used to plan a stable gait for a biped-walking machine that has a trunk (inverted pendulum). So, a RNN is trained to determine a compensative trunk motion that makes the actual ZMP get closer to the planned ZMP. In this context, an identification scheme is presented to obtain the vector of parameters of the RNN. A first order standard back-propagation with momentum (BPM) is used to adjust free parameters for the network. Artificial neural network brings up important features for function approximation. This was the main reason to use an RNN to determine the trunk motion. The proposed scheme is simulated on a 10-degree-of-freedom biped robot. The results confirm the convergence of the proposed scheme, proving this is a new way to solve this classical problem in the biped-walking machine area.

Keywords: Zero moment point, gait synthesis, robot biped, postural stability, artificial neural network

Introduction

Biped walking robot can be classified into two major research areas. The first one considers static gait planning problems and the other, considers dynamic biped locomotion problems. The gait of a biped-walking robot is statically stable if the gravity line of its center of mass (or its ground projection of the center of mass, GCoM) falls within the convex hull of the foot-support area (support polygon). A dynamic locomotion might be considered if the GCoM goes beyond the limits of the support polygon. In this case, an uncompensated moment on the foot-support arises, causing it to rotate around a point on the polygon boundary (Goswami, 1999). Since the 80's, large quantity of research has been carried out related to dynamic biped locomotion. Lately, this research area has improved considerably, by using artificial intelligence approaches.

One can guarantee a dynamic gait if the uncompensated moment on the foot-support is controlled. An interesting way to control it is to endow a biped-walking machine with an inverted pendulum to simulate a trunk. In this case, the motion pattern of the biped-walking machine can be implemented to consider the relationship between the inertial forces and the ground-reaction force. This method is the well-known zero-moment point (ZMP) criterion. Besides, the trunk can be used to plan an arbitrary motion pattern for the lower-limbs, which satisfies some restrictions, e.g., geometry of the floor (Li, et al., 1992).

A great deal of research has been using the ZMP criterion to improve the stability of biped-walking machine that has a trunk. The algorithms used can be described as below:

1. To plan the ZMP trajectory for the biped-walking machine;

2. To plan an arbitrary motion pattern for the lower-limbs;

3. To compute the trunk motion to control the uncompensated moment on the foot-support;

4. To make the biped-walking machine to gait, according to the above steps.

According to Takanishi, A. (1989), when the walking-biped machine shows steady walking, each center of mass of the lower-limbs and the ZMP movements can be planned periodically, by considering the moving Cartesian coordinate system that is set on the center of the robot's body, in parallel to the inertial coordinate system. In this case, the solution of trunk motion has a periodic solution and can be represented as a Fourier series. Thus, using Fast Fourier Transformation (FFT) to obtain an approximated periodic solution for the trunk motion commonly solves step #3.

However, this paper has the intention of introducing a new way to compute step #3, by utilizing an RNN to obtain an approximated solution for the trunk motion, based on the ZMP criterion. Artificial neural networks bring up important features for function approximation. This was the main reason to use a recurrent neural network in this classical problem in robot field research. An identification scheme is presented to obtain the parameters vector of the RNN, which utilizes a first order standard back-propagation with momentum (BPM). After that, an RNN is trained to determine a compensative trunk motion that makes the actual ZMP get closer to the planned ZMP. This scheme is simulated on a 10-degree-of-freedom biped robot. The results confirm the convergence of the proposed scheme, proving that is a new way to solve this classical problem in the biped-walking machine area.

Nomenclature

Greek Symbols

Biped Walking Robot

The biped-walking machine can be considered a multi-body mechanical system that possesses a series of rigid bodies (named links) connected by revolute joints. This section discusses a kinematics model for the biped-walking machine, according to Bezerra (2002); a synthesis gait; and a modeling of the trunk motion.

Kinematics Model

Figure 1 shows the kinematics model for the walking-biped machine. The Cartesian coordinates systems were distributed by employing Denavit and Hartenberg's rules. In this case, links and joints are marked in a systematic way. The ankle is marked by link-1j, the lower-leg by link-2j, the upper-leg by link-3j, and the hip by link-4j, where j can be assumed 1 for the leg-1 or 2 for the leg-2. Link-5 is the width of the robot's body. The angles of the joints are marked in the same form. The exceptions are the joints of the trunk that were identified as roll (q_z) and yaw (q_y) angles. The trunk was identified as Link-0. There are no feet in this model, which has a single point to touch the ground surface (plane YZ). The inertial coordinated system is (x₀, y₀, z₀). The y-axis points forward, the x-axis points upwards, and the z-axis is the cross product of the x-axis and y-axis. It is assumed that the moving Cartesian coordinate system is placed on the center of body, which is parallel to the inertial frame.

One derives the joints' angles from ankle and hip motions to the leg-j, by applying the inverse kinematics (Shih et al., 1992). In this case, Eq. (1) shows the position vector from the ankle (p_a) to the hip (p_h).

where:

Equations (2) to (3) result from Eq. (1b).

Equations (4) to (5) result from Eq. (1a).

where:

Figure 2 shows the angles of the joints of the inverted pendulum, in spherical coordinates.

If given the Cartesian coordinates (x,y,z) of the trunk, one can compute the yaw and roll angles, as suggested by equations (6) and (7).

Equations (2) to (7) allow the computation the angles of the joints of the biped-walking machine.

Planning the Gait

The reference trajectory for the walking biped robot can be planned by a few methods. For example, minimizing some chosen variables, by adopting a reference trajectory that mimics the human's gait.

Equations (8) and (9) define the references trajectories for point P₂ and for the hip joint of the swinging leg (Juang, 2000), respectively.

where:

d is the step length (meters);

s is the height of the swinging ankle (meters);

n is the index of the sampling;

N is the total sampling number for a step;

d_j is the length of link-j (meters);

v_j is the half weight of the body (v₁ = d₄, and v₂ = -d₄).

Additionally, the following hypotheses complete the planned reference trajectory.

H₁ the center of body does not rotate during the motion, so R0 = I3x3;

H₂ the swinging ankle is set parallel to the plane surface during the motion, so R2 = I3x3.

Modeling of the Trunk Motion

Figure 3 shows a biped-walking machine in the sagittal plane. Hypothetically, ith link has a ith lumped mass (m_i), which is located by a position vector r_i, whose origin is an inertial reference frame; a_i is the second time derivative of the position vector (r_i); it also shows a moving Cartesian coordinate system that is sets on the center of the robot's body. Finally, single point E, which is the contact between the foot (by modeling a single point) and the plane surface. The ground condition is assumed to be rigid and without slip.

The biped-walking machine can be considered as a rigid system. Figure 4 shows the ith mass in a three-dimensional space. Let x_oy_oz_o be a coordinate system that is fixed in time (inertial reference frame). Point E represents a stationary reference point that is located by a position vector s. R and M denote the resulting ground reaction force and moment, respectively; f_i is the force due to gravity.

We can compute the equation for the rotational dynamic equilibrium for the biped-walking machine, by applying D'Alembert's principle (Goswami, 1999). By taking moments at origin O, the Eq. (10) can be derived without considering the tangential components.

where:

Solving Eq. (10) for the components of vector s results in:

Equations (11) and (12) describe the coordinates of point E on the ground (plane YZ). At this point, the total moment, which was generating due to gravity and inertial forces, is equal to zero.

Solution for the Trunk Motion

Introduction

When a biped-walking machine has a steady walking gait, the ZMP and lower-limbs movement can be considered periodical. Thus, the motion of the trunk can be computed to ensure no foot rotation. Figure 5 shows a possible ZMP motion. The first phase of the robot's movement is considered the "transient phase;" the robot is an upright position when beginning its movement. After that, the movement is considered the "permanent phase."

Equations (11) and (12) (Takanishi, 1989) can be solved for the trunk motion by considering a linear model, and by ensuring that the trunk (m0) does not move vertically. Also, for easier consideration the motions of the trunk and the lower-limbs of the biped-walking machine are related to a moving Cartesian coordinate system, placed in the center of the body (see Fig.3). Figure 6 shows the relationship between the moving Cartesian coordinate system and the inertial frame, where hypothetically, they are parallel.

Equation (13) shows the mathematical relationship between the moving Cartesian coordinate system and the inertial frame.

The following hypotheses are considered:

H₃ height of the trunk is maintained constant during the movement, = constant;

H₄ height of the body is maintained constant during the movement, .

These hypotheses, which permit a linear model to be obtained, imply that:

H₅ second time derivative of the height of the trunk is null,

H₆ second time derivative height of the body is null, .

By considering Eq. (13) and the hypotheses H₃ to H₆, equations (14) and (15) can be derived from equations (11) and (12), resulting the second time linear differential equations that describes the trunk motion in relation to the moving Cartesian coordinate system.

where:

Equations (14) and (15) can be solved to determine a trajectory for the trunk. However, this model has poles in the half right complex plane, therefore, the trajectory may not converge. However, when the biped-walking machine has a steady walking gait, the lower-limbs and the ZMP movements present periodically patterns in the moving Cartesian coordinate system. Therefore, k₁ and k₂ are known periodical functions, and the model may have a periodic solution as its particular solution. During many years, the solution of movement of the trunk has been represented by Fourier series, and by using Fast Fourier Transforms (FFT) approximate periodic solutions for equations (14) and (15). Apart from this, when this method is involved with repeating algorithm, one can obtain a strict periodic solution for the nonlinear equation (Yamaguchi, et. al., 1993; Takanishi, 1989).

In the next section, recurrent artificial neural network architecture is proposed, aimed at solving this classical problem in robot field research.

Recurrent Artificial Neural Network

Figure 7 shows the architecture for adjustment of the parameters of the recurrent artificial neural network (RNN). The outputs of the RNN are estimates of the coordinates of the trunk in relation to the moving Cartesian coordinate system, namely, and .

Equations (18) and (19) use the outputs of the RNN and are employed to estimate the ZMP coordinates. The x-coordinate is considered constant during the movement.

The difference between the estimated ZMP and the planned ZMP is used to adjust the free parameters (weights) of RNN, via an interactive process, called "error back-propagation algorithm." This objective is to minimize an index of performance defined in terms of the error signal e, as defined by Eq. (20).

where:

Equations (21) and (22) represent the internal model of the RNN:

where:

is the kth element of the matrix of input data.

The adjustments for the weights of the RNN are made according to Eq. (20), computed for each pattern presented to the network. Equations (23) and (24) are used to correct the free parameters.

where:

h is the learning-rate parameter;

l is the momentum term of the back-propagation algorithm.

The minus sign in equations (23) and (24) is used to accounts for the gradient descent in weight space. Equations (25) and (26) define the delta rule and the momentum update, respectively.

By using equations (14), (15), (18), (19), (20), (21) and (22) one can compute Eq. (25). Equations (27) and (28) compute the weight relative to the output of the RNN. Equations (29) and (30) compute the weight relative to the input of the RNN.

Computation Implementation and Simulation Results

Figure 8 is the flowchart of the general computation work that was implemented in this paper. All the codes were developed in Matlab^® .

Figures 9 and 10 show the normalized input of the neural network, namely k1(t) and k2(t), and the ZMP planning, respectively. Equation (31) was applied to normalize the input.

The planned ZMP had just two points: one at the beginning of the movement and another at the end of the movement. In the trained RNN, a discrete model for planned ZMP was obtained, considering that the movement is a straight line (Fig. 10). To obtain the input, it was necessary to plan the lower-limbs movement of the biped-walking machine robot. Figures 11 to Fig. 20 show the movements and accelerations in the X-direction and Y-direction (calculated on each center of mass). They were obtained by using the equations (8) and (9). The following parameters were adopted: d=65.0 10^-3 m (step length), s=20.0 10^-3 m (height of the swinging ankle) and N=25 (total of samples). Accelerations were calculated by using numerical computation, according to Eq. (32).

To compute Eq. (32) three data of each variable are necessary, e.g., the X-position at t -1, t, and t+1, are needed. At the beginning of the movement of the robot, there are only two instants of the variable-position, t and t+1. To complete the data to compute Eq. (32), the initial condition is duplicated at the beginning of the movement of the robot. Thus, after the beginning of the movement of the robot, instants: t-1, t and t+1 are stored in the memory, to compute Eq. (32).

After obtaining the normalized input and planned ZMP, the architecture for the identification process for the RNN parameters (Fig. 7) was applied. A multi-layer perceptron recurrent neural network – with biases, a sigmoid layer (hidden layer), and a linear output layer – was trained, via back propagation improved by momentum. To correct the free parameters, the sequential mode training (Von Zuben, 2000) was used. It is an iterative algorithm for the update of the weights. The values were: h=0.01 (learning-rate parameter), l=0.95 (momentum term), and x=0.003 m (project parameters). The authors adopted 40 (forty) hidden neurons, and a tapped delay line of length 1 (one). The outputs of the RNN estimate the movements of the trunk in the YZ plane, satisfying the ZMP criterion.

Figures 21 to Fig. 25 show the results during the training of the specific RNN. Figure 21 shows the norm of the gradient (upon graph); the norm of the hidden and output neurons (middle graph); and the averaged squared error (lower graph).

Figure 22 shows the trunk movements on the ground surface (upper graph), and the accelerations of the trunk (lower graph).

Figure 23 shows the ZMP required (upper graph) and the ZMP calculated from data estimated by RNN plus the preset lower legs movement (lower graph).

Figures 24 and 25 show the joint trajectories corresponding to the walking pattern in Fig. 11 to Fig.16, relative to legs one (1) and two (2), respectively.

Figure 26 shows the joint trajectories of the trunk, calculated by using Eq. (6) and Eq. (7). The estimates variables computed by Eq. (21) and Eq. (22) were used here.

Comments and Conclusions

The main objective of this paper was to develop a new way to compute the motion of the trunk (inverted pendulum) that satisfies the ZMP criterion. A recurrent neural network (RNN) approach was created. Architecture for the identification process of the parameters of the RNN was established as well as equations that govern the adjustment weights of the network. The adjustment of the weights was made according to Eq. (20), computed for each pattern presented to the network. The first order standard back propagation with momentum was used. The errors are the difference between the planned ZMP and the estimated ZMP. The criterion to stop the training of RNN was the averaged squared error, which presented a 0.009-m value after 20 epochs.

The proposed scheme was used to simulate a 10-degree-of-freedom biped robot, which is in an upright condition at the beginning of the movement. The simulation showed that RNN could be used as a solution for the trunk motion, allowing initial conditions of the experiment to be included. The convergence of the architecture was achieved.

Therefore, the reference trajectories of the joints can be obtained, in order to feed a controller.

Paper accepted December, 2002. Technical Editor: Alisson Rocha Machado

• Bezerra, C. A. D., 2002. "A biped robot machine design for walking in an unknown environment." School of Mechanical Engineering, the State University of Campinas. Doctoral thesis, (in Portuguese).
• Goswami, A., 1999. "Postural Stability of Biped Robots and the Foot-Rotation Indicator (FRI) Point." The International Journal of Robotics Research, Vol. 18, No. 6, pp. 523-533, June.
• Juang, Jih-Gau, 2000. "Fuzzy Neural Network Approaches for Robotics Gait Synthesis." IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics, Vol. 30, No 4, August 2000.
• Li, Qinghua, et al. 1992. "Learning Control of Compensative Trunk Motion for Biped Walking Robot based on ZMP Stability Criterion." In Proceedings of the 1992, IEEE/RSJ International Conference on Intelligent Robots and Systems. Raleigh, NC. July, 7-10, 1992.
• Shih, Ching-Long, et al. 1993. "Inverse Kinematics and Inverse Dynamics for Control of a Biped Walking Machine." Journal of Robotics Systems 10 (4), pp. 531-555.
• Von Zuben, F. J. 2000. Notas de Aula do curso IA353 Redes Neurais. UNICAMP/FEEC/DCA.
• Yamagushi, Jin-ichi, et. al. 1992. "Development of a biped walking robot compensative for three-axis by trunk motion." In Proceedings of the 1993, IEEE/RSJ International Conference on Intelligent Robots and Systems. Yokohama, Japan. July, 26-30, 1993.
• Takanashi, A., (1989). "Robot Biped Walking Stabilized with Trunk Motion." Proceedings of ARW on Robotics and Biological System.

Publication Dates

• Publication in this collection
  18 Mar 2004
• Date of issue
  Mar 2003

History

• Received
  Dec 2002