Wireless Communication Networks between Distributed Autonomous Systems Using Self-Tuning Extremum Control

Transcription

1 AIAA Conference <br>and<br>aiaa Unmanned...Unlimited Conference 6-9 April 2009, Seattle, Washington AIAA Wireless Communication Networs between Distributed Autonomous Systems Using Self-Tuning Extremum Control Deo-Jin Lee 1, Khim Kam 2, Isaac Kaminer 3, Douglas Horner 4, Anthony Healey 5, Sean Kragelund 6 Klas Andersson 7, and Kevin Jones 8 Naval Postgraduate School, Monterey, CA, This paper represents an effective optimization approach for building high bandwidth wireless communication networs between distributed autonomous systems using unmanned aerial vehicles as airborne relay nodes. A self-tuning extremum control technique is developed to find an optimal location of the aerial vehicles to provide maximum communication throughputs. The idea behind the self-tuning control is to use an on-line gradient estimator to identify the derivative of a cost function and to use this as an input to a gradient-based hill-climbing algorithm. The on-line estimation of the gradient of a performance function is achieved by utilizing a perturbation-based pee-seeing approach which provides a quantitative gradient value of the cost function in a numerical way. Flight experiments are conducted to evaluate the performance of the proposed airborne wireless sensor networing control algorithm. I. Introduction N recent years, unmanned aerial vehicles (UAVs) have received lots of attention in both military and civil Iapplications. 1 This is because of the advances in wireless communication technologies and embedded autopilot systems that integrate a micro processor and sensors with fast computational power. Due to these technological advances, a collection of interconnected distributed wireless networs of sensing and computing devices with multimodal capabilities, referred to as wireless sensor networs (WSNs), can be deployed for a wide variety of real-time monitoring and control applications. 2-5 Furthermore, in a wireless networ-centric environment, teams of heterogeneous autonomous vehicles can be deployed in a cooperative manner to conduct wide-area sensing, surveillance, and communication relay in various environments and for a broad range of applications. 5,6 Coordinated autonomous operations by teams of heterogeneous vehicles such as aerial, surface, and underwater robots will increase the functionality of distributed sensing for shared situational awareness. Under this operational concept, unmanned aerial vehicles play an important role of long-range sensors as well as communication relay nodes. However, cooperative operations between multiple autonomous vehicles using UAVs as sensing and relaying agents are constrained by sensor ranges, communication limits, and operational environments. 6 Stable communications between a distributed autonomous system (DAS) of networed vehicles and sensing nodes as well as autonomy of the unmanned vehicles will be ey technologies for high-performance and remote operation in these applications. A challenging tas for successful networ communications in a DAS which uses UAVs as airborne sensing and relay nodes is optimizing the real-time flight trajectory to extend the communications range during coordinated autonomous operations. 6 The concept of communication relay using UAVs was proposed in the literature 7 where the UAVs are used as platforms for a high capacity trun radio relay and battlefield broadcast systems. More research has been conducted on this type of communication networs. 6~11 While most recent research deals with theoretical concepts, few in- 1 NRC Research Associate/Adjunct Research Professor, Center for Autonomous Vehicle Research, Dept. of Mech. & Astronautical Engineering, member AIAA, djlee@nps.edu/apollo17@gmail.com. 2 Research Assistant, Dept. of Mech. & Astronautical Engineering, student member AIAA. 3 Professor, Center for Autonomous Vehicle Research, Dept. of Mech. & Astronautical Engineering member AIAA. 4 Director, Center for Autonomous Vehicle Research, Dept. of Mech. & Astronautical Engineering. 5 Distinguished Professor, Center for Autonomous Vehicle Research, Dept. of Mech. & Astronautical Engineering. 6 Research Associate, Center for Autonomous Vehicle Research, Dept. of Mech. & Astronautical Engineering. 7 Research Associate, Dept. of Mech. & Astronautical Engineering, member AIAA. 8 Research Associate Professor, Dept. of Mech. & Astronautical Engineering, Associate Fellow AIAA. 1 Copyright 2009 by by the authors. Published by the, Inc., with permission.

2 flight experiments have successfully demonstrated this capability with real-time autonomous control of aerial vehicles. Frew and his colleagues 9,10 have conducted research on this topic and developed a Lyapunov guidance vector field (LGVF) based control algorithm that taes gradient inputs from a perturbation-based extremum seeing approach in order to control the UAV positioning to optimize communication lins. The operational capability of unmanned vehicles is usually limited by the range of communication systems. To increase and allow stable real-time transmission of data bac to the command control station, it is necessary to develop a real-time robust control algorithm that will force the networed autonomous vehicles to reposition themselves autonomously to maintain an optimal loitering flight path. This flight path will maximize the signal throughput of the communication lins between the heterogeneous vehicles and the ground control station. The overall concept of the sensor networ using aerial vehicles as relay nodes is illustrated in Fig. 1. The main objective of this paper is to develop autopilot guidance and control algorithms that will allow UAVs to adopt and maintain an optimal loitering posture that ensures the best high-bandwidth communications lin between a command center and multiple survey vehicles or vehicles in the operating area. The UAV motion control is a complex optimization problem with constraints imposed from the geometry between the UAV and the wireless nodes. In this paper, a selftuning extremum control technique is developed to find an optimal location of the UAV which provides maximum communication throughputs. To accomplish this goal, two efficient methods are proposed. The first approach focuses on developing a communication propagation model which generates signal-to-noise ratio (SNR) values of the communication performance function being used as reference communication signal inputs for controlling the UAVs. A sophisticated signal-to-noise ratio model is developed, which incorporates free-space propagation loss, antenna pattern loss and UAV ban angle effects in order to analyze their impact on the communication channel s signal strength. 12 This model can enhance the reliability of a guidance algorithm by fusing model-based SNR values and observed SNR data from the onboard radio communication hardware. Moreover, this model maes it possible to estimate an overall lin budget by predicting the received power, the SNR and lin margin of a receiver. The second strategy includes the development of a self-adaptive extremum control algorithm which sees an optimum location of the UAV for maximum communication throughput by embedding the signals obtained from the communication model into the real-time feedbac controller. The idea behind self-estimating extremum control is to use an on-line gradient estimator to identify the derivative of the cost function and to use this as an input to the gradient-based hillclimbing algorithm. The on-line estimation of the gradient of a performance function is achieved by utilizing an extremum-seeing approach 13 which does not require an analytical expression for the gradient of the cost function but provides a quantitative gradient value of the cost function in a numerical way to drive the set point of a dynamic system to an optimal one. The proposed self-estimating control structure is analogous to a self-tuning adaptive controller architecture, except that the recursive estimator is replaced by an extremum-seeing approach to provide estimates of the gradient of a performance function. 14 The objective function in the optimal control is the value of a signal-to-noise ratio (SNR) which is a function of the location of the sensor nodes, the UAV position, and the attitude of the UAV. Figure 1 Concept of Wireless Sensor Networing Using Unmanned Aerial Vehicles as Relay Nodes 2

3 In this paper, a theoretical technique which continuously steers the UAV position to the optimal location is proposed. In addition, flight test results are presented to verify the effectiveness of the proposed methods and provide a unique distinction from other approaches in this sensor networing research area. The performance of the proposed self-tuning extremum controller is evaluated via flight tests conducted within the USSOCOM-sponsored Tactical Networ Topologies (TNT) Cooperative Field Experimentation Program 12 to measure the received signal strength of the wireless lins sensed by each networ node. For flight tests, we have utilized commercially-available broadband mobile ad-hoc networing (MANET) equipment for both stationary and mobile nodes within a localized wireless infrastructure. The remainder of this paper is organized as follows. In Section II a signal-to-noise ratio (SNR) mathematical model is described. Section III describes the development of the self-estimating control approach including the overview of the extremum-seeing approach. Section IV represents the UAV flight systems. Section V presents flight test results. Conclusion and discussion is presented in section VI. II. Antenna Path Loss Modeling In most communication networ systems, it is desirable to maximize the received signal power. The lin quality of the communication networ is dependent on the received signal power as well as the noise level of the system. Signal-to-noise ratio (SNR) is defined as a measure of the ratio between the received power and the noise power. 15 The higher the SNR, the larger the signal level as compared to the noise level, which results in a lower bit error rate in signal reception and means better lin quality. According to the Shannon-Hartley theorem 16, the theoretical maximum channel capacity (C, in bits per second) is proportional to the SNR and the bandwidth (W, Hz) of the channel as C = W log (1 + SNR) (1) 2 Assuming that the capacity of the channel is fully utilized, an increase in SNR will lead to an increase in the throughput of the channel. There are various sources of noise in the communication system which can come from the natural environment, system devices, and movement and orientation of the transmitter and receiver. Generally the mechanisms behind electromagnetic wave propagation are diverse, but can be attributed to direct line-of-sight (LOS) path, deflection, reflection, and scattering. 15 In this section, a communication propagation model is developed to predict the signal-to-noise ratio (SNR) of the communication lins, which is used as a reference SNR signal for the inputs to a feedbac controller that is based on a real-time extremum-seeing approach. The propagation model is also used to analyze the variation of free-space propagation loss, antenna pattern loss, and the effect of UAV orientation on the signal-to-noise ratio in the communication lin. First, the free space path loss, which represents signal attenuation as a positive value measured in db, is modeled as the difference between the transmitted power and the received power. The formula is based on Friis transmission formula 16, which is one of most dominant path loss features affecting wave propagation in the radio channel. For this model development, let us assume that p ( t) = [ x( t), y( t), z( t)] T presents the UAV trajectory resolved in the local tangent coordinates (East, North, Up) as ( ) () x t = vcos ψ ( t) y t = vsin ψ ( t) ψ () t = vκ (2) where v is the speed of the UAV, ψ ( t) is the heading command, and κ is a bounded curvature. The control inputs can be the heading and the speed, but in this wor it is assumed that the speed is constant, v = constant. Thus the UAV is controlled by commanded heading rate, ucom () t = ψ () t. The relation of the heading and the ban angle is represented by 3

4 2 1 = φ() t tan v gr v g 1 = tan ψ ( t) (3) where R is the radius of curvature where the speed and the heading rate are related by v/ R = ψ. The heading is defined as the heading of the UAV with respect to the positive x-axis. The path loss model computes the path loss in db between the UAV and ground relay nodes. The model for the path loss formula is based on Friis transmission formula and is expressed by 15 where f is a frequency in MHz and dt ( ) is distance in m. L ( db) = log( f ) + 20log( d( t)) (4) p ( node, i ) ( node, i ) ( node, i ) dt () = xt () x + yt () y + zt () z (5) z y AC AB θ x Figure 2 Geometry for Antenna Pattern Loss Computation Generally, the antenna gains are varying as a function of the azimuth and elevation of an antenna. The attenuation due to the antenna pattern effect can be computed by doing a ray-tracing of the wave propagation path between the transmitter and the receiver to determine the angle of departure and arrival of the ray-path. To tae into account the signal attenuation of the antenna pattern loss between the UAV receiver and the ground relay nodes an antenna pattern loss model is derived. The equations to compute the antenna pattern loss are computed by using the geometry shown in Fig. 2. θ () t = tan i 1 ( zt () znode, i ) ( xt () xnode, i ) + ( yt () ynode, i ) 2 2 (6) The angle of arrival is defined as the angle between the incident ray and the horizontal wing of a UAV. This model assumes that the antenna of the ground relay node is pointing at the UAV and there is no antenna pattern loss for the transmitter. Hence the departure angle is not used. 12 4

5 UAV straight and level (no ban) Ban angle moves the incident ray toward the vertical and increases the angle of arrival Ban angle moves the incident ray toward the horizontal and decreases the angle of arrival. Figure 3 Arrival angle with respect to the transmitter antenna Fig. 3 describes the geometric effect of the arrival angle due to the ban angle of the UAV. In the case of head-on or tail-chase scenarios, the antenna pattern will not be affected by the ban angle while the effect of the ban angle is the worst in the crossing scenario (UAV is perpendicular to the ground relay node). In this model, the effect of the ban angle on the antenna pattern is accounted for by a sin term as shown below. ϕi () t is the bearing angle defined as the angle between the positive x-axis and the ground projection (AB) of the direct line-of-sight (LOS) ray path yt () y 1 node, i ϕi () t = tan xt () x node, i (7) The arrival angle γ ( t) is approximated by i ( ) γ () t = θ () t φ()sin t ϕ () t ψ() t (8) i i i where φ () t is the ban angle and ψ () t is the heading angle of the UAV. Using a loo-up table, the antenna pattern loss ( L ap ) can be determined by using the above relations. For example, Fig. 4 illustrates the effect of the arrival angle on the antenna pattern loss. If the arrival angle is 30 deg, the pattern loss will be -3dB from the loo-up table. Figure 4 Antenna Pattern Loss in the Horizontal and Vertical Planes The lin budget model computes the received power, the signal-to-noise ratio (SNR) and lin margin of the receiver. The equations for the lin budget are given by 15 P ( dbm) = P ( dbm) + G ( db) + G ( db) L ( db) L ( db) (9) r t t r p ap 5

6 Lin Margin( dbm) = P ( dbm) R ( dbm) (10) r sen Pr ( dbm) λ 2 GG t SNR( dbm) = = ( ) P ( dbm) 4π d L n ap r (11) where Pr ( dbm ) is the receiver power, Pt ( dbm ) is the transmitter power (28dBm), Rsen ( dbm ) is the receiver sensitivity (-74 dbm), Gt ( db ) is transmitter antenna gain (14 db), Gr ( db ) is receiver antenna gain (2.2 db), 2 Lp ( db) (4 π d / λ) is path loss which denotes the loss associated with propagation of electromagnetic waves, Lap ( db ) is antenna pattern loss, Pn ( db ) is noise power (-95 dbm), Psen ( dbm ) is receiver sensitivity (-74 dbm), d is the relative distance between the UAV and the sensor node, λ = c/ f where f is the transmission frequency in 8 Hz and c = 3 10 m/s. 15 The received signal strength can be roughly characterized by direct propagated signal and the sum of reflection, diffraction, and scattering subsides. III. Self-Tuning Extremum Controller for UAV Location Optimization In this paper, a self-estimating extremum control algorithm is developed, which allows the aerial relay vehicles to reposition themselves autonomously to maintain an optimal loitering flight path that maximizes the quality of the communication lin between a command station and a remote user vehicle. The control algorithm for the optimal localization of a small UAV is developed by integrating a hill-climbing extremum control with a derivative-free gradient estimation algorithm which numerically computes the on-line gradient values of the SNR cost function as the figure of merit. The overall structure of the proposed self-estimating extremum controller is illustrated in Fig. 5. The on-line estimation of the gradient of a performance function is achieved by utilizing an extremum-seeing technique 13, which is a derivative-free recursive estimator that does not require specific nowledge of a mathematical model of a performance function. The aim of the self-tuning control algorithms is to build a robust controller combined with on-line gradient-seeing identifier. The technical motivation is that analytical gradienttype optimizers can t be applied to the problems where there is no nowledge of the derivatives of the system performance index. θ * J J y = J ( θ ) * θ θ θˆ S ˆ θ ˆ d θ = α dt ( J ) θ α J θ J Figure 5 Self-Estimating Extremum Control Architecture This section first outlines the basic theory of extremum control and discusses the implementation of the on-line gradient estimator, a perturbation-based extremum-seeing technique. A. Gradient Descent Extremum Control Algorithms The principle behind the gradient based optimization methods lies in the fact that an extremum has a gradient with the magnitude of zero. 17 To reach the objective, it is necessary for an objective function to be a smooth function with nown parameters, and if noise is present in the cost function then it is necessary to have a good estimate of the 6

7 gradient by applying a filter. On the other hand, if a mathematical model of the cost function is not available, the gradient computation is not trivial in optimization control applications. As an alternative way of computing the gradient of a cost function, the extremum-seeing approach that provides quantitative gradient value of the cost function in a numerical way is applied. In this section, a hill-climbing extremum control is reviewed. Assume that the nonlinear dynamic and measurement model is given by x = + 1 f( x, u) y = J ( x ) (12) n where x R is the n-dimensional state, given : n n l f R R, u R is the control input, and y R is a scalar n measurement cost function with J : R R. Based on the measurement of the system state, and cost function values, the pea-seeing controller is expected to regulate the state as guided by the search sequence, and in turn minimizes the performance output. Then, the pea-seeing control problem is interpreted as ( ) min J ( ) subject to + 1 =, x D x x f x u (13) Consider a gradient based search method such as the steepest descent approach. Each iteration of a search loop computes a direction of the state. The search provides the following 18 1 α α ( ) J ( ) 1 where ( ) J ( ) x = x + d = x B H x x (14) + 1 d BH x x decides the direction of the search and is required to be a descent/ascent direction which allows the cost function J to be either reduced or increased gradually along the direction, and α > 0 is the step length along the direction d and is a positive value which decides the convergence speed. B is a suitable 2 H x J x, and B = I is for the gradient method. It is noted that approximation of the Hessian matrix ( ) ( ) T this method is called a gradient-lie method since it leads to ( ) ( ) 2 J = J 2 method uses the following direction 18 = J ( ) + β 1 d x x. While the conjugate gradient d x d (15) where β is a scalar to be selected in such a way that the directions d turn out to be mutually orthogonal with respect to a suitable scalar product. In the steepest descent direction, the descent direction is decided by d = J( x ) and for the ascent direction, d = J ( x ) denoted by J J J ( x) = ( x),, ( x) (16) x1, x n, T The direction is a descent direction if the following is satisfied ( ) J( ) J ( x ) T d J x < 0 if x 0 T d = 0 if = 0 (17) Different choices of the direction vector d correspond to different methods. For the Gauss-Newton method, the direction is computed by ( ) J ( ) d = H x x (18) 1 7

8 where H is a positive definite Hessian matrix given by 2 J h ( x ) = ( x ), i, j = 1,, n (19) ij xi, xj, The gradient method or steepest descent method sets the gradient direction to be ( ) d = B J x (20) 1 After the direction d is computed, it is necessary to find an optimal step length α that can give an adequate reduction of computational time. A method for computing the step size α consists of the following onedimensional minimization problem It is not feasible to find an optimal step length α> 0 α> 0 ( α ) min φα ( ) = min J x + d (21) α that needs minimization of φ except for in special cases. 18 Thus, in this paper, practical strategies for performing an inexact line search to identify a step length are introduced. The first method adopted is the Armijo condition that prevents steps that are too long via a sufficient decrease criterion 19 J( x + α d ) J( x ) + cα d J( x ) (22) T 1 Also, the Wolfe condition which restricts steps that are too short via a curvature criterion can be used d J( x + α d ) c d J( x ) (23) T T 2 where the values of the parametric constants are set to 0< c1 < c2 < 1, and this ensures that acceptable points exist. 19 To reach the objective it is assumed that the objective function is nown and should be a smooth function with nown parameters. If the cost function contains noise, a filter can implemented to produce a good estimate of the gradient. On the other hand, if a mathematical model of the cost function is not available, the gradient computation is not trivial in dynamic optimization applications. For computing the gradient of a cost function, an extremumseeing approach 13 that provides quantitative gradient values of the cost function in a numerical way can be applied. θ J y = J ( θ ) θ θˆ S y L ˆ θ wl S + w L y H S S + w H J acos( ωt) sin ( ωt) Figure 6 Real-Time Gradient Estimation Using Pea-Seeing Approach 8

9 B. On-Line Gradient Estimation Using Pea-Seeing Approach Now consider a way of estimating the gradient of a cost function. The maximization of the communication lins between heterogeneous unmanned vehicles could be accomplished by maximizing the signal-to-noise ratio (SNR) between them. However, the function of the SNR value is not nown in general, and thus it is difficult to apply directly an optimization technique, which is based on an analytical expression of the gradient of the cost function, to find an optimal location for the relay UAV. As promising solutions to the optimization without nowledge of the cost functional model, a perturbation-based extremum seeing control technique 13 is adopted, which is suggested originally by Leblanc 17 and has been further extended by Kristic 13 and Speyer 20 providing stability and convergence of extremum-seeing control approaches. The perturbation-based extremum-seeing control is an adaptive optimization algorithm designed to drive the set point of a dynamic system to an optimal one, which provides quantitative gradient value of the cost function in a numerical way as inputs to the gradient descent based controller. The typical extremum-seeing control structure is described in Fig. 6 where the high-pass filter plays a role of taing the gradient of the cost function and gives the rate of change of the cost function, and the low-pass filter taes out high frequency noise terms from the cost signal. For a further detailed illustration, suppose a general map function ( ) y = J θ (24) where θ R is an adjustable parameter, and y R is the performance output, and J : R Ris an extremum around the extremum value θ * * R. The objective of an extreme seeing mechanism is to steer θ to θ in realtime such that the cost function reaches the extremum J * = J( θ * ). In order to see the extremum, the cost function is perturbed locally around the current value of the parameter θ with a sinusoidal periodic signal, and uses the corresponding change of the objective output to estimate its local gradient. The gradient estimate is then used to update the parameter. If ˆ θ is assumed to be the current value of the parameter, and asinw t is a small sinusoidal perturbation around ˆ θ, then the output of the objective function is expressed by 13 ( ˆ ) ( ˆ J θ sin θ) ˆ y = J + a wt J + a sinw t (25) θ θ = θ The constant term in the output y is removed by applying a high-pass filter (HPF) as a differentiator J yh a sinw t (26) θ θ = ˆ θ Demodulating y H with a sinusoidal signal sinw t divides the signal into a low-frequency signal and a highfrequency signal 1 J 1 J ς = a a cos 2w t (27) 2 θ θ= ˆ θ 2 θ θ= ˆ θ Low-pass filtering of the demodulated the signal, ς, provides an estimate of the local gradient of ( ) J θ 1 J y L a (28) 2 θ θ = ˆ θ The estimated gradient can be expressed in terms of the parameter change and ˆ 1 J θ = a 2 θ θ = ˆ θ (29) 9

10 where is a parameter to be selected. Since the objective function is assumed to be a smooth quadratic type function with an extremum value around the set point, it can be expressed by Then, the gradient can be approximated locally around 1 ( ) ( ) ( )( ˆ J θ = J θ + J θ θ θ ) 2 (30) 2 * θ as J J ˆ = J θ θ θ= ˆ θ ( θ θ ) (31) * Denote θ = ˆ θ θ the convergence error, and taing a derivative of the error leads to θ = ˆ θ. Then the differential equation of the error can be computed by 1 θ aj ( θ ) θ 2 (32) which becomes stable with a proper choice of the parameters, a and, i.e., aj ( θ ) 0 <. 13 Thus, the error θ converges asymptotically to the extremum point θ * *, at least locally around θ. The criterion for selecting the perturbation frequency is that it should be sufficiently higher than the cut-off frequencies of the low-pass and highpass filters used in estimating the cost gradient. C. UAV Heading Controller Design Now consider designing a self-estimating extremum optimizer which regulates the UAV to reach the optimal location by integrating the gradient-based hill climbing optimization algorithm in Eq. (14) with the gradient estimator of the extremum seeing approach in Eq. (29). The previously explained gradient estimation of a performance function is achieved by utilizing the extremum-seeing approach, which is a numerical estimator and does not require specific nowledge of a mathematical model of a performance function. The variations of the SNR performance function are a nonlinear function of several variables such as the relative distance between the UAV and the remote node and attitude of the UAV in flight. In this paper, for simplicity, it is assumed that the UAV has a constant speed with constant level flight ( ht () = 0, where h is the altitude above the mean sea level) as shown in the UAV motion model described in Eq. (2). In this case, the heading angle or ban angle is the only control variable, and the commanded control input is only a heading rate command, ucom () t = vκ = ψ com () t. If it is assumed that the ascent direction of q is equal to q = J ( x ), then this gradient computation requires 2-dimensional gradient calculation. Now, using the fact that the components of the UAV position vector is an implicit function of the heading angle based on Eq. (2), x() t = f1( ψ(), t ) y () t = f2( ψ() t ), the cost function in the optimization can be written as an implicit function of the heading angle only, J ( x( t), y ( t), h( t)) = J( ψ ( t)), which reduces the dimension of the gradient calculation to a scalar parameter. Specifically, following the gradient-based steepest descent approach, the descent direction of d becomes equal to d = J( x) = J / ψ, and the gradient of the cost function is obtained by using the numerical extremum-seeing approach instead of applying a direct analytical derivation method. Then the heading control can be decided by 1 utilizing the gradient method by replacing the general state vector with the heading angle x ψ R as ψ = 1 ψ + α + J (33) ψ where α is the step-length parameters which can be either constant or time-varying, and it is assumed that the gradient term, Jψ = J / ψ R, can be obtainable from the pea-seeing approach explained in the previous subsection. Now rearranging the above equation gives 10

11 ψ 1 ψ = + α J ψ = α J ψ ψ (34) Assuming the variation of the heading angle and the cost function at each time is small, and taing the derivative of the variation terms on both sides of Eq. (34) leads to the following relation dψ () t d = α() t ( Jψ ) (35) dt dt Now, suppose that the characteristics of the figure of merit of the cost function is quadratic in terms of the heading angle variable, then the performance function can be expressed by μ J ( ψˆ() t ) = J * + ( ψˆ() t ψ * ) 2 + w() t (36) 2 * where J is the maximum attainable value of the cost function, ψ * is the heading angle which maximizes the performance function, ψˆ( t) is the current heading angle estimate, and μ is the sensitivity of the quadratic curve which relates heading angle to the indicated SNR, and wt () is a zero-mean white noise term which can be filtered out by applying a low-pass filter. It is assumed that the parameters which characterize the optimum values are unnown, but constant parameters. Taing a gradient of the cost function with respect to the current estimate ψ ˆ( t) provides the following ( ψˆ () t ) J Jψˆ () t = μ ψ ψˆ () t * ( ˆ () t ψ ) (37) Taing a time derivative of the above gradient term again leads to d = (38) dt ( J ˆ ()) ( ˆ ψ t μψ() t ) Finally, substituting Eq. (38) into Eq. (35) gives the heading-rate control input as ψ com dψ () t d () t = = α() t J dt dt = μα( t) ψˆ ( t) ( ψ ) (39) Note that the rate of the estimate of the current heading angle ψˆ () t can be obtained from Eq. (29) after applying the low-pass filter in the process of the extremum-seeing loop. We showed that the real-time heading-rate command for the control input to the autopilot of the UAV is calculated by integrating Eq. (39) with the heading-rate term produced from the extremum-seeing approach. At the final stage of the extremum control approach when the UAV reaches the optimal location, it is necessary to mae the UAV fly around the set point rather than fly directly to the point or pass over the point. Thus a steadystate heading ψ ss is introduced to guarantee that the UAV will orbit with a constant radius R ss at the final stage. The heading-rate command is expressed by ψ = ( ) ˆ com ψ ss + μα t ψ ( t) (40) where ψ is a steady-state heading input to be selected and is related to a final approach circle radius, R = v/ ψ. ss ss ss 11

12 The time rate of change of the estimated heading angle is provided from the extremum seeing stage. Finally, the control input ut ( ) ψ ( t) to the UAV is expressed by com ψ com () t = ψ ss if ψ com () t ψ ss = v / Rss εss ut () = ψ () () ˆ com t = ψ ss + μα t ψ () t other (41) where ε ss is a criterion which guarantees the bounded motion of the UAV at the final stage. This heading control input regulates the UAV system to follow the ascending direction of the cost function value until the UAV reaches the maximum point of the cost function. Once the UAV gets close to the optimal set point, it switches to a steadystate heading control mode to orbit around the optimal point with a predefined constant radius. D. Adaptive Convergence Control The fixed step gradient descent algorithm can be improved by applying the Armijo-Wolfe conditions in Eqs. (22) and (23). However, the computation of the adaptive time step α is based on the assumption that it is necessary for the cost function to be a sufficient decrease criterion. If the cost function is subject to unnown disturbances or noises, the computation of the gradient is no longer trivial. In this section, instead of calculating the condition directly, the adaptive time-step scaling factor α is computed by using a more intuitive method based on the concept of the Armijo-Wolfe conditions. If the time rate of the gradient value of the SNR cost function Δ J + 1 = J+ 1 J or d( Jψˆ () t )/ dt is greater than τ tv : a specified threshold value, the time-step scale factor is ept either the same as the previous one or scaled up by multiplying a third parameter γ 1, ΔJ + 1 τ tv, α+ 1 = α γ. On the other hand, if the change of the gradient value of the SNR is smaller than the threshold value, the time-step is scaled down by multiplying a third scale factor 0< γ < 1, Δ J + 1 < τ tv, α+ 1 = α γ, which allows the UAV to have an almost straight flight path. In this way, the adaptive time-step α is computed by α 0< γ < 1, if Δ J > τtv τtv = γα, where γ 1, else Δ J + 1 < (42) where ΔJ + 1 J+ 1 J. This algorithm not only provides fast convergence properties but also reduces the unnecessary repeated circular motion of the UAV, which results from a direct searching mode to the optimal location. E. Distributed Objective Functions In order to obtain the gradient for the heading controller, it is necessary to define a cost function that is an input to the extremum-seeing step. With a single communication node, the cost function will be the SNR model itself used in Eq. (11), J = SNR. For multiple communication nodes, however, the cost function is defined with interactions between the multi-nodes, and it is necessary to satisfy constraints. A straight forward method to define a figure of merit for the cost function is to calculate an average value by summing each SNR function, multiplied by a proper weight value W i i J N = WSNR, and W = 1 (43) opt i i i i= 1 i= 1 N However, it is rather infeasible to calculate the weights which also can be affected by uncertain environments. Therefore, the optimal cost function is modeled by using a logarithmic function in terms of the inverse of each signal power, 1/ SNR i 12

13 J opt N 1 = η log i= 1 SNR i 1 1 = η log + + SNR1 SNR N (44) where the parameter η is a shaping scale parameter that must be selected by the user. Suppose we have two communication nodes (i, j) with a single relaying UAV and they are all in a linear networ such that a node can send data to the next neighbor node. Then, two relative cost functions are defined by J = SNR ( d ), J = SNR ( d ) (45) iu, iu, iu, u, j u, j u, j where Jiu, is the SNR between the i ground node and UAV, which is a function of the relative distance d il, between the node and the UAV. Similarly, the cost function J u, j is the SNR between the UAV and the j ground node. Then, the input cost function of the extremum-seeing control for the UAV motion control is calculated by ( iu u j) J = min J, J (46) cost,, where min ( i ) denotes a function which computes a minimum value at each time step. IV. UAV Flight Systems A. Unmanned Aerial Platform A Rascal UAV as a rapid flight test prototyping system (RTFPS) for small unmanned aerial vehicles (SUAVs) has been developed at the Center for Autonomous Vehicle Research in the Naval Postgraduate School, and it is utilized for hardware-in-the-loop simulation and flight test experimentation. 21 The new RTFPS integrated avionics system architecture that includes all the principal components along with the Piccolo plus autopilot 22, and an overhead view of the avionics are shown in Fig. 7. Figure 7 Hardware integration and implementation architecture The Piccolo plus autopilot is used for primary flight control, with its dedicated 900MHz serial data lin. In the integrated flight system, two PC computers are integrated. One computer is used for a tas of guidance, navigation and control (GNC), and the other plays the role of a gateway computer to bridge an onboard LAN and a wireless mesh networ. The latter is provided by a Persistent Systems Wave Relay router 24 as shown in Fig 8, which is optimized for mobile ad-hoc broadband networing. A Pelco NET300T video server 25 is used to stream the analog video feed from the camera, and all networ devices are lined through a Linsys 5-port hub. Analog image taen from a CCD camera 26 are transmitted through the PelcoNet video server (NET300T) across an Ethernet networ integrated in the SUAV system to a ground control center. The NET300T can display the video on a PC through any 13

14 Web browser. The integrated avionics pacage weighs 1 lb, and requires about 25 W power with all components active. Figure 8 Wave-Relay QUAD Radio Router B. Data Acquisition and Management System In addition to a 900 MHz wireless lin dedicated to the safe operation of multiple UAVs from the ground control station, a second wireless communication lin, the Wave Relay 2.4 GHz mesh networ, was added to the aerial vehicle. 12 The radio is connected to a 3 db omni-directional antenna mounted on the belly of the aerial robot for transmitting and receiving messages. Currently, the onboard PC-104 computer on the UAV is unable to extract the SNR value directly from the Wave Relay s broadcast management message. As such, the SNR value is extracted using a Linux machine located at the ground station and sent to the local host computer prior to transmission to the onboard computer. The information flow for the SNR from the ground PC to the UAV is illustrated in Fig 9. There is a switch on the local host computer which controls the type of SNR reading sent to the UAV. Currently, the switch can select one of the following three types of SNR inputs: 1) a model-based SNR, generated using the mathematical SNR model based on the current position and orientation of the UAV, 2) the actual SNR measurements computed by the Wave Relay radios, and 3) SNR estimate based on the mathematical SNR model and updated by the actual SNR reading from the Wave Relay radios. Figure 9 Flight Test Systems for Extremum Control based on SNR Data 14

15 A. Flight Test for SNR Model Verification V. Flight Test Results The SNR model verification flight test with the Rascal UAV was conducted at McMillan Air field in Camp Roberts, California on August 1, The primary objective was to collect SNR measurements between the Rascal UAV and a ground control station (GCS) node at different locations and altitudes. The SNR measurements obtained from the flight test were used to validate the mathematical SNR model. The set-up for the flight test was as follows; the ground control station was located at (latitude) and (longitude) with three (9 db) verticallypolarized sector antennas, and a 2.2 db omni-directional antenna was installed on the belly of the Rascal UAV. The flight test data were obtained from two separate sources. The SNR data were obtained by the SNR logger from the Wave Relay networ while the UAV flight parameters (such as latitude/longitude, altitude, roll, yaw and pitch) were obtained from the Piccolo onboard the Rascal UAV. These two sources of data were synchronized using the time stamp found on both data sets. The position, heading, and ban angle information of the Rascal UAV were used to drive the mathematical Rascal SNR model. Figure 10 shows the UAV trajectories from the GCS node. In Fig. 11 the simulated 2D SNR map is shown where the blue line indicates a strong SNR. From these plots it is seen that the signal strength is inversely proportional to the radial distance between the UAV and the GCS at ranges far away from the GCS (paths A, B, C & D of the flight trajectory). Next, we tested for throughput limitations in the GCS-UAV lin (directional 30 dbi antenna) by changing altitude up to 300 m. The primary objective was to collect detailed signal strength measurements for a ground vehicle as sensed by the UAV. The Racal UAV flew a 0.5 ilometer radius in the local tangent coordinates (east-north-up) above the ground control center (unamplified, 3 dbi antenna). Fig. 12 shows the Rascal signal strength recorded on the ground at different altitudes where a 3D plot of the SNR strength map of the aerial vehicle with respect to a GCS located at the origin (0 m, 0 m) is provided. It is shown that the throughput is subject to the variation of the altitude of the UAV and there is a certain altitude limit which provides a minimum strength and this is related to the departure angle of the beam of the ground antenna. Figure 13 shows the error of the simulated model with respect to the actual SNR test data in this region. The maximum errors obtained from the proposed mathematical model and the flight test were within 15%, and the SNR values are close to each other. This test result indicates that the mathematical model represents signal characteristics well enough as well as environmental effects around the UAV and the nodes. This mathematical model can be refined by using system identification techniques by utilizing the real-flight test data. Figure 10 Relative Location of the GCS node and Remote Node at McMillan Air Field in Camp Roberts 15

16 Figure 11 Simulated 2D SNR Map between the Rascal UAV and the GCS Node along the Rascal Flight Trajectory Figure 12 Simulated 3D SNR Map between the Rascal UAV and the GCS Node along the Rascal Flight Trajectory 16

17 Figure 13 Relative SNR Errors (in %) between Simulated SNR and Actual Flight Data from Wave Relay B. Flight Test for Self-Tuning Extremum Controller The communication lins flight test was conducted as part of the TNT experimentation program (TNT-09-01) at McMillan Air field in Camp Roberts, California on November 20, The two transmitting ground nodes depicted in Fig. 14 acted as the command station and the survey vehicle while the UAV functioned as the relay vehicle. The primary objective of this flight test was to validate the designed onboard adaptive optimization algorithm that would drive the Rascal UAV to an optimal loitering flight path and maximize the SNR between the two nodes and the UAV. The overall setup for these flight tests is described already in Fig. 9. The aerial vehicle is equipped with 2.2 db omni-directional antenna and the location of the sensor nodes and terrain of the air field is shown in Fig 14. Table 1 and 2 show the detailed information of the communication nodes used in the flight test. Wave Relay Node No. Antenna Position Altitude Table 1 Ground Control Station (GCS) Node 3 x 9 db vertically polarized sector antenna (SA ) (lat), (longitude) 273m (above mean sea level) Wave Relay Node No. Antenna Position Altitude Table 2 Remote Node 3 x 9 db vertically polarized sector antenna (SA ) (latitude), (longitude) 310m (above mean sea level) 17

18 Figure 14 Relative Location of the GCS node and Remote Node [From Google Earth] Figure 15 shows the flight trajectory of the UAV during the test where the circular lines in the figure are contour lines of constant SNR generated from the static SNR map in east-north coordinates for a stationary (non-dynamic) UAV with fixed altitude, heading, and ban angle. Initially, the UAV was in a holding pattern orbiting north of the GCS node. When the control algorithm was activated, the UAV started to move in the direction of the steepest increase in the SNR value. When the UAV reached the region of pea SNR, a steady-state heading command was passed to mae the UAV orbit around the optimal point. It was observed that the orbit around the optimal point was elongated and not a perfect circular path. Figures 16 and 17 show the actual and simulated SNR variations as a function of time between the UAV (sender) and GCS node (receiver) and between the remote node (sender) and UAV (receiver), respectively. The difference between the observed and simulated SNR values is small, and the maximum error range between them is less than 5 db. As expected, the algorithm drove the UAV to fly towards the optimal region that gave the best possible SNR (~23 db) between the two nodes. The results indicate that the self-estimating extremum control is robust even to low signal-to-noise ratio (SNR) values and to unexpected noises or disturbances. Figure 18 shows the error between the actual data and simulated one from the SNR model. It is seen that the simulated SNR compared nicely with the actual SNR data from the flight test. The error of the simulated model with respect to the actual SNR data was within 15% except for the holding area where the error was more than 30%. The holding area is near the GCS node and the model is not accurate at this near range. Hence, the simulated SNR reading between the GCS and UAV tends to show a large error in the holding area. 18

19 Figure 15 UAV Flight Trajectory over Static SNR Countour Map with Two Transmitting Ground Nodes Figure 16 SNR Variation as a Function of Time between UAV (sender) and GCS Node (receiver) 19

20 Figure 17 SNR Variation as a Function of Time between the Romote Node (sender) and UAV (receiver) Figure 18 SNR Error (in %) between Simulated Reading and Actual Flight Test Data 20

21 VI. Conclusions This paper has represented an effective self-tuning optimization approach for building high bandwidth wireless communication networs between distributed autonomous systems using a gradient-descent based self-tuning extremum control technique. A mathematical propagation model was developed to predict the lin quality (SNR, received power, and lin budget) of a communication lin between the UAV and ground nodes. The model was then used to generate simulated SNR measurements for the extremum control algorithm. The proposed real-time selftuning extremum control algorithms which drive the UAV to reposition itself autonomously in order to maintain an optimal loitering flight path to maximize communication throughput between two transmitting ground nodes were validated through actual flight tests. These flight experiments successfully demonstrated the effectiveness of the self-adaptive extremum control algorithm for controlling the UAV to the optimal flight path using the model-based SNR signals. It turns out that the performance of the proposed self-estimating extremum control is robust even to low signal-to-noise ratio observations. Acnowledgments This wor was supported by USSOCOM under the NPS-SOCOM TNT cooperative and the Office of Naval Research (ONR). The research of the first author is supported by the National Research Council Associateship tenured at the Center for Autonomous Vehicle Research at the Naval Postgraduate School. References 1 D. A. Schoenwald, AUVs: In Space, Air, Water, and on the Ground, IEEE Control Systems Magazine, vol. 20, no. 6, pp , Dec C.-Y. Chong and S. P. Kumar, Sensor Networs: Evolution, Opportunities, and Challenges, Proceedings of the IEEE, vol. 91, no. 8, pp , Aug I J. Cortes, S. Marinez, T. Karatas, F. Bullo, Coverage Control for Mobile Sensing Networs, IEEE Transactions on Robotics and Automation, vol. 20, no. 2, pp , April B Sinopoli, C. Sharp, L. Schenato, S. Schaffert, and S. S. Sastry, Distributed Control Applications Within Sensor Networs, Proceedings of the IEEE, vol. 91, no. 8, pp , Aug P. Ö gren, E. Fiorelli, and N. E. Leonard, Cooperative Control of Mobile Sensor Networs: Adaptive Gradient Climbing in a Distributed Environment, IEEE Transactions on Automatic Contro, vol. 49, no. 8, pp , Aug D. P. Horner, and A. J. Healey, Use of Artificial Potential Fields for UAV Guidance and Optimization of WLAN Communications, in Proceedings of the 2004 IEEE/EOS Autonomous Underwater Vehicles Conference, Maine, pp , June Maj. F. J. Pinney, D. Hampel, and S. DiPierro, Unmanned Aerial Vehicle Communications Relay, in Proceedings of the IEEE, pp.47-51, P. Basu, J. Redi, and V. Shurbanov, Coordianted Flocing of UAVs for Improved Connection of Mobile Ground Nodes, in Proceedings of the IEEE Military Communications Conference, pp , Cory R. Dixon and Eric W. Frew, Cooperative Electronic Chaining Using Small Unmanned Aircraft, AIAA 2007 Conference and Exhibit, E. W. Frew, T. X. Brown, Airborne Communication Networs for Small Unmanned Aircraft Systems, Proceedings of the IEEE, vol. 96, no.12, Dec P Zhan, K. Yu, and A. Lee Swindlehurst, Wireless Relay Communication Using an Unmanned Aerial Vehicle, IEEE 7 th Worshop on Signal Processing Advances in Wireless Communications, K. Y. Kam, High Bandwidth Communications Lins between Heterogeneous Autonomous Vehicles Using Sensor Networ Modeling and Extremum Control Approaches, M.S. Thesis, Dept. of Mechanical and Astronautical Engineering, Naval Postgraduate School, Dec K. B. Ariyur and M. Krstic, Real-Time Optimization by Extremum Seeing Control, Hoboen, NJ, John Wiley & Sons, Inc., P. G. Scotson and P. E. Wellstead, Self-Tuning Optimization of Spar Ignition Automative Engines, IEEE Control Systems Magazine, vol. 10, no. 3, pp , April T. S. Rappaport, Wireless Communications: Principles and Practice, 2nd ed.upper Saddle River, N.J: Prentice Hall PTR, C. E. Shannon, Communication In The Presence Of Noise, Proceedings of the IEEE, vol. 86, pp , J. Sternby, A Review of Exremum Control, Technical Report, Department of Automatic Control, Lund Institute of Technology, April A. Quarteroni, R. Sacco, and F. Saleri, Numerical Mathematics, New Yor, NY, Springer-Verlag, Inc., C. Zhang and R. Ordonez, Numerical Optimization-Based Extremum Seeing Control with Application to ABS Design, IEEE Transactions on Automatic Control, vol. 52, no. 3, pp , March