Distributed Synchronization in Wireless Networks

Transcription

1 [ Osvaldo Simeone, Umbero Spagnolini, Yeheskel Bar-Ness, and Seven H. Srogaz ] [Global synchronizaion via local connecions] Disribued Synchronizaion in Wireless Neworks DIGIAL VISION & PHOODISC Alarge number of applicaions in disribued (sensor or ad hoc) wireless neworks is enabled by, or benefi from, he availabiliy of a common ime scale among he paricipaing nodes. Examples range from he racking of moving objecs via sensor neworks o coordinaed medium access conrol (MAC) or cooperaive ransmission. Achieving and mainaining synchronizaion in such scenarios poses new challenges in erms of scalabiliy and energy efficiency and offers new opporuniies hrough he inerplay wih specific disribued esimaion/deecion applicaions. In his conex, an ineresing soluion, which is currenly being invesigaed, is provided by disribued synchronizaion schemes based on he exchange of local ime informaion among neighboring nodes a he physical layer (e.g., via ransmission of a rain of common waveforms ha follows he local clock). his aricle presens a survey of curren research on disribued synchronizaion for decenralized wireless neworks and illusraes he role of signal processing herein, wih emphasis on physical layer-based synchronizaion schemes. he opic is inroduced by racing is origin in he parallel invesigaions carried ou independenly in mahemaical biology and communicaion heory. Available models are discussed and compared. Open problems, such as he rade-off of complexiy versus accuracy and faul olerance, are oulined and some soluions are provided using ools from signal processing and algebraic graph heory. Available analyical resuls are also repored, along wih numerical examples ha corroborae he main conclusions, lending evidence o some ineresing phenomena, such as small-world effecs on disribued synchronizaion. he close relaionship beween disribued synchronizaion and disribued esimaion/deecion applicaions is discussed as well, showing he synergy beween hese wo problems. Finally, synchronizaion of nonperiodic signals (chaos) is briefly ouched upon for compleeness and for is (debaed) applicabiliy o poin-o-poin wireless sysems. Digial Objec Idenifier 0.09/MSP /08/$ IEEE IEEE SIGNAL PROCESSING MAGAZINE [8] SEPEMBER 2008

2 INRODUCION Synchronizaion refers o he process of achieving and mainaining coordinaion among independen local clocks via he exchange of local ime informaion. Differen synchronizaion schemes differ in he way such informaion is encoded, exchanged, and processed by he clocks oward he end of overcoming he unavoidable nuisance effecs of inaccurae clocks and propagaion/processing delays. Wireless communicaions provide he naural plaform for he exchange of local ime informaion beween synchronizing clocks. Conversely, synchronizaion of local clocks enables a wealh of signal processing and communicaion applicaions in wireless neworks. I is his muual link beween synchronizaion and wireless neworks, wih emphasis on decenralized srucures such as ad hoc and sensor neworks, ha consiues he main subjec of his aricle. A BRIEF HISORY OF MUUAL IME SYNCHRONIZAION By he end of he 9h cenury, synchronizaion of a disan clock o a reference ime, also referred o as unidirecional or maserslave synchronizaion, became a sandard engineering procedure hanks, firs, o he advances in elegraphy and, laer, wireless ransmission. Railroad ransporaion, geodesy (measuremen of longiude), and localizaion were jus a few of he applicaions enabled by his pervasive new echnology. Synchronizaion of a pair of disan clocks easily qualifies as an early signal processing problem in he conex of wired or wireless communicaions: esimae he ime offse beween wo clocks from measuremens affeced by propagaion delays and random hardware (and human) imperfecions []. he idea of synchronized ime spurred an inense debae in physics and philosophy ha evenually produced Einsein s heory of relaiviy []. In his regard, i is ineresing o quoe H. Poincaré: Simulaneously is a convenion, nohing more han he coordinaion of clocks by a cross exchange of elecromagneic signals aking ino accoun he ransi ime of he signal. Node Node 2 Node 2 Asynchronous Medium Access Conrol [FIG] An applicaion of synchronizaion in wireless neworks: coordinaed (synchronous) medium access conrol improves specral/energy efficiency wih respec o asynchronous soluions by avoiding collisions and idle periods. 2 3 Synchronous Medium Access Conrol (DMA) 2 3 In he years following hese effors, scieniss wondered a he evidence of synchronizaion among disribued periodic evens in a number of naural phenomena. As lae as 96, Joy Adamson wroe in awe of he observaion of he synchronous flashing of fireflies [2]: a grea bel of ligh, some en fee wide, formed by housands upon housands of fireflies...he fluorescen band composed of hese iny organisms lighs up and goes ou wih a precision ha is perfecly synchronized, and one is lef wondering wha means of communicaions hey possess which enables hem o coordinaed heir shining as hough conrolled by a mechanical device. Oher ypical examples of sponaneous synchrony are he aciviy of individual fibers in hear muscles o produce he hearbea or he synchronous hand clapping in a concer hall [2], [3]. Analyical modeling of he dynamic esablishmen of synchrony, even in he simples absracions of such scenarios, challenged mahemaical biologiss for decades and culminaed in he landmark work by Winfree in 967 [4] and laer Kuramoo [5]. I was followed by more recen analyses of Crawford (see [6] and [7] for reviews and references), Mirollo, and Srogaz [8]. In parallel, he communicaions communiy sared developing a heory of disribued synchronizaion for elecommunicaions neworks in he 960s in order o suppor he deploymen of digial swiching in he elephone nework. his work led o he heory developed by Lindsey e al. [9]. DISRIBUED SYNCHRONIZAION IN WIRELESS NEWORKS In he area of wireless neworks, cellular elephony has monopolized he aenion of researchers and indusry for many years. In his radiional infrasrucured scenario, synchronizaion of mobile saions can be achieved by exploiing a maser-slave srucure wih he base saion broadcasing a beacon or raining signal. Applicaions encompass scheduling a he MAC layer and coheren ransmission/recepion a he physical layer. Disribued synchronizaion based on wireless communicaions Collision plays a minor role in his conex and has been considered for frame iming synchronizaion among base saions in [0]. More recenly, disribued wireless nework srucures, such as ad hoc, sensor, or vehicular neworks, have sared o arac significan ineres. In such scenarios, he availabiliy of a common ime scale, or of synchronized local oscillaors, enables a number of unique funcionaliies a differen layers of he proocol sack. Some represenaive examples are: signal processing applicaions: daa fusion of ime-sensiive measuremens in disribued esimaion and racking for monioring or surveillance based on sensor neworks [] IEEE SIGNAL PROCESSING MAGAZINE [82] SEPEMBER 2008

3 specral and energy-efficien neworking: coordinaed MAC schemes such as ime division muliple access or varians, which overcome he shorcomings of collision-based schemes in erms of bandwidh efficiency [2] (see Figure ); energy-efficien MAC ha explois sleep scheduling [3] cooperaive ransmission: collaboraive ransmission hrough space-ime coding [4], which requires muual ime synchronizaion (also referred o as disribued synchronizaion hroughou he aricle) or disribued beamforming, which demands muual carrier synchronizaion [5]. If a fixed or mobile access poin is available whose ransmission radius covers he enire nework (e.g., a mobile fusion cener in sensor neworks [7]), hen nework synchronizaion can be achieved in principle by having he access poin broadcas a beacon iming signal, as in cellular neworks. his possibiliy is, for insance, enabled in he IEEE sandard for sensor neworks (and associaed commercial aciciy in he ZigBee alliance) [2]. Moreover, in an oudoor environmen wih loose consrains on he energy consumpion (such as vehicular neworks), saellie-based synchronizaion can be employed. However, in his aricle we focus on fully disribued scenarios where no such possibiliies exis, hus making disribued synchronizaion (as opposed o maser-slave poin-o-poin synchronizaion via a cenral node) he only available soluion [6]. As in he problem of synchronizing wo disan clocks hrough elecric signals ackled in he lae 9h cenury, muual synchronizaion in disribued wireless neworks hinges on he exchange of local ime informaion beween pairs of nodes. Common complicaions of boh problems are: i) he presence of random delays beween ransmission and recepion of a iming signal, which depends no only on propagaion bu also on he ineviable processing laency a boh sides of he link, and ii) hardware and clock inaccuracies. However, disribued synchronizaion in wireless neworks provides a unique se of challenges for boh design and analysis, which call for a variey of ools from signal processing, auomaic conrol, and algebraic graph heory, jus o menion a few. On he one hand, designing muual synchronizaion in wireless neworks requires o accoun for he following specific issues: Uncoupled Clocks: Node Node 2 Node 3 Frequency Synchronizaion: Node Node 2 Node 3 Node Node 2 Node 3 Energy efficiency: In he presence of baery-powered nodes, he rade-off beween energy consumpion and nework performance becomes an essenial meri crierion [2]. Scalabiliy: Cerain disribued neworks, such as microsensor neworks, are envisaged o be composed of a large number of nodes, in which case well-behaved scaling performance of synchronizaion is a criical issue [8]. Applicaion specificiy: In sensor neworks, performance is defined in erms of applicaion-specific crieria [9], hus rendering he design of muual synchronizaion and he given signal processing funcionaliy horoughly inerwined [20] [22]. On he oher hand, as discussed below, an analysis of he sysem ofen requires consideraion of he dynamic behavior of a (n ) (n) (n+) 2 (n ) 2 (n) 2 (n+) 3 (n ) 3 (n) 3 (n+) [FIG2] Clocks i (n) for N = 3 nodes in he case of: (a) uncoupled nodes; (b) frequencysynchronous nodes wih common frequency /; (c) fully synchronized nodes. (a) (n ) (n) (n+) 3 (n ) 2 (n ) 2 (n) 2 (n+) 3 (n) (b) Full (Frequency and Phase) Synchronizaion: (n ) (n) (n+) 2 (n ) 2 (n) 2 (n+) (n ) 3 (n) 3 (n+) (c) 3 (n+) 2 IEEE SIGNAL PROCESSING MAGAZINE [83] SEPEMBER 2008

4 possibly large se of coupled oscillaors, which calls for he sabiliy analysis of a sysem of coupled linear or nonlinear equaions. his is generally an involved ask, especially in he presence of deerminisic or random nuisance parameers. PACKE-COUPLING VERSUS PULSE-COUPLING FOR MUUAL SYNCHRONIZAION IN WIRELESS NEWORKS For he ime being, we focus he discussion on ime synchronizaion for is pracicaliy and wide range of applicaions in disribued wireless neworks, posponing he discussion on muual carrier synchronizaion and is echnological challenges o a laer secion. o illusrae he problem, we consider Figure 2 and define i (n) as he ime of he nh ick (n = 0,, 2,...) of he ih clock (i =, 2,...,N, where N is he oal number of nodes). In Figure 2, he clock a each node is represened by a periodic rain of pulses corresponding o ime insans i (n). In case nodes are uncoupled, i.e., no local iming informaion is exchanged, he clocks remain asynchronous wih generally differen local periods i (n) i (n ) = i, and phases i (n) [Figure 2(a)]. On he conrary, if we allow each node, such as he ih, o gaher informaion abou he relaive ime offses j (n) i (n) wih respec o a subse of he oher nodes ( j i), a synchronized sae migh be evenually achieved [Figure 2(b) and (c)]. Noice ha he way his ime offse informaion j (n) i (n) is exchanged and processed disinguishes differen Node Node 2 [FIG3] A graphic illusraion of he signal ransmied by N = 3 nodes for pulse-coupled clocks: each node sends a rain of waveforms g() (on a dedicaed bandwidh or on an overlay sysem) for every ick of he local clock. 5 (n) 5 2 Node 2 g( 4 (n) q 54 ) 2 2 g( i (n)) (n ) (n) (n+) 2 (n ) 2 (n) 2 (n+) (n ) 3 (n) 3 (n+) y 5 (n,) [FIG4] A skech of he signal received by he fifh node ou of a se of N = 5 pulse-coupled nodes in he nh period of is local clock. 5 (n) 4 (n)+q 54 (n)+q 5 3 (n)+q 53 2 (n)+q 52 synchronizaion echniques. We say ha a condiion of frequency synchronizaion o a common frequency / is achieved if he local periods i (n) i (n ) = are he same for all clocks [Figure 2(b)], whereas full synchronizaion is aained if clocks ick a he same imes, i.e., i (n) = j (n), i j [Figure 2(c)]. In he secion Clocks and Synchronizaion, we furher specify and elaborae on hese conceps. Differen approaches o muual ime synchronizaion are classified according o he mechanism adoped for compuing and processing local ime differences j (n) i (n) wihin he nework. In paricular, wo main families of echniques have been considered. radiional mehods based on packe coupling prescribe he periodic exchange of packes carrying ime samps ha conain he local ime j (n) a he sender, hrough eiher poin-o-poin or broadcas connecions [6]. he main sources of errors for packe-based echniques are he random delays associaed wih he consrucion of a packe, queuing a he MAC layer, propagaion, and processing of he packe a he receiver side. In fac, hese delays imply ha node i acually receives he iming packe from a node j a ime j (n) + q ij, where q ij is he random delay beween he wo nodes, hus making he ime informaion on j (n) conained in he packe oudaed. Differen echniques have been designed o miigae he effecs of hese random facors according o diverse principles, such as synchronizaion beween receivers of he same packe raher han beween ransmier and receiver. he sae of he ar in packe-based echniques repors synchronizaion accuracies of he order of milliseconds o 5 (n)+ 5 2 microseconds [6]. Moreover, he need for exchanging of a large number of packes is common o all packe-based mehods. his in urn enails large compuaional complexiy, energy expendiure, and poor scalabiliy. o obviae o he drawbacks of packe-based soluions, more recenly, here has been ineres in physical layer-based schemes, where he local iming informaion in encoded direcly in he ransmission imes of given waveforms g(). In paricular, each node radiaes a periodic rain of waveforms n g( i(n)), according o is local clock, on eiher a dedicaed bandwidh or on an overlay sysem such as ulra-wideband (UWB); see Figure 3. he updae of each local clock is hen carried ou by processing he received signal, which is a combinaion of waveforms ransmied by neighboring nodes (see Figure 4). Possible processing echniques include ime-ofarrival esimaors bu efficien synchronizaion echniques can be devised ha do no need o explicily perform such IEEE SIGNAL PROCESSING MAGAZINE [84] SEPEMBER 2008

5 operaion [23], [24]. Pulse-coupled synchronizaion is naurally scalable, since he operaions performed a each node are independen of he number of nodes available in he nework and has limied complexiy, requiring only simple processing a he baseband level (see he secion rading Accuracy for Bandwih and Complexiy ). REMARK he waveform g() can be a (possibly band-pass) pulse, as illusraed in Figure 3, bu can also have a differen shape, such as a pseudorandom sequence or he odd waveform invesigaed in [8] o sudy scalabiliy of disribued synchronizaion. While bearing his in mind, in order o comply wih he curren lieraure (see [8] and [25]), we will refer o physical layer-based synchronizaion as pulse coupling (as opposed o packe coupling). A landmark work in he area of pulse-coupled ime synchronizaion for wireless neworks is [25] by Hong and Scaglione, where he auhors invesigae a direc applicaion of he model of inegrae-and-fire pulse-coupled oscillaors sudied in he conex of mahemaical biology in [8]. he resuls of [25] have been exended in [26] by considering he convergence analysis in presence of more realisic neares neighbor communicaions. Moreover, applicaions of pulse-coupled synchronizaion o signal processing problems have been discussed in [20] (change deecion) and [22] (daa fusion). A differen approach o pulsecoupled synchronizaion is he use of linear processing a he baseband level according o he mechanism of discree-ime, phase-locked loops (PLLs). Firs-order, pulse-coupled linear discree-ime PLLs wih frequency-synchronous clocks (i.e., i = for each i =,...,N) have been proposed in [0] and hen [23]. Moreover, [27] sudies a mahemaically equivalen mehod, where ime informaion is exchanged via packes. A general model for pulse-coupled linear PLLs is proposed in [24] ha can be seen as he discree ime counerpar of he coninuously coupled (linearized) analog PLLs considered by Lindsey e al. [9]. REMARK 2 An imporan echnological limiaion is he half-duplex consrain imposed on wireless ransceivers by he srong self-inerference beween ransmi and receive pahs (i.e., nodes canno ransmi and receive simulaneously). Noice, however, ha here is some evidence ha full-duplex ransmission is echnologically feasible [28]. wo soluions can be devised o implemen pulse-coupled synchronizaion wihin such a consrain. he firs approach is o choose an impulsive waveform g() wih a shor duraion and le nodes swich from ransmi o receive mode (and vice versa) before and afer ransmission of a pulse, hus having a refracory ime during ransmission where nodes are no able o receive (see he secion rading Accuracy for Bandwidh and Complexiy and [25] for furher deails). A second soluion is o selec any waveform wih he desired resoluion properies and o ransmi a rain n N g( i(n)) where N is a (e.g., randomly seleced) subse of clock periods: accordingly, each node ransmis is synchronizing signal in some periods while in oher lisens o he signal received by oher nodes. CLOCKS AND SYNCHRONIZAION In his aricle, we are concerned wih a populaion of N clocks coupled hrough a wireless channel, as skeched in Figure 3. According o Alber Einsein [], by clock we undersand any hing characerized by a phenomenon passing periodically hrough idenical phases so ha we mus assume, by virue of he principle of sufficien reason, ha all ha happens in a given period is idenical wih all ha happened in an arbirary period. A clock is hen a ime measuremen device consising of an oscillaor and an accumulaor, as deailed below for boh analog and discree ime clocks. In his secion, we firs discuss he baseline scenario of uncoupled clocks [recall Figure 2(a)] and hen inroduce he basics of muual synchronizaion via clock coupling [Figure 2(b) and (c)]. UNCOUPLED CLOCKS Here we illusrae he behavior of uncoupled clocks, accouning for he reference case where each node runs is own local clock wihou exchanging iming informaion wih he ohers. ANALOG CLOCKS An analog clock, such as he ih, is characerized by an oscillaor s i () = cos i (), () where i () is he oal insananeous phase (accumulaor), which, in case of uncoupled nodes, evolve as i () = i (0) + 2π i + ζ(), (2) where i), he free-running oscillaion period, reads i = nom + i, wih nom being he nominal period and i a random offse from he nominal value (relaed o he frequency offse or skew), ha depends on hardware imperfecions; ii) ζ() is a ypically nonsaionary random process modelling phase noise [9]. Moreover, we have seleced an arbirary iniial ime insan = 0 for all he clocks and i (0) is he iniial phase ( i () = 0 for < 0). A more general model for () could be considered ha accouns for frequency drifs [9]. In his aricle, for he sake of simpliciy, we will no elaborae on his addiional nuisance parameer. DISCREE-IME CLOCKS A discree-ime clock can be seen as a sequence i (n) of significan ime insans of an analog clock (e.g., upward zero crossing poins: i ( i (n)) = n 2π), where index n = 0,, 2,...runs over he periods of he oscillaor. In paricular, from (2), an uncoupled discree ime clock evolves as i (n) = i (0) + n i + υ(), (3) IEEE SIGNAL PROCESSING MAGAZINE [85] SEPEMBER 2008

6 where υ() is he addiive noise erm ha accouns for phase noise. As explained in he previous secion, Figure 2(a) shows he behavior of N uncoupled clocks { i (n)} N i=, assuming for simpliciy no phase noise. I is apparen ha he nodes, if isolaed, remain asynchronous. COUPLED CLOCKS he goal of a coupling mechanism among he clocks is o drive he laer o synchroniciy, possibly wihin a given olerance. Before furher elaboraing on he basic ideas behind (eiher packe or pulse) clock coupling for analog and discree ime clocks, we formalize he noions of synchronized saes, inuiively inroduced hrough he discussion on Figure 2 in he previous secion. For analog clocks, we have wo condiions of ineres: Frequency synchroniciy: for sufficienly large, here exiss a common period of oscillaion for all he nodes so ha i () = i ( + ), i =, 2,...,N. (4) Full (frequency and phase) synchroniciy: for sufficienly large, we have i () = j () for each i j. (5) Noice ha for analog clocks, full synchroniciy implies he exisence of a common ime scale a all imes. On he oher hand, for discree ime clocks, nodes are said o be synchronous if hey agree on he ime insans i (n) corresponding o he icks of he local clocks, which enails ha a common noion of ime does no exis for he period elapsed beween wo icks. More specifically, for discree ime clocks, we have he following wo condiions: Frequency synchroniciy [Figure 2(b)]: for n sufficienly large, here exiss a common period of oscillaion for all he nodes so ha α 5 5 α 5 α 54 α 45 α 2 α 2 α α 4 4 [FIG5] Example of a conneciviy graph G: local ime informaion is exchanged along he edges of he graph weighed by he coupling srenghs α ij. A key role in he analysis of disribued synchronizaon is played by he Laplacian marix L = I A, where A is he adjacency marix of he conneciviy graph G ([A] ij = α ij for i j and [A] ii = 0). 2 4 α 32 α 34 α 23 α 43 3 i (n + ) i (n) =, i =,...,N. (6) Full (frequency and phase) synchroniciy [Figure 2(b)]: for n sufficienly large, we have i (n) = j (n) for each i j. (7) In his aricle, we focus on diffusion proocols for he exchange of local ime informaion. his class encompasses, among he ohers, he packe-coupling mehod of [27], pulse coupling (see [8], [25], and [26]), and he synchronizaion of analog clocks according o he sandard Kuramoo model [5] [7] or he analog PLLs in [9]. Moreover, as illusraed in he secion Disribued Consensus for Muliagen Coordinaion, diffusion synchronizaion proocols have srong connecions wih signal processing applicaions such as disribued esimaion [2], [29], deecion [30], and consensus [3] problems. he basic mechanism is as follows. Each node ransmis (diffuses) is local ime [eiher phase j () or clock ick j (n)] o is neighboring nodes, where he definiion of neighbors usually idenifies hose nodes ha receive a sufficienly large power from he sender. We recall ha he iming informaion j (n) can be encoded eiher as a ime samp in a packe (packe coupling) [27] or simply in he ransmission ime of a given waveform g( j (n)) (pulse coupling) in he case of discree ime clocks. For analog clocks, a signal proporional o he local oscillaor s j () needs o be radiaed by each node, as discussed in he secion Coninuously Coupled Analog Clocks. he goal of each recipien, such as he ih, is o measure he phase or ime differences beween he local clock and he clocks of neighboring senders [ j () i () or j (n) i (n), respecively], and o correc he local clock accordingly, despie he nuisance erm due o propagaion delays. CONNECIVIY GRAPH AND LAPLACIAN MARIX From he presenaion above, i is clear ha he achievemen of a synchronized sae sricly depends on he opology of he connecions beween clocks, since each node ransmis is local ime informaion only o neighbors. he sandard way o represen his relaionship beween nodes is by means of a conneciviy graph G, as he one skeched in Figure 5 for N = 5. In paricular, node i receives he synchronizaion signal from j (i.e., j is a neighbor of i ) if here exiss an edge direced from i o j. Moreover, his edge is weighed by a posiive value α ij, ha represens he relaive srengh of he signal received by i from j wih respec o he oher neighbors of i (we have he normalizaion condiion j α ij = ). For insance, a ypical choice for parameers α ij is he following [0], [23]: α ij = P ij j I i P ij, (8) where P ij is he power received by he ih node from he jh and I i is he se of neighbors of i (I i ={j : P ij > P 0 },wih P 0 being a power hreshold). herefore, he edge weigh α ij depends on he disance beween nodes i and j hrough pah loss aenuaion, IEEE SIGNAL PROCESSING MAGAZINE [86] SEPEMBER 2008

7 and on possibly random facors such as fading and shadowing. Noice ha he graph is ypically direced (α ij α ji ) and furhermore, i is bidirecional (i.e., α ij > 0 if and only if α ji > 0) unless differen nodes have differen power consrains (so ha, given a pair of nodes, one node may be wihin he ransmission radius of he oher bu no vice versa). As illusraed in he res of he aricle, diffusion synchronizaion proocols for boh analog and discree cases can be described by linear dynamic sysems (see () and (7) for a preview) whose sysem marix is linearly relaed o a key algebraic quaniy ha describes he conneciviy graph G, namely he Laplacian marix L [32]. his is defined as L = I A, where A is he adjacency marix of he graph ([A] ij = α ij for i j and [A] ii = 0). I is hen clear ha he performance of muual synchronizaion depends on he nework opology (conneciviy graph G) hrough he eigensrucure of he Laplacian marix L. As elaboraed in he following (and wih some deails in Algebraic Graph heory and Disribued Synchronizaion ), of paricular relevance is he null space of marix L, ha is someimes referred o as he synchronizaion subspace. Specifically, he mulipliciy of he zero-eigenvalue λ(l) = 0 deermines wheher a synchronized sae is evenually achieved or no, while he lef eigenvecor v = [v v N ] corresponding o λ(l) = 0 (v L = 0) yields he seady-sae frequency and phases of he clocks (see (2), (4), and (9) for a preview). As a final remark, in he discussion above, we have limied he scope o ime-invarying and deerminisic opologies, bu he analysis can be exended o boh ime-varying [3], [33] and random [34] opologies. We will provide some commens on hese imporan cases in he following, and we poin o references for furher deails. CONINUOUSLY COUPLED ANALOG CLOCKS In his secion, we sudy he problem of disribued synchronizaion of coupled analog clocks. he ineres of such problem for wireless communicaions is relaed o applicaions such as, e.g., cooperaive beamforming or frequency division muliple access in ad hoc neworks. Moreover, i is hisorically he firs sudied model of disribued synchronizaion, and ses he ground for he discussion on discree-ime clocks in he nex secion. Wih coupled analog clocks, each node ransmis a signal proporional o is local oscillaor s i () in () and updaes he insananeous phase i () based on he signal received from oher nodes. Noice ha his procedure assumes ha each node is able o ransmi and receive coninuously and a he same ime (full duplex, see Remark 2). he basic mechanism of coninuously coupled clocks is phase locking (see Figure 6). Each node, say he ih, measures hrough is phase deecor (PD) he convex combinaion of phase differences N i () = α ij f( j () i ()), (9) j=, j i where j () i () is he phase difference wih respec o node j, and f( ) and α ij are phase deecor-specific feaures, namely a nonlinear funcion and convex combinaion weighs (i.e., N j= α ij = and α ij 0),respecively (recall he discussion in he previous secion). Noice ha he choice of a convex combinaion in (9) ensures ha he oupu of he phase deecor i () akes values in he range beween he minimum and he maximum of phase differences f ( j () i ()). Finally, he difference i () (9) is fed o a loop filer ε(s), whose oupu drives he volage conrolled oscillaor (VCO), which updaes he local phase as i () = 2π N + ε 0 α ij f( j () i ()), (0) i j=, j i REMARK 3 he (average) accuracy of differen clocks is someimes measured in parsper-million (PPM) by calculaing he average (absolue value of) he clock error afer one second. here exiss a clear rade-off beween accuracy and power consumpion. For insance, accuracies of ypical clocks range beween around 0 4 and 0 PPM wih corresponding power consumpions on he order of μw and hundreds of megawas, respecively [2]. s 3 () ε(s) VCO ΔΦ 3 () PD 3 ΔΦ PD () [FIG6] Block diagram of N = 3 coninuously coupled oscillaors (PD: phase deecor; VCO: volage conrolled oscillaor). s () ΔΦ 2 () PD s 2 () ε(s) VCO ε(s) VCO 2 IEEE SIGNAL PROCESSING MAGAZINE [87] SEPEMBER 2008

8 where we have considered for simpliciy a simple loop filer ε(s) = ε 0. Moreover, in our presenaion above, we have assumed absence of phase noise [see (2)], insananeous coupling among he clocks (i.e., irrelevan propagaion delays: q ij = 0) and ime-invarian nework opology (consan coefficien α ij ). hese assumpions will be assumed hroughou he paper unless oherwise saed (see he secion Signal Processing Aspecs of Disribued ime Synchronizaion for a discussion on more general and realisic models). KURAMOO S MODEL he firs model of coupled analog oscillaors was proposed by Kuramoo in he conex of mahemaical biology [5] and has araced considerable aenion since is definiion because of he many challenges i poses o mahemaicians (see he survey papers [6] and [7]). he basic Kuramoo model corresponds o he general sysem (0) wih a sinusoidal phase deecor f(x) = sin (x), all-o-all conneciviy, i.e., α ij = /N for i, j =,...,N (fully meshed conneciviy graph), and a simple loop filer ε(s) = ε 0 (firs-order PLLs). he analysis in [5] is concerned wih he assessmen of he seady sae (equilibrium poin) of he sysem. In paricular, he auhor discovered ha (assuming unimodal local frequency disribuion) here exiss a criical value of he loop gain ε 0, say ε0, such ha if ε 0 >ε0 he populaion of clocks aains a sae of parial (frequency and phase) synchronizaion in which par of he oscillaors is phase locked and par is ou of synchrony (full synchronizaion is evenually achieved for ε 0 ), whereas if ε 0 <ε0 he clocks remain in an incoheren sae. A horough undersanding of he sabiliy properies of he sysem has proved o be elusive for many years and a few quesions are sill open, see [6] for a recen review. As final remarks, we refer he reader o [3] for an applicaion of Kuramoo s model o he sudy of he dynamics of hand clapping in a concer hall. Kuramoo s model is hardly direcly applicable o wireless neworks, for wo main reasons: i) he assumpion of coninuous coupling among he clocks, which requires full-duplex ransceivers (see discussion above) and ii) he assumpion on all-o-all (mesh) conneciviy (bu see [7] for exensions of he basic Kuramoo model o more general scenarios). CONINUOUSLY COUPLED LINEAR PLLS In he conex of synchronizaion for (wired) elecommunicaion neworks, Lindsey e al. [9] sudied he general model of coupled analog clocks (0) for linear phase deecors ( f(x) = x), arbirary connecions α ij (under he consrain of convexiy), and loop filers ε(s) = ε 0 /( s/μ) (second-order PLLs). Noice ha oher ypes of second-order PLLs ha include also a zero in he loop filer (e.g., proporional-plusinegral loop filers) are possible and may bring significan benefis, especially in erms of sabiliy (see [35]). he model in [9] hen alleviaes he problem ii) menioned above by allowing arbirary conneciviy, while simplifying he analysis hrough linearizaion. Linearizaion allows o readily use ools from algebraic graph heory in order o sudy convergence, and makes he analysis flexible enough o enable assessmen of he effecs of nuisance parameers such as possible communicaions delays and clock imperfecions. o elaborae on his poin, le us focus on he case of coninuously coupled linear PLLs wih ε(s) = ε 0 (firs-order PLLs). hen, he se of PLLs (0) can be cas as a vecor linear imeinvarian differenial equaion () = ω ε 0 L (), () where we have defined he vecors Φ( ) = [ ( ) N ( )], ω = [2π/ 2π/ N ] and marix L is he graph Laplacian associaed wih he conneciviy graph ha describes he nework (recall he secion Conneciviy Graph and Laplacian Marix and Figure 5). As deailed in Algebraic Graph heory and Disribued Synchronizaion, he convergence (synchronizaion) properies of sysem () depend on he nework opology (conneciviy graph) hrough he eigenvalues of he Laplacian marix L. In paricular, a sufficien condiion for convergence (asympoic sabiliy) is ha he conneciviy graph is srongly conneced (i.e., here exiss a leas one pah beween every pair of nodes). Under he condiion ha () is asympoically sable, [9] finds ha he seady-sae soluion of () is characerized by frequency synchronizaion (4) wih common frequency given by he weighed combinaion N = v i, (2) i = i where vecor v = [v v N ] plays a cenral role and is he normalized lef eigenvecor of L corresponding o he zero eigenvalue: L v = 0 and N i = v i = (recall he secion Conneciviy Graph and Laplacian Marix ). Even if he formalism of he algebraic graph heory was no employed in [9], reinerpreing he resuls of [9] in his ligh allows a unified presenaion of diffusion-based synchronizaion schemes, including synchronizaion discree-ime clocks (see he secion Pulse-Coupled Discree-ime Clocks ) and disribued esimaion/deecion and consensus (see he secion Disribued Consensus for Muliagen Coordinaion ). Noice ha he enries of vecor v are real and posiive if he graph is srongly conneced by virue of he Perron-Frobenius heorem (see also Algebraic Graph heory and Disribued Synchronizaion ), so ha he common frequency / (2) is a convex combinaion of all he local frequencies {/ i } N i =. While frequency synchronizaion is aained, full frequency and phase synchronizaion (5) is generally no achieved, and he seady-sae phases are mismached by an amoun relaed o he deviaions beween local and common frequency ω = ω 2π/,where = [ ] () 2π ( ) + v (0) L ω + L ω. (3 ) ε 0 ε 0 IEEE SIGNAL PROCESSING MAGAZINE [88] SEPEMBER 2008

9 Noice ha he second erm in he righ hand side of (3) is a phase common o all clocks and he hird represens he phase mismach (see Algebraic Graph heory and Disribued Synchronizaion for deails). As a special case of hese resuls, if no deviaion among local frequency exiss ( i = nom ), hen from (2) he common frequency is / = / nom, and, from (3), full frequency and phase synchronizaion is achieved wih (recall ha ω = 0) i () 2π N + v j j (0). (4) j= he resuls summarized above are exended in [9] o more complex scenarios wih loop filers ε(s) = ε 0 /( s/μ), delays and phase noise. In paricular, similarly o convenional PLLs, i is shown ha adding a pole μ in he loop filer ε(s) (secondorder PLLs) reduces he seady-sae phase error [see (3)] bu, a he same ime, reduces he sabiliy margin. hese resuls can be seen as he naural exension of known conclusions in he conex of classical (maser-slave poin-o-poin) PLLs [35]. Moreover, propagaion delays are shown o cause seady-sae phase mismach. Furher discussion on he laer opic is provided in he secion he Impac of Propagaion and Processing Delays and Phase Noise for he case of discree ime PLLs. PULSE-COUPLED DISCREE IME CLOCKS In his secion, we review he wo approaches proposed for pulsecoupled discree ime clocks (Figure 3): inegrae-and-fire oscillaors [8], [25], and disribued discree ime PLLs [0], [23], [24] (see also [27]). As in he previous secion, in order o simplify he presenaion, we focus on a scenario wih absence of phase noise and delays (q ij = 0). Moreover, we limi he scope o infinie-resoluion ime deecors: ha is, we assume ha each node is able o deec he ime of arrival j (n) of any pulse received from is neighboring nodes. Clearly, in pracice, here exiss a rade-off beween resoluion on one hand, and bandwidh and complexiy on he oher. More general models wih phase noise, delays, and xi() finie-resoluion ime deecors will be discussed in he secion Signal Processing Aspecs of Disribued ime Synchronizaion. xi() PULSE-COUPLED INEGRAE-AND-FIRE OSCILLAORS his model was firs sudied in he conex of mahemaical biology in [8] and hen applied by [25] o wireless neworks. In order o enable he analysis, i is assumed ha no frequency mismach among differen nodes is presen ( i = nom ). he impac of a frequency mismach has been invesigaed via numerical simulaions in [25]. Moreover, according o he model, each node is equipped wih an inegrae-and-fire oscillaor, as skeched in Figure 7(a). Adaping he noaion of [8] o fi our overview, his oscillaor is described, when isolaed, by a sae variable x i () = g( i ()), where g( ) is a periodic funcion (wih period 2π) such ha in each period i is smooh, monoonically increasing from zero o one, and concave. As before, he icks i (n) of he clock correspond o he ime insans when he phase reurns, afer one period, o 2π, or equivalenly when he sae variables charges up o is maximum value x i ( i (n)) = and hen reurns o zero. he model of inegrae-and-fire oscillaors prescribes he following coupling mechanism among clocks, illusraed in Figure 7(b). Upon deecion of he pulse sen by any node j a ime j (n) (propagaion delays are negleced in his model), he i h clock modifies he sae funcion by adding a value ε owards he goal of selecing a firing insan ha is closer o ha of clock j { x i ( j (n) + xi ( ) = j (n) ) + ε if x i ( j (n) ) + ε< 0 oherwise and adjuss he phase i () accordingly. Convergence of pulse-coupled inegrae-and-fire clocks can be evaluaed for arbirary connecions α ij by casing he problem as he sudy of asympoic sabiliy of a sysem of differenial equaions [26]. Using Lyapunov sabiliy heory, convergence is shown o depend on he properies of he graph Laplacian L (see he secion Coupled Clocks and Algebraic Graph heory and Disribued Synchronizaion, ) similarly o he case of analog oscillaors. he main drawbacks of he model of inegrae-and-fire oscillaors when applied o wireless neworks are: i) i is hard o exend he analysis o realisic and complex scenarios wih inaccurae clocks, propagaion delays, or ime-varying channels; ii) he sysem design is no flexible enough o gran / nom 0.2 j (n) / nom (b) [FIG7] Pulse-coupled inegrae-and-fire clocks: (a) Sae funcion x i ( ) for isolaed clocks; (b) Sae funcion x i ( ) behavior in presence of a received pulse. (a) ε IEEE SIGNAL PROCESSING MAGAZINE [89] SEPEMBER 2008

10 degrees of freedom for he achievemen of addiional relevan goals, such as rading complexiy for accuracy, securiy, ec. PULSE-COUPLED DISCREE-IME PLLs An alernaive model for pulse-coupled clocks was proposed in [0], [23], and [24] based on disribued discree-ime PLLs. he approach can be seen as he discree counerpar of he sysem of coupled analog PLLs illusraed in he secion Coninuously Coupled Linear PLLs. From is analog predecessor, he sysem of pulse-coupled discree ime PLLs inheris he linear naure ha enables analysis and flexible sysem design using sandard ools from algebraic graph heory and signal processing (see also he secion Signal Processing Aspecs of Disribued ime Synchronizaion ). A sysem of pulse-coupled discree-ime PLLs is exemplified by Figure 8. Similarly o he analog case (see Figure 6), based on he received signal, each node calculaes a convex combinaion of he ime differences (ime difference deecor) N i (n) = α ij ( j (n) i (n)), (5) j=,i j ha is fed o a loop filer ε(z). Considering for simpliciy loop filers ε(z) = ε 0 (firs-order PLLs), we have N i (n + ) = i (n) + i + ε 0 α ij ( j (n) i (n)). j=, i j (6) o furher analyze he sysem, le us cas he sysem (6) as a vecor ime invarian difference equaion 3 (n) ε(z) VCC 3 Δ 3 () D (n) 2 (n) [FIG8] Block diagram of N = 3 pulse-coupled discree ime clocks (D: ime difference deecor; VCC: volage conrolled clock). D Δ D () Δ 2 (n) ε(z) VCC (n + ) (n) = ε 0 L (n), (7) where we defined he vecors (n) = [ (n) N (n)] and = [ N ] (noice ha he case of PLLs wih frequencysynchronous clocks sudied in [0], [23], and [27] corresponds o (6) wih = 0). From comparison of (7) and (), i is apparen ha he same ools and resuls derived in he coninuous case can be applied o he pulse-coupled discree ime case. In paricular, convergence is guaraneed under he same condiions (see Algebraic Graph heory and Disribued Synchronizaion ), and he seady-sae soluions are characerized by frequency synchronizaion (6) wih common frequency given by he weighed combinaion of local frequencies (2), bu generally mismached phases [i.e., absence of full synchrony (7)] wih ( ) (n) n + v (0) L + L, (8 ) ε 0 ε 0 where = [similar o (3)]. As previously discussed, an imporan special case of hese resuls occurs when here is no frequency mismach beween he clocks ( i = nom ), in which case he common frequency equals he nominal local frequency / = / nom, and, from (8), full frequency and phase synchronizaion is achieved [similarly o (4)] ε(z) VCO 2 N i (n) n+ v j j (0). (9) j= Moreover, adding a pole μ in he loop filer ε(z) = ε 0 /( μz ) (secondorder PLLs) can be shown o reduce he seady-sae phase error in (8) (by a facor μ) a he expenses of a reduced sabiliy margin [24], while full synchronizaion can be in principle achieved wih proporional-plus-inegral loop filers (see [35]). Here we consider a simple numerical example for he nework of N = 4 nodes illusraed in he box of Figure 9. We assume he weighs (8) (wih P 0 = 0), a pah loss model P ij = /d 3 ij (d ij is he disance beween node i and j ), and frequency synchronous clocks wih =, and plo he evoluion of he phases i (n) n for ε 0 = 0.3 and μ = 0. Afer a brief ransien where he nodes end o synchronize in pairs beween neighbors, he sysem reaches he seady sae o he condiion (9), where v = /4 for his specific opology. IEEE SIGNAL PROCESSING MAGAZINE [90] SEPEMBER 2008

11 REMARK 4 For boh analog (secion Coninuously Coupled Analog Clocks ) and discree-ime (secion Pulse-Coupled Discree-ime PLLs ) coupled clocks, we have assumed a homogeneous scenario where all he nodes use idenical loop filers. Exension of he analysis o a heerogeneous seup calls for subsiuion of he loop gains ε(s), and ε(z ), wih diagonal marices conaining he local loop filers a he N nodes. his model requires furher sudy. IMPAC OF OPOLOGY AND SMALL-WORLD EFFECS OF SHADOWING he convergence properies of disribued synchronizaion depend on he opology of he nework, which is in urn defined by he weighing facors α ij (recall he secion Conneciviy Graph and Laplacian Marix and Figure 5). Here we illusrae he performance of pulse-coupled PLLs for a nework of randomly locaed nodes wih weighs α ij (8) (P 0 = 4) and log-normal shadowing. More specifically, he power received over disance d ij is P ij = 0 ν 0 /d 3 ij, where ν a zero-mean Gaussian random wih sandard deviaion σ. As a performance measure, we evaluae he sandard deviaion ξ(n) of he clocks, where ξ 2 (n) = ( 2 N N i (n) N N k (n)), (20) i = k= versus ime n, averaged over random locaion of nodes and shadowing. he iniial phases (0) are seleced randomly in he se (0, ) (and (n) = 0 for n < 0), while he local free-oscillaion periods are seleced independenly in he inerval ± 0.0. he dashed lines in Figure 0 are obained from he asympoic resul (8). I can be seen ha increasing he amoun of shadowing in he model (i.e., he sandard deviaion σ) improves boh he convergence speed and he asympoic 0.8 phase error of he sysem of disribued PLLs. he beneficial impac of shadowing can be inerpreed as an insance of he fac, repored in [2] and [36], ha disribued agreemen on a graph improves if he graph has he feaures of a smallworld nework. A small-world nework is characerized by he exisence of pahs made of a small number of edges beween any wo nodes. In fac, shadowing breaks a few close connecions and, due o he long ails of he log-normal disribuion, creaes a few long links, hus enhancing he small-world properies of he conneciviy graph [2] (n) n 4 (n) n i (n) n 3 (n) n 2 (n) n SIGNAL PROCESSING ASPECS OF DISRIBUED IME SYNCHRONIZAION In he previous secions, we have shown ha basic analysis of disribued synchronizaion relies on linear algebraic, and more specifically graph algebraic, conceps (namely, on he eigensrucure of he Laplacian marix associaed wih he conneciviy graph G). In his secion, we elaborae on various furher aspecs of he analysis and design of disribued ime synchronizaion, where signal processing ools play a major role. We focus on pulse-coupled discree-ime PLLs for heir pracical relevance in wireless neworks. A firs, we remove he assumpion of infinie-resoluion ime error deecors in he secion rading Accuracy for Bandwidh and Complexiy. Nex, we address he issue of faul olerance and securiy. Finally, we discuss he impac of propagaion and processing delays and phase noise. RADING ACCURACY FOR BANDWIDH AND COMPLEXIY In his secion, we remove he assumpion made in he previous secion of infinie resoluion of he ime difference deecors for discree-ime PLLs. he main goal of he secion is o illusrae he rade-offs available in he sysem design beween accuracy and complexiy of he receiver (which in urns ranslaes ino power consumpion). o sar, we recall ha any ih node receives, in an inerval of duraion i around is local clock ick i (n) a combinaion of he waveforms ransmied by oher nodes, as skeched in Figure 4. Based on his signal, he node needs o esimae he ime differences j (n) i (n) in order o correc he local clock [i.e., o decide he nex clock ick i (n + )] according o he PLL mechanism of Figure 8. A firs approach o evaluae he ime differences j (n) i (n) would be o perform an esimaion of ime-ofarrivals j (n) (recall ha we are neglecing he delays q ij a his 2 d 3 = d 24 = 3 4 d 2 = d 34 = i (0) 4 i = n [FIG9] Phases of he N = 4 pulse-coupled discree-ime clocks shown in he box versus period n ( i =,ε 0 = 0.3,μ= 0 and (0) = [ ] ). IEEE SIGNAL PROCESSING MAGAZINE [9] SEPEMBER 2008

12 sage) based on he knowledge of he ransmied waveform g(). However, his choice would enail a large compuaion complexiy. A more efficien approach ha avoids explici esimae of he ime of arrival is he cener of mass ime-deecor proposed in [23]. Le us assume a square roo Nyquis waveforms g() wih given roll-off. According o he proposal in [23], he receiver performs baseband filering mached o he ransmied waveform g() (or an approximaion hereof) and hen samples he received signal a some muliple L of / s, where s is he peak-o-firs-zero ime for he auocorrelaion of g(). Based on he samples (indexed by m) received in he nh observaion window, {y i (n, m)}, he ih node does no explicily calculaes he single ime differences j (n) i (n). Insead, i direcly esimaes he convex combinaion of ime differences i (n) (5), using he definiion (8) for he convex weighs α ij, as he cener of mass of he received signal i (n) = m I α im = α im m s L, (2) y i (n, m) 2 k I y i(n, k) 2, (22) ξ (n) σ = 3 db 0.02 σ = 4 db σ = 0 db σ =.5 db σ = 2 db n [FIG0] Small-world effecs of shadowing: sandard deviaion of he clocks for discree-ime PLLs versus ime n for differen values of he sandard deviaion of shadowing σ. Dashed lines correspond o he analyical resul (8) (ε 0 = 0.6, μ = 0.4). ξ(n) Infinie Resoluion Refracory ime n [FIG] rade-off accuracy versus complexiy: Sandard deviaion of he clocks (20) versus ime n for differen values of he oversampling facor L and of he pole μ. Also shown for reference is he ideal case of infinie-resoluion ime difference deecors ( i = for i =,...,N,ε 0 = 0.9, s = 0.0 and refracory ime due o he half-duplex consrain equal o s ). 2 d 3 = d 24 = Implemenaion ( μ = 0.2, 0.4, 0.6; L = 5) 3 4 d 2 = d 34 = 0.5 Implemenaion ( = 0; L =, 2, 5, 5) μ where se I excludes he samples in he possible refracory period around he firing ime i (n) due o he half-duplex consrain (recall Remark 2). Noice ha his mehod does no require knowledge of he received powers, and ha is complexiy (i.e., number of operaions) is independen of he number of nodes in he nework. From he discussion above, we have idenified hree degrees of freedom in he sysem design for rading accuracy wih complexiy: ) he finie swiching ime from ransmi o receive mode ha defines he refracory ime inerval, which depends on he RF hardware employed 2) he oversampling facor L, which can be increased a he expense of compuaional and hardware complexiy a he baseband level 3) he possible presence of a loop filer wih pole μ, which increases he number of operaions o be performed and hus he compuaional complexiy (see he secion Pulse-Coupled Discree-ime PLLs ). Here we evaluae he impac of hese parameers on he performance (accuracy) of he synchronizaion scheme. Le us consider he simple nework wih N = 4 nodes shown in he box in Figure. Figure shows he sandard deviaion ξ(n) (20) of he iming vecor (n) versus n averaged wih respec o noise a he receiver side. I can be seen ha he finie resoluion of he sysem produces a performance floor for increasing n, ha can be lowered by increasing he oversampling facor L. In any case, an upper bound on he synchronizaion accuracy is se by he refracory period. his bound is reached for n and L sufficienly large. Adding a pole in he loop can increase he convergence speed as shown in Figure for μ = 0.2, 0.4, 0.6. Finally, Figure shows ha an upper bound on IEEE SIGNAL PROCESSING MAGAZINE [92] SEPEMBER 2008

13 he performance of he pracical implemenaion discussed here is se by he performance of he ideal sysem wih infinie resoluion sudied in he previous secions [see (7)]. FAUL-OLERANCE AND SECURIY Disribued wireless neworks, such as ad hoc and sensor, are ofen designed under he assumpion ha all he nodes in he nework are benign and well funcioning, and hus comply wih he preesablished behavior envisaged by he nework designer. However, in many applicaions, such an assumpion is easily oo opimisic, and robus design has o accoun for he possible aciviy of eiher fauly or malicious nodes. Here we discuss how simple signal processing echniques can enhance resilience of disribued synchronizaion o such phenomena. A simple approach o secure muual synchronizaion follows he idea of [37]. In a basic discree-ime PLL, he ime difference deecor evaluaes he convex combinaion i (n) (5) of he clock errors j (n) i (n). Using only his measure, i is no possible for he nodes o recognize ouliers ha may disrup he synchronizaion process. his goal calls for robus approaches ha evaluae he dispersion of he clock errors j (n) i (n) around he weighed average i (n), by, e.g., compuing he variance σi 2(n) = N j=, j i α ij ( j (n) i (n) i (n)) 2,and hen updae he local clock by considering only he se of clock differences j (n) i (n) ha are wihin a given fracion βσ i (n) from he average i (n). An example of he performance of such a scheme is shown in Figure 2. Consider a nework of N = 20 randomly disribued nodes in a square region of uni area wih frequency-synchronous clocks ( i = nom = ), ou of which four nodes are malfuncioning or malicious, having clocks running as i (n) = n+ θ i (n), where phases θ i (n) are seleced independenly and uniformly disribued in he se (0, ). he 0.35 dashed line corresponds o he performance of he sysem wih no malicious nodes, 0.3 which, as expeced, leads o an asympoically vanishing error ξ(n). On he conrary, in a 0.25 scenario wih malicious nodes, he iming error of he basic scheme (evaluaed only on he sixeen nonmalicious nodes) increases 0.2 linearly, hus showing ha he nework is no able o reach even frequency synchronizaion. However, he secure scheme, for 0.5 hreshold β small enough, manages o 0. mainain a consan error ξ(n) over n, hus showing ha i is able o approximaely achieve full synchronizaion wihin a limied 0.05 (here 5%) iming error. ξ (n) HE IMPAC OF PROPAGAION AND PROCESSING DELAYS AND PHASE NOISE In his secion, we remove wo furher assumpions ha have been made hroughou he aricle, namely negligible propagaion delays and absence of phase noise (see he secion Clocks and Synchronizaion ). In order o simplify he analysis, we ackle he wo problems separaely. PROPAGAION AND PROCESSING DELAYS Here we sudy he impac of delays on he performance of pulsecoupled PLLs bu similar conclusions hold also for he coninuous case as shown in [9]. Assume (for simpliciy) a frequency synchronous nework wih common local frequency / and firs-order PLLs. Moreover, le q ij be he (finie) propagaion delay beween he ih and he jh node (by symmery, we have q ij = q ji ). he ime a which he nh pulse emied by node j [a ime j (n)] is recorded by he ih is j (n) + q ij, so ha he iming deecion error of he ih PLL N α ij ( j (n) + q ij i (n)) = i (n) + N α ij q ij, j=, j i j=, j i (23) conains an addiive erm o he iming error i (n) defined in (5). Now, inroducing he effecive local frequency / i as N i = + ε 0 α ij q ij, (24) j=, j i i urns ou ha he model ha accoun for propagaion delays boils down o he frequency-asynchronous model (7). In oher words, propagaion delays have he effec of inroducing an equivalen frequency offse beween he local clocks, Robus Scheme Basic Scheme = β β = 0. No Malfuncioning or Malicious Nodes n [FIG2] Securiy in disribued synchronizaion: Sandard deviaion of he clocks (20) versus ime n for he basic and secure scheme in case we have four malfuncioning or malicious nodes ou of a oal number of N = 20 nodes (ε 0 = 0.6 and μ = 0). IEEE SIGNAL PROCESSING MAGAZINE [93] SEPEMBER 2008