Network Control by Bayesian Broadcast

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT-33, NO. 3, MAY 1987 323 Network Contro by Bayesian Broadcast RONALD L. RIVEST Abstract-A transmission contro strategy is described for sotted- ALOHA-type broadcast channes with ternary feedback. At each time sot, each station estimates the probabiity that n stations are ready to transmit a packet for each n, using Bayes rue and the observed history of coisions, successfu transmissions, and hoes (empty sots). A station transmits a packet in a probabiistic manner based on these estimates. Tbis strategy is caed Bayesian broadcast. An eegant and very practica strategy-pseudo-bayesian broadcast-is then derived by approximating the probabiity estimates with a Poisson distribution with mean Y and further simpifying. Each station keeps a copy of V, transmits a packet with probabiity 1 / Y, and then updates Y in two steps:. For coisions, increment v by (e - 2)- = 1.39221 *.*. For successes and hoes, decrement Y by 1. Set Y to max (V + x, ), where x is an estimate of the arriva rate A of new packets into the system. Simuation resuts are presented showing that pseudo-bayesian broadcast performs we in practice, and methods that can be used to prove that certain versions of pseudo-bayesian broadcast are stabe for X < e - are discussed. W I. IIwR~IXJCTI~N E PROPOSE a new strategy for the probem of controing traffic on a oca-area or sateite broadcast communications network. We begin by first presenting a strategy (caed Bayesian broadcast) which is powerfu but unikey to be cost effective in practice. The name Bayesian broadcast was chosen because each station uses Bayes rue to estimate dynamicay the probabiity that n stations are active, for each n. The stations cacuate a broadcast (or transmissio: probabiity that is optimum given the avaiabe goba information. This strategy is essentiay equivaent to a proposa of An and Geenbe [, pp. 305-3061, based on earier work by Sega [2]. Our new strategy (which we ca pseudo-bayesian broadcast) is an extremey simpe and eegant approximation to Bayesian broadcast. We give simuation resuts showing that pseudo-bayesian broadcast is exceptionay effective and stabe in practice. Consider a network with some number (possiby infinite) of stations. Each station is given packets to transmit by an associated processor. In practice a station may have a queue of packets ready to send if its processor is generating packets more quicky than the station can transmit them. However, we assume that each station has at most one packet to transmit at any time. We say a station is Manuscript received September 30, 1986. This work was supported by the Nationa Science Foundation under Grant MCS-80-06938. The author is with the MIT Laboratory for Computer Science, Room N&%3-324, 545 Technoogy Square, Cambridge, MA 02139, USA. IEEE Log Number 8611430. 0018-9448/87/0500-0323$01.00 01987 IEEE active if it has a packet to transmit; otherwise, it is inactive. We assume time is divided into sots, each ong enough to transmit one packet (the sotted ALOHA or S- ALOHA mode). Our procedures generaize for other modes; however, we do not treat these issues here. When a sot begins each active station must decide, either deterministicay or stochasticay, whether or not to transmit its packet. There are three possibe outcomes: a hoe if no stations transmit; 0 a success if one station transmits; or a coision if more than one station transmits. We assume that each station in the network can te which of the three possibe outcomes has occurred-this is the ternary feedback mode. We assume that the network objective is to minimize the average deay experienced by a packet between the time it is given to a station and the time it is successfuy transmitted; by Litte s resut [3] this is equivaent to minimizing the average backog in the network. Each station wi have a common contro strategy specifying how often it wi transmit packets, incuding how often it wi retransmit a packet which was invoved in a coision. Our approach has the foowing genera form. Just before sot t begins, each station k in the network computes a vaue for its broadcast probabiity b,,,. Then station k wi transmit a packet (if it has one) with probabiity b, f, independent of whether previous attempts had been made to transmit that packet. We assume that each station k computes b,,, from the gobay avaiabe network history, indicating whether each sot was a hoe, a success, or a coision. Since the stations ony use goba information to compute the broadcast probabiities b,, 1, each station wi compute the same vaue b, for b,,,, and our Bayesian updating procedure wi be reativey straightforward. In Section II we deveop the theory of Bayesian broadcast, showing how each station can choose a broadcast probabiity for each sot which is optimum given the avaiabe goba information. However, the fu Bayesian broadcast is a bit demanding to impement, so in Section III we provide a very simpe practica impementation based on these ideas, which we ca pseudo-bayesian broadcast. In Section IV we present some very encouraging experimenta resuts on the average backog when using pseudo-bayesian broadcast. In Section V of this paper we review reated work on this probem, and reate our resuts to this work.

324 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT-33, NO. 3, MAY 1987 Let Nt denote the number of active stations at time t (i.e., the number of stations which are ready to transmit a packet). This vaue wi decrease with successes and increase when a processor gives an inactive station packet. To motivate our deveopment, we begin by considering the compete knowedge case where each station knows the vaue of N, before sot t begins. This is unreaistic, since N, cannot be determined from the avaiabe information, but it is of interest to determine how the stations shoud act in this case. How ikey is it to have a hoe, success, or coision for a given broadcast probabiity b, ( and waiting probabiity w, = 1 - b,) and given vaue N, = n? The probabiities are P (hoe IN, = n) = H,,(n) = WY (1) P (success IN, = n) = S,,(n) = n * b, * WY- (2) P (coision II. BAYESIAN BROADCAST IN, = n) = C,,(n) = 1 - Hb(n) - S,,(n). The optimum vaue for b, is b, = /N,; (4 this maximizes S,,(N,). Note that b, depends ony on N,. If b, is chosen optimay as /N,, the expected number of stations attempting to transmit wi be one, and the probabiities of hoes, successes, and coisions wi be c,n,(nt) = - H,N,(Nt) - s,nttnt) = - f- t7) (3) (6) (The approximations hod for arge N,.) However, the stations wi typicay not know the correct vaue for N,. For exampe, some inactive stations may have received newy generated packets during sot t - 1 which they wi be ready to transmit during sot t. In the first procedure we describe, which we ca the Bayesian broadcast agorithm, each station wi use the evidence avaiabe up to time t to estimate the ikeihood P n,i that N, = n for each n 2 0. That is, P n.t = Pr(N, = n), for n = 0,... (8) given the avaiabe evidence. We ca this procedure Bayesian broadcast, since it reies on Bayesian reasoning to estimate j, = ( po, 1, p, f,... ). The Bayesian broadcast procedure described here is not new; it is essentiay the same as the proposa of An and Geenbe [, pp. 305-3061 based on the technique proposed by Sega[2] to estimate recursivey the number of stations waiting to transmit a packet. The ony difference between our formuation and theirs is that in our version new packets are not transmitted immediatey with probabiity one, but rather with the same transmission probabiity that is used for the backogged packets. In the Bayesian broadcast procedure, each station begins with the initia distribution PO = (LO, 0,. ** )-it assumes that a stations are inactive. Each station wi compute the same vector j?, using the avaiabe goba feedback information. The vector p, = ( po, [, * * *) summarizes the goba information avaiabe about N,. With the Bayesian broadcast procedure, each station performs the foowing four steps during each time sot. Compute the optima broadcast probabiity b, from the initia probabiity vector pt. If the station is active, transmit its packet with probabiity b,. Perform a Bayesian update of j, (the initia probabiity distribution for N,) to obtain j: (the fina probabiity distribution for N,), using the evidence (hoe, success, or coision) observed in time sot t. Convert the fina probabiities & for N, into initia probabiities P,+t for N,,, by considering the generation of new packets and the fact that a packet may have been successfuy transmitted during time sot t (i.e., modeing the fow of packets into and out of the system). In Sections II-A-C beow we consider the detais invoved in the preceding steps. A. Computing the Broadcast Probabiity One can choose b, to maximize the expected chance of a success, even though there is uncertainty about N, as summarized in jt, since E(P( successat timet)) = Cp,,; S,,(n). (9) n Given jt, this is a poynomia in the unknown variabe b,. In practice there woud be at most a finite number of nonzero coefficients at any time, so that we can compute the vaue 6, which maximizes (9) by differentiating and root finding. (In practice the computation required to compute b, woud probaby be excessive. One coud use the approximation (E(N,))-. However, we beieve that in practice the pseudo-bayesian broadcast agorithm to be described ater wi be the best choice.) B. Bayesian Updating of the Probabiity Vector We now describe how each station computes its fina probabiity distribution for N,, given that sot t was a hoe, a success, or a coision. This probem is we suited for an appication of Bayes rue: P@HPW) 00) P(H1E) = P(E). (The fina probabiity P(HIE) of a hypothesis H, after evidence E is received, is equa to the initia probabiity P(H) of H times the probabiity P(EIH) that E wi

RIVEST: NETWORK CONTROL BY BAYESIAN BROADCAST 325 occur given H, divided by the overa probabiity P(E) of evidence E.) We have a hypothesis N, = n, for each n 2 0. The vaues p,, f are the initia probabiities of these hypotheses before the evidence from time sot t is considered. Let p,, f denote the fina probabiity P( N, = n IE,) where E, is the sot t evidence (hoe, success, or coision), and et K = (P&t9P;,t7... ) (11) denote the corresponding fina probabiity vector. The p;, f are easiy obtained using Bayes rue by mutipying each initia probabiity p,,, r by the appropriate ikeihood Hb,( n), S$,(n), or C,!(n) according to whether a hoe, success, or coision was observed, and then normaizing so that the P n,t add up to one. This competes our description of how each station incorporates the sot t evidence into its probabiity distribution for N,. The resuting distribution makes the best possibe use of the gobay avaiabe information; one cannot improve over this appication of Bayes rue. (See [4, ch. 391.) However, it may be possibe to achieve better resuts by using information oca to the transmitting processors, or by somehow using a strategy to minimize the ong-run deay which is not a one-step ook-ahead strategy as is Bayesian broadcast. We do not pursue these possibiities here. C. Converting the Fina Probabiities j: into the Initia Probabiities ji,, 1 Finay, we convert j&!, the fina probabiity distribution for N,, into an initia distribution for N,,,. Why might Pi, tt be different than P;,~? First, if sot t was a success, the expected number of active stations wi decrease by one. Second, we expect some inactive stations to receive new packets from their processors during sot t, so the expected number of active stations wi increase for this reason. 1) Modeing Successfu Packet Transmission: We mode the effect of successes as foows. We et p$ denote a station s estimate of the probabiity that the number of active stations is n, taking into account the evidence from the channe, incuding the effect of a success on the number of active stations. If time sot t contained a success, then we set P n,t = P, +1, f> forn = ;.. (12) (note that p,$, f = 0 if E, = success); otherwise, we set P n,f = PA,,, forn = 0,.... (13) The vector p; = (p&, P;,~,... ) is used as input into the next step, where the generation of new packets is taken into account. 2) Modeing the Generation of New Packets: There are many ways to mode the generation of new packets. We are actuay concerned with the rate at which stations become active, i.e., convert from having no packet to send during time sot t to having a packet to send during time sot t + 1. We take the usua approach and assume that new packets arrive according to a Poisson distribution with parameter A, and that X i,s estimated reasonaby accuratey (et us ca the estimate A). We can compute the initia probabiities for N, + i : P n,t+ = i Pi: 1. Px(n -A. (14 j=o Here Pi( n - j) denotes the vaue of the Poisson density function at point n - j; i.e., the estimated probabiity that n -j new packets wi arrive during a time sot. This competes our description of the Bayesian broadcast procedure since we now have our initia estimates for the distribution of N,,, for the next time sot. III. THE PSEUDO-BAYESIAN BROADCAST ALGORITHM We now present a practica impementation of the above ideas, which we ca the pseudo-bayesian broadcast agorithm. We derive this agorithm by assuming that j, can be reasonaby approximated by a Poisson distribution with mean v; the station s vaue of v at time t represents the station s estimate of N,. We use the notation v rather than the subscripted form v for convenience in this section: v now denotes a changeabe contro parameter for the stations. Let --Y n P,(n) = y (15) denote the Poisson density at n for Poisson parameter v. Each station wi keep ony v, rather than the vector j,, and wi approximate the initia probabiity p,,, f by P,,(n). To deveop the pseudo-bayesian broadcast and probabiity updating procedure, we first consider the equations that woud be used for a,true Bayesian update of the Poisson approximation for j, if b, is the actua broadcast probabiity (and w, = 1 - b,). These equations represent the unnormaized fina probabiity vaues: PV(n). H,,(n) = epybt. P,,,(n) (16) P,(n). S,,(n) = vb,. evb + PVWr(n - 1) (17) Pv(n). cb,(n> = dn> ( - Hb,(n) - sb,(n)). (18) From (9) and (17) it is easy to compute the best broadcast probabiity: (19) no compicated root-finding is needed. Thus we have derived our first practica benefit from the Poisson approximation: it becomes triva to compute the desired broadcast probabiity b,. We next consider the probem of updating v in as Bayesian a manner as possibe, whie preserving our Poisson approximation. We sha see that for hoes and successes we can use Bayes rue exacty, whie for coisions we must introduce an approximation error to preserve the Poisson approximation.

326 IEEE TRANSACTIONS ON INFORMATION THEORY,VOL.IT-33,N0. 3,MAY1987 From (16) we see that for hoes the Bayesian updating takes a simpe form since the resuting distribution wi aso be Poisson with mean VW, = max (v -,O). In other words, when a hoe occurs, the stations reduce their estimate of the expected number of active stations by one, uness v is aready ess than one, in which case they set v to zero. From (17) we see that for successes the Bayesian updating and success modeing aso takes a simpe form. Here (17) wi yied a Poisson distribution with mean v - 1 shifted one pace to the right. However, the effect of modeing a successfu transmission shifts the distribution one pace to the eft. The net resut is that the Poisson assumption remains vaid, and each station shoud decrement its state variabe v by 1. If there is a coision, Bayes rue wi not yied a Poisson distribution for the fina probabiities. However; we approximate the resut by a Poisson distribution by setting v to be the mean of the resuting distribution, which is (using x to denote v. b,): v+ II XL ex - x- which simpifies in the case v 2 1, b, = /v to (20) 1 v+---- e-2 (21) (It is somewhat surprising that we get a constant increment to v in this case.) For v < 1, (20) is reasonaby we approximated by 2.39221 (22) which is the vaue (20) yieds for v = 1. Using (22) is equivaent to requiring that v 2 1 at a times. The foowing agorithm makes this simpification. We now summarize the above anaysis, assumptions, and approximations in the foowing presentation of the pseudo-bayesian broadcast agorithm. The Pseudo-Bayesian Broadcast Procedure: Each station maintains a copy of v and, during each sot, a a broadcasts with probabiity /v if it has a packet; decrements v by 1 if the current sot is a hoe or a success, and increments v by (e - 2)- = 1.392211... if the current sot is a coision; sets v to maxa(v + i, ), where fi is an estimate of the arriva rate h of new packets into the system. (For exampe, one might estimate the arriva rate by the observed average success rate, or use the constant vaue x = e-. See Section V for some discussion of this issue.) We note that the pseudo-bayesian broadcast procedure actuay ony needs binary feedback since it ony needs to distinguish coisions from noncoisions. We note that since each station now ony maintains a singe parameter v, it woud be simpe to broadcast v with every packet. In this way stations which have just powered-up can synchronize easiy. IV. EXPERIMENTALRESULTS The pseudo-bayesian broadcast procedure was simuated for o6 time sots (40 trias of 25 000 steps each) for a number of different Poisson arriva rates A. For each tria the average backog (i.e., the average of N, over the 25 000 steps) was computed as a sampe data point. The mean Nr and standard deviation ur of these 40 sampe points are given in Tabe I. TABLE1 MEANANDSTANDARDDEVIATIONFORBACKLOG x NY 0: 0.10 0.144 0.0069 0.15 0.28 0.012 0.20 0.555 0.85 0.25 1.00 0.097 0.30 2.31 0.32 0.32 3.73 0.54 0.34 7.03 1.58 0.35 12.35 3.82 0.36 28.38 20.86 0.37 63.11 39.7 For these experiments the arriva rate X was estimated by setting i to 0.500 initiay and then using the recursion A = 0.995r; + 0.005s (23) where s = 1 if the current sot contained a success, and s = 0 otherwise. This estimates the average success rate, which wi ony be a good estimate of the average arriva rate when the scheme is acting in a stabe manner. Tsitsikis [5] discusses this issue further. In Tabe II we give Fy, the average (over the 40 trias) frequency of having no backog during a tria of 25 000 steps, and Ly,, the average ast step number when there was no backog. The ast statistic supports the view that the method is stabe for h = 0.36 and unstabe for X = 0.37. TABLE11 FREQUENCYOFNOBACKLOGANDLASTINSTANCEOFNOBACKLOG x F,a LT 0.10 2320 24991 0.15 3274 24995 0.20 3154 24964 0.25 4204 24993 0.30 3714 24966 0.32 3235 24967 0.34 2312 24867 0.35 1701 24433 0.36 1026 22361 0.37 424 13605 It is cear from (6) that we shoud not expect to be abe to hande X > e- = 0.3678.... We see that the agorithm becomes unstabe for X > e-, as expected.

RIVEST: NETWORK CONTROL BY BAYESIAN BROADCAST 327 V. DISCUSSION In this section we discuss the pseudo-bayesian broadcast strategy in reation to previous work. Gaager [6] provides an exceent overview of the state of the art in mutiaccess channes, and the specia issue of this TRANSACTIONS [7] contains many exceent papers on this topic. Tanenbaum [8] surveys a number of possibe approaches to this probem, in a more genera framework. Note that the S-ALOHA mode used here differs from the more common Ethernet mode [9] since here we have fixed ength packets and fixed ength time sots. In the origina ALOHA scheme [o] a station broadcasts a new packet in the next sot. (A poicy of this sort is caed immediate first transmission (IFT) instead of deayed first transmission (DFT).) If a coision occurred, then the packet becomes backogged and is broadcast in succeeding sots with a fixed probabiity f unti it is successfuy transmitted. We note that in our scheme new packets and backogged packets are treated identicay -the contro poicy does not distinguish them. The idea of decreasing the broadcast probabiities in the event of a coision, and increasing it in the event of a success and/or a coision, is not new. For exampe, Gera and Keinrock [] discuss a number of adaptive strategies for the S-ALOHA network, some of which do not distinguish new from backogged packets, and which may be sensitive to the observed congestion on the channe. An and Geenbe [] discuss other variations, incuding a version of the Bayesian broadcast procedure as noted previousy. Hajek and Van Loon [12] present another adaptive scheme. Their scheme was IFT, and mutipies f (the transmission probabiity for backogged packets) by 1.518 for hoes, by 1.000 for successes, and by 0.559 for coisions (keeping f within some pre-estabished bounds as we). For the agorithm of Hajek and Van Loon [12] it is reported that the average number of backogged packets for X = 0.32 is approximatey 5.0 (To compare our resuts with theirs, subtract X from our vaues of Nr since they do not count newy arrived packets in the backog.) We see that the pseudo-bayesian broadcast agorithm appears to offer significanty improved performance over the Hajek and Van Loon agorithm. (To be fair, we note that their main objective was to prove that their agorithm was stabe for X < e-.) Merakos and Kazakos [13] have made a carefu study of Hajek and Van Loon s scheme and have rederived in a different way the parameters (1.518, 1.000, 0.559) mentioned earier. They aso study the effect of errors in the feedback process on the performance of the scheme. Kumar and Merakos [14] present a reated scheme for updating broadcast probabiities, but which is not provaby stabe for a X < e-. The cosest previous work is that of Key [15]. In his scheme each station maintains a variabe v, which -as in our scheme-is intended to track the vaue of N,, the number of active stations. As in our scheme, an active station transmits with probabiity /v. He considers the cass of schemes parameterized by (a, b, c), where v is increased by a in the case of a hoe, by b in the case of a successfu transmission, and by c in the case of a coision. The pseudo-bayesian broadcast scheme is a member of this cass, with parameters (-1 + A, -1 + i, (e - 2)- + i). This particuar choice of parameters was not mentioned by Key. The most important question is, of course, Is the pseudo-bayesian broadcast scheme stabe for a X < e-? The answer turns out to be a conditiona yes (conditiona on the manner in which the stations compute A). Key [15] has shown that the ratio K = N,/v wi drift towards unity whenever (a, b, c) are chosen so that [(a - c)epk + (b - c)ke- + c] (24) (the expected drift of v) is negative for K < 1 and positive for K > 1, and furthermore that h - Ke- < K[(a - c)e- + (b - c)keek + c] (25) for K > 1. In our case the above inequaities hod for X < e-, assuming that stations use fi = e- uniformy. The drift (24) wi have the correct sign when the stations overestimate the arriva rate X by using the upper bound e-. This anaysis was pointed out in Tsitsikis [5], who suggested as we that using fi = e- may be the most robust approach to estimating X. (The reader may aso want to consut Hajek [16] for a more fuy described version of a drift anaysis of this sort.) Mikhaiov [17] has presented techniques that are capabe of proving the stabiity of this type, according to Key [15] and Tsitsikis [5]. (The author has not seen the detais of these methods.) Tsitsikis [5] has presented a nice direct proof that the pseudo-bayesian broadcast agorithm is, indeed stabe whenever X < e-, assuming that A < A, i.e., that the estimate of X is aways an overestimate. Tsitsikis aso shows that if X > A, then instabiities may arise. VI. CONCLUSION We beieve the proposed pseudo-bayesian broadcast procedure wi be found to be exceptionay effective in practice since it makes neary the best possibe use of the information avaiabe on the network in determining the broadcast probabiities to use. ACKNOWLEDGMENT I woud ike to thank Bard Boom, Robert Gaager, Bruce Hajek, Pierre Humbet, Tom Leighton, Chares Leiserson, and John Tsitsikis for hepfu suggestions and discussions. I woud aso ike to thank Michee Lee and Brian Rogoff for heping to obtain the simuation resuts. Finay, the referees provided invauabe advice and pointers to the reevant iterature.

328 REFERENCES PI B. T. An and E. Geenbe, Etude et simuation de reseaux de diffusion sous controe, RA IRO Informatique/Comput. Sci., vo. 11, pp. 301-321, 1977. PI A Sega, Recursive estimation from discrete-time point processes, IEEE Trans. Inform. Theorv. vo. IT-22. vv. 422-431, Juy 1976. [31 L. Keinrock, *Queueing systems, vo. -i Theory. New York: Wiey, 1975. [41 P. Whitte, Optimization over Time, vo. II. New York: Wiey, 1983, ch. 39. [51 J. N. Tsitsikis, Anaysis of a mutiaccess contro scheme, MIT LIDS Rep. LIDS-P-1534. [61 R. G. Gaager, A perspective on mutiaccess channes, IEEE Trans. Inform. Theory, vo. IT-31, pp. 124-142, Mar. 1985. [71 J. L. Massey, Ed., Spkia Issue on kxndom-access Communications, IEEE Trans. Inform. Theory, vo. IT-31, Mar. 1985. PI A. S. Tanenbaum, Network protocos, Computing Surveys, vo. 13, pp. 453-489, Dec. 1981. t91 R. M. Metcafe and D. R. Boggs, Ethernet: Distributed packet switching for oca computer networks, Commun. Ass. Compu?. IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT-33, NO. 3, MAY 1987 Mach., vo. 19, pp. 395-404, Juy 1976. WI N. Abramson, Packet switching with sateites, in Proc. AFZPS Nat. Computer Conf., vo. 42, 1973, pp. 695-702. 1111 M. Gera and L. Keinrock, Cosed oop stabiity contros for S-ALOHA sateite communications, in Proc. Fifth Data Communications Symp., Sept. 1977, pp. 2.10-2.19. WI B. Hajek and T. Van Loon, Decentraized dynamic contro of a mutiaccess broadcast channe, IEEE Trans. Automat. Contr., vo. AC-27, pp. 559-569, June 1982. 1131 L. Merakos and D. Kazakos, On retransmission contro poicies in mutipe-access communication networks, IEEE Trans. Automat. Contr., vo. AC-30, pp. 109-117, Feb. 1985. P41 P. R. S. Kumar and L. Merakos, in Proc. 23rd IEEE Conf. Decision and Contro, Dec. 1984, pp. 1143-1 AX. [I51 F. P. Key, Stochastic modes of computer communication systems, J. Roy. Statist. Sot., Series B, vo. 47, pp. 379-395, 1985. PI B. Hajek, Hitting-time and occupation time bounds impied by drift anaysis with appications, in Advances App. Probab., vo. 14, pp. 502-525,1982. ii71 V. A. Mikhaiov, Methods of random mutipe access, Candidate Eng. thesis, Moscow Inst. of Physics and Techno., Moscow, USSR, 1979.