GAME THEORETIC FLOW AND ROUTING CONTROL FOR COMMUNICATION NETWORKS by Ismet Sahn B.S., Cukurova Unversty, 996 M.S., Unversty of Florda, 00 Submtted to the Graduate Faculty of School of Engneerng n partal fulfllment of the requrements for the degree of Doctor of Phlosophy Unversty of Pttsburgh 006
UNIVERSITY OF PITTSBURGH SCHOOL OF ENGINEERING Ths dssertaton was presented by Ismet Sahn It was defended on August 4, 006 and approved by Marwan A. Smaan, Bell of PA/Bell Atlantc Professor, Department of Electrcal and Computer Engneerng J. Robert Boston, Professor, Department of Electrcal and Computer Engneerng Lus F. Chaparro, Assocate Professor, Department of Electrcal and Computer Engneerng Chng-Chung L, Professor, Department of Electrcal and Computer Engneerng Prashant Krshnamurthy, Assocate Professor, Graduate Program n Telecommuncatons & Networkng Dssertaton Drector: Marwan A. Smaan, Bell of PA/Bell Atlantc Professor, Department of Electrcal and Computer Engneerng
GAME THEORETIC FLOW AND ROUTING CONTROL FOR COMMUNICATION NETWORKS Ismet Sahn, PhD Unversty of Pttsburgh, 006 As the need to support hgh speed data exchange n modern communcaton networks grows rapdly, effectve and far sharng of the network resources becomes very mportant. Today s communcaton networks typcally nvolve a large number of users that share the same network resources but may have dfferent, and often competng, objectves. Advanced network protocols that are mplemented to optmze the performance of such networks typcally assume that the users are passve and are wllng to accept compromsng ther own performance for the sake of optmzng the performance of the overall network. However, consderng the trend towards more decentralzaton n the future, t s natural to assume that the users n a large network may take a more actve approach and become more nterested n optmzng ther own ndvdual performances wthout gvng much consderaton to the overall performance of the network. A smlar stuaton occurs when the users are members of teams that are sharng the network resources. A user may fnd tself cooperatng wth other members of ts team whch tself s competng wth the other teams n the network. Game theory appears to provde the necessary framework and mathematcal tools for formulatng and analyzng the strategc nteractons among users, or teams of users, of such networks. In ths thess, we nvestgate networks n whch users, or teams of users, ether compete or cooperate for the same network resources. We consdered two mportant network topologes and used many examples to llustrate the varous soluton concepts that we have nvestgated.. Frst we consder two-node
parallel lnk networks wth non-cooperatve users tryng to optmally dstrbute ther flows among the lnks. For these networks, we establshed a condton whch guarantees the exstence and unqueness of a Nash equlbrum for the lnk flows. We derved an analytcal expresson for the Nash equlbrum and nvestgated ts propertes n terms of the network parameters and the users preferences. We showed that n a compettve envronment users can acheve larger flow rates by properly emphaszng the correspondng term n ther utlty functons, but that ths can only be done at the expense of an ncrease n the expected delay. Next, we consdered a general network structure wth multple lnks, multple nodes, and multple competng users. We proved the exstence of a unque Nash equlbrum. We also nvestgated many of ts ntutve propertes. We also extended the model to a network where multple teams of users compete wth each other whle cooperatng wthn the teams to optmze a team level performance. For ths model, we studed the Nonnferor Nash soluton and compared ts results wth the standard Nash equlbrum soluton. v
TABLE OF CONTENTS ACKNOWLEDGEMENTS... X.0 INTRODUCTION....0. COMMUNICATION NETWORKS... 4.. Local Area Networks (LANs)... 6.. The Internet... 8. GAME THEORY..... Strategc Games..... The Nash Equlbrum... 3..3 The Stackelberg Strategy... 5..4 Pareto Domnance and Effcency (Optmalty)... 6.3 A BRIEF SURVEY OF PREVIOUS RESEARCH.... FLOW AND ROUTING CONTROL FOR PARALLEL-LINK NETWORKS.. 30 MATHEMATICAL FORMULATION AND DERIVATION OF THE NASH EQUILIBRIUM... 3. PROPERTIES OF THE NASH EQUILIBRIUM... 36.3 ILLUSTRATIVE EXAMPLES... 39.4 CONCLUSIONS... 47 3.0 FLOW CONTROL FOR GENERAL MULTI-NODE MULTI-LINK COMMUNICATION NETWORKS... 49 3. MODEL FORMULATION... 50 3. EXISTENCE AND UNIQUENESS OF THE NASH EQUILIBRIUM... 53 3.3 ILLUSTRATIVE EXAMPLE... 58 3.4 CONCLUSION... 69 v
4.0 COOPERATIVE FLOW CONTROL FOR GENERAL COMMUNICATION NETWORKS... 7 4. MODEL AND FORMULATION... 7 4. SIMULATION RESULTS... 75 4.3 CONCLUSION... 83 5.0 CONLUSION AND FUTURE WORK... 84 BIBLIOGRAPHY... 87 v
LIST OF TABLES Table.. Costs for user and user n Example.... Table.. Nash equlbra for seven dfferent cases... 4 Table.. Nash equlbra for fve dfferent choces... 4 v
LIST OF FIGURES Fgure.. A Mesh Network... 6 Fgure.. An Ethernet Network... 7 Fgure.3. A small nternet.... 0 Fgure.4. The level curves of J and J... 8 Fgure.. Regon of feasblty for Example.... 40 Fgure.. Flow rate of user on lnk... 43 Fgure.3. Total flow on lnk... 44 Fgure.4. (a) Total flow and (b) resdual capacty on each lnk... 46 Fgure 3.. A four-lnk three-user network... 5 Fgure 3.. Network topology for the llustratve example... 59 Fgure 3.3. Flow rates of all users as a functon of the preference parameter of user.... 60 Fgure 3.4. Flow rates of all users as a functon of the capacty of lnk 7.... 6 Fgure 3.5. Flow rates of all users as a functon of the capacty of lnk 8.... 6 Fgure 3.6. Flow rates for all users as the preference parameter of all users vares... 64 Fgure 3.7. Resdual capactes n some lnks as the preference parameter of all users vares... 65 Fgure 3.8. Flow rates of all users as the preference parameter of the new user vares... 66 Fgure 3.9. Flow rates trajectores for all users for the synchronous algorthm... 68 Fgure 3.0. Flow rates trajectores for all users for the asynchronous algorthm... 68 Fgure 3.. Plots of the Eucldean error norms... 69 Fgure 4.. A network wth 6 lnks and users.... 76 Fgure 4.. Utltes of the users when they are cooperatve and non-cooperatve... 77 Fgure 4.3. Cooperatve and non-cooperatve flow rates of users n Example 4... 78 Fgure 4.4. The contours of the utlty functons of user (red) and (blue)... 79 Fgure 4.5. A network wth 6 lnks and 4 users... 80 v
Fgure 4.6. Flow rates of all users n Example 4..... 8 Fgure 4.7. Utltes of the teams n Example 4..... 8 x
ACKNOWLEDGEMENTS I would lke to thank to my advsor Dr. Marwan A. Smaan for hs supervson and motvaton. I also would lke to thank J. Robert Boston, Lus F. Chaparro, Chng-Chung L, and Prashant Krshnamurthy for servng on my commttee and for ther constructve crtcsm. I thank to my parents Sukrye and Mehmet Sahn, my brother, ssters, and my frend Sarah Colvn for ther motvaton and support durng my doctoral educaton. x
.0 INTRODUCTION Ths thess s concerned wth the optmzaton of mult-node mult-user communcaton networks by usng concepts from game theory. Ths approach allows us to study cooperatve and non-cooperatve nteractons among the users n the networks. The classcal approach requres users to optmze the overall performance of the network [,]. However, a user n a large network such as the Internet may choose to optmze ts own performance rather than the overall network performance [3]. Game theory becomes an mportant tool to model and analyze these types of network [4-6]. Problems n optmal routng, flow control, prcng polcy, and bandwdth allocaton have all been recently formulated and solved usng cooperatve and non-cooperatve soluton concepts from game theory. Effcent use of network resources by a central control system can become dffcult for the networks wth large number of users. Therefore, decentralzed control strateges have ganed consderable mportance as they remove the complexty of a central control archtecture. Snce users n a non-cooperatve network make ther own flow and routng control decsons, the teratve algorthms allowng these users to evaluate ther optmal strateges can consttute the core of decentralzed flow and routng control systems for next generaton networks. These algorthms may concevably be mplemented n the Internet by usng the capablty of IPv4 and IPv6 (Internet Protocol verson 4 and 6 respectvely) whch provde network users to route ther flow on a specfc path [,].
Compettve users decde on ther flow rate and/or routng based on ther utlty functons whch translate ther needs and desres. Utlty functons that have been used n most models typcally represent a combnaton of two objectves: () maxmzng the flow rate and () mnmzng the congeston delay experenced by the user s data. There are numerous ways of combnng these two objectves nto a sngle optmzaton crteron for each user. One approach [7-] s to consder a utlty functon for each user n the form of beneft/cost, also known as a power crteron. In ths approach, the beneft term represents the total throughput and the cost term represents a measure of the average expected delay. Maxmzng the utlty functon n ths case wll acheve the desred two objectves. Another approach that has been consdered s a utlty functon for each user n the form of beneft cost [3-6]. Typcally, the beneft term s a weghted measure of the throughput and the cost term s a weghted measure of the expected delay. The weghts can be vewed as representng the user s preferences for one term over the other. Maxmzng ths utlty functon essentally means ncreasng the throughput whle smultaneously reducng the expected delay. Snce the users may dffer n ther throughput needs and tolerance for the expected delays, and they consequently can adjust the relatve mportance of the beneft and cost terms by selectng approprate preference constants (.e. weghts) for each of these terms n ther own utlty functon. In ths thess, we consder both beneft/cost and beneft - cost types of utlty functons. Snce communcaton networks and game theory are focal ponts of ths nvestgaton, to facltate the understandng of the concepts gven later n ths thess, a bref revew of these two topcs s gven n Chapter. After gvng a short ntroducton of the prncples of how today s communcaton networks work, we descrbe varous soluton concepts n game theory.
In Chapter, we consder a smple two-node parallel lnk network wth multple competng users. Each user decdes on ts total throughput and on the throughput sent on each lnk so as to maxmze ts own utlty functon whch s n the beneft - cost form. The model also provdes the users the flexblty of choosng dfferent preference parameters for dfferent lnks. Ths allows them to adjust the amount of throughput sent on each lnk based on ther prevous satsfacton and experences from these lnks. We gve a condton under whch the exstence and unqueness of a Nash equlbrum s guaranteed. We also study some ntutve propertes of the Nash equlbrum and present two examples to demonstrate these propertes. In Chapter 3, we consder a general network envronment wth multple nodes and multple lnks. Users of ths network can enter and ext from any node n the network. The route of each user s fxed; along a path n the network connectng a source node to a destnaton node specfc for that user. Snce users share the same network resources, they decde on the flow rate compettvely by maxmzng a power-crteron utlty functon. For ths model, ths utlty functon s more advantageous than the beneft-cost form of utlty functons because t results n more analytcally tractable solutons. We prove the exstence of a unque Nash equlbrum and study some ntutve propertes of ths equlbrum. Next, we derve both synchronous and asynchronous numercal algorthms for the users so that they can evaluate ther Nash equlbrum flow rates based on the nformaton obtaned from the network. As ponted out prevously, these numercal algorthms are very desrable because they encourage the establshment of dstrbutve networks. In Chapter 4 to complete the analyss of mult-node mult-lnk networks, we also consder the same general network envronment but wth users that decde to cooperate wth each other. The resultant soluton s called Pareto domnant soluton whch usually gves larger level of 3
satsfacton to every user than the level of satsfacton they would receve from Nash equlbrum flow rates. Next, we expand the model from a sngle team to multple teams whch compete wth each other. We assume that each team has a leader whch ensures cooperaton wthn the team but competton wth the other teams. In the new model, the optmzaton s performed at two level: () The leader of each team competes wth the leaders of other teams n order to acheve the best utlty for ts team. Each team leader tres to optmze a socal utlty functon whch s the scaled sum of all the utlty functons of the users n ts team, and () At the team level, the team members cooperate to acheve a team level satsfacton rather than an ndvdual level satsfacton. The resultant equlbrum s called Nonnferer Nash equlbrum [7, 8]. By means of two examples, we llustrate the Nash, Pareto, and Nonnferor Nash equlbrums and compare ther performances. In Chapter 5, we summarze some mportant concludng remarks n ths thess and we also present some suggestons on possble extensons of the current research.. COMMUNICATION NETWORKS Processng power of the early computers was lmted and few of these computers could run calculaton-dense applcatons. In the 960 s, the U.S. Department of Defense Advanced Research Project Agency (ARPA) decded to develop frst data network, ARPANET, so that all research groups can share these hghly powered computers []. The prncple purpose was to desgn a relable data network whch can perform even though some parts of the network fal. Ths early network has evolved and eventually led to today s Internet, the global network nterconnectng networks around the world. 4
Communcaton networks can be classfed nto three dfferent networks, namely crcut swtched networks, packet swtched networks, and vrtual swtched network based on the swtchng type.. In crcut swtched networks, a crcut between a source node and a destnaton node s set up before the communcaton starts. Here, crcut refers to all network resources such as swtches, routers, communcaton lnks, etc. allocated for the communcaton between the source and the destnaton nodes. The man characterstc of ths network s that network resources allocated for a communcaton between two nodes are dedcated to ths communcaton. Therefore, these resources are not avalable to other users untl the communcaton ends and the resources are released. However, dedcated resources can guarantee a qualty of servce n these networks. In packet swtched networks, each source node splts ts data nto small blocks of nformaton bts, called packets, then sends these packets to the destnaton. Snce each packet has the source and destnaton addresses, a packet can be treated as an ndependent entty so that multple sources can send ther data through the same network resources. In the vrtual swtched networks, each communcaton sesson between two nodes starts wth settng up a vrtual crcut between these two nodes. Both nodes use ths vrtual path to send and receve packets untl the end of communcaton. Vrtual crcuts support dfferent level of qualty servces whle many users stll can share the same network resources. Communcaton networks can also be classfed based on the locaton. A network consstng of computers n a buldng or n a small geographcal area s called local area network (LAN). A wde area network (WAN) can contan computers n a very large geographcal area. To understand how computer networks work, frst we wll ntroduce local area network, and then we wll dscuss how an nternet works. We wll try to keep the dscusson as compact as possble by gvng only the general underlyng concepts and gnorng the detals. 5
.. Local Area Networks (LANs) Early communcaton networks used a connecton scheme n whch there was at least one lnk between any two nodes as llustrated n Fgure.. They are also called pont-to-pont networks or mesh networks. Fgure.. A Mesh Network The number of lnks n mesh networks grows exponentally for each addtonal computer. Therefore, the constructon cost of these networks can be prohbtvely hgh. Another mportant shortcomng of these networks s that many lnks are not used most of the tme. In modern LANs, almost all resources n the network are shared by all computers, therefore, the cost of buldng networks reduces and these resources are used more effcently. Only one transmsson at a tme can take place n shared networks (n the medum) whle others have to wat ther turn. 6
There are many LAN technologes wth ther own specfcaton about the topology of the network, the format of the packet, the modulaton scheme, etc. Topology of the network specfes how to connect computers wth each other. We use the term frame nstead of packet for a gven LAN technology snce each LAN technology has ts own packet specfcaton. We wll brefly explan how an Ethernet network, a well known LAN technology, operates. A typcal Ethernet network usng 0Base-T wrng scheme s shown n Fgure.. Each node n the network needs a network nterface card, called Ethernet card whch has a unque Ethernet address that comes wthn ts electronc crcutry. The Ethernet hub located at the center of the network s also a crcut that regulates the transmssons n the network. It allows only one user to transmt at a tme, thus, manly performng a multplexng functon. Fgure.. An Ethernet Network An Ethernet card s a collecton of crcuts prnted on a board. It s plugged nto the mother board of the computer. The Ethernet address s also called the physcal address of the node. 7
Suppose that node A wth Ethernet address a has some data to transmt to node B wth Ethernet address b n Fgure.. Frst, node A checks the network for any on-gong transmsson. If there s no on-gong transmsson, node A, usng the standard Ethernet frame format, puts the source and destnaton addresses, and ts data nto frames then t sends the frames to the hub. After recevng the frames, the hub copes each frame to all other lnks so that all computers can get the frames. Eventually, each computer extracts the destnaton Ethernet address from the frames t receved and compares t wth ts own Ethernet address. They wll match only for node B, thus, only node B wll keep the frames whle other nodes do not. Two or more transmssons can not take place at the same tme n an Ethernet snce all lnks and the hub are used for only one transmsson. Therefore, these Ethernets are called shared Ethernets. If two or more nodes start transmsson at the same tme, then the hub wll nform all nodes that a collson happened. Beng aware of the collson, the source nodes wat for random perods of tme before re-transmttng. The mechansm of how to access to the medum, descrbed very shortly the above, s an mplementaton of CSMA/CD (carrer senstve multple access / collson detecton) protocol n Ethernet. Due to performance concerns, there s a lmt on the number of nodes that can be connected to the same Ethernet hub. Performance can be ncreased by nterconnectng Ethernets wth swtches or routers... The Internet An nternet conssts of two or more networks such as Ethernets connected by routers. Each node n an nternet has a unque network address and an Ethernet address. Unlke Ethernet addresses, a network address s usually based on the geographcal poston of the node. Routers 8
make the decson of where to send the packet based on the network address not the Ethernet address. A smple nternet llustrated n Fgure.3 has fve Ethernets whch are nterconnected by four routers: R, R, R3, and R4. Suppose that node A wants to send some data to node B. The Ethernet addresses of node A and B are a and b and ther network addresses are A and B, respectvely. Node A sends ts Ethernet frame [a, r A, B data] to the router t s connected to. After recevng the frame, router R makes some modfcatons on the frame such as t throws away the Ethernet addresses and puts the nformaton nto a packet wth IP (Internet Protocol) format. Then R checks ts routng table to fnd out whch router destnaton address B s connected to. In ths case t s R4. R sends the IP packet [A, B data] to R4. R4 checks ts routng table and delver the packet to R3. Fnally, R3 fnds out that the destnaton s n an Ethernet and sends the Ethernet frame [r3, b A, B data] to the Ethernet hub whch copes the frame to ts all output ports and eventually, the destnaton node b receves the frame. The global Internet nterconnects the networks around the world. Two mportant protocols, TCP (Transfer Control Protocol) and IP (Internet Protocol), usually referred together as TCP/IP, are used for exchangng data n the Internet. IP specfes how to assgn a unque network address to each computer n the Internet and how to send a packet from a source node to a destnaton node. IP defnes a unversal packet format to be able to perform packet delvery n the Internet whch contans varous networks wth dfferent frame formats and technologes. IP supports best effort servce for delverng packets from a source to a destnaton node n the Internet. It doesn t guarantee that the packets mght be delayed, duplcated, lost, or corrupted wth transmsson errors. 9
TCP protocol has emerged to prevent these undesrable effects and provde a relable transportaton n the Internet. Ths protocol acheves relablty by means of an acknowledgment mechansm whch requres a destnaton node to notfy the source node about whether t receved the transmtted packets successfully. Ths protocol also performs flow and congeston control based on nformaton avalable to t so that network resources are used effcently. Fgure.3. A small nternet. 0
. GAME THEORY Game theory s an mportant mathematcal tool to analyze many problems orgnatng from varous dscplnes. Problems dealng wth cooperatve and compettve enttes n engneerng, economcs, poltcal scence, and many other felds are modeled and analyzed usng game theoretc prncples. Two mportant soluton concepts from game theory are Nash equlbrum and Pareto domnance whch are usually used to nvestgate cooperatve and competng enttes respectvely. A player, also called user n the context of communcaton networks, s the basc entty n models of game theory. Each user has to make a decson about whch acton t 3 should take from a set of avalable actons. Each user optmally chooses ts acton among the set of all actons to maxmze a utlty functon (payoff functon) or to mnmze a cost functon. These functons translate the level of satsfacton assocated wth each possble acton. In other works, these functons defne a preference relaton for the users. For nstance, f x and y are two possble choces, a ratonal user wll take acton x f the utlty of choosng x s hgher than the utlty of choosng y, or f the cost of choosng x s lower than the cost of choosng y. Games arse n stuatons where there s nterdependence among users. That s, n stuatons where one user s payoff s not only a functon of ts choce but also a functon of all other users choces. For nstance, the amount of proft of a company depends on ts prce settng as well as on other companes prce settngs. Therefore, to maxmze ts utlty functon or mnmze ts cost functon, a user makes ts decson by takng nto account other users possble decsons. 3 A user can be thought as a software entty whch makes optmzatons and decsons n computer networks. So we wll refer each of them as t.
Even though games can be classfed nto many dfferent categores, we wll emphasze only three of them. a) Based on cooperaton among the users, a game can be cooperatve or noncooperatve. In non-cooperatve games, each user s only concerned about ts own payoff and doesn t pay attenton to other s payoffs. On the other hand, n cooperatve games, users collaborate to ncrease ther ndvdual as well ther mutual (socal) payoff functons. b) Games can be strategc (also called statc) or repeated (also called dynamc). In strategc games, users smultaneously and only once make decsons. Dfferently, n repeated games, users nteract more than once and play the game many tmes. In these games, users future payoffs depend on ther current strategc choces. c) In nonherarchcal games, no user enforces ts strategy to other users. However, n herarchcal games, a user called the leader 4 can mpose ts strategy on another user whch s called the follower. The soluton concept for these games s called Stackelberg equlbrum [9]. The leader makes a choce to maxmze ts own utlty ahead of the follower and then allow the follower to know ts choce. Based on the leader s choce, the follower decdes on ts actons to maxmze ts own utlty functon. In ths thess, we wll concentrate manly on non-cooperatve and cooperatve strategc games... Strategc Games A strategc game, sometmes also called one shot game, has users who make ther decsons only once, smultaneously, and ndependently [5,6]. Once means that users nteract only one tme and they fnsh the game by announcng ther decsons smultaneously. Smultaneous decson makng can be realzed n many ways. In one scheme, each user sends ts decson to a 4 There can be more than one leader n a herarchcal game. We focus on the games wth only one leader.
central computer (not necessarly at the same tme) and then the central computer publshes all users decsons at once. In statc games, users are aware of each others utlty functons so they know the strategc nteractons n a game. Let N = (,,..., N ) be the set of users and N be the number of users n the game. A generc user has to choose ts acton 5 a from ts set of possble actons A. The outcome of the game s gven by an acton profle a = a a a N (,,..., ) whch s a collecton of choces made by all users. An acton profle belongs to the set A whch s the Cartesan product of acton sets of all users,.e. A = ser has the utlty functon U : A R fnes N A. U that de the user s preference relatons. Notce that the utlty of user does not only depend on ts acton a A but also depends on the choces of all users a A gven the above, a strategc game can be denoted by N,( A ),( ). In summary, by usng the notaton U. We nterchangeably use U ) ( a an d U ( a, a ) to stress the effect of all other users actons on user s utlty. The decsons made by all users except user s denoted by a A, + N to be pr ecse, a = ( a, a,..., a, a,..., a ) = ( a k \{ }. Smlarly, ) k N A denotes the set A k = k N \{ } A... The Nash Equlbrum Nash equlbrum s one of the most commonly used soluton concepts when there s no cooperaton among the users or when t s hard to mpose cooperaton. In these cases, regardless of what other users do, each user wants to maxmze ts own payoff. Nash equlbrum s safe 5 The words acton and strategy are used nterchangeably even though they are not synonyms. 3
aganst greedy efforts of any user who wants to ncrease ts payoff by devatng from t. A user can only have worse payoff f t changes ts Nash strategy unlaterally. Therefore, the Nash strategy s an equlbrum and can be thought as a steady-state pont of the game. Non-cooperatve games naturally lead to dstrbuted control systems n whch decson makers are ndvduals. A central authorty that makes decsons for all users s not needed n dstrbuted systems. Ths elmnates the complex control sgnalng schemes whch are necessary for a central optmzaton. Therefore, the overall system desgn 6 s much smplfed resultng nto easly expandable communcaton network. Now, let us gve a precse defnton of the Nash equlbrum [5,6]. Defnton : The acton profle a * ( A ) ( U ) A s a Nash equlbrum of the strategc game N,, f t satsfes the followng property for each user N : U a a U a a a A * * * (, ) (, ) for all () It s clear from the defnton that each user N wll only lose f t changes ts Nash * strategy a to any other choce a. As a result, all users n the game wll prefer not to change ther Nash strateges. Frst, let us gve the defnton of the best actons 7 of a user and then gve another defnton of Nash equlbrum. Defnton : Gven other users actons a A, the set of best actons of user s gven by B ( a ): ' ' { } B ( a ) = a A : U ( a, a ) U ( a, a ) for all a A () 6 However the complexty of the cell phone mght ncrease. 7 Best acton s also called best response or ratonal response. 4
Defnton 3: A Nash equlbrum of the strategc game,( N A ),( U ) s an acton profle a wth the property: * * a B ( a ) for all N (3) Based on ths defnton we can evaluate a Nash equlbrum by obtanng the best responses of all users and then search for an acton profle a such that * * a B ( a ) for all N. The second step corresponds to fndng the soluton of the problem wth N varable and N equatons, f best response functons, B s, are sngleton-valued functons...3 The Stackelberg Strategy In prevous secton, we consdered games n whch the users were able to make decsons ndependently and smultaneously. The Nash equlbrum s the approprate soluton concept for these games. But there are stuatons where one user has a domnant role over other users and affects ther choces. Also, there are cases where there s no smple way for the users to make ther decsons smultaneously and one user can declare ts choce before other users [9]. These cases can be modeled and analyzed as Stackelberg games. In these games, as mentoned before, there are two types of users, namely, the leader 8 and the follower. The leader s the user whch mposes ts decson over the follower by declarng ts decson before the follower. The follower s the user whose decson s affected by the leader s decson and t reacts to the leader s decson ratonally [5,6]. Let A and A be the acton (strategy) sets of user and user respectvely. User and user would lke to mnmze ther cost functons 8 Sometmes, the leader s also called the manager. 5
J( a, a) and J( a, a), respectvely. Let us gve the formal defnton of the Stackelberg equlbrum for user to be the leader [9]: Defnton 4: S S ( a, a ) s called a Stackelberg strategy par of the game f. There exsts a mappng T : A, such that, for any fxed A a A, J ( Ta, a ) J ( a, a ) for all a A.. There exsts a a S S S A such that J Ta a J T ) (, ) ( a, a fo r all a A. Let us gve another defnton of the Stackelberg equlbrum wth user as the leader. Defnton 5: ( a, a ) B S S S S ( a, a ) s a Stackelberg strategy par wth player as leader f and only f and J ( a, a ) J ( a, a ), for all ( a, a ) B (4) S S where B s the best acton set of user. The leader evaluates ts Stackelberg strategy by obtanng the best response of the follower for ts all actons a A and then by performng a maxmzaton based on ts acton and the follower s best response...4 Pareto Domnance and Effcency (Optmalty) It s well known that the Nash equlbrum s not usually effcent [0]. The neffcency of a non-cooperatve system can be nterpreted as the cost of havng a dstrbuted control system. 6
In some cases, users can cooperate or they can be forced to cooperate to have a more effcent operatng pont so that some or all users can beneft from t. A Pareto domnant acton profle ncreases some users utlty wthout hurtng any users utlty. On the other hand, for any acton * profle a A, we can fnd an acton profle a n Pareto effcent (optmal) acton profle set * A A that Pareto domnates the acton profle a. That s, * * a A s Pareto optmal f we cannot * fnd any a A that Pareto domnates a. Pareto domnance and optmalty are formally gven n the followng defnton. Defnton 6: An acton profle â s sad to be Pareto domnant over another acton profle a f U ( aˆ ) U ( a) for all N and U ( aˆ ) > U ( a) for some N. Moreover, an acton profle s sad to be Pareto effcent (optmal) f there s no other acton profle a such that * a U a U a * ( ) ( ) * for all N and U ( a) > U ( a ) for some N. above. Let us gve an example that demonstrates the game theoretcal concepts descrbed the Example.: Consder a statc game where there are two users. The cost functons of user and user are J ( x, x ) = ( x.5) + ( x 0.5) x x + x + x (5) J ( x, x ) = 3( x.) + ( x.) x x + x + x (6) respectvely. Let us consder the non-cooperatve case n whch each user wants to mnmze ts own cost functon. The level curves of and J can be seen n Fgure.4. User and user choose actons x R and x J R respectvely. Notce that acton sets of both users are the 7
same,.e. X X R. Let X = X X be the acton profle set of the game. If user had control on both actons x and x, t would choose the acton profle (.66,.33) that * corresponds to the mnmum of, denoted by J n Fgure.4. Smlarly, f user had control J on both actons t would choose the acton profle (.43,.4) that corresponds to the mnmum * of, denoted by. J J Fgure.4. The level curves of J and J 8
The best actons of user, B( x), can be drawn by jonng the ponts at whch lnes of constant x are tangent to the level curves of. Smlarly, the best actons of user, B ( x ), J can be drawn by jonng the ponts at whch lnes of constant J x are tangent to the level curves of. Although the best actons can be found by means of level curves, ther analytcal expressons can be easly found for ths example. We mnmze for gven x to fnd B ( x ). J Therefore, wll be only a functon of J x when x s fxed. The mnmzer of J satsfes the frst order necessary condton (FONC): dj( x) * = ( x.5) x + = 0 x = B( x) = 0.5x + (7) dx Second order necessary condton (SONC), d J x dx ( ) = > 0 (8) s also satsfed, thus, * x s the mnmzer for gven x. In the same way, the best acton set of user can be found: B ( x ) = 0.5x +.7 (9) Fndng best acton sets are complete. Now, from the second defnton of the Nash equlbrum, we have to fnd an acton profle ( x, x ) such that ( x, x ) B and ( x, x ) B (0) The ntersecton pont of and B satsfes ths condton. Therefore, equate two best responses: B N N B = B x 4 = 0.5x +.7 ( x, x ) = (3.8,3.6). N In Fgure.4 the pont N corresponds to the Nash equlbrum of the game ( x, x N ). 9
Now let us fnd the Stackelberg equlbrum when user s the leader and user s the follower. The leader knows the follower s best acton for ts each acton. Therefore, the leader performs an optmzaton based on the follower s best acton. We nsert B ( x ) nto the leader s cost functon J( x, B( x)) and mnmze J( x, B( x)) wth respect to x to obtan the leader s strategy: ( ) J ( x, B ( x )) = ( x.5) + (0.5x +.7) 0.5 x (0.5x +.7) + x + (0.5x +.7) () dj S S S.5 4 0.66 ( ) 3.03 x x and x B x dx = = = = = () The pont ndcated by S corresponds to the Stackelberg equlbrum of the game, ( S S x, x ) = (.66,3.03), when the frst user s the leader. Smlarly, for the case n whch user s the leader and user s the follower, the Stackelberg equlbrum can be evaluated as S S ( x, x ) = (.5,), whch s denoted by S n Fgure.4. * * The curve jonng the ponts of tangency between the level curves of and J n Fgure.4 corresponds to the Pareto optmal ponts of the game. In ths case, the users cooperate to mnmze a common cost functon gven as follows: J J( x, x ) = α J + ( α) J 0 α (3) It s clear that f both users mnmze J, ndrectly, one user chooses ts acton profle that mnmzes ts own cost functon as well another s cost functon. Let us fnd an analytcal expresson for Pareto optmal acton profles: J = α ( x.5) + ( x 0.5) xx + x+ x + ( α) 3( x.) + ( x.) xx + x + x (4) Wrte FONCs: 0
J = α( 4x +.) + 6x x 6. = 0 x (5) J = 3.4α + x x 3.4 = 0 x (6) Smultaneous soluton of equatons (5) and (6) results n: P 7.8α + 5.8 P 3.6α 36.α + 6.6 = and x = x 8α 8α (7) It s easy to check that ( P P x, x ) satsfes SONCs, therefore t s the mnmzer of J. Let us wrte the Pareto optmal acton set * X formally becomes: = (, ): =, =,0 α (8) 8α 8α X * x x x 7.8α + 5.8 x 3.6α 36.α + 6.6 Now consder the acton profles (.65,.8), (.43,.4), and (0.90, 0.47) ndcated by R, S, and T n Fgure.4, respectvely. The costs of user and user for these acton profles are gven n Table.. Table.. Costs for user and user n Example. when acton profles R, S, and T are used. J J R.9. S.5. T 3.5 4.
The acton profle S Pareto domnates the acton profle T because both users have smaller costs at profle S than at profle T. Smlarly R also Pareto domnates T. A pont n a Pareto optmal set doesn t necessarly mean to domnate all other ponts. For example, even R X *, t does not domnate the profle S as the cost of user s hgher at profle R than profle S. It s also easy to see that there s no other profle that Pareto domnates R. Therefore, t s a pont n Pareto optmal acton profle set..3 A BRIEF SURVEY OF PREVIOUS RESEARCH Models of game theory have been extensvely appled to a wde varety of problems arsng from communcaton networks. Problems of routng control [3,0,5,7,8,-33], flow control [7-9, -3, 34-4], capacty allocaton [43-46], and prcng[47-49] the network resources have receved consderable attenton n the control and communcaton lterature. In a routng control problem, a user tres to fnd the path(s) through whch t should send ts throughput demand to have the best performance wth respect to some performance crteron. In a flow control problem, a user asks how much flow t should send to the network to fnd ts optmum flow control strategy. Dfferently, n the capacty allocaton problems, users decde how much of the network resources they should allocate. Lastly, prcng of the network resources has been used to ncrease the use of network effcency. The exstence and unqueness of the Nash equlbrum consttutes one of the most mportant problems. If there exsts a unque Nash equlbrum, then we can analytcally or numercally obtan ths equlbrum pont. We may also try to tune the network parameters so that effcent use of the network resources s acheved at the Nash equlbrum.
There are few communcaton network models for whch the unqueness of the Nash equlbrum s establshed [50]. In [3], the authors provde a smple example to llustrate the dffculty of havng a unque Nash equlbrum. In the example, a network wth only four nodes has two Nash equlbrums. Therefore, usually, the unqueness of the Nash equlbrum s establshed case by case. Rosen s dagonal strct concavty (DSC) condtons are used to establsh the unqueness of the Nash equlbrum n some studes. These condtons guarantee the exstence and unqueness of the Nash equlbrum for convex games. DSC condtons are shown to be satsfed for general topology networks wth polynomal costs n []. Also, these condtons are satsfed by the parallel lnk networks wth two users and two lnks and by general topology networks under lght traffc condtons as n [3]. A user n a compettve routng network makes ts decson of how to splt ts gven throughput demand nto the avalable paths. Orda et al [3] study ths problem for parallel lnk and general topology networks wth L lnks and I users. In ths game, each user decdes on the amount of flow for each lnk to mnmze ts own cost functon wthout payng attenton to other users performance. In other words, by usng the notaton n the paper, user mnmzes ts own cost by adjustng ts flow confguraton vector f { f, f,..., f L } = where f denotes the amount of flow user puts on lnk l. The outcome of the game s expressed by the system flow l confguraton that s the Nash equlbrum of the game gven by I F = (f,f,...,f ). The Nash equlbrum s nvestgated for a wde varety of cost functons that satsfy some mld convexty condtons. The cost functon of each user s expressed as the sum of all lnk costs. The flow of a generc user through a lnk ncurs some cost, called the lnk cost. The exstence of Nash equlbrum s establshed for both networks for ths cost functon. The unqueness of the Nash 3
equlbrum for the network of parallel lnks s establshed under the assumpton that all users send ther flows over the same set of lnks. The unqueness of the general topology networks demands even more restrctve condtons. It s establshed by means of Rosen s DSC condtons whch hold for a lghtly loaded general topology networks. The Nash equlbrum s also stable and unque for a specal case of the parallel lnk networks wth two users and two lnks. In addton to the exstence and unqueness of Nash equlbrum analyss, ntutve propertes of the Nash equlbrum called monotoncty propertes are also explored. For nstance, the user wth hgher throughput demand uses a larger porton of each lnk than the user wth lower throughput demand. They show that the general topology networks mght not have these propertes. La and Anantharam consder the same compettve routng problem descrbed above n a dynamc game context n []. The users are the Network Access Provders (NAPs) who compete wth each other to support the best servce to ther ndvdual network users. They nteract many tmes wth each other untl the state of the network changes consderably due to the change of the number of the network users, or the topology of the network, or the load over the network. In practce, NAPs can communcate wth each other before they make ther decsons; so they can negotate about the effectve use of the network. It s natural to consder that each user wants to ncrease ts own performance. Therefore, none of the users mght desre to cooperate unless there s a reward for cooperaton or mght keep ts agreement unless there s a punshment for devatng from t. These nteractons among the users cannot be analyzed by statc games. Dynamc games provde a better understandng of ths stuaton. In ths settng, t s possble to have a Nash equlbrum that yeld a mnmum total system cost and every user of the network has a cost that s not larger than that of the statc game settng. That s the case for 4
parallel lnk networks and the Nash equlbrum whch s socally optmal s called subgameperfect Nash equlbrum (SPNEP). SPNEP also exsts for general topology networks f all users have the same source and destnaton nodes and f some lght techncal condtons hold for the network. It mght not exst n a general topology network wth users havng dfferent source and destnaton nodes. Routng n communcaton networks and transportaton networks have some smlartes n nature. In transportaton networks, one drver has neglgbly small effect on the other drvers and the soluton concept s Wardrop equlbrum [50]. On the other hand, users flows n the communcaton networks are not neglgble. Altman et al [] consder the routng control problem n general topology networks wth polynomal costs. Users have a fxed amount of flow demand to send to the destnaton and they have polynomal cost functon that s borrowed from the road traffc context. The cost functon s defned by the US Bureau of Publc Roads. The exstence and unqueness of the Nash equlbrum are establshed for the general topology networks. Moreover, t s shown that the Nash equlbrum does also result n a socally optmal network operatng pont at whch the total cost of the network s mnmum. It s well known that the Nash equlbrum s not usually effcent. Therefore, dfferent mechansms are suggested to have an operatng pont whch s more effcent then the Nash equlbrum [3,33,43]. Desgn parameters of the network and prcng the network resources are two examples of these mechansms. Korls et al [33] consder the problem of archtectng a non-cooperatve network wth I users compettvely routng ther flows through the network just lke the users n [3]. In ths study, the neffcency of the Nash equlbrum s overcome by means of two technques. The frst technque s employed n the provsonng phase,.e., durng the constructon of the network. The desgner of the network adjusts the capacty of each lnk so 5
that the resultant network yelds a Nash equlbrum whch s system wde effcent. System wde effcency means that the overall network performance s optmum wth respect to some performance crtera, n ths case the crteron s the mnmum network cost. The desgner s able to acheve ths goal snce t s assumed that users are ratonal and ther performance crtera are known. A mappng, called Nash mappng, assgns each capacty confguraton of the network to a unque Nash equlbrum (system flow confguraton). The unque Nash mappng allows them to compare the Nash equlbrums to obtan the capacty confguraton that yelds the mnmum total system cost. To be able to compare dfferent Nash equlbrums, t s also assumed that all users send flows over the same set of lnks for dfferent lnk capacty settngs. For the network of parallel lnks, addng all avalable capacty to the lnk whch has ntally hghest capacty s the user prce optmal soluton. A capacty confguraton s the user prce optmal f t mnmzes all users prces. Prce of a user, called margnal cost n economy, s the partal dervatve of cost functon wth respect to the user s own flow. The second technque for overcomng the neffcency of the Nash equlbrum s employed n the run tme. The network manager has control over some porton of the flow to route through the network. The manager adjusts the amount of flow for each lnk just lke other users to lead to a Nash equlbrum at whch the mnmum average network delay s acheved. Thus, the manager plays a socal role n ths game to ncrease the effcency of the system. Snce the manager actvely adjusts the avalable capacty to other users by ts flow; ths technque s smlar to the capacty assgnment technque durng the provsonng phase. But, dfferently, the manager s flow consumes some resources and ncurs some cost to the network. Authors show that the manager s flow demand must exceed a threshold to acheve ts goal. Interestngly, the threshold decreases wth ncreasng load n the network. In other words, the manager s job s easer for a heavly loaded network. Lastly, 6
n ths study, the authors also nvestgate Braess paradox. That s, the addton of some more capacty to a network can degrade all users performances. The authors prove that ths paradox does not occur n parallel lnk networks. One way to prevent ths paradox from occurrng n general topology networks s that the avalable capacty should be added to all lnks of the network unformly. The routng control problems we descrbed earler nvolves routng of a gven flow demand through the network. The nature of the problem s completely dfferent f the users are to send an unspecfed amount of flow on the network and the problem becomes an optmal routng and flowng control problem. In other words, users smultaneously decde on ther flow and routng strateges. Each user fnds ts optmum amount of flow for each lnk. Altman et al [0] consder ths problem for parallel lnk networks for an arbtrarly large number of users. Users am to maxmze ther utlty functon whch s n the form of beneft/cost. As mentoned before, ths form of the utlty functons s recognzed as the power crteron. A postve power of the total throughput of the user s consdered to be the beneft term whle the average expected delay experenced by the user s flow s consdered to be the cost term. Maxmzng ths form of utlty functon can be nterpreted as maxmzng the throughput whle mnmzng the delay. The power of the total throughput, adjustng the tradeoff between throughput and delay, can be user specfc. For ths non-concave utlty functon the authors fnd an explct expresson of the soluton for a sngle user that uses M parallel lnks. Interestngly, t s possble to have an optmal flow confguraton for the user such that some lnks are not used. They also show that there exsts a unque Nash equlbrum as the number of users becomes arbtrarly large. The large number of users results nto the Nash equlbrum whch has delay-equalzng property that delay for each lnk becomes the same. 7
As mentoned before, prcng mechansm s also wdely used to utlze the resources effectvely and ncrease the system performance. Korls et al [47] consder a prcng mechansm that leads to mnmum network congeston. The authors suggest that prce of a lnk s drectly proportonal to the congeston level of the lnk. Therefore, hghly congested lnks wll have hgh prces that wll produce an ncentve for users to use the lghtly congested lnks wth low prces. The manager s objectve s to have the network operate at a target pont at whch the mnmum network congeston cost s acheved. A prcng vector of the network contans the prces for all lnks. The manager chooses the prcng vector and all network users use ths prcng vector to decde on ther flow rates by mnmzng ther cost functons. Flow rates of all users consttute the system flow confguraton. The authors prove that there exsts a unque prce vector that nduces a unque Nash equlbrum at whch the system flow confguraton and the manager s target system operatng pont matches up for parallel lnk networks. For a general network case such as the Internet where users cost or utlty functons are not known, they ntroduce an adaptve algorthm to fnd the prce vector. They obtan the suffcent condtons for the algorthm to converge to the optmum prce vector. Rhee and Konstantopoulos [45] consder a bandwdth allocaton problem for a parallel lnk network. Each user reserves the amount of bandwdth that maxmzes ts own utlty functon. Utlty functons are assumed to be concave and smooth. In ths model, users can have dfferent utlty functons and the users throughput s bounded between a maxmum and a mnmum value. In prevous works, each user had a fxed bandwdth demand or the users demands were lmted by the capacty of the resources. The exstence and unqueness of the Nash equlbrum s establshed for parallel lnk networks and also for general topology networks wth users establshng vrtual paths along a fxed route. That s, no user splts ts flow over the lnks but 8
each user sends ts flow along one vrtual path n the fxed routng scheme. Gauss-Sedel teratve method s shown to be convergng to the unque Nash equlbrum for networks wth one lnk and many users. Lastly, n study [5], an elastc traffc model s studed. Traffc arsng as a result of controllng flow rates wth respect to avalable bandwdth wthn a network s referred as elastc. In ths model, each user chooses a prce per unt tme. Based on ths prce, the network assgns a flow rate to ths user by optmzng a network performance crteron. For ths network model, the authors establshes the stablty of the algorthms whch are based on addtvely ncreasng and multplcatvely decreasng flow rates. They generalze these results to large scale broadband communcaton systems n [53]. 9
.0 FLOW AND ROUTING CONTROL FOR PARALLEL-LINK NETWORKS In ths chapter, we consder a two-node parallel lnk network wth multple competng users. Ths smple type of network s encountered n many of today s communcaton networks n several dfferent ways [43]. For nstance, n a broadband network, users are assgned some preallocated network resources and ndependent vrtual paths are created by splttng the avalable bandwdth. Each vrtual path may be consdered as a lnk n a parallel lnk network model. Smlarly, to smplfy routng n a complex communcaton network, users may be restrcted to send ther data flow on a specfed number of paths between any two nodes n the network. Another example s that of an enterprse whch s served by many ISPs. The connectons to the ISPs can be modeled as parallel lnks and the enterprse may have dfferent preferences for each connecton based on the prce and ts prevous satsfacton wth that ISP. Each user of the two-node parallel lnk network decdes on ts flow rate whch maxmzes a beneft-cost form of utlty functon. Snce our network conssts of several parallel lnks, we provde each user wth the flexblty of usng dfferent preference constants for dfferent lnks n ther utlty functons. That s, we allow the preference constants to be lnk-dependent. In dong ths, our model gves each user the opton to choose the preference constants not only to balance between throughput and delay but also to reduce the usage of lnks that they perceve to be defectve and ncrease the usage of lnks that they perceve to best meet ther needs. 30