Simultaneous Adversarial Multi-Robot Learning

Size: px
Start display at page:

Download "Simultaneous Adversarial Multi-Robot Learning"

Transcription

1 Simultneous Adversril Multi-Robot Lerning Michel Bowling nd Mnuel Veloso Computer Science Deprtment Crnegie Mellon University Pittsburgh PA, Abstrct Multi-robot lerning fces ll of the chllenges of robot lerning with ll of the chllenges of multigent lerning. There hs been gret del of recent reserch on multigent reinforcement lerning in stochstic gmes, which is the intuitive extension of MDPs to multiple gents. This recent work, lthough generl, hs only been pplied to smll gmes with t most hundreds of sttes. On the other hnd robot tsks hve continuous, nd often complex, stte nd ction spces. Robot lerning tsks demnd pproximtion nd generliztion techniques, which hve only received extensive ttention in single-gent lerning. In this pper we introduce GrWoLF, generl-purpose, sclble, multigent lerning lgorithm. It combines grdient-bsed policy lerning techniques with the WoLF ( Win or Lern Fst ) vrible lerning rte. We pply this lgorithm to n dversril multirobot tsk with simultneous lerning. We show results of lerning both in simultion nd on the rel robots. These results demonstrte tht GrWoLF cn lern successful policies, overcoming the mny chllenges in multi-robot lerning. 1 Introduction Multi-robot lerning is the chllenge of lerning to ct in n environment contining other robots. These other robots, though, hve their own gols nd my be lerning s well. Other dpting robots mke the environment no longer sttionry, violting the Mrkov property tht trditionl singlegent behvior lerning relies upon. Multi-robot lerning combines ll of these multigent lerning chllenges with the problems of lerning in robots, such s continuous stte nd ction spces nd miniml trining dt. A gret del of recent work on multigent lerning hs looked t the problem of lerning in stochstic gmes [Littmn, 1994; Singh et l., 2000; Bowling nd Veloso, 2002; Greenwld nd Hll, 2002]. Stochstic gmes re nturl extension of Mrkov decision processes (MDPs) to multiple gents nd hve been well studied in the field of gme theory. The trditionl solution concept for the problem of simultneously finding optiml policies is tht of Nsh equilibri. An equilibrium is simply policy for ll of the plyers where ech is plying optimlly with respect to the others. This concept is powerful solution for these gmes even in lerning context, since no gent could lern better policy when ll the gents re plying n equilibrium. Multigent lerning in stochstic gmes, thus fr, hs only been pplied to smll gmes with enumerble stte nd ction spces. Robot lerning tsks though hve continuous stte nd ction spces, nd typiclly with more thn just couple dimensions. Discretiztions of this spce into n enumerble stte set lso do not typiclly perform well. In ddition, dt is considerbly more costly to gther, nd millions of trining runs to lern policies is not fesible. Typicl solutions to this robot lerning problem is to use pproximtion to mke the lerning trctble nd generliztion for more efficient use of trining experience. In this pper we introduce new lgorithm, GrWoLF 1, tht combines pproximtion nd generliztion techniques with the WoLF multigent lerning technique. We show empiricl results of pplying this lgorithm to problem of simultneous lerning in n dversril robot tsk. In Section 2, we give brief overview of key concepts from multigent lerning long with the the forml model of stochstic gmes. In Section 3, we describe prticulr dversril robot tsk nd its chllenges for lerning. In Section 4, we present the min components of GrWoLF: policy grdient scent nd the WoLF vrible lerning rte. In Section 5, we present experimentl results of pplying this lgorithm to our dversril robot tsk, nd then conclude. 2 Multigent Lerning Multigent lerning hs focused on the gme theoretic frmework of stochstic gmes. A stochstic gme is tuple (n, S, A 1...n, T, R 1...n ), where n is the number of gents, S is set of sttes, A i is the set of ctions vilble to gent i (nd A is the joint ction spce A 1... A n ), T is trnsition function S A S [0, 1], nd R i is rewrd function for the ith gent S A R. This looks very similr to the MDP frmework except we hve multiple gents selecting ctions nd the next stte nd rewrds depend on the joint ction of the gents. Another importnt difference is tht ech gent 1 GrWoLF is short for Grdient-bsed Win or Lern Fst, nd the hs the sme sound s in grdient.

2 hs its own seprte rewrd function. The gol for ech gent is to select ctions in order to mximize its discounted future rewrds with discount fctor γ. Stochstic gmes re very nturl extension of MDPs to multiple gents. They re lso n extension of mtrix gmes to multiple sttes. Ech stte in stochstic gme cn be viewed s mtrix gme with the pyoffs for ech joint ction determined by the mtrices R i (s, ). After plying the mtrix gme nd receiving their pyoffs the plyers re trnsitioned to nother stte (or mtrix gme) determined by their joint ction. We cn see tht stochstic gmes then contin both MDPs nd mtrix gmes s subsets of the frmework. Stochstic Policies. Unlike in single-gent settings, deterministic policies in multigent settings cn often be exploited by the other gents. Consider the children s gme rock-pperscissors. If one plyer were to ply ny ction deterministiclly, the other plyer could win every time by selecting the ction tht defets it. This fct requires the considertion of mixed strtegies nd stochstic policies. A stochstic policy, π : S P D(A i ), is function tht mps sttes to mixed strtegies, which re probbility distributions over the plyer s ctions. We lter show tht stochstic policies re lso useful for grdient-bsed lerning techniques. Nsh Equilibri. Even with the concept of stochstic policies there re still no optiml policies tht re independent of the other plyers policies. We cn, though, define notion of best-response. A policy is best-response to the other plyers policies if it is optiml given their policies. The mjor dvncement tht hs driven much of the development of mtrix gmes, gme theory, nd even stochstic gmes is the notion of best-response equilibrium, or Nsh equilibrium [Nsh, Jr., 1950]. A Nsh equilibrium is collection of strtegies for ech of the plyers such tht ech plyer s strtegy is best-response to the other plyers strtegies. So, no plyer cn do better by chnging strtegies given tht the other plyers lso don t chnge strtegies. Wht mkes the notion of equilibrium compelling is tht ll mtrix gmes nd stochstic gmes hve such n equilibrium, possibly hving multiple equilibri. Zero-sum, two-plyer gmes, like the dversril tsk explored in this pper, hve single Nsh equilibrium. 2 Lerning in Stochstic Gmes. Stochstic gmes hve been the focus of recent reserch in the re of reinforcement lerning. There re two different pproches being explored. The first is tht of lgorithms tht explicitly lern equilibri through experience, independent of the other plyers policy, e.g., [Littmn, 1994; Greenwld nd Hll, 2002]. These lgorithms itertively estimte vlue functions, nd use them to compute n equilibrium for the gme. A second pproch is tht of best-response lerners, e.g., [Clus nd Boutilier, 1998; Singh et l., 2000; Bowling nd Veloso, 2002]. These lerners explicitly optimize their rewrd with respect to the 2 There cn ctully be multiple equilibri, but they ll hve equl pyoffs nd re interchngeble. Figure 1: An dversril robot tsk. The top robot is trying to get inside the circle while the bottom robot is trying to stop it. The lines show the stte nd ction representtion, which is described in Section 4.3. other plyers possibly chnging policies. This pproch, too, hs strong connection to equilibri. If these lgorithms converge when plying ech other, then they must do so to n equilibrium [Bowling nd Veloso, 2002]. Neither of these pproches, though, hve been scled beyond gmes with few hundred sttes. Gmes with very lrge number of sttes, or gmes with continuous stte spces, mke stte enumertion intrctble. Since previous lgorithms in their stted form require the enumertion of sttes either for policies or vlue functions, this is mjor limittion. In this pper we exmine lerning in n dversril robot tsk, which cn be thought of s continuous stte stochstic gme. Specificlly, we build on the ide of best-response lerners using grdient techniques [Singh et l., 2000; Bowling nd Veloso, 2002]. We first describe our robot tsk nd then describe our lgorithm nd results. 3 An Adversril Robot Tsk Consider the dversril robot tsk shown in Figure 1. The robot t the top of the figure, the ttcker, is trying to rech the circle in the center of the field, while the robot closer to the circle, the defender, is trying to prevent this from hppening. If the ttcker reches the circle, it receives rewrd of one nd the defender receives rewrd of negtive one. These re the only rewrds in the tsk. When the ttcker reches the circle or ten seconds elpses, the tril is over nd the robots reset to their initil positions, where the ttcker is meter from the circle nd the defender hlf-wy between. The robots simultneously lern in this environment ech seeking to mximize its own discounted future rewrd. For ll of our experiments the discount fctor used ws 0.8 for ech full second of dely. The robots themselves re prt of the CMDrgons robot soccer tem, which competes in the RoboCup smll-size legue. There re two detils bout the robot pltform tht re relevnt to this lerning tsk. First, the lerning lgorithm itself is situted within lrge nd complex rchitecture of

3 existing cpbility. The tem employs globl vision system mounted over the field. This input is processed by n elborte trcking module tht provides ccurte positions nd velocities of the robots. These positions comprise the input stte for the lerning lgorithm. The tem lso consists of robust modules for obstcle voidnce nd motion control. The ctions for the lerning lgorithm then involve providing trget points for the obstcle voidnce module. Situting the lerning within the context of this lrger rchitecture focuses the lerning. Rther thn hving the robot lern to solve well understood problems like pth plnning or object trcking, the lerning is directed t the hert of the problem, the multirobot interction. Second, the system control loop tht is prtilly described bove hs inherent, though smll, ltency. Specificlly, fter n observed chnge in the world 100ms will elpse before the robot s response is executed. This ltency is overcome for ech robot s own position nd velocity by predicting through this ltency using knowledge of the pst, but not yet executed, ctions. Since the ctions of the opponent robots re not known, this prediction is not possible for other robots. Ltency effectively dds n element of prtil observbility to the problem, since the gents do not hve the complete stte of the world, nd in fct hve seprte views of this stte. Notice, tht this lso dds tcticl element to successful policies. A robot cn fke the opponent robot by chnging its direction suddenly, knowing the other robot will not be ble to respond to this chnge for full ltency period. This tsk involves numerous chllenges for existing multigent lerning techniques. These chllenges include continuous stte nd ction spces, elements of prtil observbility due to system ltency, nd violtion of the Mrkov ssumption since mny of the system components hve memory, e.g., the trcking nd the obstcle voidnce modules. In fct, these limittions my even mke equilibri cese to exist ltogether [Bowling nd Veloso, 2002b]. This is further reson for exploring best-response lerning techniques, which don t directly seek to lern n equilibrium. We now present Gr- WoLF, best-response lerning lgorithm tht cn hndle the chllenges of multi-robot lerning. 4 GrWoLF GrWoLF, or Grdient-bsed WoLF, combines two key ides from reinforcement lerning: policy grdient lerning nd the WoLF vrible lerning rte. Policy grdient lerning is technique to hndle intrctble or continuous stte spces. WoLF is multigent lerning technique tht encourges best-response lerning lgorithms to converge in situtions of simultneous lerning. We briefly describe these techniques nd how they re combined. 4.1 Policy Grdient Ascent We use the policy grdient technique presented by Sutton nd collegues [2000]. Specificlly, we define policy s Gibbs distribution over liner combintion of fetures of cndidte stte nd ction pir. Let θ be vector of the policy s prmeters nd let φ s be n identiclly sized feture vector for stte s nd ction, then the Gibbs distribution defines stochstic policy ccording to, eθ φs π θ (s, ) = b eθ φ. sb Sutton nd collegues min result ws convergence proof for the following policy itertion rule tht updtes policy s prmeters in Gibbs distribution ccording to, θ k+1 = θ k + δ k d π k (s) φ s π θk (s, )f wk (s, ) (1) s f wk (s, ) is n independent pproximtion of Q π θ k (s, ) with prmeters w, which is the expected vlue of tking ction from stte s nd then following the policy π θk. For the Gibbs distribution, Sutton nd collegues showed tht for convergence this pproximtion should hve the following form, [ f wk (s, ) = w k φ s ] π θk (s, b)φ sb. (2) b As they point out, this mounts to f w being n pproximtion of the dvntge function, A π (s, ) = Q π (s, ) V π (s), where V π (s) is the vlue of following policy π from stte s. It is this dvntge function tht we estimte nd use for grdient scent. Using this bsic formultion we derive n on-line version of the lerning rule, where the policy s weights re updted with ech stte visited. The summtion over sttes cn be removed by updting proportiontely to tht stte s contribution to the policy s overll vlue. Since we re visiting sttes onpolicy then we only need to weight lter sttes by the discount fctor to ccount for their smller contribution. If t time hs pssed since the tril strt, this turns Eqution 1 into, θ k+1 = θ k + γ t δ k φ s π θk (s, )f wk (s, ). (3) Since the whole lgorithm is on-line, we do the policy improvement step (updting θ) simultneously with the vlue estimtion step (updting w). We do vlue estimtion using grdient-descent Srs(λ) [Sutton nd Brto, 1998] over the sme feture spce s the policy. This requires mintining n eligibility trce vector, e. The updte rule is then, if t time k the system is in stte s nd tkes ction trnsitioning to stte s in time t nd then tking ction, we updte the trce nd the weight vector using, e k+1 = λγ t e k + φ s, (4) ( ) r + γ w k+1 = w k + e k+1 α t Q wk (s, ) k, (5) Q wk (s, ) where λ is the Srs prmeter nd α k is n ppropritely decyed lerning rte. The ddition of rising γ to the power t llows for ctions to tke differing mounts of time to execute, s in semi-mrkov decision processes [Sutton nd Brto, 1998]. The policy improvement step then uses Eqution 3 where s is the stte of the system t time k nd the ction-vlue estimtes from Srs, Q wk, re used to compute the dvntge term. We then get, f wk (s, ) = Q wk (s, ) π θk (s, )Q wk (s, ). (6)

4 This forms the crux of GrWoLF. Wht remins is the selection of the lerning rte, δ k. This is where the WoLF vrible lerning rte is used. 4.2 Win or Lern Fst WoLF ( Win or Lern Fst ) is method by Bowling nd Veloso [2002] for chnging the lerning rte to encourge convergence in multigent reinforcement lerning scenrio. Notice tht the policy grdient scent lgorithm bove does not ccount for non-sttionry environment tht rises with simultneous lerning in stochstic gmes. All of the other gents ctions re simply ssumed to be prt of the environment nd unchnging. WoLF provides simple wy to ccount for other gents through djusting how quickly or slowly the gent chnges its policy. Since only the rte of lerning is chnged, lgorithms tht re gurnteed to find (loclly) optiml policies in nonsttionry environments retin this property even when using WoLF. In stochstic gmes with simultneous lerning, WoLF hs both theoreticl evidence (limited to two-plyer, two-ction mtrix gmes), nd empiricl evidence (experiments in gmes with enumerted stte spces) tht it encourges convergence in lgorithms tht don t otherwise converge. The intuition for this technique is tht lerner should dpt quickly when it is doing more poorly thn expected. When it is doing better thn expected, it should be cutious, since the other plyers re likely to chnge their policy. This implicitly ccounts for other plyers tht re lerning, rther thn other techniques tht try to explicitly reson bout the others ction choices. The WoLF principle nturlly lends itself to policy grdient techniques where there is well-defined lerning rte, α k. With WoLF we replce the originl lerning rte with two lerning rtes αk w < αl k to be used when winning or losing, respectively. One determintion of winning nd losing tht hs been successful is to compre the vlue of the current policy V π (s) to the vlue of the verge policy over time V π (s). So, if V π (s) > V π (s) then the lgorithm is considered winning, otherwise it is losing. With the policy grdient technique bove, we cnnot esily compute the verge policy. Insted we exmine the pproximte vlue, using Q w, of the current weight vector θ with the verge weight vector over time θ. Specificlly, we re winning if nd only if, π θk (s, )Q wk (s, ) > π θk (s, )Q wk (s, ). (7) When winning in prticulr stte, we updte the prmeters for tht stte using α w k, otherwise αl k. 4.3 Our Tsk Returning to the robot tsk shown in Figure 1, in order to pply GrWoLF we need to select policy prmeteriztion. Specificlly we need to find mpping from the continuous spce of sttes nd ctions to useful feture vector, i.e., to define φ s for given s nd. The filtered positions nd velocities of the two robots form the vilble stte informtion, nd the vilble ctions re points on the field for nvigtion. By observing tht the rdil ngle of the ttcker with respect to the circle is not relevnt to the tsk we rrive t seven dimensionl input spce. These seven dimensions re shown by the white overlid lines in Figure 1. We chose to use tile coding [Sutton nd Brto, 1998], lso known s CMACS, to construct our feture vector. Tile coding uses number of lrge offset tilings to llow for both fine discretiztion nd lrge mount of generliztion. We use 16 tiles whose size ws 800mm in distnce dimensions nd 1600mm/s in velocity dimensions. We simplify the ction spce by requiring the ttcker to select its nvigtion point long perpendiculr line through the circle s center. This is shown by the drk overlid line in Figure 1. This line, whose length is three times the distnce of the ttcker to the circle, is then discretized into seven points evenly distributed. The defender uses the sme points for its ctions but then nvigtes to the point tht bisects the line from the ttcker to the selected ction point. The robot s ction is lso included in the tiling s n eighth dimension with tile size for tht dimension of the entire line, so ctions re distinguishble but there is lso generliztion. Agents select ctions every tenth of second, or fter every three frmes, unless the feture vector hs not chnged over tht time. This keeps the robots from oscillting too much during the initil prts of lerning. 5 Results Before presenting results on pplying GrWoLF to this problem we first consider some issues relted to evlution. 5.1 Evlution Evlution of multi-robot lerning lgorithms present number of chllenges. The first is the problem of dt gthering on rel robot pltform. Lerning often requires fr more trils thn is prcticl to execute in the physicl world. We believe nd demonstrte tht the GrWoLF technique is prcticl for the time constrints of rel robots. Yet, from reserch stndpoint, we wnt thorough nd sttisticlly significnt evlution, which requires fr more trils thn just those used for lerning. We solve this problem by using both simultor of our robot tem s well s the robots themselves to show tht the pproch is both prcticl for robots while still providing n extensive nlysis of the results. The second chllenge is the problem of evluting the success of simultneous lerning. The trditionl single-gent evlution tht shows performnce over time converging to some optiml vlue is not pplicble. Multigent domins hve no single optiml vlue independent of the other gents behvior. The optiml vlue is chnging over time s the other gents lso lern. This is especilly true of self-ply experiments where both gents re using n identicl lerning lgorithm, nd ny performnce success by one gent is necessrily performnce filure for the other. On the other hnd we would still wnt lerning robots, even in self-ply, to improve their policy over time. In this pper, our min evlution compres the performnce of the lerned policy with the the performnce of the initil policy before lerning. The initil policy is rndom selection of the vilble ctions, nd by design of the vilble ctions is ctully firly cpble policy for both gents. We lso use the evlution technique of chllengers, which ws first exmined by

5 s s Attcker s Expected Rewrd Attcker s Expected Rewrd Defender Attcker 0.2 L v. R R v. R R v. L Figure 2: The vlue of lerned policies compred to rndom opponent in simultion. Lines to the right of the brs show stndrd devitions. 7.2s s Fst Slow WoLF WoLF Slow Fst Figure 3: The vlue of lerned policies plying their chllengers in simultion. Lines to the right of the brs show stndrd devitions. Littmn [1994]. This technique trins worst-cse opponent, or chllenger, for prticulr policy to see the generlity of the lerned policy. In this pper we present chllenger results demonstrting tht the lerned policies re indeed robust, nd tht the WoLF vrible lerning rte plys criticl role in keeping the lerning wy from esily exploited policies. 5.2 Experiments In ll of the performnce grphs in this section, the y- xis shows the ttcker s expected discounted rewrd, which roughly corresponds to the expected time it tkes for the ttcker to rech the circle. On the right of the grph the rnge is shown in seconds. All mesurements of expected discounted rewrd re gthered empiriclly over the course of 1000 trils. All trining occurred over 4000 trils, nd tkes pproximtely 6-7 hours of trining time on the robot pltform or in simultion. Unless otherwise noted, the trining ws lwys done with simultneously lerning opponent, both using GrWoLF. The experiments in simultion were repeted nine times nd the verges re shown in the grphs. We first exmine the performnce of the policies lerned in simultion, nd then exmine the performnce of lerning on the robot. Figure 2 shows the results of vrious lerned policies when plying ginst n opponent following the rndom policy, which ws the strting point for lerning. The middle br, R v. R, corresponds to the expected vlue of both plyers following the rndom policy. L v. R corresponds to the vlue of the ttcker following the policy lerned in self-ply ginst rndom defender. R v. L corresponds to the vlue of rndom ttcker ginst the defender policy lerned in self-ply. Notice tht, s ws desired, lerning does improve performnce over the strting policy. The lerned ttcker policy ginst rndom does fr better thn the rndom ttcker policy ginst the lerned defender. These experiments demonstrte tht GrWoLF improves considerbly on its strting policy. The next experiment explores how robust the lerned policy is nd the effect of the WoLF component. Figure 3 shows results of chllenger experiments. Policies re trined using simultneous lerning. The policy is then fixed nd chllenger policy is trined for 4000 trils, to specificlly mesure the policy s worst-cse performnce. The better the worst-cse performnce, the less exploitble nd more robust the policy is to unknown opponents. We trined chllenger ginst the policies lerned fter 3000, 3500, nd 4000 trils, verging together the results. We used this experiment to investigte the robustness of the lerned policy nd the ffect of the WoLF vrible lerning rte on the GrWoLF lgorithm. The left side of the grph shows the worst-cse performnce of lerned defender policies, while the right side shows the worst-cse performnce for ttcker policies. WoLF corresponds to the described GrWoLF lgorithm, Slow does not use vrible lerning rte but rther lwys uses the WoLF s winning rte, while Fst lwys uses its losing rte. First, notice tht the distnce between the worst-cse performnce of the defender nd the worst-cse performnce of the ttcker (the third nd fourth column of Tble 3, respectively) is quite smll. This demonstrtes tht the lerned policies re quite close to the equilibrium policy, if one exists. This lso mens tht the lerned policies re robust nd difficult to exploit. Second, notice tht the WoLF lerned defender policy performs better ginst its chllenger, i.e., keeps its chllenger to lower rewrd, thn either of the two lerning rtes it switches between. For the ttcker, the lerned policy performs better ginst its chllenger thn the fst lerning rte, but is not significntly different thn the slow lerning rte. There re couple of possible resons for this. One explntion is due to the initiliztion of vlues. Since ll vlues were intilized to zero for both sides, this mounts to n optimistic initiliztion for the defender, nd pessimistic one for the ttcker (s ll rewrds re non-negtive for the ttcker.) This my men the ttcker considers itself winning fr more often thn the defender, cusing the slower lerning rte to be employed most of the time. There is evidence to this effect when exmining the percentge of updtes tht use δ w versus δ l. During trining, the ttcker used the slower, winning rte for 92.3% of its updtes, while the defender used the winning rte for only 84.2%. The effect of vlue initiliztion on Gr- WoLF is n interesting top to be explored further. Overll, lthough the results re certinly not s drmtic s the unpproximted results [Bowling nd Veloso, 2002], WoLF still

6 Attcker s Expected Rewrd LL v. R R v. R R v. LL Figure 4: The vlue of lerned policies plying the rndom policy on the rel robots. Trining ws done with the first hlf of the trils in simultion nd the lst hlf on the robots. seems to be providing converging effect. Finlly, we exmine results of using GrWoLF on the rel robots. We took policies tht were trined for 2000 trils in simultion nd then did n dditionl 2000 trils of trining on the robots. We evluted the resulting policies ginst the rndom policy s we did with the simultor results in Figure 2. These results re shown in Figure 4 In the rel robots the ttcker is nerly impossible to keep from reching the circle. The defender cn t best only slow down its progress, nd this is even true for rndom ttcker policy. This cn be seen in Figure 4 by the much higher rewrds s compred to the simultor results. Despite this, these results re qulittively identicl to the results produced in simultion. Simultneous lerning improves the performnce for both the ttcker nd defender over their initil policies. Finlly, s n dversril environment we would expect good policies to be stochstic in nture. This is true of ll of the lerned policies in both simultion nd on the robots. For exmple, the most probble ction in ny given stte hs probbility on verge round 40% in the ttcker s lerned policy, nd 70% in the defender s. In some sttes, though, the lerned policies re nerly deterministic. So, the lerned policies re indeed stochstic to void exploittion, while still lerning optiml ctions in sttes tht don t depend on the other gent s behvior. 6 Conclusion We hve introduced, GrWoLF, generl-purpose multigent lerning lgorithm cpble of lerning robot tsks in multirobot, nd even dversril, environments. We showed results of this lgorithm lerning in one prticulr dversril robot tsk, both in simultion nd from ctul robot experience. These results both demonstrted the effectiveness of the policy grdient lerner, nd the importnce of the WoLF vrible lerning rte. It should be noted tht the use of Sutton nd collegues prticulr policy grdient formultion is not criticl. It would be interesting to combine WoLF with other policy grdient techniques, such s [Willims, 1992; Bxter nd Brtlett, 2000]. 2.7s 4.1s Acknowledgements This reserch ws sponsored by Grnts F nd DABT The content of this publiction does not necessrily reflect the position of the funding gencies nd no officil endorsement should be inferred. The uthors lso thnk Brett Browning nd Jmes Bruce for the development of the CMDrgons 02 robots used in this work. References [Bxter nd Brtlett, 2000] Johnthn Bxter nd Peter L. Brtlett. Reinforcement lerning in POMDP s vi direct grdient scent. In Proceedings of the Seventeenth Interntionl Conference on Mchine Lerning, pges 41 48, Stnford University, June Morgn Kufmn. [Bowling nd Veloso, 2002] Michel Bowling nd Mnuel Veloso. Multigent lerning using vrible lerning rte. Artificil Intelligence, 136: , [Bowling nd Veloso, 2002b] Michel Bowling nd Mnuel M. Veloso. Existence of multigent equilibri with limited gents. Technicl report CMU-CS , Computer Science Deprtment, Crnegie Mellon University, [Clus nd Boutilier, 1998] Croline Clus nd Crig Boutilier. The dynmics of reinforcement lerning in coopertive multigent systems. In Proceedings of the Fifteenth Ntionl Conference on Artificil Intelligence, Menlo Prk, CA, AAAI Press. [Greenwld nd Hll, 2002] Amy Greenwld nd Keith Hll. Correlted Q-lerning. In Proceedings of the AAAI Spring Symposium Workshop on Collbortive Lerning Agents, [Littmn, 1994] Michel L. Littmn. Mrkov gmes s frmework for multi-gent reinforcement lerning. In Proceedings of the Eleventh Interntionl Conference on Mchine Lerning, pges Morgn Kufmn, [Nsh, Jr., 1950] John F. Nsh, Jr. Equilibrium points in n- person gmes. PNAS, 36:48 49, [Singh et l., 2000] Stinder Singh, Michel Kerns, nd Yishy Mnsour. Nsh convergence of grdient dynmics in generl-sum gmes. In Proceedings of the Sixteenth Conference on Uncertinty in Artificil Intelligence, pges Morgn Kufmn, [Sutton nd Brto, 1998] Richrd S. Sutton nd Andrew G. Brto. Reinforcement Lerning. MIT Press, [Sutton et l., 2000] Richrd S. Sutton, Dvid McAllester, Stinder Singh, nd Yishy Mnsour. Policy grdient methods for reinforcement lerning with function pproximtion. In Advnces in Neurl Informtion Processing Systems 12, pges MIT Press, [Willims, 1992] R. Willims. Simple sttisticl grdient following lgorithms for connectionisht reinforcement lerning. Mchine Lerning, 8: , 1992.

MAXIMUM FLOWS IN FUZZY NETWORKS WITH FUNNEL-SHAPED NODES

MAXIMUM FLOWS IN FUZZY NETWORKS WITH FUNNEL-SHAPED NODES MAXIMUM FLOWS IN FUZZY NETWORKS WITH FUNNEL-SHAPED NODES Romn V. Tyshchuk Informtion Systems Deprtment, AMI corportion, Donetsk, Ukrine E-mil: rt_science@hotmil.com 1 INTRODUCTION During the considertion

More information

Example. Check that the Jacobian of the transformation to spherical coordinates is

Example. Check that the Jacobian of the transformation to spherical coordinates is lss, given on Feb 3, 2, for Mth 3, Winter 2 Recll tht the fctor which ppers in chnge of vrible formul when integrting is the Jcobin, which is the determinnt of mtrix of first order prtil derivtives. Exmple.

More information

CHAPTER 2 LITERATURE STUDY

CHAPTER 2 LITERATURE STUDY CHAPTER LITERATURE STUDY. Introduction Multipliction involves two bsic opertions: the genertion of the prtil products nd their ccumultion. Therefore, there re two possible wys to speed up the multipliction:

More information

METHOD OF LOCATION USING SIGNALS OF UNKNOWN ORIGIN. Inventor: Brian L. Baskin

METHOD OF LOCATION USING SIGNALS OF UNKNOWN ORIGIN. Inventor: Brian L. Baskin METHOD OF LOCATION USING SIGNALS OF UNKNOWN ORIGIN Inventor: Brin L. Bskin 1 ABSTRACT The present invention encompsses method of loction comprising: using plurlity of signl trnsceivers to receive one or

More information

Energy Harvesting Two-Way Channels With Decoding and Processing Costs

Energy Harvesting Two-Way Channels With Decoding and Processing Costs IEEE TRANSACTIONS ON GREEN COMMUNICATIONS AND NETWORKING, VOL., NO., MARCH 07 3 Energy Hrvesting Two-Wy Chnnels With Decoding nd Processing Costs Ahmed Arf, Student Member, IEEE, Abdulrhmn Bknin, Student

More information

Algorithms for Memory Hierarchies Lecture 14

Algorithms for Memory Hierarchies Lecture 14 Algorithms for emory Hierrchies Lecture 4 Lecturer: Nodri Sitchinv Scribe: ichel Hmnn Prllelism nd Cche Obliviousness The combintion of prllelism nd cche obliviousness is n ongoing topic of reserch, in

More information

Experiment 3: Non-Ideal Operational Amplifiers

Experiment 3: Non-Ideal Operational Amplifiers Experiment 3: Non-Idel Opertionl Amplifiers Fll 2009 Equivlent Circuits The bsic ssumptions for n idel opertionl mplifier re n infinite differentil gin ( d ), n infinite input resistnce (R i ), zero output

More information

Experiment 3: Non-Ideal Operational Amplifiers

Experiment 3: Non-Ideal Operational Amplifiers Experiment 3: Non-Idel Opertionl Amplifiers 9/11/06 Equivlent Circuits The bsic ssumptions for n idel opertionl mplifier re n infinite differentil gin ( d ), n infinite input resistnce (R i ), zero output

More information

The Discussion of this exercise covers the following points:

The Discussion of this exercise covers the following points: Exercise 4 Bttery Chrging Methods EXERCISE OBJECTIVE When you hve completed this exercise, you will be fmilir with the different chrging methods nd chrge-control techniques commonly used when chrging Ni-MI

More information

Section 16.3 Double Integrals over General Regions

Section 16.3 Double Integrals over General Regions Section 6.3 Double Integrls over Generl egions Not ever region is rectngle In the lst two sections we considered the problem of integrting function of two vribles over rectngle. This sitution however is

More information

Y9.ET1.3 Implementation of Secure Energy Management against Cyber/physical Attacks for FREEDM System

Y9.ET1.3 Implementation of Secure Energy Management against Cyber/physical Attacks for FREEDM System Y9.ET1.3 Implementtion of Secure Energy ngement ginst Cyber/physicl Attcks for FREED System Project Leder: Fculty: Students: Dr. Bruce cillin Dr. o-yuen Chow Jie Dun 1. Project Gols Develop resilient cyber-physicl

More information

The Math Learning Center PO Box 12929, Salem, Oregon Math Learning Center

The Math Learning Center PO Box 12929, Salem, Oregon Math Learning Center Resource Overview Quntile Mesure: Skill or Concept: 300Q Model the concept of ddition for sums to 10. (QT N 36) Model the concept of sutrction using numers less thn or equl to 10. (QT N 37) Write ddition

More information

Synchronous Machine Parameter Measurement

Synchronous Machine Parameter Measurement Synchronous Mchine Prmeter Mesurement 1 Synchronous Mchine Prmeter Mesurement Introduction Wound field synchronous mchines re mostly used for power genertion but lso re well suited for motor pplictions

More information

Lecture 20. Intro to line integrals. Dan Nichols MATH 233, Spring 2018 University of Massachusetts.

Lecture 20. Intro to line integrals. Dan Nichols MATH 233, Spring 2018 University of Massachusetts. Lecture 2 Intro to line integrls Dn Nichols nichols@mth.umss.edu MATH 233, Spring 218 University of Msschusetts April 12, 218 (2) onservtive vector fields We wnt to determine if F P (x, y), Q(x, y) is

More information

Engineer-to-Engineer Note

Engineer-to-Engineer Note Engineer-to-Engineer Note EE-297 Technicl notes on using Anlog Devices DSPs, processors nd development tools Visit our Web resources http://www.nlog.com/ee-notes nd http://www.nlog.com/processors or e-mil

More information

Module 9. DC Machines. Version 2 EE IIT, Kharagpur

Module 9. DC Machines. Version 2 EE IIT, Kharagpur Module 9 DC Mchines Version EE IIT, Khrgpur esson 40 osses, Efficiency nd Testing of D.C. Mchines Version EE IIT, Khrgpur Contents 40 osses, efficiency nd testing of D.C. mchines (esson-40) 4 40.1 Gols

More information

Outcome Matrix based Phrase Selection

Outcome Matrix based Phrase Selection Outcome Mtrix bsed Phrse Selection Aln R Wgner Georgi Tech Reserch Institute 50 4 th Street NW, Atlnt GA 0-08 Abstrct This rticle presents method for using outcome mtrices for socil phrse selection. An

More information

University of North Carolina-Charlotte Department of Electrical and Computer Engineering ECGR 4143/5195 Electrical Machinery Fall 2009

University of North Carolina-Charlotte Department of Electrical and Computer Engineering ECGR 4143/5195 Electrical Machinery Fall 2009 Problem 1: Using DC Mchine University o North Crolin-Chrlotte Deprtment o Electricl nd Computer Engineering ECGR 4143/5195 Electricl Mchinery Fll 2009 Problem Set 4 Due: Thursdy October 8 Suggested Reding:

More information

First Round Solutions Grades 4, 5, and 6

First Round Solutions Grades 4, 5, and 6 First Round Solutions Grdes 4, 5, nd 1) There re four bsic rectngles not mde up of smller ones There re three more rectngles mde up of two smller ones ech, two rectngles mde up of three smller ones ech,

More information

Intention reconsideration in theory and practice

Intention reconsideration in theory and practice Intention reconsidertion in theory nd prctice Simon Prsons nd Ol Pettersson nd lessndro Sffiotti nd Michel Wooldridge bstrct. utonomous gents operting in complex dynmic environments need the bility to

More information

Exercise 1-1. The Sine Wave EXERCISE OBJECTIVE DISCUSSION OUTLINE. Relationship between a rotating phasor and a sine wave DISCUSSION

Exercise 1-1. The Sine Wave EXERCISE OBJECTIVE DISCUSSION OUTLINE. Relationship between a rotating phasor and a sine wave DISCUSSION Exercise 1-1 The Sine Wve EXERCISE OBJECTIVE When you hve completed this exercise, you will be fmilir with the notion of sine wve nd how it cn be expressed s phsor rotting round the center of circle. You

More information

Geometric quantities for polar curves

Geometric quantities for polar curves Roerto s Notes on Integrl Clculus Chpter 5: Bsic pplictions of integrtion Section 10 Geometric quntities for polr curves Wht you need to know lredy: How to use integrls to compute res nd lengths of regions

More information

A Slot-Asynchronous MAC Protocol Design for Blind Rendezvous in Cognitive Radio Networks

A Slot-Asynchronous MAC Protocol Design for Blind Rendezvous in Cognitive Radio Networks Globecom 04 - Wireless Networking Symposium A Slot-Asynchronous MAC Protocol Design for Blind Rendezvous in Cognitive Rdio Networks Xingy Liu nd Jing Xie Deprtment of Electricl nd Computer Engineering

More information

A Development of Earthing-Resistance-Estimation Instrument

A Development of Earthing-Resistance-Estimation Instrument A Development of Erthing-Resistnce-Estimtion Instrument HITOSHI KIJIMA Abstrct: - Whenever erth construction work is done, the implnted number nd depth of electrodes hve to be estimted in order to obtin

More information

Study on SLT calibration method of 2-port waveguide DUT

Study on SLT calibration method of 2-port waveguide DUT Interntionl Conference on Advnced Electronic cience nd Technology (AET 206) tudy on LT clibrtion method of 2-port wveguide DUT Wenqing Luo, Anyong Hu, Ki Liu nd Xi Chen chool of Electronics nd Informtion

More information

Domination and Independence on Square Chessboard

Domination and Independence on Square Chessboard Engineering nd Technology Journl Vol. 5, Prt, No. 1, 017 A.A. Omrn Deprtment of Mthemtics, College of Eduction for Pure Science, University of bylon, bylon, Irq pure.hmed.omrn@uobby lon.edu.iq Domintion

More information

Interference Cancellation Method without Feedback Amount for Three Users Interference Channel

Interference Cancellation Method without Feedback Amount for Three Users Interference Channel Open Access Librry Journl 07, Volume, e57 ISSN Online: -97 ISSN Print: -9705 Interference Cncelltion Method without Feedbc Amount for Three Users Interference Chnnel Xini Tin, otin Zhng, Wenie Ji School

More information

Adaptive Network Coding for Wireless Access Networks

Adaptive Network Coding for Wireless Access Networks Adptive Network Coding for Wireless Access Networks Tun Trn School of Electricl Engineering nd Computer Science Oregon Stte University, Corvllis, Oregon 9733 Emil: trntu@eecs.orst.edu Thinh Nguyen School

More information

Make Your Math Super Powered

Make Your Math Super Powered Mke Your Mth Super Powered: Use Gmes, Chllenges, nd Puzzles Where s the fun? Lern Mth Workshop model by prticipting in one nd explore fun nocost/low-cost gmes nd puzzles tht you cn esily bring into your

More information

10.4 AREAS AND LENGTHS IN POLAR COORDINATES

10.4 AREAS AND LENGTHS IN POLAR COORDINATES 65 CHAPTER PARAMETRIC EQUATINS AND PLAR CRDINATES.4 AREAS AND LENGTHS IN PLAR CRDINATES In this section we develop the formul for the re of region whose oundry is given y polr eqution. We need to use the

More information

Synchronous Machine Parameter Measurement

Synchronous Machine Parameter Measurement Synchronous Mchine Prmeter Mesurement 1 Synchronous Mchine Prmeter Mesurement Introduction Wound field synchronous mchines re mostly used for power genertion but lso re well suited for motor pplictions

More information

Polar Coordinates. July 30, 2014

Polar Coordinates. July 30, 2014 Polr Coordintes July 3, 4 Sometimes it is more helpful to look t point in the xy-plne not in terms of how fr it is horizontlly nd verticlly (this would men looking t the Crtesin, or rectngulr, coordintes

More information

Dataflow Language Model. DataFlow Models. Applications of Dataflow. Dataflow Languages. Kahn process networks. A Kahn Process (1)

Dataflow Language Model. DataFlow Models. Applications of Dataflow. Dataflow Languages. Kahn process networks. A Kahn Process (1) The slides contin revisited mterils from: Peter Mrwedel, TU Dortmund Lothr Thiele, ETH Zurich Frnk Vhid, University of liforni, Riverside Dtflow Lnguge Model Drsticlly different wy of looking t computtion:

More information

Postprint. This is the accepted version of a paper presented at IEEE PES General Meeting.

Postprint.   This is the accepted version of a paper presented at IEEE PES General Meeting. http://www.div-portl.org Postprint This is the ccepted version of pper presented t IEEE PES Generl Meeting. Cittion for the originl published pper: Mhmood, F., Hooshyr, H., Vnfretti, L. (217) Sensitivity

More information

Foot-Pedal: Haptic Feedback Human Interface Bridging Sensational Gap between Remote Places

Foot-Pedal: Haptic Feedback Human Interface Bridging Sensational Gap between Remote Places Foot-Pedl: Hptic Feedbck Humn Interfce Bridging Senstionl Gp between Remote Plces Mincheol Kim 1, De-Keun Yoon 2, Shin-Young Kim 1, Ji-Hi Cho 1, Kwng-Kyu Lee 1, Bum-Je You 1,3 1 Center of Humn-centered

More information

9.4. ; 65. A family of curves has polar equations. ; 66. The astronomer Giovanni Cassini ( ) studied the family of curves with polar equations

9.4. ; 65. A family of curves has polar equations. ; 66. The astronomer Giovanni Cassini ( ) studied the family of curves with polar equations 54 CHAPTER 9 PARAMETRIC EQUATINS AND PLAR CRDINATES 49. r, 5. r sin 3, 5 54 Find the points on the given curve where the tngent line is horizontl or verticl. 5. r 3 cos 5. r e 53. r cos 54. r sin 55. Show

More information

Design-weighted Regression Adjusted Plus-Minus

Design-weighted Regression Adjusted Plus-Minus Design-weighted Regression Adjusted Plus-Minus Schuckers, Im, Mcdonld, McNulty August 3, 208 Schuckers, Im, Mcdonld, McNulty CASSIS-Schuckers August 3, 208 / 26 A etter title Design-weighted Regression

More information

PB-735 HD DP. Industrial Line. Automatic punch and bind machine for books and calendars

PB-735 HD DP. Industrial Line. Automatic punch and bind machine for books and calendars PB-735 HD DP Automtic punch nd bind mchine for books nd clendrs A further step for the utomtion of double loop binding. A clever nd flexible mchine ble to punch nd bind in line up to 9/16. Using the best

More information

Automatic Heuristic Construction in a Complete General Game Player

Automatic Heuristic Construction in a Complete General Game Player Automtic Heuristic Construction in Complete Generl Gme Plyer Gregory Kuhlmnn, Kurt Dresner nd Peter Stone Deprtment of Computer Sciences, The University of Texs t Austin 1 University Sttion C0500, Austin,

More information

Spiral Tilings with C-curves

Spiral Tilings with C-curves Spirl Tilings with -curves Using ombintorics to Augment Trdition hris K. Plmer 19 North Albny Avenue hicgo, Illinois, 0 chris@shdowfolds.com www.shdowfolds.com Abstrct Spirl tilings used by rtisns through

More information

Math Circles Finite Automata Question Sheet 3 (Solutions)

Math Circles Finite Automata Question Sheet 3 (Solutions) Mth Circles Finite Automt Question Sheet 3 (Solutions) Nickols Rollick nrollick@uwterloo.c Novemer 2, 28 Note: These solutions my give you the nswers to ll the prolems, ut they usully won t tell you how

More information

Joanna Towler, Roading Engineer, Professional Services, NZTA National Office Dave Bates, Operations Manager, NZTA National Office

Joanna Towler, Roading Engineer, Professional Services, NZTA National Office Dave Bates, Operations Manager, NZTA National Office . TECHNICA MEMOANDM To Cc repred By Endorsed By NZTA Network Mngement Consultnts nd Contrctors NZTA egionl Opertions Mngers nd Are Mngers Dve Btes, Opertions Mnger, NZTA Ntionl Office Jonn Towler, oding

More information

Jamming-Resistant Collaborative Broadcast In Wireless Networks, Part II: Multihop Networks

Jamming-Resistant Collaborative Broadcast In Wireless Networks, Part II: Multihop Networks Jmming-Resistnt ollbortive Brodcst In Wireless Networks, Prt II: Multihop Networks Ling Xio Ximen University, hin 361005 Emil: lxio@xmu.edu.cn Huiyu Di N Stte University, Rleigh, N 27695 Emil: huiyu di@ncsu.edu

More information

Multi-beam antennas in a broadband wireless access system

Multi-beam antennas in a broadband wireless access system Multi-em ntenns in rodnd wireless ccess system Ulrik Engström, Mrtin Johnsson, nders Derneryd nd jörn Johnnisson ntenn Reserch Center Ericsson Reserch Ericsson SE-4 84 Mölndl Sweden E-mil: ulrik.engstrom@ericsson.com,

More information

CSI-SF: Estimating Wireless Channel State Using CSI Sampling & Fusion

CSI-SF: Estimating Wireless Channel State Using CSI Sampling & Fusion CSI-SF: Estimting Wireless Chnnel Stte Using CSI Smpling & Fusion Riccrdo Crepldi, Jeongkeun Lee, Rul Etkin, Sung-Ju Lee, Robin Krvets University of Illinois t Urbn-Chmpign Hewlett-Pckrd Lbortories Emil:{rcrepl,rhk}@illinoisedu,

More information

Sequential Logic (2) Synchronous vs Asynchronous Sequential Circuit. Clock Signal. Synchronous Sequential Circuits. FSM Overview 9/10/12

Sequential Logic (2) Synchronous vs Asynchronous Sequential Circuit. Clock Signal. Synchronous Sequential Circuits. FSM Overview 9/10/12 9//2 Sequentil (2) ENGG5 st Semester, 22 Dr. Hden So Deprtment of Electricl nd Electronic Engineering http://www.eee.hku.hk/~engg5 Snchronous vs Asnchronous Sequentil Circuit This Course snchronous Sequentil

More information

Synchronous Generator Line Synchronization

Synchronous Generator Line Synchronization Synchronous Genertor Line Synchroniztion 1 Synchronous Genertor Line Synchroniztion Introduction One issue in power genertion is synchronous genertor strting. Typiclly, synchronous genertor is connected

More information

Outline. A.I. Applications. Searching for the solution. Chess game. Deep Blue vs. Kasparov 20/03/2017

Outline. A.I. Applications. Searching for the solution. Chess game. Deep Blue vs. Kasparov 20/03/2017 Outline Giorgio Buttzzo E-mil: g.buttzzo@sssup.it Scuol Superiore Snt Ann retis.sssup.it/~giorgio/slides/neurl/inn.html Motivtions Neurl processing Lerning prdigms Associtive memories Pttern recognitions

More information

Redundancy Data Elimination Scheme Based on Stitching Technique in Image Senor Networks

Redundancy Data Elimination Scheme Based on Stitching Technique in Image Senor Networks Sensors & Trnsducers 204 by IFSA Publishing, S. L. http://www.sensorsportl.com Redundncy Dt Elimintion Scheme Bsed on Stitching Technique in Imge Senor Networks hunling Tng hongqing Technology nd Business

More information

Student Book SERIES. Fractions. Name

Student Book SERIES. Fractions. Name D Student Book Nme Series D Contents Topic Introducing frctions (pp. ) modelling frctions frctions of collection compring nd ordering frctions frction ingo pply Dte completed / / / / / / / / Topic Types

More information

CS 135: Computer Architecture I. Boolean Algebra. Basic Logic Gates

CS 135: Computer Architecture I. Boolean Algebra. Basic Logic Gates Bsic Logic Gtes : Computer Architecture I Boolen Algebr Instructor: Prof. Bhgi Nrhri Dept. of Computer Science Course URL: www.ses.gwu.edu/~bhgiweb/cs35/ Digitl Logic Circuits We sw how we cn build the

More information

Improving synchronized transfers in public transit networks using real-time tactics

Improving synchronized transfers in public transit networks using real-time tactics Improving synchronized trnsfers in public trnsit networks using rel-time tctics Zhongjun Wu 1,2,3, Grhm Currie 3, Wei Wng 1,2 1 Jingsu Key Lbortory of Urbn ITS, Si Pi Lou 2#, Nnjing, 210096, Chin 2 School

More information

B inary classification refers to the categorization of data

B inary classification refers to the categorization of data ROBUST MODULAR ARTMAP FOR MULTI-CLASS SHAPE RECOGNITION Chue Poh Tn, Chen Chnge Loy, Weng Kin Li, Chee Peng Lim Abstrct This pper presents Fuzzy ARTMAP (FAM) bsed modulr rchitecture for multi-clss pttern

More information

Area-Time Efficient Digit-Serial-Serial Two s Complement Multiplier

Area-Time Efficient Digit-Serial-Serial Two s Complement Multiplier Are-Time Efficient Digit-Seril-Seril Two s Complement Multiplier Essm Elsyed nd Htem M. El-Boghddi Computer Engineering Deprtment, Ciro University, Egypt Astrct - Multipliction is n importnt primitive

More information

Nevery electronic device, since all the semiconductor

Nevery electronic device, since all the semiconductor Proceedings of Interntionl Joint Conference on Neurl Networks, Orlndo, Florid, USA, August 12-17, 2007 A Self-tuning for Rel-time Voltge Regultion Weiming Li, Xio-Hu Yu Abstrct In this reserch, self-tuning

More information

EET 438a Automatic Control Systems Technology Laboratory 5 Control of a Separately Excited DC Machine

EET 438a Automatic Control Systems Technology Laboratory 5 Control of a Separately Excited DC Machine EE 438 Automtic Control Systems echnology bortory 5 Control of Seprtely Excited DC Mchine Objective: Apply proportionl controller to n electromechnicl system nd observe the effects tht feedbck control

More information

Section 17.2: Line Integrals. 1 Objectives. 2 Assignments. 3 Maple Commands. 1. Compute line integrals in IR 2 and IR Read Section 17.

Section 17.2: Line Integrals. 1 Objectives. 2 Assignments. 3 Maple Commands. 1. Compute line integrals in IR 2 and IR Read Section 17. Section 7.: Line Integrls Objectives. ompute line integrls in IR nd IR 3. Assignments. Red Section 7.. Problems:,5,9,,3,7,,4 3. hllenge: 6,3,37 4. Red Section 7.3 3 Mple ommnds Mple cn ctully evlute line

More information

Available online at ScienceDirect. Procedia Engineering 89 (2014 )

Available online at   ScienceDirect. Procedia Engineering 89 (2014 ) Aville online t www.sciencedirect.com ScienceDirect Procedi Engineering 89 (2014 ) 411 417 16th Conference on Wter Distriution System Anlysis, WDSA 2014 A New Indictor for Rel-Time Lek Detection in Wter

More information

Direct AC Generation from Solar Cell Arrays

Direct AC Generation from Solar Cell Arrays Missouri University of Science nd Technology Scholrs' Mine UMR-MEC Conference 1975 Direct AC Genertion from Solr Cell Arrys Fernndo L. Alvrdo Follow this nd dditionl works t: http://scholrsmine.mst.edu/umr-mec

More information

ABB STOTZ-KONTAKT. ABB i-bus EIB Current Module SM/S Intelligent Installation Systems. User Manual SM/S In = 16 A AC Un = 230 V AC

ABB STOTZ-KONTAKT. ABB i-bus EIB Current Module SM/S Intelligent Installation Systems. User Manual SM/S In = 16 A AC Un = 230 V AC User Mnul ntelligent nstlltion Systems A B 1 2 3 4 5 6 7 8 30 ma 30 ma n = AC Un = 230 V AC 30 ma 9 10 11 12 C ABB STOTZ-KONTAKT Appliction Softwre Current Vlue Threshold/1 Contents Pge 1 Device Chrcteristics...

More information

A Novel Back EMF Zero Crossing Detection of Brushless DC Motor Based on PWM

A Novel Back EMF Zero Crossing Detection of Brushless DC Motor Based on PWM A ovel Bck EMF Zero Crossing Detection of Brushless DC Motor Bsed on PWM Zhu Bo-peng Wei Hi-feng School of Electricl nd Informtion, Jingsu niversity of Science nd Technology, Zhenjing 1003 Chin) Abstrct:

More information

Autonomous Robotic Exploration Using Occupancy Grid Maps and Graph SLAM Based on Shannon and Rényi Entropy

Autonomous Robotic Exploration Using Occupancy Grid Maps and Graph SLAM Based on Shannon and Rényi Entropy Autonomous Robotic Explortion Using Occupncy Grid Mps nd Grph SLAM Bsed on Shnnon nd Rényi Entropy Henry Crrillo, Philip Dmes, Vijy Kumr, nd José A. Cstellnos Abstrct In this pper we exmine the problem

More information

April 9, 2000 DIS chapter 10 CHAPTER 3 : INTEGRATED PROCESSOR-LEVEL ARCHITECTURES FOR REAL-TIME DIGITAL SIGNAL PROCESSING

April 9, 2000 DIS chapter 10 CHAPTER 3 : INTEGRATED PROCESSOR-LEVEL ARCHITECTURES FOR REAL-TIME DIGITAL SIGNAL PROCESSING April 9, 2000 DIS chpter 0 CHAPTE 3 : INTEGATED POCESSO-LEVEL ACHITECTUES FO EAL-TIME DIGITAL SIGNAL POCESSING April 9, 2000 DIS chpter 3.. INTODUCTION The purpose of this chpter is twofold. Firstly, bsic

More information

DYE SOLUBILITY IN SUPERCRITICAL CARBON DIOXIDE FLUID

DYE SOLUBILITY IN SUPERCRITICAL CARBON DIOXIDE FLUID THERMAL SCIENCE, Yer 2015, Vol. 19, No. 4, pp. 1311-1315 1311 DYE SOLUBILITY IN SUPERCRITICAL CARBON DIOXIDE FLUID by Jun YAN, Li-Jiu ZHENG *, Bing DU, Yong-Fng QIAN, nd Fng YE Lioning Provincil Key Lbortory

More information

& Y Connected resistors, Light emitting diode.

& Y Connected resistors, Light emitting diode. & Y Connected resistors, Light emitting diode. Experiment # 02 Ojectives: To get some hndson experience with the physicl instruments. To investigte the equivlent resistors, nd Y connected resistors, nd

More information

Fuzzy Logic Controller for Three Phase PWM AC-DC Converter

Fuzzy Logic Controller for Three Phase PWM AC-DC Converter Journl of Electrotechnology, Electricl Engineering nd Mngement (2017) Vol. 1, Number 1 Clusius Scientific Press, Cnd Fuzzy Logic Controller for Three Phse PWM AC-DC Converter Min Muhmmd Kml1,, Husn Ali2,b

More information

Solutions to exercise 1 in ETS052 Computer Communication

Solutions to exercise 1 in ETS052 Computer Communication Solutions to exercise in TS52 Computer Communiction 23 Septemer, 23 If it occupies millisecond = 3 seconds, then second is occupied y 3 = 3 its = kps. kps If it occupies 2 microseconds = 2 6 seconds, then

More information

High Speed On-Chip Interconnects: Trade offs in Passive Termination

High Speed On-Chip Interconnects: Trade offs in Passive Termination High Speed On-Chip Interconnects: Trde offs in Pssive Termintion Rj Prihr University of Rochester, NY, USA prihr@ece.rochester.edu Abstrct In this pper, severl pssive termintion schemes for high speed

More information

Digital Design. Sequential Logic Design -- Controllers. Copyright 2007 Frank Vahid

Digital Design. Sequential Logic Design -- Controllers. Copyright 2007 Frank Vahid Digitl Design Sequentil Logic Design -- Controllers Slides to ccompny the tetook Digitl Design, First Edition, y, John Wiley nd Sons Pulishers, 27. http://www.ddvhid.com Copyright 27 Instructors of courses

More information

Genetic Representations for Evolutionary Minimization of Network Coding Resources

Genetic Representations for Evolutionary Minimization of Network Coding Resources Genetic Representtions for Evolutionry Minimiztion of Network Coding Resources Minkyu Kim 1, Vrun Aggrwl 2, Un-My O Reilly 2, Muriel Médrd 1, nd Wonsik Kim 1 1 Lortory for Informtion nd Decision Systems

More information

This is a repository copy of Effect of power state on absorption cross section of personal computer components.

This is a repository copy of Effect of power state on absorption cross section of personal computer components. This is repository copy of Effect of power stte on bsorption cross section of personl computer components. White Rose Reserch Online URL for this pper: http://eprints.whiterose.c.uk/10547/ Version: Accepted

More information

DESIGN OF CONTINUOUS LAG COMPENSATORS

DESIGN OF CONTINUOUS LAG COMPENSATORS DESIGN OF CONTINUOUS LAG COMPENSATORS J. Pulusová, L. Körösi, M. Dúbrvská Institute of Robotics nd Cybernetics, Slovk University of Technology, Fculty of Electricl Engineering nd Informtion Technology

More information

On the Prediction of EPON Traffic Using Polynomial Fitting in Optical Network Units

On the Prediction of EPON Traffic Using Polynomial Fitting in Optical Network Units On the Prediction of EP Trffic Using Polynomil Fitting in Opticl Networ Units I. Mmounis (1),(3), K. Yinnopoulos (2), G. Ppdimitriou (4), E. Vrvrigos (1),(3) (1) Computer Technology Institute nd Press

More information

arxiv: v1 [cs.ro] 13 Oct 2016

arxiv: v1 [cs.ro] 13 Oct 2016 Sim-to-Rel Robot Lerning from Pixels with Progressive Nets Andrei A. Rusu, Mtej Vecerik, Thoms Rothörl, Nicols Heess, Rzvn Pscnu, Ri Hdsell rxiv:161.4286v1 [cs.ro] 13 Oct 216 Google DeepMind London, UK

More information

arxiv: v1 [cs.cc] 29 Mar 2012

arxiv: v1 [cs.cc] 29 Mar 2012 Solving Mhjong Solitire ords with peeking Michiel de Bondt rxiv:1203.6559v1 [cs.cc] 29 Mr 2012 Decemer 22, 2013 Astrct We first prove tht solving Mhjong Solitire ords with peeking is NPcomplete, even if

More information

Kirchhoff s Rules. Kirchhoff s Laws. Kirchhoff s Rules. Kirchhoff s Laws. Practice. Understanding SPH4UW. Kirchhoff s Voltage Rule (KVR):

Kirchhoff s Rules. Kirchhoff s Laws. Kirchhoff s Rules. Kirchhoff s Laws. Practice. Understanding SPH4UW. Kirchhoff s Voltage Rule (KVR): SPH4UW Kirchhoff s ules Kirchhoff s oltge ule (K): Sum of voltge drops round loop is zero. Kirchhoff s Lws Kirchhoff s Current ule (KC): Current going in equls current coming out. Kirchhoff s ules etween

More information

High-speed Simulation of the GPRS Link Layer

High-speed Simulation of the GPRS Link Layer 989 High-speed Simultion of the GPRS Link Lyer J Gozlvez nd J Dunlop Deprtment of Electronic nd Electricl Engineering, University of Strthclyde 204 George St, Glsgow G-lXW, Scotlnd Tel: +44 4 548 206,

More information

Misty. Sudnow Dot Songs

Misty. Sudnow Dot Songs Sudnow Dot Songs isty T The Dot Song is nottionl system tht depicts voiced chords in wy where the non-music reder cn find these firly redily. But the Dot Song is not intended be red, not s sight reder

More information

(CATALYST GROUP) B"sic Electric"l Engineering

(CATALYST GROUP) Bsic Electricl Engineering (CATALYST GROUP) B"sic Electric"l Engineering 1. Kirchhoff s current l"w st"tes th"t (") net current flow "t the junction is positive (b) Hebr"ic sum of the currents meeting "t the junction is zero (c)

More information

RSS based Localization of Sensor Nodes by Learning Movement Model

RSS based Localization of Sensor Nodes by Learning Movement Model RSS bsed Locliztion of Sensor Nodes by Lerning Movement Model 1 R.ARTHI, 2 P.DEVARAJ, 1 K.MURUGAN 1 Rmnujn Computing Centre, Ann University, Guindy, Chenni, Indi 2 Deprtment of Mthemtics, College of Engineering,

More information

BP-P2P: Belief Propagation-Based Trust and Reputation Management for P2P Networks

BP-P2P: Belief Propagation-Based Trust and Reputation Management for P2P Networks BP-PP: Belief Propgtion-Bsed Trust nd Reputtion Mngement for PP Networs Ermn Aydy School of Electricl nd Comp. Eng. Georgi Institute of Technology Atlnt, GA 333, USA Emil: eydy@gtech.edu Frmrz Feri School

More information

S1 Only VEOG HEOG. S2 Only. S1 and S2. Computer. Subject. Computer

S1 Only VEOG HEOG. S2 Only. S1 and S2. Computer. Subject. Computer The Eects of Eye Trcking in VR Helmet on EEG Recordings Jessic D. Byliss nd Dn H. Bllrd The University of Rochester Computer Science Deprtment Rochester, New York 14627 Technicl Report 685 My 1998 Astrct

More information

Application of Wavelet De-noising in Vibration Torque Measurement

Application of Wavelet De-noising in Vibration Torque Measurement IJCSI Interntionl Journl of Computer Science Issues, Vol. 9, Issue 5, No 3, September 01 www.ijcsi.org 9 Appliction of Wvelet De-noising in Vibrtion orque Mesurement Ho Zho 1 1 Jixing University, Jixing,

More information

Network Sharing and its Energy Benefits: a Study of European Mobile Network Operators

Network Sharing and its Energy Benefits: a Study of European Mobile Network Operators Network Shring nd its Energy Benefits: Study of Europen Mobile Network Opertors Mrco Ajmone Mrsn Electronics nd Telecommunictions Dept Politecnico di Torino, nd Institute IMDEA Networks, mrco.jmone@polito.it

More information

A New Stochastic Inner Product Core Design for Digital FIR Filters

A New Stochastic Inner Product Core Design for Digital FIR Filters MATEC Web of Conferences, (7) DOI:./ mtecconf/7 CSCC 7 A New Stochstic Inner Product Core Design for Digitl FIR Filters Ming Ming Wong,, M. L. Dennis Wong, Cishen Zhng, nd Ismt Hijzin Fculty of Engineering,

More information

Robustness Analysis of Pulse Width Modulation Control of Motor Speed

Robustness Analysis of Pulse Width Modulation Control of Motor Speed Proceedings of the World Congress on Engineering nd Computer Science 2007 WCECS 2007, October 24-26, 2007, Sn Frncisco, USA obustness Anlysis of Pulse Width Modultion Control of Motor Speed Wei Zhn Abstrct

More information

Effect of High-speed Milling tool path strategies on the surface roughness of Stavax ESR mold insert machining

Effect of High-speed Milling tool path strategies on the surface roughness of Stavax ESR mold insert machining IOP Conference Series: Mterils Science nd Engineering PAPER OPEN ACCESS Effect of High-speed Milling tool pth strtegies on the surfce roughness of Stvx ESR mold insert mchining Relted content - Reserch

More information

Application Note. Differential Amplifier

Application Note. Differential Amplifier Appliction Note AN367 Differentil Amplifier Author: Dve n Ess Associted Project: Yes Associted Prt Fmily: CY8C9x66, CY8C7x43, CY8C4x3A PSoC Designer ersion: 4. SP3 Abstrct For mny sensing pplictions, desirble

More information

A New Algorithm to Compute Alternate Paths in Reliable OSPF (ROSPF)

A New Algorithm to Compute Alternate Paths in Reliable OSPF (ROSPF) A New Algorithm to Compute Alternte Pths in Relile OSPF (ROSPF) Jin Pu *, Eric Mnning, Gholmli C. Shoj, Annd Srinivsn ** PANDA Group, Computer Science Deprtment University of Victori Victori, BC, Cnd Astrct

More information

A Comparative Analysis of Algorithms for Determining the Peak Position of a Stripe to Sub-pixel Accuracy

A Comparative Analysis of Algorithms for Determining the Peak Position of a Stripe to Sub-pixel Accuracy A Comprtive Anlysis of Algorithms for Determining the Pek Position of Stripe to Sub-pixel Accurcy D.K.Nidu R.B.Fisher Deprtment of Artificil Intelligence, University of Edinburgh 5 Forrest Hill, Edinburgh

More information

EE Controls Lab #2: Implementing State-Transition Logic on a PLC

EE Controls Lab #2: Implementing State-Transition Logic on a PLC Objective: EE 44 - Controls Lb #2: Implementing Stte-rnsition Logic on PLC ssuming tht speed is not of essence, PLC's cn be used to implement stte trnsition logic. he dvntge of using PLC over using hrdwre

More information

AN IMPROVED METHOD FOR RADIO FREQUENCY DIRECTION FINDING USING WIRELESS SENSOR NETWORKS

AN IMPROVED METHOD FOR RADIO FREQUENCY DIRECTION FINDING USING WIRELESS SENSOR NETWORKS AN IMPROVED METHOD FOR RADIO FREQUENCY DIRECTION FINDING USING WIRELESS SENSOR NETWORKS Mickey S. Btson, John C. McEchen, nd Murli Tumml Deprtment of Electricl nd Computer Engineering Nvl Postgrdute School

More information

A Stochastic Geometry Approach to the Modeling of DSRC for Vehicular Safety Communication

A Stochastic Geometry Approach to the Modeling of DSRC for Vehicular Safety Communication A Stochstic Geometry Approch to the Modeling of DSRC for Vehiculr Sfety Communiction Zhen Tong, Student Member, IEEE, Hongsheng Lu 2, Mrtin Henggi, Fellow, IEEE, nd Christin Poellbuer 2, Senior Member,

More information

PRO LIGNO Vol. 11 N pp

PRO LIGNO Vol. 11 N pp THE INFLUENCE OF THE TOOL POINT ANGLE AND FEED RATE ON THE DELAMINATION AT DRILLING OF PRE-LAMINATED PARTICLEBOARD Mihi ISPAS Prof.dr.eng. Trnsilvni University of Brsov Fculty of Wood Engineering Address:

More information

Understanding Basic Analog Ideal Op Amps

Understanding Basic Analog Ideal Op Amps Appliction Report SLAA068A - April 2000 Understnding Bsic Anlog Idel Op Amps Ron Mncini Mixed Signl Products ABSTRACT This ppliction report develops the equtions for the idel opertionl mplifier (op mp).

More information

SOLVING TRIANGLES USING THE SINE AND COSINE RULES

SOLVING TRIANGLES USING THE SINE AND COSINE RULES Mthemtics Revision Guides - Solving Generl Tringles - Sine nd Cosine Rules Pge 1 of 17 M.K. HOME TUITION Mthemtics Revision Guides Level: GCSE Higher Tier SOLVING TRIANGLES USING THE SINE AND COSINE RULES

More information

Topic 20: Huffman Coding

Topic 20: Huffman Coding Topic 0: Huffmn Coding The uthor should gze t Noh, nd... lern, s they did in the Ark, to crowd gret del of mtter into very smll compss. Sydney Smith, dinburgh Review Agend ncoding Compression Huffmn Coding

More information

BP-P2P: Belief Propagation-Based Trust and Reputation Management for P2P Networks

BP-P2P: Belief Propagation-Based Trust and Reputation Management for P2P Networks 1 9th Annul IEEE Communictions Society Conference on Sensor, Mesh nd Ad Hoc Communictions nd Networs (SECON) BP-PP: Belief Propgtion-Bsed Trust nd Reputtion Mngement for PP Networs Ermn Aydy School of

More information

Color gamut reduction techniques for printing with custom inks

Color gamut reduction techniques for printing with custom inks Color gmut reduction techniques for printing with custom inks Sylvin M. CHOSSON *, Roger D. HERSCH * Ecole Polytechnique Fédérle de usnne (EPF) STRCT Printing with custom inks is of interest oth for rtistic

More information

Theme: Don t get mad. Learn mod.

Theme: Don t get mad. Learn mod. FERURY When 1 is divided by 5, the reminder is. nother wy to sy this is opyright 015 The Ntionl ouncil of Techers of Mthemtics, Inc. www.nctm.org. ll rights reserved. This mteril my not be copied or distributed

More information