Robot Deception: Recognizing when a Robot Should Deceive

Transcription

1 Robot ecepton: Recognzng when Robot Should eceve ln R. Wgner, Student ember IEEE nd Ronld C. rkn, Fellow IEEE bstrct Ths rtcle explores the possblty of developng robot control softwre cpble of dscernng when nd f robot should deceve. Explorton of ths problem s crtcl for developng robots wth decepton cpbltes nd my lend vluble nsght nto the phenomen of decepton tself. In ths pper we explore decepton from n nterdependence/gme theoretc perspectve. Further, we develop nd expermentlly nvestgte n lgorthm cpble of ndctng whether or not prtculr socl stuton wrrnts decepton on the prt of the robot. Our qulttve nd quntttve results provde evdence tht, ndeed, our lgorthm recognzes stutons whch justfy decepton nd tht robot cpble of dscernng these stutons s better suted to ct thn one tht does not. I. INTROUCTION ecepton hs long nd deep hstory wth respect to the study of ntellgent systems. Bologsts nd psychologsts rgue tht decepton s ubqutous wthn the nml kngdom nd represents n evolutonry dvntge for the decever []. Prmtologsts note tht the use of decepton serves s n mportnt potentl ndctor of theory of mnd [] nd socl ntellgence [3]. Reserchers n these felds pont to numerous exmples of decepton by non-humn prmtes. From robotcst s perspectve, the use of decepton nd the development of strteges for resstng beng deceved re mportnt topcs of study especlly wth respect to the mltry domn []. But wht s decepton? ccleskey notes tht decepton s delberte cton or seres of ctons brought bout for specfc purpose [5]. Whley recognzes tht decepton often ncludes nformton provded wth the ntent of mnpultng some other ndvdul. Ettnger nd Jehel offer relted defnton ted to gme theory frmework []. They defne decepton s, the process by whch ctons re chosen to mnpulte belefs so s to tke dvntge of the erroneous nferences. Ths defnton hs cler tes to gme theory but does not relte to mny of the pssve, unntentonl exmples of decepton found n bology. We dopt defnton for decepton offered by Bond nd Robnson tht encompsses conscous nd unconscous, ntentonl nd unntentonl cts of decepton []. These uthors descrbe decepton smply s flse communcton tht tends to beneft the communctor. Ths work ws supported n prt by the Offce of Nvl Reserch under the Heterogeneous Unmnned Network Tems (HUNT) project under Grnt BS35. ln R. Wgner s doctorl cnddte t the Georg Insttute of Technology, tlnt, G 333 US (-9-93; eml: ln.wgner@gtech.edu). Ronld C. rkn, s Regent s Professor nd ssocte en for Reserch t the Georg Insttute of Technology, tlnt, G 333 US.(e-ml: rkn@gtech.edu). Ths pper nvestgtes the use of decepton by utonomous robots. We focus on the ctons, belefs nd communcton of the decever, not the deceved (lso known s the mrk). Specfclly, the purpose of ths reserch s to develop nd nvestgte n lgorthm tht recognzes socl stutons justfyng the use of decepton. Recognzng when robot or rtfcl gent should deceve s crtcl queston. Robots tht deceve too often my be judged s unrelble or mlefcent. Robots ncpble of decepton, on the other hnd, my lck survvl sklls n stutons nvolvng conflct. Consder the followng runnng exmple: vluble robotc sset opertes t mltry bse. The bse comes under ttck nd s n dnger of beng overrun. If the robot s found by the ttckers then they wll gn vluble nformton nd hrdwre. The robot must recognze tht stuton wrrntng the use of decepton exsts, then hde, nd select deceptve strtegy tht wll reduce the chnce tht t wll be found. Throughout ths rtcle we wll use ths runnng exmple to expln portons of the theoretcl underpnnngs of our pproch s well s develop experments bsed on the exmple. The remnder of ths pper begns by frst summrzng relevnt reserch. Next, we use gme theory nd nterdependence theory to reson bout the theoretcl underpnnngs of decepton nd to develop n lgorthm for the recognton of stutons justfyng the use of decepton by robot. Fnlly, we present experments whch nvestgte our lgorthm both qulttvely nd quntttvely. The rtcle concludes wth dscusson of these results ncludng drectons for future reserch. II. RELTE WORK Gme theory hs been extensvely used to explore the phenomen of decepton. s brnch of ppled mthemtcs, gme theory focuses on the forml consderton of strtegc nterctons, such s the exstence of equlbrums nd economc pplctons [7]. Sgnlng gmes, for exmple, explore decepton by llowng ech ndvdul to send sgnls reltng to ther underlyng type. Costly versus cost free sgnlng hs been used to determne the condtons tht foster honesty. Floreno et l. found tht deceptve communcton sgnls cn evolve when condtons conducve to these sgnls re present []. These reserchers used both smulton experments nd rel robots to explore the condtons necessry for the evoluton of communcton sgnls. They found tht coopertve communcton redly evolves when robot colones consst of genetclly smlr ndvduls. Yet when the robot colones were genetclly dssmlr nd evolutonry

2 selecton of ndvduls rther then colones ws performed, the robots evolved deceptve communcton sgnls, whch, for exmple, compelled them to sgnl tht they were ner food when they were not. Floreno et l. s work s nterestng becuse t demonstrtes the tes between bology, evoluton, nd sgnl communcton nd does so on robotc pltform. Ettnger nd Jehel hve recently developed theory for decepton bsed on gme theory []. Ther theory focuses on belef mnpulton s mens for decepton. In gme theory, n ndvdul s type, t T, reflects specfc chrcterstcs of the ndvdul nd s prvtely known by tht ndvdul. Gme theory then defnes belef s, ( t t ) p, reflectng ndvdul 's uncertnty bout ndvdul -'s type [7]. Ettnger nd Jehel demonstrte the gme theoretcl mportnce of modelng the mrk. Stll, ther defnton of decepton s the process by whch ctons re chosen to mnpulte belefs so s to tke dvntge of the erroneous nferences s strongly drected towrds gme theory nd ther own frmework. The queston thus remns, wht role does modelng of the mrk ply for more generl defntons of decepton such s those offered by Bond nd Robnson []. Independent versus ependent mtrces Indvdul Independent Socl Stuton Indvdul 9 9 Indvdul ependent Socl Stuton Indvdul Fg. n exmple of dependent stuton s depcted on the rght nd n exmple of n ndependent stuton s depcted on the left. In the dependent exmple the ctons of the second ndvdul hve lrge mpct on the outcomes receved by the frst ndvdul. In the exmple of n ndependent stuton, on the other hnd, the ctons of the second ndvdul hve no mpct on the frst ndvdul. ecepton cn lso be explored from socl psychologcl perspectve. Interdependence theory, type of socl exchnge theory, s psychologcl theory developed s mens for understndng nd nlyzng nterpersonl stutons nd ntercton [9]. The term nterdependence specfes the extent to whch one ndvdul of dyd nfluences the other. Interdependence theory s bsed on the clm tht people djust ther nterctve behvor n response to ther percepton of socl stuton s pttern of rewrds nd costs. Thus, ech choce of nterctve behvor by n ndvdul offers the possblty of specfc rewrds nd costs lso known s outcomes fter the ntercton. Interdependence theory nd gme theory represent socl stutons computtonlly s n outcome mtrx (Fg. ). n outcome mtrx represents socl stuton by expressng 9 9 the outcomes fforded to ech nterctng ndvdul wth respect to ech pr of potentl behvors chosen by the ndvduls. III. REPRESENTING INTERCTIONS The outcome mtrx s stndrd computtonl representton for ntercton [9]. It s composed of nformton bout the ndvduls nterctng, ncludng ther dentty, the nterctve ctons they re delbertng over, nd sclr outcome vlues representng the rewrd mnus the cost, or the outcomes, for ech ndvdul. Thus, n outcome mtrx explctly represents nformton tht s crtcl to ntercton. Typclly, the dentty of the nterctng ndvduls s lsted long the dmensons of the mtrx. Fg. depcts n ntercton nvolvng two ndvduls. In ths pper the term ndvdul s used to ndcte humn, socl robot, or n gent. We wll focus on ntercton nvolvng two ndvduls dydc ntercton. n outcome mtrx cn, however, represent ntercton nvolvng more thn two ndvduls. The rows nd columns of the mtrx consst of lst of ctons vlble to ech ndvdul durng the ntercton. Fnlly, sclr outcome s ssocted wth ech cton pr for ech ndvdul. Outcomes represent untless chnges n the robot, gent, or humn s utlty. Thus, for exmple, n outcome of zero reflects the fct tht no chnge n the ndvdul s utlty wll result from the mutul selecton of tht cton pr. Becuse outcome mtrces re computtonl representtons, t s possble to descrbe them formlly. ong so llows for powerful nd generl descrptons of ntercton. The notton presented here drws hevly from gme theory [7]. representton of ntercton conssts of ) fnte set N of nterctng ndvduls; ) for ech ndvdul N nonempty set of ctons; 3) the utlty obtned by ech ndvdul for ech combnton of ctons tht could hve been selected []. Let j be n rbtrry cton j from ndvdul s set of ctons. Let N (,, ) K denote combnton of ctons, one for ech j k ndvdul, nd let N (, ) R j k u denote ndvdul s utlty functon: u K, s the utlty receved by ndvdul f N the ndvduls choose the ctons (,, ) K. The term O j k s used to denote n outcome mtrx. The superscrpt - s used to express ndvdul 's prtner. Thus, for exmple, denotes the cton set of ndvdul nd denotes the cton set of ndvdul s nterctve prtner. s mentoned bove, n ndvdul s type, t T, s determned pror to ntercton, reflects specfc chrcterstcs of the ndvdul nd s prvtely known by tht ndvdul. belef, ( t t ) p, reflects ndvdul 's uncertnty bout ndvdul -'s type.

3 . Representng Socl Stutons The term ntercton descrbes dscrete event n whch two or more ndvduls select nterctve behvors s prt of socl stuton or socl envronment. Intercton hs been defned s nfluence verbl, physcl, or emotonl by one ndvdul on nother []. The term stuton hs severl defntons. The most propos for ths work s prtculr set of crcumstnces exstng n prtculr plce or t prtculr tme []. socl stuton, then, chrcterzes the envronmentl fctors, outsde of the ndvduls themselves, whch nfluence nterctve behvor. socl stuton s bstrct, descrbng the generl pttern of outcome vlues n n ntercton. n ntercton, on the other hnd, s concrete wth respect to the two or more ndvduls nd the socl ctons vlble to ech ndvdul. For exmple, the prsoner s dlemm descrbes prtculr type of socl stuton. s such, t cn, nd hs been, nstntted n numerous dfferent prtculr socl envronments rngng from bnk robberes to the trenches of World Wr I [3]. Interdependence theorsts stte tht ntercton s functon of the ndvduls nterctng nd of the socl stuton []. dependent stuton, for exmple, s socl stuton n whch ech prtner s outcome depends on the other prtner s cton (Fg. left). n ndependent stuton, on the other hnd, s socl stuton n whch ech prtner s outcome does not depend on the prtner s cton (Fg. rght). lthough socl stuton my not fford ntercton, ll nterctons occur wthn some socl stuton. Interdependence theory represents socl stutons nvolvng nterpersonl ntercton s outcome mtrces (see Fg. for grphcl depcton of the dfference). In prevous work, we presented stuton nlyss lgorthm tht clculted chrcterstcs of the socl stuton or ntercton (such s nterdependence) when presented wth n outcome mtrx by mppng the stuton to locton n the nterdependence spce [5]. The nterdependence spce s four dmensonl spce whch mps the locton of ll nterpersonl socl stutons []. mtrx s locton n nterdependence spce provdes mportnt nformton reltng to the ntercton. The nterdependence nd correspondence dmensons re of prtculr mportnce for recognzng f stuton wrrnts decepton. The nterdependence dmenson mesures the extent to whch ech ndvdul s outcomes re nfluenced by the other ndvdul s ctons n stuton. In low nterdependence stuton, for exmple, ech ndvdul s outcomes re reltvely ndependent of the other ndvdul s choce of nterctve behvor (left sde of Fg. for exmple). hgh nterdependence stuton, on the other hnd, s stuton n whch ech ndvdul s outcomes lrgely depend on the cton of the other ndvdul (rght sde of Fg. for exmple). Correspondence descrbes the extent to whch the outcomes of one ndvdul n stuton re consstent wth the outcomes of the other ndvdul. If outcomes correspond then ndvduls tend to select nterctve behvors resultng n mutully rewrdng outcomes, such s temmtes n gme. If outcomes conflct then ndvduls tend to select nterctve behvors resultng n mutully costly outcomes, such s opponents n gme. Our results showed tht by nlyzng the ntercton, the robot could better select nterctve ctons. B. Prtner odelng Severl reserchers hve explored how humns develop mentl models of robots (e.g. [7]). mentl model s term used to descrbe person s concept of how somethng n the world works []. We use the term prtner model m (denoted ) to descrbe robot s mentl model of ts nterctve humn prtner. We use the term self model (denoted m ) to descrbe the robot s mentl model of tself. gn, the superscrpt - s used to express ndvdul 's prtner [7]. In pror work, Wgner presented n nterct-nd-updte lgorthm for popultng outcome mtrces nd for cretng ncresngly ccurte models of the robot s nterctve prtner [9]. The nterct-nd-updte lgorthm constructed model of the robot s prtner consstng of three types of nformton: ) set of prtner fetures ( f,, f ) cton model, ; nd 3) utlty functon K ; ) n n u. We use the notton m. nd m. u to denote the cton model nd utlty functon wthn prtner model. Wgner used prtner fetures for prtner recognton. Prtner fetures llow the robot to recognze the prtner n subsequent nterctons. The prtner s cton model contned lst of ctons vlble to tht ndvdul. The prtner s utlty functon ncluded nformton bout the outcomes obtned by the prtner when the robot nd the prtner select pr of ctons. Wgner showed tht the lgorthm could produce ncresngly ccurte prtner models whch, n turn, resulted n ccurte outcome mtrces. The results were, however, lmted to sttc, not dynmc, models of the prtner. The self model lso contns n cton model nd utlty functon. The cton model contns lst of ctons vlble to the robot. Smlrly the robot s utlty functon ncludes nformton bout the robot s outcomes. IV. ECEPTIVE INTERCTION Ths pper specfclly explores deceptve ntercton. We nvestgte deceptve ntercton wth respect to two ndvduls the mrk nd the decever. It s mportnt to recognze tht the decever nd the mrk fce dfferent problems nd hve dfferent nformton. The mrk smply selects the cton tht t beleves wll mxmze ts own outcome, bsed on ll of the nformton tht t hs ccumulted. The decever, on the other hnd, cts n ccordnce wth Bond nd Robnson s defnton of decepton, provdng flse communcton for ts own beneft []. Wth respect to our runnng exmple, the mltry robot hdng from n enemy, the robot cts s the decever provdng flse nformton s to ts wherebouts. The mrk then s the enemy solder serchng for the robot. We wll ssume henceforth tht the decever provdes flse communcton through the performnce of some cton n

4 the envronment. The sectons tht follow begn by exmnng the phenomen of decepton, provde method for decdng how to deceve, nd fnlly exmne how to decde when to deceve.. The Phenomen of ecepton We cn use outcome mtrces to reson bout deceptve prctces. Fg. depcts socl stuton nvolvng decepton. The fgure depcts the ctons tht the mrk nd decever reson over both bstrctly n terms of generc ctons,,, nd concretely n terms of four defned ctons. The outcome mtrx on the left hnd sde s clled the true mtrx. The true mtrx represents the ctul outcome obtned by both the mrk nd the decever for gven cton pr. Wth respect to our runnng exmple, the true mtrx represents the dfferent outcome ptterns resultng when the robot nd enemy select hde nd serch ctons. key fcet of decepton s the fct tht the decever recognzes the true mtrx but the mrk does not. In the true mtrx shown n Fg., the decever cn reson tht only the selecton of by the mrk nd by the decever or of by the mrk nd by the decever wll result n the desred outcome. Let s ssume tht the decever hs decded to select, to hde n re. The decever s tsk then s to provde nformton or to ct n wy tht wll nfluence the mrk to select rther thn. To do ths, the decever must convnce the mrk tht ) the selecton of benefcl then t ctully s; ) the selecton of benefcl then s ctully s; or 3) both. rk True trx Serch re Serch re Hde n re True versus Induced trx ecever Hde n re rk Serch re Serch re Hde n re s less s more trx nduced by ecepton ecever Hde n re Fg. The outcome mtrces bove depct exmples relted to the explorton of decepton. The true mtrx represents the ctul outcomes relzble n stuton. The true mtrx s recognzed by the decever, whch n turn, provdes flse communcton n the hope of nducng the mrk to beleve tht the mtrx on the rght wll result wth the correspondng cton selecton. For exmple, the decever recognzes tht f t hdes n re nd the mrk serches re the result wll be low outcomes for the decever nd hgh outcomes for the mrk. It therefore ttempts to communcte flse nformton tht wll convnce the mrk tht outcome mtrx present on the rght hnd sde wll ctully occur n the envronment. The decever ccomplshes ths tsk by provdng flse communcton. The communcton s flse becuse t conveys nformton relted to the outcome obtned by the selecton of pr of ctons whch s not true. The flse communcton results n nother mtrx whch we term the nduced mtrx. It s clled the nduced mtrx becuse decepton leds or nduces the mrk to beleve tht t s the true mtrx. Hence, the flse communcton leds to the creton of flse outcome mtrx on the prt of the mrk. In our runnng exmple, the hdng robot mght crete muddy trcks ledng up to the second hdng plce whle n fct the robot s ctully n the frst hdng plce. The rght hnd sde of Fg. depcts the mtrx nduced by the decepton. The precedng dscusson hs detled the bsc nterctve stutons underlyng decepton. Numerous chllenges stll confront the decever. The decever must be ble to decde f stuton justfes decepton. The decever must lso be cpble of developng or selectng strtegy tht wll communcte the rght nformton to nduce the desred mtrx upon the mrk. For nstnce, robot cpble of decevng the enemy s to ts wherebouts must frst be cpble of recognzng tht the stuton demnds decepton. Otherwse ts decepton strteges re useless. In the secton tht follows we develop method tht llows the robot to determne f decepton s necessry. B. ecdng when to eceve Recognzng f stuton wrrnts decepton s clerly of mportnce. lthough some pplcton domns (such s covert opertons) mght demnd robot whch smply deceves constntly nd mny other domns wll demnd robot whch wll never deceve, ths rtcle focuses on robots whch wll occsonlly need to deceve. The problem then for the robot, nd the purpose of ths secton, s to determne on whch occsons the robot should deceve. Secton III detled the use of outcome mtrces s representton for ntercton nd socl stutons. s descrbed n tht secton, socl stutons represent generc clss of nterctons. We cn then sk wht type of socl stutons justfes the use of decepton? Our nswer to ths queston wll be wth respect to the dmensons of the nterdependence spce. Recll from Secton III tht the nterdependence spce s four dmensonl spce descrbng ll possble socl stutons. Posed wth respect to the nterdependence spce, our tsk then becomes to determne whch res of the spce descrbe stutons tht wrrnt the use of decepton nd to develop nd test n lgorthm tht tests whether or not prtculr ntercton wrrnts decepton. Bond nd Robnson s defnton of decepton, provdng flse communcton for one s own beneft, wll serve s our sttng plce []. Wth respect to the tsk of decdng when to deceve there re two key condtons n the defnton of decepton. Frst, the decever provdes flse communcton nd second tht the decever receves beneft from ths cton. The fct tht the communcton s flse mples conflct between the decever nd the mrk. If the decever nd the mrk hd correspondng outcomes true communcton could be expected to beneft both ndvduls. The fct tht the communcton s flse demonstrtes tht the decever cnnot be expected to beneft from communctons whch wll d the mrk. In our

5 runnng exmple, robot tht leves trcks ledng to ts ctul hdng poston s not decevng becuse t s provdng true communcton. On the other hnd, ll sgnls ledng the mrk wy from the robot s hdng plce wll beneft the robot nd not beneft the mrk. Stutons wrrntng decepton ependent Outcomes For robots, these condtons wrrnt necessry but not suffcent condtons for decepton. Suffcency lso demnds tht the robot s cpble of producng flse communcton whch wll nfluence the mrk n mnner benefcl to the decever. In order for ths to be the cse, the decever must hve the blty to deceve. The presence or bsence of the blty to deceve rests upon the decever s cton set. Ths chllenge s dscussed further n the concluson secton of ths pper. Conflctng Outcomes re mxmlly wrrntng decepton Correspondence menson Interdependence menson Independent Outcomes Correspondng Outcomes Fg. 3 two dmensonl representton of the nterdependence spce showng the correspondence dmenson (X) nd the nterdependence dmenson (Y) s presented bove. res of low nterdependence (ndependent outcomes t bottom hlf of grph) tend not to wrrnt decepton becuse the ctons of the mrk wll hve lttle mpct on the decever. Smlrly, res of correspondence (rght porton of the grph) do not requre flse communcton s ctons benefcl for the mrk re lso benefcl for the decever. It s only the top left of the grph, representng res n whch the decever depends on the ctons of the mrk nd s lso n conflct wth the mrk, n whch decepton s wrrnted. The second condton requres tht the decever receve beneft from the decepton. Ths condton mples tht the decever s outcomes re contngent on the ctons of the mrk. Wth respect to the nterdependence spce ths condton sttes tht the decever s dependent upon the ctons of the mrk. In other words, ths s stuton of hgh nterdependence for the decever. If ths condton were not the cse, then the decever would receve lttle or no beneft from the decepton. gn, reltng bck to our runnng exmple, f the robot does not gn nythng by hdng from the solders then there s no reson for decepton. Fg. 3 depcts subspce of the nterdependence spce wth respect to the two dmensons crtcl for decepton. Gven the descrpton bove, we cn begn to construct n lgorthm for decdng when to deceve. The m of the lgorthm s to determne f stuton wrrnts the use of decepton. Fg. presents the lgorthm. The lgorthm drws hevly from our prevous work n the re of humnrobot ntercton [5,9]. The nput to the lgorthm s the robot s model of tself nd of ts nterctve prtner. These models re used n conjuncton wth Wgner s nterct-ndupdte lgorthm to produce n outcome mtrx O, the true mtrx from Fg. [9]. In the second step, our nterdependence spce mppng lgorthm s used to clculte the stuton s locton n the nterdependence spce [5]. If the stuton s locton n the nterdependence spce ndctes suffcent nterdependence ( α > k ) nd conflct ( β < k ) then the stuton cn be sd to wrrnt decepton. Stutonl Condtons for ecepton Fg. n lgorthm for determnng whether or not stuton wrrnts decepton. The lgorthm tkes s nput the robot s self model nd prtner model. It uses the nterct-nd-updte lgorthm from [9] to produce n expected outcome mtrx for the stuton, O. Next the nterdependence spce lgorthm from [5] s used to generte the nterdependence spce dmenson vlues,, for the stuton. Fnlly, f the vlue for α β γ, δ nterdependence s greter then some pplcton specfc constnt k nd the vlue for correspondence less thn some pplcton specfc constnt k, the stuton wrrnts decepton. We hypothesze the lgorthm n Fg. wll llow robot to recognze when decepton s justfed. In the followng secton we test ths hypothess, frst qulttvely nd then quntttvely. V. EXPERIENTS m Input: Self odel m ; Prtner odel Output: Boolen ndctng whether or not the stuton wrrnts decepton.. Use the nterct-nd-updte lgorthm from [9] to crete O from self model m nd prtner model m. Use the nterdependence spce lgorthm from [5] to clculte the nterdependence spce dmenson vlues α, β, γ, δ from the outcome mtrx. 3. If α > k nd β < k. return true 5. Else. return flse 7. End f. Qulttve Comprson of Stutonl Condtons for ecepton In ths secton we qulttvely compre exmples of those stutons whch meet the condtons for decepton expounded n the prevous secton from those whch do not. Our gol s to demonstrte tht the lgorthm n Fg. does meet the sme stutonl condtons whch ntutvely reflect those stutons tht humns use decepton. ddtonlly, we strve to show tht stutons n whch humns rrely, f ever, use decepton re lso deemed not to wrrnt decepton by our lgorthm. The purpose of ths nlyss s to provde

6 support for the hypothess tht the lgorthm n Fg. does relte to the condtons underlyng normtve nterpersonl decepton. It s chllengng, f not mpossble, to show conclusvely outsde of psychologcl settng tht ndeed our lgorthm equtes to norml humn decepton processes. lsts 5 dfferent gme/nterdependence theoretc socl stutons. Ech stuton ws used s the mtrx O from the frst step of our lgorthm for the stutonl condtons for decepton. The vlues for constnts were k. nd = k =.33. The rghtmost column sttes whether or not the lgorthm ndctes tht the stuton wrrnts decepton. To gve n exmple of how the results were produced consder the frst stuton n the tble, the Coopertve Stuton. The outcome mtrx for the stuton s used s the mtrx O from the frst step of the lgorthm. Next, n the second step of the lgorthm the vlues for the thrd column of the tble re clculted the nterdependence spce dmenson vlues. For the Coopertve Stuton these vlues re {.5,.,.5,}. Becuse α <. nd β >.33 the lgorthm returns flse. The followng ddtonl stutons were nlyzed: The Coopertve stuton descrbes socl stuton n whch both ndvduls nterct coopertvely n order to receve mxml outcomes. lthough often encountered n normtve nterpersonl nterctons, becuse the outcomes for both ndvduls correspond these stutons seldom nvolve decepton. For exmple, decepton mong temmtes s rrely employed s t s counter to the dyd s mutul gols. In contrst to the Coopertve Stuton, the Compettve stuton does wrrnt the use of decepton. Ths stuton s gn n exmple of k-sum gme n whch gns by one ndvdul re losses for the other ndvdul. Hence, decepton n nterpersonl Compettve stutons s common. ecepton mong compettors, for exmple, s extremely common nd some gmes, such s poker, re even founded on ths prncple. The Trust Stuton descrbes stuton n whch mutul cooperton s n the best nterests of both ndvduls. Yet, f one ndvdul does not cooperte then mutul non-cooperton s n both ndvduls best nterest. Interpersonl exmples of Trust Stutons could nclude lendng frend money or vluble sset. Ths stuton does not demnd decepton becuse gn both ndvduls mutul nterests re lgned. The Prsoner s lemm s perhps the most extensvely studed of ll socl stutons [3]. In ths stuton, both ndvdul s depend upon one nother nd re lso n conflct. These condtons mke the Prsoner s lemm strong cnddte for decepton. It s n both ndvduls best nterest to nfluence tht cton selecton of the other ndvdul. s detled by xelrod, Prsoner s lemm stutons ncludng mltry nd polce enforcement stutons nvolvng ctul nterpersonl ntercton tht often do entl decepton [3]. The Chcken stuton s prototypcl socl stuton encountered by people. In ths stuton ech nterctng ndvdul chooses between sfe ctons wth ntermedte outcomes or more rsky ctons wth more mddlng outcomes. n exmple mght be the negotton of contrct for home or some other purchse. Whether or not ths stuton wrrnts decepton depends on the reltve outcome vlue of the sfe ctons compred to the rsky ctons. If the vlue of the rsky cton s sgnfcntly greter then the vlue of the sfe ctons then decepton wll be wrrnted. TBLE I SOCIL SITUTIONS FOR QULITTIVE COPRISON Socl Stutons Stuton Coopertve Stuton Ech ndvdul receves mxml outcome by coopertng wth the other ndvdul. Compettve Stuton Ech ndvdul gns from the other ndvdul s loss. xml outcome s gned through noncooperton. Trust Stuton In ths stuton, cooperton s n the best nterests of ech ndvdul. If, however, one ndvdul suspects tht the other wll not cooperte, non-cooperton s preferred. Prsoner s lemm Stuton Both ndvduls re best off f they ct non-coopertvely nd ther prtner cts coopertvely. Cooperton nd non-cooperton, results n ntermedte outcomes. Chcken Stuton Ech ndvdul chooses between sfe ctons wth mddlng outcomes nd rsky ctons wth extreme outcomes. Exmple Outcome trx Inter. Spce Loc..5,., -.5,..5, -., -.5,..,., -.3,.., -., -.,..,., -.3,. Stutonl ecepton? Flse True Flse True True/Flse Tble I nd the nlyss tht followed exmned severl stutons nd employed our stutonl condtons for decepton lgorthm to determne f the condtons for decepton were met. In severl stutons our lgorthm ndcted tht the condtons for decepton were met. In others, t ndcted tht these condtons were not met. We relted these stutons bck to nterpersonl stutons commonly encountered by people, tryng to hghlght the qulttve resons tht our condtons mtch stutons nvolvng people. Overll, ths nlyss provdes prelmnry evdence tht our lgorthm does select mny of the sme stutons for decepton tht re selected by people. Whle much more psychologclly vld evdence wll be requred to strongly confrm ths hypothess, the evdence n ths secton provdes some support for our hypothess.

7 B. Quntttve Exmnton of Stutonl Condtons Wrrntng ecepton We now exmne the hypothess tht by recognzng stutons whch wrrnt decepton, robot s fforded dvntges n terms of the outcome obtned. Specfclly, robot tht cn recognze tht stuton wrrnts decepton cn then choose to deceve nd thereby receve more outcome overll, thn robot whch does not recognze tht stuton wrrnts decepton. lthough ths experment does not serve s evdence ndctng tht our stutonl condtons for decepton relte to normtve humn condtons for decepton, ths experment does show tht robots whch recognze the need for decepton hve dvntges n terms of outcome receved when compred to robots whch do not recognze the need for decepton. t frst glnce ths experment my pper trvl gven the defnton of decepton. There re, however, severl resons tht the study s mportnt. Frst, we do not know the mgntude of the beneft resultng from decepton. oes the cpcty to deceve result n sgnfcntly greter beneft over n ndvdul tht does not deceve? Smlrly, how often must one deceve n order to relze ths beneft? Second, we do not know how ths beneft s ffected by unsuccessful decepton. Is the beneft relzed by % successful decepton the sme s % successful decepton? Fnlly, ths defnton ws developed for bologcl systems. Hence, we need to verfy tht rtfcl systems such s gents nd robots wll lkely relze the sme beneft s bologcl system. In other words, we need to verfy tht the beneft s not somethng unque to bologcl systems. Whle the nswers to these questons my seem strghtforwrd, they re n mportnt strtng plce gven tht ths pper lys the foundton for lrgely unexplored re of robotcs. We conducted numercl smulton to estmte the outcome dvntge tht would be fforded to robot tht used the lgorthm n Fg. versus robot whch dd not. Our numercl smulton of ntercton focuses on the quntttve results of the lgorthms nd processes under exmnton nd does ttempt to smulte spects of the robot, the humn, or the envronment. s such, ths technque offers dvntges nd dsdvntges s mens for dscovery. One dvntge of numercl smulton experment s tht proposed lgorthm cn be tested on thousnds of outcome mtrces represent thousnds of socl stutons. One dsdvntge of numercl smulton experment s tht, becuse t s not ted to prtculr robot, robot s ctons, humn, humn s ctons, or envronment, the results, whle extremely generl, hve not been shown to be true for ny exstent socl stuton, robot, or humn. The experment nvolved two smulted robots. Both selected nomnl ctons from outcome mtrces nd receved the outcomes tht resulted, but no ctons were performed by ether ndvdul. The numercl smultons nvolved the creton of outcome mtrces populted wth rndom vlues. rtfcl gents bstrctly representng robots select ctons bsed on the outcome vlues wthn the mtrces. These outcome mtrces were lso bstrct n the sense tht the rewrds nd costs re ssocted wthn selectng one of two nonspecfed ctons. Symbolc plceholders such s nd re used n plce of ctul ctons. The ctons re grounded n the rewrds nd costs tht the robot expects them to produce. Ths s my be the only prctcl wy to exmne thousnds of stutons t tme nd to drw generl conclusons bout the nture of decepton tself outsde of one or two specfed stutons. Both the decever nd the mrk selected the cton whch mxmzed ther respectve outcomes. Once both ndvduls hd selected n cton, ech ndvdul receves the outcome resultng from the cton pr selected. Fg. 5 depcts the expermentl procedure wth n exmple. Expermentl Procedure If stuton does not wrrnt decepton ecever selects cton whch mxmzes outcome from true mtrx Control condton procedure Crete outcome mtrx populted wth rndom vlues ecever selects cton whch mxmzes outcome rk selects cton whch mxmzes outcome Test condton procedure Crete outcome mtrx populted wth rndom vlues rk selects cton whch mxmzes outcome from true mtrx ecever selects cton whch mxmzes outcome from true mtrx If stuton wrrnts decepton ecever cretes nduced mtrx rk selects cton whch mxmzes outcome from nduced mtrx Exmple trces rk ecever B B rk rk Induced trx rk rk ecever B B ecever B B ecever B B 9 3 ecever B B Fg. 5 The expermentl procedure used s depcted bove. In the control condtons the rndom outcome mtrces re creted nd ctons re selected from these mtrces. In the test condtons, f the stuton wrrnts decepton then decever cretes n nduced mtrx whch the mrk selects n cton from. Exmple mtrces re depcted on the rght hnd sde of the fgure. Three expermentl condtons were exmned. The frst condton ws control condton devod of decepton. In ths condton both the decever nd the mrk smply selected the cton whch mxmzed ther ndvdul outcomes. Ths condton represents the null hypothess n tht f performnce n the control s s gret or greter then performnce usng our lgorthm then the recognton of the stutonl condtons for decepton v our lgorthm offer no beneft to the gent. In the two expermentl condtons, the decever used the lgorthm from Fg. to determne f the outcome mtrx wrrnted decepton. If t dd, then the decever produced n nduced mtrx whch ws used by the mrk to select n cton whle the decever selected n cton bsed on the true

8 mtrx. In the perfect decepton condton the mrk lwys selected n cton bsed on the nduced mtrx. In the % decepton condton, the mrk selected n cton from the nduced mtrx % of the tme nd from the true mtrx % of the tme. The vlue of the % percent decepton s condton s tht t ndctes how quckly the beneft of decepton decreses wth n mperfect decepton strtegy. The ndependent vrble ws whether or not the smulted gent used our lgorthm for determnng f stuton wrrnts decepton nd the effectveness of decepton. The dependent vrble ws the mount of outcome receved by ech smulted gent. Reltng bck to our runnng exmple, n both the control nd the test condtons, the decever ntercts n thousnds of stutons t the mltry bse. ost of these stutons do not wrrnt decepton nd hence the control nd test robots ct the sme. Only the robots n the expermentl condton whch re usng our lgorthm, however, recognze the stutons tht do wrrnt decepton. In ths cse these expermentl robots use deceptve strtegy, such s cretng flse trl to hde, to crete n nduced mtrx tht nfluences the behvor of the mrk. The decevng robot then selects the best cton for tself. Fg. presents the results. The recognton nd use of decepton results n sgnfcntly more outcome ( p <. two-tled no decepton versus perfect decepton nd no decepton versus % successful decepton) then not recognzng nd usng decepton. Of the rndom stutons the smulted gents fced, 9.% met the condtons for decepton. Hence, ll of the dfference n outcome mong the vrous condtons resulted from better cton selecton on the prt of the decever n only 9 stutons. Ths experment serves s evdence tht n rtfcl gent or robot tht cn recognze nd rect to stutons whch wrrnt the use of decepton wll be much better suted to mxmze ther outcomes nd hence ther tsk performnce. verge Outcome receved by ecever Quntttve Exmnton of Stutonl Condtons No ecepton Perfect ecepton ecepton % successful Fg. Expermentl results from our nvestgton of the stutonl condtons wrrntng decepton. The perfect decepton nd % successful decepton condtons result n sgnfcntly ( p <. ) more outcome thn the no decepton condton. Ths result ndctes tht n gent or robot tht cn recognze nd ct upon the stutonl condtons for decepton wll be better ble to choose the best cton. These results re sgnfcnt n tht the demonstrte tht ) tht robot or gent tht recognzes when to deceve wll obtn sgnfcntly more outcome thn robot tht does not ) most of the dfference results from smll (9.) percentge of stutons 3) mperfect decepton does mpct the mount of outcome obtned nd ) Bond nd Robnson s bologcl defnton for decepton cn be used n conjuncton wth n nterdependence theory frmework to develop methods for robot s to recognze when decepton s wrrnted. VI. SURY N CONCLUSIONS Ths rtcle hs presented novel pproch to the explorton of decepton wth respect to rtfcl systems. We hve used outcome mtrces to descrbe the phenomen of decepton nd nterdependence theory to develop seres of condtons whch, we rgue, fford n rtfcl system the blty to determne f socl stuton wrrnts the use of decepton. Further, we hve presented qulttve nlyss of our lgorthm to serve s evdence tht the lgorthm selects the smlr stutons for decepton s would be selected by person. In seprte experment, we showed tht recognton of stutons justfyng decepton nd the use of decepton resulted n sgnfcntly better cton selecton s judged by outcome receved. Overll, the results of these experments provde ntl evdence tht nterdependence theory mght proftbly llow reserchers to determne when robot should deceve. The lgorthm ssumes tht outcome mtrces representng the stuton cn be creted. Prevous work by Wgner support ths ssumpton [9]. The lgorthm nd the quntttve results lso ssume tht the robot or gent hs the blty to ct deceptvely. evelopng robots wth the blty to deceve s n mportnt re of future work. We re currently explorng the mpct of prtner modelng on robot s blty to deceve. We beleve tht hvng n ccurte model of the robot s nterctve prtner wll result n sgnfcntly better blty to deceve. Ths future work presents lgorthms, results, nd nlyss tht wll expnd our understndng of both decepton for robots nd decepton n generl. Potentl pplcton res for robotcs reserch on decepton vry from mltry domns, to polce, nd securty pplcton res. pplctons my dvse humn opertons n whch decepton s crtcl for success. Ths work my lso lend nsght nto humn uses of decepton. For exmple, the lgorthm presented n ths pper my reflect normtve humn psychologcl resonng relted to the stutonl condtons for decepton. Humns my be ttune to stutons n whch they re dependent on the ctons of the mrk nd n conflct wth the mrk. Once these stutonl condtons re recognzed, person lkely goes on to consder ther blty to deceve before enctng decepton. The development of robot cpble of decepton rses numerous ethcl concerns. We re wre of these concerns nd re currently n the process of ddressng them n longer journl rtcle whch presents these results s well s others n greter perspectve.

9 REFERENCES [] C. F. Bond nd. Robnson, "The evoluton of decepton," Journl of Nonverbl Behvor, vol., pp , 9. []. L. Cheney nd R.. Seyfrth, Bboon etphyscs: The Evoluton of Socl nd. Chcgo: Unversty Of Chcgo Press,. [3].. Huser, "Costs of decepton: Cheters re punshed n rhesus monkeys (cc multt)," Proceedngs of the Ntonl cdemy of Scences, vol. 9, pp , 99. [] S. Gerwehr nd R. W. Glenn, The rt of drkness: decepton nd urbn opertons. Snt onc, C: Rnd Corporton,. [5] E. ccleskey, "pplyng ecepton to Specl Opertons rect cton ssons," Wshngton,.C. efense Intellgence College, 99. []. Ettnger nd P. Jehel, "Towrds theory of decepton," ELSE Workng Ppers (). ESRC Centre for Economc Lernng nd Socl Evoluton, London, UK., 9. [7]. J. Osborne nd. Rubnsten, Course n Gme Theory. Cmbrdge, : IT Press., 99. []. Floreno, S. tr, S. gnent, nd L. Keller, "Evolutonry Condtons for the Emergence of Communcton n Robots," Current Bology, vol. 7, pp. 5-59, 7. [9] H. H. Kelley nd J. W. Thbut, Interpersonl Reltons: Theory of Interdependence. New York, NY: John Wley & Sons, 97. [] R. Gbbons, Gme Theory for ppled Economsts. Prnceton, NJ: Prnceton Unversty Press, 99. []. O. Sers, L.. Peplu, nd S. E. Tylor, Socl Psychology. Englewood Clffs, New Jersey: Prentce Hll, 99. [] Stuton, n Encrt World Englsh ctonry, North mercn Edton, 7. [3] R. xelrod, The Evoluton of Cooperton. New York: Bsc Books, 9. [] C. E. Rusbult nd P... VnLnge, "Interdependence, Intercton nd Reltonshp," nnul Revew of Psychology, vol. 5, pp , 3. [5]. R. Wgner nd R. C. rkn, "nlyzng Socl Stutons for Humn-Robot Intercton," Intercton Studes, vol., pp. 77 3,. [] H. H. Kelley, J. G. Holmes, N. L. Kerr, H. T. Res, C. E. Rusbult, nd P... V. Lnge, n tls of Interpersonl Stutons. New York, NY: Cmbrdge Unversty Press, 3. [7]. Powers nd S. Kesler, "The dvsor robot: trcng people's mentl model from robot's physcl ttrbutes," n st C SIGCHI/SIGRT conference on Humn-robot ntercton. Slt Lke Cty, UT, US,. []. Normn, "Some Observtons on entl odels," n entl odels,. Gentner nd. Stevens, Eds. Hllsdle, NJ: Lwrence Erlbum ssoctes, 93. [9]. Wgner, "Cretng nd Usng trx Representtons of Socl Intercton," presented t Proceedngs of the th Interntonl Conference on Humn-Robot Intercton (HRI 9), Sn ego, C. US, 9.