CyclopsRing: Enabling Whole-Hand and Context-Aware Interactions Through a Fisheye Ring

Transcription

1 CylopsRing: Enling Whole-Hn n Context-Awre Intertions Through Fisheye Ring Liwei Chn Yi-Ling Chen Chi-Ho Hsieh Rong-Ho Ling Bing-Yu Chen Grute Shool of Mei Design, Keio University Ntionl Tiwn University liwei@km.keio..jp {yilinghenntu, r , rhling, roin}@ntu.eu.tw ABSTRACT This pper presents CylopsRing, ring-style fisheye imging werle evie tht n e worn on hn weings to enle whole-hn n ontext-wre intertions. Oserving from entrl position of the hn through fisheye perspetive, CylopsRing sees not only the operting hn, ut lso the environmentl ontexts tht involve with the hn-se intertions. Sine CylopsRing is finger-worn evie, it lso llows users to fully preserve skin feek of the hns. This pper emonstrtes proof-of-onept evie, reports the performne in hn-gesture reognition using rnom eision forest (RDF) metho, n, upon the gesture reognizer, presents set of intertion tehniques inluing onfinger pinh-n-slie input, in-ir pinh-n-motion input, plm-writing input, n their intertions with the environmentl ontexts. The experiment otine n 84.75% reognition rte of hn gesture input from tse of seven hn gestures ollete from 15 prtiipnts. To our knowlege, CylopsRing is the first ring-werle evie tht supports whole-hn n ontext-wre intertions. Author Keywors Werle evies, ring, whole-hn intertion, plm touh input, finger touh input, hn gesture input ACM Clssifition Keywors H.5.m. Informtion Interfes n Presenttion (e.g. HCI): Misellneous INTRODUCTION Designing werle interfes for hn-se intertions is n intriguing hllenge. Previous reserh hs emonstrte interfes in vrious werle forms worn t vrious oy positions, suh s essories set on hes [29], hests [3], shoulers [12] or hns [17]. The two key mesures for ssessing these interfes, whih shoul e onsiere simultneously, re their forms n emerging intertions. Ring werles re generlly onsiere oth ttrtive n hllenging, euse of their miniture forms s well s their im to ugment the most powerful interfe - humn hns. Permission to mke igitl or hr opies of ll or prt of this work for personl or lssroom use is grnte without fee provie tht opies re not me or istriute for profit or ommeril vntge n tht opies er this notie n the full ittion on the first pge. Copyrights for omponents of this work owne y others thn ACM must e honore. Astrting with reit is permitte. To opy otherwise, or repulish, to post on servers or to reistriute to lists, requires prior speifi permission n/or fee. Request permissions from Permissions@m.org. UIST 15, Novemer 08 11, 2015, Chrlotte, NC, USA Copyright 2015 ACM /15/11...$ DOI: Figure 1. By supporting whole-hn n ontext-wre intertions, CylopsRing llows users to intert with rel-worl ojets suh s the smrt lmp y pinh-n-motion input. Unfortuntely, most existing ring werles were exlusively esigne for ugmenting the wering fingers, lowering the opportunity to tke the full vntges of the hns. CylopsRing This pper presents CylopsRing, ring-style fisheye imging werle evie worn on the hn weing. By oserving from entrl position of the hn through the hn-entri fisheye perspetive, CylopsRing n pture the whole frontl skin region of the hn. Thus, this region of the hn eomes useful intertive surfe n llows for performing hn gesture reognition. A further enefit of its extremely wie fiel of view is tht CylopsRing n inorporte environmentl ontexts into hn-se intertions. Finlly, the form ftor of CylopsRing is ring, whih preserves skin hpti feek of the hns. Figure 1 illustrtes senrio in whih CylopsRing is use to ontrol smrt lmp y pinh-n-motion input. The proof-of-onept prototype emonstrte the fesiility of gesture reognition using RDF-se lgorithms, n set of intertive tehniques omining whole-hn n ontextwre intertions. In the experiment, RDF-se pixel lssifition hieve 84.75% reognition rte for hn gesture input from tse of seven hn gestures ollete from 15 prtiipnts, whih inite the effetiveness of the fisheye imges for rih hn-se intertions. Contriution The min ontriution of this pper is the onept of using fisheye hn-entri view of the hn to enle whole-hn n ontext-wre intertions. This work (1) evelope proof-of-onept prototype, (2) emonstrte the effetiveness of CylopsRing for hn gesture input using rnom eision forest (RDF) metho, n (3) presente set of wholehn n ontext-wre intertions.

2 RELATED WORK This pper is relte to werle interfes for hn-se intertions, prtiulrly those in whih finger ring is use s the interfe. Hn-Bse Intertions Through Boy Werles Beuse they n etet intertions t istne, mers n e mounte on ifferent oy lotions to enle hnse intertions. Depening on the mounting position (e.g., hes [6, 20, 29], hests [3, 11, 18, 19], shoulers [12], foot [1] n wrists [17, 24]), hn-se intertions hve ifferent enefits. For exmple, he-mounte mers n potentilly ugment ny ojets uner users perspetives with intertive funtions, inluing users hns, ut require users visul ttentions. Chest- n shouler-mounte mers require users moving n fing the operting hns to the mer s fiel-of-view. Wrist-mounte mers re generlly onsiere effetive for hn-se intertions euse they re ner the hns. However, the limittion is olusion t the ngles of wrists. CylopsRing vois this limittion, ut limits to gestures where suffiient prts of fingers re visile to the mer. Unlike mers, low-level sensing pprohes re enefite from iret ontt with the skin. The wrist is ommon lotion for implementing suh sensing tehniques. Aousti sensors [14, 21], for exmple, hve een use to etet tpping positions on the hn. Hn gestures n e reognize y swept frequeny pitive sensing [26], forerm eletromyogrphy [25], n y mesuring wrist ontours [10] or pressures me to the wrist [8]. Low-level sensing tehniques re enefiil for lightweight power n omputtionl onsumption ut suffer from limite input pilities (e.g., isrete input), prtiulrly when the evies re worn n esigne in ring-like forms [23]. In ontrst, the selete form ftor for CylopsRing is finger ring to llow rih n ontinuous hn-se intertions. Hn-Bse Intertion Through Finger Werles Finger werles re ttrtive forms, ut their hllenge is the limite re for implementing funtions. Therey, the inputs for most existing finger werles re exlusively ssoite with the instrumente fingers. For exmple, iring [23] etete tpping on the oy of the ring. Using mgneti loliztion, Arr [13] enle roun-evie input with the mgnet-instrumente finger, utrk [5] enle touh intertion on the plm, n FingerP [4] turne fingertips into touh interfe. LightRing [16] enle the instrumente fingers to perform touh intertion on ny surfe. MgiFinger [15] n EyeRing [22] propose mer-instrumente fingers to intert with environmentl ontexts tht re in ssoition with the finger through the mer view. In omprison, CylopsRing emonstrtes the potentils to ugment the whole hn of the instrumente finger n support ontext-wre intertions. HARDWARE PROTOTYPE The min omponent of CylopsRing is miniture fisheye mer mounte on ring ple of oserving the entire = + + Color mer with 185 fisheye lens Figure 2. () CylopsRing omprises miniture 185-egree fisheye mer mounte on the ege of 3D printe ring. () Wering it on fingers n position the evie t ertin hn weings. frontl skin region of the user s hn. Here, we emonstrte the omponents of our hrwre prototype, n present the vrition of wering the evie t ifferent hn weings n the enefits of our hoie to wer it in inex finger. Fisheye Ring Devies Figure 2 shows tht the hrwre prototype onsists of miniture mer with mini-fisheye lens, whih hs fol length of 1.2 mm n n perture of F1.8 to enle 185- egree fiel-of-view. The mer is 14 mm in imeter n 15 mm in height. A 1/3 SONY CCD sensor ws use to quire imges t 30 fps. A speil ring ws frite to hol the fisheye mer y the ege suh tht the fisheye mer is positione t hn weing while users wering it s shown in Figure 2. The prototype quire this unusul esign to ompenste for the reltively lrge oy of the fisheye lens (i.e., 14 mm in imeter) while llowing the lens to e positione s lose s possile to entrl lotion of the hn. The size of the fisheye lenses is expete to erese with future vnes in mer optis, whih will ompenste for esign ompromises in the prototype. For referene, the NnEye mer mnufture y AWAIBA 1, espite oes not provie the require viewing ngles, provies 120-egree fiel-of-view n mesures 1.0 mm 1.0 mm 1.7 mm in three imensions. Plement of the ring evie Figure 3 isplys the oserve imges of user performing two gestures tken from ifferent hn weings y wering the ring on the thum, inex finger, mile finger, n ring finger. Different weings otin onsierly ifferent views of the sme hn gestures. For the PINCH gesture (Figure 3), for exmple, the views from the inex, mile, n ring fingers lerly revel the irle forme y the inex finger n thum t vrious istnes, ut the stright fingers re rely visile. In omprison, the evie t the thum nnot view the irle ut n lerly view the stright fingers s if it oul ount the fingers for knowing the gesture types. In the GUN gesture (Figure 3), the evie t the mile n ring fingers re mostly olue y the urling fingers. The views from the inex finger n thum re lso very ifferent. 1

3 Figure 4. Foregroun Extrtion Using Skin Color. Figure 3. Plement of the ring in ifferent hn weings. The urrent implementtion still requires n exessively lrge re (e.g., 1 squre m) to ommote the evie on the fingers. The evie is positione on the weing of the inex n mile fingers y wering the evie on the inex finger (Figure 2) euse the inex finger is more flexile to yiel spe in the weing for the evie without preventing users from performing gestures. RECOGNIZING HAND GESTURES To perform gesture reognition, the imges pture y the fisheye mer were first roppe y winow to exlue some irrelevnt regions. For effiieny, the imges were further resize to to extrt the skin olor regions. The inry msk imges were then use for RDF gesture reognizer trining n testing. Foregroun Extrtion Using Skin Color The foregroun region ws extrte y using the metho esrie in [32] to etet skin olor. The oserve imge is first onverte into YCCr olor spe, n skin olor pixel is extrte if its Cr n C vlues fll into the rnges [145, 185] n [85, 115], respetively. Smll kgroun noises re remove y morphologil opertions. Owing to the unique plement of CylopsRing, the rel hn regions lwys orrespon to los of skin olors tht penetrte the peripherl ounry of the fisheye imge. Speifilly, the floting omponents in the entrl region of the imge my e filtere out to otin len msk of hn regions. Notly, the los other thn hn regions my still orrespon to relevnt ontextul ojets, e.g., humn fes. CylopsRing lso retins these los for ontext-wre intertion. Figure 4 illustrtes n exmple of skin olor etetion n the orresponing inry msk. Rnom Deision Forest for Gesture Clssifition The RDF is generi t-riven lerning lgorithm tht is wiely use for omputer vision [7, 27]. Reent HCI reserh hs lso use RDF methos for gesture reognition (e.g., inir hn gestures [28] n hn gestures on n ove keyors [30]) n for full oy posture reognition vi hestmounte fisheye mer [3]. As esrie in [27], hn gesture reognition is formulte s pixel lssifition prolem y using RDF lssifiers. Similr to [3, 30], the RDF lssifiers re trine from intensity ifferene fetures erive from pirs of rnomly generte offset vetors from the signture imges. Inste of the motion history imges (MHIs) use in previous works, this stuy iretly employe inry imges otine y performing skin olor etetion in stti gesturl imges (Figure 4). Unlike [3], itionl sensor t re not use to filitte the reognition tsks. More etils of RDFs re referre to the relevnt works. Evlution: 84.75% Rte of Hn Gesture Input Experimentl Settings As presente in Figure 5, the experiment inlue seven hn gestures, inluing four pinh gestures n three pplitionrelte gestures. Notly, eh pinh gesture inlues two tpping positions: one t the fingertip n the other t the mile of the finger. Prtiipnts Fifteen prtiipnts (7 femles) were reruite from our eprtment. They were ge etween 23 n 27. We ttempte to reruit prtiipnts with wiely vrying height (men = 167m, st = 7.6) to inrese the vrine in plm size. The length of their hns from the tip of the mile finger to the wrist were reore (men = m, st = 1.21 m). All prtiipnts were right hne. Trining Dt Aquisition Dt were quire in two sessions with 3-y intervl. After helping the prtiipnts put on the evie, the experimenter expline the gestures to e performe. The experimenter then ske the prtiipnts to prtie mking ll gestures to ensure tht they fully unerstoo the etils. During the stuy, monitor prompte the prtiipnts with photogrph of eh tril gesture. Eh prtiipnt performe four rouns of the entire gesture set. Eh prtiipnt performe 28 lele gesture imges (7 gestures 4 repetitions). Finlly, we reore short vieo of non-gesture hn motions y sking users to sully streth or url their fingers. Performne Evlution A tehnil evlution ws performe to etermine the reognition rtes hieve y pplying RDF lssifier. Prmeter settings re summrize s follows: tree numer T = 3 n mximl epth D = 19; for eh trining imge, 2000 pixels were rnomly smple s t points; 2000 nite fetures n 50 nite threshols were evlute for splitting n intermeite noe. Leve-one-person-out ross-vlition ws performe in the evlution. The RDF lssifier hieves the reognition rtes to nerly 84.75% for hn gesture reognition. Figure 5 shows the onfusion mtries. Generlly, the gesture lssifier worke well for most lsses. The ses

4 intive n n tive tiveintive Figure 6. The stte trnsition of on-finger pinh-n-slie input. From left, the user performs the intertion on the inex finger (from intive to tive stte) n then releses the pinh (return to intive stte) intivetive Figure 5. Experimentl results of hn gesture reognition using RDF metho. () Seven hn gestures oul e reognize. () Exmples of the skin foregroun region. () Confusion mtrix of gesture reognition. of flse reognition minly ome from the two hngun gestures, whih re visully similr in their inry representtions n thus iffiult to e istinguishe from eh other. LIMITATIONS While the fisheye view llows rih set of hn gestures, CylopsRing nnot ifferentite gestures in whih the issimilr prts of the gestures re eyon its fiel of view. For exmple, experiments showe tht CylopsRing hs iffiulty using the thum to reognize G UN n G UN F IRE gestures. This limittion n e resolve y wering multiple CylopsRings. For exmple, n lterntive fisheye view from the thum with seon CylopsRing onsierly omplements the fiel of view of the first CylopsRing. In ition, our urrent implementtion only opte olor informtion. A future version of CylopsRing with epth sensing pility will further improve reognition of nturl hn intertions. INTERACTION TECHNIQUES The hn gesture reognition pility of CylopsRing ws further exploite y implementing other intertion tehniques, inluing on-finger, on-plm touh intertions, n hn gesture intertion with rel-worl ojets. Eh intertion tehnique is relize y pplying heuristis for hnentri fisheye imges. The intertion esign inlue tivtion/etivtion gesture frmework to enle users to swith mong vrious intertion tehniques. By eteting pre-efine hn gestures, CylopsRing tivtes speifi intertion tehnique n either pplies the orresponing heuristis or isengges from the urrent intertion. The etivtion gesture O PEN H AND ws selete euse it is nturlly performe when users omplete n input n relx. To voi ientl inputs, CylopsRing swithes to new intertion tehnique only when it is in intive stte. On-Finger Pinh-n-Slie Input Users trnsform their fingers into funtionl sliers y first pinhing finger (on either fingertips or the mile of the finger) to trigger n then sliing long the speifi finger. The four P INCH gestures n O PEN H AND re exploite tivte n etivte the intertion, respetively. Figure 7. Intertion with multiple on-finger sliers y vrying the size of the lose re forme y pinh gesture. Effets of () eresing the vlue of the ring-finger slier, n () inresing the vlue of the inex-finger slier. Figure 6 illustrtes the stte trnsition of the pinh-n-slie intertion performe with the inex finger. When CylopsRing is in intive stte (O PEN H AND), it ontinuously etets the ourrene of pinh gestures. When it etets pinh gesture, it tivtes the slier ssoite with the orresponing fingers. In this exmple, the intertion is triggere on the inex finger s shown in Figure 6. In tive stte (Figure 6), finger-slier input is enle y fining the reltively lrge enlose omponent forme y the pinh gesture in the fisheye imges. The vrition in the omponent re is use to lter the slier vlues (Figure 7). When the user releses the pinh gesture, the slier orresponing to the inex finger is etivte. CylopsRing then etets the next tivtion gesture (Figure 6). Plm-Writing Input The fisheye view enles prtil oservtion of the skin region of the plm in the flt hn posture n full oservtion of tht in the hlf-urve hn posture. The plm n then e use s touhp for writing with fingers or pens. This intertion involves one tivtion gesture: the user ens his thum to enle writing input moe. Figure 8 illustrtes the stte trnsition of plm-writing input. Typilly, the inex finger of one hn is the input evie use to write on the plm of the other hn. Therefore, fter entering the tive stte, CylopsRing tkes the first pture imge s the referene imge n extrts the skin olor region s kgroun. Figure 9 shows tht, in tive stte, the finger or olorppe pen n e use to write on the plm. We exploit the well-known Viol-Jones ojet etetor [31] to lolize the fingernil (Figure 9-) or olor lo extrtion n trking to ientify the p of the pen (Figure 9e-h) so s to estimte the stroke trjetory. The fingernil etetor ws trine with 800 positive n 1600 negtive trining smples. Flse posi-

5 intive intivetive tive tiveintive Figure 8. The stte trnsition of plm-writing input. From left, the user ens his thum to tivte the intertion, n relxes his thum to en the intertion. intive intivetive f g tiveintive Figure 10. The stte trnsition of in-ir pinh-n-motion input. Strting from the left, the user performs pinh gesture (from intive to tive stte), moves to rw n in-ir stroke, n releses the pinh (return to intive stte) to omplete the intertion. e tive h Figure 9. Plm-writing input. () The user writes on the plm with the inex finger. () The user writes in ifferent olors using olor pens. tive results were filtere out y exluing winows etete outsie of the kgroun region. Size/temporl oherene ws lso onsiere when etermining the most likely results (Figure 9). The lowest prt of the foregroun skin olor region (Figure 9) ws then use to ue the seletion of the finl etetion winow. The strokes entere y the user were reonstrute y exploiting the following heuristi to mp the positions of the etete fingernils or olor los to lol oorinte system (x-y) efine on the plm of the user. The X oorintes of the etete fingernils or olor los in the fisheye imge re mppe to the x oorintes y simply pplying sling funtion. The y oorintes re otine y using mpping funtion, whih is inversely proportionl to winow size or lo re. Intuitively, s the fingernil or pen p moves loser to the mer, the pereive winow size or lo re inreses, whih ereses the Y oorinte. In-Air Pinh-n-Motion Input Figure 10 shows how CylopsRing uses finger pinh gestures for in-ir pinh-n-motion input. This intertion is enle y four tivtion gestures: the thum touhes the tip of one of the four fingers. Notly, pinhing with ifferent fingers inreses the input molity. Figure 11 shows tht, one CylopsRing tivtes the intertion, motion input is enle y estimting hn movements from isplements of mthe SURF fetures [2] in the onseutive unistorte fisheye imges. A roust estimtion of glol mer motion is otine y isring the motion vetors of outliers tht re too fr from the men motion vetor in mgnitue or orienttion. The motion trjetory is forme y ontenting ll the men motions until the user releses the pinh gesture. Context-wre Intertion Tehniques The fisheye mer enles CylopsRing to pereive the environment n enles the user to intert with rel worl Figure 11. Intertion using in-ir pinh-n-motion input. () The user performs the pinh gesture on the inex finger. () SURF fetures re ompute in the unistorte imge. () The isplements of mthe fetures from the lst frme. ojets suh s smrt pplines visile to the fisheye imges uring intertion. The intertion with rel worl ojets n e relize y ommon omputer vision tehniques suh s ojet n fe reognition. Pinh-n-motion input with ojet reognition Figure 12 shows how the user performs the pinh-nmotion tehnique esrie ove to intert with smrt lmp. In this exmple, the user triggers the seletion of evie y performing pinh gesture on the smrt lmp. CylopsRing then etets the evie type y reognizing the AR tg [9] on the lmp, whih is visile in the fisheye imge. To turn on the lmp, the user performs orresponing opertion (e.g., rwing tik in the ir) n then releses the pinh to issue the ommn. For ojet reognition, feture mthing tehniques suh s SURF n e use inste of the AR tg. Humn intertion with fe reognition Comining fe etetion n reognition enle the use of pinh gestures to intert with nery users. Figure 12 shows possile senrio in whih CylopsRing sens igitl files from one user to nother user. When the user triggers seletion tion y performing pinh gesture, CylopsRing etets fes in its fiel of view. From he-mounte isply, the user sees n unistorte version of CylopsRings view in whih the etete fes re highlighte. The ross isplye t the enter of the imge helps the user to im t fe when moving his hn with the pinh gesture. When the fe of the reeiving user is verifie, the sening user releses the pinh gesture, n the igitl ontent is elivere. Finger opy funtions Another interesting pility is riging the physil n yer worls y igitizing physil ontent. For exmple, Figure 13 shows how CylopsRing n e use to opy printe imge y rgging the inex finger ross the imge. The inex finger is use to speify the region of interest n CylopsRing perform sn n opy opertions. This intertion

6 e f g h Figure 12. Aitionl ontext relte funtions n e inorporte with in-ir pinh-n-motion input. () Diret intertion with smrt lmp through n AR tg. () Pik-n-rop intertion with person vi fe etetion. Figure 13. Finger opy of n imge on the physil pper. () Photogrph tken from thir-person view. () The rw fisheye imge. () Unistorte imge. () Fingertip position. (e) Mthe SURF fetures. (f) The finger stroke. (g) Retifie imge. (h) Croppe imge. is relize y using simple heuristi to ientify the tip of the inex finger. The onsistent sptil reltionship of fingers with respet to the fisheye mer simplifies the serh for the pixel of the gretest x oorinte in the skin region of the left-upper qurter in the unistorte imges (Figure 13). ugmenting CylopsRing with 6 DOF IMU to im pistol or to etet grene eing thrown. The next step is to etermine the finger stroke on the pper. For this purpose, homogrphy trnsformtions etween every two onseutive unistorte imges (Figure 13e) re ompute y ientifying t lest four pirs of SURF fetures therein. In this se, glol homogrphy is suffiient to moel the projetive trnsformtion etween two imges sine the physil ojet eing snne is of plnr surfe. The homogrphies re then use to re-projet ll the fingertip positions onto the first unistorte imge where the stroke is initite to form the stroke trjetory (Figure 13f). While the stroke roughly ptures the igonl line of the ontent-of-interest, we n not simply tke the retngle efine y the igonl line unless the CylopsRing perpeniulrly looks t the pper. We thus retify the imge y pplying preefine homogrphy trnsformtion Ho, to lirte the reltive orienttion etween the CylopsRing n the snne ojet, whih is ssume to e plnr surfe (e.g., tlog in this exmple). Given tht users perform the intertion with similr inline ngles, Ho n e estimte one y reorienting the imge to n upright ngle. Figure 13g shows n exmple of orienttion retifition n Figure 13h is the orresponing roppe region y using the stroke s the igonl of miniml ouning ox. EXAMPLE APPLICATIONS In this setion, we emonstrte severl pplitions to showse the pility of CylopsRing to relize vriety of whole-hn n ontext-wre intertions. Whole-hn intertions Gesturl intertion for virtul relity Hn gestures in VR gming llow plyers to perform vrious hn gestures tht re metphorilly relte funtions. In the first-person shooting gme in Figure 14, the plyer performs G UN gesture to swith wepon to the pistol rrel. Bening own the thum fires the gun, pinhing with the thum n inex finger retrieves grene. Intertivity is enhne y Single-hne intertion for smrt wthes Touh intertion on smrt wthes is sujet to the isply size, yet the touh intertion requires users oth hns. Onfinger pinh-n-slier input (Figure 14) n e use to support single-hnely intertion with smrt wthes. Plm-Touh Intertion for he-mounte isplys CylopsRing trnsforms the plm of the hn into remote touh p for glss isplys. Inste of pressing uttons on the glsses or relying on n externl touh p for input, the user simply writes on the plm to perform touh ontrol of the he-mounte isplys. Figure 14 shows how plm-writing input is use to nvigte n to reply to messges. Context-wre intertions Cyer-worl Clipor The yer-worl lipor of CylopsRing llows to igitize physil ontent. In Figure 14, the user rgs n inex finger ross the region of interest, e.g., photo on n tul piee of pper, the ontent of the speifie region is igitize n iretly psse to the neighoring tlet. Here, this intertion is implemente with finger-opy funtions. Pinh into the Context For ontextul intertions, CylopsRing quikly etermines the intention of the user euse its mer is typilly ime t the ojet with whih the user is interting. Therefore, CylopsRing enles intuitive intertion with rel-worl ojets y pinh gestures. For exmple, in smrt home, users my selet smrt evie with the pinh gesture n issue ommn y rwing in-ir strokes (Figure 14e). Figure 14f shows nother exmple of file shring with other users. A pinh gesture is use to pik up igitl ontent in the smrt phone. The ontent is then roppe y relesing the pinh gesture, whih triggers the sen ommn. CONCLUSION This stuy presente CylopsRing, whih is the first finger ring evie tht supports oth whole-hn n ontext-wre intertions. Hn gesture reognition experiments showe 84.75% ury in reognizing seven hn gestures ross

7 e f Figure 14. Exmples of pplitions of CylopsRing for gesturl intertion in virtul relity, hn se input for smrt evies n vrious ontextwre intertions. 15 prtiipnts. Built upon gesture reognition, this work emonstrte set of hn-se intertions inluing onfinger slier input, in-ir pinh-n-motion input, n plmwriting input, n their intertions with the environmentl ontext. In future works, we will investigte the use of RDF methos for ontinuous trking of hn skeletons. Aitionl ojetives inlue using miro lens tehnology to miniturize CylopsRing n inresing the fiel of view y ommoting multiple miniturize wie-ngle lens in the esign for single finger ring. ACKNOWLEDGMENTS This work ws supporte in prt y Ministry of Siene n Tehnology, Tiwn, Ntionl Tiwn University n Intel Corportion uner Grnts MOST I , MOST E , MOST E MY3, NTU-ICRP-104R7501, NTU-ICRP-104R7501-1, n lso JST ACCEL Emoie Mei Projet. REFERENCES 1. Billy, G., Müller, J., Rohs, M., Wigor, D., n Krtz, S. ShoeSense: A new perspetive on gesturl intertion n werle pplitions. In Pro. ACM CHI 12 (2012), By, H., Tuytelrs, T., n Vn Gool, L. Surf: Speee up roust fetures. In Computer Vision ECCV 2006, vol of Leture Notes in Computer Siene. 2006, Chn, L., Hsieh, C.-H., Chen, Y.-L., Yng, S., Hung, D.-Y., Ling, R.-H., n Chen, B.-Y. Cylops: Werle n single-piee full-oy gesture input evies. In Pro. ACM CHI 15 (2015), Chn, L., Ling, R.-H., Tsi, M.-C., Cheng, K.-Y., Su, C.-H., Chen, M. Y., Cheng, W.-H., n Chen, B.-Y. FingerP: Privte n sutle intertion using fingertips. In Pro. ACM UIST 13 (2013), Chen, K.-Y., Lyons, K., White, S., n Ptel, S. utrk: 3D input using two mgneti sensors. In Pro. ACM UIST 13 (2013), Colço, A., Kirmni, A., Yng, H. S., Gong, N.-W., Shmnt, C., n Goyl, V. K. Mime: Compt, low power 3D gesture sensing for intertion with he mounte isplys. In Pro. ACM UIST 13 (2013), Criminisi, A., n Shotton, J. Deision Forests for Computer Vision n Meil Imge Anlysis. Springer, Dementyev, A., n Priso, J. A. WristFlex: Low-power gesture input with wrist-worn pressure sensors. In Pro. ACM UIST 14 (2014), Fil, M. ARTg, fiuil mrker system using igitl tehniques. In Pro. IEEE CVPR 05, vol. 2 (June 2005), vol Fukui, R., Wtne, M., Gyot, T., Shimosk, M., n Sto, T. Hn shpe lssifition with wrist ontour sensor: Development of prototype evie. In Pro. UiComp 11 (2011), Gustfson, S., Bierwirth, D., n Buish, P. Imginry interfes: Sptil intertion with empty hns n without visul feek. In Pro. ACM UIST 10 (2010), Hrrison, C., Benko, H., n Wilson, A. D. OmniTouh: Werle multitouh intertion everywhere. In Pro. ACM UIST 11 (2011), Hrrison, C., n Huson, S. E. Arr: Wireless, high-preision, n unpowere finger input for very smll moile evies. In Pro. ACM UIST 09 (2009), Hrrison, C., Tn, D., n Morris, D. Skinput: Appropriting the oy s n input surfe. In Pro. ACM CHI 10 (2010), Jing, L., Cheng, Z., Zhou, Y., Wng, J., n Hung, T. Mgi ring: A self-ontine gesture input evie on finger. In Pro. ACM MUM 13 (2013), 39:1 39: Kienzle, W., n Hinkley, K. Lightring: Alwys-ville 2D input on ny surfe. In Pro. ACM UIST 14 (2014),

8 17. Kim, D., Hilliges, O., Izi, S., Butler, A. D., Chen, J., Oikonomiis, I., n Olivier, P. Digits: Freehn 3D intertions nywhere using wrist-worn gloveless sensor. In Pro. ACM UIST 12 (2012), Lolir, Christin, G. S., n Buish, P. Pinhwth: A werle evie for one-hne mirointertions. 19. Mistry, P., n Mes, P. SixthSense: A werle gesturl interfe. In Pro. ACM SIGGRAPH Asi 09 Skethes (2009), 11:1 11: Mistry, P., Mes, P., n Chng, L. WUW - wer ur worl: A werle gesturl interfe. In Pro. ACM CHI EA 09 (2009), Mujiiy, A., Co, X., Tn, D. S., Morris, D., Ptel, S. N., n Rekimoto, J. The soun of touh: On-oy touh n gesture sensing se on trnserml ultrsoun propgtion. In Pro. ACM ITS 13 (2013), Nnykkr, S., Shilkrot, R., Yeo, K. P., n Mes, P. Eyering: A finger-worn input evie for semless intertions with our surrounings. In Pro. ACM AH 13 (2013), Ogt, M., Sugiur, Y., Osw, H., n Imi, M. iring: Intelligent ring using infrre refletion. In Pro. ACM UIST 12 (2012), Prätorius, M., Vlkov, D., Burgher, U., n Hinrihs, K. DigiTp: An eyes-free VR/AR symoli input evie. In Pro. ACM VRST 14 (2014), Spons, T. S., Tn, D. S., Morris, D., Blkrishnn, R., Turner, J., n Lny, J. A. Enling lwys-ville input with musle-omputer interfes. In Pro. ACM UIST 09 (2009), Sto, M., Poupyrev, I., n Hrrison, C. Touhé;: Enhning touh intertion on humns, sreens, liquis, n everyy ojets. In Pro. ACM CHI 12 (2012), Shotton, J., Fitzgion, A., Cook, M., Shrp, T., Finohio, M., Moore, R., Kipmn, A., n Blke, A. Rel-time humn pose reognition in prts from single epth imges. In Pro. IEEE CVPR 11 (2011), Song, J., Sörös, G., Pee, F., Fnello, S. R., Izi, S., Keskin, C., n Hilliges, O. In-ir gestures roun unmoifie moile evies. In Pro. ACM UIST 14 (2014), Tmki, E., Miyki, T., n Rekimoto, J. Briny hn: An er-worn hn gesture intertion evie. In Pro. ACM CHI EA 09 (2009), Tylor, S., Keskin, C., Hilliges, O., Izi, S., n Helmes, J. Type-hover-swipe in 96 ytes: A motion sensing mehnil keyor. In Pro. ACM CHI 14 (2014), Viol, P., n Jones, M. Rpi ojet etetion using ooste se of simple fetures. In Pro. IEEE CVPR 01 (2001), Yng, X.-D., Hsn, K., Brue, N., n Irni, P. Surroun-see: Enling peripherl vision on smrtphones uring tive use. In Pro. ACM UIST 13 (2013).