Foot-Pedl: Hptic Feedbck Humn Interfce Bridging Senstionl Gp between Remote Plces Mincheol Kim 1, De-Keun Yoon 2, Shin-Young Kim 1, Ji-Hi Cho 1, Kwng-Kyu Lee 1, Bum-Je You 1,3 1 Center of Humn-centered Interction for Coexistence, Seoul, 02792, Kore (Tel : +82-2-958-7391; E-mil: bll@chic.re.kr, sini13@chic.re.kr, jhcos7@chic.re.kr, ybj@chic.re.kr) 2 Hyundi Hevy Industries, Ulsn, 682-792, Kore (Tel : +82-52-202-2114; E-mil: mnipulbility@gmil.com) 3 Center for Robotics Reserch, KIST, Seoul, 02792, Kore (Tel : +82-2-958-5760; E-mil: ybj@kist.re.kr) Abstrct Teleconferencing or teleopertion hs been used for some yers in ctul fields including mny different types of compnies, fculties, hospitls nd even in gme industries. The technology involves cpturing the user s movements nd replying it t remote loction. However, tody s technology reches to not just presenting oneself virtully t remote loction, but lso letting the user experience the remote environment without hving to ctully exist in the plce. This llows the user to feel nd mnipulte the remote environment or nother person in more intuitive mnner nd n ctul physicl interction between the user nd the remote environment possible. This pper suggests new type of humn interfce which the user cn mnipulte with his or her feet llowing the hnds to move freely during the interction. The proposed mechnism is foot-pedl type hptic feedbck humn interfce nd is equipped with 4 different sensors tht cn observe the user s movements in 4 different DOFs. In ddition, the interfce is composed of 2 independent motors which cn provide the user with the hptic feedbck from the remote environment. In this pper, vrious control lgorithms re presented which decide the ction or hptic feedbck from the interfce depending on the remote surroundings. The performnce of the proposed interfce is verified through severl experiments in different scenrios. Keywords Humn-Computer Interction, Humn Interfce, Hptics, Sensors, Motion Cpture. 1. Introduction During mny yers, there hs been introduced the concept of teleconferencing. In globl compnies where the representtives from mny countries round the world cnnot regulrly mke physicl confronttion, the most common wy of hving conference meeting ws to conduct meetings through teleconferencing [1]. Although the persons do not mke physicl ppernce t the ppointment room, they would gther round the tble telegrphiclly nd hve discussion together. On tht course, people my be using video cmers to cpture the ppernces of themselves nd present oneself to one nother. These imge sensors cn then be sid to be used for cpturing imges. However, wht if people t the meeting wnt to feel something together? If the compny serves in clothes mnufcturing field, people might wnt to feel one of the proposed products, or even try to wer it. This kind of feedbck cnnot be stisfied by mere imge sensors. Therefore, in order to enble such chllenging scenrios, n lterntive input interfce is necessry tht cn send out the user s movement informtion nd receive the environment feedbck informtion simultneously. Another common scenrio where people use foot-bsed input interfces is in VR gming industries [2, 3]. Since the user hs to control the movement of the virtul chrcter nd interct with the virtul environment t the sme time, often the user is forced to use foot-bsed interfces for movements, leving their hnds free to do something else. For exmple, in first person shooter gme, the user moves the virtul chrcter by controlling their feet, through foot-bsed input interfces, nd tkes cre of other businesses such s shooting with his/her hnds or looking round with his/her hed. If the user hd to use hnds to control the movement of the virtul chrcter s well, the sitution would be no different from using keybord, mking the gming experience less indulging. The rest of the pper will be orgnized s follows: fter the introduction, erly works on foot-bsed interfces will be reviewed in Sec. 2. A new type of interfce device is introduced nd described in Sec. 3. Corresponding experiments nd results for performnce verifiction will be explined in Sec. 4. Finlly, conclusions nd future works will be delt in Sec. 5. 2.1 Erly Works 2. Foot-Bsed Interfces Due to lrgely growing gming industries, mny different ttempts in order to control the user in the virtul environment hve been mde. Alredy, lots of demonstrtion videos re uploded on the internet, djoining the foot-bsed interfces nd mnipultion of the virtul chrcter [4]. Reltively well-known mong them re tredmill-type interfces nd joystick-type interfces. Tredmill-type interfces include hrdwre tht resembles tredmill where user wlks upon it wering specilized pir of shoes designed for the hrdwre. As the outer look
suggests, the tredmill-bsed designs hold n dvntge in tht the usge is more strightforwrd compred to other types of devices. A nturl downside would be it is esy for the user to get tired using it. One of the most well-known models of tredmill-type interfces is shown in the left of Fig. 1. Virtuix Omni [5] from Virtuix hs tredmill on the bottom of the interfce nd hs 1-DOF height sensor included t the wist. This enbles the cpturing of the user s movement in 2D, plus the height. In overll, Foot-Pedl currently consists of 2 motors nd 2 encoders. Trnsltion prt hs 1 motor nd 1 corresponding encoder, nd so does the rottion prt. Since the device fully supports 2D movement of the user, ll types of motion in 2D re plusible. The motors equipped t ech DOF re intended to produce hptic feedbck tht will enhnce the feeling of relity provided by Foot-Pedl. The overll design is depicted in Fig. 2. Fig. 1. Virtuix Omni from Virtuix (left) nd 3D Rudder from 3DRudder (right). A well-known joystick-type interfce is 3D Rudder [6] from 3DRudder which is shown in the right of Fig. 1. The user cn control the movement by tilting nd rotting the device with his/her feet. The upside of such devices is tht it requires little energy to mnipulte compred to strightforwrd tredmill-types. 2.2 Limittions Virtuix Omni s height sensor is limited in wy tht the rnge of the dt is somewht confined in smll vicinity. Therefore, it might be suitble for first person shooter gmes, but it is unsuitble for situtions where lrge displcement must be mde in ll three directions. 3D Rudder on the other hnd, is suitble for such usge. However, since the device supports only tilting nd rotting, it is not ble to distinguish motion between lrge nd smll displcements. The biggest obstcle for both types of interfces to overcome is the unvilbility of hptic feedbck. Setting side the teleconferencing of clothes mnufcturer scenrio, even in gming environment, hptic feedbck could ply n importnt role in mking n indulging experience. For instnce, if the virtul chrcter fces stedy wll, the user could receive feedbck such s not being ble to mnipulte the interfce in the forwrd direction. This will give the user more relity-bsed experience compred to trditionl one-wy communiction tht common input devices provide. In order to provide decent hptic feedbck nd cpture the movement of the user properly, new foot-bsed humn interfce clled Foot-Pedl is introduced in this study. Fig. 2. Overll design of Foot-Pedl nd the corresponding coordinte system. B. Dt Acquisition From the user, Foot-Pedl receives the movement informtion through the encoders equipped t ech DOF. Since it is not binry sign implying on/off sttus, the output signl from the encoder, which is continuous, cn be vriously interpreted. In the lter experiments, the movement of the Foot-Pedl is converted into scled vlue, which is then clculted to the velocity of object being mnipulted. C. Hptic Feedbck In ddition, Foot-Pedl is equipped with two BLDC motors tht cn generte hptic response from the environment nd control trnsltion nd rottion movements ccordingly. Since the equipment is directly in contct with humn feet, it hs series elstic ctutor (SEA) structure [7] with both ctutors, providing n elstic contct between the user nd the hrdwre. Not only does it hve n dvntge over elsticity, but this kind of structure comes lso convenient in giving hptic feedbck to the user. Figure 3 shows the detiled look of the SEA structure. Fig. 3. Series Elstic Actutor (SEA) structures. 3.1 Hrdwre A. Overll Look 3. Foot-Pedl + b x = f f (1) = f f (2) o o h f = k x x ) + b( x x ) (3) ( o o
The reltionship between the vribles follows the fundmentl physics lws regrding mss-spring-dmper systems s shown in Eq. (1, 2, 3), where x o nd q o refer to the displcement of the user-mnipulted trnsltion nd rottion prts, nd x nd q refer to the displcement of the motor-driven trnsltion nd rottion prts respectively. Also, m o, m, k nd b refer to the effective msses, stiffness nd dmping of the models respectively. f, f h nd f refer to n rtificil force in the impednce model, force exerted by the humn user nd force exerted from the mechnicl springs respectively nd likewise goes for τ s in rottion. In Foot-Pedl, there re two dditionl encoders t the trnsltion prt nd the rottion prt which cpture the displcement of the min body, x o. This wy, by clculting the pplied force from the springs due to the displcement, impednce control becomes fesible. I other words, the SEA structure plys role of force sensor in comprbly economic mnner. By providing impednce control, Foot-Pedl cn be useful interfce in bridging the senstionl gp between the user nd remote or virtul environment. 3.2 Softwre & Algorithms Aside from impednce control, Foot-Pedl hs gone through number of experiments with remotely controlled two-wheeled nvigtionl robot with dditionl lgorithms. For sfe nvigtion, we hve defined vribles nmed sfety fctors. They refer to function of the scled distnce to nerby obstcle in the corresponding direction. The distnce between the robot nd obstcles were mesured by Microsoft Kinect. More detils regrding the experiment will be discussed in Sec. 4. We simply defined the sfety fctors S trns nd S rot s proportionl gin vlues of the pplied force nd torque. Equtions (4) nd (5) show how these re formulted. From Eq. (1), the sfety fctor is multiplied to the mount of exerted force by spring s in Eq. (4). Likewise, rottion prt is pplied with the similr logic s shown in Eq. (5). reltive ngle from the center of the robot to obstcles t 30Hz. The obstcle informtion (distnce nd ngle) is trnsferred to Foot-Pedl through TCP/IP sockets t the sme speed. Bsed on the received informtion, we performed impednce control on both motors t trnsltion nd rottion in order to give hptic feedbck to the user. If n obstcle comes too close to the robot, certin impednce is pplied so tht the user could feel something residing in tht direction. In ddition, input velocity commnds to the nvigtionl robot is clculted from the encoder redings from trnsltion nd rottion motion. The commnds re trnsferred through TCP/IP sockets t 100Hz. Then the mobile robot converts the velocity commnds into motor velocities t ech wheel t 100Hz. Fig. 4. Block digrm of communiction structure. 4.2 Results nd Discussion First, we confirmed the correspondence between the input from Foot-Pedl nd the output wheel velocities of the tele-operted robot. As Figs. 5 nd 6 suggest, the motion in Foot-Pedl nd the motion of the nvigtionl robot re in sync, forming proportionl reltionship. + b x = f S f (4) trns q + b q = Srot m τ τ (5) By introducing sfety fctors, Foot-Pedl receives the obstcle informtion from the distnt robot s hptic feedbck. The user now perceives the existence of n obstcle in terms of force pplied to his/her feet. This wy, even if the user desires to proceed further into direction leding to n obstcle, Foot-Pedl will not move, leding to diminished velocity of the nvigtionl robot. 4. Experiments 4.1 Experiment Setting In order to verify the performnce of the proposed mechnism, severl experiments involving remote nvigtionl robot hve been crried out. The robot hs differentil wheel mechnism nd cn detect nerby obstcles using n RGB-D cmer. The depth mp of the RGB-D cmer is used to clculte the distnce nd Fig. 5. Trnsltionl motion by Foot-Pedl
Fig. 6. Rottionl motion by Foot-Pedl. The reltionship between the scled distnce to the obstcle nd the trnsltionl sfety fctor is shown in Eq. (6). Bsiclly, the force input is diminished to zero s the robot gets closer to n obstcle. This is demonstrted by fixing the nvigtionl robot in plce nd hnging it in the ir so tht the wheels could run free while supposed obstcle (person) is closing in nd bck out. Figure 7 depicts the experiment environment nd the corresponding grph of the reltionship between the sfety fctor nd the scled distnce. log 10 ( 99 Scled Distnce + 1) S trns = (6) 2 By mesuring the force pplied to the spring blnce, we were ble to verify the force feedbck from the environment to Foot-Pedl. Figure 8 shows s the person closing in on the robot, nd the corresponding force feedbck increses. The force feedbck from Foot-Pedl follows the inverse logrithmic function tht cuses Foot-Pedl to ct s stiff, rigid structure if n obstcle is too close nerby. Therefore, it cn be concluded tht Foot-Pedl cn be novel tool to not only cpture the user s feet movement dt through equipped sensors, but lso trnsfer hptic senstion from remote or virtul environment to the user. Fig. 7. Plot of scled distnce to obstcle vs. sfety fctor. 5. Conclusions In this study, we proposed new type of humn interfce tht supports both cpturing of the user s trnsltionl nd rottionl feet movements nd hptic feedbck to the user from the environment which fcilittes the indulging senstion. Compred to the other types of interfces in the mrket, equipped with specilized hrdwre, Foot-Pedl is unique in wy tht force control or impednce control cn be implemented. Moreover, it does not show ny downsides in performing the mnipultion of nvigtionl robot. The performnce of the suggested mechnism is verified through severl experiments involving the ctul mnipultion of robot, demonstrtion of the correltion between the sfety fctor nd the scled distnce to the obstcle, nd the force feedbck generted due to proximity of the obstcles to the robot. Our future reserch my include, developing new version of Foot-Pedl tht is lighter nd more compct thn the current version. Moreover, kinds of supported movements re to be widened to rising or lowering foot which re essentil in stepping up or down stirs. Possibly in the future, we hope to extend our reserch to using Foot-Pedl to mnipulte not only tele-presence robots, but lso virtul vtrs existing in the virtul relity. Combined with HMDs nd methodologies which support hnd movement recognition interfces, we believe tht Foot-Pedl cn be the next-genertion foot-bsed interfce
tht comes fully functionl, comptible nd convenient to its users. Pin Mngement, Cyberpsychology, Behvior, nd Socil Networking, Vol. 17, No. 6, pp. 414-422, 2014. [3] L. Avil nd M. Biley, Virtul Relity for the Msses, IEEE Computer Grphics nd Applictions, Vol. 34, No. 5, pp. 103-104, 2014. [4] https://www.youtube.com/wtch?v=c2kzrywocak, Virtuix Omni CES 2016, YouTube, 2016. [5] http://www.virtuix.com/, Virtuix Omni. [6] http://www.3drudder.com/, 3D Rudder. [7] G. A. Prtt nd M. M. Willimson, Series Elstic Actutors, Intelligent Robots nd Systems 95. 'Humn Robot Interction nd Coopertive Robots', Proceedings. 1995 IEEE/RSJ Interntionl Conference on, Vol. 1, pp. 399-406, 1995. Fig. 8. Plot of scled distnce to obstcle vs. force feedbck. Acknowledgement This work ws supported by the Globl Frontier R&D Progrm on <Humn-centered Interction for Coexistence> funded by the Ntionl Reserch Foundtion of Kore grnt funded by the Koren Government(MSIP)(2010-0029759). References [1] C. Greenhlgh nd S. Benford, MASSIVE: Collbortive Virtul Environment for Teleconferencing, ACM Trnsctions on Computer-Humn Interction (TOCHI), Vol. 2, No. 3, pp. 239-261, 1995. [2] B. K. Wiederhold, A. Soomro, G. Riv, nd M. D. Wiederhold, Future Directions: Advnces nd Implictions of Virtul Environments Designed for