Fault Estimation and Accommodation for Virtual Sensor Bias Fault in Image-Based Visual Servoing using Particle Filter

Transcription

1 IEEE Transactons on Industral Informatcs Fault Estmaton and Accommodaton for Vrtual Sensor Bas Fault n Image-Based Vsual Servong usng Partcle Flter Men Van, Shuz Sam Ge, Fellow, IEEE, Darusz Ceglarek, Senor Member, IEEE Abstract Ths study develops a fault estmaton and accommodaton scheme for the mage-based vsual servong (IBVS) system to elmnate the effects of the faults due to the mage feature extracton task, whch s named as bas vrtual sensor fault. Frst, a bas vrtual sensor fault n vsual servong s declared. Then, fault dagnoss (FD), whch ncludes fault detecton, solaton and estmaton, s desgned based on the means of partcle flter (PF). Fnally, a fault accommodaton law s developed based on the nformaton obtaned from the fault estmaton to compensate for the effects of the fault n the system. The proposed fault estmaton and accommodaton s verfed through smulaton and expermental studes, and the results show that the system can estmate and elmnate the unknown fault effects effectvely. Index Terms Fault dagnoss, Fault tolerant control, Imagebased vsual servong, Partcle flter, Robot control. I I. INTRODUCTION MAGE BASED VISUAL SERVOING (IBVS) has been proven as an effectve method for the robotc system guded by vsual nformaton due to ts easy n mplementaton and hgh accuracy []. There are ncreasng numbers of applcatons whch can sgnfcantly beneft by usng the IBVS approaches to extract poston of the geometrc feature/target whch has nherent error. For example, robotc laser weldng process guded by real-tme seam trackng or edge detecton (weldng on the fly) []; or applcaton of laser or whte lght scanners used for n-process or n-lne D parts geometry nspectons []. However, the current vsual servong approaches encounter some lmtatons as dscussed below. In tradtonal vsual servong approaches, the mage features are defned based on the geometrc characterstc of the object such as Manuscrpt receved Aug, 6; revsed Dec 8, 6, Feb 8, 7 and May, 7; accepted June, 7. Ths work was partly n collaboraton wth Computatonal Intellgent and Applcatons Research Group at Nottngham Trent Unversty. (Correspondng author: Men Van). Men Van s wth the Scence and Technology, Nottngham Trent Unversty, Nottngham, Unted Kngdom (e-mal: men.van@ntu.ac.uk, vanmen@gmal.com), Orcd ID: S. S. Ge s wth the Department of Electrcal and Computer Engneerng and Socal Robotcs Laboratory, Natonal Unversty of Sngapore, 758, Sngapore (e-mal: samge@nus.edu.sg). Darusz Ceglarek s wth the WMG, Unversty of Warwck, CV7AL, Unted Kngdom (e-mal: D. J. Ceglarek@warwck.ac.uk). ponts, ellpses, straght lnes, or segments, etc [-5]. The control law s calculated based on the dsplacement of the desgned mage features durng vsual servong. Based on ths prncple, the robot tracks the object precsely when the dsplacements of all the desgned mage features are precsely dentfed. However, ths major task has sometme been faled due to the effects of the complex envronment durng vsual servong [5-6]. In general, the falures can be caused by: () mage sngulartes [6-7], () nadequate feld of vew (FOV) of camera [8-9]: due to the vsblty constrant of the camera, some features may go out of FOV of camera durng vsual servong, and () the envronment noses: due to the change of the envronment such as lght condton, obstacles durng vsual servong, some desgned mage features could be occluded, or some undesred mage features could be appearng. In order to dscard the mage sngulartes, effectve vsual features such as polar features [6] or moment [7] have been proposed. To avod the loss of features due to the vsblty constrant, numerous publshed lteratures have been developed to ncrease the FOV of camera [8-]. The nnovatons of these methods have been thoroughly revewed n []. To reduce the collson wth obstacle, surface laser scannng has been developed []. Although these approaches can effectvely avod the falures due to the lack of FOV of camera, collson wth obstacle and mage sngulartes, the falure of the mage feature extracton task due to the mage noses has not been consdered yet; n fact ths falure scenaro s usually occurred n real applcatons. For example, the bas fault due to mage nose n seam extracton of robotc weldng system as llustrated n Fg. : the demand of the feature extracton task n seam extracton for V-groove type s to dentfy the three mage feature ponts, as shown n Fg. ; however, due to the smlar property of the desred feature pont and the nose feature pont, the system extract the nose mage feature pont nstead of the desred feature pont, and thus the dsplacement of the desgned feature wll be calculated ncorrectly, as shown n Fg.. In order to montor the falure due to the feature extracton task, fault dagnoss observer based on Kalman flter has been developed [, ]. In ths approach, a nonlnear dynamc model of the vsual servong, n whch the camera velocty s defned as the nput and the dsplacements of the feature ponts are defned as the output, was nvestgated. Based on the defned dynamc 55-6 IEEE. Personal use s permtted, but republcaton/redstrbuton requres IEEE permsson. See for more nformaton.

2 model of the vsual servong, the falure of the feature extracton tasks can be consdered as the vrtual sensor faults [, ]. Then, a fault detecton and solaton scheme was establshed based on the Kalman flter. However, ths approach has not yet consdered the fault estmaton, whch s a crucal task to dentfy the severty level of the fault. In addton, the approach has not yet nvestgated for the fault accommodaton, whch s extremely desred n real applcatons to compensate for the effect of the fault to guarantee that the system can guarantee the desred performance even n the presence of fault. In ths paper, as a second part of our prevous approach [], we nvestgate a fault dagnoss scheme, whch ncludes fault detecton, solaton and estmaton, for IBVS. The fault dagnoss observer s desgned based on Partcle flter (PF). The PF s employed because t has a good capablty to handle nonlnear and non-gaussan models, as well as t s robust and flexble compared to Kalman flter or other flters [5-8]. After a fault s dagnosed, t s desred that the controller should be reconfgured to reduce the effect of the fault [9- ]. Ths task s known as fault accommodaton, or fault tolerant control (FTC). Generally, there are two ways to compensate for the effects of a fault n the system []: () Passvely, FTC s desgned based on the assumpton that the set of possble system faults can be predcted n advance, and a fxed control law s desgned based on the predcted fault for both normal and fault operatons []. However, the pror knowledge of the possble system fault s dffcult to be obtaned n vsual servong system snce the level of nose of the system s dffcult to get n advance. () Actvely, namely actve FTC (AFTC), the control law s adjusted based on the fault nformaton, whch s obtaned from a fault dagnoss observer scheme [-]. The operaton of the AFTC conssts of two stages. In the frst stage, a FD observer s desgned to estmate the system faults onlne. In the second stage, the system uses the obtaned fault nformaton to reconfgure the control law. Compared to the passve FTC, the actve approach has a better performance when the magntude of fault s correctly estmated, and thus s desred n real applcatons. In summary, the contrbuton of ths paper can be marked by the followng sgnfcant ponts: Vrtual sensor fault n vsual servong system s broadly revewed. A FD observer s desgned to detect, solate, and estmate the severty of vrtual sensor fault, based on Partcle flter. An AFTC control law s developed to compensate the effect of faults n the system. The rest of ths paper s organzed as follows. In secton II, problem formulaton s stated. In secton III, FD and FTC strateges based on Partcle flter are presented. In secton IV, we verfy the proposed methodologes based on smulaton study. The performance of the proposed strateges s further verfed through expermental study detaled n secton V. The concluson and future work are provded n secton VI. II. PROBLEM FORMULATION Consderng the pnhole perspectve model of camera used n vsual servong system [, ], the projecton of the D T ponts P x, y, z,,..., n nto the mage plane of the T camera s s u, v,,..., n, where []: u x s = () v z y where s the focal length of the camera. The relatonshp between the dsplacement of the feature pont n the mage plane s and the spatal velocty of camera Vc [ x, y, z, x, y, z ] T can be expressed as: s L s, z Vc, where,,,..., n () L s, z L s z s the nteracton matrx, and u uv ( u ) v z z v ( v ) uv u z z z s the depth of the mage features and s assumed to be known []. The target of the control law n vsual servong system s to mnmze the error e, whch s defned as the dfferent between the current feature pont s and ther goal value s, * e s - s + Vc ˆL s e. The tradtonal control law s desgned as () where s a postve gan and pseudonverse of L s [ L, L,.. Ln ] T. Dscretzaton s appled to () []: =A +B + + k+ k k k k k k =C k +ζk + ˆL s s an approxmaton of T where s u, v,..., un, vn denotes the state varable of the system, KVc s the control nput, where K s the samplng tme, k s used to represent the system uncertanty, whch s defned as the error when dscretzng () to () [], and ζ are the model uncertanty and measurement nose, respectvely. The coeffcent matrx are defned as T A=I n, n, B= Ls L, L..., Ln n,6 (5) C=I n, n (6) When a vrtual sensor fault occurs, the true value of the desgned mage feature may not be determned. Ths means that the nput sgnal s measured as s ( t) s( t) s( t), where s( t ) s the true sgnal and s( t) s the fault sgnal. In the presence of a vrtual sensor bas fault, the system dynamcs s changed to k+=a k +B k k + k + k (7) k =C( k + ( t))+ζk ()

3 c) Fg.. Sensor bas fault n weld seam extracton of a weldng robot system. normal operaton, vrtual sensor bas fault, c) weldng robot system. []. where the fault functon s defned by, t Ts ( t) (8) s( t), t Ts where T s s the tme that the fault occurs. The objectve of ths paper s twofold: () desgn a fault dagnoss scheme based on Partcle flter to detect, solate, and estmate the unknown fault ( t), and () desgn a fault accommodaton scheme such that the vsual servong can selfcompensate the effect of faults and contnue to work relably wth an acceptable performance even though the faults stll exst n the system. III. FAULT DIAGNOSIS AND FAULT TOLERANT CONTROL BASED ON PARTICLE FILTER Based on the property of the faults, fault dagnoss problem can be categorzed nto three major tasks [9-]: ) Fault detecton: makes a bnary decson whether and when any abnormal event n the montored system happens, or f everythng works well, ) Fault solaton: dentfes the root of the fault, ) Fault dentfcaton or fault estmaton: specfes the magntude of the fault. In the followng the Partcle flterbased fault dagnoss s presented. A. Partcle Flter Consder the dynamc system of nterest s descrbed by k f ( k, k) (9) k h( k,ζ k ) where k s the state varable, k s the measurement. The system and measurement noses k and ζ k are assumed to be ndependent of k. However, unlke the Kalman flter, they need not be Gaussan dstrbuted. Snce the vsual servong system descrbed n () may not be a Gaussan dstrbuted system due to the uncertanty, the Partcle flter would be effectve to approxmate the system states. From a Bayesan perspectve, the problem of the state estmaton s to calculate the probablty densty functon (pdf) p( k : k ) of the state k based on the sensor data avalable up to tme k,,,..., : k k. Startng from the values of the ntal condton p( ) p( ) and the pdf, p( k : k ) at tme k, there are two steps to update the pdf at the tme k, p( k : k ). ) Predcton step: p( k : k ) p( k k) p( k : k) d k () ) Update step: p( k k ) p( k : k) p( k : k ) () p( k : k ) where p( k : k ) s a normalzng factor that depends on the pdf p( k k ). In theory, the Bayesan flter can estmate the true state varable by usng the two above recursve steps () and (). However, the approach can only gve optmal soluton f the system can satsfy two assumptons: the noses are Gaussan dstrbuton and the system s lnear. However, these assumptons are not usually satsfed n real applcatons. To overcome the lmtaton, Partcle flter, whch s an approxmaton method of Bayesan flter, has been proposed. The PF approxmates the pdf usng a set of N partcles, { k, k }, where k presents the th partcle and k presents ts assocated weghts. In lterature, many algorthms have been developed for the Partcle flter. In ths paper, we use sequental mportance resamplng (SIR) [5] due to ts effcent and smple mplementaton. In the followng, the structure of the SIR algorthm s presented. SIR Algorthm [5]. ) For =,, N, a new partcle k s generated based on the pdf k k as k p k k p and the correspondng weght s computed. N ) Compute the sum of weghts w k and then normalze the partcle weghts: k w k. ) Do a samplng process:.. Start from c, construct the cumulatve sum of weghts (CSW) by computng c c k for,..., N... Gve and generate a startng pont from the unform dstrbuton U, N... For j,..., N Make j N ( j ). Whle j c make. Assgn kj k. j Assgn k N

4 B. Fault Detecton and Isolaton Based On Partcle Flter The PF estmate output for the dynamc model descrbed n (7) s: ˆ =C ˆ () where ˆ s the PF state estmaton outputs, whch are determned as the output of the SIR algorthm appled for the system (7). In fault dagnoss task, t s crucal sgnfcant to choose the effectve resdual, whch can be used to easly dstngush between normal condton and fault condton when the system changng from a normal operaton to a fault operaton, and the correspondng threshold. In ths paper, the error e, whch s defned as n (), s chosen as the resdual. e -ˆ () In normal operaton, the Partcle flter state tends to approxmate the state varable of the system. Thus, from (7) and (), the resdual e tends to approxmate the system uncertantes and noses, e, where ζ. Assumpton : the system uncertantes and noses are bounded by eth, where e Th s a known constant. The assumpton s reasonable n real applcaton because the nose value s usually bounded by a constant. In practce, the bound value of the system uncertanty and nose are usually obtaned by experments. Snce e when the system n normal operaton, the bound value of the uncertanty and noses,, can be estmated based on the bound value of the error e. In ths paper, we employ ths method. The procedure to obtan the bound value s performed offlne, and s as follows. Frstly, a desred mage s obtaned by movng the robot to the target poston and capturng the desred mage. Then, startng from an arbtrary poston, but guarantee that the object s placed wthn the FOV of camera, the robot s commanded to track the object usng the control law (). The fault dagnoss observer based on Partcle flter s employed and the resdual e s obtaned when the system n normal operaton. As shown n Fgs. and (wll be dscussed latter), the resdual e converges close to zero wth small varaton due to the noses and uncertantes. The bound value e Th s selected such that t s bgger than the peak value of the varaton. Robustness property: the robustness property of the fault detecton scheme s to prevent a false alarm due to the effects of the system uncertantes and noses pror to the fault occurrng. Snce e eth when the system n normal operaton, by choosng e Th as the threshold, the robustness of the fault detecton can be guaranteed. Fault decson s made when the resdual, ( e ), surpass ts correspondng threshold e Th. From (), the change of the state varables u or v can be represented by the change of s. Thus, n order to facltate the process of fault detecton and solaton of a feature pont, the resduals of two state varables u and v should be represented by s, as follows: TABLE I FAULT-SIGNATURE TABLE Fault r r r r r n None Sen. Sen. Sen. Sen. Sen. n Fg.. Fault dagnoss and fault tolerant control scheme for vsual servong system. es e e u v () where e u and e v represent the PF estmaton errors of the state varables u and v of the feature pont, respectvely, and e s s used to represent the PF estmaton error of the feature pont. Then, the decson rule s defned as f es Th r (5) f es Th where Th e Th e u Th, where v e u Th u and e v Th v, s a chosen threshold. The robustness property of the fault dagnoss system s guaranteed and can be explaned as follows: when the system n normal operaton, the resdual s approxmated as es, and based on the assumpton, the resdual s always smaller than the selected threshold value,.e, es Th and r. However, when a fault occurs, the resdual s approxmated as es. Ths resdual sgnal wll overshoots the threshold value es Th and r, the fault decson wll be made. Fault detecton and solaton rules are defned as n Table I. In summary, fault detecton system for a gven vsual servong system s confgured and calbrated based on the followng steps: Step : Identfy the system uncertantes and noses,, by an offlne experment procedure explaned above. Step : Determne the bound value, e Th, of the system uncertanty and nose.

5 Startng ponts End ponts KF UKF PF v (pxels) 6 s s s s Error Cartesan velocty Feature error (pxel) u (pxels) v x v y v z x z y u u u u v v v v c) Fg.. Trackng performance of smulated vsual servong when the system n normal operaton. Image space, control nputs (noted that x y n the fgure), c) mage error Fg.. Resdual values and the selected thresholds when the system n normal operaton. Error Error Error Error Step : Set the bound value e Th as the threshold. Step : Make a fault decson f the resdual, e, surpass ts correspondng threshold e Th. Remark : Table I s also used to defne multple faults workng condtons. For example, when faults occur n the sensors, and at the same tme, the correspondng resduals are r, r and r. Remark : The threshold value s chosen as the same as the bound value of the system uncertantes and noses. Therefore, for a gven vsual servong system, the threshold value should Error Error Error Error Error c) Error d) Error Fg. 5. Comparson between Kalman flter (KF), unscented Kalman flter (UKF) and Partcle flter (PF). be calbrated based on the level of the system uncertantes and noses: the bgger the system uncertantes and noses, the bgger threshold need to be selected. However, t s necessary to obtan the precse bound value of the system uncertantes and noses pror to mplementng the fault dagnoss system. C. Fault Estmaton In the prevous secton, the analyses show that the PF approxmates the mage feature states wth a very small error when the system n normal operaton, es. However, when a fault occurs at the tme T s, the estmaton error e s tends to approxmate the fault component, es. Because s usually much smaller compared to, the estmaton error approxmates the fault magntude, es. Thus, the fault magntude can be approxmated as

6 Startng ponts End Ponts e u 5 Fault magntude: X:, Y=8.8 KF UKF PF v (pxels) Cartesan velocty 6 8 s s s s 6 8 u (pxels) v x v y v z x z y Fg. 6. Trackng performance of smulated vsual servong when the faults n the ponts, and occur. Image space, control nputs. Error Error Error Error Fg. 7. Resdual values when the faults n the ponts, and occur. Δs( ) t tts e (6) tts However, Ts s an unknown tme. It can only be predcted by usng the nformaton obtaned from the fault detecton and solaton scheme. Therefore, f we denote T d as the tme when the fault s detected, fault magntude can be estmated as Δs( t) tt ett (7) s d where et T d denotes the PF estmaton error at the tme T d. In practce, f the fault detecton and solaton scheme works well, we wll have Td Ts. D. Fault Tolerant Control After a fault s dagnosed, t s desred that the fault should be compensated to reduce ts effects n the system. In the tradtonal vsual servong, the system s controlled by the conventonal law (). The desred system performance s satsfed only when t operates n normal condton, wheren the system gets the feedback from the correct mage feature nput s. However, when a fault occurs, the fault feature value s ( t) s( t) s( t) s used as the nput sgnal to the controller e v e u e v e u e v Fault magntude: X:, Y= Fault magntude: X:, Y:6. Fault magntude: X:, Y: Fault magntude: X:, Y:97.8 Fault magntude: X:, Y: c) Fg. 8. KF, UKF and PF state estmaton errors when the faults n the ponts, and occur. KF, UKF and PF estmaton errors for the feature pont, KF, UKF and PF estmaton errors for the feature pont, and c) KF, UKF and PF estmaton errors for the feature pont. that wll generate ncorrect control nput and decrease the trackng performance consequently. In order to ncrease the system performance, the correct mage feature value s( t ) at the tme T s should be reconstructed and fed back to the controller nstead of the fault value s ( t ). The correct feature value can be smply calculated as s( t) s ( t) s( t). However, snce the correct fault value s( t) cannot be calculated, ts estmaton value, whch s obtaned from the fault estmaton scheme (7), s used nstead. Then, the value of the correct mage feature at the tme T s, sˆ( t ), can be estmated as: sˆ( t) s ( t) ett d (8) Afterward, when a fault s detected, to reduce ts effect on the system, the controller s reconfgured as ˆL + ( ( ) e s * V s t ) (9) c s tt d Fnally, the whole FTC law for the vsual servong system s desgned as:

7 P F TABLE II COMPARISON BETWEEN KALMAN FILTER, UNSCENTED KALMAN FILTER (UKF) AND PARTICLE FILTERS (PFS) Method Pont Pont Pont Pont Computaton RMSE VAR RMSE VAR RMSE VAR RMSE VAR tme KF UKF N= N= ## * ˆL s ( s s ) t Td Vc () + * ˆL s ( s ( t) e s ) t Td The overall FD and FTC schemes developed n ths paper are llustrated n Fg.. IV. SIMULATION STUDY In ths secton, the performances of the vsual servong system wth and wthout FTC are smulated to show the effectveness of the FD and FTC schemes. The target used n ths smulaton s masked by four feature ponts. The mage resoluton s x pxel. The samplng tme s 5 frameper-second (fps). The number of partcles s set as N=5; ths value s chosen based on the tral and error valdaton through several experments. Eght nternal states, u, v, u, v, u, v and u, v are approxmated by the PF. Startng from the ntal camera locaton, where the four ponts can be seen by dashed lnes n the mage space n Fg., the target of the vsual servong system s to locate the camera at the poston such that the four ponts can be seen by dot-dashed lnes n the mage space n Fg.. In order to compare the trackng performance of the system among normal operaton, fault operaton wthout FTC and fault operaton wth FTC, the vsual servong system s modeled n three dfferent workng condtons. In the frst case, the vsual servong system s modeled to operate n normal condton. In the second case, a multple faults condton s generated to the system wthout FTC to llustrate both the sngle and multple faults effects. In the thrd case, the proposed FTC control law s employed to reduce the effects of faults generated n the second case. A. Vsual Servong System n Normal Operaton Consderng the operaton of the system n normal operaton, as shown n Fg., the PF approxmates the nonlnear vsual servong system wth a small error due to the uncertantes and nose,, as shown n Fg.. We can see from Fg. that the camera tracks the object very well. From Fg., the PF estmaton errors converge close to zero very quck (after a few teratons). To dstngush between the effects of the uncertantes and faults, the threshold values Th are selected as the red lne, as shown n Fg.. To further evaluate the performance of the PF to approxmate the system states, we smulate the system wth the measurement nose 5. In addton, we compare the performance of the PF wth dfferent number of partcles used wth Kalman flter (KF) and unscented Kalman flter (UKF). The approxmaton errors of KF, UKF and PFs are shown n Fg. 5. For easy n comparson, the root mean square error v (pxels) Cartesan velocty 6 8 Startng ponts End ponts 6 8 u (pxels) v x v y v z x z y Fg. 9. Trackng performance of smulated vsual servong when the faults exsted n feature ponts, and wth FTC. Image space, control nputs (noted that x y n the fgure). (RMSE) and standard devaton (STD) and the computaton tme of these methods are also reported n Table II. From the results we can see that the PF flters provde a lower RMSE and STD than the KF and UKF. On the other hand, the used of hgher number of partcle provdes a better performance: the performance of the PF wth N=5 s better than the PF wth the lower number partcles (N=). However, there s a tradeoff between the approxmaton capablty and the computaton tme of the PF. The hgher number of partcles the better approxmaton performance but hgher computaton tme, and vce versa. Snce the PF generated the resdual smaller than KF and UKF, the threshold for the system usng PF can be set as a smaller value compared to the use of KF or UKF. The lower threshold value has several advantages such as reducng the detecton tme and ncreasng the senstvty of the fault detecton system []. However, accordng to the Table I, the computaton tme of the PFs s much hgher compared to the KF and UKF. Fortunately, ths stll guarantees the real tme computaton of vsual servong system. B. Vsual Servong System wth Assumed Vrtual Sensor Faults In order to show the effects of vrtual sensor bas fault n the vsual servong system and to verfy the performance of the developed FD and FTC schemes, we generate a bas fault to the vsual servong system. Partcularly, we smulate the

8 system wth a multple faults condton, s [, ], s [5, 5] and s [,]. The three faults are assumed to be occurred at the same tme at the teraton. Fgure 6 llustrates the varaton of the system performance when the system changes from normal operaton to fault operaton. Comparson results between Fg. and Fg. 6 show that the moton of the camera s ncorrect when the dsplacement of a feature s ncorrectly extracted. In partcular, due to the effect of the vrtual sensor faults, the correspondng velocty control nput s dscontnuous at the teraton, as shown n Fg. 6, that wll make the vsual servong system unstable. Due to the effect of the fault, the convergence of the PF s broken, as shown n Fg. 7. The resduals of the feature ponts, and overshoot the correspondng thresholds at the teraton, ndcatng that the faults are exsted n the vrtual sensors, and. Thus, n ths experment, the system has detected and solated the faults successfully. In the next, we consder fault estmaton performance. At the teraton, where the exstng faults n the feature ponts, and were detected and solated successfully, fault estmatons were then calculated by usng the formulaton defned n (7). Fg. 8 shows the fault estmaton results usng the KF, UKF and PF. Accordng to (7) and Fg. 8, the computed fault estmatons usng PF are ˆ s e [8.8, 7.7] for t T d ˆ s e tt d [6.,.8] ˆ s e tt d [97.8,.] ˆ s, ˆ s the vrtual sensor, for the vrtual sensor, and for the vrtual sensor. The estmated fault values, and ˆ s, are very close to the assumed fault values, s, s and s. In addton, the comparson results shown n Fg. 8 verfy that the PF provdes better fault estmaton compared to the KF and UKF. Thus, from the comparson results shown n Fg. 5 and Fg. 8, we can verfy that the PF gves hgher performance than the KF and UKF for both fault detecton and estmaton. C. Vsual Servong System Wth Fault Tolerant Control As shown n Fg. 5, the effects of the faults generate a dscontnuous control nput, and consequently break the stablty of the system. To reduce the effects of the faults, the developed FTC law n () s employed based on the feedback nformaton of the estmated fault usng PF obtaned n Fg. 8. The performance of the developed FTC for the vsual servong s shown n Fg. 9. Comparson results between Fg. 9 and Fgs. and 5 show that the FTC system compensates the effects of the faults very well. The system performance of the vsual servong under FTC s comparable to the normal operaton. The velocty control nput of the system wth FTC s contnuous, as shown n Fg. 9. Thus, we can conclude that the fault has been accommodated successfully. Remark : If we consder the sensor fault only and assume that the actuator (camera moton) s always healthy, the proposed fault dagnoss and accommodaton can detect and compensate for the heavy sensor fault case, where all the sensors are faled at the same tme. However, f we consder Fg.. Experment setup of eye-n-hand vsual servong. v (pxels) Cartesan Velocty s s s s Startng ponts End Ponts 5 6 u (pxels) V x V y V z x y z - 5 Fg.. Trackng performance of expermented vsual servong when the system n normal operaton. Image space, control nputs. both the actuator and sensor fault scenaros, the control system wll solate a wrong actuator fault nstead of sensor faults f all the sensors are faled at the same tme []. Fortunately, ths stuaton s rarely occurrng n real applcatons. Remark : There are many actve fault tolerant approaches publshed n the lterature []. Among them, the approach, whch uses a nomnal controller plus a fault compensator, whch s taken from a fault estmaton scheme, s mostly appled because t s smple n desgn and effectve n fault compensaton [, ]. Ths paper follows ths desgn procedure. It s obvous to see that the performance of ths FTC strategy s manly dependent on the performance of the fault estmaton. Therefore, nstead of comparng the performance among AFTC approaches, we compare the performance of fault estmaton usng KF, UKF and PF. The results shown that the PF provdes very accurate fault estmaton, and thus the developed AFTC would be effectve compared to other FTC approaches. V. EXPERIMENTAL STUDY In order to show the trackng performance of the system wth FD and FTC, a lab expermental setup s developed as

9 Fg.. Resdual values of expermented vsual servong and the selected threshold values. Error Error Error Error Fg. 5. PF estmaton errors when the expermented vsual servong sytem n normal operaton. Error Error Error Error 5 Startng ponts End Ponts Fg.. Image feature n normal extracton, vrtual sensor bas fault. v (pxels) Catersan velocty s s s s Startng ponts End Ponts 5 6 u (pxels).5 Effect of fault -.5 V x V y V z x y z - 5 Fg.. Trackng performance of vsual servong when the fault exsted n the vrtual sensor (feature pont ) wthout FTC. Image space, control nputs. shown n Fg.. The Baxter ndustral robot [5] s used to perform experment. The Baxter s a new generaton ndustral robot and has been wdely usng n ndustral applcaton and research. The Baxter has two ndependent arms and each has seven degree-of-freedom (DOF). Each arm was attached wth an eye-n-hand confguraton. In ths study, we used the leftarm and left-hand camera to do experments. The object to be tracked ncludes four feature ponts, as shown n Fg.. The camera capturng rate s fps (frame/s), and the mage nformaton s sent nto the host Lnux PC to processng. The camera of the Baxter robot has 6x pxels resoluton and has an effectve focal length of. mm. v (pxels) Cartesan Velocty s s s s 5 6 u (pxels).5 The effect of fault s reduced -.5 V x V y V z x y z - 5 Fg. 6. Trackng performance of the expermented vsual servong system when the fault exsted n the feature pont wth FTC. Image space, control nputs. Faults are ntroduced n the vrtual sensor by changng the dsplacements of the mage feature ponts at an arbtrary tme. In the followng, we present the performance of the vsual servong system wthout FTC and wth FTC when the system n normal and fault operatons. A. Vsual Servong n Normal Operaton In fault-free workng condton, the vsual servong system tracks the object very well, as shown n Fg.. The PF estmaton errors, whch are used as the resduals n ths paper, are shown n Fg.. The results from Fg. show that the PF estmaton errors are quckly convergent after a few teratons. As analyzed n secton IIIB, the resdual value obtaned when the system n fault-free operaton s the uncertanty and nose components,, of the system. Thus, to avod any ncorrect fault decson due to the effects of the system uncertantes and noses, the threshold values Th are selected to be bgger than the bound value of. The selected thresholds are the red lnes shown n Fg..

10 B. Vsual Servong System under Vrtual Sensor Bas Fault wthout Fault Tolerant Control In the presence of a bas sensor fault, the controller system read a nose feature value nstead of the true desgned feature, as an example shown n Fg.. To smulate a bas fault, we change the dsplacement of the feature pont at the teraton 5, as llustrated n Fg.. Fg. shows the transton of the trackng performance when changng from normal operaton to fault operaton. From Fg., due to the presence of the fault at the teraton 5, the computed control velocty nput s changed suddenly. Due to the effects of the fault, the convergence of the PF estmaton error s broken, as shown n Fg. 5. In Fg. 5, the resdual of the feature pont overshoots the correspondng threshold at the teraton 5, and thus the fault decson s made. C. Vsual Servong wth Fault Tolerant Control Under Vrtual Sensor Fault The generated bas vrtual sensor fault provdes dscontnuous control nput as shown n Fg.. To tackle ths problem, the developed FTC law n () s employed. Fg. 6 shows the results of the fault compensaton. By comparng Fg. 6 wth Fg., we can see that the effects of the fault n the computed control nput are much reduced. From ths, we can conclude that the fault has been estmated precsely and ts effects n the vsual servong system have been correctly compensated by the developed FTC law. Remark 5: The performance of the fault estmaton and accommodaton s dependent on the level of uncertanty and nose of the vsual servong system. VI. CONCLUSION Ths paper revews the falure scenaros of the feature extracton task n vsual servong system, namely vrtual sensor bas fault. Then, the PF-based FD s developed to montor the presence of the faults. An AFTC s developed based on the estmated fault nformaton to compensate for the effects of the faults. Both the desgned FD and FTC schemes have a smple structure and easy mplementaton n real applcaton. Smulaton and expermental results verfy that the presence of the falures due to the sensor bas faults can be detected accurately and ts effects can be compensated effectvely. Accordng to [], the falure of vsual servong system could be caused by ncorrect robot moton, namely actuator fault. Fault estmaton and accommodaton for the falures of actuator fault wll be nvestgated n our future work. REFERENCES [] M. Keshmr, W. F. Xe, and A. Mohebb, Augmented mage-based vsual servong of a manpulator usng acceleraton command, IEEE Trans. Ind. Electron., vol. 6, no., pp. 5-55,. [] D. Ceglarek, M. Colledan, J. Vancza, D. Y. Km, C. Marne, M. Kogel- Hollacher, A. Mstry, L. Bolognese, Rapd Deployment of Remote Laser Weldng Processes n Automotve Assembly Systems, Annals of the CIRP, vol. 6, no., pp. 89-9, 5. [] Y. Dng, P. Km, D. Ceglarek, J. Jn, Optmal Sensor Dstrbuton for Varaton Dagnoss n Mult-staton Manufacturng Processes, IEEE Trans. Robot. Auto., vol. 9, no., pp ,. [] A. Hajloo, M. Keshmn, W. F. Xe, T. T. Wang, Robust onlne model predctve control for a constraned mage based vsual servong, IEEE Trans. Ind. Electron., vol. 6, no., pp. -5, 6. [5] W. F. Xe, Z. L, X. W. Tu, C. Perron, Swtchng control of magebased vsual servong wth laser ponter n robotc manufacturng systems, IEEE Trans. Ind. Electron., vol. 56, no., pp. 5-59, 9. [6] P. Corke, F. Spndler, F. Chaumetter, "Combnng Cartesan and polar coordnates n IBVS," n Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., (IROS 9), Oct. 9, pp [7] J. Wang, H. Cho, Mcropeg and hole algnment usng mage moments based vsual servong method, IEEE Trans. Ind. Electron., vol. 5, no., pp. 86-9, 8. [8] N. G. Aracl, E. Mals, R. A. Santonja, C. P. Vdal, Contnuous vsual servong despte the changes of vsblty n mage features, IEEE Trans. Robot., vol., no., pp. 7-7,. [9] G. Ches, K. Hashmoto, D. Prattchzzo, and A. Vcno, Keepng features n the feld of vew n Eye-In-Hand vsual servong: A swtch approach, IEEE Trans. Robot., vol., no. 5, pp. 98-9,. [] M. Kazem, K. K. Gupta, M. Mehrandezh, Randomzed knodynamc plannng for robust vsual servong, IEEE Trans. Robot., vol. 9, no. 5, pp. 95-,. [] M. Baumann, S. Leonard, E. A. Croft, J. J. Lttle, Path plannng for mproved vsblty usng a probablstc road map, IEEE Trans. Robot., vol. 6, no., pp. 97-,. [] M. Van, D. Wu, S. S. Ge, H. Ren, "Fault Dagnoss n Image-Based Vsual Servong wth Eye-n-Hand Confguraton Usng Kalman Flter," IEEE Trans. Ind. Inform., vol., no. 6, pp , 6. [] L. C. B. Precado, O. Y. Sergyenko, J. C. R. Qunonex, X. Garca, V. V. Tyrase, M. R. Lopez, D. H. Balbuena, P. Mercorell, M. Podrygalo, A. Gurko, I. Tabakova, O. Starostenko, Optcal D laser measurement system for navgaton of autonomous moble robot, Optcs and Lasers n Engneerng, vol. 5, pp ,. [] M. Van, D. Wu, S. S. Ge, H. Ren, Condton montorng for mage based vsual servong usng Kalman Flter, Advances n Vsual Computng, pp. 8-85, 5. [5] M. S. Arulampalam, S. Maskell, N. Gordon, ans T. Clapp, "A tutoral on partcle flters for onlne nonlnear/non-gaussan Bayesan trackng," IEEE Trans. Sgnal Process., vol. 5, no., pp. 7-88,. [6] T. We, Y. Huang, C. L. Phlp Chen, Adaptve sensor fault detecton and dentfcaton usng partcle flter algorthms, IEEE Trans. Syst. Man Cyber., vol. 9, no., pp. -, 9. [7] N. Wdynsk, M. Mgnotte, A multscale partcle flter framework for contour detecton, IEEE Trans. Pattern. Analy. Machne. Intel., vol. 6, no., pp. 9-95,. [8] B. Zhao, R. Skjetne, M. Blanke, and F. Dukan, "Partcle flter for fault dagnoss and robust navgaton of underwater robot," IEEE Trans. Control Syst. Tech., vol., no. 6, pp. 99-7,. [9] S. Laghrouche, J. Lu, F. S. Ahmed, M. Harmouche and M. Wack, "Adaptve Second-Order Sldng Mode Observer-Based Fault Reconstructon for PEM Fuel Cell Ar-Feed System," IEEE Trans. Control Syst. Tech., vol., no., pp. 98-9, 5. [] J. Lu, W. Luo, X. Yang and L. Wu, "Actve fault tolerant control systems," IEEE Trans. Ind. Electron., vol. 6, no. 5, pp. 6-7, 6. [] M. Mahmoud, J. Jang, Y. Zhang, Actve fault tolerant control systems: stochastc analyss and synthess, Lecture Notes n Control and Informaton Scences, Sprnger Scence & Busness Meda, vol. 87,. [] M. Van, S. S. Ge, H. Ren, "Robust Fault-Tolerant Control for a Class of Second-Order Nonlnear Systems Usng an Adaptve Thrd-Order Sldng Mode Control," IEEE Trans. Systems, Man, and Cybern.: Systems, vol. 7, no., pp. -8, 7. [] M. Van, H. J. Kang, Y. S. Suh, K. S. Shn, A robust fault dagnoss and accommodaton scheme for robot manpulators, Int. J. Control Aut. Syst., vol., no., pp ,. [] M. Van, S. S. Ge, H. Ren, "Fnte Tme Fault Tolerant Control for Robot Manpulators Usng Tme Delay Estmaton and Contnuous Nonsngular Fast Termnal Sldng Mode Control," IEEE Trans. Cybern., vol.pp, no.99, pp.-, do:.9/tcyb , 6. [5] C. Ftzgerald, "Developng baxter," n Proc. Int. Conf. technol. Practcal Robot Appl., Woburn MA, USA,, pp. -6.

11 Men Van receved hs B.S. degree n Electrcal Engneerng from Danang Unversty of Technology, DaNang cty, Vetnam, n 9. He receved hs Ph.D. degree n the School of Electrcal Engneerng, Unversty of Ulsan, South Korea n 5. He was a Post-Doctoral Research Fellow wth the Advanced Robotcs Centre, Faculty of Engneerng, Natonal Unversty of Sngapore, Sngapore and the WMG, Unversty of Warwck, UK, from 5. He s currently wth the School of Scence and Technology, Nottngham Trent Unversty, UK. Hs research nterests nclude Assstve robotcs, Robot control, Robot-vson system, Fault dagnoss and fault tolerant, Machne learnng, and Sensng and Percepton. Shuzh Sam Ge (S 9 M 9 SM F 6) receved the B.Sc. degree from the Bejng Unversty of Aeronautcs and Astronautcs, Bejng, Chna, n 986, and the Ph.D. degree from the Imperal College of Scence, Technology and Medcne, Unversty of London, London, U.K., n 99. He s the Drector of Socal Robotcs Laboratory, Smart Systems Insttute, and the Professor wth the Department of Electrcal and Computer Engneerng, Natonal Unversty of Sngapore, Sngapore. He has co-authored seven books and over nternatonal journal and conference papers. Hs current research nterests nclude socal robotcs, adaptve control, ntellgent systems, and artfcal ntellgence. Prof. Ge s an Edtor-n-Chef of the Internatonal Journal of Socal Robotcs, Sprnger. He has served/been servng as an Assocate Edtor for a number of flagshp journals, ncludng the IEEE TRANSACTIONS ON AUTOMATIC CONTROL, the IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, the IEEE TRANSACTIONS ON NEURAL NETWORKS, and Automatca. He also serves as a Book Edtor of the Taylor & Francs Automaton and Control Engneerng Seres. At the IEEE Control Systems Socety, he served/serves as the Vce Presdent for Techncal Actvtes, from 9 to, the Vce Presdent of Membershp Actvtes, from to, a Member of Board of Governors of the IEEE Control Systems Socety, from 7 to 9. He s a fellow of the Internatonal Federaton of Automatc Control, the Insttute of Engneerng and Technology, and Sngapore Academy of Engneerng. Darusz Ceglarek (SM ) s EPSRC Star Research Char at WMG, Unversty of Warwck and a CIRP Fellow. Prevously, he was Professor n Industral and Systems Engneerng at Unversty of Wsconsn, Madson. He receved hs Ph.D. n Mechancal Engneerng from Unversty of Mchgan-Ann Arbor n 99. Hs research focusses on dgtal manufacturng, nprocess qualty control and root cause analyss across desgn, manufacturng and servce. Hs research has been funded by: US (NSF, NIST), Unted Kngdom (EPSRC, InnovateUK and HVM Catapult) and EU (FP7, Mare Cure) and ndustry (more than OEMs and SMEs). He has publshed over 5 papers and receved several Best Paper Awards. He has receved numerous awards ncludng 7 UK EPSRC Star Award, US NSF CAREER Award; 999 Outstandng Research Scentst Award from Unversty of Mchgan; the 998 Dell K. Allen Outstandng Young Manufacturng Engneer of the Year Award from the SME. He has served on numerous Edtoral Boards and s an Assocate Edtor (Europe) of the ASTM Smart and Sustanable Manufacturng Systems Journal. Prof. Ceglarek served as Char of the Qualty, Statstcs and Relablty Secton of INFORMS; Program Char for the ASME Desgn-for-Manufacturng Lfe Cycle Conferences, Assoc Edtor of the IEEE Trans., and of the ASME Trans, J. Manuf Sys & Eng.