MANY real-world optimization problems can be summarized. Push and Pull Search for Solving Constrained Multi-objective Optimization Problems

Transcription

1 JOURNAL OF LATEX CLASS FILES, VOL., NO. 8, AUGUST Push and Pull Search for Solving Constrained Multi-objective Optimization Problems Zhun Fan, Senior Member, IEEE, Wenji Li, Xinye Cai, Hui Li, Caimin Wei, Qingfu Zhang, Fellow, IEEE, Kalyanmoy Deb, Fellow, IEEE, and Erik D. Goodman arxiv:79.9v [cs.ne] Sep 7 Abstract This paper proposes a push and pull search (PPS) framework for solving constrained multi-objective optimization problems (CMOPs). To be more specific, the proposed PPS divides the search process into two different stages, including the push and pull search stages. In the push stage, a multiobjective evolutionary algorithm (MOEA) is adopted to explore the search space without considering any constraints, which can help to get across infeasible regions very fast and approach the unconstrained Pareto front. Furthermore, the landscape of CMOPs with constraints can be probed and estimated in the push stage, which can be utilized to conduct the parameters setting for constraint-handling approaches applied in the pull stage. Then, a constrained multi-objective evolutionary algorithm (CMOEA) equipped with an improved epsilon constraint-handling is applied to pull the infeasible individuals achieved in the push stage to the feasible and non-dominated regions. Compared with other CMOEAs, the proposed PPS method can more efficiently get across infeasible regions and converge to the feasible and nondominated regions by applying push and pull search strategies at different stages. To evaluate the performance regarding convergence and diversity, a set of benchmark CMOPs is used to test the proposed PPS and compare with other five CMOEAs, including MOEA/D-CDP, MOEA/D-SR, C-MOEA/D, MOEA/D- Epsilon and MOEA/D-IEpsilon. The comprehensive experimental results demonstrate that the proposed PPS achieves significantly better or competitive performance than the other five CMOEAs on most of the benchmark set. Index Terms Push and Pull Search, Constraint-handling Mechanisms, Constrained Multi-objective Evolutionary Algorithms. I. INTRODUCTION MANY real-world optimization problems can be summarized as optimizing a number of conflicting objectives simultaneously with a set of equality and/or inequality constraints, which are denoted as constrained multi-objective optimization problems (CMOPs). Without lose of generality, a CMOP considered in this paper can be defined as follows []: minimize subject to F(x) = ( f (x),..., f m (x)) T g i (x), i =,..., q h j (x) =, j =,..., p x R n where F(x) = ( f (x), f (x),..., f m (x)) T is an m-dimensional objective vector, and F(x) R m. g i (x) is an inequality constraint, and q is the number of inequality constraints. h j (x) = is an equality constraint, and p represents the number () of equality constraints. x R n is an n-dimensional decision vector. When solving CMOPs with both inequality and equality constraints, we usually convert the equality constraints into inequality ones by introducing an extremely small positive number ɛ. The detailed transformation is given as follows: h j (x) ɛ h j (x) () To deal with a set of constraints in CMOPs, the overall constraint violation is a widely used approach, which summarizes them into a single scalar, as follows: q p φ(x) = min(g i (x), ) + min(h j (x), ) () i= Given a solution x k R n, if φ(x k ) =, x k is a feasible solution. All the feasible solutions constitute a feasible region S, which is defined as S = {x φ(x) =, x R n }. For any two solutions x a, x b S, x a is said to dominate x b if f i (x a ) f i (x b ) for each i {,..., m} and F(x a ) F(x b ), denoted as x a x b. If there is no other solution in S dominating solution x, then x is called a Pareto optimal solution. All of the Pareto optimal solutions constitute a Pareto optimal set (PS). The mapping of the PS in the objective space is called a Pareto optimal front (), which is defined as = {F(x) x PS}. A key issue in CMOEAs is to maintain a balance the objectives and constraints. In fact, most constraint-handling mechanisms in evolutionary computation are applied for this purpose. For example, the penalty function approach adopts a penalty factor λ to maintain the balance between the objectives and constraints. It converts a CMOP into an unconstrained MOP by adding the overall constraint violation multiplied by a predefined penalty factor λ to each objective []. In the case of λ =, it is called a death penalty approach [], which means that infeasible solutions are totally unacceptable. If λ is a static value during the search process, it is called a static penalty approach []. If λ is changing during the search process, it is called a dynamic penalty approach []. In the case of λ is changing according to the information collected from the search process, it is called an adaptive penalty approach [] [9]. In oder to avoid tuning the penalty factors, another type of constraint-handling methods is suggested, which compares the objectives and constraints separately. Representative examples include constraint dominance principle (CDP) [], epsilon j=

2 JOURNAL OF LATEX CLASS FILES, VOL., NO. 8, AUGUST constraint-handling method (EC) [], stochastic ranking approach (SR) [], and so on. In CDP [], three basic rules are adopted to compare any two solutions. In the first rule, given two solutions x i, x j R n, if x i is feasible and x j is infeasible, x i is better than x j. In the case of x i and x j are both infeasible, the one with a smaller constraint violation is better. In the last rule, x i and x j are both feasible, and the one dominating the other is better. CDP is a popular constraint-handling method, as it is simple and has no extra parameters. However, it is not suitable for solving CMOPs with very small and narrow feasible regions []. For many generations, most or even all solutions in the working population are infeasible when solving CMOPs with this property. In addition, the diversity of the working population can hardly be well maintained, because the selection of solutions is only based on the constraint violations according to the second rule of CDP. In order to solve CMOPs with small and narrow feasible regions, the epsilon constraint-handling (EC) [] approach is suggested. It is similar to CDP except for the relaxation of the constraints. In EC, the relaxation of the constraints is controlled by the epsilon level ε, which can help to maintain the diversity of the working population in the case when most solutions are infeasible. To be more specific, if the overall constraint violation of a solution is less than ε, this solution is deemed as feasible. The epsilon level ε is a critical parameter in EC. In the case of ε =, EC is the same as CDP. Although EC can be used to solve CMOPs with small feasible regions, controlling the value of ε properly is non-trivial at all. Both CDP [] and EC [] first compare the constrains, then compare the objectives. SR [] is different from CDP and EC in terms of the order of comparison. It adopts a probability parameter p f [, ] to decide if the comparison is based on objectives or constraints. For any two solutions, if a random number is less than p f, the one with the nondominated objectives is better. The comparison is based on objectives. On the contrary, if the random number is greater than p f, the comparison is first based on the constraints, then on the objectives, which is the same as that of CDP. In the case of p f =, SR is equivalent to CDP. In recent years, many work has been done in the field of many-objective evolutionary algorithms (MaOEAs) [], which gives us a new way to solve CMOPs. In order to balance the constraints and the objectives, some researchers adopt multi-objective evolutionary algorithms (MOEAs) or MaOEAs (in the case of the number of objectives is greater than three) to deal with constraints []. For an M-objective CMOP, its constraints can be converted into one or k extra objectives. Then the M-objective CMOP is transformed into an (M + )- or (M + k)-objective unconstrained MOP, which can be solved by MOEAs or MaOEAs. Representative examples include Cai and WangâĂŹs Method (CW) [] and the in-feasibility driven evolutionary algorithms (IDEA) [7]. To maintain a good balance between the constraints and the objectives, some researchers combine several constrainthandling mechanisms, which can be further divided into two categories, including adopting different constraint-handling mechanisms in different evolutionary stages or in different subproblems. For example, the adaptive trade-off model (ATM) [8] uses two different constraint-handling mechanisms, including multi-objective approach and the adaptive penalty functions in different evolutionary stages. The ensemble of constraint-handling methods (ECHM) [9] uses three different constraint-handling techniques, including epsilon constrainthandling (EC) [], self-adaptive penalty functions (SP) [9] and superiority of feasible solutions (SF) []. Three subpopulations are generated in ECHM, and each sub-population uses a different constraint-handling method. In this paper, we propose a biphasic CMOEA, namely push and pull search (PPS), to balance the objectives and the constraints. Unlike the above-mentioned constraint-handling methods, the PPS divide the search process into two different stages. In the first stage, only the objectives are optimized, which means the working population is pushed toward the unconstrained without considering any constraints. Furthermore, the landscape of constraints in CMOPs can be estimated in the push stage, which can be applied to conduct the parameters setting of constraint-handling approaches applied in the pull stage. In the pull stage, an improved EC [] constraint-handling approach is adopted to pull the working population to the constrained. In summary, it provides a new framework and has the following potential advantages. ) It has the ability to get across large infeasible regions in front of the constrained. Since the constraints are ignored in the push stage, the infeasible regions in front of the real present no barriers for the working population. ) It facilitates the parameters setting in the constrainthandling methods. Since the landscape of constraints has already been explored by the push process, a lots of information can be discovered and gathered to guide the search and parameter setting in the pull stage. The rest of the paper is organized as follows. Section II introduces the general idea of PPS. Section III gives an instantiation of PPS in the framework of MOEA/D, called PPS- MOEA/D. Section IV designs a set of experiments to compare the proposed PPS-MOEA/D with five other CMOEAs, including MOEA/D-IEpsilon, MOEA/D-Epsilon, MOEA/D- SR, MOEA/D-CDP and C-MOEA/D. Finally, conclusions are made in section V. II. THE GENERAL FRAMEWORK OF PUSH AND PULL SEARCH Constraints define the infeasible regions in the decision space, which may also affect the s of CMOPs in the objective space. The influence of the infeasible regions on the s can be generally classified into four different situations. For each situation, the search behavior of PPS is illustrated by Fig. - respectively, which can be summarized as follows. ) block the way towards the, as illustrated by Fig. (a). In this circumstance, the unconstrained is the same as the constrained, and PPS has significant advantages compared with other CMOEAs. Since the constraints are ignored in the push stage of PPS, the infeasible regions have no effect

3 JOURNAL OF LATEX CLASS FILES, VOL., NO. 8, AUGUST on the searching of PPS. Fig. (a)-(e) show the push process in different stages, where the working population crosses the infeasible regions in this case without any extra efforts. As the constrained is the same as the unconstrained, the true has already been approximated by the working population in the push process, the pull search has no affect on the working population, as shown in Fig. (f). ) The unconstrained is covered by infeasible regions and becomes no more feasible. Every constrained Pareto optimal point lies on some constraint boundaries, as illustrated by Fig. (a). In this circumstance, PPS first approaches the unconstrained by using the push strategy as illustrated by Fig. (a)-(c). After the working population approaches the unconstrained, the pull strategy is applied to pull the working population towards the real, as illustrated by Fig. (d)-(f). ) make the original unconstrained partially feasible, as illustrated by Fig. (a). In this situation, some parts of the true have already been achieved during the push search as illustrated by Fig. (c). In the pull stage, infeasible solutions are pulled to the feasible and non-dominated regions as illustrated by Fig. (d)-(f). Finally, the whole real has been found by PPS. ) may reduce the dimensionality of the of a CMOP, as illustrated by Fig. (a). In the push stage, a set of non-dominated solutions without considering constraints are achieved as illustrated by Fig. (a)-(c). Then, infeasible solutions in the working population are pulled toward the feasible and non-dominated regions in the pull search process, as shown in Fig. (d)-(f). From the above analysis, it can be observed that PPS can deal with CMOPs with all situations of s affected by constraints. The main steps of PPS includes the push and pull search processes. However, when to apply the push or pull search is very critical. A strategy to switch the search behavior is suggested as follows. r k max{rz k, rn k } ɛ () where r k represents the max rate of change between the ideal and nadir points during the last l-th generation. ɛ is a userdefined parameter, here we set ɛ = e. The rates of change of the ideal and nadir points during the last l-th generation are defined in Eq. and Eq. respectively. rz k = max { i=,...,m rn k = max { i=,...,m z k i z k l i max{ z k l i, e } } () n k i nk l i max{ n k l i, e } } () where z k = (z k,..., zk m), n k = (n k,..., nk m) are the ideal and nadir points in the k-th generation, and z k i = min j=,..., N f i (x j ), n k i = max j=,..., N f i (x j ). z k l = (z k l,..., zm k l ), n k l = (n k l,..., nm k l ) are the ideal and nadir points in the (k l)-th generation. At the beginning of the search, r k is initialized to.. At each generation, r k is updated according to Eq.. If r k is less or equal then the predefined threshold ɛ, the search behavior is switched to the pull search. To summarize, PPS divides its search process into two different stages, including the push search and pull search. At the first stage, the push search without considering constraints is adopted to approximate the unconstrained. Once Eq. is satisfied, the pull search is used to pull the infeasible solutions to the feasible and non-dominated regions. In this situation, constraints are considered in the pull search process. PPS terminates when a predefined halting condition is met. In the following section, we will describe the instantiation of the push and pull strategy in a MOEA/D framework. III. AN INSTANTIATION OF PPS IN MOEA/D In this section, the details of an instantiation of the push search method, as well as the method adopted in the pull search, and the proposed PPS in the framework of MOEA/D are described. A. The push search In the push search stage, an unconstrained MOEA/D is used to search for non-dominated solutions without considering any constraints. When solving a CMOP by using MOEA/D, we can decompose the CMOP into a set of single constrained optimization subproblems, and optimize them simultaneously in a collaborative way. Each subproblem is associated with a decomposition function by using a weight vector λ i. In the decomposition-based selection approach, an individual is selected into next generation based on the value of the decomposition function. There are three popular decomposition approaches, including weighted sum [], Tchebycheff [] and boundary intersection approaches []. In this paper, we adopt the Tchebycheff decomposition method, with the detailed definition given as follows. g te (x, λ i, z ) = max j=,...,m λ i j ( f j (x) z j ) (7) where λ i is a weight vector, and j=,...,m λ i j =, λi j. z is the ideal points as described in section II. In the push search stage, a newly generated solution x is kept to the next generation based on the value of g te as described in Algorithm. B. The pull search In this process, some infeasible solutions are pulled to the feasible and non-dominated regions. To achieve this, a constraint-handling mechanism is adopted to punish the infeasible solutions in the pull search stage. An improved epsilon constraint-handling to deal with constraints is proposed with the detailed formulation given as follows.

4 JOURNAL OF LATEX CLASS FILES, VOL., NO. 8, AUGUST (a) (b) (c) (d) (e) Fig.. block the way towards the, and the unconstrained is the same as the constrained. (a)-(e) show the different stages of the push search process, and the working population can get across the infeasible regions without any extra efforts dealing with constraints. (f) shows the pull search process, which is the same as (e) in this particular case, since the real is the same as the unconstrained, and has already been achieved by the working population in the push search process. (f) with constraints without constraints with constraints without constraints with constraints without constraints (a) (b) (c) with constraints without constraints with constraints without constraints with constraints without constraints (d) (e) Fig.. The unconstrained is covered by infeasible regions and becomes no more feasible. The real lies on some boundaries of constraints. (a)-(c) show the push search process, and the working population can get across the infeasible regions without any barriers. (e)-(f) show the pull search process, the infeasible solutions in the working population are pulled to the feasible and non-dominated regions gradually. (f)

5 JOURNAL OF LATEX CLASS FILES, VOL., NO. 8, AUGUST with constraints without constraints with constraints without constraints with constraints without constraints (a) (b) (c) with constraints without constraints with constraints without constraints with constraints without constraints (d) (e) (f) Fig.. make the original unconstrained partially feasible. (a)-(c) show the push search process, and some parts of the real have been found in this process. In the pull stage, infeasible solutions have been pulled to the feasible and non-dominated regions gradually as shown in (d)-(f). without constraints without constraints without constraints (a) (b) (c) without constraints without constraints without constraints (d) (e) Fig.. reduce the dimensionality of the. (a)-(c) show the push search process of PPS, and a set of non-dominated solutions are achieved without considering any constraints. (d)-(f) show the pull search process of PPS, many infeasible solutions are pulled to the feasible and non-dominated regions, which reside in a reduced dimensionality of the space. (f)

6 JOURNAL OF LATEX CLASS FILES, VOL., NO. 8, AUGUST Algorithm : Push Subproblem Function result = PushSubproblems(x j,y j ) result = f alse if g te (y i λ j, z ) g te (x j λ j, z ) then x j = y j result = ture end 7 return result 8 end ε(k) = { ( τ)ε(k ), if r f k < α ε()( k T c ) cp, if r f k α where r f k is the ratio of feasible solutions in the k-th generation. τ is to control the speed of reducing the relaxation of constraints in the case of r f k < α, and τ [, ]. α is to control the searching preference between the feasible and infeasible regions, and α [, ]. cp is to control the speed of reducing relaxation of constraints in the case of r f k α. ε(k) is updated until the generation counter k reaches the control generation T c. ε() is set to the maximum overall constraint violation of the working population at the end of the push search. The above proposed epsilon setting method in Eq. 8 is different from our previous work [] and []. In this paper, ε() is set to the maximum overall constraint violation of the working population at the end of the push search, while ε() is set to the overall constraint violation of the top θ-th individual in the initial population in [] and []. In the case of r f ==, ε() = according to the epsilon setting in [] and [], which can not help to explore the infeasible regions and get across large infeasible regions. To avoid this situation, ε(k) is increased if r f k is greater than a predefined threshold in [] and []. However, increasing ε(k) properly is still an issue that needs to be further investigated. In this paper, ε(k) does not need to be increased to get across large infeasible regions, because the working population has already gotten across large infeasible regions in the push stage. In the pull stage, a newly generated solution x is selected into the next generation based on the value of g te and the overall constraint violation φ(x) as illustrated by Algorithm. C. PPS Embedded in MOEA/D Algorithm outlines the psuecode of PPS-MOEA/D, a PPS embedded in MOEA/D. A CMOP is decomposed into N single objective subproblems, and these subproblems are initialized at line. The max rate of change of ideal and nadir points is initialized to., and the flag of search stage PushStage = true. Then, the algorithm repeats lines -7 until T max generations have been reached. The value of ε(k) and the search strategy are set at lines - based on Eq. and Eq. 8. ε(k) and ε() are initialized at line 7, and the function MaxViolation() calculates the maximum overall constraint violation in the current working population. Lines (8) Algorithm : Pull Subproblem Function result = PullSubproblems(x j,y j,ε(k)) result = f alse if φ(y j ) ε(k) and φ(x j ) ε(k) then if g te (y i λ j, z ) g te (x j λ j, z ) then x j = y j result = ture 7 end 8 else if φ(y j ) == φ(x j ) then 9 if g te (y j λ j, z ) g te (x j λ j, z ) then x j = y j result = ture end else if φ(y j ) < φ(x j ) then x j = y j result = ture end 7 return result 8 end -9 show the process of generating a new solution. The updating of the ideal points are illustrated by lines -. Lines - show the updating of subproblems, and different search strategies are adopted according to the state of the searching. In the case of PushStage == true, the push search is adopted, otherwise, the pull search is used. The generation count k is updated at line. The set of feasible and nondominated solutions N S is updated according to the nondominated ranking in NSGA-II [] at line. Remark: The proposed PPS is a general framework for solving CMOPs. It can be instantiated in many different MOEAs, even though only PPS-MOEA/D is realized in this paper. At each search stage, many information can be gathered to extract useful knowledge which can be used to conduct the searching in both search stages. In fact, knowledge discovery is a critical step in the PPS framework. In this paper, we only adopts some statistic information. For example, the maximum overall constraint violation at the end of the push search stage is adopted to set the value of ε(). The ratio of feasible solutions at the pull search stage is used to control the value of ε(k). In fact, many data mining methods and machine learning approaches can be integrated in the PPS framework for solving CMOPs more efficiently. A. Experimental Settings IV. EXPERIMENTAL STUDY To evaluate the performance of the proposed PPS method, the other five CMOEAs, including MOEA/D- IEpsilon, MOEA/D-Epsilon, MOEA/D-SR, MOEA/D-CDP and C-MOEA/D, are tested on LIR-CMOP- []. The detailed parameters are listed as follows: ) The mutation probability Pm = /n (n denotes the dimension of a decision vector). The distribution index in the polynomial mutation is set to. ) DE parameters: CR =., f =..

7 JOURNAL OF LATEX CLASS FILES, VOL., NO. 8, AUGUST 7 Algorithm : PPS-MOEA/D Input: N: the number of subproblems. T max : the maximum generation. N weight vectors: λ,..., λ N. T: the size of the neighborhood. δ: the selecting probability from neighbors. n r : the maximal number of solutions replaced by a child. Output: N S : a set of feasible non-dominated solutions Decompose a CMOP into N subproblems associated with λ,..., λ N. Generate an initial population P = {x,..., x N }. For each i =,..., N, set B(i) = {i,..., i T }, where λ i,..., λ i T are the T closest weight vectors to λ i. Initialize the rate of change of ideal and nadir points (maxchange =.) over the previous L generations. Set PushStage = true, k =. while k T max do if k >= L then maxchange = CalcMaxChange(k) according to Eq. ; if k < T c then if maxchange α And PushStage == true then 7 PushStage = f alse; ε(k) = ε() = MaxViolation(); 8 end 9 if PushStage == f alse then Update ε(k) according to Eq. 8; end else ε(k) = ; end Generate a random permutation rp from {,..., N}. for i to N do 7 Generate a random number r [, ], j = rp(i). 8 if r < δ then S = B(j) else S = {,..., N} ; 9 Generate y j through the DE operator, and perform polynomial mutation on y j. for t to m do if zt > f t (y j ) then zt = f t(y j ) ; end Set c =. while c n r or S do select an index j from S randomly. if PushStage == true then 7 result = PushSubproblems(x j, y j );// Algorithm ; 8 else 9 result = PullSubproblems(x j, y j, ε(k)); // Algorithm ; end if result == true then c = c + ; S = S\{ j}; end end k = k + ; NS = NDSelect(NS P); // According to NSGA-II []; 7 end ) Population size: N =. Neighborhood size: T =. ) Halting condition: each algorithm runs for times independently, and stops until, function evaluations are reached. ) Probability of selecting individuals from its neighborhood: δ =.9. ) The max number of solutions updated by a child: nr =. 7) Parameters setting in PPS-MOEA/D: T c = 8, α =.9, τ =., cp =, l =. 8) Parameters setting in MOEA/D-IEpsilon: T c = 8, α =.9, τ =., and θ =.N. 9) Parameters setting in MOEA/D-Epsilon: T c = 8, cp =, and θ =.N. ) Parameter setting in MOEA/D-SR: S r =.. B. Performance Metric To measure the performance of PPS-MOEA/D, MOEA/D- IEpsilon, MOEA/D-Epsilon, MOEA/D-CDP, MOEA/D-SR and C-MOEA/D, two popular metrics the inverted generation distance (IGD) [] and the hypervolume [] are adopted. Inverted Generational Distance (IGD): The IGD metric reflects the performance regarding convergence and diversity simultaneously. The detailed definition is given as follows: IGD(P, A) = y P d(y, A) P d(y, A) = min { mi= (y y A i y i) } where P denotes a set of representative solutions in the real, A is an approximate achieved by a CMOEA. m denotes the number of objectives. For LIR-CMOPs with two objectives, points are sampled uniformly from the true to construct P. For LIR-CMOPs with three objectives, points are sampled uniformly from the to constitute P. It is worth noting that a smaller value of IGD may indicate better performance with regards to diversity and/or convergence. Hypervolume (HV): HV reflects the closeness of the set of non-dominated solutions achieved by a CMOEA to the real. The larger HV means that the corresponding non-dominated set is closer to the true. (9) HV(S) = VOL( [ f (x), z r ]...[ f m(x), zm]) r () x S where VOL( ) is the Lebesgue measure, m denotes the number of objectives, z r = (z r,..., zr m) T is a user-defined reference point in the objective space. For each LIR-CMOP, the reference point is placed at. times the distance to the nadir point of the true. It is worth noting that a larger value of HV may indicate better performance regarding diversity and/or convergence.

8 JOURNAL OF LATEX CLASS FILES, VOL., NO. 8, AUGUST 8 C. Discussion of Experiments ) A Comparison between PPS-MOEA/D and MOEA/D- IEpsilon: The difference between PPS-MOEA/D and MOEA/D-IEpsilon is that MOEA/D-IEpsilon does not apply the push search without considering any constraints, and they adopt different epsilon setting methods. To illustrate the effect of the PPS framework, the performance regarding IGD and HV metrics of PPS-MOEA/D and MOEA/D-IEpsilon on LIR- CMOP- is compared. The statistical results of the IGD values on LIR-CMOP- achieved by PPS-MOEA/D and MOEA/D-IEpsilon in independent runs are listed in Table I. For LIR-CMOP-, LIR- CMOP-8, LIR-CMOP and LIR-CMOP, PPS-MOEA/D performs significantly better than MOEA/D-IEpsilon. For LIR- CMOP9, MOEA/D-IEpsilon is significantly better than PPS- MOEA/D. For the rest of test instances, MOEA/D-IEpsilon and PPS-MOEA/D have no significant difference. The statistical results of the HV values on LIR-CMOP- achieved by PPS-MOEA/D and MOEA/D-IEpsilon in independent runs are listed in Table II. For LIR-CMOP-, LIR-CMOP-8 and LIR-CMOP, PPS-MOEA/D is significantly better than MOEA/D-IEpsilon. There is no significantly difference between PPS-MOEA/D and MOEA/D-IEpsilon on LIR-CMOP and LIR-CMOP-. For LIR-CMOP9, PPS is significantly worse than MOEA/D-IEpsilon. Since the real of LIR-CMOP9 locates on its unconstrained and has several disconnected parts, as illustrated by Fig. (a), PPS-MOEA/D should performs well on this problems. However, the statistic results of both IGD and HV indicate that PPS-MOEA/D is significantly worse than MOEA/D-IEpsilon. One possible reason is that PPS-MOEA/D has not converged to the whole unconstrained in the push search stage. To verify this hypothesis, we adjust the value of l in Eq. () and Eq. (), which is a critical parameter to switch the search behavior. The mean values of IGD and HV values of MOEA/D-IEpsilon and PPS-MOEA/D with different values of l are shown in Fig. (b) and Fig. (c), respectively. We can observe that, in cases of l 7, PPS-MOEA/D is better than MOEA/D-IEpsilon. To summarize, PPS-MOEA/D is superior to MOEA/D- IEpsilon on eight test problems, is not significantly different from MOEA/D-IEpsilon on five test problems, is inferior to MOEA/D-IEpsilon on only one test instance. It can be concluded that the PPS framework can help to enhance the performance of a CMOEA in terms of the IGD and HV metrics. ) A Comparison of PPS-MOEA/D and the other four decomposition-based CMOEAs: The statistical results of the IGD values on LIR-CMOP- achieved by PPS-MOEA/D and the other four decomposition-based CMOEAs in independent runs are listed in Table I. According to the Wilcoxon- Test in this table, it is clear that PPS-MOEA/D is significantly better than MOEA/D-Epsilon, MOEA/D-CDP, MOEA/D-SR, C-MOEA/D on all of the fourteen tested problems in terms of the IGD metric. The statistical results of the HV values on LIR-CMOP- achieved by PPS-MOEA/D and the other four decompositionbased CMOEAs in independent runs are listed in Table II. It can be observed that PPS-MOEA/D is significantly better than MOEA/D-Epsilon, MOEA/D-CDP, MOEA/D-SR and C- MOEA/D on all the test instances in terms of the HV metric. It is clear that the proposed PPS-MOEA/D performs better than the other four decomposition-based CMOEAs on all of the test instances. ) The advantages of PPS-MOEA/D: The box plots of IGD values for the six tested CMOEAs on LIR-CMOP, LIR- CMOP7 and LIR-CMOP are shown in Fig.. It is clear that the mean IGD value of PPS-MOEA/D is the smallest among the six tested CMOEAs. To further illustrate the advantages of the proposed PPS-MOEA/D, the populations achieved by each tested CMOEAs on LIR-CMOP, LIR-CMOP7, and LIR- CMOP during the independent runs are plotted in Fig. 8, Fig. 9 and Fig., respectively. For LIR-CMOP, there are two large infeasible regions in front of the, and the unconstrained is the same as the constrained, as illustrated by Fig. 7(a). In Fig. 8, we can observe that PPS-MOEA/D can get across the large infeasible regions at each running time, while the other five CMOEAs are sometimes trapped into local optimal. For LIR-CMOP7, there are three large infeasible regions, and the unconstrained is covered by infeasible regions and becomes no more feasible as illustrated by Fig. 7(b). In Fig. 9, we can observe that PPS-MOEA/D can converge to the real at each running time, while the other five CMOEAs can not always converge to the real. For LIR-CMOP, infeasible regions make the original unconstrained partially feasible, as illustrated by Fig. 7(c). The of LIR-CMOP is disconnected and has seven Pareto optimal solutions. PPS-MOEA/D can find the seven discrete Pareto optimal solutions at each running time, while the other five CMOEAs can not find all the seven Pareto optimal solutions sometimes, as shown in Fig.. In Fig. (b), it seems that MOEA/D-IEpsilon has found all the Pareto optimal solutions. However, MOEA/D-IEpsilon can not find all the Pareto optimal solutions more than fifteen out of runs. For MOEA/D-Epsilon, MOEA/D-CDP, MOEA/D-SR and C- MOEA/D, some Pareto optimal solutions are not found during the thirty independent runs, as illustrated by (c)-(f). V. CONCLUSION This paper proposes a general PPS framework to deal with CMOPs. More specifically, the search process of PPS is divided into two stages, including the push and pull search processes. At the push stage, constraints are ignored, which can help PPS to get across infeasible regions in front of the unconstrained s. Moreover, the landscape affected by constraints can be estimated at the push stage, and this information, such as the ration of feasible solutions and the maximum overall constraint violation, can be applied to conduct the settings of parameters coming from the constrainthandling mechanisms in the pull stage. When the max rate of change between ideal and nadir points is less or equal than a predefined threshold, PPS is switched to the pull search process. The infeasible solutions achieved in the push stage

9 JOURNAL OF LATEX CLASS FILES, VOL., NO. 8, AUGUST 9 TABLE I IGD RESULTS OF PPS AND THE OTHER FIVE CMOEAS ON LIR-CMOP-. Test Instances PPS-MOEA/D MOEA/D-IEpsilon MOEA/D-Epsilon MOEA/D-CDP MOEA/D-SR C-MOEA/D LIR-CMOP mean.e- 7.97E-.7E-.7E-.8E-.E- std.98e-.e-.887e-.9e-.e- 7.8E- LIR-CMOP mean.7e-.e-.9e-.e- 9.E-.97E- std 7.8E-.9E-.8E-.E- 7.E-.E- LIR-CMOP mean 8..E-.E- 8.8E-.8E-.77E-.8E- std.8e-.e-.e-.e- 7.9E-.E- LIR-CMOP mean.77e-.88e-.e-.7e-.99e-.9e- std.e-.e-.e-.e-.e-.e- LIR-CMOP mean.87e-.e-.e+.e+.e+.e+ std 9.E-.79E-.979E-.E-.E-.99E- LIR-CMOP mean.9e-.e-.9e+.9e+ 9.7E-.7E+ std.99e-.e-.97e-.98e-.899e-.8e- LIR-CMOP7 mean.797e-.7e-.9e+.e+.79e+.e+ std 9.8E-.E-.9E-.8E- 7.8E-.E- LIR-CMOP8 mean.778e-.79e-.e+.7e+.e+.79e+ std 7.8E-.97E-.E-.9E- 7.E-.7E- LIR-CMOP9 mean 9.9E-.977E-.9E-.8E-.8E-.8E- std.9e-.7e-.e-.e-.77e-.e- LIR-CMOP mean.8e-.e-.e-.9e-.9e-.e- std 7.7E- 7.E-.E-.8E-.8E-.E- LIR-CMOP mean.8e-.89e-.8e-.8e-.7e-.8e- std.9e-.79e- 9.8E- 9.E- 7.9E- 8.99E- LIR-CMOP mean.7e-.8e-.e-.9e-.e-..e- std.e-.78e- 8.98E- 9.8E-.E- 9.8E- LIR-CMOP mean.e-.e-.e+.e+ 8.8E-.8E+ std.77e-.9e-.e-.7e-.7e-.77e- LIR-CMOP mean.9e-.e-.e+.7e+.7e+.e+ std.9e-.7e-.8e-.97e-.7e-.98e- Wilcoxon-Test (S-D-I) WilcoxonâĂŹs rank sum test at a. significance level is performed between PPS-MOEA/D and each of the other five CMOEAs. and denote that the performance of the corresponding algorithm is significantly worse than or better than that of PPS-MOEA/D, respectively. Where âăźs-d-iâăź indicates PPS-MOEA/D is superior to, not significantly different from or inferior to the corresponding compared CMOEAs. TABLE II HV RESULTS OF PPS-MOEA/D AND THE OTHER FIVE CMOEAS ON LIR-CMOP-. Test Instances PPS-MOEA/D MOEA/D-IEpsilon MOEA/D-Epsilon MOEA/D-CDP MOEA/D-SR C-MOEA/D LIR-CMOP mean.e+.e+ 9..E- 7.8E- 9.9E- 7.E- std.8e-.e-.8e- 8.97E-.9E-.E- LIR-CMOP mean.9e+.8e+.8e+.e+.7e+.7e+ std.e-.8e-.88e-.78e-.e- 9.99E- LIR-CMOP mean 8.7E- 8.8E- 7.97E-.8E-.9E-.77E- std.e-.99e-.99e-.e-.7e-.89e- LIR-CMOP mean.9e+.9e+.e+ 7.9E- 8.7E- 7.8E- std.7e-.8e-.9e-.e- 8.99E-.E- LIR-CMOP mean.e+.e+.97e-.e-.87e- 9.7E- std.99e-.e-.e-.e-.89e-.99e- LIR-CMOP mean.9e+ 9.9E-.98E-.88E-.E-.8E- std.77e-.e-.e-.87e-.e-.7e- LIR-CMOP7 mean.e+.8e+.e-.7e E-.E- std.e-.98e- 9.7E- 9.77E-.7E+ 7.E- LIR-CMOP8 mean.7e+.9e+.e-.7e-.99e+.8e- std.9e-.8e-.88e-.e+.98e+.e- LIR-CMOP9 mean.7e+.77e+.77e+.77e+.7e+.77e+ std.e-.878e-.8e-.8e-.e-.8e- LIR-CMOP mean.e+.e+.88e+.879e+.97e+.888e+ std.77e-.8e-.e-.e-.7e- 9.7E- LIR-CMOP mean.9e+.e+.e+.e+.8e+.e+ std.7e-.8e-.7e-.8e-.899e-.e- LIR-CMOP mean.e+.9e+.88e+.87e+.e+.89e+ std..e-.78e-.8e-.8e-.7e-.e- LIR-CMOP mean.7e+.7e+..e-.8e-.89e+.9e- std.7e-.e-.e+.e+.7e+.7e+ LIR-CMOP mean.9e+.8e+.e+ 8.8E-.7E+.79E- std.e-.9e-.e+.99e+.88e+.98e- Wilcoxon-Test (S-D-I) WilcoxonâĂŹs rank sum test at a. significance level is performed between PPS-MOEA/D and each of the other five CMOEAs. and denote that the performance of the corresponding algorithm is significantly worse than or better than that of PPS-MOEA/D, respectively. Where âăźs-d-iâăź indicates PPS-MOEA/D is superior to, not significantly different from or inferior to the corresponding compared CMOEAs.

10 JOURNAL OF LATEX CLASS FILES, VOL., NO. 8, AUGUST without constraints Constraint boundary Mean of HV Mean of IGD Indexes of CMOEAs (a) The landscape of LIR-CMOP Indexes of CMOEAs (b) IGD (c) HV Fig.. The feasible and infeasible regions of LIR-CMOP9 are plotted in (a). In (b)-(c), the mean values of IGD and HV of MOEA/D-IEpsilon and PPS-MOEA/D with different values of l are plotted. In the horizontal axises of (b)-(c), the index represents MOEA/D-IEpsilon, indexes - denote PPS-MOEA/D with l =,,,,,, 7, 8, 9 and IGD IGD IGD..... (a) LIR-CMOP (b) LIR-CMOP7 (c) LIR-CMOP Fig.. The box plots of IGD values for the six tested CMOEAs. The horizontal axis shows the index of each tested CMOEAs, where,,,, and denote PPS-MOEA/D, MOEA/D-IEpsilon, MOEA/D-Epsilon, MOEA/D-CDP, MOEA/D-SR and C-MOEA/D respectively. with constraints without constraints without constraints Constraint boundary... (a) LIR-CMOP Fig. 7. (b) LIR-CMOP7... (c) LIR-CMOP Illustrations of the feasible and infeasible regions of LIR-CMOP, LIR-CMOP7 and LIR-CMOP. are pulled to the feasible and non-dominated area by adopting an improved epsilon constraint-handling technique. The value of epsilon level can be set properly according to the maximum overall constraint violation obtained at the end of the push search stage. The comprehensive experiments indicate that the proposed PPS achieves competitive or statistically significant better results than the other five CMOEAs on most of the benchmark problems. It is also worthwhile to point out that there has been very little work regarding using information of landscape affected by constraints to solve CMOPs. In this context, the proposed PPS provides a viable framework. Obviously, a lot of work need to be done to improve the performance of PPS, such as, the augmented constraint-handling mechanisms in the pull stage, the enhanced strategies to switch the search behavior and the data mining methods and machine learning approaches integrated in the PPS framework. For another future work, the proposed PPS will be implemented in the nondominated framework, such as NSGA-II, to further verify the effect of PPS. More other CMOPs and real-world constrained engineering optimization problems will also be used to test the performance of the PPS embedded in different MOEA frameworks.

11 JOURNAL OF LATEX CLASS FILES, VOL., NO. 8, AUGUST True PPS-MOEA/D True MOEA/D-IEpsilon (x).. (x) (x) (a) PPS-MOEA/D True MOEA/D-SR True C-MOEA/D (x) (x) 8 (c) MOEA/D-Epsilon True MOEA/D-CDP (x) (b) MOEA/D-IEpsilon (x) True MOEA/D-Epsilon 8 (x) (x) (x) (f) C-MOEA/D True PPS-MOEA/D True MOEA/D-IEpsilon (x)... (x) 8 (x) (b) MOEA/D-IEpsilon (c) MOEA/D-Epsilon 8 True MOEA/D-CDP (x) (a) PPS-MOEA/D True MOEA/D-SR True C-MOEA/D (x) (x) True MOEA/D-Epsilon 8 (x) (x) (x) (e) MOEA/D-SR (x) The non-dominated solutions achieved by each algorithm on LIR-CMOP during the independent runs are plotted in (a)-(f). (x) (d) MOEA/D-CDP Fig. 9. (x) (d) MOEA/D-CDP Fig. 8. (x) (e) MOEA/D-SR (x) (f) C-MOEA/D The non-dominated solutions achieved by each algorithm on LIR-CMOP7 during the independent runs are plotted in (a)-(f).

12 JOURNAL OF LATEX CLASS FILES, VOL., NO. 8, AUGUST True PPS-MOEA/D True MOEA/D-IEpsilon True MOEA/D-Epsilon (x) (x) (x) 8 (x) (a) PPS-MOEA/D 8 (x) (b) MOEA/D-IEpsilon (x) (c) MOEA/D-Epsilon True MOEA/D-CDP 8 True MOEA/D-SR True C-MOEA/D (x) (x) (x) Fig.. (x) (d) MOEA/D-CDP 8 (x) (e) MOEA/D-SR (x) (f) C-MOEA/D The non-dominated solutions achieved by each algorithm on LIR-CMOP during the independent runs are plotted in (a)-(f). ACKNOWLEDGMENT This research work was supported by Guangdong Key Laboratory of Digital Signal and Image Processing, the National Natural Science Foundation of China under Grant (77, 9,, 787), Jiangsu Natural Science Foundation (BK88) and Science and Technology Planning Project of Guangdong Province, China (B). REFERENCES [] K. Deb, Multi-objective optimization using evolutionary algorithms. John Wiley & Sons,, vol.. [] C. A. C. Coello, Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art, Computer Methods in Applied Mechanics and Engineering, vol. 9, no., pp. 87,. [] T. BDack, F. Hoffmeister, and H. Schwefel, A survey of evolution strategies, in Proceedings of the th international conference on genetic algorithms, 99, pp. 9. [] A. Homaifar, C. X. Qi, and S. H. Lai, Constrained optimization via genetic algorithms, Simulation, vol., no., pp., 99. [] J. A. Joines and C. R. Houck, On the use of non-stationary penalty functions to solve nonlinear constrained optimization problems with ga s, in Evolutionary Computation, 99. IEEE World Congress on Computational Intelligence., Proceedings of the First IEEE Conference on. IEEE, 99, pp [] J. C. Bean and A. ben Hadj-Alouane, A dual genetic algorithm for bounded integer programs, 99. [7] D. W. Coit, A. E. Smith, and D. M. Tate, Adaptive penalty methods for genetic optimization of constrained combinatorial problems, INFORMS Journal on Computing, vol. 8, no., pp. 7 8, 99. [8] A. Ben Hadj-Alouane and J. C. Bean, A genetic algorithm for the multiple-choice integer program, Operations research, vol., no., pp. 9, 997. [9] Y. G. Woldesenbet, G. G. Yen, and B. G. Tessema, Constraint handling in multiobjective evolutionary optimization, IEEE Transactions on Evolutionary Computation, vol., no., pp., June 9. [] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, A fast and elitist multiobjective genetic algorithm: NSGA-II, IEEE Transactions on Evolutionary Computation, vol., no., pp. 8 97, Apr. [] T. Takahama and S. Sakai, Constrained optimization by ε constrained particle swarm optimizer with ε-level control, in Soft Computing as Transdisciplinary Science and Technology. Springer,, pp [] T. P. Runarsson and X. Yao, Stochastic ranking for constrained evolutionary optimization, IEEE Transactions on evolutionary computation, vol., no., pp. 8 9,. [] Z. Fan, W. Li, X. Cai, K. Hu, H. Lin, and H. Li, Angle-based constrained dominance principle in moea/d for constrained multi-objective optimization problems, in IEEE Congress on Evolutionary Computation,, pp. 7. [] B. Li, J. Li, K. Tang, and X. Yao, Many-objective evolutionary algorithms: A survey, ACM Computing Surveys (CSUR), vol. 8, no., p.,. [] E. Mezura-Montes and C. A. Coello Coello, Constraint-handling in nature-inspired numerical optimization: Past, present and future, Swarm and Evolutionary Computation, vol., no., pp. 7 9, Dec.. [] Z. Cai and Y. Wang, A multiobjective optimization-based evolutionary algorithm for constrained optimization, Evolutionary Computation, IEEE Transactions on, vol., no., pp. 8 7,. [7] T. Ray, H. K. Singh, A. Isaacs, and W. Smith, Infeasibility driven evolutionary algorithm for constrained optimization, in Constrainthandling in evolutionary optimization. Springer, 9, pp.. [8] Y. Wang, Z. Cai, Y. Zhou, and W. Zeng, An adaptive tradeoff model for constrained evolutionary optimization, IEEE Transactions on Evolutionary Computation, vol., no., pp. 8 9, 8. [9] B. Y. Qu and P. N. Suganthan, Constrained multi-objective optimization algorithm with an ensemble of constraint handling methods, Engineering Optimization, vol., no., pp.,. [] K. Deb, An efficient constraint handling method for genetic algorithms, Computer methods in applied mechanics and engineering, vol. 8, no., pp. 8,. [] K. Miettinen, Nonlinear Multiobjective Optimization. Springer Science & Business Media, 999, vol.. [] Q. Zhang and H. Li, MOEA/D: A multiobjective evolutionary algorithm based on decomposition, IEEE Transactions on evolutionary computation, 7.

13 JOURNAL OF LATEX CLASS FILES, VOL., NO. 8, AUGUST [] Z. Fan, W. Li, X. Cai, H. Li, H. Huang, and Z. Cai, An improved epsilon constraint handling method embedded in MOEA/D for constrained multi-objective optimization problems, in IEEE Symposium Series on Computational Intelligence, Dec. [] Z. Fan, W. Li, X. Cai, H. Huang, Y. Fang, Y. You, J. Mo, C. Wei, and E. D. Goodman, An improved epsilon constraint-handling method in MOEA/D for cmops with large infeasible regions, arxiv preprint arxiv:77.877, 7. [] P. A. Bosman and D. Thierens, The balance between proximity and diversity in multiobjective evolutionary algorithms, Evolutionary Computation, IEEE Transactions on, vol. 7, no., pp. 7 88,. [] E. Zitzler and L. Thiele, Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach, IEEE Transactions on Evolutionary Computation, vol., no., pp. 7 7, 999.