HELPFUL OR UNHELPFUL: A LINEAR APPROACH FOR RANKING PRODUCT REVIEWS

Transcription

1 Zhang & Tran: Helpful or Unhelpful: A Lnear Approach for Rankng Product Revews HELPFUL OR UNHELPFUL: A LINEAR APPROACH FOR RANKING PRODUCT REVIEWS Rchong Zhang School of Informaton Technology and Engneerng Unversty of Ottawa 800 Kng Edward Ave., Ottawa, Ontaro, K1N 6N5, Canada rzhan025@ste.uottawa.ca Thomas Tran School of Informaton Technology and Engneerng Unversty of Ottawa 800 Kng Edward Ave., Ottawa, Ontaro, K1N 6N5, Canada ttran@ste.uottawa.ca ABSTRACT Most E-commerce web stes and onlne communtes provde nterfaces and platforms for consumers to express ther opnons about a specfc product by wrtng personal revews. The fast development of E-commerce has caused such a huge amount of onlne product revews to become avalable to consumers that t s mpossble for potental consumers to read through all the revews and to make a quck purchasng decson. Revew readers are asked to vote f a revew s Helpful or Unhelpful and the most postvely voted revews are placed on the top of product revew lst. However, the accumulaton of votes takes tme for a revew to be fully voted and newly publshed revews are always lsted at the bottom of the revew lst. Ths paper proposes a lnear model to predct the helpfulness of onlne product revews. Revews can be quckly ranked and classfed by our model and revews that may help consumers better than others wll be retreved. We compare our model wth several machne learnng classfcaton algorthms and our expermental results show that our approach effectvely classfes onlne revews. Also, we provde an evaluaton measurement to udge the performance of the helpfulness modelng algorthm and the results show that the helpfulness scores predcted by our approach consstently follow the changng trend of the true helpfulness values. Keywords: recommender system, onlne product revew, helpfulness, evaluaton 1. Introducton Due to the fast development of Internet and E-commerce, more and more onlne revews aggregaton web stes, such as Epnons.com etc., have provded consumers wth platforms to exchange ther opnons about products, servces, and merchants. Onlne product revews provded by consumers who prevously purchased products have become a maor nformaton source for consumers and marketers regardng product qualty [Hu & Zhang 2008]. Park et al. [2007] confrmed that the qualty of revews has a postve effect on product sales and consumers purchase ntentons ncrease wth the quantty of product revews. On the E-commerce web stes, such as Amazon.com and Ebay.com, consumers are asked to wrte revews and rate products or servces by a number of stars after they fnshed a transacton. Most of exstng recommendaton approaches [Goldberg et al. 1992; Resnck et al. 1994; Sarwar et al. 2001] are merely based on the ratng of products. Wth a star ratng scale, users can not get `real semantcs' of revew statements. Snce product revews represent revewers' feelngs, experences and opnons on a specfc product, they are more useful than product ratngs and therefore can better help potental consumers make purchase decsons. Search engnes are good tools to assst n lookng for nformaton; however, there are too many search results returned from a search engne and not all of them are revews. For nstance, f we nput `Cyber-shot Dgtal Camera Revew' n Google, 278,000 web pages wll be returned. Ths s absolutely a too huge result set for consumers to go through. Moreover, n an onlne communty, such as Epnon.com or Amazon.com, more than 1000 revews for a specfc product are submtted by dfferent consumers. Therefore, t s mportant to rank and classfy product revews so that they can be accessed easly and used effectvely by consumers. Revew aggregaton web stes provde a functon for consumers to vote whether a revew s Helpful or Unhelpful. But ths progress takes tme far before a really helpful revew s dscovered and the most recent Page 220

2 Journal of Electronc Commerce Research, VOL 11, NO 3, 2010 publshed revews wll always be the least voted ones. Our goal s to develop a model that flters out revews whch are most lkely helpful to consumers and that provde more valuable nformaton for consumer's decson makng. Such a model can save a great deal of consumers' effort n surfng for relable and helpful revews. Most of recent researches focus on topcal categorzaton, sentment classfcaton and polarty dentfcaton of consumer revews. In ths paper, we propose a lnear-tme model, whch utlzes the helpfulness gan of each term occurrng n revew documents, n order to calculate the helpfulness score of product revews. Revews provded by all members of a communty can be analyzed by our model and the helpfulness of each revew can be returned by our algorthm. Moreover, we provde a metrc, whch utlzes the log-lkelhood functon of voters' opnon to evaluate the performance of the revew helpfulness assessment algorthms. Not lke the conventonal researches only comparng the observed and the predcted helpfulness value, our evaluaton measurement consders both the opnon of each voter and the number of voters who have voted. We examne the performance of our model on the revews collected from Amazon.com and conclude that our approach can quckly and effectvely dscover the helpfulness of onlne product revews. Our model can also be used to classfy product revews as Helpful or Unhelpful. Emprcal results ndcate that the classfcaton ablty of our model outperforms or performs the same as other commonly used methods n comparson wth other classfers. The remander of ths paper s organzed as follows: Secton 2 dscusses related work. Secton 3 descrbes our proposed approach n detals ncludng score calculatng and predcton generatng. Secton 4 presents our expermental evaluaton whch ncludes a comparson between the proposed model and other machne learnng methods. Secton 5 provdes further dscusson on the value and applcatons of the proposed model; and fnally, Secton 6 concludes the paper and suggests some future research drectons. 2. Related Work Park and Km [2008] nvestgated the relatonshp between dfferent types of revews and consumers. They fnd that consumer concerns vary at each stage of the product lfe cycle, and suggest marketers develop dfferent strateges for dfferent types of consumers. Lee et al. [2008] examned the effect of the qualty of negatve onlne revews on product atttude and dscover that hgh-nvolvement consumers consder the qualty of negatve revews and low-nvolvement consumers tend to conform to other revewer atttudes regardless of the qualty. In [Vermeulen and Seegers 2009], authors studed the effect of onlne hotel revews on consumer consderaton and conclude that postve revews have a postve mpact on consumer behavor. These observatons ndcate that onlne product revews play an mportant role for consumers to make purchasng decsons. Some research works have been done on sentment classfcaton, also known as polarty classfcaton for onlne product revews. To dstngush or predct whether consumers lke a product or not based on ther revews, authors propose a method to predct the semantc orentaton of adectves by a supervsed learnng algorthm n [Hatzvassloglou & McKeown 1997]. Turney presents an unsupervsed learnng algorthm to classfy revews as recommended or not recommended by analyzng the semantc orentaton based on mutual nformaton [Turney 2001]. In [Yu & Hatzvassloglou 2003], the authors propose a classfcaton approach to separate sentences as postve or negatve. In [Pang et al. 2002], authors classfy move revews as postve or negatve by several machne learnng methods, namely Nave Bayes, Maxmum Entropy, and Support Vector Machnes, and they also use dfferent features such as ungram, bgram, poston and the combnaton of these features. The results show that ungram presence feature was the most effectve and the SVM performed the best for sentment classfcaton. The effect of onlne product revews on product sales s also a study area. Hu and Zhang [2008] dscover that consumers consder not only the revew ratngs but also the contextual nformaton lke revewer's reputaton. They also fnd that the mpact of onlne revews on sales dmnshes over tme. Some studes have been done n the area of revew mnng and summarzng. In [Zhuang et al. 2006], authors mne and summarze move revews based on a mult-knowledge approach whch ncludes WordNet, statstcal analyss and move knowledge. Hu and Lu [2004] summarze product revews by mnng opnon features. Evaluatng the qualty and helpfulness of revews or posts on web forums attracts more and more researchers' attentons. Km et al. [2006] delver a method to automatcally assess revew helpfulness. They use SVM to tran the system and fnd the length of the revew; the ungrams and the product ratng are the most useful features. Wemer et al. [2007a] propose an automatc algorthm to assess the qualty of posts on web forums usng features such as surface, lexcal, syntactc, forum specfc and smlarty features. Then the authors extend the method nto the onlne dsscusson messages on software and fnd the SVM classfcaton performs very well [Wemer et al. 2007b]. Lu et al. [2008] present a nonlnear regresson model for the helpfulness predcton. Three groups of factors whch mght affect the value of helpfulness are analyzed and the model s bult upon on these three groups of factors. The results from applyng ther model show that the performance s better than the SVM regresson model. Page 221

3 Zhang & Tran: Helpful or Unhelpful: A Lnear Approach for Rankng Product Revews In ths paper, we propose a lnear-tme helpfulness assessment approach. Wth ths approach, onlne product revews can be effectvely ranked and classfed, and the most helpful revews can be retreved to assst consumers n makng purchase decsons. 3. Proposed Approach Our work focuses on analyzng the revews and to fnd hgh qualty and helpful revews. In ths secton, we dscuss how to estmate the helpfulness and buld the helpfulness functon Revew Helpfulness Consumers publsh ther revews about products on the revew aggregaton web stes or the web communtes after they fnsh a transacton. They submt ther revews nto the web ste lke epnon.com to share wth other possble consumers. Consumers can vote revews as Helpful or Unhelpful after they read through them. Let C be the set of consumers, P be the set of Products, D be the set of revew documents, and V be the set of votes whch s consumers' opnon about revews (vote ncludes Helpful, Unhelpful ). Consumer C = { c1, c2, c3,, c M} Product P = { p1, p2, p3,, p N} Revew D = { d1, d2, d3,, d I } Vote V s the set of all votes defned as follows: We denote v c, d as the consumer c m 's opnon on revew document m d. It s formulated as: 1 f cm voted d as Helpful, or vc, d = m 0 f cm voted d as Unhelpful. (1) Revew helpfulness s the percepton that the revew d D can be used to assst the consumers to understand the product pn P. For a revew d, ts helpfulness can be calculated as the rato of the number of consumers voted d as Helpful to the total number of consumers who have voted for d. Let the number of all Helpful votes about revew d be denoted as v d. Let the total number of all Unhelpful votes about revew d be denoted as v d. We denote revew d 's helpfulness as: vd hd ( ) =. vd v d (2) The postve vote fracton of greater than 0.9 can be seen as true helpful and smaller than 0.1 can be seen as true unhelpful. The mean of the helpfulness value of the onlne revews whch have a helpfulness value between 0.1 and 0.9 s 0.6. Therefore, we predefne 0.60 as a threshold for helpful revews. If the revew's helpfulness s greater than 0.60, we say t s helpful. The onlne revew conssts of words, whch nclude opnon words, product features, product parts and other words. The mportance of each word to the helpfulness of the revew can be calculated from a tranng data whch contans the vote nformaton provded by prevous revew readers. In the followng subsecton, we defne the formulaton of helpfulness gan whch represents the mportance of word for the class of Helpful Helpfulness Gan Pang et al. [2002] classfed move revews as postve or negatve by several machne learnng methods and dscovered that the best result for classfyng revew documents as postve or negatve was obtaned by usng Boolean values of ungram features. We use the bag-of-words model to represent the documents and buld our language model. Each feature s a non-stop stemmed word and the value of ths feature s a Boolean value of the occurrence of the word on the revew. We ntroduce the Shannon's nformaton entropy concept [Shannon 2001] to measure the amount of nformaton n revews. For the onlne revew classfcaton problem, the entropy can be extended as follows: Let S = { s1, s2,, s q } be the set of categores n the revew space. The expected nformaton needed to classfy a revew s: m H( S) = Pr( s)log Pr( s) =1 (3) The average amount of nformaton contrbuted by a term t n a class s wll be: Page 222

4 Journal of Electronc Commerce Research, VOL 11, NO 3, 2010 m H( S t) = Pr( s t)log Pr( s t) =1 (4) Informaton Gan s derved from entropy and can be understood as the expected entropy reducton by knowng the exstence of a term t. G( t) = H( S) H( S t) (5) It s often employed as a term goodness crteron n the feld of machne learnng [Yang & Pedersen 1997] and s often used as a feature selecton method n text classfcaton. In [Yang and Pedersen 1997], nformaton gan of term t was extended and defned as follows: q q q G( t) = P ( s )log Pr( s ) Pr( t) Pr( s t)log Pr( s t) Pr( t ) Pr( s t )log Pr( s t ) r =1 =1 =1 Pr( s ) s the probablty of documents n category s among all documents Pr( t ) s the probablty of documents whch contan term t among all documents Pr( s ) t s the probablty of documents whch contan term t and whch s ncluded n category s out of all documents whch contan t Pr( s t ) s the probablty of documents whch do not contan term t and whch belongs to category s out of all documents whch do not contan t The above formula calculates the reducton of entropy by knowng the occurrence of a specfed term. It consders not only the term's occurrence, but also the term's non-occurrence. Ths value ndcates the term's contrbuton and predctng ablty. A word has hgher helpfulness gan whch means t has more contrbuton for classfcaton. For a bnary classfcaton, ths value can be used to measure the amount of contrbuton of term t to a s. class In our case, only two categores, Helpful and Unhelpful, s consdered. Let s 1 be Unhelpful and s 2 be Helpful. In order to provde the dfference of predcton ablty for two categores, we provde the formulaton of helpfulness gan whch represents a term's contrbuton amount to the class of Helpful revews. The average helpfulness values of revew documents where a term t occurs, denoted by h( D t ), s ntroduced as a factor to calculate the helpfulness gan of each term: f P(s 1 t )<P(s 2 t ) then gan( t ) = G( t )* h( D t ), otherwse gan( t ) = G( t )*(1 h( D t )). The helpfulness gan of a term t s calculated as: G( t )* h( D t ) f P( s1 t ) < P( s2 t ), gan( t ) = G( t )*(1 h( D t )) otherwse. (7) where s 1 s the category of Unhelpful, s 2 s the category of Helpful, and h( D t ) s the mean of helpfulness value of the revew documents where word t occurred. The helpfulness gan of term t, whch represents the mportance and the predcton ablty of words, s addressed by Eq. (7) Predcton Computaton From the dscusson n prevous secton we understand that helpfulness gan represents a words' ablty of correctly predctng a documents allocaton to the category of Helpful or Unhelpful revews. So, the summarzaton of the helpful gan of all words n a revew ndcates the revew's helpfulness. In our approach, the revew's content (words) wll be analyzed and the helpfulness gan wll be calculated for each word n product revews. In order to predct the helpfulness of a revew d, we propose the helpfulness score functon as follows: W score( d ) = gan( t )* f ( d, t ) =1 (8) Where gan(t ) s the th stemmed word's helpfulness gan and W s the number of stemmed non-stop words n revew r (6) Page 223

5 Zhang & Tran: Helpful or Unhelpful: A Lnear Approach for Rankng Product Revews 1 f term t occurs n d, or f ( d, t ) = 0 f term t does not occur n d. (9) Eq. (6) can be seen as the total helpful nformaton delvered by a revew document I and we we utlze ths functon to model the helpfulness value of revews. Ths value may be greater than 1, so we ntroduce a normalzaton factor to ensure that the calculated score value remans n the range of {0,1}. As a result, tuples of <d, score(d )> are returned from our algorthm. score(r ) s the revew d 's predcted helpfulness score. Fnally, onlne product revews wll be ranked based on ther correspondng score(d ) values. Revews wth hgher score values s more helpful than others. Wth a set T of tranng revews and a set T of test revews, the helpfulness predcton process s shown as follows: 1. Fnd the gan values for every non-stop word from T. 2. Calculate the helpfulness score for every revew of T by Eq. (8). 3. Normalze the helpfulness score. 4. Sort T n descendng order based on ther helpfulness score. Our approach also can be extended to a classfcaton system by addng two more steps: 1. Fnd the gan values for every non-stop word from T. 2. Calculate the helpfulness score for each revew n T and select a helpfulness threshold for classfcaton. 3. Calculate the helpfulness score for every revew n T by Eq. (8). 4. Normalze the helpfulness score. 5. Sort T n descendng order based on ther scores. 6. Classfy revew document as Helpful or Unhelpful based on whether the score of a document s greater than the threshold. 4. Expermental Evaluaton In ths secton, we frst ntroduce the evaluaton method used n our experments whch utlzes log-lkelhood as the measurement to evaluate the performance of the helpfulness assessment for revews. Then we descrbe the data set and the expermental steps. At the end, we analyze the expermental results and evaluate the performance of our approach Evaluaton Precson, recall, F-score [Rsbergen 1979] are commonly used n evaluatng nformaton retreval systems. Precson s defned as the rato of retreved helpful revews to the total number of revew retreved. Recall s defned as the rato of the number of retreved helpful revews to the total number of helpful revews. F-score s defned as the harmonc mean of above two measures and s calculated by: 2* Precson* Recall F = Precson Recall (10) When analyzng the predctve success of helpfulness assessment algorthms, there exst varous accepted evaluaton metrcs. Correlaton coeffcent between the predcted helpfulness rankng and the observed helpfulness rankng s often used to evaluate the performance of helpfulness assessng models. Also, other evaluaton metrcs commonly used n learnng rankng area, such as precson at poston n, mean average precson and normalzed dscount cumulatve Gan, can be used to evaluate the helpfulness assessng systems. These evaluaton metrcs do not take the voters populaton sze nto account. In order to precsely ustfy the goodness-of-ft and the rankng performance of our model, we brng forward the log-lkelhood of voter's opnon as Helpful to evaluate the dfference between the predcted helpfulness score and the real observed helpfulness value from the data set. Wth respect of a fxed set of voters whch can be observed n the tranng set, we can assume the helpfulness value s a fxed number of the data set. Gven a fxed helpfulness value, f we take a voter c m randomly from the voters' populaton on a specfc revew document d, the vote or opnon of ths randomly selected voter about the revew document d can be modeled as an ndependent, dentcally dstrbuted (..d.) Bernoull random varable governed by the populaton parameter and the dstrbuton of voter opnons can be formulzed as: h( d ), f vc, = 1; m d pv ( c, ) = m d 1 h ( d ), else. (11) Page 224

6 Journal of Electronc Commerce Research, VOL 11, NO 3, 2010 Then the probablty of a voter wll vote Helpful for a revew document d s * ( = 1) = ( ) d v v (1 ( )) d p v h d h d. Therefore, we can use ths value as the benchmark to compare wth the cm, d pv ( = 1) of predcted helpfulness score: cm, d vd cm, d ( =1) = ( ) (1 ( )) v d p v score d score d, (12) where score(d ) s the predcted helpfulness score. In the followng sectons, we nvestgate the performance of our algorthm by comparng the log-lkelhood of the observed and predcted voters' opnon Data set Our experments focus on the product categores of dgtal cameras. We crawled 7054 dgtal camera revews from Amazon.com, of whch 1468 revew documents have been evaluated by at least 5 consumers as helpful or unhelpful. We delete a lst of 571 stop words [Buckley 1985] and use the bag-of-word model to represent text and buld our language model. We apply Porter Stemmng algorthm [Porter 1980] to all the words n our data set and each feature s a non-stop stemmed word and the value of ths feature s a Boolean value of the occurrence of the word on the revew. After the parsng and stemmng to all the revews, a document term matrx s returned assocated wth the helpfulness value of each document. In ths data set, non-stop stemmed words are detected. We make use of 1468 revew documents whch have been voted by more than 5 voters and make use of the vocabulary of words or terms as the feature set of the model to buld a document-term matrx and each the matrx has been normalzed to zero-mean. We defne that f the helpfulness of a revew (percentage of helpful votes) s greater than 60%, the revew wll be marked as Helpful, otherwse t s Unhelpful. We randomly select 600 dgtal camera revews to execute the experments. We use 10-fold cross valdaton to evaluate our approach. Revews are randomly dvded nto 10 equal-szed folds, of whch 9 folds of revews are used for tranng the model and one fold used as test data Results and Analyss We choose the helpfulness score of the V + th (the number of helpful revews n the tranng set) sorted revew as the helpfulness whch s used to classfy revews. Fgure 1 and fgure 2 show the resulted helpfulness score of the tranng set and the testng set from one of the 10-folds evaluatons. Fgure 1 shows the tranng revews' helpfulness scores resulted from our algorthm. Fgure 1: Score Values of Revews n the Tranng Set Page 225

7 Zhang & Tran: Helpful or Unhelpful: A Lnear Approach for Rankng Product Revews Clearly, most of the score values of the helpful data are greater than the threshold and most of the scores of the unhelpful revew documents are smaller than the threshold. In other words, most of the Helpful revews have larger helpfulness scores than Unhelpful. Ths result hghly ndcates that the helpfulness score functon can model the helpfulness of onlne product revews. So, wth the rankng of helpfulness scores, most of the helpful revews can be retreved on the top of the sorted revews' lst. Fgure 2 shows the score values of the testng data whch contans 30 Helpful revews and 30 Unhelpful revews. It llustrates that most of the testng Helpful revews' predcted helpfulness scores are greater than the threshold whch s learned from the tranng set, and most of the testng Unhelpful revews' predcted helpfulness scores are smaller than the threshold. Ths clear dstncton ndcates that our approach can correctly group the product revews nto Helpful and Unhelpful. In order to ustfy how the predcted helpfulness score dffers from the true helpfulness value of the data set and * the goodness-of-ft of our model, we frst sort revew documents wth decreasng log-lkelhood of p ( v = 1), then we plot the log-lkelhood of * p ( vc, 1) m d from the data set and the log-lkelhood of cm, d cm, d pv ( = 1) resulted from our model for each revew n the sorted revews lst. Fgure 3 shows the performance of our algorthm workng wth Helpful Revews, and Fgure 4 shows the performance of our algorthm dealng wth Unhelpful Revews. These two fgures are acheved from one of the 10-fold cross valdatons whch ncludes 540 tranng revew documents (270 Helpful revews and 270 Unhelpful revews) and 60 testng revew documents (30 Helpful revews and 30 Unhelpful revews) n each fold to evaluate our approach. The expermental results of the Helpful and Unhelpful revews have the same pattern and exhbt the same decreasng trend as the orgnal helpfulness value of the data set. It hghly ndcates that our approach can effectvely predct the helpfulness of the revew documents. Fgure 2: Score Values of Revews n the Testng Set Page 226

8 Journal of Electronc Commerce Research, VOL 11, NO 3, 2010 Fgure 3: Log-lkelhood of Postve Revews Fgure 4: Log-lkelhood of Negatve Revews Table 1 shows the classfcaton performance of our model. We use the 10-fold cross valdaton, whch makes our results less prone to random varaton, and the classfcaton precson of our model s 76.7% for Helpful revews and 73.7% for Unhelpful revews. Precson, recall, and F-score of Nave Bayes, Decson Tree, SMO and our model s compared n Table 1. For both the Helpful and Unhelpful revews, the precson and recall of our Page 227

9 Zhang & Tran: Helpful or Unhelpful: A Lnear Approach for Rankng Product Revews approach outperforms Nave Bayes and Decson Tree. In comparson wth SMO, the recall of our approach s 3.7% hgher than the recall of Helpful revews and the precson s 3.3% lower than the precson of SMO method. The expermental result led us to conclude that the classfcaton capablty our model performs better or at least the same as other commonly used classfers. Table 1: Performance of varous classfcaton methods and our model (10-fold cross-valdaton) Precson Recall F-measure Helpful Unhelpful Helpful Unhelpful Helpful Unhelpful Nave Bayes SMO DecsonTree Our Model Dscusson The model dscussed n ths paper analyzes the helpfulness gan of each word from the tranng set whch s calculated from the nformaton gan of a term and the average helpfulness value of the revew documents where ths word occurs. Wth ths helpfulness gan, features' contrbuton to the class of Helpful s dscovered. The emprcal results show that our model performs very well for our data set of Amazon.com. We ntroduce the log-lkelhood functon of the voter's opnon as a metrc to evaluate the performance of helpfulness assessment n ths paper. We compare the predcted helpfulness score and the helpfulness value of the data set by ths evaluaton measurement. The expermental results show that the predcted helpfulness score calculated by our algorthm consstently follow the changng trend of helpfulness value of the data set. Wth the help of our approach, consumers can easly assess the Helpful revews and don't have to spend effort searchng by themselves. In comparson wth other classfcaton approaches, the evaluaton results ndcate that our approach performs better or the same as other common used machne learnng methods. However, we make use of a lnear summarzaton approach to model the helpfulness of revews whch means that the tme complexty of our approach s lnear and s lower than other machne learnng approaches. It has been observed that the precson of our model s better for Helpful revew documents than Unhelpful ones. The most lkely reason s that our algorthm fnd more terms wth hgher gan value n the helpful category than n the unhelpful one. In ths paper, the experments were run for helpfulness dscovery usng only ungrams. Word n-gram based text representaton can also be used to represent onlne revew documents. The performance of nvolvng word n-gram features should be mproved by nvolvng more features. The confguraton of our algorthm s very smple, and other corpora can be easly used for the helpfulness evaluaton. The threshold we predefned for separatng Helpful and Unhelpful s 0.6. Ths value wll affect the performance of our model applyng to other categores of onlne product revews. A threshold choosng measurement, as to acheve a maxmzed value of F-measure, may be ntroduced to balance the classfcaton performance. Many onlne shoppng web stes provde facltes for consumers to share ther experence and revew product. Ths forms a huge nformaton source for onlne users and t becomes more and more dffcult to easly compare revews and make decsons. Our model can serve as a revew recommendaton system to provde the most helpful revews to potental consumers. Moreover, not only product revews but also other sources of product and servce related nformaton such as users' ratngs, opnons and comments that can be obtaned from dfferent onlne user clubs, communtes or forums across the Internet can be ftted as the nput to our model; and ether useful revews (or smlar nformaton), or products/servces and vendors are the output of our model. Obvously, a recommender system that ncorporates the functon of recommendng useful revews would be much more helpful, convenent and surely attract more potental buyers. 6. Concluson and Future Work Ths paper proposes a model for modelng the helpfulness of onlne product revews based on the ungram features. Usng our model, onlne product revews can be classfed and ranked based on the score values and the most helpful revews can be dentfed. Ths helps consumers complete ther nformaton search and make purchase decsons easly and quck. We utlze the helpfulness gan, whch represents the classfcaton capablty of each term, to model the helpfulness value of revews. In comparson wth other helpfulness assessment methods, our model s smpler, easer to mplement and more understandable. The lnear-tme complexty of our model ndcates Page 228

10 Journal of Electronc Commerce Research, VOL 11, NO 3, 2010 the helpfulness model can be learned quckly. We also proposed a new evaluaton measurement to udge the performance of helpfulness assessment algorthms whch not only compares the dfference between the observed and predcted values, but also takes the voters' populaton sze nto account. Moreover, the expermental results show that our approach acheves good performance n rankng and classfyng revew documents. In ths paper, we have assumed that all consumers have the same preferences for onlne revews and do not consder the dfference between ndvduals. In the future research, n order to mprove the accuracy of personalzaton, the smlarty of voters should be consdered n our model and more data would be collected to examne the generalzaton power of our model. Our ntended future research would also take other feature sets, whch may affect the qualty of revews, nto consderaton. These nclude such features as when a revew was publshed, how the consumers rated the product, and the number of features whch were mentoned n the revew. In our proposed model, we dd not detect the ntellgent spammng revews whch only consst of the words whch have hgh gan values. We are gong to nclude ths functon n our future works. The expermental results led us to conclude that our approach s able to effcently predct the helpfulness of specfc categores of revew documents. Approaches that dstngush relevant and non-relevant tems [Butler et. al 2001] mght help us to extend our model to a more general case that can dscover a helpful onlne revew whch s relevant to the tem whch users are nterested n. REFERENCES Buckley, C., Implementaton of the Smart Informaton Retreval System, Techncal Report TR85-686, Dept. of Computer Scence, Cornell Unv., May Butler, J., D.J. Morrce, P.W. Mullarkey, A multple attrbute utlty theory approach to rankng and selecton, In: Management Scence 47(6), pp Goldberg, D., D. Nchols, B.M. Ok, and D. Terry, Usng collaboratve flterng to weave an nformaton tapestry, Communcaton. ACM, 35(12): Hatzvassloglou, V. and K. R. McKeown, Predctng the semantc orentaton of adectves, In Proceedngs of the eghth conference on European chapter of the Assocaton for Computatonal Lngustcs, pages , Morrstown, NJ, USA. Assocaton for Computatonal Lngustcs Hu, M. and B. Lu, Mnng and summarzng customer revews. In KDD '04: Proceedngs of the tenth ACM SIGKDD nternatonal conference on Knowledge dscovery and data mnng, pages , New York, NY, USA. ACM Km, S.M., P. Pantel, T. Chklovsk, and M. Pennacchott. Automatcally assessng revew helpfulness, In Proceedngs of the 2006 Conference on Emprcal Methods n Natural Language Processng, pages , Sydney, Australa. Assocaton for Computatonal Lngustcs Lee, J., D.H. Park, and T. Han, The effect of negatve onlne consumer revews on product atttude: An nformaton processng vew. Electronc Commerce Research and Applcatons 7, 3, Lu, Y., X. Huang, A. An, and X. Yu, Modelng and predctng the helpfulness of onlne revews, In ICDM '08: Proceedngs of the 2008 Eghth IEEE Internatonal Conference on Data Mnng, pages , Washngton, DC, USA. IEEE Computer Socety Hu, N., L. L. and Zhang, J. J. Do onlne revews affect product sales? Informaton Technology and Management Orkn, M. and Drogn, R. Vtal Statstcs, McGraw-Hll, 1990 Pang, B., Lee, L., and Vathyanathan, S. Thumbs up? sentment classfcaton usng machne learnng technques, In EMNLP '02: Proceedngs of the ACL-02 conference on Emprcal methods n natural language processng, pages 79-86, Morrstown, NJ, USA. Assocaton for Computatonal Lngustcs Park, D.H. and S. Km, The effects of consumer knowledge on message processng of electronc word-of-mouth va onlne consumer revews., Electronc Commerce Research and Applcatons 7, 4, Park, D.H., J. Lee, and I. Han, The effect of on-lne consumer revews on consumer purchasng ntenton: The moderatng role of nvolvement, Int. J. Electron. Commerce, 11(4): Porter, M.F., An algorthm for suffx strppng, Program, 14(3) pp Resnck, P., N. Iacovou, M. Suchak, P. Bergstrom, and J. Redl, Grouplens: an open archtecture for collaboratve flterng of netnews, In CSCW '94: Proceedngs of the 1994 ACM conference on Computer supported cooperatve work, pages , New York, NY, USA. ACM Press Sarwar, B. M., G. Karyps, J.A. Konstan, and J. Redl, Item-based collaboratve flterng recommendaton algorthms. In World Wde Web, pages Shannon, C. E. A mathematcal theory of communcaton, SIGMOBILE Mob.Comput. Commun. Rev., 5(1): Page 229

11 Zhang & Tran: Helpful or Unhelpful: A Lnear Approach for Rankng Product Revews Turney, P. D. Thumbs up or thumbs down? semantc orentaton appled to unsupervsed classfcaton of revews, In ACL '02: Proceedngs of the 40th Annual Meetng on Assocaton for Computatonal Lngustcs, pages , Morrstown, NJ, USA. Assocaton for Computatonal Lngustcs Rsbergen, C. J. Informaton Retreval, Dept. of Computer Scence, Unversty of Glasgow, Second edton Vermeulen, I.E. and D. Seegers, Tred and tested: The mpact of onlne hotel revews on consumer consderaton, Toursm Management 30, 1, Wemer, M and I. Gurevych, Predctng the perceved qualty of web forum posts, Proceedngs of the Conference on Recent Advances n Natural Language Processng (RANLP). 2007a Wemer, M., I. Gurevych, and M. Muhlhauser, Automatcally assessng the post qualty n onlne dscussons on software, In Proceedngs of the 45th Annual Meetng of the Assocaton for Computatonal Lngustcs Companon Volume Proceedngs of the Demo and Poster Sessons, pages , Prague, Czech Republc. Assocaton for Computatonal Lngustcs. 2007b Yang, Y. and J.O. Pedersen, A comparatve study on feature selecton n text categorzaton, In ICML '97: Proceedngs of the Fourteenth Internatonal Conference on Machne Learnng, pages , San Francsco, CA, USA. Morgan Kaufmann Publshers Inc Yu, H. and V. Hatzvassloglou, Towards answerng opnon questons: separatng facts from opnons and dentfyng the polarty of opnon sentences, In Proceedngs of the 2003 conference on Emprcal methods n natural language processng, pages , Morrstown, NJ, USA. Assocaton for Computatonal Lngustcs Zhuang, L., F. Jng, and X.Y. Zhu, Move revew mnng and summarzaton, In CIKM '06: Proceedngs of the 15th ACM nternatonal conference on Informaton and knowledge management, pages 43-50, New York, NY, USA. ACM.2006 Page 230