Case-Studies in Association Rule Mining for Recommender Systems

Transcription

1 Case-Studies in Association Rule Mining for Recommender Systems Barry Smyth, Kevin McCarthy, James Reilly, Derry O Sullivan and Lorraine McGinty Smart Media Institute, Department of Computer Science, University College Dublin (UCD), Dublin, Ireland barry.smyth@ucd.ie David C. Wilson Department of Software and Information Systems University Of North Carolina at Charlotte, USA davils@uncc.edu Abstract Recommender systems combine ideas from information retrieval, machine learning and user profiling research in order to provide end-users with more proactive and personalized information retrieval applications. Two popular approaches have come to dominate. Content-based techniques leverage the availability of rich item descriptions to identify new items that are similar to those that a user has liked in the past. In contrast, collaborative filtering techniques rely on the availability of user profiles in which sets of items have been rated. They recommend new items to a target user on the basis that similar users have preferred these items in the past. In this paper we will present two casestudies of how association rule mining techniques have been used to significantly enhance the power of content-based and collaborative filtering recommender systems. I. INTRODUCTION We live in an age of information that provides for an unprecedented level of access to all forms of electronic data, products and services. However, users are finding it increasingly difficult to locate the right information at the right time and while Web search engines serve as our primary information access tool we are frequently frustrated by their inability to accurately capture our information retrieval needs [?], [?]. Over the past number of years the field of recommender systems has emerged with the stated goal of providing more proactive, intelligent and personalized information retrieval systems that are better able to cater for the needs of users in a variety of circumstances [?], [?]. Recommender systems combine ideas from artificial intelligence, machine learning, information retrieval and user profiling and have been applied successfully in a variety of domains including e-commerce and online shopping; see for e.g. [?], [?]. Over time two particular approaches to recommendation have emerged to dominate research and development. Content-based and casebased recommenders adopt a traditional information retrieval stance, relying on feature-based descriptions of information items as the basis for recommendation [?], [?], [?]. A typical content-based recommender selects items for recommendation on the grounds that they are similar to items that the user has liked in the past. In contrast, collaborative filtering approaches to recommendation adopt a very different standpoint, under the assumption that content-based descriptions may not be available. Instead they rely on ratings-based user profiles profiles containing items that the user has previously rated according to their appeal or relevance. Recommendations are produced by locating users with similar ratings histories and by selecting items from these profiles that have been highly rated but that are absent from the target user s profile [?], [?], [?]. In our research we have developed a wide range of recommender systems that include content-based and collaborative filtering aspects (e.g. [?], [?], [?], [?]). On their own the content-based and collaborative approaches suffer from a number of drawbacks, but taken together they can form the basis of a powerful hybrid recommendation strategy [?]. In recent years we have investigated how data-mining techniques may be able to improve the performance of content-based, collaborative and hybrid recommender systems [?], [?], [?]. In this paper we describe two case-studies from this work, where we have used association rule mining techniques to improve the performance of collaborative filtering and content-based recommenders. In Section II we describe how the well-

2 known sparsity problem in collaborative filtering can be addressed by mining similarity knowledge from ratingsbased profiles. In Section III we describe how similar techniques can be used in a content-based recommender in order to discover feature patterns that may help the user to refine his/her recommendations. II. CASE-STUDY 1: MINING SIMILARITY KNOWLEDGE In this case-study we are interested in ways of improving collaborative filtering recommendation techniques by enhancing their rudimentary notions of similarity. Collaborative filtering algorithms make use of simple ratings-based profiles and generate recommendations on the basis of similarity between the target user and neighbouring profiles. However, the standard approach to similarity used in collaborative filtering systems relies heavily on direct overlap between ratings, which is problematic due to the so-called sparsity problem. The sparsity problem tells us that, on average, two users are unlikely to have rated many of the same items and so there will be little direct overlap between their profiles. This is problematic when it comes to identifying similar profiles because it means that there is no direct way to measure the similarity between two profile cases unless we have access to similarity knowledge that allows us to compare non-identical profile items. Unfortunately this similarity knowledge is usually hard to come-by and in this case-study we report on the results of an investigation into the use of association rule mining techniques in order to discover this type of knowledge. A. Learning Item-Item Similarity Knowledge There are many automated techniques that could be used to derive various sorts of similarity knowledge. The approach we have chosen is to apply data mining techniques, in particular the Apriori algorithm [?], to extract association rules between items in user-profile cases. To illustrate this we will use examples from the PTV system. PTV is a recommender system designed to make TV program suggestions to users based on their learned viewing preferences; PTV profiles contain lists of positively and negatively rated TV programs [?]. By discovering relationships between programs beyond simple direct overlap, we may be able to cover more potential profile matches and so make more informed recommendations. For example, a person that likes X-Files and Frasier would not normally be comparable to a person that likes Friends and ER, but Rule Support Confidence Friends Frasier 12% 25% Friends ER 14% 37% Frasier ER 10% 22% TABLE I EXAMPLE RULES Friends Frasier ER Friends Frasier ER TABLE II EXAMPLE RULE SIMILARITY MATRIX discovering a relationship between Frasier and Friends would provide a basis for profile matching. Treating user profiles as transactions and the program ratings therein as itemsets, the Apriori algorithm can be used to derive a set of rules and associated confidence levels between programs. Table I shows some example rules that were generated by running Apriori on our PTV data set. The confidence values are taken as similarity scores and used to fill in a program similarity matrix, as shown in Table II. While the Apriori approach is capable of generating rules with multiple antecedents and consequents, we have limited the initial phase of the work to rules with single-program antecedents and consequents. The derivation of direct program similarity information naturally suggests two extensions to further elaborate the similarity matrix. First, the generated rules can be chained together to provide indirect relationships between programs. In doing so, a choice has to be made as to how to calculate the indirect rule confidence; for example we could compute the minimum or maximum of a sequence of similarities and in the evaluation section below we consider a range of alternative combination methods. Second, it seems intuitive to think that a rule of the form likes Friends likes Frasier could imply the reverse. If it were possible to establish this symmetry the additional coverage could be doubled. B. Enhanced Profile Similarity The availability of item similarity knowledge facilitates a new type of similarity-based recommendation strategy that combines elements from case-based (or content-based) and collaborative recommendation techniques. It facilitates the use of more sophisticated similarity metrics on ratings-based profile data, which in turn makes it possible to leverage indirect similarities between profile cases, and so generate improved recommendation lists.

3 This new recommendation strategy consists of two basic steps: 1) The target profile, t is compared to each profile case, s, to select the k most similar cases. 2) The items contained within these selected cases (but absent in the target profile) are ranked according to the relevance to the target, and the r most similar items are returned as recommendations 1) Profile Matching: The profile similarity metric is presented in Equation 1 as the weighted-sum of the similarities between the items in the target and source profile cases (see also Equation 2). In the situation where there is a direct correspondence between an item in the source, s i, and the target, t j, then maximal similarity is assumed. However, the nature of ratings-based profile cases is such that these direct correspondences are rare and in such situations the similarity value of the source profile item is computed as the mean similarity between this item and the k most similar items in the target profile case (t 1...t k ). P Sim(t, s, k) = s iɛs w i ISim(t, s i, k) (1) ISim(t, s i, k) = 1 if t j = s i (2) j=1..k = sim(t j, s i) (3) k Notice, that if k = 1 and there is a perfect one-toone correspondence between the target and source profile cases, then this profile similarity metric is equivalent to the traditional weighted-sum similarity metric used in many case-based and content-based recommenders. However, when there is no such correspondence, or when certain pairs of target-source items have no similarity value, then this metric is still capable of calculating an overall similarity score. 2) Recommendation Ranking: Once the most similar profile cases (S) to the target have been identified a set of ranked item recommendations can be produced. There are three factors to consider when ranking these recommendations. First, we want to give priority to those items that have a high similarity to the target profile case. Second, items that occur in many of the retrieved profile cases should be preferred to those that occur in few profile cases. Finally, items that are recommended by profiles that are similar to the target should be preferred to items that are recommended by less similar profiles. Accordingly we compute the relevance of an item, s i from a retrieved profile case, s, with respect to the target profile, t, as shown in Equation 4; where S S is the set of retrieved profile cases that contain s i. Rel(s i, t, S) = ISim(s i, t, k) S S C. Evaluation sɛs P Sim(s, t) (4) In order to evaluate our approach to mining and applying similarity knowledge, we conducted a series of experiments using real user-data from 622 PTV customer profiles. We were particularly interested in the increased similarity coverage offered by our data mining technique and the contribution that this new similarity knowledge had when it came to making high quality recommendations. For reasons of space we can only briefly report on our evaluation and the results below represent just a fraction of the evaluation work that has been carried out. For more information the interested reader is referred to [?], [?] 1) Recommendation Quality: In this experiment we test how this new similarity knowledge impacts on recommendation quality. After generating direct and indirect similarity rules, a profile case is selected from the case-base. A parameterized percentage of the items in the selected case are removed from consideration. The remainder of the profile case is then used as the basis for retrieval, using our similarity metric and recommendation ranking. Accuracy was calculated using three metrics: M 1: M 2: M 3: Percentage of removed items that were recommended in each case. Percentage of profiles in which at least one removed item was recommended. A combination of the above: percentage of removed items that were recommended, given a recommendation could be made. Metrics 1 and 2 give us an overall sense of recommendation performance whereas metric 3 provides a more localised account of performance. Hence we were more concerned about the performance of the first 2 metrics in these experiments as they give us a better sense of model accuracy. We were most interested in looking for uplift in recommendation quality when comparing our new recommendation technique (using direct and indirect similarity rules) to a pure collaborative filtering technique. We compared a number of different variations of our new technique against standard collaborative filtering benchmarks. Our variations included a version that makes use

4 Fig PTV recommendation accuracy. user to provide information on a specific product feature. On common form of feedback is called critiquing: the user indicates a preference over a particular feature of a recommended item. For example, when shopping for a PC a user might indicate that they like the current suggestion but they are looking for something cheaper ; cheaper is a critique over the price feature of the PC case. Critiques were originally proposed by the wellknown FindMe recommender systems [?] and we will focus on the use of critiquing in this case-study. Sometimes it is useful to critique multiple features (compound critiques). For example, a PC shopper may ask for a more powerful PC, meaning that they are looking for a faster processor, more memory and a larger hard-disk; that is, 3 individual (unit) critiques are encoded in parallel. In this case-study we describe of direct similarity rules only (Direct), plus a number of versions that made use of indirect rules whose similarities.3 scores Similarity werecoverage combined using a variety of combination Increasing recent work investigating how a recommender can au- n the next set methods of experiments, (Max, Min, we Average, were interested Mult). inthe evaluating two standard the degreetomatically which discover useful compound critiques during imilarity coverage benchmark couldcollaborative be improved byfiltering the generated techniques rule sets. (denoted For the first theexperi- recommendation session and how these critiques ent, the density CF and of the CFRR) generated encoded item-item a standard similarity collaborative matrix wasfiltering taken our can measure be used to improve recommendation efficiency. We f similarityapproach; coverage. CF We produces varied Apriori an unranked confidence setlevels of recommenda- each whereas run we generated CFRR ranks the Apriori recommendation direct rule set, according as well as a maximal from 40% to 5% willatassume 5% a conversational recommender system in ntervals. Ontions, the image of the FindMe recommender systems [?]. ndirect ruletoset the bydegree chaining ofdirect similarity rules of andtheir filledsource in the profiles item similarity to the matrix, takng the matrix density relative to the items that participated in the rule set. Results for Each recommendation session is initiated by an initial target [?], [?]. TV showed direct item similarities provide an average of 10% coverage, but user there query is a and this results in the retrieval of the most On the PTV dataset, results of the first metric showed arked increase for the indirect similarity rules (65% coverage in the best case). similar product case (the recommended case) and a Of course, that the using Apriori direct method rules does (20%) notoutperformed generate association indirect rules rules for every set ofpro- le programme, (18%), andwhich in factin Apriori turn ignores outperformed a great many pure programmes CF (8%). Abecause compound their critiques. The user will have the opportunity critiques, both fixed unit critiques and dynamic requency fails similar the Apriori patternthreshold was seen tests. in Nevertheless metric 2, although when wemetric add these 3 newly to accept genrated this case, thereby ending the recommendation rules showed to the full pure item-item CF tomatrix, have awehigher found accuracy an increase(30%) density thanfromsession, 0.6% toor to critique it. If they critique the case, the.6%. Again, either a similar the pattern direct or wasthe found indirect themethods Físchlár (.17% (Direct: to.65%) 24%, dataset critique [11] acts as a filter over the remaining cases, such s well as Combined Indirect: (.04% 22%) to (Figure.26%), 1). Browse On the (.044% faceto of.29%), it these Record results (.05% that to.14%) the case chosen for the next cycle is the one that nd Play (.06% suggest to.17%). that the learned similarity knowledge is proving is compatible with the critique and maximally similar to to be useful from a recommendation standpoint, at least the previously recommended case. In this section we will.4 Recommendation in relation Quality to the critical : Accuracy performance metrics, metric describe how dynamic critiques can be discovered and ith encouraging 1 and results 2. These for increasing results also the similarity suggest that coverage, the indirect we designed selected experi-foents to test similarity the effect of knowledge this new similarity is, as expected, knowledgepoorer on recommendation quality cycle. quality. presentation as part of each recommendation ased on our than earlier the direct experiments, similarity we chose knowledge. representative These performance confidence levels of 10% or the PTV, benefits Físchlár, have Playalso and been Record found datasets in aand variety 30% of onother Browse datasets and Combined A. Critique Patterns these [?]. specified levels, we have shown an item-item matrix coverage In order to discover a useful set of compound critiques atasets. Using ncrease when moving from CF to both direct and indirect rule approaches. from a set of product cases we need to first understand The finaliii. experiment CASE-STUDY tested the2: accuracy MINING ofcompound recommendation CRITIQUES on both datasets. howafter these cases relate to the current recommendation enerating direct In and our indirect second similarity case-study rules, we a profile will case look is at selected a form from we the need caseto understand their basic feature differences of content-based recommendation called conversational so that we may then look for frequently occurring recommendation. Conversational recommenders guide a sets of differences to act as our compound critiques. user through a sequence of iterations, suggesting specific The first step is to generate a set of so-called critique items, and using feedback from users to refine their patterns from the remaining cases. Each remaining case suggestions in subsequent iterations [?], [?], [?], [?]. is compared to the current recommended case and the Different recommender systems look for different types relative feature differences make up the critique pattern. of feedback from users; for example, they might ask the Figure 2 illustrates what we mean with the aid of an user to rate a set of suggestions or they might ask the example. It shows the current case that has been selected

5 Fig. 2. Critique patterns capture the basic feature differences that exist between a current case and a recommendation candidate. for recommendation to the user as part of the current cycle and also a case, c, from the case-base. The current case describes a 1.4GHz, desktop PC with 512Mb of RAM, a 14 monitor and a 40Gb hard-drive, all for 1500 euro. Case c describes a 900MHz, desktop with 512MB or RAM, a 12 monitor and a 30Gb hard-drive for 3000 euro. The resulting critique pattern reflects how case c differs from current case in terms of individual feature critiques. For example, the critique pattern shown includes a < critique for processor speed we will refer to this as [Speed <] because the comparison case has a slower processor than the current recommended case. Similarly, the pattern includes the critique [P rice >] because the comparison case is more expensive than the current case. B. Learning Compound Critiques To identify compound critiques we must look for recurring patterns within a set of critique patterns. This bears a strong resemblance to so-called market-basket analysis problem, where the task is to find regularities in the shopping behaviour of customers [?] by identifying sets of products that tend to be purchased together: each critique pattern is equivalent to the shopping basket for a single customer, and the individual critiques correspond to the items in this basket. The combinatorics of a typical task domain thousands of products and customers make this a challenging problem, leading to an explosion in the number of possible groups of recurring items. However, it is not so acute in our critiquing scenario because there are only a limited number of possible critiques. For instance, each numeric feature can have a < or a > critique and each nominal feature can have a = or a! = critique, so there are only 2n possible critiques in a case-base where the cases are made up of n individual features. The Apriori algorithm [?] can be used to efficiently locate compound critiques. It characterises recurring item subsets as association rules of the form A B: from the presence of a certain set of critiques (A) one can infer the presence of certain other critiques (B). For example, one might learn that from the presence of the critique, [P rice <], we can infer the presence of [Memory <] with a high degree of probability; in other words the pattern {[P rice <],[M emory <]} is commonplace. Our proposal is to use Apriori, during each recommendation cycle, to generate a collection of compound critiques (frequent itemsets over the pattern-base), and to then select a small subset of the best of these compound critiques for presentation to the user to complement the standard unit critiques. C. Rating Compound Critiques From a user interface and availability viewpoint it is obviously not practical to present large numbers of different compound critiques, as feedback options, to the user during each cycle. And yet large numbers of compound critiques are likely to be discovered. A filtering strategy is needed so that we can select the most useful critiques, say the top 5, for presentation purposes. One option is to use the support value of the critique as the basis for filtering. For example, by selecting critiques with low support values we will be selecting critiques that are capable of eliminating many cases, but in doing so these critiques are less likely to be chosen by the user because they are less likely to lead to their desired target case. Conversely we could select critiques with high support values, on the grounds that they are likely to lead to the target case; however they are unlikely to eliminate many cases because high support critiques are, by definition, compatible with many remaining cases. In the next section we will evaluate these two strategies with respect to their ability to improve recommendation performance. D. Evaluation In this experiment we will investigate how our approach to compound critiquing impacts on recommendation performance compared to a standard critique-based recommender that uses fixed unit critiques. We will use the well-known standard PC dataset [?] as the basis for this evaluation. It consists of 120 PC product cases, each described in terms of 8 features including type, manufacturer, processor, memory, etc; this dataset is available for download at Once again we are limited by the level of detail that we can provide and the interested reader is referred to [?] for further information.

6 1) Methodology: We examine three compound critiquing variations, each distinguished according to how the set of top 5 critiques is chosen for presentation to the user during the current cycle: (1) LS - the top 5 critiques with the lowest support are chosen; (2) HS - the top 5 critiques with the highest support are chosen; (3) RAND - A random set of 5 critiques are chosen. So the above provide three system variations based on compound critiquing and in addition we also include a standard, unit critiquing approach (STD) as a benchmark. Ideally we would like to have carried out an online evaluation with live-users, but unfortunately this was not possible. In the alternative we chose to opt for an offline evaluation and used the approach described by [?], [?], [?]. Accordingly, each case (base) in the casebase is temporarily removed and used in two ways. First, it serves as the basis for a set of queries by taking random subsets of its features. We focus on subsets of 1, 3 and 5 features to allow us to distinguish between hard, moderate and easy queries, respectively. Second, we select the case that is most similar to each original base. These cases are the recommendation targets for the experiments. Thus, the base represents the ideal query for a user, the generated query is the initial query that the user provides to the recommender, and the target is the best available case for the user. Each generated query is a test problem for the recommender, and in each recommendation cycle the user picks a critique that is compatible with the known target case; that is, a critique that, when applied to the remaining cases, results in the target case being left in the filtered set of cases. In a typical cycle there may be a number of critiques (unit and compound) that satisfy this condition and the actual one chosen by the user depends on the system being used: LS picks the compound critique with the lowest support, HS picks the one with the highest support, and RAND picks a random critique. Each leave-one-out pass through the case-base is repeated (30 times for the PC dataset) and recommendation sessions terminate when the target case is returned. 2) Results: Ultimately we are interested in how the use of compound critiques affects recommendation efficiency; that is, the number of cycles before the target case is returned to the user. We are looking for an efficiency improvement over the standard critiquing approach (STD). If this is to be achieved then the user will have to select compound critiques frequently, during the course of a recommendation session, and when selected these critiques must be able to filter out more cases than the alternative unit critique. To test this we run our leave- Fig. 3. Average session length by query size for the PC domain. one-out test for each dataset and algorithm combination, noting the length of each recommendation session for each query. Figure 3 presents average session lengths for each algorithm, separating out queries of different lengths (difficulty levels). The idea here is to understand the performance implications as initial user queries become more complete; queries that contain only a single feature are likely to be vague and difficult to satisfy compared to more complete queries that contain up to 5 features. The results are very clear, pointing to a significant advantage for LS, but little or no advantage for HS. As expected the average session length falls with increasing initial query length but it is interesting to note that the scale of any advantage due to dynamic critiquing appears to be greater for the more difficult (smaller) initial queries. For example, for difficult queries (with a single specified feature), we find that LS enjoys a session length reduction of just over 40%, relative to STD. This falls to almost 32% for the moderate queries (containing 3 specified features) and falls again to just under 26% for the relatively easy queries with 5 fully specified features. This is to be expected perhaps since the easier queries naturally result in shorter sessions and thus there are fewer opportunities for compound critiques to be chosen, and hence fewer opportunities for their benefit to be felt. It is worth noting that similar results have been reproduced in other product domains. IV. CONCLUSIONS Recommender systems are an important tool in the quest to provide users with improved information access facilities. And while recommender systems have enjoyed some considerable success in a range of domains, technological shortcomings still exist in relation to their core content-based and collaborative recommendation

7 approaches. In our research we have looked at how data mining techniques can be used to solve some of these shortcomings. In this paper we have brought elements of this research together. We have presented two detailed case-studies that explains how association rule mining techniques can be used to improve content-based and collaborative filtering recommender systems by offering meaningful performance benefits. ACKNOWLEDGMENT This material is based on works supported by Science Foundation Ireland (SFI) under Grant No. 03/IN.3/I361.