WikiSent : Weakly Supervised Sentiment Analysis Through Extractive Summarization With Wikipedia

Transcription

1 WikiSent : Weakly Supervised Sentiment Analysis Through Extractive Summarization With Wikipedia Subhabrata Mukherjee, Pushpak Bhattacharyya Dept. of Computer Science and Engineering, IIT Bombay European Conference on Machine Learning ECML PKDD 2012, Bristol, U.K., Sept, 2012

2 World Knowledge 2 Extensive world knowledge required to perform analysis of reviews Distinguish between What s and About s of the review Filter out concepts irrelevant to the reviewer opinion about the movie Retain only the objects of interest and corresponding opinions Department of Computer Science and Engineering, IIT Bombay 7/23/2013

3 A Negative Review best remembered for his understated performance as dr. hannibal lecter in michael mann's forensics thriller, manhunter, scottish character actor brian cox brings something special to every movie he works on. usually playing a bit role in some studio schlock ( he dies halfway through the long kiss goodnight ), he's only occasionally given something meaty and substantial to do. if you want to see some brilliant acting, check out his work as a dogged police inspector opposite frances mcdormand in ken loach's hidden agenda. cox plays the role of big john harrigan in the disturbing new indie flick l. i. e., which lot 47 picked up at sundance when other distributors were scared to budge big john feels the love that dares not speak its name, but he expresses it through seeking out adolescents and bringing them back to his pad. what bothered some audience members was the presentation of big john in an oddly empathetic light. he's an even-tempered, funny, robust old man who actually listens to the kids' problems ( as opposed to their parents and friends, both caught up in the high-wire act of their own confused lives. ) he'll have sex-for-pay with them only after an elaborate courtship, charming them with temptations from the grown-up world.

4 Facets of a Movie Review

5 Facets of a Movie Review John Travolta is considered by many to be a has-been, or a one-hit wonder Leornardo DeCaprio is an awesome actor.

6 Facets of a Movie Review Born into a family of thespians -- parents Roger Winslet and Sally Bridges-Winslet John Travolta is considered by many to be a has-been, or a one-hit wonder were both stage actors, maternal grandparents Oliver and Linda Leornardo DeCaprio is an awesome actor. Bridges ran the Reading Repertory Theatre, and uncle Robert Bridges was a fixture in London's West End theatre district Kate Winslet came into her talent at an early age.

7 Facets of a Movie Review Born into a family of thespians -- parents Roger Winslet and Sally Bridges-Winslet John The role Travolta that were transformed both is considered stage Winslet actors, by maternal from many art to house grandparents be a attraction has-been, Oliver to or international and a one-hit Linda star wonder was Bridges Rose ran DeWitt the Reading Bukater, Repertory the passionate, Theatre, rosy-cheeked and uncle Robert aristocrat Bridges in was James a fixture in London's West End Leornardo theatre Cameron's district DeCaprio Kate Titanic is Winslet (1997). awesome came into actor. her talent at an early age.

8 Facets of a Movie Review Born into a family of thespians -- parents Roger Winslet and Sally Bridges-Winslet John The role Travolta that were transformed both is considered stage Winslet actors, by maternal from many art to house grandparents be a attraction has-been, Oliver to or international and a one-hit Linda star wonder was I cancelled the date with my girlfriend Bridges ran the Reading Repertory Theatre, just to watch my favorite star Rose DeWitt Bukater, the passionate, rosy-cheeked and uncle Robert aristocrat Bridges in was James a fixture in London's West End Leornardo theatre Cameron's featuring district DeCaprio Kate in Titanic is this Winslet movie. (1997). awesome came into actor. her talent at an early age.

9 Facets of a Movie Review Born into a family of thespians -- parents Roger Winslet and Sally Bridges-Winslet L.I.E. stands for Long Island Expressway, which slices through the strip malls and John The role Travolta that were transformed both is considered stage Winslet actors, by maternal from many art to house grandparents be a attraction has-been, Oliver to or international and a one-hit Linda star wonder was middle-class I cancelled homes the date of suburbia. with my Filmmaker girlfriend Michael Bridges ran the Reading Repertory Theatre, just to watch Cuestamy uses favorite as a star (pretty Rose DeWitt Bukater, the passionate, rosy-cheeked and uncle Robert aristocrat Bridges in was James a fixture transparent) metaphor of dangerous in London's West End Leornardo theatre Cameron's featuring escape district DeCaprio Kate in Titanic is this for Winslet movie. his 15-year old protagonist, Howie (1997). awesome came into actor. her talent at an early (Paul Franklin Dano). age.

10 Facets of a Movie Review Born into a family of thespians -- parents Roger Winslet and Sally Bridges-Winslet L.I.E. stands for Long Island Expressway, which slices through the strip malls and John The He's role Travolta that were even-tempered, transformed both is considered stage Winslet actors, funny, by maternal from robust many art old to house grandparents be man a attraction who has-been, actually Oliver to or international listens and a one-hit Linda to the star wonder kids' was middle-class I cancelled homes the date of suburbia. with my Filmmaker girlfriend Michael Bridges ran the Reading Repertory Theatre, just to watch Cuestamy uses favorite as a star (pretty problems Rose (as DeWitt opposed Bukater, to their parents passionate, and friends, rosy-cheeked and uncle both Robert caught aristocrat Bridges up in inthe was James high-wire a fixture transparent) metaphor of dangerous in London's West End Leornardo theatre Cameron's featuring escape district DeCaprio Kate in Titanic is this for Winslet movie. his 15-year old protagonist, Howie act of their own confused (1997). awesome lives.). came into actor. her talent at an early (Paul Franklin Dano). age.

11 Facets of a Movie Review Born into a family of thespians -- parents Roger Winslet and Sally Bridges-Winslet L.I.E. stands for Long Island Expressway, which slices through the strip malls and John The He's role Travolta that were even-tempered, transformed both is considered stage Horror Winslet actors, funny, movies by maternal from robust many are art supposed old to house grandparents be man a attraction who has-been, to actually scary. Oliver to or international listens and a one-hit Linda to the star wonder kids' was middle-class I cancelled homes the date of suburbia. with my Filmmaker girlfriend Michael Bridges Rose ran DeWitt the Reading Bukater, Repertory passionate, Theatre, just to watch Cuestamy uses favorite as a star (pretty There problems is (as axiom opposed that directors their parents who have and a friends, rosy-cheeked big and hit uncle with both Robert their caught aristocrat debut Bridges up have in inthe was James a high-wire big a fixture bomb transparent) metaphor of dangerous in London's West End Leornardo theatre Cameron's featuring escape district DeCaprio Kate in Titanic is this for Winslet movie. his 15-year old protagonist, Howie act of with their their own second confused (1997). awesome film. lives.). came into actor. her talent at an early (Paul Franklin Dano). age.

12 Facets of a Movie Review Born into a family of thespians -- parents Roger Winslet and Sally Bridges-Winslet L.I.E. stands for Long Island Expressway, which slices through the strip malls and John The While He's role Travolta the that were movie even-tempered, transformed both is is considered brutal, stage Horror the Winslet actors, funny, movies violence by maternal from robust many are is art supposed neither old to house grandparents be man a very attraction who has-been, to graphic actually scary. Oliver to nor international listens and a gratuitous. one-hit Linda to the star wonder kids' It may was middle-class I cancelled homes the date of suburbia. with my Filmmaker girlfriend Michael Bridges Rose ran DeWitt the Reading Bukater, Repertory passionate, Theatre, just to watch Cuestamy uses favorite as a star (pretty There scare problems is the little (as axiom opposed ones, that but directors the their teen-age parents who have audience and a friends, rosy-cheeked big and for hit uncle with which both Robert their caught it aristocrat debut aimed Bridges up have in will inthe was James appreciate a high-wire big a fixture bomb transparent) metaphor of dangerous in London's West End Leornardo theatre Cameron's featuring escape district DeCaprio Kate in Titanic is this for Winslet movie. his 15-year old protagonist, Howie the man-eating act of with chomping their their own second confused that runs (1997). awesome film. through lives.). came into actor. the her film. talent at an early (Paul Franklin Dano). age.

13 Facets of a Movie Review Born into a family of thespians -- parents Roger Winslet and Sally Bridges-Winslet L.I.E. stands for Long Island Expressway, which slices through the strip malls and So my grandson gives me passes to this new picture One Night at McCool's because John The While He's role Travolta the that were movie even-tempered, transformed both is is considered brutal, stage Horror the Winslet actors, funny, movies violence by maternal from robust many are is art supposed neither old to house grandparents be man a very attraction who has-been, to graphic actually scary. Oliver to nor international listens and a gratuitous. one-hit Linda to the star wonder kids' It may was middle-class I cancelled homes the date of suburbia. with my Filmmaker girlfriend Michael Bridges Rose ran DeWitt the Reading Bukater, Repertory passionate, Theatre, just to watch Cuestamy uses favorite as a star (pretty There the scare problems free is the screening little (as axiom opposed ones, that is the but directors same the their teen-age night parents who have audience and that a friends, horrible rosy-cheeked big and for hit uncle with which both show Robert their caught with aristocrat debut aimed Bridges those up have in will in poor the was James appreciate a high-wire prisoners big a fixture bomb transparent) metaphor of dangerous in London's West End Leornardo theatre Cameron's featuring escape district DeCaprio Kate in Titanic is this for Winslet movie. his 15-year old protagonist, Howie trapped the the island man-eating who act of with chomping their bugs. their own second "Go," confused that runs he (1997). awesome film. says, through lives.). came "it's into actor. the just her film. like talent Rush-o-Man." at early (Paul Franklin Dano). age.

14 Wikipedia Extensive World Knowledge required to perform this analysis Wikipedia is used to create a topic-specific, extractive summary of a review The extract is classified with a Lexicon, instead of the entire review

15 Feature Extraction from Wikipedia

16 Feature Extraction from Wikipedia METADATA Harry Potter and the Deathly Hallows Part 1 is a 2010 fantasy film[5] directed by David Yates and the first of two films based on the novel Harry Potter and the Deathly Hallows by J. K. Rowling. It is the seventh instalment in the Harry Potter film series, written by Steve Kloves and produced by David Heyman, David Barron and Rowling. The story follows Harry Potter on a quest to find and destroy Lord Voldemort's secret to immortality the Horcruxes. The film stars Daniel Radcliffe as Harry Potter, alongside Rupert Grint and Emma Watson as Harry's best friends Ron Weasley and Hermione Granger. It is the sequel to Harry Potter and the Half- Blood Prince and is followed by the concluding film, Harry Potter and the Deathly Hallows Part 2. Principal photography began on 19 February 2009 and was completed on 12 June 2010.[6] Part 1 was released in 2D cinemas and IMAX formats worldwide on 19 November 2010.

17 Feature Extraction from Wikipedia METADATA PLOT Harry Further Potter information: and the Harry Deathly Potter Hallows and the Part Deathly 1 is a Hallows 2010 fantasy Novel film[5] Plot directed by David Yates and the first of two films based on the Minister novel Rufus Harry Scrimgeour Potter and the addresses Deathly the Hallows wizarding by J. media K. Rowling. stating It is that the the seventh Ministry instalment of Magic in will the remain Harry Potter strong film as series, Lord Voldemort written by Steve gains Kloves power and throughout produced the by David wizarding Heyman, and David Muggle Barron worlds. and Rowling. Severus Snape The story arrives follows at Malfoy Harry Manor Potter to on inform a quest Lord to Voldemort find and destroy and his Death Lord Voldemort's Eaters of Harry's secret departure to immortality from No. the 4 Privet Horcruxes. Drive. The Voldemort film stars commandeers Daniel Radcliffe Lucius as Malfoy's Harry Potter, wand, alongside as Voldemort's Rupert Grint own wand Emma cannot Watson be used as to Harry's kill Harry; best their friends wands Ron are Weasley "twins". and Hermione Meanwhile, Granger. the Order It is of the the sequel Phoenix to Harry arrive Potter at Privet and Drive the Halfand Blood Prince and is followed by the concluding film, Harry Potter escort Harry to safety using Polyjuice Potion to create six decoy and Harrys. the Deathly During their Hallows flight Part to the 2. Burrow, they are ambushed by Principal Death Eaters, photography who kill Mad-Eye began on Moody 19 February and Hedwig, 2009 and and injure was completed George Weasley. on 12 June They 2010.[6] arrive at Part the 1 Burrow, was released where in Harry 2D cinemas has a and vision IMAX of Ollivander formats worldwide being tormented on 19 November by Voldemort, who claims that the wand-maker had lied to him by informing him of the only way to kill Harry: obtaining another's wand.

18 Feature Extraction from Wikipedia METADATA CREW PLOT Harry Further Directed Potter information: by : David and the Yates Harry Deathly Potter Hallows and the Part Deathly 1 is a Hallows 2010 fantasy Novel film[5] Plot Produced directed by by David : David Yates Heyman,David and the first Barro, of two J. K. films Rowling based on the Minister Screenplay novel Rufus Harry by : Scrimgeour Potter Steve and Kloves the addresses Based Deathly on the Hallows Harry wizarding Potter by J. media K. and Rowling. the stating It is that the seventh instalment in the Harry Potter film series, written by Steve Deathly the Kloves Hallows Ministry and by of produced J. Magic K. Rowling will remain strong as Lord Voldemort gains power throughout the by David wizarding Heyman, and David Muggle Barron worlds. and Rowling. Severus Starring : Snape The Daniel story arrives Radcliffe, follows at Malfoy Rupert Harry Manor Grint, Potter Emma to on inform a Watson quest Lord to Voldemort find and destroy and Music his by Death Lord : Alexandre Voldemort's Eaters Desplat of Harry's secret departure to immortality from No. the 4 Privet Horcruxes. Drive. Voldemort Themes: film stars John commandeers Williams Daniel Radcliffe Lucius as Malfoy's Harry Potter, wand, alongside as Voldemort's Rupert Grint own Emma Watson as Harry's best friends Ron Weasley and Cinematography wand cannot : be Eduardo used to Serra kill Harry; their wands are "twins". Hermione Meanwhile, Granger. the Order It is of the the sequel Phoenix to Harry arrive Potter at Privet and Drive the Halfand Blood Editing Prince by: Mark and Day is followed by the concluding film, Harry Potter escort Harry to safety using Polyjuice Potion to create six decoy and Studio Harrys. the :Heyday Deathly Films During their Hallows flight Part to the 2. Burrow, they are ambushed by Principal Death Distributed Eaters, photography by: who Warner kill Bros. Mad-Eye began Pictures on Moody 19 February and Hedwig, 2009 and and injure was completed George Weasley. on 12 June They 2010.[6] arrive at Part the 1 Burrow, was released where in Harry 2D cinemas has a and vision IMAX of Ollivander formats worldwide being tormented on 19 November by Voldemort, who claims that the wand-maker had lied to him by informing him of the only way to kill Harry: obtaining another's wand.

19 Feature Extraction from Wikipedia CHARACTER METADATA CREW PLOT Harry Further Daniel Directed Potter Radcliffe information: by : David and as the Harry Yates Deathly Potter, Hallows the and film's the Part protagonist. Deathly 1 a Hallows 2010 fantasy Novel film[5] Plot Rupert Produced directed Grint by as by Ron David : David Weasley, Yates Heyman,David Harry's and the best first Barro, friend of two J. and K. films Rowling Hermione's based the Minister romantic Screenplay novel Rufus interest. Harry by : Scrimgeour Potter Steve and Kloves the addresses Based Deathly on the Hallows Harry wizarding Potter by J. media K. and Rowling. the stating It is that the seventh instalment in the Harry Potter film series, written by Emma Steve Deathly the Watson Kloves Hallows Ministry as and by of Hermione produced J. Magic K. Rowling will Granger, remain Harry's strong other as Lord best Voldemort friend and gains Ron's power throughout the by David wizarding Heyman, and David Muggle Barron worlds. and Rowling. Severus Starring romantic : Snape The Daniel interest. story arrives Radcliffe, follows at Malfoy Rupert Harry Manor Grint, Potter Emma to on inform a Watson quest to find and destroy Helena Lord Voldemort and Music his by Bonham Death Lord : Alexandre Voldemort's Carter Eaters Desplat as secret Bellatrix to Lestrange, of Harry's departure immortality a from No. Death the 4 Privet Horcruxes. Eater and Drive. Sirius Voldemort film Black's stars cousin commandeers Daniel and Radcliffe murderer. Themes: John Williams Lucius as Malfoy's Harry Potter, wand, alongside as Voldemort's Rupert Grint own Robbie wand Coltrane Emma cannot Watson as be Rubeus used as to Harry's Hagrid, kill Harry; best Harry's their friends wands half-giant Ron are Weasley friend, "twins". and Hermione gamekeeper Cinematography Meanwhile, Granger. at Hogwarts. : Eduardo Serra the Order It is of the the sequel Phoenix to Harry arrive Potter at Privet and Drive the Halfand Blood Prince and is followed by the concluding film, Harry Potter escort Warwick Editing by: Harry Davis Mark to safety as Day Griphook, using Polyjuice a goblin Potion and former to create employee six decoy at and Harrys. Gringotts Studio the :Heyday Deathly During Bank. Films their Hallows Davis flight replaced Part to the 2. Verne Burrow, Troyer, they who are ambushed portrayed the by Principal Death character Distributed Eaters, photography physically by: who Warner kill in Bros. Mad-Eye the began first Pictures on film, Moody 19 though February and Hedwig, Davis 2009 had and and dubbed injure was completed George Griphook's Weasley. on lines. 12 June They 2010.[6] arrive at Part the 1 Burrow, was released where in Harry 2D cinemas has a and vision Tom IMAX Felton of Ollivander formats as Draco worldwide being Malfoy, tormented on a Death 19 November Eater by Voldemort, and son of who Lucius claims and that Narcissa the wand-maker Malfoy. had lied to him by informing him of the only way Ralph to Fiennes kill Harry: as obtaining Lord Voldemort, another's wand. film's main antagonist

20 Feature Extraction from Wikipedia CHARACTER METADATA CREW PLOT Harry Further Daniel Directed Potter Radcliffe information: by : David and as the Harry Yates Deathly Potter, Hallows the and film's the Part protagonist. Deathly 1 a Hallows 2010 fantasy Novel film[5] Plot Rupert Produced directed DOMAIN Grint by as by Ron David : David SPECIFIC Weasley, Yates Heyman,David Harry's and the FEATURE best first Barro, friend of two J. and K. films LIST Rowling Hermione's based the Minister romantic Screenplay novel Rufus interest. Harry by : Scrimgeour Potter Steve and Kloves the addresses Based Deathly on the Hallows Harry wizarding Potter by J. media K. and Rowling. the stating It is that the seventh instalment in the Harry Potter film series, written by Emma Steve Deathly the Watson Kloves Hallows Ministry as and by of Hermione produced J. Magic K. Rowling will Granger, remain Harry's strong other as Lord best Voldemort friend and gains Ron's power throughout the by David wizarding Heyman, and David Muggle Barron worlds. and Rowling. Severus Starring romantic : Snape The Daniel interest. story arrives Radcliffe, follows at Malfoy Rupert Harry Manor Grint, Potter Emma to on inform a Watson quest to find and destroy Helena Lord Voldemort and Music Movie, his by Bonham Death Staffing, Lord : Alexandre Voldemort's Carter Eaters casting, Desplat of Harry's secret Bellatrix Writing, departure to Lestrange, immortality Theory, from Writing, a No. Death the 4 Privet Horcruxes. Rewriting, Eater and Drive. Sirius Voldemort Screenplay, film Black's stars commandeers Format, cousin Daniel Treatments, and Radcliffe murderer. Lucius as Scriptments, Malfoy's Harry Potter, wand, Synopsis, alongside as Voldemort's Logline, Themes: John Williams Rupert Grint own Robbie Pitching, wand Coltrane Certification,scripts, Emma cannot Watson as be Rubeus used as to Harry's Hagrid, kill Budget, Harry; best Harry's Ideas, their friends wands half-giant Funding, Ron are Weasley friend, budgeting, "twins". and Hermione gamekeeper Cinematography Funding, Plans, Meanwhile, Granger. at Hogwarts. Grants, : Eduardo the Order It is Pitching, Serra of the the sequel Tax, Phoenix to Harry Contracts, arrive Potter law, at Privet and Copyright, Drive the Halfand Blood Prince and is followed by the concluding film, Harry Potter escort Warwick Editing Pre-production, by: Harry Davis Mark to safety as Day Budgeting, Griphook, Scheduling, using Polyjuice a goblin Potion and Pre-production, former to create employee film six decoy at, and Harrys. Gringotts Studio stock, the :Heyday Story, Deathly During Bank. boarding, Films their Hallows Davis flight replaced Part plot, to the 2. Verne Casting Burrow, Troyer,, Directors, they who are ambushed portrayed Location, the by Principal Death character Distributed Scouting,.. Eaters, photography physically by: who Warner kill in Bros. Mad-Eye the began first Pictures on film, Moody 19 though February and Hedwig, Davis 2009 had and and dubbed injure was completed George Griphook's Weasley. on lines. 12 June They 2010.[6] arrive at Part the 1 Burrow, was released where in Harry 2D cinemas has a and vision Tom IMAX Felton of Ollivander formats as Draco worldwide being Malfoy, tormented on a Death 19 November Eater by Voldemort, and son of who Lucius claims and that Narcissa the wand-maker Malfoy. had lied to him by informing him of the only way Ralph to Fiennes kill Harry: as obtaining Lord Voldemort, another's wand. film's main antagonist

21 Feature List Creation

22 Feature List Creation Metadata and Plot sentences are POS-tagged Nouns retrieved Stemmed Added to the Plot list Character information also added to Plot list

23 Feature List Creation Metadata and Plot sentences are POS-tagged Nouns retrieved Stemmed Added to the Plot list Character information also added to Plot list Domain specific feature list extracted from Wiki articles on films and movies Added to MovieFeature list

24 Feature List Creation Metadata and Plot sentences are POS-tagged Nouns retrieved Stemmed Added to the Plot list Character information also added to Plot list Domain specific feature list extracted from Wiki articles on films and movies Added to MovieFeature list Crew information added to Crew list

25 Feature List Creation Metadata and Plot sentences are POS-tagged Nouns retrieved Stemmed Added to the Plot list Character information also added to Plot list Domain specific feature list extracted from Wiki articles on films and movies Added to MovieFeature list Crew information added to Crew list Laptop and Printer domain data used to filter frequent occurring concepts Non-overlapping domains Frequently occurring terms in all the domains added to FreqWords list The FreqWords are pruned from the other feature lists

26 Why only Nouns? 26 To restrict genre-specific concepts to be entities Harry acted as if nothing has happened vs. Kate Winslet acted awesome in the movie. Here act is present as a Verb in both the sentences First sentence belongs to the plot (Category 5) Second sentence depicts the reviewer opinion (Category 8) Difference lies in the presence of different subjects of interest with the Verb Our focus is to capture the subjects and objects in the sentence which give direct clues about the category of the reviewer statement, so that feature lists are as pure as possible Department of Computer Science and Engineering, IIT Bombay 7/23/2013

27 Extractive Summary

28 Semi-Supervised Learning of Parameters 28 Department of Computer Science and Engineering, IIT Bombay 7/23/2013

29 Algorithm

30 Algorithm

31 WikiSent 31 Input Movie Review Extractive Opinionated Summary Polarity Output Mobile, Printer Reviews Wikipedia Frequent Words List Metadata, Plot, Crew, Cast, Story Characters, Movie Features Sentiment Lexicon World Knowledge Department of Computer Science and Engineering, IIT Bombay 7/23/2013

32 Algorithm Demonstration

33 Algorithm Demonstration Best remembered for his understated performance as Dr. Hannibal Lecter in Michael Mann's forensics thriller, Manhunter, Scottish character actor Brian Cox brings something special to every movie he works on.

34 Algorithm Demonstration Best remembered for his understated performance as Dr. Hannibal Lecter in Michael Usually playing a bit role in some studio schlock (he dies halfway through The Long Kiss Mann's forensics thriller, Manhunter, Scottish character actor Brian Cox brings Goodnight), he's only occasionally given something meaty and substantial to do. something special to every movie he works on.

35 Algorithm Demonstration Best remembered for his understated performance as Dr. Hannibal Lecter in Michael If Usually you Mann's want playing to forensics see a bit some role thriller, brilliant in some Manhunter, acting, studio check schlock Scottish out (he character his dies work halfway as through The Long Kiss actor a dogged Brian Cox police brings inspector Goodnight), opposite he's only occasionally given something meaty and substantial to do. something Frances McDormand special to every in Ken movie Loach's he works Hidden on. Agenda.

36 Algorithm Demonstration Best remembered for his understated performance as Dr. Hannibal Lecter in Michael If Usually Cox you Mann's want plays playing to the forensics see a role bit some of role thriller, Big brilliant in John some studio schlock (he dies halfway through The Long Kiss Manhunter, acting, Harrigan check Scottish in the out disturbing character his work new as actor a indie dogged Brian flick Cox police L.I.E., brings inspector which Goodnight), Lot 47 picked opposite he's up only occasionally given something meaty and substantial to do. something Frances at Sundance McDormand special when to other every in Ken distributors movie Loach's he works Hidden were on. scared Agenda. to budge.

37 Algorithm Demonstration Best remembered for his understated performance as Dr. Hannibal Lecter in Michael If Usually Cox you Big Mann's want plays John playing to the forensics feels see a role bit some the of role in some studio schlock (he dies halfway through The Long Kiss thriller, love Big brilliant John that Manhunter, dares acting, Harrigan not check Scottish speak in the out its disturbing character his name, work but new as actor he a indie dogged expresses Brian flick Cox police L.I.E., it brings through inspector which Goodnight), Lot 47 picked opposite seeking he's up only occasionally given something meaty and substantial to do. something Frances at out Sundance adolescents McDormand special when to other every bringing Ken distributors movie Loach's them he works back Hidden were to on. his scared Agenda. pad. to budge.

38 Algorithm Demonstration Best remembered for his understated performance as Dr. Hannibal Lecter in Michael If Usually What Cox you Big Mann's want bothered plays John playing to forensics feels see a some role bit some the of role in some studio schlock (he dies halfway through The Long Kiss thriller, audience love Big brilliant John that Manhunter, dares acting, members Harrigan not check Scottish speak was in the out the its disturbing character his presentation name, work but new as actor he a of indie dogged expresses Brian Big flick John Cox police L.I.E., it in brings through inspector an which oddly Goodnight), Lot 47 picked opposite seeking he's up only occasionally given something meaty and substantial to do. something Frances at out Sundance adolescents McDormand special when empathetic to other every bringing Ken distributors light. movie Loach's them he works back Hidden were to on. his scared Agenda. pad. to budge.

39 Algorithm Demonstration He's Best remembered for his understated performance as Dr. Hannibal Lecter in Michael Usually an playing even-tempered, a bit role in funny, some robust studio old schlock who (he actually dies halfway listens through to the The kids' Long problems If What Cox you Big Kiss Mann's want bothered plays Johnto forensics feels see some role some the of thriller, audience love Big brilliant John that Manhunter, dares acting, members Harrigan not check Scottish speak was in the out the its disturbing character his presentation name, work but new as actor he a of indie dogged expresses Brian Big flick John Cox police L.I.E., it in brings through inspector an which oddly (as Goodnight), opposed to Lot 47 picked opposite he's their seeking up only parents occasionally and friends, given both something caught meaty up in the and high-wire substantial act to of do. their something Frances at out Sundance adolescents McDormand special when empathetic to other every bringing Ken distributors light. movie Loach's them he works back Hidden were to on. his scared Agenda. pad. to budge. own confused lives.).

40 Algorithm Demonstration He's Best remembered for his understated performance as Dr. Hannibal Lecter in Michael If Usually What He'll an Cox you Big have playing even-tempered, Mann's want bothered plays John sex-for-pay to a bit role in some studio schlock (he dies halfway through The Long Kiss forensics feels see some role some the of thriller, audience love Big with funny, brilliant John that them robust Manhunter, dares acting, members Harrigan only not after old check Scottish speak was in an the elaborate who actually out the its disturbing character his presentation name, work courtship, listens but new as actor he a of indie dogged expresses Brian Big charming to the kids' flick John Cox police L.I.E., it in brings through them problems inspector an which oddly with (as Goodnight), opposed to Lot 47 picked opposite seeking he's their up only parents occasionally given something meaty and substantial to do. something Frances at out Sundance temptations and friends, adolescents McDormand special when empathetic from to other both every bringing Ken grown-up caught up in the high-wire act of their distributors light. movie Loach's them world he works back Hidden were to on. his scared Agenda. pad. to budge. own confused lives.).

41 Lexicons and Datasets 41 Department of Computer Science and Engineering, IIT Bombay 7/23/2013

42 Lexicons and Datasets 42 Lexicons SentiWordNet (Esuli et al., 2006) Subjectivity Lexicon (Wilson et al., 2005) General Inquirer (Stone et al., 1966) Department of Computer Science and Engineering, IIT Bombay 7/23/2013

43 Lexicons and Datasets 43 Lexicons SentiWordNet (Esuli et al., 2006) Subjectivity Lexicon (Wilson et al., 2005) General Inquirer (Stone et al., 1966) Baseline 1 Bag-of-Words based on the 3 lexicons Baseline 2 SO-CAL (Taboada et al., 2011) Baseline 3 All the semi-supervised and unsupervised systems in the domain Department of Computer Science and Engineering, IIT Bombay 7/23/2013

44 Lexicons and Datasets 44 Lexicons SentiWordNet (Esuli et al., 2006) Subjectivity Lexicon (Wilson et al., 2005) General Inquirer (Stone et al., 1966) Baseline 1 Bag-of-Words based on the 3 lexicons Baseline 2 SO-CAL (Taboada et al., 2011) Baseline 3 All the semi-supervised and unsupervised systems in the domain Movie Review Dataset (Pang et al., 2002) 1000 positive and 1000 negative reviews (labeled at document level) 27,000 untagged reviews Department of Computer Science and Engineering, IIT Bombay 7/23/2013

45 45 Accuracy Comparison with All the Semisupervised and Unsupervised Systems in the Same Dataset Department of Computer Science and Engineering, IIT Bombay 7/23/2013

46 Accuracy Comparison with Different Baselines 46

47 Accuracy Comparison with Different Baselines 47 Baseline 1 : Simple Bag-of-Words Only SentiWordNet 52.1 Only Subjectivity 60.3 Only GI 63.5 Baseline 2 : Worse, Median and Best Performing Lexicons with SO-CAL SentiWordNet-Full SentiWordNet-Basic GI-Full GI-Basic Subjectivity-Full Subjectivity-Basic WikiSent with Different Lexicons WikiSent+Subjectivity 71.1 WikiSent+SentiWordNet 73.3 WikiSent+GI 76.85

48 Accuracy Comparison with Different Baselines 48 Systems Baseline 1 : Simple Classification Bag-of-Words Method Accuracy Turney SO value Only SentiWordNet Unsupervised (PMI) Only Subjectivity Taboada SO-CAL Basic [1] Lexicon Generation Only GI Taboada SO-CAL Full [1] Lexicon Generation Baseline 2 : Worse, Median and Best Performing Lexicons with SO-CAL Shi [28], Dasgupta [29] Unsupervised Eigen Vector 70.9 SentiWordNet-Full SentiWordNet-Basic Clustering Socher [30] RAE GI-Full Semi Supervised Auto Encoders Lin [32] LSM GI-Basic Unsupervised without prior info Lin [32] LSM Subjectivity-Full Weakly Supervised Subjectivity-Basic with prior info Li [33] WikiSent with Semi Different Supervised Lexicons 10% doc. Label 60 WikiSent+Subjectivity 71.1 Li [33] Semi Supervised 40% doc. Label 60 WikiSent+SentiWordNet 73.3 WikiSent Wikipedia+GI Lexicon WikiSent+GI 76.85

49 Trend Analysis 49 Department of Computer Science and Engineering, IIT Bombay 7/23/2013

50 Trend Analysis 50 Department of Computer Science and Engineering, IIT Bombay 7/23/2013

51 Trend Analysis 51 Department of Computer Science and Engineering, IIT Bombay 7/23/2013

52 Trend Analysis 52 Department of Computer Science and Engineering, IIT Bombay 7/23/2013

53 Trend Analysis 53 WikiSent Bag-of-Words Baseline 1 Positive Reviews (%) Negative Reviews (%) Department of Computer Science and Engineering, IIT Bombay 7/23/2013

54 Drawbacks 54 Absence of a co-reference resolution module false negative Synonymous concepts not handled Does not matter much as genre-specific concepts like acting, direction, story-writer occur in same lexical form Reviewer opinion bias can affect the system false positive Absence of WSD module affects lexicon-based classification Department of Computer Science and Engineering, IIT Bombay 7/23/2013