agreementwle

JournalofArticialIntelligenceResearch30(2007)457-500Submitted05/07;published11/07UsingLinguisticCuesfortheAutomaticRecognitionofPersonalityinConversationandTextFrancoisMairessef.
mairesse@sheffield.
ac.
ukDepartmentofComputerScience,UniversityofSheeld211PortobelloStreet,SheeldS14DP,UnitedKingdomMarilynA.
Walkerm.
a.
walker@sheffield.
ac.
ukDepartmentofComputerScience,UniversityofSheeld211PortobelloStreet,SheeldS14DP,UnitedKingdomMatthiasR.
Mehlmehl@email.
arizona.
eduDepartmentofPsychology,UniversityofArizona1503EUniversityBlvd.
Building68,Tucson,AZ85721,USARogerK.
Moorer.
k.
moore@dcs.
shef.
ac.
ukDepartmentofComputerScience,UniversityofSheeld211PortobelloStreet,SheeldS14DP,UnitedKingdomAbstractItiswellknownthatutterancesconveyagreatdealofinformationaboutthespeakerinadditiontotheirsemanticcontent.
Onesuchtypeofinformationconsistsofcuestothespeaker'spersonalitytraits,themostfundamentaldimensionofvariationbetweenhumans.
Recentworkexplorestheautomaticdetectionofothertypesofpragmaticvariationintextandconversation,suchasemotion,deception,speakercharisma,dominance,pointofview,subjectivity,opinionandsentiment.
Personalityaectstheseotheraspectsoflinguisticproduction,andthuspersonalityrecognitionmaybeusefulforthesetasks,inadditiontomanyotherpotentialapplications.
However,todate,thereislittleworkontheautomaticrecognitionofpersonalitytraits.
ThisarticlereportsexperimentalresultsforrecognitionofallBigFivepersonalitytraits,inbothconversationandtext,utilisingbothselfandobserverratingsofpersonality.
Whileotherworkreportsclassicationresults,weexperimentwithclassication,regressionandrankingmodels.
Foreachmodel,weanalysetheeectofdierentfeaturesetsonaccuracy.
Resultsshowthatforsometraits,anytypeofstatisticalmodelperformssignicantlybetterthanthebaseline,butrankingmodelsperformbestoverall.
Wealsopresentanexperimentsuggestingthatrankingmodelsaremoreaccuratethanmulti-classclassiersformodellingpersonality.
Inaddition,recognitionmodelstrainedonobservedpersonalityperformbetterthanmodelstrainedusingself-reports,andtheoptimalfeaturesetdependsonthepersonalitytrait.
Aqualitativeanalysisofthelearnedmodelsconrmspreviousndingslinkinglanguageandpersonality,whilerevealingmanynewlinguisticmarkers.
1.
IntroductionPersonalityisthecomplexofalltheattributes—behavioural,temperamental,emotionalandmental—thatcharacteriseauniqueindividual.
Itiswellknownthatutterancesconveyagreatdealofinformationaboutthespeakerinadditiontotheirsemanticcontent.
Onesuchtypeofinformationconsistsofcuestothec2007AIAccessFoundation.
Allrightsreserved.
Mairesse,Walker,Mehl&Moorespeaker'spersonalitytraits,themostfundamentaldimensionofvariationbetweenhumans.
PersonalityistypicallyassessedalongvedimensionsknownastheBigFive:Extraversionvs.
Introversion(sociable,assertive,playfulvs.
aloof,reserved,shy)Emotionalstabilityvs.
Neuroticism(calm,unemotionalvs.
insecure,anxious)Agreeablenessvs.
Disagreeable(friendly,cooperativevs.
antagonistic,faultnding)Conscientiousnessvs.
Unconscientious(self-disciplined,organisedvs.
inecient,care-less)Opennesstoexperience(intellectual,insightfulvs.
shallow,unimaginative)Thesevepersonalitytraitshavebeenrepeatedlyobtainedbyapplyingfactoranalysestovariouslistsoftraitadjectivesusedinpersonalitydescriptionquestionnaires(sampleadjectivesabove)(Norman,1963;Peabody&Goldberg,1989;Goldberg,1990).
ThebasisforsuchfactoranalysesistheLexicalHypothesis(Allport&Odbert,1936),i.
e.
thatthemostrelevantindividualdierencesareencodedintothelanguage,andthemoreimportantthedierence,themorelikelyitistobeexpressedasasingleword.
Despitesomeknownlimits(Eysenck,1991;Paunonen&Jackson,2000),overthelast50yearstheBigFivemodelhasbecomeastandardinpsychologyandexperimentsusingtheBigFivehaveshownthatpersonalitytraitsinuencemanyaspectsoftask-relatedindividualbehaviour.
Forexample,thesuccessofmostinterpersonaltasksdependsonthepersonalitiesoftheparticipants,andpersonalitytraitsinuenceleadershipability(Hogan,Curphy,&Hogan,1994),generaljobperformance(Furnham,Jackson,&Miller,1999),attitudetowardmachines(Sigurdsson,1991),salesability(Furnhametal.
,1999),teachereectiveness(Rushton,Murray,&Erdle,1987),andacademicabilityandmotivation(Furnham&Mitchell,1991;Komarraju&Karau,2005).
However,todatetherehasbeenlittleworkontheautomaticrecognitionofpersonalitytraits(Argamon,Dhawle,Koppel,&Pennebaker,2005;Mairesse&Walker,2006a,2006b;Oberlander&Nowson,2006).
RecentworkinAIexploresmethodsfortheautomaticdetectionofothertypesofprag-maticvariationintextandconversation,suchasemotion(Oudeyer,2002;Liscombe,Ven-ditti,&Hirschberg,2003),deception(Newman,Pennebaker,Berry,&Richards,2003;Enos,Benus,Cautin,Graciarena,Hirschberg,&Shriberg,2006;Graciarena,Shriberg,Stolcke,Enos,Hirschberg,&Kajarekar,2006;Hirschberg,Benus,Brenier,Enos,Fried-man,Gilman,Girand,Graciarena,Kathol,Michaelis,Pellom,Shriberg,&Stolcke,2005),speakercharisma(Rosenberg&Hirschberg,2005),mood(Mishne,2005),dominanceinmeetings(Rienks&Heylen,2006),pointofvieworsubjectivity(Wilson,Wiebe,&Hwa,2004;Wiebe,Wilson,Bruce,Bell,&Martin,2004;Wiebe&Rilo,2005;Stoyanov,Cardie,&Wiebe,2005;Somasundaran,Ruppenhofer,&Wiebe,2007),andsentimentoropinion(Turney,2002;Pang&Lee,2005;Popescu&Etzioni,2005;Breck,Choi,&Cardie,2007).
Incontrastwiththesepragmaticphenomena,whichmayberelativelycontextualisedorshort-lived,personalityisusuallyconsideredtobealongerterm,morestable,aspectofindividuals(Scherer,2003).
However,thereisevidencethatpersonalityinteractswith,andaects,theseotheraspectsoflinguisticproduction.
Forexample,therearestrongrelationsbetweentheextraversionandconscientiousnesstraitsandthepositiveaects,andbetween458RecognisingPersonalityinConversationandTextneuroticismanddisagreeablenessandvariousnegativeaects(Watson&Clark,1992).
Ly-ingleadstoinconsistenciesinimpressionsoftheagreeablenesspersonalitytraitacrossmodes(visualvs.
acoustic),andtheseinconsistenciesareusedascuesfordeceptiondetectionbyhumanjudges(Heinrich&Borkenau,1998).
Outgoingandenergeticpeople(i.
e.
extravert)aremoresuccessfulatdeception,whileapprehensive(i.
e.
neurotic)individualsarenotassuccessful(Riggio,Salinas,&Tucker,1988),andindividualswhoscorehighlyontheagree-ablenessandopennesstoexperiencetraitsarealsobetteratdetectingdeception(Enosetal.
,2006).
Featuresusedtoautomaticallyrecogniseintroversionandextraversioninourstudiesarealsoimportantforautomaticallyidentifyingdeception(Newmanetal.
,2003).
Speakercharismahasbeenshowntocorrelatestronglywithextraversion(Bono&Judge,2004),andindividualswhodominatemeetingshavesimilarcharacteristicstoextraverts,suchasverbosity(Rienks&Heylen,2006).
OberlanderandNowson(2006)suggestthatopinionminingcouldbenetfrompersonalityinformation.
Thusthisevidencesuggeststhatincorporatingpersonalitymodelsintotheseothertasksmayimproveaccuracy.
Wealsohypothesisethatcomputationalrecognitionofuserpersonalitycouldbeuse-fulinmanyothercomputationalapplications.
Identicationofleadersusingpersonalitydimensionscouldbeusefulinanalysingmeetingsandtheconversationsofsuspectedter-rorists(Hoganetal.
,1994;Tucker&Whittaker,2004;Nunn,2005).
Datingwebsitescouldanalysetextmessagestotrytomatchpersonalitiesandincreasethechancesofasuccessfulrelationship(Donnellan,Conger,&Bryant,2004).
Tutoringsystemsmightbemoreeectiveiftheycouldadapttothelearner'spersonality(Komarraju&Karau,2005).
Automaticallyidentifyingtheauthor'spersonalityinacorpuscouldalsoimprovelanguagegeneration,asindividualdierencesinlanguageaectthewaythatconceptsareexpressed(Reiter&Sripada,2004).
Studieshavealsoshownthatusers'evaluationofconversationalagentsdependsontheirownpersonality(Reeves&Nass,1996;Cassell&Bickmore,2003),whichsuggestsarequirementforsuchsystemstoadapttotheuser'spersonality,likehumansdo(Funder&Sneed,1993;McLarney-Vesotski,Bernieri,&Rempala,2006).
Whileinsomeapplicationsitwouldbepossibletoacquirepersonalityinformationbyaskingtheuserorauthordirectly(John,Donahue,&Kentle,1991;Costa&McCrae,1992),hereweexplorewhetheritispossibletoacquirepersonalitymodelsfortheBigFivepersonalitytraitsbyobservationofindividuallinguisticoutputsintextandconversation.
Todate,weknowofonlytwostudiesbesidesourownonautomaticrecognitionofuserpersonality(Argamonetal.
,2005;Mairesse&Walker,2006a,2006b;Oberlander&Nowson,2006).
Otherworkhasappliedclassicationmodelstotherecognitionofpersonalityintextsandblogpostings.
Toourknowledge,theresultspresentedherearethersttoexaminetherecognitionofpersonalityindialogue(Mairesse&Walker,2006a,2006b),andtoapplyregressionandrankingmodelsthatallowustomodelpersonalityrecognitionusingthecontinuousscalestraditionalinpsychology.
Wealsosystematicallyexaminetheuseofdierentfeaturesets,suggestedbypsycholinguisticresearch,andreportstatisticallysignicantresults.
WestartinSection2byreviewingthepsychologyndingslinkingpersonalityandlanguage;thesendingsmotivatethefeaturesusedinthelearningexperimentsdescribedinSection3.
Section3overviewsthemethodsweusetoautomaticallytrainpersonalitymodels,usingbothconversationandwrittenlanguagesamples,andbothself-ratingsandobserverratingsofpersonalitytraits.
Weexploretheuseofclassicationmodels(Section4),459Mairesse,Walker,Mehl&Mooreregressionmodels(Section5),andrankingmodels(Section6),andtheeectofdierentfeaturesetsonmodelaccuracy.
Theresultsshowthatforsometraits,anytypeofstatisticalmodelperformssignicantlybetterthanthebaseline,butrankingmodelsperformbestoverall.
Inaddition,modelstrainedonobservedpersonalityscoresperformbetterthanmodelstrainedusingself-reports,andtheoptimalfeaturesetisdependentonthepersonalitytrait.
Therulesderivedandfeaturesusedinthelearnedmodelsconrmpreviousndingslinkinglanguageandpersonality,whilerevealingmanynewlinguisticmarkers.
WedelaythereviewofArgamonetal.
(2005)andOberlanderandNowson(2006)toSection7,whenwecanbettercomparetheirresultswithourown,andsumupanddiscussfutureworkinSection8.
2.
PersonalityMarkersinLanguageWhydowebelieveitmightbepossibletoautomaticallyrecognisepersonalityfromlinguisticcuesPsychologistshavedocumentedtheexistenceofsuchcuesbydiscoveringcorrelationsbetweenarangeoflinguisticvariablesandpersonalitytraits,acrossawiderangeoflinguisticlevels,includingacousticparameters(Smith,Brown,Strong,&Rencher,1975;Scherer,1979),lexicalcategories(Pennebaker&King,1999;Pennebaker,Mehl,&Niederhoer,2003;Mehl,Gosling,&Pennebaker,2006;Fast&Funder,2007),n-grams(Oberlander&Gill,2006),andspeech-acttype(Vogel&Vogel,1986).
Asthecorrelationsreportedintheliteraturearegenerallyweak(seeSection3.
3),itisnotclearwhetherthesefeatureswillimproveaccuraciesofstatisticalmodelsonunseensubjects.
OfallBigFivetraits,extraversionhasreceivedthemostattentionfromresearchers.
However,studiesfocusingsystematicallyonallBigFivetraitsarebecomingmorecommon.
2.
1MarkersofExtraversionWesummarisevariousndingslinkingextraversionandlanguagecuesinTable1,fordierentlevelsoflanguageproductionsuchasspeech,syntaxandcontentselection.
Are-viewbyFurnham(1990)describeslinguisticfeatureslinkedtoextraversionandothertraits,andDewaeleandFurnham(1999)reviewstudiesfocusingonthelinkbetweenextraversionandbothlanguagelearningandspeechproduction.
Findingsincludethatthereisahighercorrelationbetweenextraversionandorallan-guage,especiallywhenthestudyinvolvesacomplextask.
Extravertstalkmore,louderandmorerepetitively,withfewerpausesandhesitations,theyhavehigherspeechrates,shortersilences,ahigherverbaloutput,alowertype/tokenratioandalessformallan-guage,whileintrovertsuseabroadervocabulary(Scherer,1979;Furnham,1990;Gill&Oberlander,2002).
Extravertsalsousemorepositiveemotionwords,andshowmoreagree-mentsandcomplimentsthanintroverts(Pennebaker&King,1999).
ExtravertstudentslearningFrenchasasecondlanguageproducemoreback-channels,andhaveamoreim-plicitstyleandalowerlexicalrichnessinformalsituations.
Itseemsthatthemorecomplexthetaskandthehigherthelevelofanxiety,theeasieritistodierentiatebetweenintrovertsandextraverts(Dewaele&Furnham,1999).
HeylighenandDewaele(2002)alsonotethatextraversionissignicantlycorrelatedwithcontextuality,asopposedtoformality.
Contextualitycanbeseenahighrelianceonsharedknowledgebetweenconversationalpartners,leadingtotheuseofmanydeictic460RecognisingPersonalityinConversationandTextLevelIntrovertExtravertConversationalListenInitiateconversationbehaviourLessback-channelbehaviourMoreback-channelbehaviourTopicSelf-focusedNotself-focused*selectionProblemtalk,dissatisfactionPleasuretalk,agreement,complimentStrictselectionThinkoutloud*SingletopicManytopicsFewsemanticerrorsManysemanticerrorsFewself-referencesManyself-referencesStyleFormalInformalManyhedges(tentativewords)Fewhedges(tentativewords)SyntaxManynouns,adjectives,prepositions(explicit)Manyverbs,adverbs,pronouns(implicit)ElaboratedconstructionsSimpleconstructions*ManywordspersentenceFewwordspersentenceManyarticlesFewarticlesManynegationsFewnegationsLexiconCorrectLoose*RichPoorHighdiversityLowdiversityManyexclusiveandinclusivewordsFewexclusiveandinclusivewordsFewsocialwordsManysocialwordsFewpositiveemotionwordsManypositiveemotionwordsManynegativeemotionwordsFewnegativeemotionwordsSpeechReceivedaccentLocalaccent*SlowspeechrateHighspeechrateFewdisuenciesManydisuencies*ManyunlledpausesFewunlledpausesLongresponselatencyShortresponselatencyQuietLoudLowvoicequalityHighvoicequalityNon-nasalvoiceNasalvoiceLowfrequencyvariabilityHighfrequencyvariabilityTable1:Summaryofidentiedlanguagecuesforextraversionandvariousproductionlev-els,basedonpreviousstudiesbyScherer(1979),Furnham(1990),PennebakerandKing(1999),DewaeleandFurnham(1999),Gill(2003),Mehletal.
(2006).
Asterisksindicatethatthecueisonlybasedonahypothesis,asopposedtostudyresults.
expressionssuchaspronouns,verbs,adverbsandinterjections,whereasformallanguageislessambiguousandassumeslesscommonknowledge.
Inordertomeasurethisvariation,HeylighenandDewaelesuggesttheuseofametriccalledformality,denedas:F=(nounfreq+adjectivefreq+prepositionfreq+articlefreq-pronounfreq-verbfreq-adverbfreq-interjectionfreq+100)/2Theyarguethatthismeasureisthemostimportantdimensionofvariationbetweenlinguisticexpressions,asshowninBiber'sfactoranalysisofvariousgenres(Biber,1988).
Inadditiontointroversion,theauthorsalsondthatformalitycorrelatespositivelywiththelevelofeducationandthefemininityofthespeaker.
Situationalvariablesrelatedtotheuseofformallanguagearetheaudiencesize,thetimespanbetweendialogues,theunavailabilityoffeedback,dierenceofbackgroundsandspatiallocationbetweenspeakers,aswellastheprecedingamountofconversation.
461Mairesse,Walker,Mehl&MooreScherer(1979)showsthatextravertsareperceivedastalkinglouderandwithamorenasalvoice,andthatAmericanextravertstendtomakefewerpauses,whileGermanex-travertsproducemorepausesthanintroverts.
Thuspersonalitymarkersareculture-dependent,evenamongwesternsocieties.
OberlanderandGill(2006)usecontentanalysistoolsandn-gramlanguagemodelstoidentifymarkersinextravertandintrovertemails.
Theyreplicatepreviousndingsandidentifynewpersonalitymarkerssuchasrstpersonsingularpronouns(e.
g.
,Idon't)andformalgreetings(e.
g.
,Hello)forintroversion,whilelessformalphrasessuchasTakecareandHicharacteriseextraverts.
2.
2MarkersofOtherBigFiveTraitsPennebakerandKing(1999)identifymanylinguisticfeaturesassociatedwitheachoftheBigFivepersonalitytraits.
TheyusetheirLinguisticInquiryandWordCount(LIWC)tooltocountwordcategoriesofessayswrittenbystudentswhosepersonalityhasbeenassessedusingaquestionnaire.
Theauthorsndsmallbutsignicantcorrelationsbetweentheirlinguisticdimensionsandpersonalitytraits.
Neuroticsusemore1stpersonsingularpronouns,morenegativeemotionwordsandlesspositiveemotionwords.
Ontheotherhand,agreeablepeopleexpressmorepositiveandfewernegativeemotions.
Theyalsousefewerarticles.
Conscientiouspeopleavoidnegations,negativeemotionwordsandwordsreectingdiscrepancies(e.
g.
,shouldandwould).
Finally,opennesstoexperienceischaracterisedbyapreferenceforlongerwordsandwordsexpressingtentativity(e.
g.
,perhapsandmaybe),aswellastheavoidanceof1stpersonsingularpronounsandpresenttenseforms.
Additionally,Mehletal.
(2006)studymarkersofpersonalityasperceivedbyobservers.
Theyndthattheuseofwordsrelatedtoinsightandtheavoidanceofpasttenseindicatesopennesstoexperience,andswearingmarksdisagreeableness.
Thesameauthorsalsoshowthatsomelinguisticcuesvarygreatlyacrossgender.
Forexample,malesperceivedasconscientiousproducemorellerwords,whilefemalesdon't.
Genderdierencesarealsofoundinmarkersofself-assessedpersonality:theuseof2ndpersonpronounsindicatesaconscientiousmale,butanunconscientiousfemale.
GillandOberlander(2003)studycorrelatesofemotionalstability:theyndthatneu-roticsusemoreconcreteandfrequentwords.
However,theyalsoshowthatobserversdon'tusethosecuescorrectly,asobserverreportsofneuroticismcorrelatenegativelywithself-reports.
Concerningprosody,Smithetal.
(1975)alsoshowthatspeechrateispositivelycorre-latedwithperceivedcompetence(conscientiousness),andthatspeechratehasaninverted-Urelationshipwithbenevolence(agreeableness),suggestinganeedfornon-linearmodels.
Sometraitshaveproducedmorendingsthanothers.
Areasonforthismightbethatsomearemorereectedthroughlanguage,likeextraversion.
However,itispossiblethatthisfocusisaconsequenceofextraversionbeingcorrelatedwithlinguisticcuesthatcanbeanalysedmoreeasily(e.
g.
,verbosity).
462RecognisingPersonalityinConversationandText3.
ExperimentalMethodWeconductasetofexperimentstoexaminewhetherautomaticallytrainedmodelscanbeusedtorecognisethepersonalityofunseensubjects.
Ourapproachcanbesummarisedinvesteps:1.
Collectindividualcorpora;2.
Collectassociatedpersonalityratingsforeachparticipant;3.
Extractrelevantfeaturesfromthetexts;4.
Buildstatisticalmodelsofthepersonalityratingsbasedonthefeatures;5.
Testthelearnedmodelsonthelinguisticoutputsofunseenindividuals.
Thefollowingsectionsdescribeeachofthesestepsinmoredetail.
3.
1SourcesofLanguageandPersonalityIntrovertExtravertI'vebeenwakingupontimesofar.
WhatIhavesomereallyrandomthoughts.
Ihasitbeen,5daysDearme,I'llneverwantthebestthingsoutoflife.
keepitup,beingsuchnotamorningButIfearthatIwanttoomuch!
personandall.
ButmaybeI'lladjust,WhatifIfallatonmyfaceandornot.
Iwantinternetaccessinmydon'tamounttoanything.
ButIroom,Idon'thaveityet,butIwillfeellikeIwasborntodoBIGthingsonWedIthink.
Butthatain'tsoononthisearth.
Butwhoknows.
.
.
Thereenough,causeIgotcalculushomework[.
.
.
]isthisPersianpartytoday.
NeuroticEmotionallystableOneofmyfriendsjustbargedin,andIIshouldexcelinthissportbecauseIjumpedinmyseat.
Thisiscrazy.
Iknowhowtopushmybodyharderthanshouldtellhimnottodothatagain.
anyoneIknow,nomatterwhatthetestII'mnotthatfastidiousactually.
Butalwayspushmybodyharderthaneveryonecertainthingsannoyme.
Thethingselse.
Iwanttobethebestnomatterthatwouldannoymewouldactuallywhatthesportorevent.
Ishouldalsoannoyanynormalhumanbeing,soIbegoodatthisbecauseIlovetorideknowI'mnotafreak.
mybike.
Table2:Extractsfromtheessayscorpus,forparticipantsratedasextremelyintrovert,extravert,neurotic,andemotionallystable.
WeusethedatafromPennebakerandKing(1999)andMehletal.
(2006)inourex-periments.
Therstcorpuscontains2,479essaysfrompsychologystudents(1.
9millionwords),whoweretoldtowritewhatevercomesintotheirmindfor20minutes.
ThedatawascollectedandanalysedbyPennebakerandKing(1999);asampleisshowninTable2.
463Mairesse,Walker,Mehl&MooreIntrovertExtravert-Yeahyouwoulddokilograms.
YeahIsee-That'smyrstyogurtexperiencehere.
whatyou'resaying.
Reallywatery.
Why-OnTuesdayIhaveclass.
Idon'tknow.
-Damn.
Newgame.
-Idon'tknow.
A16.
Yeah,thatiskindofcool.
-Oh.
-Idon'tknow.
Ijustcan'twaittobewith-That'ssorude.
That.
youandnothavetodothiseverynight,-Yeah,buthe,theylikeeachother.
youknowHelikesher.
-Yeah.
Youdon'tknow.
Isthereabedin-TheyaregoingtoendupbreakingupthereWellokjust.
.
.
andhe'sgoingtobelike.
UnconscientiousConscientious-WiththeChinese.
Getittogether.
-Idon't,Idon'tknowforafactbut-Itriedtoyellatyouthroughthewindow.
IwouldimaginethathistoricallywomenOh.
xxxx'sfuckingadumbass.
Lookatwhohaveenteredprostitutionhavedonehim.
Lookathim,dude.
Lookathim.
Iso,noteveryone,butforthemajorityoutwishwehadacamera.
He'sfuckingbrushingofextremedesperationandIthink.
Idon'thist-shirtwithatoothbrush.
Getakickknow,ithinkpeopleunderstandthatofit.
Don'tstealnothing.
desperationandtheydon'tdon'tsee[.
.
.
]Table3:ExtractsfromtheEARcorpus,forparticipantsratedasextremelyintrovert,ex-travert,unconscientious,andconscientious.
Onlytheparticipants'utterancesareshown.
PersonalitywasassessedbyaskingeachstudenttollintheBigFiveInventoryquestion-naire(Johnetal.
,1991),whichasksparticipantstoevaluateona5pointscalehowwelltheirpersonalitymatchesaseriesofdescriptions.
ThesecondsourceofdataconsistsofconversationextractsrecordedusinganElectroni-callyActivatedRecorder(EAR)(Mehl,Pennebaker,Crow,Dabbs,&Price,2001),collectedbyMehletal.
(2006).
Topreservetheparticipants'privacy,onlyrandomsnippetsofconver-sationwererecorded.
Thiscorpusismuchsmallerthantheessayscorpus(96participantsforatotalof97,468wordsand15,269utterances).
Whiletheessayscorpusconsistsonlyoftexts,theEARcorpuscontainsbothsoundextractsandtranscripts.
Thiscorpusthereforeallowsustobuildmodelsofpersonalityrecognitionfromspeech.
Onlytheparticipants'ut-terancesweretranscribed(notthoseoftheirconversationalpartners),makingitimpossibletoreconstructwholeconversations.
Nevertheless,theconversationextractsarelessformalthantheessays,andpersonalitymaybebestobservedintheabsenceofbehaviouralcon-straints.
Table4showsthatwhiletheessayscorpusismuchlargerthantheEARcorpus,theamountofdatapersubjectiscomparable,i.
e.
766wordspersubjectfortheessaysand1,015fortheEARcorpus.
Table3showsexamplesofconversationsfromtheEARcorpusfordierentpersonalitytraits.
Forpersonalityratings,theEARcorpuscontainsbothself-reportsandratingsfrom18independentobservers.
Psychologistsuseself-reportstofacilitateevaluatingthepersonal-ityofalargenumberofparticipants,andtherearealargenumberofstandardself-reporttests.
ObserverswereaskedtomaketheirjudgmentsbyratingdescriptionsoftheBigFiveInventory(John&Srivastava,1999)ona7pointscale(fromstronglydisagreetostrongly464RecognisingPersonalityinConversationandTextDatasetEssaysEARSourceoflanguageWrittenSpokenPersonalityreportsSelfreportsSelfandobserverNumberofwords1.
9million97,468Subjects2,47996Wordspersubject766.
41,015.
3Table4:ComparisonoftheessaysandEARcorpora.
agree),withoutknowingtheparticipants.
Observersweredividedintothreegroups,eachratingonethirdoftheparticipants,afterlisteningtoeachparticipant'sentiresetofsoundles(130lesonaverage).
Thepersonalityassessmentwasbasedontheaudiorecordings,whichcontainmoreinformationthanthetranscripts(e.
g.
,ambientsounds,includingcap-turedconversations).
Mehletal.
(2006)reportstronginter-observerreliabilitiesacrossallBigFivedimensions(intraclasscorrelationsbasedonone-wayrandomeectmodels:meanr=0.
84,pwledge,thereareonlytwootherstudiesontheautomaticrecognitionofper-sonality.
Bothofthesestudieshavefocusedontheclassicationofwrittentextsbasedonself-reports,ratherthanusingcontinuousmodellingtechniquesaswedohere.
Argamonetal.
(2005)usetheessayscorpusofPennebakerandKing(1999),sotheirresultsaredirectlycomparabletoours.
Asinourwork,theyuseatop-downapproachtofeaturedenition:theirfeaturesetconsistsofrelativefrequenciesof675functionwordsandwordcategoriesbasedonnetworksofthetheoryofSystemicFunctionalGrammar.
However,theysimplifythetaskbyremovingthemiddlethirdofthedataset,therebypotentiallyincreasingprecisionatthecostofreducingrecalltoamaximumof67%.
TheytrainSMOmodelsonthetopthirdandlowerthirdoftheessayscorpusforthetwopersonalitytraits487Mairesse,Walker,Mehl&MooreConscientiousnessmodelwithallfeatures#Positiverulesα#Negativerulesα1Occup≥1.
210.
3711Swear≥0.
20-0.
182Insight≥2.
150.
3612WPS≥6.
25-0.
193Posfeel≥0.
300.
3013Pitch-mean≥229-0.
204Int-stddev≥7.
830.
2914Othref≥7.
64-0.
205Nlet≥3.
290.
2715Humans≥0.
83-0.
216Comm≥1.
200.
2616Swear≥0.
93-0.
217Nphon≥2.
660.
2517Swear≥0.
17-0.
248Nphon≥2.
670.
2218Relig≥0.
32-0.
279Nphon≥2.
760.
2019Swear≥0.
65-0.
3110K-F-nsamp≥3290.
1920Int-max≥86.
84-0.
50Table25:BestRankBoostmodelbasedonEARconversationsforconscientiousness.
Rows1-10representtherulesproducingthehighestscoreincrease,whilerows11-20indicateevidencefortheotherendofthescale,i.
e.
unconscientiousness.
ofextraversionandemotionalstability,achievingaccuraciesonthissubsetofthedataof58%forbothtraits.
Webelieveitislikelythatpersonalityrecognitionmodelsneedtobebasedonthefullrangeofvaluestobeusefulinanypracticalapplication.
Nevertheless,inordertodoadirectcomparison,wealsoremovedthemiddlethirdoftheessaysdatasetandtrainedanSMOclassierwiththeLIWCfeatures.
Weobtain57%classicationaccuracyforextra-versionand60%foremotionalstability,whereaswhenthesamealgorithmisappliedtothewholecorpus,weobtainaccuraciesof55%forextraversionand57%foremotionalstability,signicantlyoutperformingthebaseline(seeTable12).
UsingtheEARconversationaldataandobserverreports,accuraciesofourSMOmodelsremainat65%forextraversionbutincreaseto63%foremotionalstability(seeTable14).
TheseresultssuggestthatourfeaturesetincombinationwiththatofArgamonetal.
couldpossiblyimproveperformance,asbothfeaturesetsperformcomparably.
Usingtheirfeatures,Argamonetal.
identifythatrelativefrequenciesofasetoffunctionwordsarethebestpredictorforextraversion,suggestingthatthosethatrefertonormsandcertaintyarethemostsalient.
Concerningemotionalstability,thefeaturesetcharacterisingappraisalproducesbyfarthebestresults.
AppraisalfeaturesarerelativefrequenciesofpositiveandnegativewordsaswellasfrequenciesofeachcategoryintheAttitudenetwork(e.
g.
,aect,appreciation,judgement,etc.
).
Theyndthatneuroticstendtousemorewordsrelatedtonegativeappraisalandaect,butfewerappreciationappraisalwords,suggestingthattheyfocusmoreontheirpersonalfeelings.
OberlanderandNowson(2006)followabottom-upfeaturediscoverymethodbytrain-ingNaiveBayesandSMOmodelsforfouroftheBigFivetraitsonacorpusofpersonalweblogs,usingn-gramfeaturesextractedfromthedataset.
InordertobeabletocomparewithArgamonetal.
,theyreportexperimentswheretheyremovetextswithnon-extremepersonalityscoresfromtheircorpus,buttheyalsoreportexperimentsapplyingclassicationalgorithmstosevendierentwaysofpartitioningthewholecorpusintoclasses,motivatedasapproximatingacontinuousmodellingapproach.
Although,theirresultsaren'tdirectly488RecognisingPersonalityinConversationandTextcomparabletooursbecausetheyarebasedondierentcorpora,wereporttheresultsthatuseallinstancesofthedataset,aswebelievethatdiscardingsomeofthetestdataincreasesprecisionatthecostofmakingrecallunacceptablylow.
WhenbuildingNaiveBayesmodelsusingthemostfrequentbi-gramsandtri-gramscomputedoverthefullcorpus,OberlanderandNowson(2006)ndthatthemodelofagree-ablenessistheonlyoneoutperformingthebaseline(54%accuracy,nolevelofsignicancementioned).
Whenkeepingonlyn-gramsthataredistinctiveoftwoextremesetsofagiventrait,accuraciesrangefrom65%forextraversionto72%foremotionalstability.
Finally,whenapplyinganautomaticfeatureselectionalgorithmtothelteredset,accuraciesin-creasetorangefrom83%foremotionalstabilityto93%foragreeableness.
Whentestingwhetherthesemodelsgeneralisetoadierentcorpusofweblogs,NowsonandOberlander(2007)reportbinaryclassicationaccuraciesrangingfrom55%forextraversionto65%forconscientiousness.
Interestingly,modelstrainedonthemostextremeinstancesoftheoriginalcorpusseemtooutperformmodelstrainedonthefullcorpus,althoughnolevelofsignicanceismentioned.
Thesestudiesshowthatcarefulfeatureselectiongreatlyimprovesclassicationaccuracy,andthatn-gramscanbeappropriatetomodelself-reportsofper-sonality,although,asOberlanderandNowsonpointout,suchfeaturesarelikelytoovert.
Itwouldthereforebeinterestingtotestinfutureworkwhetherthefeaturesetsusedheregeneralisetoanotherdataset.
OberlanderandNowson(2006)alsoreportresultsfor3-wayand5-wayclassication,inordertoapproximatethener-grainedcontinuouspersonalityratingsusedinpsychology(aswedowiththescalarmodelswepresenthere).
Theyobtainamaximumof44.
7%forextraversionwith5bins,usingrawn-grams(baselineis33.
8%).
Theseresultsarenotdirectlycomparabletooursbecausetheyareonadierentcorpus,withdierentfeaturesets.
Moreover,wehavenotprovidedresultsonsuchmultipleclassicationexperiments,becausesuchmodelscannottakeintoaccountthefactthatthedierentclassesarepartofatotalordering,andthustheresultingmodelsareforcedtoignoretheimportanceoffeaturesthatcorrelatewiththatorderingacrossallclasses.
Webelievethatregressionandrankingmodelsaremoreappropriateforner-grainedpersonalityrecognition(seeSections5and6).
Toevaluatethisclaim,werstmappedtheoutputofthebestclassiertoarankingandcompareditwiththeRankBoostmodels.
WetrainedaNaiveBayesclassierontheEARcorpuswithobserverreportsandallfeatures,using5equalsizebins.
2Foreachtestfoldofa10-foldcross-validation,wecomputedtherankinglossproducedbytheclassierbasedontheorderingoftheveclasses.
ResultsinTable26showthatRankBoostsignicantlyoutperformstheclassierforfourtraitsoutofve(pwledge,theresultspresentedherearethersttodemonstratestatisticallysignicantresultsfortextsandtorecognisepersonalityinconversation(Mairesse&Walker,2006a,2006b).
Wepresenttherstresultsapplyingregressionandrankingmodelsinordertomodelpersonalityrecognitionusingthecontinuousscalestraditionalinpsychology.
Wealsosystematicallyexaminetheuseofdierentfeaturesets,suggestedbypreviouspsy-cholinguisticresearch.
Althoughthesefeatureshavebeensuggestedbythepsycholinguisticliterature,reportedcorrelationswithpersonalityratingsaregenerallyweak:itwasnotobviousthattheywouldimproveaccuraciesofstatisticalmodelsonunseensubjects.
Computationalworkonmodellingpersonalityhasprimarilyfocusedonmethodsforexpressingpersonalityinvirtualagentsandtutorialsystems,andconceptsrelatedtoper-sonalitysuchaspoliteness,emotion,orsocialintelligence(Walker,Cahn,&Whittaker,1997;Andre,Klesen,Gebhard,Allen,&Rist,1999;Lester,Towns,&FitzGerald,1999;Wang,Johnson,Mayer,Rizzo,Shaw,&Collins,2005)interalia.
Studieshaveshownthatuserevaluationsofagentpersonalitydependontheuser'sownpersonality(Reeves&Nass,1996;Cassell&Bickmore,2003),suggestingthatanabilitytomodeltheuser'spersonality3.
Anonlinedemoandapersonalityrecognitiontoolbasedonthemodelspresentedinthispapercanbedownloadedfromwww.
dcs.
shef.
ac.
uk/cogsys/recognition.
html490RecognisingPersonalityinConversationandTextisrequired.
Modelssuchaswepresentherefortheautomaticrecognitionofuserpersonalityisonewaytoacquiresuchausermodel(Chu-Carroll&Carberry,1994;Thompson,G¨oker,&Langley,2004;Zukerman&Litman,2001).
Weplantotestthesemodelsasusermodelsinthecontextofanadaptivedialoguesystem.
Table27summarisesresultsforallthepersonalitytraitsandrecognitiontasksweanalysed.
Whatclearlyemergesisthatextraversionistheeasiesttraittomodelfromspokenlanguage,followedbyemotionalstabilityandconscientiousness.
Concerningwrit-tenlanguage,modelsofopennesstoexperienceproducethebestresultsforallrecognitiontasks.
Wecanalsoseethatfeatureselectionisveryimportant,assomeofthebestmodelsonlycontainasmallsubsetofthefullfeatureset.
Prosodicfeaturesareimportantformod-ellingobservedextraversion,emotionalstabilityandopennesstoexperience.
MRCfeaturesareusefulformodelsofemotionalstability,whileLIWCfeaturesarebenecialforalltraits.
Wealsoanalysedqualitativelywhichfeatureshadthemostinuenceinspecicmodels,forallrecognitiontasks,aswellasreportingcorrelationsbetweeneachfeatureandpersonalitytraitsinSection3.
3.
Althoughtheparametersofthealgorithmshavenotbeenoptimised,thebottomofTable27seemstoindicatethatsimplemodelslikeNaiveBayesorregressiontreestendtooutperformmorecomplexones(e.
g.
,supportvectormachines),conrmingresultsfromOberlanderandNowson(2006).
However,ourexperimentsonthelargeressayscorpus(morethan2,400texts)showthatsupportvectormachinesandboostingalgorithmspro-ducehigherclassicationaccuracies.
ItisthereforelikelythatthosealgorithmswouldalsoperformbetteronspokendataiftheyweretrainedonamuchlargercorpusthantheEARdataset,andiftheirparameterswereoptimised.
Wehypothesisedthatmodelsofobservedpersonalitywilloutperformmodelsofself-assessedpersonality.
Ourresultsdosuggestthatobservedpersonalitymaybeeasiertomodelthanself-reports,atleastinconversationaldata.
FortheEARcorpus,wendmanygoodresultswithmodelsofobservedpersonality,whilemodelsofself-assessedpersonalityneveroutperformthebaseline.
Thismaybeduetoobjectiveobserversusingsimilarcuesasourmodels,whileself-reportsofpersonalitymaybemoreinuencedbyfactorssuchasthedesirabilityofthetrait(Edwards,1953).
Hogan(1982)introducedthedistinctionbetweentheagent'sandtheobserver'sperspectiveinpersonalityassessment.
Whiletheagent'sperspectiveconceptuallytapsintoaperson'sidentity(or'personalityfromtheinside'),theobserver'sperspectiveincontrasttapsintoaperson'sreputation(or'personalityfromtheoutside').
Bothfacetsofpersonalityhaveimportantpsychologicalimplications.
Aperson'sidentityshapesthewaythepersonexperiencestheworld.
Aperson'sreputation,however,ispsychologicallynotlessimportant:itdetermineswhetherpeoplegethiredorred(e.
g.
,reputationofhonesty),getmarriedordivorced,getadoredorstigmatised.
Becauseitishardertoassess,thisobserver'sperspectivehasreceivedcomparativelylittleattentioninpsychology.
Giventhatineverydaylifepeopleactasobserversofotherpeople'sbehavioursmostofthetime,theexternalperspectivenaturallyhasbothhightheoreticalimportanceandsocialrelevance(Hogan,1982).
RecentresearchexploringthisissueinpsychologyisbasedontheBrunswikianLensmodel(Brunswik,1956),whichhasbeenusedextensivelyinrecentyearstoexplainthe'kerneloftruth'inthesocialperceptionofstrangers.
Useofthelensmodelinpersonalityresearchreectsthewidelysharedassumptionsthattheexpressionofpersonalityiscommu-491Mairesse,Walker,Mehl&MooreTaskClassicationRegressionRankingBaselinen/anone50%n/anone0%n/anone0.
50Self-reportmodelstrainedonwrittendata(essays):ExtraversionADALIWC56%LRMRC1%RankLIWC0.
44EmotionalstabilitySMOLIWC58%M5LIWC4%RankLIWC0.
42AgreeablenessSMOLIWC56%LRLIWC2%RankLIWC0.
46ConscientiousnessSMOLIWC56%M5LIWC2%RankLIWC0.
44OpennesstoexperienceSMOLIWC63%M5all7%RankLIWC0.
39Observerreportmodelstrainedonspokendata(EAR):ExtraversionNBall73%REPLIWC24%Rankprosody0.
26EmotionalstabilityNBall74%M5prosody15%RankMRC0.
39AgreeablenessNBall61%M5Rall*3%Rankall0.
31ConscientiousnessNBall68%M5RLIWC18%Rankall0.
33OpennesstoexperienceNBprosody65%M5type*1%RankLIWC0.
37Table27:Comparisonofthebestmodelsforeachtrait,forallthreerecognitiontasks.
Eachtableentrycontainsthealgorithm,thefeatureset,andthemodelperformance.
SeeSections3.
2and3.
4fordetails.
Dependingonthetask,theevaluationmet-riciseitherthe(1)classicationaccuracy;(2)percentageofimprovementovertheregressionbaseline;(3)rankingloss.
Asterisksindicateresultsthataren'tsignicantatthepwledgmentsWewouldliketothankJamesPennebakerforgivingusaccesstotheessaysdata.
ThisworkwaspartiallyfundedbyaRoyalSocietyWolfsonResearchMeritAwardtoMarilynWalker,andbyaViceChancellor'sstudentshiptoFrancoisMairesse.
493Mairesse,Walker,Mehl&MooreReferencesAllport,G.
W.
,&Odbert,H.
S.
(1936).
Traitnames:apsycho-lexicalstudy.
PsychologicalMonographs,47(1,WholeNo.
211),171–220.
Andre,E.
,Klesen,M.
,Gebhard,P.
,Allen,S.
,&Rist,T.
(1999).
Integratingmodelsofpersonalityandemotionsintolifelikecharacters.
InProceedingsoftheWorkshoponAectinInteractions-TowardsanewGenerationofInterfaces,pp.
136–149.
Argamon,S.
,Dhawle,S.
,Koppel,M.
,&Pennebaker,J.
(2005).
Lexicalpredictorsofpersonalitytype.
InProceedingsoftheJointAnnualMeetingoftheInterfaceandtheClassicationSocietyofNorthAmerica.
Biber,D.
(1988).
VariationacrossSpeechandWriting.
CambridgeUniversityPress.
Boersma,P.
(2001).
Praat,asystemfordoingphoneticsbycomputer.
GlotInternational,5(9/10),341–345.
Bono,J.
E.
,&Judge,T.
A.
(2004).
Personalityandtransformationalandtransactionalleadership:ameta-analysis.
JournalofAppliedPsychology,89(5),901–910.
Breck,E.
,Choi,Y.
,&Cardie,C.
(2007).
Identifyingexpressionsofopinionincontext.
InTwentiethInternationalJointConferenceonArticialIntelligence(IJCAI).
Brunswik,E.
(1956).
PerceptionandtheRepresentativeDesignofPsychologicalExperi-ments.
UniversityofCaliforniaPress,Berkeley,CA.
Byrne,D.
,&Nelson,D.
(1965).
Attractionasalinearfunctionofproportionofpositivereinforcements.
JournalofPersonalityandSocialPsychology,1,659–663.
Cassell,J.
,&Bickmore,T.
(2003).
Negotiatedcollusion:Modelingsociallanguageanditsrelationshipeectsinintelligentagents.
UserModelingandUser-AdaptedInteraction,13,89–132.
Chu-Carroll,J.
,&Carberry,S.
(1994).
Aplan-basedmodelforresponsegenerationincollaborativetask-orienteddialogue.
InProceedingsofthe12thNationalConferenceonArticialIntelligence(AAAI),pp.
799–805.
Coltheart,M.
(1981).
TheMRCpsycholinguisticdatabase.
QuarterlyJournalofExperi-mentalPsychology,33A,497–505.
Costa,P.
T.
,&McCrae,R.
R.
(1992).
NEOPI-RProfessionalManual.
PsychologicalAssessmentResources,Odessa,FL.
Dewaele,J.
-M.
,&Furnham,A.
(1999).
Extraversion:theunlovedvariableinappliedlinguisticresearch.
LanguageLearning,49(3),509–544.
Donnellan,M.
B.
,Conger,R.
D.
,&Bryant,C.
M.
(2004).
TheBigFiveandenduringmarriages.
JournalofResearchinPersonality,38,481–504.
494RecognisingPersonalityinConversationandTextEdwards,A.
L.
(1953).
Therelationshipbetweenthejudgeddesirabilityofatraitandtheprobabilitythatitwillbeendorsed.
JournalofAppliedPsychology,37,90–93.
Enos,F.
,Benus,S.
,Cautin,R.
,Graciarena,M.
,Hirschberg,J.
,&Shriberg,E.
(2006).
Personalityfactorsinhumandeceptiondetection:Comparinghumantomachineperformance.
InProceedingsofICSLP.
Eysenck,H.
J.
(1991).
Dimensionsofpersonality:16,5or3criteriaforataxonomicparadigm.
PersonalityandIndividualDierences,12(8),773–790.
Fast,L.
A.
,&Funder,D.
C.
(2007).
Personalityasmanifestinworduse:Correlationswithself-report,acquaintance-report,andbehavior.
JournalofPersonalityandSocialPsychology,inpress.
Freund,Y.
,Iyer,R.
,Schapire,R.
E.
,&Singer,Y.
(1998).
Anecientboostingalgorithmforcombiningpreferences.
InProceedingsofthe15thInternationalConferenceonMachineLearning,pp.
170–178.
Funder,D.
C.
(1995).
Ontheaccuracyofpersonalityjudgment:Arealisticapproach.
PsychologicalReview,102,652–670.
Funder,D.
C.
,&Sneed,C.
D.
(1993).
Behavioralmanifestationsofpersonality:Anecolog-icalapproachtojudgmentalaccuracy.
JournalofPersonalityandSocialPsychology,64(3),479–490.
Furnham,A.
(1990).
Languageandpersonality.
InGiles,H.
,&Robinson,W.
(Eds.
),HandbookofLanguageandSocialPsychology.
Winley.
Furnham,A.
,Jackson,C.
J.
,&Miller,T.
(1999).
Personality,learningstyleandworkperformance.
PersonalityandIndividualDierences,27,1113–1122.
Furnham,A.
,&Mitchell,J.
(1991).
Personality,needs,socialskillsandacademicachieve-ment:Alongitudinalstudy.
PersonalityandIndividualDierences,12,1067–1073.
Gill,A.
,&Oberlander,J.
(2003).
Perceptionofe-mailpersonalityatzero-acquaintance:Extraversiontakescareofitself;neuroticismisaworry.
InProceedingsofthe25thAnnualConferenceoftheCognitiveScienceSociety,pp.
456–461.
Gill,A.
(2003).
PersonalityandLanguage:TheProjectionandPerceptionofPersonalityinComputer-MediatedCommunication.
Ph.
D.
thesis,UniversityofEdinburgh.
Gill,A.
J.
,&Oberlander,J.
(2002).
Takingcareofthelinguisticfeaturesofextraversion.
InProceedingsofthe24thAnnualConferenceoftheCognitiveScienceSociety,pp.
363–368.
Goldberg,L.
R.
(1990).
Analternative"descriptionofpersonality":TheBig-Fivefactorstructure.
JournalofPersonalityandSocialPsychology,59,1216–1229.
Graciarena,M.
,Shriberg,E.
,Stolcke,A.
,Enos,F.
,Hirschberg,J.
,&Kajarekar,S.
(2006).
Combiningprosodic,lexicalandcepstralsystemsfordeceptivespeechdetection.
InProceedingsofIEEEICASSP.
495Mairesse,Walker,Mehl&MooreHeinrich,C.
U.
,&Borkenau,P.
(1998).
Deceptionanddeceptiondetection:Theroleofcross-modalinconsistency.
JournalofPersonality,66(5),687–712.
Herbrich,R.
,Graepel,T.
,&Obermayer,K.
(2000).
Largemarginrankboundariesforordinalregression.
InSmola,A.
J.
,Bartlett,P.
,Sch¨olkopf,B.
,&Schuurmans,D.
(Eds.
),AdvancesinLargeMarginClassiers,pp.
115–132.
MITPress,Cambridge,MA.
Heylighen,F.
,&Dewaele,J.
-M.
(2002).
Variationinthecontextualityoflanguage:anempiricalmeasure.
ContextinContext,SpecialissueofFoundationsofScience,7(3),293–340.
Hirschberg,J.
,Benus,S.
,Brenier,J.
M.
,Enos,F.
,Friedman,S.
,Gilman,S.
,Girand,C.
,Graciarena,M.
,Kathol,A.
,Michaelis,L.
,Pellom,B.
,Shriberg,E.
,&Stolcke,A.
(2005).
Distinguishingdeceptivefromnon-deceptivespeech.
InProceedingsofInterspeech'2005-Eurospeech.
Hogan,R.
(1982).
Asocioanalytictheoryofpersonality.
NebraskaSymposiumofMotivation,30,55–89.
Hogan,R.
,Curphy,G.
J.
,&Hogan,J.
(1994).
Whatweknowaboutleadership:Eective-nessandpersonality.
AmericanPsychologist,49(6),493–504.
John,O.
P.
,Donahue,E.
M.
,&Kentle,R.
L.
(1991).
The"BigFive"Inventory:Versions4aand5b.
Tech.
rep.
,Berkeley:UniversityofCalifornia,InstituteofPersonalityandSocialResearch.
John,O.
P.
,&Srivastava,S.
(1999).
TheBigFivetraittaxonomy:History,measurement,andtheoreticalperspectives.
InPervin,L.
A.
,&John,O.
P.
(Eds.
),Handbookofpersonalitytheoryandresearch.
NewYork:GuilfordPress.
Komarraju,M.
,&Karau,S.
J.
(2005).
TherelationshipbetweentheBigFivepersonalitytraitsandacademicmotivation.
PersonalityandIndividualDierences,39,557–567.
Lester,J.
C.
,Towns,S.
G.
,&FitzGerald,P.
J.
(1999).
Achievingaectiveimpact:Visualemotivecommunicationinlifelikepedagogicalagents.
TheInternationalJournalofArticialIntelligenceinEducation,10(3-4),278–291.
Liscombe,J.
,Venditti,J.
,&Hirschberg,J.
(2003).
Classifyingsubjectratingsofemotionalspeechusingacousticfeatures.
InProceedingsofInterspeech'2003-Eurospeech.
Mairesse,F.
,&Walker,M.
(2006a).
Automaticrecognitionofpersonalityinconversation.
InProceedingsofHLT-NAACL.
Mairesse,F.
,&Walker,M.
(2006b).
Wordsmarkthenerds:Computationalmodelsofper-sonalityrecognitionthroughlanguage.
InProceedingsofthe28thAnnualConferenceoftheCognitiveScienceSociety,pp.
543–548.
Mairesse,F.
,&Walker,M.
(2007).
PERSONAGE:Personalitygenerationfordialogue.
InProceedingsofthe45thAnnualMeetingoftheAssociationforComputationalLin-guistics(ACL),pp.
496–503.
496RecognisingPersonalityinConversationandTextMallory,P.
,&Miller,V.
(1958).
Apossiblebasisfortheassociationofvoicecharacteristicsandpersonalitytraits.
SpeechMonograph,25,255–260.
Marcus,D.
K.
,Lilienfeld,S.
O.
,Edens,J.
F.
,&Poythress,N.
G.
(2006).
IsantisocialpersonalitydisordercontinuousorcategoricalAtaxometricanalysis.
PsychologicalMedicine,36(11),1571–1582.
McLarney-Vesotski,A.
R.
,Bernieri,F.
,&Rempala,D.
(2006).
Personalityperception:Adevelopmentalstudy.
JournalofResearchinPersonality,40(5),652–674.
Mehl,M.
R.
,Gosling,S.
D.
,&Pennebaker,J.
W.
(2006).
Personalityinitsnaturalhabi-tat:Manifestationsandimplicitfolktheoriesofpersonalityindailylife.
JournalofPersonalityandSocialPsychology,90,862–877.
Mehl,M.
,Pennebaker,J.
,Crow,M.
,Dabbs,J.
,&Price,J.
(2001).
TheElectronicallyActivatedRecorder(EAR):Adeviceforsamplingnaturalisticdailyactivitiesandconversations.
BehaviorResearchMethods,Instruments,andComputers,33,517–523.
Mishne,G.
(2005).
Experimentswithmoodclassicationinblogposts.
InProceedingsofACMSIGIR2005WorkshoponStylisticAnalysisofTextforInformationAccess.
Nass,C.
,&Lee,K.
(2001).
Doescomputer-synthesizedspeechmanifestpersonalityexper-imentaltestsofrecognition,similarity-attraction,andconsistency-attraction.
JournalofExperimentalPsychology:Applied,7(3),171–181.
Newman,M.
L.
,Pennebaker,J.
W.
,Berry,D.
S.
,&Richards,J.
M.
(2003).
Lyingwords:Predictingdeceptionfromlinguisticstyle.
PersonalityandSocialPsychologyBulletin,29,665–675.
Norman,W.
T.
(1963).
Towardanadequatetaxonomyofpersonalityattributes:Replicatedfactorstructureinpeernominationpersonalityrating.
JournalofAbnormalandSocialPsychology,66,574–583.
Nowson,S.
,&Oberlander,J.
(2007).
Identifyingmorebloggers:Towardslargescaleperson-alityclassicationofpersonalweblogs.
InProceedingsoftheInternationalConferenceonWeblogsandSocialMedia.
Nunn,S.
(2005).
Preventingthenextterroristattack:Thetheoryandpracticeofhome-landsecurityinformationsystems.
JournalofHomelandSecurityandEmergencyManagement,2(3).
Oberlander,J.
,&Gill,A.
J.
(2006).
Languagewithcharacter:Astratiedcorpuscom-parisonofindividualdierencesine-mailcommunication.
DiscourseProcesses,42,239–270.
Oberlander,J.
,&Nowson,S.
(2006).
Whosethumbisitanywayclassifyingauthorperson-alityfromweblogtext.
InProceedingsofthe44thAnnualMeetingoftheAssociationforComputationalLinguistics(ACL).
497Mairesse,Walker,Mehl&MooreOudeyer,P.
-Y.
(2002).
Novelusefulfeaturesandalgorithmsfortherecognitionofemotionsinspeech.
InProceedingsofthe1stInternationalConferenceonSpeechProsody,pp.
547–550.
Pang,B.
,&Lee,L.
(2005).
Seeingstars:Exploitingclassrelationshipsforsentimentcategorizationwithrespecttoratingscales.
InProceedingsofthe43rdAnnualMeetingoftheAssociationforComputationalLinguistics(ACL),pp.
115–124.
Paunonen,S.
V.
,&Jackson,D.
N.
(2000).
WhatisbeyondtheBigFiveplenty!
.
JournalofPersonality,68(5),821–836.
Peabody,D.
,&Goldberg,L.
R.
(1989).
Somedeterminantsoffactorstructuresfrompersonality-traitdescriptor.
JournalofPersonalityandSocialPsychology,57(3),552–567.
Pennebaker,J.
W.
,Francis,M.
E.
,&Booth,R.
J.
(2001).
InquiryandWordCount:LIWC2001.
LawrenceErlbaum,Mahwah,NJ.
Pennebaker,J.
W.
,&King,L.
A.
(1999).
Linguisticstyles:Languageuseasanindividualdierence.
JournalofPersonalityandSocialPsychology,77,1296–1312.
Pennebaker,J.
W.
,Mehl,M.
,&Niederhoer,K.
(2003).
Psychologicalaspectsofnaturallanguageuse:Ourwords,ourselves.
AnnualReviewofPsychology,54,547–577.
Popescu,A.
,&Etzioni,O.
(2005).
Extractingproductfeaturesandopinionsfromreviews.
InProceedingsofHTL-EMNLP.
Reeves,B.
,&Nass,C.
(1996).
TheMediaEquation.
UniversityofChicagoPress.
Reiter,E.
,&Sripada,S.
G.
(2004).
Contextualinuencesonnear-synonymchoice.
InProceedingsoftheInternationalNaturalLanguageGenerationConference,pp.
161–170.
Rienks,R.
,&Heylen,D.
(2006).
Dominancedetectioninmeetingsusingeasilyobtainablefeatures.
InBourlard,H.
,&Renals,S.
(Eds.
),RevisedSelectedPapersofthe2ndJointWorkshoponMultimodalInteractionandRelatedMachineLearningAlgorithms,Vol.
3869ofLectureNotesinComputerScience.
SpringerVerlag.
Riggio,R.
E.
,Salinas,C.
,&Tucker,J.
(1988).
Personalityanddeceptionability.
PersonalityandIndividualDierences,9(1),189–191.
Rosenberg,A.
,&Hirschberg,J.
(2005).
Acoustic/prosodicandlexicalcorrelatesofcharis-maticspeech.
InProceedingsofInterspeech'2005-Eurospeech.
Rushton,J.
P.
,Murray,H.
G.
,&Erdle,S.
(1987).
Combiningtraitconsistencyandlearn-ingspecicityapproachestopersonality,withillustrativedataonfacultyteachingperformance.
PersonalityandIndividualDierences,8,59–66.
Schapire,R.
(1999).
Abriefintroductiontoboosting.
ProceedingsoftheSixteenthInterna-tionalJointConferenceonArticialIntelligence,2,1401–1406.
498RecognisingPersonalityinConversationandTextScherer,K.
R.
(1979).
Personalitymarkersinspeech.
InScherer,K.
R.
,&Giles,H.
(Eds.
),Socialmarkersinspeech,pp.
147–209.
CambridgeUniversityPress.
Scherer,K.
R.
(2003).
Vocalcommunicationofemotion:Areviewofresearchparadigms.
SpeechCommunication,40(1-2),227–256.
Siegman,A.
,&Pope,B.
(1965).
Personalityvariablesassociatedwithproductivityandverbaluencyintheinitialinterview.
InProceedingsofthe73rdAnnualConferenceoftheAmericanPsychologicalAssociation.
Sigurdsson,J.
F.
(1991).
Computerexperience,attitudestowardcomputersandpersonalitycharacteristicsinpsychologyundergraduates.
PersonalityandIndividualDierences,12(6),617–624.
Smith,B.
L.
,Brown,B.
L.
,Strong,W.
J.
,&Rencher,A.
C.
(1975).
Eectsofspeechrateonpersonalityperception.
LanguageandSpeech,18,145–152.
Somasundaran,S.
,Ruppenhofer,J.
,&Wiebe,J.
(2007).
Detectingarguingandsentimentinmeetings.
InProceedingsofthe8thSIGdialWorkshoponDiscourseandDialogue.
Stolcke,A.
,Ries,K.
,Coccaro,N.
,Shriberg,E.
,Bates,R.
,Jurafsky,D.
,Taylor,P.
,Martin,R.
,Ess-Dykema,C.
V.
,&Meteer,M.
(2000).
Dialogueactmodelingforautomatictaggingandrecognitionofconversationalspeech.
ComputationalLinguistics,26(3),339–371.
Stoyanov,V.
,Cardie,C.
,&Wiebe,J.
(2005).
Multi-perspectivequestionansweringusingtheOpQAcorpus.
InProceedingsofHLT-EMNLP.
Thompson,C.
A.
,G¨oker,M.
H.
,&Langley,P.
(2004).
Apersonalizedsystemforconver-sationalrecommendations.
JournalofArticialIntelligenceResearch,21,393–428.
Tucker,S.
,&Whittaker,S.
(2004).
Accessingmultimodalmeetingdata:Systems,prob-lemsandpossibilities.
LectureNotesinComputerScience,MachineLearningforMultimodalInteraction,3361,1–11.
Turney,P.
D.
(2002).
ThumbsuporthumbsdownSemanticorientationappliedtoun-spervisedclassicationofreviews.
InProceedingsofthe40thAnnualMeetingoftheAssociationforComputationalLinguistics(ACL),pp.
417–424.
Vogel,K.
,&Vogel,S.
(1986).
L'interlangueetlapersonalitedel'apprenant.
InternationalJournalofAppliedLinguistics,24(1),48–68.
Walker,M.
,Cahn,J.
E.
,&Whittaker,S.
J.
(1997).
Improvisinglinguisticstyle:Socialandaectivebasesforagentpersonality.
InProceedingsofthe1stConferenceonAutonomousAgents,pp.
96–105.
Walker,M.
,&Whittaker,S.
(1990).
Mixedinitiativeindialogue:aninvestigationintodiscoursesegmentation.
InProceedingsofthe28thAnnualMeetingoftheAssociationforComputationalLinguistics(ACL),pp.
70–78.
499Mairesse,Walker,Mehl&MooreWang,N.
,Johnson,W.
L.
,Mayer,R.
E.
,Rizzo,P.
,Shaw,E.
,&Collins,H.
(2005).
Thepolitenesseect:Pedagogicalagentsandlearninggains.
FrontiersinArticialIntel-ligenceandApplications,125,686–693.
Watson,D.
,&Clark,L.
A.
(1992).
Ontraitsandtemperament:Generalandspecicfactorsofemotionalexperienceandtheirrelationtothevefactormodel.
JournalofPersonality,60(2),441–76.
Webb,N.
,Hepple,M.
,&Wilks,Y.
(2005).
Erroranalysisofdialogueactclassication.
InProceedingsofthe8thInternationalConferenceonText,SpeechandDialogue.
Wiebe,J.
,&Rilo,E.
(2005).
Creatingsubjectiveandobjectivesentenceclassiersfromunannotatedtexts.
InProceedingsofthe6thInternationalConferenceonIntelligentTextProcessingandComputationalLinguistics.
Wiebe,J.
,Wilson,T.
,Bruce,R.
,Bell,M.
,&Martin,M.
(2004).
Learningsubjectivelanguage.
ComputationalLinguistic,30(3),277–308.
Wilson,T.
,Wiebe,J.
,&Hwa,R.
(2004).
Justhowmadareyoundingstrongandweakopinionclauses.
InProceedingsofthe19thNationalConferenceonArticialIntelligence(AAAI),pp.
761–769.
Witten,I.
H.
,&Frank,E.
(2005).
DataMining:Practicalmachinelearningtoolsandtechniques.
MorganKaufmann,SanFrancisco,CA.
Zukerman,I.
,&Litman,D.
(2001).
Naturallanguageprocessingandusermodeling:Syner-giesandlimitations.
UserModelingandUser-AdaptedInteraction,11(1-2),129–158.
500