ontologiesoscommerce

QALM:aBenchmarkforQuestionAnsweringoverLinkedMerchantWebsitesDataAmineHallili1,ElenaCabrio2,3,andCatherineFaronZucker11Univ.
NiceSophiaAntipolis,CNRS,I3S,UMR7271,SophiaAntipolis,Franceamine.
hallili@inria.
fr;faron@unice.
fr2INRIASophiaAntipolisMediterranee,SophiaAntipolis,Franceelena.
cabrio@inria.
fr3EURECOM,SophiaAntipolis,FranceAbstract.
Thispaperpresentsabenchmarkfortrainingandevaluat-ingQuestionAnsweringSystemsaimingatmediatingbetweenauser,expressinghisorherinformationneedsinnaturallanguage,andseman-ticdatainthecommercialdomainofthemobilephonesindustry.
WerstdescribetheRDFdatasetweextractedthroughtheAPIsofmer-chantwebsites,andtheschemasonwhichitrelies.
Wethenpresentthemethodologyweappliedtocreateasetofnaturallanguagequestionsexpressingpossibleuserneedsintheabovementioneddomain.
Suchquestionsethasthenbeenfurtherannotatedbothwiththecorrespond-ingSPARQLqueries,andwiththecorrectanswersretrievedfromthedataset.
1IntroductionTheevolutionofthee-commercedomain,especiallytheBusinessToClient(B2C),hasencouragedtheimplementationandtheuseofdedicatedapplica-tions(e.
g.
QuestionAnsweringSystems)tryingtoprovideend-userswithabet-terexperience.
Atthesametime,theuser'sneedsaregettingmoreandmorecomplexandspecic,especiallywhenitcomestocommercialproductswhosequestionsconcernmoreoftentheirtechnicalaspects(e.
g.
price,color,seller,etc.
).
Severalsystemsareproposingsolutionstoanswertotheseneeds,butmanychal-lengeshavenotbeenovercomeyet,leavingroomforimprovement.
Forinstance,federatingseveralcommercialknowledgebasesinoneknowledgebasehasnotbeenaccomplishedyet.
Also,understandingandinterpretingcomplexnaturallanguagequestionsalsoknownasn-relationquestionsseemstobeoneoftheambitioustopicsthatsystemsarecurrentlytryingtogureout.
InthispaperwepresentabenchmarkfortrainingandevaluatingQuestionAnswering(QA)Systemsaimingatmediatingbetweenauser,expressinghisorherinformationneedinnaturallanguage,andsemanticdatainthecommercialdomainofthemobilephoneindustry.
WerstdescribetheRDFdatasetthatwehaveextractedthroughtheAPIsofmerchantsites,andtheschemasonwhichitrelies.
Wethenpresentthemethodologyweappliedtocreateasetofnaturallan-guagequestionsexpressingpossibleuserneedsintheabovementioneddomain.
SuchquestionsethasthenbefurtherannotatedbothwiththecorrespondingSPARQLqueries,andwiththecorrectanswersretrievedfromthedataset.
2AMerchantSitesDatasetfortheMobilePhonesIndustryThissectiondescribestheQALM(QuestionAnsweringoverLinkedMerchantwebsites)ontology(Section2.
1),andtheRDFdataset(Section2.
2)webuiltbyextractingasampleofdatafromasetofcommercialwebsites.
2.
1QALMOntologyTheQALMRDFdatasetreliesontwoontologies:theMerchantSiteOntology(MSO)andthePhoneOntology(PO).
TogethertheybuilduptheQALMOn-tology.
4MSOmodelsgeneralconceptsofmerchantwebsites,anditisalignedtothecommercialpartoftheSchema.
orgontology.
MSOiscomposedof5classes:mso:Product,mso:Seller,mso:Organization,mso:Store,mso:ParcelDelive-ry,andof29properties(e.
g.
mso:price,mso:url,mso:location,mso:seller)declaredassubclassesandsubpropertiesofSchema.
orgclassesandproperties.
Weaddedtothemmultilinguallabels(bothinEnglishandinFrench),thatcanbeexploitedbyQAsystemsinparticularforpropertyidenticationinthequestioninterpretationstep.
WereliedonWordNetsynonyms[2]toextractasmuchlabelsaspossible.
Forexample,thepropertymso:pricehasthefollowingEnglishlabels:"price","cost","value","tari","amount",andthefollowingFrenchlabels:"prix","cout","couter","valoir","tarif","s'elever".
POisadomainontologymodelingconceptsspecictothephoneindus-try.
Itiscomposedof7classes(e.
g.
po:Phone,po:Accessory)whicharede-claredassubclassesofmso:Product,andof35properties(e.
g.
po:handsetType,po:operatingSystem,po:phoneStyle).
2.
2QALMRDFDatasetOurnalgoalistobuildauniedRDFdatasetintegratingcommercialproductdescriptionsfromvariouse-commercewebsites.
Inordertoachievethisgoal,weanalyzethewebservicesofthee-commercewebsitesregardlessoftheirtype(eitherSOAPorREST).
Tofeedourdataset,wecreateamappingbetweentheremotecallstothewebservicesandtheontologyproperties,thatwestoreinaseparateleforreuse.
Inparticular,webuilttheQALMRDFdatasetbyextractingdatafromeBay5andBestBuy6commercialwebsitesthroughBestBuyWebserviceandeBayAPI.
TheextractedrawdataistransformedintoRDFtriplesbyapplyingtheabovedescribedmappingbetweentheQALMontology4Availableatwww.
i3s.
unice.
fr/qalm/ontology5http://www.
ebay.
com/6http://www.
bestbuy.
com/andtheAPI/webservice.
Forinstance,themethodgetPrice()intheeBayAPIismappedtothepropertymso:priceintheQALMontology.
Currently,theQALMdatasetcomprises500000productdescriptionsandupto15millionstriplesextractedfromeBayandBestBuy.
73QALMQuestionSetInordertotrainandtoevaluateaQAsystemmediatingbetweenauserandsemanticdataintheQALMdataset,asetofquestionsrepresentingusersre-questsinthephoneindustrydomainisrequired.
Uptoourknowledge,theonlyavailablestandardsetsofquestionstoevaluateQAsystemsoverlinkeddataaretheonesreleasedbytheorganizersoftheQALD(QuestionAnsweringoverLinkedData)challenges.
8HoweversuchquestionsareovertheEnglishDBpediadataset9,andthereforecoverseveraltopics.
Forthisreason,wecreatedasetofnaturallanguagequestionsforthespeciccommercialdomainofthephoneindustry,followingtheguidelinesdescribedbytheQALDorganizersforthecreationoftheirquestionsets[1].
Morespecically,thesequestionswerecre-atedby12externalpeople(studentsandresearchersinothergroups)withnobackgroundinquestionanswering,inordertoavoidabiastowardsaparticularapproach.
Toaccomplishthetaskofquestioncreation,eachpersonwasgiveni)thelistoftheproducttypespresentintheQALMdataset(mainlycomposedofITproductsasphonesandaccessories);ii)thelistofthepropertiesoftheQALMontologypresentedasproductfeaturesinwhichtheycouldbeinterestedin;andtheywereaskedtoproducei)both1-relationand2-relationquestions,andii)atleast5questionseach.
Thequestionsweredesignedtopresentpotentialuserquestionsandtoincludeawiderangeofchallengessuchaslexicalambiguitiesandcomplexsyntacticalstructures.
SuchquestionswerethenannotatedwiththecorrespondingSPARQLqueries,andthecorrectanswersretrievedfromthedataset,inordertoconsiderthemasareliablegoldstandardforourbenchmark.
Thenalquestionsetcomprises70questions;itisdividedintoatrainingset10andatestsetofrespectively40and30questions.
AnnotationsareprovidedinXMLformat,andaccordingtoQALDguidelines,thefollowingattributesarespeciedforeachquestionalongwithitsID:aggregation(indicateswhetheranyoperationbeyondtriplepatternmatchingisrequiredtoanswerthequestion,e.
g.
,counting,ltering,ordering),answertype(givestheanswertype:resource,string,boolean,double,date).
Wealsoaddedtheattributerelations,toindicatewhetherthequestionisconnectedtoitsanswerthroughoneormorepropertiesoftheontology(values:1,n).
Finally,foreachquestionthecorrespondingSPARQLqueryisprovided,aswellastheanswersthisqueryreturns.
Examples1and2showsomequestionsfromthecollectedquestionset,connectedtotheiranswersthrough1propertyormorethan1propertyoftheontology,respectively.
In7Availableatwww.
i3s.
unice.
fr/QALM/qalm.
rdf8http://greententacle.
techfak.
uni-bielefeld.
de/~cunger/qald/9http://dbpedia.
org10Availableatwww.
i3s.
unice.
fr/QALM/training_questions.
xmlparticular,questions14and50fromExample2requirealsotocarryoutsomereasoningontheresults,inordertorankthemandtoproducethecorrectanswer.
Example1.
1-relationquestions.
id=36.
Givemethemanufacturerswhosupplyon-earheadphones.
id=52.
WhatcolorsareavailablefortheSamsungGalaxy5id=61.
WhichproductsofAlcatelareavailableonlineExample2.
n-relationsquestions.
id=14.
Whichcellphonecase(anymanufacturer)hasthemostratingsid=50.
WhatisthehighestcameraresolutionofphonesmanufacturedbyMotorolaid=58.
IwouldliketoknowinwhichstoresIcanbuyApplephones.
4ConclusionsandOngoingWorkThispaperpresentedabenchmarktotrainandtestQAsystems,composedofi)theQALMontologies;ii)theQALMRDFdatasetofproductdescriptionsex-tractedfromeBayandBestBuy;andiii)theQALMQuestionSet,containing70naturallanguagequestionsinthecommercialdomainofphonesandaccessories.
Asforfuturework,wewillconsideraligningtheQALMontologytotheGoodRelationsontologytofullycoverthecommercialdomain,andtobenetfromthesemanticscapturedinthisontology.
WealsoconsiderimprovingtheQALMRDFdatasetbyi)extractingRDFdatafromadditionalcommercialwebsitesthatprovidewebservicesorAPIs;andii)directlyextractingRDFdataintheSchema.
orgontologyfromcommercialwebsiteswhosepagesareautomaticallygeneratedwithSchema.
orgmarkup(e.
g.
Magento,OSCommerce,Genesis2.
0,Prestashop),toextendthenumberofaddressedcommercialwebsites.
Inparallel,wearecurrentlydevelopingtheSynchroBotQAsystem[3],anontology-basedchatbotforthee-commercedomain.
WewillevaluateitbyusingtheproposedQALMbenchmark.
AcknowledgementsWethankAmazon,eBayandBestBuyforcontributingtothisworkbysharingwithuspublicdataabouttheircommercialproducts.
TheworkofE.
CabriowasfundedbytheFrenchGovernmentthroughtheANR-11-LABX-0031-01program.
References1.
Cimiano,P.
,Lopez,V.
,Unger,C.
,Cabrio,E.
,Ngomo,A.
C.
N.
,Walter,S.
:Multi-lingualquestionansweringoverlinkeddata(qald-3):Laboverview.
In:CLEF.
pp.
321–332(2013)2.
Fellbaum,C.
:WordNet:AnElectronicLexicalDatabase.
BradfordBooks(1998)3.
Hallili,A.
:Towardanontology-basedchatbotendowedwithnaturallanguagepro-cessingandgeneration.
In:Proc.
ofESSLLI2014-StudentSession,Posterpaper(2014)