refer663uu.com

ShapeNet:AnInformation-Rich3DModelRepositoryhttp://www.
shapenet.
orgAngelX.
Chang1,ThomasFunkhouser2,LeonidasGuibas1,PatHanrahan1,QixingHuang3,ZimoLi3,SilvioSavarese1,ManolisSavva1,ShuranSong2,HaoSu1,JianxiongXiao2,LiYi1,andFisherYu21StanfordUniversity—2PrincetonUniversity—3ToyotaTechnologicalInstituteatChicagoAuthorslistedalphabeticallyAbstractWepresentShapeNet:arichly-annotated,large-scalerepositoryofshapesrepresentedby3DCADmodelsofob-jects.
ShapeNetcontains3DmodelsfromamultitudeofsemanticcategoriesandorganizesthemundertheWord-Nettaxonomy.
Itisacollectionofdatasetsprovidingmanysemanticannotationsforeach3Dmodelsuchasconsis-tentrigidalignments,partsandbilateralsymmetryplanes,physicalsizes,keywords,aswellasotherplannedanno-tations.
Annotationsaremadeavailablethroughapub-licweb-basedinterfacetoenabledatavisualizationofob-jectattributes,promotedata-drivengeometricanalysis,andprovidealarge-scalequantitativebenchmarkforresearchincomputergraphicsandvision.
Atthetimeofthistechni-calreport,ShapeNethasindexedmorethan3,000,000mod-els,220,000modelsoutofwhichareclassiedinto3,135categories(WordNetsynsets).
InthisreportwedescribetheShapeNeteffortasawhole,providedetailsforallcurrentlyavailabledatasets,andsummarizefutureplans.
1.
IntroductionRecenttechnologicaldevelopmentshaveledtoanex-plosionintheamountof3Ddatathatwecangenerateandstore.
Repositoriesof3DCADmodelsareexpandingcon-tinuously,predominantlythroughaggregationof3Dcontentontheweb.
RGB-Dsensorsandothertechnologyforscan-ningandreconstructionareprovidingincreasinglyhigherdelitygeometricrepresentationsofobjectsandrealenvi-ronmentsthatcaneventuallybecomeCAD-qualitymodels.
Atthesametime,therearemanyopenresearchprob-lemsduetofundamentalchallengesinusing3Dcontent.
Computingsegmentationsof3Dshapes,andestablishingcorrespondencesbetweenthemaretwobasicproblemsingeometricshapeanalysis.
Recognitionofshapesfrompar-Contactauthors:{msavva,haosu}@cs.
stanford.
edutialscansisaresearchgoalsharedbycomputergraphicsandvision.
Sceneunderstandingfrom2Dimagesisagrandchallengeinvisionthathasrecentlybenetedtremendouslyfrom3DCADmodels[28,34].
Navigationofautonomousrobotsandplanningofgraspingmanipulationsaretwolargeareasinroboticsthatbenetfromanunderstandingof3Dshapes.
Attherootofalltheseresearchproblemsliestheneedforattachingsemanticstorepresentationsof3Dshapes,anddoingsoatlargescale.
Recently,data-drivenmethodsfromthemachinelearn-ingcommunityhavebeenexploitedbyresearchersinvisionandNLP(naturallanguageprocessing).
"Bigdata"inthevisualandtextualdomainshasledtotremendousprogresstowardsassociatingsemanticswithcontentinbothelds.
Mirroringthispattern,recentworkincomputergraphicshasalsoappliedsimilarapproachestospecicproblemsinthesynthesisofnewshapevariations[10]andnewarrange-mentsofshapes[6].
However,acriticalbottleneckfacingtheadoptionofdata-drivenmethodsfor3Dcontentisthelackoflarge-scale,curateddatasetsof3Dmodelsthatareavailabletothecommunity.
Motivatedbythefar-reachingimpactofdataseteffortssuchasthePennTreebank[20],WordNet[21]andIma-geNet[4],whichcollectivelyhavetensofthousandsofci-tations,weproposeestablishingShapeNet:alarge-scale3Dmodeldataset.
Makingacomprehensive,semanticallyen-richedshapedatasetavailabletothecommunitycanhaveimmenseimpact,enablingmanyavenuesoffutureresearch.
InconstructingShapeNetweaimtofulllseveralgoals:Collectandcentralize3Dmodeldatasets,helpingtoorganizeeffortintheresearchcommunity.
Supportdata-drivenmethodsrequiring3Dmodeldata.
Enableevaluationandcomparisonofalgorithmsforfundamentaltasksinvolvinggeometry(e.
g.
,segmen-tation,alignment,correspondence).
Serveasaknowledgebaseforrepresentingreal-worldobjectsandtheirsemantics.
1ThesegoalsimplyseveraldesiderataforShapeNet:Broadanddeepcoverageofobjectsobservedintherealworld,withthousandsofobjectcategoriesandmillionsoftotalinstances.
Categorizationschemeconnectedtoothermodalitiesofknowledgesuchas2Dimagesandlanguage.
Annotationofsalientphysicalattributesonmodels,suchascanonicalorientations,planesofsymmetry,andpartdecompositions.
Web-basedinterfacesforsearching,viewingandre-trievingmodelsinthedatasetthroughseveralmodali-ties:textualkeywords,taxonomytraversal,imageandshapesimilaritysearch.
Achievingthesegoalsandprovidingtheresultingdatasettothecommunitywillenablemanyadvancesandapplica-tionsincomputergraphicsandvision.
Inthisreport,werstsituateShapeNet,explainingtheoverallgoalsoftheeffortandthetypesofdataitisin-tendedtocontain,aswellasmotivatingthelong-termvi-sionandinfrastructuraldesigndecisions(Section3).
Wethendescribetheacquisitionandvalidationofannotationscollectedsofar(Section4),summarizethecurrentstateofallavailableShapeNetdatasets,andprovidebasicstatisticsonthecollectedannotations(Section5).
Weendwithadis-cussionofShapeNet'sfuturetrajectoryandconnectitwithseveralresearchdirections(Section7).
2.
BackgroundandRelatedWorkTherehasbeensubstantialgrowthinthenumberofof3Dmodelsavailableonlineoverthelastdecade,withrepos-itoriesliketheTrimble3DWarehouseprovidingmillionsof3Dpolygonalmodelscoveringthousandsofobjectandscenecategories.
Yet,therearefewcollectionsof3Dmod-elsthatprovideusefulorganizationandannotations.
Mean-ingfultextualdescriptionsarerarelyprovidedforindivid-ualmodels,andonlinerepositoriesareusuallyeitherun-organizedorgroupedintogrosscategories(e.
g.
,furniture,architecture,etc.
[7]).
Asaresult,theyhavebeenpoorlyutilizedinresearchandapplications.
Therehavebeenpreviouseffortstobuildorganizedcol-lectionsof3Dmodels(e.
g.
,[5,7]).
However,theyhaveprovidedquitesmalldatasets,coveredonlyasmallnum-berofsemanticcategories,andincludedfewstructuralandsemanticannotations.
Mostofthesepreviouscollectionshavebeendevelopedforevaluatingshaperetrievalandclas-sicationalgorithms.
Forexample,datasetsarecreatedan-nuallyfortheShapeRetrievalContest(SHREC)thatcom-monlycontainssetsofmodelsorganizedinobjectcate-gories.
However,thosedatasetsareverysmall—themostrecentSHRECiterationin2014[17]containsa"large"datasetwitharound9,000modelsconsistingofmodelsfromavarietyofsourcesorganizedinto171categories(Table1).
ThePrincetonShapeBenchmarkisprobablythemostwell-knownandfrequentlyused3Dshapecollectiontodate(withover1000citations)[27].
Itcontainsaround1,8003Dmodelsgroupedinto90categories,buthasnoannotationsbeyondcategorylabels.
Othercommonly-useddatasetscontainsegmentations[2],correspondences[13,12],hier-archies[19],symmetries[11],salientfeatures[3],seman-ticsegmentationsandlabels[36],alignmentsof3Dmodelswithimages[35],semanticontologies[5],andotherfunc-tionalannotations—butagainonlyforsmallsizedatasets.
Forexample,theBenchmarkfor3DMeshSegmentationcontainsjust380modelsin19objectclasses[2].
Incontrast,therehasbeenaurryofactivityoncollect-ing,organizing,andlabelinglargedatasetsincomputervi-sionandrelatedelds.
Forexample,ImageNet[4]providesasetof14Mimagesorganizedinto20Kcategoriesasso-ciatedwith"synsets"ofWordNet[21].
LabelMeprovidessegmentationsandlabelannotationsofhundredsofthou-sandsofobjectsintensofthousandsofimages[24].
TheSUNdatasetprovides3Mannotationsofobjectsin4Kcat-egoriesappearingin131Kimagesof900typesofscenes.
Recentworkdemonstratedthebenetofalargedatasetof120K3DCADmodelsintrainingaconvolutionalneu-ralnetworkforobjectrecognitionandnext-bestviewpre-dictioninRGB-Ddata[34].
Largedatasetssuchasthisandothers(e.
g.
,[14,18])haverevitalizeddata-drivenal-gorithmsforrecognition,detection,andeditingofimages,whichhaverevolutionizedcomputervision.
Similarly,largecollectionsofannotated3Ddatahavehadgreatinuenceonprogressinotherdisciplines.
Forex-ample,theProteinDataBank[1]providesadatabasewith100Kprotein3Dstructures,eachlabeledwithitssourceandlinkstostructuralandfunctionalannotations[15].
Thisdatabaseisacommonrepositoryofall3Dproteinstructuressolvedtodateandprovidesasharedinfrastructureforthecollectionandtransferofknowledgeabouteachentry.
Ithasacceleratedthedevelopmentofdata-drivenalgorithms,fa-cilitatedthecreationofbenchmarks,andlinkedresearchersandindustryfromaroundtheworld.
Weaimtoprovideasimilarresourcefor3Dmodelsofeverydayobjects.
3.
ShapeNet:AnInformation-Rich3DModelRepositoryShapeNetisalarge,information-richrepositoryof3Dmodels.
Itcontainsmodelsspanningamultitudeofseman-ticcategories.
Unlikeprevious3Dmodelrepositories,itprovidesextensivesetsofannotationsforeverymodelandlinksbetweenmodelsintherepositoryandothermultime-diadataoutsidetherepository.
LikeImageNet,ShapeNetprovidesaviewofthecon-taineddatainahierarchicalcategorizationaccordingtoWordNetsynsets(Figure1).
Unlikeothermodelreposi-tories,ShapeNetalsoprovidesarichsetofannotationsfor2BenchmarksTypes#models#classesAvg#modelsperclassSHREC14LSGTBGeneric8,98717153PSBGeneric907+907(train+test)90+92(train+test)10+10(train+test)SHREC12GTBGeneric12006020TSBGeneric10,00035228CCCCGeneric473559WMBWatertight(articulated)4002020MSBArticulated4571924BABArchitecture2257183+180(function+form)12+13(function+form)ESBCAD8674519Table1.
SourcedatasetsfromSHREC2014:PrincetonShapeBenchmark(PSB)[27],SHREC2012genericShapeBenchmark(SHREC12GTB)[16],ToyohashiShapeBenchmark(TSB)[29],Konstanz3DModelBenchmark(CCCC)[32],WatertightModelBench-mark(WMB)[31],McGill3DShapeBenchmark(MSB)[37],BonnArchitectureBenchmark(BAB)[33],PurdueEngineeringShapeBenchmark(ESB)[9].
eachshapeandcorrespondencesbetweenshapes.
Thean-notationsincludegeometricattributessuchasuprightandfrontorientationvectors,partsandkeypoints,shapesym-metries(reectionplane,otherrotationalsymmetries),andscaleofobjectinrealworldunits.
Theseattributesprovidevaluableresourcesforprocessing,understandingandvisu-alizing3Dshapesinawaythatisawareofthesemanticsoftheshape.
Wehavecurrentlycollectedapproximately3millionshapesfromonline3Dmodelrepositories,andcategorized300thousandofthemagainsttheWordNettaxonomy.
Wehavealsoannotatedasubsetofthesemodelswithshapepropertiessuchasuprightandfrontorientations,symme-tries,andhierarchicalpartdecompositions.
Wearecontin-uingtheprocessofexpandingtheannotatedsetofmodelsandalsocollectingnewmodelsfromnewdatasources.
Inthefollowingsections,wediscusshow3DmodelsarecollectedforShapeNet,whatannotationswillbeadded,howthoseannotationswillbegenerated,howannotationswillbeupdatedasthedatasetevolvesovertime,andwhattoolswillbeprovidedforthecommunitytosearch,browse,andutilizeexistingdata,aswellascontributenewdata.
3.
1.
DataCollectionTheraw3DmodeldataforShapeNetcomesfrompubliconlinerepositoriesorexistingresearchdatasets.
ShapeNetisintendedtobeanevolvingrepositorywithregularupdatesasmoreandmore3Dmodelsbecomeavailable,asmorepeoplecontributeannotations,andasthedatacapturedwithnew3Dsensorsbecomeprevalent.
Wehavecollected3Dpolygonalmodelsfromtwopopularpublicrepositories:Trimble3DWarehouse1andYobi3D2.
TheTrimble3DWarehousecontains2.
4Muser-designed3Dmodelsandscenes.
Yobi3Dcontains350Kadditionalmodelscollectedfromawiderangeofotheron-linerepositories.
Together,theyprovideadiversesetof1https://3dwarehouse.
sketchup.
com/2https://yobi3d.
comFigure1.
ScreenshotoftheonlineShapeNettaxonomyview,or-ganizingcontained3DmodelsunderWordNetsynsets.
shapesfromabroadsetofobjectandscenecategories—e.
g.
,manyorganicshapecategories(e.
g.
,humansandmammals),whicharerareinWarehouse3D,areplentifulinYobi3D.
Formoredetailedstatisticsonthecurrentlyavail-ableShapeNetmodelsrefertoSection5.
Thoughthetoolsdevelopedforthisprojectwillbegeneral-purpose,weintendtoincludeonly3Dmodelsofobjectsencounteredbypeopleintheeverydayworld.
Thatis,itwillnotincludeCADmechanicalparts,molecularstructures,orotherdomain-specicobjects.
However,wewillincludescenes(e.
g.
,ofce),objects(e.
g.
,laptopcom-puter),andpartsofobjects(e.
g.
,keyboard).
ModelsareorganizedunderWordNet[21]noun"synsets"(synonymsets).
WordNetprovidesabroadanddeeptaxonomywithover80Kdistinctsynsetsrepresentingdistinctnouncon-ceptsarrangedasaDAGnetworkofhyponymrelationships(e.
g.
,"canary"isahyponymof"bird").
ThistaxonomyhasbeenusedbyImageNettodescribecategoriesofobjectsat3multiplescales[4].
ThoughwerstuseWordNetduetoitspopularity,theShapeNetUIisdesignedtoallowmultipleviewsintothecollectionofshapesthatitcontains,includ-ingdifferenttaxonomyviewsandfacetednavigation.
3.
2.
AnnotationTypesWeenvisionShapeNetasfarmorethanacollectionof3Dmodels.
ShapeNetwillincludearichsetofannota-tionsthatprovidesemanticinformationaboutthosemod-els,establishlinksbetweenthem,andlinkstoothermodal-itiesofdata(e.
g.
,images).
TheseannotationsareexactlywhatmakeShapeNetuniquelyvaluable.
Figure2illustratesthevalueofthisdensenetworkofinterlinkedattributesonshapes,whichwedescribebelow.
Language-relatedAnnotations:Namingobjectsbytheirbasiccategoryisusefulforindexing,grouping,andlinkingtorelatedsourcesofdata.
Asdescribedinthepre-vioussection,weorganizeShapeNetbasedontheWord-Net[21]taxonomy.
Synsetsareinterlinkedwithvariousrelations,suchashyperandhyponym,andpart-wholerela-tions.
DuetothepopularityofWordNet,wecanleverageotherresourceslinkedtoWordNetsuchasImageNet,Con-ceptNet,Freebase,andWikipedia.
Inparticular,linkingtoImageNet[4]willhelptransportinformationbetweenim-agesandshapes.
Weassigneach3DmodelinShapeNettooneormoresynsetsintheWordNettaxonomy(i.
e.
,wepop-ulateeachsynsetwithacollectionofshapes).
PleaserefertoSection4.
1fordetailsontheacquisitionandvalidationofbasiccategoryannotations.
Futureplannedannotationsincludenaturallanguagedescriptionsofobjectsandobjectpart-partrelationdescriptions.
GeometricAnnotations:Acriticalpropertythatdistin-guishesShapeNetfromimageandvideodatasetsisthe-delitywithwhich3Dgeometryrepresentsreal-worldstruc-tures.
Wecombinealgorithmicpredictionsandmanualannotationstoorganizeshapesbycategory-levelgeomet-ricpropertiesandfurtherderiverichgeometricannotationsfromtheraw3Dmodelgeometry.
RigidAlignments:Establishingaconsistentcanon-icalorientation(e.
g.
,uprightandfront)foreverymodelisimportantforvarioustaskssuchasvisual-izingshapes[13],shapeclassication[8]andshaperecognition[34].
Fortunately,mostraw3Dmodeldataisbydefaultplacedinanuprightorientation,andthefrontorientationsaretypicallyalignedwithanaxis.
Thisallowsustouseahierarchicalclusteringandalignmentapproachtoensureconsistentrigidalign-mentswithineachcategory(seeSection4.
2).
PartsandKeypoints:Manyshapescontainorhavenaturaldecompositionsintoimportantparts,aswellassignicantkeypointsrelatedtoboththeirgeometryandtheirsemantics.
Forexample,oftendifferentmaterialsareassociatedwithdifferentparts.
Weintendtocap-tureasmuchofthataspossibleintoShapeNet.
Symmetry:Bilateralsymmetryplanesandrotationalsymmetriesareprevalentinarticialandnaturalob-jects,anddeeplyconnectedwiththealignmentandfunctionalityofshapes.
WerefertoSection4.
4formoredetailsonhowwecomputesymmetriesfortheshapesinShapeNet.
ObjectSize:Objectsizeisusefulformanyapplica-tions,suchasreducingthehypothesisspaceinobjectrecognition.
SizeannotationsarediscussedinSec-tion5.
2.
FunctionalAnnotations:Manyobjects,especiallyman-madeartifactssuchasfurnitureandappliances,canbeusedbyhumans.
Functionalannotationsdescribetheseusagepatterns.
Suchannotationsareoftenhighlycorrelatedwithspecicregionsofanobject.
Inaddition,itisoftenrelatedwiththespecictypeofhumanaction.
ShapeNetaimstostorefunctionalannotationsattheglobalshapelevelandattheobjectpartlevel.
FunctionalParts:Partsarecriticalforunderstand-ingobjectstructure,humanactivitiesinvolvinga3Dshape,andergonomicproductdesign.
Weplantoan-notatepartsaccordingtotheirfunction—infacttheverydenitionofpartshastobebasedonbothgeo-metricandfunctionalcriteria.
Affordances:Weareinterestedinaffordanceannota-tionsthatarefunctionandactivityspecic.
Examplesofsuchannotationsincludesupportingplaneannota-tions,andgraspableregionannotationsforvariousob-jectmanipulations.
PhysicalAnnotations:Realobjectsexistinthephysicalworldandtypicallyhavexedphysicalpropertiessuchasdimensionsanddensities.
Thus,itisimportanttostorephysicalattributeannotationsfor3Dshapes.
SurfaceMaterial:Weareespeciallyinterestedintheopticalpropertiesandsemanticnamesofsurfacemate-rials.
Theyareimportantforapplicationssuchasren-deringandstructuralstrengthestimation.
Weight:Abasicpropertyofobjectswhichisveryuse-fulforphysicalsimulations,andreasoningaboutsta-bilityandstaticsupport.
Ingeneral,theissueofcompactandinformativerep-resentationsforalltheaboveattributesovershapesraisesmanyinterestingquestionsthatwewillneedtoaddressaspartoftheShapeNeteffort.
Manyannotationsarecur-rentlyongoingprojectsandinvolveinterestingopenre-searchproblems.
4Figure2.
ShapeNetannotationsillustratedforanexamplechairmodel.
Left:linkstotheWordNettaxonomyprovidedenitionsofobjects,is-aandhas-arelations,andaconnectiontoimagesfromImageNet.
Middle-left:shapeisalignedtoaconsistentuprightandfrontorientation,andsymmetriesarecomputedMiddle-right:hierarchicaldecompositionofshapeintopartsonwhichvariousattributesaredened:names,symmetries,dimensions,materials,andmasses.
Right:part-to-partandpoint-to-pointconnectionsareestablishedatalllevelswithinShapeNetproducingadenseandsemanticallyrichnetworkofcorrespondences.
Thegraybackgroundindicatesannotationsthatarecurrentlyongoingandnotyetavailableforrelease.
3.
3.
AnnotationMethodologyThoughatrstglanceitmightseemreasonabletocollecttheannotationswedescribepurelythroughmanualhumaneffort,wewillingeneraltakeahybridapproach.
Foranno-tationtypeswhereitispossible,wewillrstalgorithmicallypredicttheannotationforeachmodelinstance(e.
g.
,globalsymmetryplanes,consistentrigidalignments).
Wewillthenverifythesepredictionsthroughcrowd-sourcingpipelinesandinspectionbyhumanexperts.
Thishybridstrategyissensibleinthecontextof3Dshapedataastherearealreadyvariousalgorithmswecanleverage,andcollectingcorre-spondingannotationsentirelythroughmanualeffortcanbeextremelylaborintensive.
Inparticular,sinceobjectsina3Drepresentationarebothmorepureandmorecompletethanobjectsinimages,wecanexpectbetterandeasiertoestablishcorrespondencesbetween3Dshapes,enablingal-gorithmictransportofsemanticannotations.
Inmanycases,thedesignofthehumanannotationinterfacesthemselvesisanopenquestion—whichstandsincontrasttolargelyman-ualimagelabelingeffortssuchasImageNet.
Asaconcreteexample,shapepartannotationcanbepresentedandper-formedinvariouswayswithdifferenttrade-offsinthetypeofobtainedpartannotation,theaccuracyandtheefciencyoftheannotationprocess.
Coupledwiththishybridannotationstrategy,wealsotakeparticularcaretopreservetheprovenanceandcon-denceofeachalgorithmicandhumanannotation.
Theanno-tationsource(whetheranalgorithm,orhumaneffort),andameasureofthetrustwecanplaceineachannotationarecriticalpiecesofinformationespeciallywhenwehavetocombine,aggregate,andreconcileseveralannotations.
3.
4.
AnnotationSchemaandWebAPIToprovideconvenientaccesstoallofthemodelandan-notationdatacontainedwithinShapeNet,weconstructanindexoverallthe3Dmodelsandtheirassociatedannota-tionsusingtheApacheSolrframework.
3Eachstoredan-notationforagiven3Dmodeliscontainedwithintheindexasaseparateattributethatcanbeeasilyqueriedandlteredthroughasimpleweb-basedUI.
Inaddition,tomakethedatasetconvenientlyaccessibletoresearchers,weprovideabatcheddownloadcapability.
4.
AnnotationAcquisitionandValidationAkeychallengeinconstructingShapeNetisthemethod-ologyforacquiringandvalidatingannotations.
Ourgoalistoprovideallannotationswithhighaccuracy.
Incaseswherefullvericationisnotyetavailable,weaimtoes-timateacondencemetricforeachannotation,aswellasrecorditsprovenance.
Thiswillenableotherstoproperlyestimatethetrustworthinessoftheinformationweprovideanduseitfordifferentapplications.
4.
1.
CategoryAnnotationAsdescribedinSection3.
2,weassigneach3DmodeltooneormoresynsetsintheWordNettaxonomy.
AnnotationModelsareretrievedbytextualqueryintotheonlinerepositoriesthatwecollected,andtheinitialcategoryannotationissettotheusedtextualqueryforeachretrieved3http://lucene.
apache.
org/solr/5model.
Afterweretrievethesemodelsweusethepopular-ityscoreofeachmodelontherepositorytosortmodelsandaskhumanworkerstoverifytheassignedcategoryannota-tion.
Thisissensiblesincethemorepopularmodelstendtobehighqualityandcorrectlyretrievedthroughthecategorykeywordtextualquery.
Westopverifyingcategoryannota-tionswithpeopleoncethepositiveratioislowerthana2%threshold.
Clean-upInorderforthedatasettobeeasilyusablebyre-searchersitshouldcontaincleanandhighquality3Dmod-els.
Throughinspection,weidentifyandgroup3Dmodelsintothefollowingcategories:single3Dmodels,3Dscenes,billboards,andbiggroundplane.
Single3Dmodels:semanticallydistinctobjects;focusofourShapeNetCoreannotationeffort.
3Dscenes:detectedbycountingthenumberofcon-nectedcomponentsinavoxelizedrepresentation.
Wemanuallyverifythesedetectionsandmarkscenesforfutureanalysis.
Billboards:planeswithapaintedtexture.
Oftenusedtorepresentpeopleandtrees.
Thesemodelsaregen-erallynotusefulforgeometricanalysis.
Theycanbedetectedbycheckingwhetherasingleplanecantallvertices.
Biggroundplane:objectofinterestplacedonalargehorizontalplaneorinfrontoflargeverticalplane.
Al-thoughwedonotcurrentlyusethesemodels,theplanecaneasilybeidentiedandremovedthroughsimplegeometricanalysis.
Wecurrentlyincludethesingle3DmodelsintheShapeNetCoresubsetofShapeNet.
4.
2.
HierarchicalRigidAlignmentThegoalofthisstepistoestablishaconsistentcanon-icalorientationformodelswithineachcategory.
Suchalignmentisimportantforvarioustaskssuchasvisualizingshapes,shapeclassicationandshaperecognition.
Figure3showsseveralcategoriesinShapeNetthathavebeencon-sistentlyaligned.
Thoughtheconceptofconsistentorientationseemsnat-ural,oneissuehastobeaddressed.
Weexplainbyanex-ample.
"armchair","chair"and"seat"arethreecategoriesinourtaxonomy,eachbeingasubcategoryofitssucces-sor.
Consistentorientationcanbewelldenedforshapesinthe"armchair"category,bycheckingarms,legsandbacks.
Yet,itbecomesdifculttodeneforthe"chair"category.
Forexample,"sidechair"and"swivelchair"arebothsub-categoriesof"chair",however,swivelchairshaveaverydifferentlegstructurethanmostsidechairs.
Itbecomesevenmoreambiguoustodenefor"seat",whichhassub-categoriessuchas"stool","couch",and"chair".
However,Figure3.
Examplesofalignedmodelsinthechair,laptop,bench,andairplanesynsets.
theconceptofanuprightorientationstillappliesthroughoutmostlevelsofthetaxonomy.
Followingtheabovediscussion,itisnaturalforustopro-poseahierarchicalalignmentmethod,withasmallamountofhumansupervision.
ThebasicideaistohierarchicallyalignmodelsfollowingthetaxonomyofShapeNetinabottom-upmanner,i.
e.
,westartfromaligningshapesinlow-levelcategoriesandthengraduallyelevatetohigherlevelcategories.
Whenweproceedtothehigherlevel,theself-consistentorientationwithinasubcategoryshouldbemaintained.
Forthealignmentateachlevel,werstuseageometricalgorithmdescribedintheAppendixA.
1,andthenaskhumanexpertstocheckandcorrectpossiblemis-alignments.
Withthisstrategy,weefcientlyobtainconsis-tentorientations.
Inpractice,mostshapesinthesamelow-levelcategoriescanbewellalignedalgorithmically,requir-inglimitedmanualcorrection.
Thoughtheproportionofmanualcorrectionsincreasesforaligninghigher-levelcate-gories,thenumberofcategoriesateachlevelbecomeslog-arithmicallysmaller.
4.
3.
PartsandKeypointsToobtainpartandkeypointannotationswestartfromsomecuratedpartannotationswithineachcategory.
Forparts,thisacquisitioncanbespeededupbyhavingalgo-rithmicallygeneratedsegmentationsandthenhavingusersacceptormodifypartsfromthese.
Weintendtoexperimentwithboth2Dand3Dinterfacesforthistask.
Wethenex-ploitanumberofdifferentalgorithmictechniquestopropa-gatethisinformationtoothernearbyshapes.
Suchmethodscanrelyonrigidalignmentsin3D,featuredescriptoralign-mentsinanappropriatelydenedfeaturespace,orgeneralshapecorrespondences.
Weiteratethispipeline,usingac-tivelearningtoestimatethe3Dmodelsandregionsofthese6modelswherefurtherhumanannotationwouldbemostin-formative,generateanewsetofcrowd-sourcedannotationtasks,algorithmicallypropagatetheirresults,andsoon.
Intheendwehaveusersverifyallproposedpartsandkey-points,asvericationismuchfasterthandirectannotation.
4.
4.
SymmetryEstimationWeprovidebilateralsymmetryplanedetectionsforall3DmodelsinShapeNetCore.
Ourmethodisamodiedversionof[22].
Thebasicideaistousehoughtransformtovoteontheparametersofthesymmetryplane.
Morespecically,wegenerateallcombinationsofpairsofver-ticesfromthemesh.
Eachpaircastsavoteofapossiblesymmetryplaneinthediscretizedspaceofplaneparame-terspartitionedevenly.
Wethenpicktheparameterwiththemostvotesasthesymmetryplanecandidate.
Asanalstep,thiscandidateisveriedtoensurethateveryvertexhasasymmetriccounterpart.
4.
5.
PhysicalPropertyEstimationBeforecomputingphysicalattributeannotations,thedi-mensionsofthemodelsneedtobecorrespondtotherealworld.
Weestimatetheabsolutedimensionsofmodelsus-ingpriorworkinsizeestimation[25],followedbyman-ualverication.
Withthegivenabsolutedimensions,wenowcomputethetotalsolidvolumeofeachmodelthroughlled-invoxelization.
WeusethespacecarvingapproachimplementedbyBinvox[23].
Categoriesofobjectsthatareknowntobecontainer-like(i.
e.
,bottles,microwaves)areannotatedassuchandonlythesurfacevoxelizationvolumeisusedinstead.
Wethenestimatetheproportionalmate-rialcompositionofeachobjectcategoryanduseatableofmaterialdensitiesalongwitheachmodelinstancevolumetocomputearoughtotalweightestimateforthatinstance.
Moredetailsabouttheacquisitionofthesephysicalattributeannotationsareavailableseparately[26].
5.
CurrentStatisticsAtthetimeofthistechnicalreport,ShapeNethasin-dexedroughly3,000,000models.
220,000modelsofthesemodelsareclassiedinto3,135categories(Word-Netsynsets).
BelowweprovidedetailedstatisticsforthecurrentlyannotatedmodelsinShapeNetasawhole,aswellasdetailsoftheavailablepubliclyreleasedsubsetsofShapeNet.
CategoryDistributionFigure4showsthedistributionsofthenumberofshapespersynsetatvarioustaxonomylevelsforthecurrentShapeNetCorecorpus.
Tothebestofourknowledge,ShapeNetisthelargestcleanshapedatasetavailableintermsoftotalnumberofshapes,averagenum-berofshapespercategory,aswellasthenumberofcate-gories.
WeobservethatShapeNetasawholeisstronglybiasedtowardscategoriesofrigidman-madeartifacts,duetothebiasofthesource3Dmodelrepositories.
Thisisincon-trasttocommonimagedatabasestatisticsthatcontainmorenaturalobjectssuchasplantsandanimals[30].
Thisdistri-butionbiasisprobablyduetoacombinationoffactors:1)meshesofnaturalobjectsaremoredifculttodesignusingcommonCADsoftware;2)3Dmodelconsumersaretypi-callymoreinterestedinarticialobjectssuchasthoseob-servedinmodernurbanlifestyles.
Theformerfactorcanbemitigatedinthenearfuturebyusingtherapidlyimprovingdepthsensingand3Dscanningtechnology.
5.
1.
ShapeNetCoreShapeNetCoreisasubsetofthefullShapeNetdatasetwithsingleclean3Dmodelsandmanuallyveriedcategoryandalignmentannotations.
Itcovers55commonobjectcat-egorieswithabout51,300unique3Dmodels.
The12objectcategoriesofPASCAL3D+[35],apopularcomputervision3Dbenchmarkdataset,areallcoveredbyShapeNetCore.
ThecategorydistributionofShapeNetCoreisshowninTa-ble2.
5.
2.
ShapeNetSemShapeNetSemisasmaller,moredenselyannotatedsub-setconsistingof12,000modelsspreadoverabroadersetof270categories.
Inadditiontomanuallyveriedcategorylabelsandconsistentalignments,thesemodelsareanno-tatedwithreal-worlddimensions,estimatesoftheirmate-rialcompositionatthecategorylevel,andestimatesoftheirtotalvolumeandweight.
Thetotalnumbersofmodelsforthetop100categoriesinthissubsetaregiveninTable3.
6.
DiscussionandFutureWorkTheconstructionofShapeNetisacontinuous,ongoingeffort.
HerewehavejustdescribedtheinitialstepswehavetakenindeningShapeNetandpopulatingacoresubsetofmodelannotationsthatwehopewillproveusefultothecommunity.
WeplantogrowShapeNetinfourdistinctdi-rections:AdditionalannotationtypesWewillintroduceseveraladditionaltypesofannotationsthathavestrongconnectionstothesemanticsandfunctionalityofobjects.
Firstly,hierar-chicalpartdecompositionsofobjectswillprovideausefulnergranularitydescriptionofobjectstructurethatcanbeleveragedforpartsegmentationandshapesynthesis.
Sec-ondly,physicalobjectpropertyannotationssuchasmateri-alsandtheirattributeswillallowhigherdelityphysicsandappearancesimulation,addinganotherlayerofunderstand-ingtomethodsinvisionandgraphics.
7Figure4.
PlotsofthedistributionofShapeNetmodelsoverWordNetsynsetsatmultiplelevelsofthetaxonomy(onlythetopfewchildrensynsetsareshownateachlevel).
Thehighestlevel(root)isatthetopandthetaxonomylevelsbecomelowerdownwardsandtotheright.
Notethebiastowardsrigidman-madeartifactsatthetopandthebroadcoverageofmanylowlevelcategoriestowardsthebottom.
IDNameNumIDNameNumIDNameNum04379243table844303593526jar59704225987skateboard15202958343car749702876657bottle49804460130tower13303001627chair677802871439bookshelf46602942699camera11302691156airplane404503642806laptop46002801938basket11304256520sofa317303624134knife42402946921can10804090263rie237304468005train38903938244pillow9603636649lamp231802747177trashbin34303710193mailbox9404530566watercraft193903790512motorbike33703207941dishwasher9302828884bench181603948459pistol30704099429rocket8503691459loudspeaker161803337140lecabinet29802773838bag8302933112cabinet157202818832bed25402843684birdhouse7303211117display109503928116piano23903261776earphone7304401088telephone105204330267stove21803759954microphone6702924116bus93903797390mug21404074963remote6702808440bathtub85702880940bowl18603085013keyboard6503467517guitar79704554684washer16902834778bicycle5903325088faucet74404004475printer16602954340cap5603046257clock65503513137helmet16203991062owerpot60203761084microwaves152Total57386Table2.
StatisticsofShapeNetCoresynsets.
IDcorrespondstoWordNetsynsetoffset,whichisalignedwithImageNet.
8CategoryNumCategoryNumCategoryNumCategoryNumCategoryNumChair696Monitor127WallLamp78Gun54FlagPole38Lamp663RoundTable120SideChair77Nightstand53TvStand38ChestOfDrawers511TrashBin117VideoGameConsole75Mug51Fireplace37Table427DrinkingUtensil112MediaStorage73AccentChair50Rack37Couch413DeskLamp110Painting73ChessBoard49LightSwitch36Computer244Clock101Desktop71Rug49Oven36Dresser234ToyFigure101AccentTable70WallUnit46Airplane35TV233Plant98Camera70Mirror45DresserWithMirror35WallArt222Armoire95Picture69Bowl44Calculator34Bed221QueenBed94Refrigerator68SodaCan44TableClock34Cabinet221Stool92Speaker68VideoGameController44Toilet34FloorLamp201EndTable91Sideboard67WallClock43Cup33Desk189Bottle88Barstool66Printer42Stapler33PottedPlant188DiningTable88Guitar65Sword40PaperBox32FoodItem180Bookcase87MediaPlayer62USBStick40SpaceShip32Laptop173CeilingLamp86Ipod59Chaise39Toy32Vase163Bench84PersonStanding57OfceSideChair39ToiletPaper31TableLamp142Book84Piano56Poster39Knife30OfceChair137CoffeeTable81Curtain55Sink39PictureFrame30CellPhone130Pencil80Candle54Telephone39Recliner30Table3.
Totalnumberofmodelsforthetop100ShapeNetSemcategories(outof270categories).
EachcategoryisalsolinkedtothecorrespondingWordNetsynset,establishingthesamelinkagetoWordNetandImageNetaswithShapeNetCore.
CorrespondencesOneofthemostimportantgoalsofShapeNetistoprovideadensenetworkofcorrespondencesbetween3Dmodelsandtheirparts.
Thiswillbeinvalu-ableforenablingmuchshapeanalysisresearchandhelpingtoimproveandevaluatemethodsformanytraditionaltaskssuchasalignmentandsegmentation.
Additionally,weplantoprovidecorrespondencesbetween3DmodelpartsandimagepatchesinImageNet—alinkthatwillbecriticalforpropagatinginformationbetweenimagespaceand3Dmodels.
RGB-DdataTherapidproliferationofcommodityRGB-Dsensorsisalreadymakingtheprocessofcapturingreal-worldenvironmentsbetterandmoreefcient.
Expand-ingShapeNettoincludeshapesreconstructedfromscannedRGB-Ddataisacriticalgoal.
Weforeseethatovertime,theamountofavailablereconstructedshapedatawillover-shadowtheexistingdesigned3DmodeldataandassuchthisisanaturalgrowthdirectionforShapeNet.
Arelatedef-fortthatwearecurrentlyundertakingistoalign3DmodelstoobjectsobservedinRGB-Dframes.
Thiswillestablishapowerfulconnectionbetweenrealworldobservationsand3Dmodels.
AnnotationcoverageWewillcontinuetoexpandthesetofannotatedmodelstocoverabiggersubsetoftheentiretyofShapeNet.
Wewillexplorecombinationsofalgorithmicpropagationmethodsandcrowd-sourcingforvericationofthealgorithmicresults.
7.
ConclusionWermlybelievethatShapeNetwillprovetobeanim-menselyusefulresourcetoseveralresearchcommunitiesinseveralways:Data-drivenresearchByestablishingShapeNetastherstlarge-scale3Dshapedatasetofitskindwecanhelptomovecomputergraphicsresearchtowardadata-drivendirectionfollowingrecentdevelopmentsinvisionandNLP.
Additionally,wecanhelptoenablelarger-scalequantitativeanalysisofproposedsystemsthatcanclarifythebenetsofparticularmethodologiesagainstabroaderandmorerepre-sentativevarietyof3Dmodeldata.
TrainingresourceByprovidingalarge-scale,richlyan-notateddatasetwecanalsopromoteabroadclassofre-centlyresurgentmachinelearningandneuralnetworkmeth-odsforapplicationsdealingwithgeometricdata.
Muchlikeresearchincomputervisionandnaturallanguageun-derstanding,computationalgeometryandgraphicsstandtobenetimmenselyfromthesedata-drivenlearningap-proaches.
BenchmarkdatasetWehopethatShapeNetwillgrowtobecomeacanonicalbenchmarkdatasetforseveralevalua-tiontasksandchallenges.
Inthisway,wewouldliketoen-gagethebroaderresearchcommunityinhelpingusdeneandgrowShapeNettobeapivotaldatasetwithlong-lastingimpact.
References[1]HelenMBerman,JohnWestbrook,ZukangFeng,GaryGilliland,TNBhat,HelgeWeissig,IlyaNShindyalov,andPhilipEBourne.
Theproteindatabank.
NucleicAcidsRes,28:235–242,2000.
2[2]XiaobaiChen,AlekseyGolovinskiy,andThomasFunkhouser.
Abenchmarkfor3Dmeshsegmentation.
ACMTOG,28(3):73:1–73:12,July2009.
29[3]XiaobaiChen,AbulhairSaparov,BillPang,andThomasFunkhouser.
Schellingpointson3Dsurfacemeshes.
ACMTOG,August2012.
2[4]JiaDeng,WeiDong,RichardSocher,Li-JiaLi,KaiLi,andLiFei-Fei.
ImageNet:Alarge-scalehierarchicalimagedatabase.
InCVPR,2009.
1,2,4[5]BiancaFalcidieno.
Aim@shape.
http://www.
aimatshape.
net/ontologies/shapes/,2005.
2[6]MatthewFisher,DanielRitchie,ManolisSavva,ThomasFunkhouser,andPatHanrahan.
Example-basedsynthesisof3Dobjectarrangements.
ACMTOG,31(6):135,2012.
1[7]Paul-LouisGeorge.
Gamma.
http://www.
rocq.
inria.
fr/gamma/download/download.
php,2007.
2[8]QixingHuang,HaoSu,andLeonidasGuibas.
Fine-grainedsemi-supervisedlabelingoflargeshapecollections.
ACMTOG,32:190:1–190:10,2013.
4,11[9]SubramaniamJayanti,YagnanarayananKalyanaraman,NatrajIyer,andKarthikRamani.
DevelopinganengineeringshapebenchmarkforCADmodels.
Computer-AidedDesign,2006.
3[10]EvangelosKalogerakis,SiddharthaChaudhuri,DaphneKoller,andVladlenKoltun.
Aprobabilisticmodelforcomponent-basedshapesynthesis.
ACMTOG,31:55,2012.
1[11]VladimirKim,YaronLipman,XiaobaiChen,andThomasFunkhouser.
Mobiustransformationsforglobalintrinsicsymmetryanalysis.
SymposiumonGeometryProcessing,July2010.
2[12]VladimirG.
Kim,WilmotLi,NiloyJ.
Mitra,SiddharthaChaudhuri,StephenDiVerdi,andThomasFunkhouser.
Learningpart-basedtemplatesfromlargecollectionsof3Dshapes.
ACMTOG,32(4):70:1–70:12,July2013.
2[13]VladimirG.
Kim,WilmotLi,NiloyJ.
Mitra,StephenDiVerdi,andThomasFunkhouser.
Exploringcollectionsof3Dmodelsusingfuzzycorrespondences.
ACMTOG,31(4):54:1–54:11,July2012.
2,4[14]JonathanKrause,MichaelStark,JiaDeng,andLiFei-Fei.
3Dobjectrepresentationsforne-grainedcategorization.
In4thInternationalIEEEWorkshopon3DRepresentationandRecognition(3dRR-13),Sydney,Australia,2013.
2[15]RomanALaskowski,EGailHutchinson,AlexDMichie,AndrewCWallace,MartinLJones,andJanetMThornton.
PDBsum:Aweb-baseddatabaseofsummariesandanalysesofallPDBstructures.
TrendsBiochem.
Sci.
,22:488–490,1997.
2[16]BoLi,AfzalGodil,MasakiAono,XBai,TakahikoFuruya,LLi,RLopez-Sastre,HenryJohan,RyutarouOhbuchi,CarolinaRedondo-Cabrera,etal.
SHREC'12track:generic3Dshaperetrieval.
In5thEurographicsConferenceon3DObjectRetrieval,2012.
3[17]BoLi,YijuanLu,ChunyuanLi,AfzalGodil,TobiasSchreck,MasakiAono,QiangChen,NihadKarimChowdhury,BinFang,TakahikoFuruya,etal.
SHREC'14track:Largescalecomprehensive3Dshaperetrieval.
InEurographicsWorkshopon3DObjectRetrieval,2014.
2[18]JoergLiebeltandCordeliaSchmid.
Multi-viewobjectclassdetectionwitha3Dgeometricmodel.
InCVPR,pages1688–1695.
IEEE,2010.
2[19]TianqiangLiu,SiddharthaChaudhuri,VladimirG.
Kim,Qi-XingHuang,NiloyJ.
Mitra,andThomasFunkhouser.
Creatingconsistentscenegraphsusingaprobabilisticgrammar.
ACMTOG,December2014.
2[20]MitchellPMarcus,MaryAnnMarcinkiewicz,andBeatriceSantorini.
Buildingalargeannotatedcorpusofenglish:ThePennTreebank.
Computationallinguistics,19(2):313–330,1993.
1[21]GeorgeA.
Miller.
WordNet:alexicaldatabaseforEnglish.
CACM,1995.
1,2,3,4[22]NiloyJMitra,MarkPauly,MichaelWand,andDuyguCeylan.
Symmetryin3Dgeometry:Extractionandapplications.
InComputerGraphicsForum,volume32,pages1–23,2013.
7[23]FakirS.
NooruddinandGregTurk.
Simplicationandrepairofpolygonalmodelsusingvolumetrictechniques.
VisualizationandComputerGraphics,IEEETransactionson,2003.
7[24]BryanCRussellandAntonioTorralba.
Buildingadatabaseof3Dscenesfromuserannotations.
InCVPR,2009.
2[25]ManolisSavva,AngelX.
Chang,GilbertBernstein,ChristopherD.
Manning,andPatHanrahan.
Onbeingtherightscale:Sizinglargecollectionsof3Dmodels.
InSIGGRAPHAsia2014WorkshoponIndoorSceneUnderstanding:WhereGraphicsmeetsVision,2014.
7[26]ManolisSavva,AngelX.
Chang,andPatHanrahan.
Semantically-Enriched3DModelsforCommon-senseKnowledge.
CVPR2015WorkshoponFunctionality,Physics,IntentionalityandCausality,2015.
7[27]PhilipShilane,PatrickMin,MichaelKazhdan,andThomasFunkhouser.
ThePrincetonshapebenchmark.
InShapeModelingApplications.
IEEE,2004.
2,3[28]ShuranSongandJianxiongXiao.
Slidingshapesfor3Dobjectdetectionindepthimages.
InECCV,2014.
1[29]AtsushiTatsuma,HitoshiKoyanagi,andMasakiAono.
Alarge-scaleshapebenchmarkfor3Dobjectretrieval:Toyohashishapebenchmark.
InAsiaPacicSignalandInformationProcessingAssociation,2012.
3[30]AntonioTorralba,BryanCRussell,andJennyYuen.
LabelMe:Onlineimageannotationandapplications.
ProceedingsoftheIEEE,98(8):1467–1484,2010.
7[31]RemcoC.
VeltkampandFBterHarr.
SHREC20073Dshaperetrievalcontest.
Technicalreport,UtrechtUniversityTechnicalReportUU-CS-2007-015,2007.
3[32]DejanVVranic.
3Dmodelretrieval.
UniversityofLeipzig,Germany,PhDthesis,2004.
310[33]RaoulWessel,InaBl¨umel,andReinhardKlein.
A3Dshapebenchmarkforretrievalandautomaticclassicationofarchitecturaldata.
InEurographics2009Workshopon3DObjectRetrieval,pages53–56.
TheEurographicsAssociation,2009.
3[34]ZhirongWu,ShuranSong,AdityaKhosla,FisherYu,LinguangZhang,XiaoouTang,andJianxiongXiao.
3DShapeNets:ADeepRepresentationforVolumetricShapes.
CVPR,2015.
1,2,4[35]YuXiang,RoozbehMottaghi,andSilvioSavarese.
BeyondPASCAL:Abenchmarkfor3Dobjectdetectioninthewild.
InWACV,2014.
2,7[36]JianxiongXiao,AndrewOwens,andAntonioTorralba.
SUN3D:AdatabaseofbigspacesreconstructedusingSfMandobjectlabels.
InICCV,pages1625–1632,2013.
2[37]JuanZhang,KaleemSiddiqi,DiegoMacrini,AliShokoufandeh,andSvenDickinson.
Retrievingarticulated3-Dmodelsusingmedialsurfacesandtheirgraphspectra.
InEnergyminimizationmethodsincomputervisionandpatternrecognition,2005.
3A.
AppendixA.
1.
HierarchicalRigidAlignmentInthefollowing,wedescribeourhierarchicalrigidalignmentalgorithminmoredetail.
Asapre-processingstep,werstsemi-automaticallyaligntheuprightorientationofeachshape.
Fortunately,mostshapesdownloadedfromthewebarebydefaultplacedintheuprightorientations.
Forthosethatarenot,welterthemoutbymanualinspection.
WethenconvertmodelstopointcloudsthroughfurthestpointsamplingandperformPCAonthepointsets.
Finally,weaskapersontopickthevectorofcorrectuprightorientationfromsixcandidatescontainingthePCAaxesandtheirreversedirections.
StartingfromaleafcategoryinShapeNet,wejointlyalignallshapesfollowingpriorwork[8].
Ifaleafcategoryhasmorethan100shapes,wefurtherpartitionitintosmaller,morecoherentclustersbyk-meansclusteringusingpose-invariantglobalfeatures,suchasphase-invariantHoGfeatures[seeappendix].
Herewebrieyreview[8].
Eachshapeisassociatedwitharandomvariable,denotingthetransformationoftheshapefromitsoriginalposetotheconsistentcanonicalpose.
Overthesetofshapes,aMarkovRandomField(MRF)isconstructed,whoseenergyfunctionmeasurestheconsistencyofallpairsofshapesafterapplyingtheirtransformations.
Inpractice,thespaceofrigidtransformationsisdiscretizedintoNbins.
WeperformMAPinferenceovertheMRFtondtheoptimaltransformationforeachshape.
Wethenmanualinspecttheresultsandcorrectoccasionalerrors.
Afterthisstep,werepresenteachleafnodecategorybytheshapeinthecentroidofthefeaturespace.
Then,wegathertherepresentativeshapesforallleafcategoriesofanintermediatecategoryandapply[8]againforjointalignment.
Thishigher-levelalgorithmicalignmentisveriedbyapersonagain.
Theprocedureisappliedalongthetaxonomyhierarchyuntiltherootnodeisreached.
11