AndroidDynamicLinker-MarshmallowWANGZhenhua,i@jackwish.
netAbstractDynamiclinker,linkssharedlibrariestogethertobeabletorun,hasbeenafundamentalmechanisminmodernoperatingsystemandrichsoftwareecosystemoverthepastdecades.
Dynamiclinkerisalwayshighlyplatform-customizedsinceit'scoupledwithbinaryformatofasystem.
ThisarticleintroducesthebasicconceptionofdynamiclinkerandtakesAndroid(Marshmallow)dynamiclinkerasexampletodiscusstheimplementation.
IntroductionDynamicLinkOpenanyprogramminglanguagebook,"HelloWorld"isusuallytherstcodeexample.
TheCexamplebelowwhichwearefamiliarwithisrelatedwithdynamiclink.
ThelifecycleofthiscodeisasFigure1(memoryrelatedpartisinbluewhilesymbolrelatedisinpurple).
#includeintmain(){printf("HelloWorld!
\n");return0;}Asweknow,functionsneedtobedeclaredanddenedbeforeuse.
Forthe"HelloWorld"example,printf()isdeclaredinstdio.
handtheimplementationisinsharedlibrarylibc.
so.
Theprocedureoflocatingthedeclarationiscompiling(pre-processmoreprecisely)whilelocatingtheimplementationislinking.
Therearetwocategoriesoflinking-staticlinkinganddynamiclinking-ofwhichthedierenceisthetimethelinkingprocedureisperformed,asFigure1demonstrated.
Staticlinkingisperformedbycompilertool-chain,gccexample.
c-staticonLinuxisanexample.
Astaticlinkedbinaryrunswithouttheneedtorelocate1symbol-printfisasitsself-denedfunction.
Whenthebinaryhello.
elfisexecuted,operatingsystemonlyneedstoloaditintomemory.
Dynamiclinkingisperformedatruntimebydynamiclinker.
Atcompiletime,compilertool-chaingeneratesdynamiclinkedbinaryhello.
elfwhichcontainstheinformationthatitdependsonlibc.
sofortheimplementationofprintf.
Atruntime,dynamiclinkerloadshello.
elf,readsthedependentdata,loadslibc.
sointomemory,andllstheaddressofprintfintohello.
elf.
Inthisway,themainfunctioncancorrectlycallsprintf.
Figure1:printfExampleofLinkWiththeabilityofdynamiclinking,developerscouldcreateandsharelibraries.
Libraryauthorcouldupdateinternalimplementationwithoutneedtoinformusers,whilelibraryusersdon'tneedtore-deploytheirapplicationsunlesslibraryinterfaceshavechange.
ThisistheinfrastructureofAPI(ApplicationProgramInterface)!
LibraryDependencyDiverseprogramsrunningonmoderncomputersystemareconstructedbylibrariestodiminishdevelopmenteortandmemoryandstorageconsumption.
2Dynamiclinkerisresponsibleforlocatinglibrariesfromstorage,loadingthemintomemoryandlinkingthereferenceofthem.
Developerscreateprogramorlibrarybasedondierentlibraries.
OnetypicaldependencyoflibraryisasFigure2(libcutils.
soofAndroid,libdl.
soisignored).
Alibraryauthorknowswhichlibrariesisdependeduponbyhislibrary,andrecordsthedependencyinthelibrary(DT_NEEDEDtableforELFformatlibrary).
Atruntime,dynamiclinkerre-buildsthedependencyofanexecutableorsharedlibraryandlinksthebinaryagainstitsdependency.
Figure2:Dependencyoflibcutils.
soThedependencyismostlyaDAG(DirectedAcyclicGraph).
Fordynamicsystemsupportingrecursivedependent,thedependencycouldbeaDCG(DirectedCyclicGraph).
Nomatterwhatkindthegraphis,dynamiclinkercansimplyvisitallnodesinthegraph,locate,loadandlinkthem.
Inadependencygraph,thereisoneandonlyonenodewhichhasnoentryedge,calledroot.
There-buildingprocedureofadependencyistraversingthegraphinDFS(DepthFirstSearch)orBFS(BreadthFirstSearch)orderstartingwithroot.
Figure2isaBFSexample.
Inthisarticle,ourdiscussionisbasedontheBFSdependency.
DocumentStructureWetakethedynamiclinkerofAndroidastheimplementationexample.
Itispartofbionic,thestandardClibrarydevelopedbyGoogleforitsAndroidoperatingsystem.
AndroidisbasedonLinuxofwhichthesharedlibraryformatisELF.
DynamiclinkerprovideslinkingserviceforAndroidsystemandapplicationsdeployedwithJNIcapability.
WerstlyintroducethebasicmechanismofdynamiclinkerofAndroid.
Thentalkabouthowspecialfeaturesisimplemented.
Afterthat,thebootstrapofdynamiclinkerisdiscussed.
And,atlast,comingtothetrickypartoflibrarydependency.
3DynamicLinkingMechanismInAndroid,dynamiclinkerisinvokedwhenSystem.
loadLibrary()isexecutedinJavaordlopen()isexecutedinnativecode.
ForJavacode,Dalvik/Android-runtimecallsintodynamiclinkerjustlikedlopen()eventually.
StartingwithLollipopMR1(OurdiscussionisbasedonMarshmallow),Androiddynamiclinkingistwo-phase:libraryloadingandlibraryrelocation.
AsFigure3shows,thelefthalfisloadingwhiletherighthalfislinking.
Duringthelibraryloadingprocedure,dynamiclinkerre-buildsthelibraryde-pendency,loadsalllibrariesofitintomemory.
Thelibraryrelocationprocedurelinksthedependency.
WetalkabouttheimportantdatastructureofAndroiddynamiclinkerrstly.
Figure3:WorkowofAndroidDynamicLinkerDataStructurePersistentDataStructureDynamiclinkerofAndroidhastwopersistentdatastructureduringthelifetimeofanapplication/program-LSPath(LibrarySearchPaths)andALList(Already-loadedLibraryList).
4LSPatharethedirectorieswherelibrariesarestored.
Dynamiclinkertraversesthesepathstohuntforalibrary.
Thesepathsarecriticaltothelibrarylocatingandaresequencedinpriority.
ALListisalistofsoinfowhichisusedtomaintainmetadataofloadedlibraries(ELFdataandmemorylayoutforexample).
DynamiclinkerobtainsdatafromALListacrossdierentlibraryloadingandlinking.
ALListgrowsandshortenswhenlibraryisloadingandunloadingrespectively.
PersistentdatastructuresareatthetopsideofFigure3.
"Storage"inFigure3meansLSPathsomehow.
TemporaryDataStructureNaturally,manytemporarydatastructuresareusedduringloadinglibrary.
Amongthem,themostimportanttwoareload_tasksandlocal_group.
Bothofthempresentthelibrarydependency.
load_tasksisaqueuecontainingthelibrariestobeloaded-asubsetlibrariesoflibrarydependencywhichhaveNOTbeenloadedintomemoryyet.
load_tasksdequeueswhenlinkerbeginstosearchalibraryandenqueuesthedependentlibrariesjustparsed(fromDT_NEEDEDtableofanELFformatlibrary).
Afterallthetreehasbeenloaded(load_tasksisemptyatthistime),local_groupisconstructedandusedforrelocation.
local_groupisaqueueofsoinfoandrepresentslibrarydependencyinBFSorder.
(Anothersimilardatastructurecalledglobal_groupwillbediscussedin"SpecialFeatures"section.
)TemporarydatastructuresarelistedatthebottomsideofFigure3.
LibraryLoadingProcedureAtthebeginning,thelibraryrequestedbyoperatingsystem-root-isaddedtoload_tasks,asFigure3.
Duringtheprocedureoflibraryloading,dynamiclinkercontinuallyloadsalllibrariesinload_tasksandupdatesitifnecessary,asthelefthalfpartofFigure3.
Alllibrariesinthelibrarydependencywillbeloadedwhenthisprocedureisnished.
LibraryLocatingDynamiclinkerextractsonename/pathfromload_tasks,andopensdirectlyifit'sabsolutepathortraversesLSPathtohuntforthelibraryotherwise.
Whenthelibraryislocatedandopened,itcouldbeasystemlibraryoraapplibrary.
systemlibraryarelibrariesloadedfromsystemlibrarypaths5like/system/lib;applibraryareloadedfromapplicationlibrarypathslike/data/data/com.
example.
app/lib.
BeforeapplicationisforkedfromZygote,dynamiclinkeronlysearchesforlibraryundersystemlibrarypaths.
Afterapplicationisforkedandlibrarypathsareset,dynamiclinkersearchesunderapplicationpathsrstly,thensystemlibrarypaths.
LibraryLoadingAfterlibraryisopenedfromstorage,andbeforeloadsthelibraryintomemory,linkerwantstoverifywhetherthelejustopenedisavalidsharedlibrary.
ItperformscheckbasedonELFdata:magicnumber,32/64bit,little/bigendian,machinetypeandsoon.
Ifanythingwrong,thislibraryandthelibrarydependencywillbeunloaded.
Ifvalidationpasses,dynamiclinkerreadsthelibraryheaderandloadsallloadablesegmentsintomemory.
ItcalculatestheneededmemorysizeofthelibrarybycheckingPT_LOADtablesofprogramheader.
Thememoryallocationissimplyviammap.
(InandbeforeJellyBean,thelibrarymemoryismanagedbyabuddymemoryallocationsystemsystem)LibraryPre-link"Pre-link"intendstobuildonemoreleveloflibrarydependencybyreadingthedependency(dynamicDT_NEEDEDsection)ofalibrary.
AlllibrarynamesrecordedinDT_NEEDEDtableareaddedtoload_tasksandtobeloaded.
It'seasilytoseethatsamelibrary(name)maybeaddedtoload_tasksmanytimeswhenloadsalibrary.
DynamiclinkertraversesALListtocheckifthelibraryhasalreadybeloadedintomemorybynameandi-node,beforeandafteropenthatlibrary.
Iffound,dynamiclinkerdropsthatload_tasksnodeandfetchesnext.
SotherearenoduplicateloadedlibrariesinALList.
TheoccurrencetimeofreadingdependencyofalibraryhaschangedacrossthedevelopmentofAndroid.
BeforeLollipop-MR1,librarylinkingisDFSwhichloadsandlinksthelibrarydependencyrecursively.
BeginningwithLollipop-MR1,librarylinkingchangestoBFS.
Thischangemakesthelibrarylinkingatwo-stageprocedure,alllibrariesinalibrarydependencyhasbeenloadedintomemorybeforeanyofthemhasbeenrelocated.
LibraryRelocationProcedureAfterlibraryloadingprocedure,thedependentrelationshipoflibariesarerecordedinsoinfo.
Dynamiclinkerreadssoinfobeginningwithroottobuild6local_group.
Relocationisperformedonlocal_group.
Mainloopofrelocationdequeuesalibraryfromlocal_groupandrelocatesit.
local_groupisBFSbuilt,sotherelocationisBFStoo.
Whenresolvingasymbolofalibrary,dynamiclinkerwalkstheRelocationSection,atableofallthingsneededtoberelocated(DT_RELorDT_RELAofELF),ofasharedlibrary.
Foreachrelocationentry,linkerreadsthesymbolindexandconvertsittosymbolname.
Withthename,linkersearchesthedenitionofitinthedependencytree-beginwiththelibraryitself,thenglobal_group(see"ExtensionofDynamicLinker")andlocal_group.
Whensearchesasymboldenitioninalibrary,dynamiclinkerchecksthesymboltable(DT_SYMTABofELF)ofit.
Thereisaccelerationmethodsforthetablelookup,DT_HASHofELFisahashlistwhichcontainsallthe"exported"or"imported"symbolofalibrary.
Thelibraryrelocationprocedureisintuitive.
Whenit'sdone,dynamiclinkercallsalllibraryconstructorsinthedependency.
Afterconstructorsarenished,thelibraryisloaded,dynamiclinkerreturnsahandlerofthislibrarytouser.
ExtensionofDynamicLinkerDynamiclinkinghassomeextensionstosupportvariousscenarios,andAndroidhasextendeddynamiclinkingfunctionsforspecicpurpose.
GenericDynamicLinkGlobalLibraryWhenalibraryisdeclaredasa"globallibrary",loadedwiththeagRTLD_GLOBAL,thedenitionofthesymbolofthelibraryhasthehighestpriorityforalllibrariesloadedafterit.
Androiddynamiclinkerbuildstheglobal_groupeverytimeatthebeginningwhenloadalibrary.
Whenrelocatingasymbol,theglobal_groupisrstlylookedup-"globallibrary"canoverlaysthesymboldenitionofthelibrariestobeloadedafterwards.
PreloadLibraryWhenabinaryexecutedwithagLD_PRELOAD.
Theselibrarieswillbeloadedbeforethebinaryisreallyexecuted.
Androiddynamiclinkerpreloadstheselibrarieswhenitinitializes.
TheselibrarieswillcarrytheagRTLD_GLOBAL.
Afterthat,"preloadlibrary"isjustlike"globallibrary".
LD_PRELOADonlytakeseectforpurenativeprograms.
7AndroidExtendedDynamicLinkAndroidsystemextendsdynamiclinkingtoimproveexperiencewhenloadslibrariesfrombothJavaandnative.
TheAPIisandroid_dlopen_ext().
TillM,featuresofthisextensionisasbelow,mostofthemareprettyeasytounderstand.
Justcopyfromthesourcele.
.
.
.
Thedatastructureofthisextensionislikethis:typedefstruct{uint64_tflags;void*reserved_addr;size_treserved_size;intrelro_fd;intlibrary_fd;off64_tlibrary_fd_offset;}android_dlextinfo;LibraryMemoryRelatedANDROID_DLEXT_RESERVED_ADDRESS:Whenset,thereserved_addrandreserved_sizeeldsmustpointtoanalready-reservedregionofaddressspacewhichwillbeusedtoloadthelibraryifitts.
Ifthereservedregionisnotlargeenough,theloadwillfail.
ANDROID_DLEXT_RESERVED_ADDRESS_HINT:AsDLEXT_RESERVED_ADDRESS,butifthereservedregionisnotlargeenough,thelinkerwillchooseanavailableaddressinstead.
LibraryOpeningRelatedANDROID_DLEXT_USE_LIBRARY_FD:Instructdlopentouselibrary_fdinsteadofopeninglebyname.
Thelenameparameterisstillusedtoidentifythelibrary.
ANDROID_DLEXT_USE_LIBRARY_FD_OFFSET:Ifopeningalibraryusinglibrary_fdreaditstartingatlibrary_fd_offset.
ThisagisonlyvalidwhenANDROID_DLEXT_USE_LIBRARY_FDisset.
ANDROID_DLEXT_FORCE_LOAD:Whenset,donotcheckifthelibraryhasalreadybeenloadedbylestat(2)s.
ThisagallowsforcedloadingofthelibraryinthecasewhenforsomereasonmultipleELFlessharethesamelename(becausethealready-loadedlibraryhasbeenremovedandoverwritten,forexample).
Notethatifthelibraryhasthesamedt_sonameasanoldoneandsomeotherlibraryhasthesonameinDT_NEEDEDlist,therstonewillbeusedtoresolveanydependencies.
8LibraryRelocationRelatedANDROID_DLEXT_WRITE_RELRO:Whenset,writetheGNURELROsectionofthemappedlibrarytorelro_fdafterrelocationhasbeenperformed,toallowittobereusedbyanotherprocessloadingthesamelibraryatthesameaddress.
ThisimpliesANDROID_DLEXT_USE_RELRO.
ANDROID_DLEXT_USE_RELRO:Whenset,comparetheGNURELROsectionofthemappedlibrarytorelro_fdafterrelocationhasbeenperformed,andreplaceanyrelocatedpagesthatareidenticalwithaversionmappedfromthele.
BootstrapofDynamicLinkerDynamiclinkerisdesignedto"link"allrelocatablebinaries,andmustmakeitselflookslikelibdl.
sotorelocatables-thelibdl.
sobinaryisjustadummylibrarywhichmakesldofcompilertool-chainhappy.
Dynamiclinkerisstaticallylinkedatcompiletimeanddoesn'tdependonanyotherresourcesexceptsystemcall.
Theself-relocatingandfakinglibdl.
soisBootstrap.
ThebootstrapofAndroiddynamiclinkerisdividedintotwosteps:1.
Initialize:hardcodedtorelocatelinkeritself.
2.
Post-initialize:prepare"linkerruntime"forlibraryloading.
InitializeDuringthisstage,allexecutedcodeisstaticallyrelocated.
Noexternvariable,externfunction,orGOTaccess.
Calledfrombegin.
SandwillcallPost-initializefunctionsafterwards.
Primaryoperationsarerelocatinglinkeritselfandcreatingthedummylibdl.
sosoinfo.
Relocatinglinkeritselfisasadstory,everythingishand-obtained.
Afterthesoinfoiswell-setuped(memoryrelatedmostly),therealrelocationisconducted.
Thenconstructorsoflinkerarecalledtoinitializelinker'sglobalvariables.
Creatingdummylibdl.
sosoinfoismainlysetandupdatethereferenceofthesoinfotohard-codedarray,symboltableforexample.
Thissoinfonodeoflibdl.
soisalwaystherstnodeofALList.
Withtheseworkdone,linkerisrelocated.
9Post-initializeAfterself-relocated,dynamiclinkerrelocatessomain-themainprocess-Zygote.
BeforerelocatesZygote,linkerasksforruntimevariablefromsystemlikeLD_LIBRARY_PATHandLD_PRELOAD.
Andthen,itrelocatesZygote.
WithZygoterelocated,loadalllibrariesdeclaredinLD_PRELOAD.
Witheverythingdone,linkernishesBootstrapandjumpstoZygote.
LibraryDependencyAsdiscussedinthebeginning,onetaskofdynamiclinkeristore-buildlibrarydependency.
There-buildingprocedureissensitivetoruntimeenvironmentinsomecornerscenario.
Figure4:TrickyLibraryDepdendencyGenerationofAndroidConsiderthattherearetwosetsoflibraries-set1andset2.
Somelibrariesinthesetwosetssharesamenamebuthavedierentdenitions.
Atthebeginning,onlyset1canbeloaded,andthenbothset1andset2canbeloaded.
Thetrickis,inanydependency,thelibraryloadedinphase1couldonlydependsonlibraryinset1,asFigure4.
Thisisbecausewheneverlibraryinset1isneeded,dynamiclinkersimplyreusesthesoinfoofit.
LD_PRELOADintraditionalLinuxandlibrariesloadedbeforeZygoteforksinAndroidaresuchscenarios.
Thisisneformostdevelopers,butcouldimpactsomeemulationsystem.
10SummaryDynamiclinkerre-buildsthedependencyofexecutables,locates,loadsandlinksit.
It'sfundamentalinfrastructureofmodernoperatingsystemandsensitivetorunningenvironment.
Dynamiclinkingisusuallyhighplatformcustomizedandrequiresbootstrap.
AndroidNincludesnamespacechangestopreventloadingofnon-publicAPIs.
ThisfeatureheavilyimpactstheecosystemofAndroid.
Intheory,namespaceenables"virtualization"indynamiclinking.
Thedynamiclinkingwediscussedinthisdocumentin"processinternal",whilenamespacecanbuildseveralvirtualspace-namespace-fordynamiclinkinginoneprocess,makingthedynamiclink"namespaceinternal".
Wewillrefertonamespaceinthefuture.
11
快云科技怎么样?快云科技是一家成立于2020年的新起国内主机商,资质齐全 持有IDC ICP ISP等正规商家。我们秉承着服务于客户服务于大众的理念运营,机器线路优价格低。目前已注册用户达到5000+!主营产品有:香港弹性云服务器,美国vps和日本vps,香港物理机,国内高防物理机以及美国日本高防物理机!产品特色:全配置均20M带宽,架构采用KVM虚拟化技术,全盘SSD硬盘,RAID10阵列, 国...
OneTechCloud发布了本月促销信息,全场VPS主机月付9折,季付8折,优惠后香港VPS月付25.2元起,美国CN2 GIA线路高防VPS月付31.5元起。这是一家2019年成立的国人主机商,提供VPS主机和独立服务器租用,产品数据中心包括美国洛杉矶和中国香港,Cera的机器,VPS基于KVM架构,采用SSD硬盘,其中美国洛杉矶回程CN2 GIA,可选高防。下面列出部分套餐配置信息。美国CN...
由于行业需求和自媒体的倾向问题,对于我们个人站长建站的方向还是有一些需要改变的。传统的个人网站建站内容方向可能会因为自媒体的分流导致个人网站很多行业不再成为流量的主导。于是我们很多个人网站都在想办法进行重新更换行业,包括前几天也有和网友在考虑是不是换个其他行业做做。这不有重新注册域名重新更换。鉴于快速上手的考虑还是采用香港服务器,这不腾讯云和阿里云早已不是新账户,考虑到新注册UCLOUD账户还算比...
android5.1为你推荐
设置xpcss3圆角用CSS3怎么实现圆角边框?ipadwifiipad插卡版和wifi版有什么区别,价格差的多么?127.0.0.1DNS老是被修改为127.0.0.1,这是为什么?用itunes备份如何使用itunes完整备份iPhone资料360chrome360Chrome 世界之窗极速浏览器 ChromePlus联通iphone4联通iphone4好用吗迅雷快鸟迅雷快鸟这种强盗软件不违规吗?micromediaMacromedia Fireworks 8是什么东西,怎么用?googleadsense10分钟申请Google Adsense是一种怎样的体验
vps代理 工信部域名备案系统 万网域名证书查询 荷兰服务器 私人服务器 服务器怎么绑定域名 网站cdn加速 共享主机 网通服务器托管 爱奇艺vip免费领取 789电视剧 银盘服务是什么 华为云盘 双线asp空间 shuang12 阿里云邮箱登陆地址 镇江高防 摩尔庄园注册 免费个人网页 小夜博客 更多