Eclipsesyntaxhighlighter

syntaxhighlighter  时间:2021-05-18  阅读:()
EmbeddingLanguagesWithoutBreakingToolsLukasRenggli,TudorGrba,OscarNierstraszSoftwareCompositionGroup,UniversityofBern,Switzerlandhttp://scg.
unibe.
ch/Abstract.
Domain-speciclanguages(DSLs)areincreasinglyusedasembeddedlanguageswithingeneral-purposehostlanguages.
DSLsprovideacompact,dedicatedsyntaxforspecifyingpartsofanapplicationrelatedtospecializeddomains.
Unfortunately,suchlanguageextensionstypicallydonotintegratewellwiththedevelopmenttoolsofthehostlanguage.
Editors,compilersanddebuggersareeitherunawareoftheextensions,ormustbeadaptedatanon-trivialcost.
WepresentanovelapproachtoembedDSLsintoanexistinghostlanguagebyleveragingtheunderlyingrepresentationofthehostlanguageusedbythesetools.
HelvetiaisanextensiblesystemthatinterceptsthecompilationpipelineoftheSmalltalkhostlanguagetoseamlesslyintegratelanguageextensions.
Wevalidateourapproachbycasestudiesthatdemonstratethreefundamentallydierentwaystoextendoradaptthehostlanguagesyntaxandsemantics.
1IntroductionGeneralpurposelanguages,bybeing"goodenough"tocodesoftwareforarbitrarydomains,arenecessarilysuboptimalformanyspecializeddomains.
Theymaybeoverlyverbose,confusingorjustplainawkwardtouse.
ManyDSLshavebeendevelopedtoaddresstheneedsofthesespecializeddomains,butmostoftheselanguagestypicallydonotintegratewellwiththehostlanguageandtools,makingitclumsytodevelopanddebugprogramswrittenintheselanguages.
Furthercomplicatingmatters,multipleDSLsmaybeactivewithinthesameapplicationsoftware.
TheseDSLsmayeitherbegloballyavailable,ortheymaybecontext-dependent,beingactiveonlywithinselectedpackagesorclasses.
DSLscomeinmanyavors.
Atoneextremewehaveso-calledinternalDSLswhichsimplymakecreativeuseofAPIsandofthehostsyntax.
SuchDSLsaresometimesreferredtoasuentinterfaces[1].
Theyprovideaseamlessintegrationinthehostlanguage,andassuchtheycanbenetfromthetoolsprovidedbythedevelopmentenvironment(e.
g.
,codeeditor,debugger)ofthehostlanguage.
However,theexpressivenessofinternalDSLsisconnedbythehostsyntax.
AttheotherendofthespectrumwendexternalDSLs[2].
Thesearetypicallydevelopedasapreprocessingsteporthroughanextensiblecompiler,thusprovidingfreedomforexpressingdiversesyntaxandsemantics.
However,whiledoingsotheybreakthetoolsofthehostdevelopmentenvironment.
InProceedingsofthe24thEuropeanConferenceonObject-OrientedProgramming(ECOOP2010),LNCS6183,pp.
380–404,Springer-Verlag,2010.
InbetweenthesetwoextremeswendembeddedDSLs,whichextendahostlanguagewithnewsyntaxandsemantics.
LanguageworkbenchessupportthedevelopmentofembeddedDSLsbyintroducingacommonrepresentationforalllanguagesandbyintegratingmultiplelanguagesintoacommontoolset.
Merniketal.
[3]pointoutthatsuchanembeddedapproach[4]leadstobetterreuseofexistinghostlanguagefeaturesandtools,andsignicantlyreducesdevelopmentandtrainingcosts.
Inpracticehowever,languageworkbenchesdonotleveragetheexistingtoolsbutprovidetheirownenvironment.
Oftentheyintroduceanon-standardlanguagerepresentationandthusposecompatibilityproblemswithexistingcode.
Table1.
TaxonomyforPidgin,CreoleandArgotembeddedlanguages.
SyntaxVocabularySemanticsDescriptionPidginApidginisasimpliedformofthehostlanguage.
Itintro-ducesanewvocabularyandnewsemanticstothecode.
CreoleAcreolechangesthesyntaxofthehostlanguageanddenesnewsemantics.
ArgotAnargotswitchesthesemanticsoftheexistinglanguage,withoutaectingitssyntax.
Weproposetoaddresstheseshortcomingsbydevelopinganapproachthatenablesmultiple,context-dependentembeddedDSLsthatleverageexistingtools.
Embeddedlanguagesextendahostlanguagebyadaptingtheexistingsyntaxandsemantics,orbyintroducingnewsyntax.
InTable1weidentifyessentiallythreedierentwaysthiscanbedone:1Pidgin.
Apidginbendsthesyntaxofthehostlanguagetoextenditssemantics[5].
Thiskindofembeddedlanguagereusesalimitedpartofthehostsyntaxandcombinesitwithanewvocabulary.
Creole.
Acreoleintroducesacompletelynewsyntaxbydeningitsowngram-marandacustomtransformationtothehostlanguagethatdenesthesemantics.
Argot.
Anargotusestheexistinghostlanguagesyntax,butchangesitsseman-tics.
Anargotreinterpretsthesemanticsofvalidhostlanguagecode,whereas1Inthedomainofnaturallanguage,a"pidgin"is"agrammaticallysimpliedformofalanguage,usedforcommunicationbetweenpeoplenotsharingacommonlanguage";a"creole"is"amothertongueformedfromthecontactoftwolanguagesthroughanearlierpidginstage";an"argot"isa"jargonorslangofaparticulargrouporclass"[NewOxfordAmericanDictionary].
pidgincodeisonlysyntacticallycorrecthostcode—ithasmeaningonlyforthepidgin.
Anembeddedlanguagemusteitherintroducenewsyntaxtothehostlanguagefortheconceptsitintroduces(acreole),oritmustadoptthehostsyntaxasis.
Ifthehostsyntaxisreused,itmusteitherbeoverloaded,reinterpretingthesyntaxinanovelway(pidgin),oritmustalterthesemanticsofthehost(argot).
Afullygeneralapproachtointegratingnewembeddedlanguagesintoanexistinghostlanguageandenvironmentmustthereforesupportallthreeclassesofembeddedlanguage.
Ourapproach.
InthispaperwepresentalanguageworkbenchcalledHelvetiafordeningembeddedlanguagesandforintegratingthemintothehostlanguage.
Helvetiaaccommodatesnewlanguagesthroughextensionpointsoftheexistingcompilerandtools.
Alllanguagesaretransformedandrepresentedintermsoftheabstractsyntaxtree(AST)ofthehostlanguage.
Thetransformationsareexpressedasrulesandtheycanbescopedtovariouscontexts.
Furthermore,theserulescanbeactiveatthesametime,allowingustoembeddierentlanguagesintoacommonhostlanguage,andtointegratethemintothehostenvironmentanditstools.
Sinceourapproachbuildsontopoftheexistinginfrastructureofthehostlanguage,existingtools,suchaseditorsanddebuggers,continuetoworkwithminimaladaptation.
Helvetiaprovidesthenecessarylow-levelinfrastructureforLanguageBoxes[6],anadaptivelanguagemodelforne-grainedlanguagechangesandlanguagecomposition.
Outline.
Section2discussestherelatedworkandenumeratestheshortcomingsoftheseapproaches.
InSection3wepresentconcretelanguageextensionsthatexemplifythethreetypesoflanguagesoursolutionsupports.
Section4illustrateshowthedierenttypesofembeddedlanguagescanbespeciedwithHelvetia.
InSection5weexplainhowHelvetialeveragesthehosttoolchaintoseamlesslyembednewlanguages.
Section6evaluatesourapproachincomparisonwithrelatedwork,andinSection7wesummarizethepaperanddiscussfuturework.
2RelatedWorkWereviewthestateoftheartinsystemsforauthoringembeddedlanguagestoestablishtheopenchallengesfacinganewapproach.
Inparticular,weconsiderhowthesesystemssupportthedevelopmentofpidgin,creoleandargotembed-dedlanguages,andweassesshowwellthesesystemsfacilitateintegrationofembeddedlanguageswiththeirrespectivehostlanguageandhostlanguagetools.
Furthermore,wecomparetheexistingapproacheswithrespecttothefollowingpropertiesoflanguageembedding:MultipleContext-DependentLanguages.
Switchingbetweendierentlan-guagesshouldbepossibleatarbitrarypointsandnotenforcetheuseofspecialsyntacticmarkers.
Itshouldbepracticabletomixandmatchdierentlanguageextensionsandthehostlanguage.
Languagechangesshouldbescopableatane-grainedlevel,tomakeitpossibletouseseveralotherwiseconictinglanguageextensionsinthesamecompilationunit.
HomogeneousToolIntegration.
Auniformtoolsetisimportanttosoftwareengineers.
Toeasethedevelopmentanduseofembeddedlanguagesalldevelopmentactivitiesshouldhappeninthesamefamiliarprogrammingenvironmentofthehostlanguage.
Nospecialcodebrowser,editors,debuggersorsourcecontrolshouldbenecessary;allexistingtoolsshouldcontinuetoworktransparentlywithdierentlanguages.
Tofacilitatedebuggingaprecisebi-directionalconnectionbetweentheoriginalsource,thevarioustransformationstagesandthenalexecutablecodeshouldbemaintained.
HomogeneousLanguageIntegration.
Themeta-languageusedtospecifynewlanguagefeaturesshouldbethesameasthehostlanguage[7].
Homo-geneouslanguageintegrationiscentralforseveralreasons:Ahomogeneoussystemisarequirementtotransparentlypassvaluesbetweenthedierentmeta-levelsandtousereectiontoreasonaboutstaticanddynamicstructureoftheapplication[8].
Table2providesanoverviewoftherelatedworksplitintofourcategories.
Foreachconsideredsystemweindicatethehostlanguage,thecapabilityofdeningpidgins,creolesandargots,andthesupportfortheaforementionedcharacteristics.
Thefollowingsectionsoerdetailsforeachindividualsystem.
2.
1ExtensibleCompilersExtensibleCompilersarebestdescribedasopentoolboxesthatprovideentrypointsintothetoolchaintoextendandchangethehostlanguage.
TheJavaAnnotationProcessingTool(APT)enablesacompiletime,read-onlyviewoftheJavaprogramstructure.
ASTscanbetransformedonlyusingaprivateAPI,whichisnotsupportedandmaybesubjecttochangeordeletion.
Assuch,argotsandpidginscanbeimplemented,butacreolewouldrequireanadditionalpreprocessingstep.
InasimilarwaypeopleareusingtheweavingmechanismofAOPtoachievesemanticchangesforpidginsandargots.
ableJ[9],Dryad[10],JastAddJ[11],andPolyglot[12]areextensibleJavacompilerframeworksandprovidethenecessaryinfrastructuretobuildargots,pidginsandcreoles.
Languageextensionsarecomposableandmodular,andaretransformedintoJavaforexecution.
Mostextensiblecompilersdenethesyntaxchangesusingaexternallanguagedenition.
TheDryadcompilerusesbytecodeasitscentralrepresentation,andthusitisnothomogeneousaswell.
NoneofthesystemsoerstightIDEintegration,andthetransformedcodecannotbedebuggedatthesourcelevel.
Xoc[13]isanextensibleCcompiler.
Xocusesasource-to-sourcetranslatorthatreadstheinput,analyzesandtransformsit,toeventuallygeneratestandardCcode.
Thesystemprovidesnoreectivefacilitiesandnotoolintegration,neitheratcompiletimenoratruntime.
Table2.
Comparisonofdierentsystemsforembeddedlanguageauthoring.
Weindicatecustomhostlanguageswithparentheses.
TypeSystemHostLanguagePidginCreoleArgotMultipleLanguagesHomogeneousToolsHomogeneousLanguageHelvetiaSmalltalkExtensibleCompilersJavaAnnotationProcessingJava··ableJJava··DryadJava··JastAddJJava··PolyglotJava·XocC··MetaProgrammingSystemsCola(various)···Converge(Python)···MetaOCamlOCaml··SchemeScheme·LanguageWorkbenchesJetBrainsMPS(Java)··IntentionalSoftware(C#openArchitectureWareJavaJavaDevelopmentToolsJavaIDEMetatoolingPlatformJavaWholePlatformJavaKatahdin(C#)···CetevaXMF(Java)···LanguageTransformationSystemsKheperaC···MontiCoreJavaMetaBorg··2.
2Meta-ProgrammingSystemsMeta-ProgrammingSystemsareprogramminglanguagesthatcomewithmeta-programmingfacilitiestargetedatcodegeneration.
Cola[14]implementsanopenobjectmodelforexperimentationwithdierentprogrammingparadigms.
ColaisbootstrappedinitselfusingaSmalltalk-likelanguagecalledPepsi.
JoltisaLisp-likelanguagethatservesasacommonabstractsyntaxandexecutablerepresentationforotherlanguages.
OMeta[15]isanobject-orientedpatternmatcherbasedonParsingExpressionGrammarsthatisusedtotransformnewlanguagestoJolt.
Evenifalllanguagesarebuiltontopofthesameinfrastructure,theauthorsdonotprovidemechanismstoeasilyembedthemintoeachother.
Furthermorethereisnocommontoolsupportforeditinganddebuggingthedierentlanguages.
Converge[16]isadynamicprogramminglanguageresemblingPython.
Aspecialblockconstruct$>isusedtoembedlanguagesintothesourcecode.
Thecreationofargotsandpidgins,i.
e.
,themodicationorextensionofexistinglanguages,isnotsupported.
Meta-programmingispossibleatcompiletimeonly.
ThereisnoIDEintegration.
MetaOCaml[17]usesmulti-stageprogrammingtogenerateandtransformcodeatruntime.
SimilarquotingmechanismsareavailableinprogramminglanguageslikeScheme,LisporTemplateHaskell,butthesemechanismsalonedonotallownewsyntaxtobeintroduced.
Bothsystemhavepowerfulmacroprogramingconstructs,makingitpossibletotweakthedefaulttowardspidginsorargots.
Typicallythesekindofsystemsareusedtogetherwithtraditionaltexteditorsandthusdonotallowaneasyadaptationtolanguagechanges.
Debuggersareavailable.
2.
3LanguageWorkbenchesLanguageWorkbenchesarecharacterizedbyaspecializedIDEwithawell-denedworkowtospecifyandusedierentlanguages.
Languagedesignersarerequiredtofollowclearlydenedstepstodescribesyntax,semanticsandeditorbehaviorofanewlanguage.
TheMetaProgrammingSystem(MPS)byJetBrains[18]andIntentionalSoftware[19]bothprovideaprogrammingenvironmenttodenenewlanguagesandtochangeexistingones.
Neithersystemusestextrepresentationforsourcecode,butinsteadtheyprovideagraphicalcelleditorthatmapsvalidprogramsdirectlytoanunderlyingabstractcoderepresentation.
MPSdenesnewlan-guagesusingdierentconceptsforstructure(semanticmodel),editor(parser),constraints,behavior,typesystems,dataowandcodegeneratorsforJava.
MPS1.
1doesnotcomewithasourceleveldebugger;debugginganderrorreportinghappensatthelevelofthegeneratedJavacode.
SimilarlyIntentionalSoftwarerequireslanguagedeveloperstodeneedit,display,andtransformationconcernsforlanguageextensions.
Asnoproductpreviewsandnodetaileddocumentationisavailable,theexactpropertiesofthissystemarenotclear.
Forexampleitisunknownhowcloselythesystemintegrateswiththehostlanguageandifitispossibletotransparentlystepwithadebuggerthroughdierentlanguages.
openArchitectureWareandWholePlatform[20]arelanguageworkbenchesthataretightlyintegratedintotheEclipseplatform.
InbothcasestemplatingsystemsareusedtogenerateexecutableJavacode.
openArchitectureWareprovidesastrongintegrationwiththeEclipsemeta-modelingfacilities,suchastheEclipseModelingFramework(EMF)andtheGraphicalModelingFramework(GMF).
Theybothprovidebasicsupportforeditorintegrationsuchassyntaxhighlightingandcodecompletionoftextuallanguages.
However,bothsystemslackdebuggingsupportandtheabilitytochangethesemanticsoftheirhostlanguage.
TheJavaDevelopmentTools(JDT)providethebasictoolstobuildEclipseplugins.
TheIDEMetatoolingPlatform(IMP)[21]isanextensibleIDEarchitec-turefortheEclipseplatform.
ContrarytoallothertoolslistedinthissectionJDTandIMPdonotprovidefunctionalityforlanguagedesignandembeddingthemselves.
TheonlypurposeofIMPistocloselyintegrateexistinglanguagesintotheEclipseIDEusingaservicearchitecture.
Atthetimebeingthereisnosupportforinteractionwithlanguageruntimesanddebuggers.
Katahdin[22]isaprogramminglanguageimplementedontopofC#.
KatahdinisadynamicallytypedlanguagethatsyntacticallyresemblesC#.
Newconstructssuchasexpressionsorstatementsaredenedbysubclassingexistingparse-treenodesthatcanthenbeaddedtothehostlanguageatruntime.
Languagescanbeenabledonaper-lebasis,orcanbeintegratedintothehostlanguagebyextendingitwithaspecickeywordsuchaslanguage{.
.
.
}andconnectingthetwogrammar-trees.
Thesemanticsofuser-denedparse-treenodesisspeciedbyoverridingcertainmethodsintherespectivenode-classes.
Codeisinterpretedbytraversingtheparse-treenodesandcallingmethodsdeningthesemantics.
Thereisasimpledebuggeravailablevisualizingtheinternalparse-treestructureoftheinterpreter.
ThedebuggerisnotabletoworkatthelevelofKatahdinprograms.
TheExtensibleProgrammingLanguage(XMF)byCetevaisaspecicationofa"superlanguage"[23].
Asuperlanguageischaracterizedthroughusability(interactive,dynamic,reection,interfaces),expressiveness(high-level,dynamictyping,garbage-collection),andextensibility(aspects,reexive,extensiblesyntax).
XMFiswritteninitself,andallowsonetoeasilydenenewlanguages.
Aspecial@languageconstructisusedtoswitchbetweendierentlanguages.
AlthoughaJavainterfaceisavailable,XMFusesitsownproprietaryvirtualmachinewritteninJava.
XMFdoesnotprovideanIDEintegration.
2.
4LanguageTransformationsLanguagetransformationsystemsdenelanguagesthroughthetransformationandcompositionoflanguagemodels.
Khepera[24]isapreprocessorthattransformssource-to-sourceandpretty-printsthegeneratedcodetoCbeforecompilingwithatraditionalcompiler.
Itparsesinputintoanabstractsyntaxtreeandperformingcomplextree-basedanalysisandmanipulation.
Alltransformationspreservetheknowledgeoftheoriginofeachnode.
Thus,intheoryKheperamakesitpossibletodevelopdebuggersfornewlanguages.
However,thesystemprovidesnoready-to-useIDEordebugger.
Furthermore,itdoesnotsupportmultiplelanguagestobeusedsimultaneously.
MontiCore[25]providesaframeworkforlanguageinheritanceandlanguageembedding.
MontiCorehasitsownsyntaxtodenegrammarsandtheirmappingtoJavatypes.
TheparseriscreatedusingtheANTLR[26]parsergenerator.
Theabstractsyntaxtreeisautomaticallyderivedfromthegrammar.
Languageinheritanceallowsonetosubclassexistinggrammarstomodifyandextend.
Languageembeddingisachievedbymanuallyintroducingasuperordinateparserforeverypairoflanguagesthatareusedtogether.
ANTLRhasbeenpatchedtosupportswappingbetweenthegrammarsonthey.
Visitorsareusedtoaddnewbehaviorforproductions,togeneratecodeandtobuildeditorsforEclipse.
ThereisnoIDEintegrationorsupportfordebugging.
MetaBorg[27]isamethodforembeddingDLSsandextendingexistinglanguages.
MetaBorgisbasedontheStratego/XT[28]toolkit,alanguagein-dependentprogramtransformationengine,hencethereisnointegrationintodevelopmentenvironmentsandmodelleveldebuggersofthehostlanguage.
3HelvetiaExempliedInthissectionweintroducethreecasesoflanguageextensions.
ThersttwocasestakethesameunderlyingAPIofagraphicalengineandtransformitintoapidgin(Section3.
1)andacreole(Section3.
2).
Wethendescribetheuseofanargotforintroducingatransactionalmemorymechanismwithoutchangingthesyntaxofthehostlanguage(Section3.
3).
TheprototypeofHelvetia2andtheexamplespresentedinthispaperareimplementedinPharo3,anopen-sourceSmalltalk[29]dialect.
ReadersunfamiliarwiththesyntaxofSmalltalkmightwanttoreadthecodeexamplesinthefollowingsectionsaloudandinterpretthemasnormalsentences.
Aninvocationtoamethodnamedmethod:with:,usingtwoargumentslookslike:receivermethod:arg1with:arg2.
Thesemicolonseparatesmessagesthataresenttothesamereceiver.
Forexample,receivermethod1:arg1;method2:arg2sendsthemessagesmethod1:andmethod2:toreceiver.
Othersyntacticele-mentsofSmalltalkare:thedottoseparatestatements:statement1.
statement2;squarebracketstodenotecodeblocksoranonymousfunctions:[statements];singlequotestodelimitstrings:'astring';anddoublequotesdelimitcomments:"comment".
Thecaret^returnstheresultofthefollowingexpression.
3.
1APidgin:MondrianMondrian[30]isagraphbasedvisualizationframeworkthatprovidesadeclarativeSmalltalkAPIforuserstospecifynewvisualizationsandcomposeexistingones.
OneofthefeaturesofMondrianisanAPItocomposecustomshapesoutofbasicones,calledFormsBuilder.
TheFormsBuilderisinspiredbyCSS3andusesagridtoalignprimitivegraphicalelementssuchastextlabelsandboxes.
Forexample,thecodebelowinListing1createsaUMLpackageshapeasdepictedinFigure1.
Thepackageshapeisbuiltfroma2*2grid.
Therstcolumnandrowaretoldtogrowtoenclosetheirchildren.
Thesecondcolumnandrowaretoldtofilltheremainingspace.
Incell(1,1)weplaceaborderedLabelShape.
Incell(1,2)weplaceaborderedRectangleShapethatspanstwohorizontalcells.
2Theimplementationalongwithitssourcecodeandexamplescanbedownloadedfromhttp://scg.
unibe.
ch/research/helvetia.
3http://www.
pharo-project.
orgFig.
1.
AUMLpackageshapeinMondrian.
aBuilderrowgrow.
"definesrowsizing"aBuilderrowfill.
aBuildercolumngrow.
"definecolumnsizing"aBuildercolumnfill.
aBuilderx:1y:1add:(LabelShapenew"definethecells"text:[:each|eachname];borderColor:#black;borderWidth:1;yourself).
aBuilderx:1y:2w:2h:1add:(RectangleShapenewborderColor:#black;borderWidth:1;width:200;height:100;yourself)Listing1.
TraditionalMondrianFormsBuilderAPI.
MondrianprovidesaninternalDSLthatoersahigh-levelinterfaceforcomposingvisualizations.
Whileitmakesthecompositioneasy,thereisstillaconsiderableamountofsyntacticnoisethatmakesthescripthardtoread.
WeincrementallybendthesyntaxofthehostlanguagetowardsamoresuitableDSL,rstbycreatingapidgin,andthenbycreatingacreole.
Ournalgoalistobeabletodenethevisualizationsusingasimplesyntaxresemblingcascadingstyle-sheets(CSS),whichoersacompactnotationtoprogrammersanddesignerstodeclarativelyspecifylayoutanddesignofwebsites.
IfwehavealookatthecodeinListing1weseethatthenoiseiscausedbycertainsemanticelementsthatarerequiredtomakethisexamplerunasSmalltalkcodeconformingtotheoriginalMondrianAPI.
Wediscoverthreethingsthatarerepetitiveandthatcouldbesimplied:1.
ThevariableaBuilderisreferencedineveryruleasanentrypointtoconstructandcongurethedierentpartsoftheforms.
2.
Thespecicationofthecellsandtheircontentisrepetitiveandratherhardtoread.
3.
Theinstantiationofdierentshapesiscumbersomeasinthiscasethehostlanguagesyntaxisratherverbose.
Thefollowingcodeaddressestheseissues:row=grow.
row=fill.
column=grow.
column=fill.
(1,1)=labeltext:[:each|eachname];borderColor:#black;borderWidth:1.
(1,2)-(2,1)=rectangleborderColor:#black;borderWidth:1;width:200;height:100.
Listing2.
Pidgin:Eliminatingsyntacticnoise.
WhiletheabovecodeissyntacticallyvalidandisparsedbythestandardSmalltalkparser,itisnotsemanticallyvalid.
Forexamplenumbersdonotimplementa,method,andcolumnisanunknownvariable.
Inourimplementation(describedinSection4.
1),theabovepidginexampleistransformedtransparentlyintothecodefromListing1.
Thiskindoftransfor-mationsimpliestheamountandcomplexityofsourcecodesignicantly.
ThetransformationisspeciedattheASTlevelusingtwotransformationrulesthatareappliedbythecompilerafterparsing.
3.
2ACreole:MondrianThepidginshowsanimprovementovertheoriginalSmalltalkcode,butourgoalistoobtainanevenmoreconciseCSS-likelanguageasinthelistingbelow:shape{cols:#grow,#fill;rows:#grow,#fill;}label{position:1,1;text:[:each|eachname];borderColor:#black;borderWidth:1;}rectangle{position:1,2;colspan:2;borderColor:#black;borderWidth:1;width:200;height:100;}Listing3.
Creole:ACSS-likesyntax.
ThecodeabovedoesnotfollowSmalltalksyntax.
Atthispoint,theassumptionofapidginrelyingonthehostsyntaxstartstogetinourway.
Thesolutionistoallowthedenitionofanewparserthathandlesthecreolesyntax.
Wetypicallyalsowanttointegratethenewlanguageconstructswiththehostlanguageorwithotherlanguageconstructs.
Inourexample,thecodetext:[:each|eachname]providessuchacaseinwhichweparameterizetheshapespecicationwithaSmalltalkexpression.
AsshowninSection4.
2,Helvetiaoersamechanismforwritingacustomparserthatcanalsoincludeproductionsexternaltothelanguageathand.
Likethiswecanaccommodateanysyntax.
3.
3AnArgot:TransactionalMemoryInourpreviouswork[31]wehavepresentedasolutionforintroducingsoftwaretransactionalmemory(STM)[32,33]atthelanguagelevelofdynamicprogram-minglanguageswithoutrequiringchangestothevirtualmachine.
Weachievedthisbypatchingthecompiler.
Asaresultwewereabletomakeallapplications,librariesandsystemcodetransaction-aware.
Howeversinceourchangesatthecompilerlevelwereratheradhoc,welosttheabilitytousethedebuggerwithinatransaction.
IntroducingSTMintoanexistinglanguageprovidesaconcreteusecaseforchangingtheexecutionsemanticswithoutchangingthesyntaxofthelanguage.
Apieceoflibrarycodethatisusedaspartoftransactionalcodeshouldcontinuetoworkwithoutrequiringanyadaptation.
Unlikeapidgin,whichbendsthehostsyntaxinwaysthatbreakthesemantics,anargotmoresubtlyreinterpretsthesemanticsofotherwisevalidcode.
Intheexamplebelowtheglobalcountervalueisincrementedby1.
Thecodedoesnotensuremutualexclusion,thusitmighthappenthatsomeoftheupdatesarelostwhenmultiplethreadsrunthismethodconcurrently.
incrementBy:anIntegervalue:=value+anIntegerWhenrunningtheabovemethodfromwithinatransaction,thechangeisdeferredtotheendofthetransaction,insteadofincrementingthevariableimmediately.
Thisallowsthesystemtocheckforconictsandrevertthechangesifnecessary.
Thus,evenifthesourcecodelooksexactlythesame,itsbehav-iorchanges.
WedescribethetransformationsappliedtotransactionalcodeinSection4.
3.
4SpecifyingEmbeddedLanguageswithHelvetiaInthissectionwepresentthespecicationsofthethreeexamplesgivenintheprevioussection.
4.
1SpecifyingtheMondrianPidginThesyntaxoftheMondrianpidgincanbeparsedbythetraditionalparseroftheSmalltalkhostlanguage.
However,weneedtoapplyseveraltransformationstogetthesemanticsright.
WedeneasetoftransformationrulesthatareappliedbyHelvetiaafterparsingthecodefromListing2:1MondrianPidginclass>>rowColumnTransformation23^TreeRulenew4expression:'row=@expr';5expression:'column=@expr';6action:[:ast|7astswapWith:(aBuilder8,(astreceiver)9,(astat:'@expr'))]1011MondrianPidginclass>>cellTransformation1213^TreeRulenew14expression:'(@x,@y)=@expr';15expression:'(@x,@y)-(@w,@h)=@expr';16action:[:ast|17astswapWith:(aBuilder18x:,(astat:'@x')19y:,(astat:'@y')20w:,(astat:'@w'ifAbsent:[1])21h:,(astat:'@h'ifAbsent:[1])22add:(,(Shapesat:(contextat:'var')name)23new,(astat:'@expr')))]Thetransformationrulesaresplitintotwomethods.
Eachofthesemethodsistaggedwiththemethodannotation(lines2and12),sothatthecompilerknowsthatithastoapplythesetransformationsbeforeperformingsemanticanalysis.
Eachruleconsistsoftwomatchexpressions(lines4–5and14–15)tondparticularparse-treenodes.
ThisfunctionalityispartoftheRefactoringEngine[34]andisprovidedbythehostenvironment.
InourcontextthesepatternsmatchthespecicconstructsweintroducedinListing2.
Theactionblocks(lines6–9and16–23)performatransformationonthematchedASTnode.
Forexampletherstactionblocktransformsexpressionsoftheformrow=growintoaBuilderrowgrow.
ItdoessobyusingasyntacticextensionofSmalltalkwithpartialevaluation[35].
Everythingthatfollowsthequasiquotemeta-characterisdelayedinexecutionandrepresentstheASToftheenclosedexpressionatruntime.
Similarlyeverythingthatfollowstheunquotemeta-character,isagainexecutedwhenperformingthecodeandisusedtocombinesmallerdelayedvalues(e.
g.
,frommatchedASTnodes)tolargerones.
Athirdoperatortocompile,evaluateandspliceintheresultatcompile-timeisavailabletoo,butnotusedintheexamplesofthispaper.
ThetwomentionedmethodsareallthatisneededtoimplementtheMondrianpidgin.
TheswapWith:methodcallreplacesthematchedASTnodewiththenewcode.
SinceallASTnodescarryinformationabouttheiroriginalsourceorigin,adebuggerisabletostepthroughandproperlyhighlighttherecomposedcodefragments.
Newlygeneratedcodeismarkedashidden,sothattheuserofthepidgindoesnotseeitinthedebugger.
4.
2SpecifyingtheMondrianCreoleIncontrasttoapidgin,acreolerequiresacustomparserandHelvetiaoersthepossibilitytodeneone.
Forexample,forthecreolewepresentedinListing3wedenethefollowinggrammarrulesdenedasindividualmethodsoftheclassCSSParser:CSSParser>>rules={rule}CSSParser>>rule=selector"{"declarations"}"CSSParser>>selector=#identifierCSSParser>>declarations=declaration{";"declaration}CSSParser>>declaration=#keywordMessageThisgrammarlooksverysimilartoExtendedBackus-NaurForm(EBNF)[36].
Infact,itisaDSLforparsergeneratorsimplementedinHelvetia.
AsanextensiontoEBNFweallowproductionstoreferencegrammarrulesofotherlanguages.
Thenameofexternalgrammarrulesareprexedwithahashcharacter#.
Forexample,theCSSselectorissimplyaSmalltalkidentier,andthedeclarationofapropertyisakeywordmessage(aSmalltalkmethodnamewitharguments,butwithoutreceiver)ofthehostlanguage.
NextwecreateanewsubclassofCSSParsercalledCSSTranslator,toreusetheabstractgrammardenitionandtoaugmentitwithproductionstotransformtheparsetreenodestothehostlanguageAST[37].
AgainweusequasiquotingtobuildtheASTofthehostlanguage.
TwoofCSSTranslator'sparsetreetransformationslooklikeinthefollowinglisting.
Theothergrammarproductionsaresimilarlydened.
1CSSTranslator>>rules2^superrules==>[:ast|(buildOn:aBuilder,ast)]34CSSTranslator>>rule5^superrule==>[:ast|selftransform:(astat:'selector')declarations:(astat:'declarations')]Thisassignssemanticactionstotheproductionsdenedinthesuperclass.
TheargumentastisacollectionofparsenodesbuiltbythegrammarproductionsFig.
2.
TheCSSParserHierarchy.
ofthesuperclass.
Lines3–4usequasiquotingtodenethemethodheaderandtoembedtheASTnodesoftherulesintoitsbody.
Lines7–9callthehelpermethodbuild:declarations:withtheselectortokenandacollectionofdeclarationmessagestobuildaSmalltalkAST.
Totellthesystemtouseourcustomparserinsteadofthedefaultone,weuseamethodannotationontheclasseswherewewanttousethecustomsyntax.
ThecodeinListing3isparsed,transformedandeventuallycompiledtobytecodeidenticaltothemethodswemanuallywroteinListing1andListing2.
MondrianCreoleclass>>cssParser^CSSTranslatorOnesmallproblematthispointisthatthesyntaxhighlighterinthecodeeditorisbroken.
Asbefore,wecreateanewsubclassofCSSParsernamedCSSHighlighterthatunderlinestheselectorsanddispatchestostandardSmalltalkhighlightingforthedenitions.
AgainthisisachievedbyoverridingtheappropriatemethodsofCSSParser.
Forexample,theselectormethodisdenedinCSSHighlighterlikethis:CSSHighlighter>>selector^superselector==>[:ast|ast->TextEmphasisunderlined]UsingaannotationwedeclarethehandlertoberesponsibleforsyntaxhighlightingoftheCSScode:MondrianCreoleclass>>cssHighlighter^CSSHighlighterAddingapidginorcreoleoveranexistingframeworkcansimplifyitsuseandreduceaconsiderableamountofsyntacticnoise.
WithouttouchingtheoriginalframeworkweareabletoprovidedierentlanguageskinsthatmightbeanappealingalternativetotheinternalDSLthatwasusedbefore.
Thechangesshowninthissectionareallthatisneededtoalsoaectthedebugger.
Figure4showsaliveresultofsteppingthroughtheexecutionofthescriptbuildingtheUMLpackageshape.
4.
3SpecifyingtheTransactionalMemoryArgotInanutshell,oursoftwaretransactionalmemoryimplementationworksasfollows.
Wecompileeverymethodinthesystemtwice,onceforthetransactionalandonceforthenon-transactionalcontext.
Onthetransactionalcodeweapplytwotransformations:(1)allstateaccessisreiedtobedispatchedthroughthetransactionalcontext,and(2)methodnamesandmethodsendsareprexedwith__atomic__.
Furthermoreweusemethodannotationstodisableorcustomizethesetransformationsincertainplaces,suchaswhenprimitivecodeiscalledorinthetransactionalinfrastructureitself.
Atransactionisstartedbyassigningatransactionmanagertoathread-localvariableandbycallingan__atomic__method.
Attheendofatransactionthecachedchangesareatomicallycheckedforconictsandappliedtotheinvolvedobjects.
Thetransactionboundariesarehandledatthelanguagelevelusingthereectivefacilitiesofthehostlanguage.
Fordetailsontheimplementationofthesemanticmodelpleaserefertoourpreviouswork[31].
ThroughtheuseofHelvetiatheargotspecicationbecomessimple.
Thecompletesetoftransformationrulesarepresentedbelow:1Objectclass>>transformAtomic23^ConditionRulenew4if:[:context|contextisTransactional]5then:(TreeRulenew6expression:'@receiver@msg:@args'do:[:ast|7astswapWith:(,(astat:'@receiver')8,('__atomic__',(astat:'@msg:'))9,(astat:'@args'))];10expression:'var:=@expr'do:[:ast|11astswapWith:(self12atomicInstVarAt:,(astbindingindex)13put:,(astat:'@expr'))];14expression:'var'do:[:ast|15astswapWith:(self16atomicInstVarAt:,(astbindingindex))])Thecodeusesthemethodannotation(line2),totellthecom-pilerthattherulesareexpectedtorunafterthesymbolshavebeenresolved(attributed).
Line4makessurethatthetransformationisonlyperformedwhencompilingcodeforthetransactionalcontext.
Lines5–16implementtheactualtransformations,exempliedinthetablebelow:6–9TransformMessageSendsselfprintString→self__atomic__printString10–13TransformInstanceVariableWritevalue:='Atomic'→selfatomicInstVarAt:2put:'Atomic'14–16TransformInstanceVariableReadvalue→selfatomicInstVarAt:2Allmessagesendsareprependedwith__atomic__toensurethattheexecutionstaysintheatomiccontext.
Allstateaccesses,suchasinstancevariablereadsandwrites,aretransformedtomessagesendsanddispatchedthroughthetransactionmanager.
Thisallowsustodelaymodicationstoobjects,sothatthechangesareonlyvisiblewithinthecurrenttransaction.
Thenumber2intheexamplesaboverefertotheindexofthenamedinstancevariablevalue.
ThisslotindexisretrievedfromtheattributedAST.
Totriggerthecompilationofatransactionalandanon-transactionversionofeverymethodwehookintotheparserusingtheannotation.
Wecopythecompilationcontextandspawnanewcompilationpathforthetransactioncontext.
(DetailsfollowinSection5.
)Objectclass>>compileTransactional:aContextaContextisTransactionalifFalse:[aContextcopybeTransactional;perform].
^nilTheimplementationoftransactionalmemoryusingHelvetiahasnumerousadvantagesoverourpreviousimplementation:Thecodebaseisnotintegratedintothecompilerinanadhocmanneranymore,butthroughclearlydenedHelvetiaextensionpoints.
Inourpreviousimplementation,wedirectlypatchedinseveralplacestheexistingcompiler.
FollowingthesupportofHelvetia,thetransformationsarespeciedatasingleplacethatisexternaltothecompiler.
Asaconsequencethecodeisnicelymodularizedandmoreconcise.
Withoutadditionalworktheuseofthedebuggerbecomesviable,becausethetransformationspreservelocationintegrity.
Inourpreviousexampleitwasnotpossibletotransparentlystepthroughtransactionalcode,asthetoolswoulddisplaythegeneratedcode.
ThenewimplementationtakesadvantageofHelvetiaandsinglesteppingthroughtransactionalcodelooksexactlythesameastheregularcode.
5LeveragingtheHostToolchainwithHelvetiaHelvetiamanagestointegratemultipleembeddedlanguageswithexistingtoolsbyleveragingandinterceptingtheexistingtoolchainandtheunderlyingrepresentationofthehostlanguage.
Helvetiaprovideshookstointerceptparsing,ASTtransformationandsemanticanalysisofthestandardcompilationtoolchain.
5.
1HomogeneousLanguageIntegrationFig.
3.
Thecodecompilationpipelineshowingmultipleinterceptionpaths.
InSmalltalktheunitofcompilationisthemethod.
Wheneveramethodissaved,itautomaticallytriggerstheSmalltalkcompilerwithinthesameenvi-ronment.
Thesourcecodeandcompiledmethodsaswellasthecompilerareallavailableatruntime.
AsseeninFigure3wehaveenhancedthestandardcompilationpipelinetobeabletointerceptthedatapassedfromonecompilationsteptotheother.
Weareabletoperformsource-to-sourcetransformationsortobypasstheregularSmalltalkparseraltogether.
FurthermoreweareabletoperformASTtransformationseitherbefore,insteadof,oraftersemanticanalysis.
Therulestointerceptthistransformationpipelinearedenedusingannotatedmethods.
ThesemethodsconstituteconventionalSmalltalkcodethatiscalledatcompiletime[38].
Theinterceptionrulesallowusnotonlytomodifydatainthepipelinebutalsotobypassconventionalcomponents.
–Arulemarkedwithallowsonetointercepttheparsingofthesourcecode.
Theresultofaparserrulecanbeeitheranewsourcestring(incaseofasource-to-sourcetransformation)oraSmalltalkAST(inwhichtheoriginalSmalltalkparserisskipped).
–ArulemarkedwithisperformedontheASTafterparsingandbeforesemanticanalysis.
Itallowsdeveloperstoapplyarbitrarytransforma-tionsontheAST.
Furthermore,itispossibletochangethedefaultsemanticanalysisandinsteadperformacustomone.
–Arulemarkedwithisperformedaftersymbolresolutionandbeforebytecodegeneration.
ThismakesitpossibletoperformtransformationsontheattributedASTaswell.
Compilationerrorsarehandledbythestandardtoolchain.
Sincealldatapassedfromonesteptothenextcarriesinformationonitsoriginalsourcelocation,theerrorlocationisdeterminedautomaticallyanditisrevealedtothetheuserthroughthetraditionalmeansofthecompiler.
Forexample,whenavariableisundeclared,itsoccurrenceishighlightedandtheuserisaskedtocorrecttheproblem.
5.
2HomogeneousToolIntegrationTocontrolsyntaxhighlighting,weusetheruledatabasetochangethedefaulthighlighting.
ThetraditionalSmalltalksyntaxhighlighterisonlyappliedtonormalSmalltalkmethods.
Assoonasthereisacustomparserinvolved,theaectedpartofthesourcecoderemainsblackunlessacustomhighlighterisprovided.
Theannotationisusedtodeneahighlightingrule.
Figure4showstheresultofthecustomhighlighterinthesourcepaneofthedebugger.
Helvetiaprovidessimilarextensionpointsforcode-completionandcontextualmenus.
Fig.
4.
TraditionalSmalltalkdebuggerwithlanguagespecicsyntaxhighlightingsteppingthroughamixtureofSmalltalkandthecreoledenedinSection3.
2.
Thelackofdedicatedtoolstondandxbugsinanewlanguageisoneofthemajordrawbackswhendesigningandusingembeddedlanguages.
Sinceourapproachusesthecodeabstractionofthehostlanguage,thestandarddebuggingtoolscontinuetowork.
Onecansetbreakpointsasinaconventionalmethods.
Steppingthroughcodewritteninamixtureoflanguagesposesnoproblemeither.
TheASTofadebuggedmethodcarriesinformationaboutthesourcerangeintheoriginalcode.
Generatedcodeeitherreusesthesourcerangesoftheparentnode,orhasnosourcerangeandisthereforeinvisibleinthedebugger.
WiththisinformationfromtheoriginalASTthedebuggerisabletoaccuratelyhighlightthecurrentexecutionpointandsteptothenextstatement,withouthavingtoknowanythingaboutthestructureofthesourcestring.
Helvetiacurrentlydoesnotchangethewaythedebuggerpresentsinforma-tion,e.
g.
,thestackframesandvariablesaredisplayedatthelevelofhostlanguage.
However,weenvisiontheadditionofnewrulestoenablethecustomizationofthedebugger'suser-interface.
Figure4showshowwestepthroughthecodeofourcreoleexample.
Thetoppartshowstheexecutionstack,withthetopmethodbeingLabelShape>>text:whichsetstheMondriantextoftheshape.
Themaineditorshowsthecreolecodecorrespondingtothecreationoftheshape.
Thisexampleshowshowthedebuggeraccommodatesboththecreolecodeandthecalledframeworkcode.
5.
3MultipleContext-DependentLanguagesHelvetiausesannotatedmethodsclass-side(static)methodstodenearuledatabasethatisqueriedbythecompilerandothertools.
Therulesaectinstance-codeofthecorrespondingclassanditssubclasses.
Todeneasystem-widerule,ithastobeinstalledwithinanextensionmethodforObject,therootoftheclasshierarchy.
Thefollowingprimitiveruletypesarecurrentlysupported:–TheConditionRulebehaveslikeacasestatement.
Anorderedlistofconditionsischeckedandtherstmatchingactionisexecuted.
Ifnomatchisfoundadefaultactionisexecutedasanalternative.
BoththeconditionandthematchactionareimplementedusingthehostlanguageandcancheckforarbitraryconditionsusingthereectiveAPI.
Thisruletypeistypicallyusedtoscopetheeectofrulestospecicpartsofthesystem.
–MatchRuleandRangeRuleuseregularexpressionstomatchsourcecode.
Thisisusefultocheckforspecicstringsinthecodewhennoparsetreeisavailableyet.
Forexample,regularexpressionsaresometimesusedtoprovidecustomsyntaxhighlightingwithinstringliteralsofthehostlanguage.
Inmanycasesmatchingtheparsetreeissimpler.
Thisruletypeisonlysupportedforrules.
–TheTreeRuleisaparsetreematcher.
UnlikestringmatchingthesepatternsworkontheASTandmakeitpossibletoecientlyndalloccurrencesofparticularnodecombinations.
Again,actioncodecanbesuppliedthatisexecutedwhenamatchisfound.
Thisruletypeisonlysupportedforandrules.
AswehaveseenintransactionalexampleinSection4.
3,rulescanbearbitrarilynested.
InsteadofattachingSmalltalkcodetoanaction,anotherrulecanbeusedthatissubsequentlyappliedinthecontextoftheparentmatch.
Furthermore,aswehaveseeninthecreoleexampleinSection4.
2,itispossibletosupplyacustomruleobjectsuchasacustomparserorsyntaxhighlighter.
6EvaluationIntherelatedwork,supportforpidgin,creoleandargotembeddedlanguagesisvariable.
Inthecategoryofextensiblecompilersusuallyalltypesaresupported.
Metaprogrammingsystemseitherdonotprovideamodelofthehostlanguagethatcanbemodied(Converge)ordonotprovidethepossibilitytochangethesyntax(MetaOCaml,Scheme).
Languageworkbenchesaredesignedtoimplementcreoles,thatistobuildnewlanguageelementsandcombinethemwithotherlanguages.
6.
1HostLanguageChoiceMetaprogrammingsystemsandlanguageworkbenchesprovidelargetoolsetstodenenewlanguages,howeverinmanycases(Converge,MPS,IntentionalSoftware,Katahdin,XMF)theyusederivatives.
Insomecases(Katahdin,XMF)theyimplementanewruntimelayerthatmakesitdiculttoreuseexistingcodeandlibraries.
Webelievethatitisbenecialfortheadaptationofalanguageauthoringsystemtobuildintoanexistinghostlanguageandleverageasmanyfeaturesaspossible.
HelvetiareusestheSmalltalkcoderepresentation,thecompletecompilertoolchainandtheexistingIDEtoprovidealightweightlanguageinte-gration.
Helvetiacodeshowsperformancepenaltyasitusesthesameruntimeinfrastructureasthehostlanguage.
Inourpreviouswork[39]wehaveevaluatedseveralhostlanguagechoicesforasystemlikeHelvetia.
Smalltalkhasproventobeagoodpracticalchoice,thoughnotarequirement:–InSmalltalkthecompilerispartofthedevelopmentenvironmentandcanbechangedonthey.
ForHelvetia,wedidsobycarefullyintroducinginter-ceptionpointsbeforeandafterthedierentcompilationsteps(Section5.
1).
Rulesaredenedusingannotatedmethodsthatareevaluatedatcompilationtime.
–RulesthatthatworkonASTnodesneedtopreservethesourcemappingwitheverytransformation.
InourcaseweusetherefactoringengineofthehostlanguagetoquerytheASTnodes.
Meta-programmingfacilities,suchasthequasiquotingfacilities[40]inSchemeorOMetaCaml'sstagingconstructs[41],greatlysimplifythegenerationofcode.
–Aswiththecompiler,editorsarerequiredtosupportextensionpointsforcustomhighlighting,codecompletion,errorreporting,etc.
InSmalltalktheeditorsareimplementedwithinthehostlanguageandcanbecustomizedbyextendingorchangingtheexistingcode.
Inourcasewedidsobyconsultingtheruledatabaseforeverymethodbeingedited.
Itisessentialthattheenvironmenthavefullaccesstotherules.
–Tosupportdebuggingofdierentlanguages,thedebuggermustbeabletouseanarbitrarysourcemappingbetweenthecustomlanguageandtheexecutablerepresentationofthehostenvironment.
Inourcasewemaintainthismappingfromthesourcestringthroughalltransformationstagesdowntothebytecode.
Thedebuggerisfedwithacustomfunctionthatmapssourcerangestobytecoderanges.
Sincethedebuggerreusesthenormalcodeeditoroftheprogrammingenvironment,syntaxhighlightingworkswithoutadditionalsupport.
–Sincealllanguagesusethesameunderlyingrepresentation,therearenodicultiestoshareapplicationstatebetweendierentpartsofthesystem.
Forexample,anewlanguageconstructcanaccesstemporaryvariables,instancevariablesorglobals.
Whenamethodisevaluated,itdoesnotmatterinwhatlanguageithasbeenimplemented.
Blockclosurescanbepassedaround,nomatterwhatorigintheyhaveandfromwhatlanguagecontexttheyareevaluated.
WeseethefollowingmainchallengestoimplementasystemlikeHelvetiainanexistingenvironmentlikeEclipse:(1)replacethedefaulteditor,compileranddebuggerwithacustomizedones,(2)connectthesetoacentralruledatabase(thisrequirescommunicationbetweendierentJavaVMs),and(3)establishane-grainedmappingbetweenbytecodeandsourcecode(bydefaultJavaonlysupportsalinebasedmapping).
6.
2MultipleEmbeddedLanguagesTheintegrationofnewembeddedlanguagesintoeachotherandintothehostlanguageissolvedindierentways.
Mostexistingsystemsdothisonaper-lebasis.
Somesystemsrequirespecialtokenstoswitchbetweenlanguages(Converge,XMF,MontiCore).
InKatahdinthistokenisfreelydenablebychangingthehostlanguage.
InHelvetiathescopeofeachlanguageisdenedinawaythatdiersfromthesystemsdiscussed.
Languageextensionsaredenedinrulesthatusereectioncapabilitiesofthehostlanguagetocheckforspecicconditions.
Namespaces,packages,class-hierarchies,orannotatedclassescandeneascope.
Atamethodlevelweareabletolookforspecicannotationsinthesourcestringorsimplytrydierentparsers.
Atasub-methodlevelweareabletolookforcertaincodestatementstotransform,eitherusingregular-expressions(beforeparsing)orusingparse-treematching(afterparsing).
Thesetechniquesenableane-grainedcontroloverthelanguages,howeverforendusersitisoftenlessevidentwhatpartsofthesystembelongtothehostlanguageorareexternallydened.
Thiscanbeaddressedbymeansoftailoredhighlightingofsuchcode.
6.
3CombiningLanguagesSincemultiplerule-setscanbeactiveatthesametime,dierentlanguageexten-sionscanbecombined.
Forexample,boththetransactionalmemoryextensionandanyoftheMondrianlanguagescanbeactiveatthesametime,sincetheydonotperformtransformationsatthesameplaceinthecompilertoolchain.
Ifconictingrulesareactive,forexampletwolanguageextensionsthatdenetheirownparser,Helvetiaraisesanerror.
Transformationrulestypicallydon'tconict,sincetheyworkonthesameASTmodel.
Rulesareperformedinadeterministicorderbasedontheirpriority.
Thankstothereectivecapabilitiesofthesystem,eachrulecandetectotheractiverulesandchoosetodisableitselforotherrulesonthey.
Inpracticeconictsarerare,becauselanguageextensionsaretypicallyscopedtoasmallportionofthesystem,suchasaclasshierarchyorapackage.
6.
4HomogeneousLanguagesThelanguagetransformationsystemsuseapreprocessor.
Thisconsiderablyslowsdownthecompilecycles,asseveraltransformationpassesandcompilationcyclesofdierentindependenttoolsareinvolved.
Furthermore,itcanbediculttodebugthegeneratedcode,asitisoftenimpossibletoprovideacorrectmappingfromgeneratedcodebacktotheoriginalsource.
Dierenthostandmetalanguagesmakeinteroperabilitymoredicult.
HelvetiamaintainsthismappingthroughoutasinglecompilerpipelinethatallowsonetousethisinformationinthestandardSmalltalkdebugger.
Transformationrulesaredenedinthehostlanguageandtakeadvantageofthereectivecapabilitiesofthesystem.
6.
5HomogeneousToolsMostsystemsprovidedebuggingtoolsforlanguagedevelopers,howevertheymostlylacksophisticateddebuggingsupportforapplicationdevelopers.
Webelievethatitiscrucialfortheendusersofalanguagetohavegooddebuggingsupport.
Implementingcustomdebuggersisexpensiveandthusseldomdoneinpractice.
Furthermoreswitchingbetweendierentdebuggersinamultilingualenvironmentiscumbersome.
Endusersdonotwanttobeforcedtolearnnewtools,butinsteadpreferthefamiliartoolsprovidedbythehostlanguageinuse.
Helvetiasupportstheuseoftheexistingdebuggingfacilitiesforlanguagedevelopersandendusers.
Whilethehostlanguagedebuggermightnotoertheoptimalabstractionforalllanguages,itoersafreeliveviewontheuntransformedsourcecodeandthecurrentexecutionpoint.
Thisissomethingthatmostothersystemstonotprovidewithoutadditionaldevelopmenteort.
7ConclusionInthispaperwehavepresentedHelvetia,anenvironmentfordeningembeddedlanguagesandforintegratingthemintothehostlanguage.
Wehaveshownhowwecanbendthesyntaxandsemanticsofthehostlanguage,byintroducingafewextensionpointsintothestandardcompilerpipeline.
Ourcontributionsarethefollowing:1.
Wehaveidentiedanddemonstratedthreefundamentaltypesofembeddedlanguages:pidginsadoptthesyntaxofthehostlanguagewhileextendingitssemantics;creolesfurtherrenepidginswiththeirowndedicatedsyntax;argotsswitchthesemanticsofthehostlanguagewithoutchangingthesyntaxandwithoutrequiringchangesinexistingapplicationcode.
2.
Wehavepresentedanovelapproachtolanguageembeddingthatleveragesthehostlanguagetoolchain.
Reusingthetraditionalcoderepresentationofthehostsystemhasnumerousadvantages.
Wehaveachievedatightintegrationofdierentlanguagesthatworkseamlesslywitheachother.
Wespecifytransformationrulesusingannotatedmethods,andspecifythescopeofthesetransformationsusingreectivefacilitiesofthehostlanguage.
Ourapproachworksnicelywithexistingcode,andintegrateswellintotheexistingtoolset.
Fine-grainedcustomizationssuchassyntaxhighlightingarereadilysupported.
3.
WehavedemonstratedafullyworkingprototypeoftheHelvetiasys-temandshowntheimplementationofthreenon-trivialembeddedDSLsindetail.
Wehavealsomentionedtwootherlanguageextensionsinthedo-mainoflanguageengineeringthathavebeenimplementedusingHelvetia,thegrammarspecicationlanguageandthequasiquotingfacility.
Further-moretheHelvetiainfrastructurehasbeenusedfortheimplementationofLanguageBoxes[6],anadaptivelanguagemodelforne-grainedlanguagechanges,languagecompositionandlanguagere-useintermsofgrammartransformation.
AcollectionofHelvetiaexamplelanguagescanbefoundathttp://scg.
unibe.
ch/research/helvetia/examples.
4.
WehaveidentiedthebasicrequirementsforthehostlanguageandhavecomparedHelvetiawithothersystemsforembeddinglanguages.
Asfutureworkweplantovalidateourapproachonawidevarietyofotherlanguageextensionsthatwehavecollectedfromanindustrialcontext.
WealsointendtolookintowaystoautomaticallyrefactorcodefromthehostlanguagetowardsaDSL,aswellashowtoautomatethecreationofDSLlayerstoimprovetheuseofexistingframeworks.
AcknowledgmentsWegratefullyacknowledgethenancialsupportoftheSwissNationalScienceFoundationfortheproject"BringingModelsClosertoCode"(SNFProjectNo.
200020-121594,Oct.
2008–Sept.
2010).
WethankCHOOSE,thespecialinterestgroupforObject-OrientedSystemsandEnvironmentsoftheSwissInformaticsSociety,foritsnancialcontributiontothepresentationofthispaper.
WealsoliketothankDamienCassou,StephaneDucasse,RalfL¨ammel,JorgeRessia,DavidR¨othlisberger,andErwannWernlifortheirfeedbackonearlierdraftsofthispaper.
References1.
Fowler,M.
:FluentInterface,onMartinFowler'sblog(December2005)http://www.
martinfowler.
com/bliki/FluentInterface.
html.
2.
Fowler,M.
:Languageworkbenches:Thekiller-appfordomain-speciclanguages(June2005)http://www.
martinfowler.
com/articles/languageWorkbench.
html.
3.
Mernik,M.
,Heering,J.
,Sloane,A.
M.
:Whenandhowtodevelopdomain-speciclanguages.
ACMComput.
Surv.
37(4)(2005)316–3444.
Hudak,P.
:Buildingdomainspecicembeddedlanguages.
ACMComputingSurveys28(4es)(December1996)5.
Spinellis,D.
:Notabledesignpatternsfordomainspeciclanguages.
JournalofSystemsandSoftware56(1)(February2001)91–996.
Renggli,L.
,Denker,M.
,Nierstrasz,O.
:Languageboxes:Bendingthehostlanguagewithmodularlanguagechanges.
In:SoftwareLanguageEngineering:SecondInternationalConference,SLE2009,Denver,Colorado,October5-6,2009.
LNCS,Springer(2009)toappear.
7.
Sheard,T.
:Accomplishmentsandresearchchallengesinmeta-programming.
In:SAIG2001:ProceedingsoftheSecondInternationalWorkshoponSemantics,Applications,andImplementationofProgramGeneration,London,UK,Springer-Verlag(2001)2–448.
Gybels,K.
,Wuyts,R.
,Ducasse,S.
,D'Hondt,M.
:Inter-languagereection—aconceptualmodelanditsimplementation.
JournalofComputerLanguages,SystemsandStructures32(2-3)(July2006)109–1249.
VanWyk,E.
,Krishnan,L.
,Bodin,D.
,Schwerdfeger,A.
:AttributeGrammar-BasedLanguageExtensionsforJava.
LNCS4609(2007)57510.
Kats,L.
C.
L.
,Bravenboer,M.
,Visser,E.
:Mixingsourceandbytecode.
Acaseforcompilationbynormalization.
InKiczales,G.
,ed.
:ProceedingsOOPSLA2008,Nashville,Tenessee,USA,ACM(October2008)91–10811.
Ekman,T.
,Hedin,G.
:TheJastAddextensibleJavacompiler.
InGabriel,R.
P.
,Bacon,D.
F.
,Lopes,C.
V.
,Jr.
,G.
L.
S.
,eds.
:ProceedingsOOPSLA2007,NewYork,NY,USA,ACMPress(2007)1–1812.
Nystrom,N.
,Clarkson,M.
R.
,Myers,A.
C.
:Polyglot:Anextensiblecompilerframeworkforjava.
In:CompilerConstruction.
Volume2622ofLNCS.
Springer-Verlag(2003)138–15213.
Cox,R.
,Bergan,T.
,Clements,A.
T.
,Kaashoek,F.
,Kohler,E.
:Xoc,anextension-orientedcompilerforsystemsprogramming.
SIGARCHComput.
Archit.
News36(1)(2008)244–25414.
Piumarta,I.
,Warth,A.
:Openreusableobjectmodels.
Technicalreport,ViewpointsResearchInstitute(2006)VPRIResearchNoteRN-2006-003-a.
15.
Warth,A.
,Piumarta,I.
:OMeta:anobject-orientedlanguageforpatternmatching.
In:DLS'07:Proceedingsofthe2007symposiumonDynamiclanguages,NewYork,NY,USA,ACM(2007)11–1916.
Tratt,L.
:TheConvergeprogramminglanguage.
TechnicalReportTR-05-01,DepartmentofComputerScience,King'sCollegeLondon(February2005)17.
Calcagno,C.
,Taha,W.
,Huang,L.
,Leroy,X.
:Implementingmulti-stagelanguagesusingASTs,GenSym,andReection.
In:ProceedingsGPCE.
Volume2830ofLNCS.
,Springer-Verlag(2003)57–7618.
Dimitriev,S.
:Languageorientedprogramming:Thenextprogrammingparadigm.
onBoardOnlineMagazine1(1)(November2004)19.
Simonyi,C.
,Christerson,M.
,Cliord,S.
:Intentionalsoftware.
In:ProceedingsOOPSLA2006,ACM(2006)451–46420.
Solmi,R.
:WholePlatform.
PhDthesis,UniversityofBologna(March2005)21.
Charles,P.
,Fuhrer,R.
M.
,Jr.
,S.
M.
S.
,Duesterwald,E.
,Vinju,J.
J.
:Acceleratingthecreationofcustomized,language-specicidesineclipse.
InArora,S.
,Leavens,G.
T.
,eds.
:ProceedingsOOPSLA2009,ACM(2009)191–20622.
Seaton,C.
:Aprogramminglanguagewherethesyntaxandsemanticsaremutableatruntime.
TechnicalReportCSTR-07-005,UniversityofBristol(June2007)23.
Clark,T.
,Sammut,P.
,Willans,J.
:Superlanguages,DevelopingLanguagesandApplicationswithXMF.
VolumeFirstEdition.
Ceteva(2008)24.
Faith,R.
E.
,Nyland,L.
S.
,Prins,J.
F.
:KHEPERA:asystemforrapidimplementa-tionofdomainspeciclanguages.
In:DSL'97:ProceedingsoftheConferenceonDomain-SpecicLanguagesonConferenceonDomain-SpecicLanguages(DSL),1997,Berkeley,CA,USA,USENIXAssociation(1997)19–1925.
Krahn,H.
,Rumpe,B.
,V¨olkel,S.
:MontiCore:Modulardevelopmentoftextualdomainspeciclanguages.
InPaige,R.
,Meyer,B.
,eds.
:Proceedingsofthe46thInternationalConferenceObjects,Models,Components,Patterns(TOOLS-Europe),Springer-Verlag(2008)297–31526.
Parr,T.
:TheDenitiveANTLRReference:BuildingDomain-SpecicLanguages.
PragmaticProgrammers(May2007)27.
Bravenboer,M.
,Visser,E.
:Concretesyntaxforobjects.
Domain-speciclanguageembeddingandassimilationwithoutrestrictions.
InSchmidt,D.
C.
,ed.
:ProceedingsOOPSLA2004,Vancouver,Canada,ACMPress(oct2004)365–38328.
Visser,E.
:ProgramtransformationwithStratego/XT:Rules,strategies,tools,andsystemsinStrategoXT-0.
9.
InLengauer,C.
,etal.
,eds.
:Domain-SpecicProgramGeneration.
Volume3016ofLNCS.
Spinger-Verlag(June2004)216–23829.
Goldberg,A.
,Robson,D.
:Smalltalk80:theLanguageanditsImplementation.
AddisonWesley,Reading,Mass.
(May1983)30.
Meyer,M.
,Grba,T.
,Lungu,M.
:Mondrian:Anagilevisualizationframework.
In:ACMSymposiumonSoftwareVisualization(SoftVis'06),NewYork,NY,USA,ACMPress(2006)135–14431.
Renggli,L.
,Nierstrasz,O.
:Transactionalmemoryinadynamiclanguage.
JournalofComputerLanguages,SystemsandStructures35(1)(April2009)21–3032.
Herlihy,M.
P.
:Wait-freesynchronization.
ACMTransactionsonProgrammingLanguagesandSystems13(1)(January1991)124–14933.
Herlihy,M.
P.
,Moss,J.
E.
B.
:Transactionalmemory:Architecturalsupportforlock-freedatastructures.
In:Proceedingsofthe20.
AnnualInternationalSymposiumonComputerArchitecture.
(1993)289–30034.
Roberts,D.
,Brant,J.
,Johnson,R.
E.
:ArefactoringtoolforSmalltalk.
TheoryandPracticeofObjectSystems(TAPOS)3(4)(1997)253–26335.
Futamura,Y.
:Partialevaluationofcomputationprocess:Anapproachtoacompiler-compiler.
HigherOrderSymbol.
Comput.
12(4)(1999)381–39136.
Wirth,N.
:WhatcanwedoabouttheunnecessarydiversityofnotationforsyntacticdenitionsCommun.
ACM20(11)(1977)822–82337.
Bracha,G.
:ExecutablegrammarsinNewspeak.
Electron.
NotesTheor.
Comput.
Sci.
193(2007)3–1838.
Tratt,L.
:Domainspeciclanguageimplementationviacompile-timemeta-programming.
ACMTOPLAS30(6)(2008)1–4039.
Renggli,L.
,Grba,T.
:WhySmalltalkwinsthehostlanguagesshootout.
In:ProceedingsofInternationalWorkshoponSmalltalkTechnologies(IWST2009),ACMDigitalLibrary(2009)Toappear.
40.
Bawden,A.
:QuasiquotationinLisp.
In:PartialEvaluationandSemantic-BasedProgramManipulation.
(1999)4–1241.
Taha,W.
:Agentleintroductiontomulti-stageprogramming.
In:Domain-SpecicProgramGeneration.
(2003)30–50

云俄罗斯VPSJusthost俄罗斯VPS云服务器justg:JustHost、RuVDS、JustG等俄罗斯vps主机

俄罗斯vps云服务器商家推荐!俄罗斯VPS,也叫毛子主机(毛子vps),因为俄罗斯离中国大陆比较近,所以俄罗斯VPS的延迟会比较低,国内用户也不少,例如新西伯利亚机房和莫斯科机房都是比较热门的俄罗斯机房。这里为大家整理推荐一些好用的俄罗斯VPS云服务器,这里主要推荐这三家:justhost、ruvds、justg等俄罗斯vps主机,方便大家对比购买适合自己的俄罗斯VPS。一、俄罗斯VPS介绍俄罗斯...

韩国服务器租用优惠点评大全

韩国服务器怎么样?韩国云服务器租用推荐?韩国服务器距离中国近,有天然的地域优势,韩国服务器速度快而且非常稳定!有不少有亚洲市场的外贸公司选择韩国服务器开拓业务,韩国服务器因自身的优势也受到不少用户的青睐。目前的IDC市场上,韩国、香港、美国三个地方的服务器几乎占据了海外服务器的百分之九十以上。韩国服务器相比美国服务器来说速度更快,而相比香港机房来说则带宽更充足,占用市场份额非常大。那么,韩国服务器...

PhotonVPS:$4/月,KVM-2GB/30GB/2TB/洛杉矶&达拉斯&芝加哥等

很久没有分享PhotonVPS的消息,最近看到商家VPS主机套餐有一些更新所以分享下。这是一家成立于2008年的国外VPS服务商,Psychz机房旗下的站点,主要提供VPS和独立服务器等,数据中心包括美国洛杉矶、达拉斯、芝加哥、阿什本等。目前,商家针对Cloud VPS提供8折优惠码,优惠后最低2G内存套餐每月4美元起。下面列出几款主机配置信息。CPU:1core内存:2GB硬盘:30GB NVm...

syntaxhighlighter为你推荐
支持ipad支持ipad支持ipadC1:山东品牌商品馆iphone连不上wifi苹果手机为什么突然连不上家里的wifi?iphone连不上wifi我的苹果手机连不上无线,其它手机能,怎么回事?只是家里的连不上ipad上网为什么我的ipad 显示无法连接到网络phpecho在php中 echo和print 有什么区别联通版iphone4s苹果4s是联通版,或移动版,或全网通如何知道?google中国地图谷歌中国地图用的是什么投影,什么坐标系
万网域名解析 sharktech wordpress技巧 免费cdn加速 智能骨干网 申请个人网站 网站卫士 国外ip加速器 如何建立邮箱 联通网站 万网空间管理 备案空间 畅行云 lamp架构 酸酸乳 金主 xuni 阿里云邮箱申请 hdchina windowsserverr2 更多