interactivelysendmail

-14-January21,1992DonLibesNationalInstituteofStandardsandTechnologyphotoDonLibesreceivedaB.
A.
inMathematicsfromRutgersUniversityandanM.
S.
inComputerSci-encefromtheUniversityofRochester.
CurrentlyattheNationalInstituteofStandardsandTechnology,DonisengagedinresearchthatwillhelpU.
S.
industrymeasurethestandardhack.
Unfortunately,NISTdoesnothaveaverygoodsenseofhumor,sohewasforcedtowritehisrstbook"LifeWithUNIX"throughPrentice-Hall.
-13-January21,1992expectisdesignedtoworkwithprogramsastheyare.
Programsneednotbechangedorrede-signed,nomatterhowpoorlywritten.
Understandably,themajorityofsystemadministratorsarereluctanttomodifyaprogramthatworksandthattheyhavenotwrittenthemselves.
MostpreferwritingshellscriptsusingtheclassicUNIXtoolsphilosophy.
expecthandlestheseproblems,solvingthemdirectlyandwithelegance.
expectscriptsaresmallandsimpleforproblemsthataresmallandsimple.
Whilenotallexpectscriptsaresmall,thescriptsscalewell.
Theyarecomparableinstyletoshellscripts,beingtask-oriented,andprovidesynergywithshellscripts,bothbecausetheycancallshellscriptsandbecalledbythem.
Usedjudiciously,expectisawelcomenewtooltotheworkbenchofallUNIXsystemadministrators.
AcknowledgmentsThisworkwasjointlyfundedbytheNISTAutomatedManufacturingResearchFacility(AMRF,project734-3385)andScienticandTechnicalResearchServices(STRS,project734-3106).
ThecallbackscriptwaswrittenbyScottPaisley.
WalterRowewrotethesendmailversion-checkingscriptmentionedinthepaper.
ThankstoWalterRoweandKenManheimerfortheirhelpfulcommentsonthispaper.
SueMulroneywashelpfulincorrectingmybadlygrammar.
AvailabilitySincethedesignandimplementationofexpectwaspaidforbytheU.
S.
government,itisinthepublicdomain.
However,theauthorandNISTwouldlikecreditifthisprogram,documentationorportionsofthemareused.
expectmaybeftp'daspub/expect/expect.
shar.
Zfromftp.
cme.
nist.
gov.
expectwillbemailedtoyou,ifyousendthemailmessagesendpub/expect/expect.
shar.
Ztolibrary@cme.
nist.
gov.
References[1]DonLibes,"expect:CuringThoseUncontrollableFitsofInteraction",ProceedingsoftheSummer1990USENIXConference,Anaheim,CA,June10-15,1990.
[2]DonLibes,"TheexpectUserManual–programmaticdialoguewithinteractiveprograms",toappearasaNISTIR,NationalInstituteofStandardsandTechnology,November,1990.
[3]JohnOusterhout,"Tcl:AnEmbeddableCommandLanguage",ProceedingsoftheWinter1990USENIXConference,Washington,D.
C.
,January22-26,1990.
[4]JohnOusterhout,"tcl(3)–overviewoftoolcommandlanguagefacilities",unpublishedmanualpage,UniversityofCaliforniaatBerkeley,January1990.
[5]AT&T,UNIXProgrammer'sManual,Section8.
[6]LarryWall,"Perl–PracticalExtractionandReportLanguage",unpublishedmanualpage,March1990.
-12-January21,1992includesahigh-levellanguagethatisinterpretedandbearsastrongsimilaritytotheshellandalsotoC.
Inthatsense,Iseelittletoargueaboutsinceexpectcandoshell-likefunctions.
Inaprevi-ouspaper[1],Ihavesuggestedtheadditionofexpect'sfeaturestotheshell.
Noonewantstolearnyetanothershell,andthereisnoreasonwhythesecapabilitiescannotbeaddedtotheshell.
PerlAmoreinterestingcomparisoniswithPerl,alanguageclaimed(bytheauthor)[6]toembodythebestaspectsoftheshell,C,awk,sed,andanumberofotherUNIXtools.
HavingspentsometimeprogramminginPerl,thereisnoquestioninmymindthatPerliscapableofsolvingthesametasksthatIhavedescribedinthispaper.
Pseudo-ttypackagesforPerlhavebeenwrittenandsend/expectutilitiescouldbewrittenalso.
Perlisaverypowerfullanguage.
Itismuchricherthanthelanguageusedbyexpect(oranyshellforthatmatter).
Thishasadvantagesanddisadvantages.
ThemostobviousdisadvantageisthatPerl'soverabundanceofoptionsandfeaturessimplyaren'tnecessaryforthetasksthatexpectaddresses.
Perl'scomplexityisreectedinitsdiskspace.
Thecomputeronmydesk,aSun3,requires270KtostorePerlandhasasignicantstartuptime.
expect,ontheotherhand,is70Kwithessentiallynostartuptime.
ThereareotherreasonsthatPerlisnotwidelyappliedtocertainproblems,butcompletingthediscussiondeservesapaperofitsown.
InsteadIwillsummarizebysayingthatexpectisappropriatetoonlyafractionofthesystemadministrationproblemsthatPerlsolves.
Thisisintentionallyso.
expectwaswrittentosolveaveryspecicproblem,anditdoesthatconciselyandefciently.
IthinkthatittswellwiththeUNIXphilosophyofsmalltools,unlikePerlwhichdemandsasignicantinvestmentinmasteringitscomplexity.
Giventhechoice,Ipredictthatmostsystemadministratorswouldchooseatoollikeexpectthattakesverylittleefforttolearn,ratherthanenteringtheworldofPerl.
EmacsEmacsisanalogoustoPerlinmanyways,includingitsexibilityandoverabundanceoffunction-ality.
Similarly,Emacscanbeusedtosolvethesesameproblems.
Andformuchthesamerea-sonsasIgaveabove,EmacsisinappropriatefortheclassofproblemsIhavesuggestedinthispaper.
Indeed,consideringthatEmacshasbeenavailableforoveradecade,andI'veneverheardofanyoneusingitthisway,I'llprofferthatEmacsissoinappropriatefortheseproblems,thatitisnotsurprisingthisusagehasneverevenoccurredtoanyone.
ConclusionUNIXshellsareincapableofcontrollinginteractiveprocesses.
Thishasbeenattherootofmanydifcultiesautomatingsystemadministrationtasks.
WhiletheUNIXcommunityisgraduallyprovidingbetterdesignedtoolsanduserinterfaces,evenmoreprogramsarebeingwrittenwithembarrassinglypooruserinterfacesatthesametime.
Thisisunderstandablebecausesystemadministratorsgivemoreprioritytosolvingaproblemsotheycangotothenextone,thangoingbacktoprettyupanoldandworkingsolution.
-11-January21,1992systems.
HowmuchmoreworkcoulditpossiblybeforyoutoadministerjustonemoresystemOh,anditrunsVMS.
")SecuritySeveraloftheexamplespresentedhavepromptedforpasswordsthataredifferentthantheusualUNIXstyle.
Normally,UNIXpromptsforpasswordsdirectlyfrom/dev/tty.
Thishastheunfor-tunatedrawbackthatyoucannotredirectstdin.
Wehaveshownhowtogetaroundthatbyusingexpect.
Ofcourse,doingthisreopensapossiblesecurityhole.
Unprivilegeduserscandetectpasswordspassedasargumentsbyusingps.
Ifpasswordsarestoredinles,lapsesinsecuritycanmakeplaintextpasswordsevidenttopeoplebrowsingthroughyourles.
Publicly-readablebackupmediaareoneofthesimplestsuchsecuritylapses.
Ifyouareatallinterestedinsecurity,Idonotrecommendstoringplaintextpasswordsinles.
Thelikelihoodofsuchapasswordbeingdiscoveredandabusedisjusttoohigh.
Ourusersstorepasswordsinles,butonlyforhighlyrestrictedaccounts,suchasfordemosoranonymousftp.
Thechancesofleakingapasswordthroughpsarelower,andcanbeloweredfurtherstillbyusingthesmallestpossiblescriptaroundthepasswordpromptingprogram.
Suchawindowisextremelysmall.
Nonetheless,securesitesshouldnottakeeventhischance.
Analternativeistohaveexpectinteractivelypromptforpasswords.
Ifyouhaveanexpectscriptthatisdoingacomplicatedseriesoftelnets,ftpsandotherthings,thescriptscanencodeevery-thingbutthepasswords.
Uponrunningsuchascript,theuserwillbeonlybepromptedonceforapassword,andnothingelse.
Thenexpectwillusethatpasswordwhenevernecessary,andcom-pletealltheotherdialoguefromdatapre-storedinles.
Insummary,expectneednotweakensecurity.
Usedwisely,expectcanevenenhancesecurity.
However,youmustusecommonsensewhenwritingscripts.
ComparisontoothersystemadministrationtoolsThissectionofthepapercanbeconsideredcontroversyorheresy,asyouwish.
Itissomewhatreligiousinthattheargumentscanonlyberesolvedbyphilosophicalchoiceratherthanlogic.
IhavekeptitdowntoaveryfewreasonstogiveyouonlythebarestfeelingsforwhatIconsiderisimportanttounderstandwhenchoosingexpectoverothersystemadministrationtools.
Asshouldbeobvious,Ithinkthereareveryfewalternativestousingexpect.
Traditionally,thepopularchoiceshavebeen1)avoidanceand2)Cprogramming.
Thesearenownolongertheonlychoices.
ShellTheshellisincapableofcontrollinginteractiveprocessesinthewaythatexpectcan.
Nonethe-less,certaincomparisonsbetweenexpectandtheshellareinevitable.
Inparticular,expect-10-January21,1992youcantransferahierarchynomatterwhatitlookslikeorhowdeepitis.
expectsupportsrecur-siveprocedures,makingthistaskashortscript.
Mysiteregularlyretrieveslargedistributions(e.
g.
,Gnu,X)thisway.
Assistingadbandother"dumb"programsQuiteoften,vendorsprovideinstructionsformodifyingsystemsintheformofadbinstructions,wheresomeinstructionsmaydependontheresultsofearlierones(i.
e.
,"eachtime_maxusersisincremented,youmustadd16to_nle").
adbhasnospecialscriptinglanguagethatsupportssuchinteraction,nordoestheshellprovidethiscapability.
expectcanperformthisinteraction,playingthepartoftheuser,bydirectlylookingattheresultsofoperations,justasauserwould.
Thistechniquecanbeappliedtoanyprogram.
Infact,expectcanactasanintermediarybetweentheuserandprogramswithpoorly-writtenuserinterfaces.
expectnormallyshowstheentiredia-loguebutcanbetoldnotto.
Thenexpectcanprompttheuserforcommandssuchasshow_maxusersinsteadofadb'snativebutcryptic_maxusers/d.
Translationscanalsobeper-formedinthereversedirection.
Ashortexpectscriptcouldlimitthedifcultyofsystemadminis-tratorswhohavenointerestinmasteringadb.
Inaddition,theabilityofsystemadministratorstoaccidentallycrashthesystembyafewerrantkeystrokeswouldbedramaticallylessened.
GreppingmonsterloglesAcommoncommandsequenceinvolveslookingatalogwith,say,grep,andtheninterruptingit(with^C)afterthelineofinterestappears.
Unfortunately,grepandotherprogramsarelimitedtotheamountofprogrammabilitytheyhave.
Forexample,grepcannotbedirectedtostopsearch-ingaftertherstmatch.
Ashortexpectscriptcansendaninterrupttogrepafterseeingtherstlinejustasiftheuserwereactuallyatthekeyboard.
Withprogramsthatgenerateloglesaslargeasagigabyte,thisisarealproblem.
Withoutexpect,theonlysolutionsaretoletgrepcontinuerunningoverthewholele,ortodedicateahumantothetaskofpressing^Cattherighttime.
expectcancutofftheprocessassoonaspos-sible,mailingtheresultsbackthesystemadministratorifnecessary.
Ingeneral,expectisusefulforsendingoddcharacterstoaprocessthatcannotbeembeddedinashellscript.
expectcanalsoexecutejobcontrolcommands(bg,fg,etc.
)inordertomediatebetweenprocessesthatwereneverdesignedtocommunicatewitheachother.
Again,thiscanrelieveahumanfromthetedioustaskofinteractivelymonitoringprograms.
Administeringnon-UNIXsystemsexpectisaUNIXprogram,yetitcanbeusedtoadministernon-UNIXsystems.
Howisthispos-sibleRunningtelnet(tip,kermit,etc.
)toanon-UNIXhost,itcanloginandperformsend/ex-pectsequencesontheremotecomputer.
Theoperatingsystemorenvironmentoftheremotecomputeriscompletelyirrelevanttoexpect,sinceallofthisisisolatedtotheexpectscriptitself.
ThisisveryusefulforsystemadministratorsthatalreadyhaveaUNIXcomputerontheirdeskbutareforcedbymanagementtoadministeranothercomputer.
("Youalreadyadminister20UNIX-9-January21,1992su,passwd,cryptandotherpassword–eatersProgramsthatreadandwrite/dev/ttycannotbeusedfromshellscriptswithouttheshellscriptaccessing/dev/tty.
Anearlierexampleshowedhowtoforcepasswdnottoreadfrom/dev/tty.
Withthistechnique,youcanchangeitsinputsourcetostdin,aparameter,orevenanenvironmentvariable.
Asanotherexample,supposeyouhavetypedacommandthatfailsbecauseyouweren'troot.
Thetypicalreactionistotypesuandthenreenterthecommand.
Unfortunately,historywon'tworkinthissituationas!
-2willjustevoketheerror-1:Eventnotfound.
Theproblemisthatyouwanttorefertoacommandthatisnowinadifferentshellinstantiation,andthereisnowaytogetbacktoit.
Asolutionistopassthefailedcommandasanargument(via!
!
)toanexpectscriptthatwillpromptyoufortherootpassword,invokesu,andthenfeedtheoriginalfailedcommandtotheresultingsuperusershell.
Iftheexpectscriptexecutesinteractasitslastaction,youwillhavetheoriginalcommandexecutedforyou(noretyping),plusyouwillgetanewsuperusershell.
Thereisnowaytodothiswithsuexceptbyresortingtotemporarylesforyourhistoryandalotofretyping.
Amorepainfulexampleisnewgrp.
Unlikesu,newgrpdoesnotallowadditionalargumentsonthecommandlinetobepassedtothenewshell.
Youmustinteractivelyenterthemafternewgrpbeginsexecuting.
Ineithercase,bothsuandnewgrpareessentiallyuselessinshellscripts.
Security–Thegoodnewsis…Earlier,Imentionedhowtobuildascriptthatwouldforceuserstochoosegoodpasswordswith-outrewritingpasswd.
Allotheralternativeseitherrewritethepasswdprogramorasktheusertoberesponsibleforchoosingagoodpassword.
Ontheoppositesideofthecoin,expectcanbeusedtotestothersitesforsecurelogins(ortobreakin,Isuppose).
Tryingtologinasrootusing,say,allthewordsinanon-linedictionary,atallthelocalhostsatasitewouldbeprohibitivelyexpensiveforahumantodo.
expectwouldworkatitrelentlessly,eventuallyndinganinsecureroot,orshowingthesitetobeprotectedbygoodpasswords.
QuestionsatboottimeWhilebooting,itisusefultovalidateimportantsystemfacts(e.
g.
,date)beforecomingupalltheway.
Ofcourse,ifnooneisstandinginfrontoftheconsole(e.
g.
,thesystembootedduetoapowerfailure)thecomputershouldcomeupanyway.
Writingsuchascriptusingtheshellispain-ful,primarilybecausearead-with-timeoutisnotdirectlyimplementedintheshell.
Inexpect,allreadstimeout.
expectcanpromptandreadfromthekeyboardjustaseasilyasfromaprocess.
TransferringhierarchieswithftpAnonymousftpisverypainfulwhenitcomestodirectoryhierarchies.
Sincethereisnorecursivecopycommand,youmustexplicitlydocdsandgets.
Youcanautomatethisinashellscript,butonlyifthehierarchyisknowninadvance.
expectcanexecuteanlsandlookattheresultssothat-8-January21,1992Inthissection,moreexampleswillbediscussed.
Becauseofspacelimitations,scriptswillnotbeshown,butallofthemhavebeenwrittenandarebeingused.
RegressiontestingTestingnewreleasesofinteractivesoftware(tip,telnet,etc.
)requiresahumantopresskeysandwatchforcorrectresponses.
Doingthismorethanafewtimesbecomesquitetiresome.
Natu-rally,peoplearemuchlesslikelytorunthoroughregressiontestsaftermakingsmallchangesthattheythinkprobablydon'taffectotherpartsofaprogram.
Regressiontestingcanalsobeusefulforyourentireinstallation.
Youcanmakeascriptthattestsallyoursite'slocalapplications,andrunitataftereachsystemupgradeorcongurationchange.
AutomatingloginsManyprogramshaveafrequentlyrepeated,well-denedsetofcommandsandanothersetthatarenotwell-dened.
Forexample,atypicaltelnetsessionalwaysbeginswithalogin,afterwhichtheusercandoanything.
Toautomatethis,expecthastheabilitytopasscontrolfromthescripttotheuser.
Atanytime,theusercanreturncontroltothescripttemporarilytoexecutesequencesofcommonlyrepeatedcommands.
Atmysite,expectisheavilyusedtoautomatetheprocessoflogginginthroughmultiplefront-endsandcommunicationswitches.
Infact,theoriginalreasonexpectwaswrittenwastocreatesixwindows,eachofwhichautomaticallyloggedintoanotherhosttorunademo.
Thegeneralideaofautomatingtelnet,ftp,andtipisveryusefulwhendealingwithhoststhatdonotsupportrloginandrcp.
ButthetechniqueisalsousefulwithnativeUNIXcommandslikesu,login,orrlogin.
expectscriptscancallanyofthem,sendingpasswordsasappropriateandthencontinuingactionsasdesired.
Whileanyofthesecommandscanbeembeddedinashellscript,theshellhasnowayoftakingcontroloverwhathappensinsideoftheseprograms.
Subsequentcommandsfromtheshellscriptdonotgetsenttothenewcontext,butareheldupuntiltheprevi-ouscommandhascompletedsothattheycanbesenttotheoriginalcontext.
expecthasnosuchproblemsswitchingcontextstocontinuecontrollinganyofthesesessions.
telnet–It'snotjustforbreakfastanymoretelnetalsofunctionsasaninterfacetotheexcitingworldofTCPsockets.
telnetcanbeusedtoaccessnon-telnetsocketsandqueryotherhostsfortheirdate(port13),time(port37),listofactiveusers(port11),userinformation(port43),networkstatus(port15),andallsortsofothergoodiesthatyoumightonlybeabletogetifyouhadpermissiontologin.
Forexample,oursiteregularlyrunsascriptthatchecks(port25)whatversionofsendmail.
cfeachofourlocalhostsisactuallyusing.
Ifwedidthisbyreadingles,wewouldneedpermissiontologin,orremotelymountlesystemsandreaddirectoriesandlesonseveralhundredhosts.
Usingtelnetismucheasier,albeitalittlestrange.
-7-January21,1992isimportantisthatexpectscriptsaresmallandsimpleforproblemsthataresmallandsimple.
expectobviatestheneedforresortingtoCjustbecauseoflimitationsonthepartoftheshell.
Example–IntelligentftpOneofoursiteadministratorswantedtospoollesinadirectory.
Later,asecondcomputerwoulduseftptopickthemupandthendeletethemfromtherstcomputer.
Hisrstattemptwastousemget*followedbymdelete*.
Unfortunately,thisdeleteslesthatarriveinthewin-dowbetweenwhenthemgetstartsandthemdeletestarts.
ThescriptfragmentinListing5solvedtheproblem.
Thescriptbeginsbyspawningftp.
Ihaveomittedseverallinesthatopenaconnectionfollowedbysendingandconrmingtheuserandpasswordinformation.
Thenextlinesendsanftpcom-mandtostorethelistofremotelesinalocallecalledlsFile.
Thiscommandisterminatedbyasemicolon,allowingtheresponsetobeveriedwithanexpectcommandonthesamelineofthescript.
exec–ExecuteaUNIXcommand.
execexecutesaUNIXcommandandsimplywaitsforittocomplete,justasifitwereinashellscript.
Inlinefour,catreturnsthelistofles,andtheirnamesarestoredinthevariable,lsVar.
execisusedagaininthenextline,thistimetodeletethelocalle,lsFile.
TheremainderofthescriptmerelyiteratesthroughthevariablelsVar,sendinggetcommandsfol-lowedbydeletecommandsforeachlefoundintheearlierls.
Otherexamplessolvedexpectaddressesasurprisinglylargeclassofsystemadministrationproblemswhichbeforenowhaveeitherbeensolvedbyavoidanceorspecialkludges.
Atthesametime,expectdoesnotattempttosubsumefunctionsalreadyhandledbyotherutilities.
Forexample,thereisnobuilt-inletransfercapability,becauseexpectcanjustcallaprogramtodothat.
Andwhiletheshellisprogrammable,itcannotinteractwithotherinteractiveprocessesanditcannotsolveanyoftheexamplesinthispaper.
Listing5Fragmentofftpspoolscript.
spawnftp.
.
.
sendls*lsFile\r;expect*success*ftp>*setlsVar[execcatlsFile]execrmlsFile\rsetlen[length$lsVar]for{seti0}{$i*senddelete$file\r;expect*success*ftp>*}-6-January21,1992Ifthescriptdoesnotmatchoneoftheprespeciedanswers,thelastcasematches.
(Theisnecessarytopreventthescriptfromtriggeringbeforetheentirequestionarrives.
)Theinteractactionpassescontrolfromthescripttothekeyboard(actuallystdin)sothatahumancananswerthequestion.
interact–Passcontrolfromscripttouserandback.
Duringinteract,theusertakescontrolfordirectinteractions.
Controlisreturnedtothescriptafterpressingtheoptionalescapecharacter.
Inthisscript,+ischosenastheescapecharacterbypassingitastheargumenttointeract.
Arealexpectscriptforfsckwoulddoseveralotherthings.
Forexample,fsckusesseveralstati-cally-sizedtables.
Forthisreason,fsckislimitedtothenumberoferrorsofonetypethatcanbexedinasinglepass.
Thismayrequirefsckberunseveraltimes.
Whilethemanualsaysthis,fsckdoesn't,andfewsystemadministratorsknowfsckthatintimately.
Whenrunfromashellscript,thislackofprogrammabilitywillcausethesystemtocomeupallthewaywithacorruptlesystem(ifthereturncodeisn'tchecked)orbeunnecessarilyrebootedseveraltimes(ifthereturncodeischecked).
Example–CallbackThescriptinListing4waswrittenbyauserwhowantedtodialupthecomputer,andtellittocallhimback.
Sincehelivedoutofthelocalcallingarea,thiswouldgetthecomputertopickuphislong-distancephonebillsforhim.
Therstlinespawnstipwhichopensaconnectiontoamodem.
Next,expectwaitsfortiptosayitisconnectedtothemodem.
Theuser'sphonenumber,passedastherstargumenttothescript,isthenfetchedandaddedtoacommandtodialaHayes-compatiblemodem.
Acarriage-returnisappendedtomakeitappearasifauserhadtypedthestring,andthemodembeginsdialing.
Thethirdlineassigns60tothevariabletimeout.
expectactuallylooksatthisvariableinordertotellithowmanysecondstowaitbeforegivingup.
Eventuallythephoneringsandthemodemanswers.
expectndswhatit'slookingforandexits.
Atthispointgettywakesup,andndingthatithasadialuplinewithDTRonit,startsloginwhichpromptstheusertologin.
Sincethescriptwasoriginallywritten,wehaveaddedafewmorelinestoautomateandverifyphonenumbersbasedontheuidrunningitpartlyforsecurity,butthefragmentshownherewasusedsuccessfullyandformstheheartofourcurrentscript.
Ironically,werecentlynoticeda60KbequivalenttocallbackonUsenetthathadnomorefunctionalitythanadozenorsolinesofexpect.
Ofcourse,notallscriptsarethisshort.
I'mlimitedtowhatcanbepresentedhere,andtheseexamplesreallyservejusttogiveyouafeelofwhatexpectdoesandhowitcanbeapplied.
WhatListing4Callbackscript.
Firstargumentisphonenumber.
spawntipmodemexpect{*connected*}sendATDT[index$argv1]\rsettimeout60expect{*CONNECT*}-5-January21,1992"Assumeayesresponsetoallquestionsaskedbyfsck;thisshouldbeusedwithextremecaution,asitisafreelicensetocontinue,evenaftersevereproblemsareencountered.
"The-noptionhasasimilarlyworthlessmeaning.
Thiskindofinterfaceisinexcusablybad,andyetmanyprogramshavethesamestyle.
Forexample,ftphasanoptionthatdisablesinteractivepromptingsothatitcanberunfromascript,butitprovidesnowaytotakealternativeactionshouldanerroroccur.
Usingexpect,youcanwriteascriptthatallowsfscktoberun,havingquestionsansweredauto-matically.
Listing2isascriptthatcanrunfsckunattendedwhileprovidingthesameexibilityasbeingruninteractively.
Thescriptbeginsbyspawningfsck.
for–Controlsiteration(looping).
Thelanguageusedbyexpectsupportscommonhigh-levelcontrolstructuressuchasif/then/else.
Inthesecondline,aforloopisusedwhichisstructuredsimilarlytotheC-languageversion.
Thebodyoftheforcontainsoneexpectcommand.
Thefollowingexpectcommanddemonstratestheabilitytolookformultiplepatternssimulta-neously.
(Thebackslashes(\)areusedtoquotecharacters–inthiscasewhitespace.
)Inaddition,eachpatterncanhaveanaccompanyingactiontoexecuteifthepatternisfound.
Thisallowsustoprespecifyanswersforspecicquestions.
WhenthequestionsUNREFFILE…CLEARorBADINODENUMBER…FIXappear,thescriptwillautomaticallyanswery.
Ifanythingelseappears,thescriptwillanswern.
Ingeneral,ifallquestionsareknownandanswerableinadvance,ascriptcanberunintheback-ground.
Withmorecomplexprogramsitmaybedesirabletotrapunexpectedquestionsandforceausertointeractivelyevaluatethem.
Listing3isascriptdoesexactlythis.
Listing2Non-interactivefsckscript.
spawnfsckfor{}1{}{expecteofbreak\{*UNREF\FILE*CLEAR\}{sendy\r}\{*BAD\INODE*FIX\}{sendy\r}\{*\}{sendn\r}}Listing3User-friendlyfsckscript.
spawnfsckfor{}1{}{expecteofbreak\{*UNREF\FILE*CLEAR\}{sendn\r}\{*BLK(S)\MISSING*SALVAGE\}{sendy\r}\{*\}{interact+}}-4-January21,1992spawn–Runsaninteractiveprogram.
Thespawnedprogramisreferredtoasthecurrentprocess.
Inthisexample,passwdisspawnedandbecomesthecurrentprocess.
Ausernameispassedasanargumenttopasswd.
expect–Looksforapatternintheoutputofthecurrentprocess.
Theargumentdenesthepattern.
Additionaloptionalargumentsprovidealternativepatternsandactionstoexecutewhenapatternisseen.
(Anexamplewillbeshownlater.
)Inthisexample,expectlooksforthepatternpassword.
Theasteriskallowsittomatchotherdataintheinput,andisausefulshortcuttoavoidspecifyingeverythingindetail.
Thereisnoactionspecied,sothecommandjustwaitsuntilthepatternisfoundbeforecontinuing.
send–Sendsitsargumentstothecurrentprocess.
Thepasswordissenttothecurrentprocess.
The\rindicatesacarriage-return.
(Allthe"usual"Cconventionsaresupported.
)Therearetwosend/expectsequencesbecausepasswdasksthepasswordtobetypedtwiceasaspellingverication.
Thereisnopointtothisinanon-interactivepasswd,butthescripthastodothisbecausepasswddoesn'tknowbetter.
Thenalexpecteofsearchesforanend-of-leintheoutputofpasswdanddemonstratestheuseofkeywordpatterns.
Anotheroneistimeout,usedtodenotethefailureofanypatterntomatch.
Here,eofisnecessaryonlybecausepasswdiscarefullywrittentocheckthatallofitsI/Osucceeds,includingthenalnewlineproducedafterthepasswordhasbeenenteredasecondtime.
Itiseasytoaddacallandtestofgrep$password/usr/dict/wordstothescripttocheckthatapassworddoesn'tappearintheon-linedictionary,however,wewillleavetheillustra-tionofcontrolstructurestothenextexample.
Example–fsckManyprogramsareostensiblynon-interactive.
Thisis,theycanruninthebackgroundbutwithaveryreducedfunctionality.
Forexample,fsckcanberunfromashellscriptonlywiththe-yor-noptions.
Themanual[5]denesthe-yoptionasfollows:Listing1Non-interactivepasswdscript.
Firstargumentisusername.
Secondargumentisnewpassword.
setpassword[index$argv2]spawnpasswd[index$argv1]expect{*password:}send$password\rexpect{*password:}send$password\rexpecteof-3-January21,1992expectisageneral-purposesystemforsolvingtheinteractiveprogramproblem,howeveritsolvesanunusuallylargenumberofproblemsinthesystemadministrationarena.
WhiletheUNIXstyleistobuildsmallprogramsthatcanbeusedasbuildingblocksintheconstructionofotherpro-gramsusingshellsandpipelines,fewsystemadministrationprogramsbehavethisway.
Traditionally,littletimewasspentdesigninggooduserinterfacesforsystemadministratortools.
Thereasonsmaybeanyorallofthefollowing:Systemadministratorswereexperiencedprogrammers,andthereforedidn'tneedallthehand-holdingthatgeneraluserprogramsrequire.
Programssuchasfsckandcrashwereruninfrequently,sotherewaslittlepointspendingmuchtimeonsuchrarelyusedtools.
Systemadministrationtoolswereusedinextremeconditions,considerednotworthprogrammingforbecauseoftheirdifcultyorrarity.
Itwasmorecost-effectivetosolvetheproblembyhandinreal-time.
Systemadministratorssolvedproblemsinsite-dependentways,neverexpect-ingtheirunderdesignedprogramstobepropagatedwidely.
Whateverthereason,theresultisthattheUNIXsystemadministrator'stoolboxislledwithrep-resentativesofsomeoftheworstuserinterfaceseverseen.
Whileonlyacompleteredesignwillhelpalloftheseproblems,expectcanbeusedtoaddressagreatmanyofthem.
Example–passwdTheexpectscriptinListing1takesapasswordasanargument,andcanberunnon-interactivelysuchasbyashellscript.
Ashellscriptcouldpromptandrejecteasilyguessedpasswords.
Alter-natively,theshellscriptcouldcallapasswordgenerator.
Suchacombinationcouldcreatelargenumbersofaccountsatatimewithoutthesystemadministratorhavingtohand-enterpasswordsasiscurrentlydone.
Admittedly,thescriptreopenstheoriginalsecurityproblemthatpasswdwasdesignedtosolve.
Thiscanbeclosedinanumberofways.
Forexample,expectcouldgeneratethepasswordsitselfbydirectlycallingthepasswordgeneratorfromwithinthescript.
ThescriptinglanguageofexpectisdenedcompletelybyLibes[1][2]andOusterhout[3][4].
Inthispaper,commandswillbedescribedastheyareencountered.
Ratherthangivingcomprehen-siveexplanationsofeachcommand,onlyenoughtounderstandtheexampleswillbesupplied.
set–Setstherstargumenttothesecond(i.
e.
,assignment).
Inline1ofthescript,therstargumenttosetispassword.
Thesecondisanexpressionthatisevaluatedtoreturnthesecondargumentofthescriptbyusingtheindexcommand.
Therstargu-mentofindexisalist,fromwhichitretrievestheelementcorrespondingtothepositionofthesecondargument.
argvreferstotheargumentsofthescript,inthesamestyleastheClanguageargv.
-2-January21,1992IntroductionUNIXsystemadministrationofteninvolvesusingprogramsdesignedforinteractiveuse.
Manysuchprograms(passwd,su,etc.
)cannotbeplacedintoshellscripts.
Someprograms(fsck,dump,etc.
)arenotspecicallyinteractive,buthavelittlesupportforautomateduse.
Forexamplethepasswdcommandpromptstheuserforapassword.
Thereisnowaytosupplythepasswordonthecommandline.
Ifyouusepasswdfromashellscript,itwillblockthescriptfromrunningwhileitpromptstheuserwhoinvokedtheshellscript.
Becauseofthis,youcannot,forexample,rejectpasswordsthatarefoundinthesystemdictionary,acommonsecuritymeasure.
Itisironicthatsecuritywasthereasonthatpasswdwasdesignedtoreaddirectlyfromthekeyboardtobeginwith.
passwdisnotaloneinthisrecalcitrantbehavior.
Manyotherprogramsdonotworkwellinsideofshellscriptsandquiteafewofthesearecrucialtoolstothesystemadministrator.
Examplesarerlogin,telnet,crypt,su,dump,adb,andfsck.
Moreproblemswillbementionedlater.
Theproblemwithalloftheseprogramsisnottheprogramsthemselves,buttheshell.
Forexam-ple,theshellcannotseepromptsfrominteractiveprogramsnorcanitseeerrormessages.
Theshellcannotdealwithinteractiveprogramsthiswaybecauseitisincapableofcreatingatwo-wayconnectiontoaprocess.
ThisisaninherentlimitationofclassicUNIXshellssuchassh,cshandksh(fromhereongenericallyreferredtoassimplytheshell).
expect–AnOverviewexpectisaprogramthatsolvesthegeneralproblemofautomatinginteractiveprograms.
expectcommunicateswithprocessesbyinterposingitselfbetweenprocesses(seeFigure1).
Pseudo-ttysareusedsothatprocessesbelievetheyaretalkingtoarealuser.
Ahigh-levelscriptenableshan-dlingofvariedbehavior.
Thescriptoffersjobcontrolsothatmultipleprogramscanbecontrolledsimultaneouslyandaffectoneanother.
Also,arealusermaytakeandreturncontrolfromandtothescriptwhenevernecessary.
Figure1.
expectiscommunicatingwith5processessimultaneously.
Thescriptisincontrolandhasdisabledloggingtotheuser.
Theuseronlyseeswhatthescriptsaystosendandisessentiallytreatedasjustanotherprocess.
expectinteractiveprocessesscriptJanuary21,1992UsingexpecttoAutomateSystemAdministrationTasksDonLibesNationalInstituteofStandardsandTechnologyMetrologyBldg,RoomA-127Gaithersburg,MD20899libes@cme.
nist.
govABSTRACTUNIXsystemadministrationofteninvolvesprogramsdesignedonlyforinteractiveuse.
Manysuchprograms(passwd,su,etc.
)cannotbeplacedintoshellscripts.
Someprograms(fsck,dump,etc.
)arenotspecificallyinteractive,buthavepoorsupportforautomateduse.
expectisaprogramwhichcan"talk"tointeractiveprograms.
Ascriptisusedtoguidethedialogue.
Scriptsarewritteninahigh-levellanguageandprovideflexi-bilityforarbitrarilycomplexdialogues.
Bywritinganexpectscript,onecanruninteractiveprogramsnon-interactively.
Shellscriptsareincapableofmanagingthesesystemadministrationtasks,butexpectscriptscancontrolthemandmanyothers.
Tasksrequiringapersondedi-catedtointeractivelyrespondingtobadlywrittenprograms,canbeautomated.
Inalargeenvironment,thetimeandaggravationsavedisimmense.
expectissimilarinstyletotheshell,andcaneasilybemasteredbyanysystemadministratorwhocanprogramintheshellalready.
Thispaperpresentsrealexam-plesofusingexpecttoautomatesystemadministrationtaskssuchaspasswdandfsck.
Alsodiscussedareanumberofothersystemadministrationtasksthatcanbeautomated.
Keywords:expect,fsck,interaction,passwd,password,programmeddialogue,security,shell,Tcl,UNIX,uucpReprintedfromProceedingsoftheFourthUSENIXLISALargeInstallationSys-temsAdministration(LISA)Conference,ColoradoSprings,CO,October17-19,1990.