MachineLearning-Lecture01

Instructor(AndrewNg):Okay.
Goodmorning.
WelcometoCS229,themachinelearningclass.
SowhatIwannadotodayisjustspendalittletimegoingoverthelogisticsoftheclass,andthenwe'llstarttotalkabitaboutmachinelearning.
Bywayofintroduction,myname'sAndrewNgandI'llbeinstructorforthisclass.
AndsoIpersonallyworkinmachinelearning,andI'veworkedonitforabout15yearsnow,andIactuallythinkthatmachinelearningisthemostexcitingfieldofallthecomputersciences.
SoI'mactuallyalwaysexcitedaboutteachingthisclass.
SometimesIactuallythinkthatmachinelearningisnotonlythemostexcitingthingincomputerscience,butthemostexcitingthinginallofhumanendeavor,somaybealittlebiasthere.
IalsowanttointroducetheTAs,whoareallgraduatestudentsdoingresearchinorrelatedtothemachinelearningandallaspectsofmachinelearning.
PaulBaumstarckworksinmachinelearningandcomputervision.
CatieChangisactuallyaneuroscientistwhoappliesmachinelearningalgorithmstotrytounderstandthehumanbrain.
TomDoisanotherPhDstudent,worksincomputationalbiologyandinsortofthebasicfundamentalsofhumanlearning.
ZicoKolteristheheadTA—he'sheadTAtwoyearsinarownow—worksinmachinelearningandappliesthemtoabunchofrobots.
AndDanielRamageis—Iguesshe'snothere—Danielapplieslearningalgorithmstoproblemsinnaturallanguageprocessing.
Soyou'llgettoknowtheTAsandmemuchbetterthroughoutthisquarter,butjustfromthesortsofthingstheTA'sdo,IhopeyoucanalreadytellthatmachinelearningisahighlyinterdisciplinarytopicinwhichjusttheTAsfindlearningalgorithmstoproblemsincomputervisionandbiologyandrobotsandlanguage.
Andmachinelearningisoneofthosethingsthathasandishavingalargeimpactonmanyapplications.
Sojustinmyowndailywork,IactuallyfrequentlyenduptalkingtopeoplelikehelicopterpilotstobiologiststopeopleincomputersystemsordatabasestoeconomistsandsortofalsoanunendingstreamofpeoplefromindustrycomingtoStanfordinterestedinapplyingmachinelearningmethodstotheirownproblems.
Soyeah,thisisfun.
Acoupleofweeksago,astudentactuallyforwardedtomeanarticlein"ComputerWorld"aboutthe12ITskillsthatemployerscan'tsaynoto.
Soit'saboutsortofthe12mostdesirableskillsinallofITandallofinformationtechnology,andtoppingthelistwasactuallymachinelearning.
SoIthinkthisisagoodtimetobelearningthisstuffandlearningalgorithmsandhavingalargeimpactonmanysegmentsofscienceandindustry.
I'mactuallycuriousaboutsomething.
Learningalgorithmsisoneofthethingsthattouchesmanyareasofscienceandindustries,andI'mjustkindofcurious.
Howmanypeopleherearecomputersciencemajors,areinthecomputersciencedepartmentOkay.
Abouthalfofyou.
HowmanypeoplearefromEEOh,okay,maybeaboutafifth.
HowmanybiologersaretherehereWow,justafew,notmany.
I'msurprised.
AnyonefromstatisticsOkay,afew.
SowherearetherestofyoufromStudent:iCME.
Instructor(AndrewNg):SayagainStudent:iCME.
Instructor(AndrewNg):iCME.
Cool.
Student:[Inaudible].
Instructor(AndrewNg):CiviandwhatelseStudent:[Inaudible]Instructor(AndrewNg):Synthesis,[inaudible]systems.
Yeah,cool.
Student:Chemi.
Instructor(AndrewNg):Chemi.
Cool.
Student:[Inaudible].
Instructor(AndrewNg):Aero/astro.
Yes,right.
Yeah,okay,cool.
AnyoneelseStudent:[Inaudible].
Instructor(AndrewNg):PardonMSNE.
Allright.
Cool.
Yeah.
Student:[Inaudible].
Instructor(AndrewNg):PardonStudent:[Inaudible].
Instructor(AndrewNg):Endo—Student:[Inaudible].
Instructor(AndrewNg):Oh,Isee,industry.
Okay.
Cool.
Great,great.
Soasyoucantellfromacross-sectionofthisclass,Ithinkwe'reaverydiverseaudienceinthisroom,andthat'soneofthethingsthatmakesthisclassfuntoteachandfuntobein,Ithink.
Sointhisclass,we'vetriedtoconveytoyouabroadsetofprinciplesandtoolsthatwillbeusefulfordoingmany,manythings.
AndeverytimeIteachthisclass,IcanactuallyveryconfidentlysaythatafterDecember,nomatterwhatyou'regoingtodoafterthisDecemberwhenyou'vesortofcompletedthisclass,you'llfindthethingsyoulearninthisclassveryuseful,andthesethingswillbeusefulprettymuchnomatterwhatyouendupdoinglaterinyourlife.
SoIhavemorelogisticstogooverlater,butlet'ssayafewmorewordsaboutmachinelearning.
IfeelthatmachinelearninggrewoutofearlyworkinAI,earlyworkinartificialintelligence.
Andoverthelast—Iwannasaylast15orlast20yearsorso,it'sbeenviewedasasortofgrowingnewcapabilityforcomputers.
Andinparticular,itturnsoutthattherearemanyprogramsortherearemanyapplicationsthatyoucan'tprogrambyhand.
Forexample,ifyouwanttogetacomputertoreadhandwrittencharacters,toreadsortofhandwrittendigits,thatactuallyturnsouttobeamazinglydifficulttowriteapieceofsoftwaretotakethisinput,animageofsomethingthatIwroteandtofigureoutjustwhatitis,totranslatemycursivehandwritinginto—toextractthecharactersIwroteoutinlonghand.
Andotherthings:OnethingthatmystudentsandIdoisautonomousflight.
Itturnsouttobeextremelydifficulttositdownandwriteaprogramtoflyahelicopter.
Butincontrast,ifyouwanttodothingsliketogetsoftwaretoflyahelicopterorhavesoftwarerecognizehandwrittendigits,oneverysuccessfulapproachistousealearningalgorithmandhaveacomputerlearnbyitselfhowto,say,recognizeyourhandwriting.
Andinfact,handwrittendigitrecognition,thisisprettymuchtheonlyapproachthatworkswell.
Itusesapplicationsthatarehardtoprogrambyhand.
LearningalgorithmshasalsomadeIguesssignificantinroadsinwhat'ssometimescalleddatabasemining.
So,forexample,withthegrowthofITandcomputers,increasinglymanyhospitalsarekeepingaroundmedicalrecordsofwhatsortofpatients,whatproblemstheyhad,whattheirprognoseswas,whattheoutcomewas.
Andtakingallofthesemedicalrecords,whichstartedtobedigitizedonlyaboutmaybe15years,applyinglearningalgorithmstothemcanturnrawmedicalrecordsintowhatImightlooselycallmedicalknowledgeinwhichwestarttodetecttrendsinmedicalpracticeandevenstarttoaltermedicalpracticeasaresultofmedicalknowledgethat'sderivedbyapplyinglearningalgorithmstothesortsofmedicalrecordsthathospitalshavejustbeenbuildingoverthelast15,20yearsinanelectronicformat.
Turnsoutthatmostofyouprobablyuselearningalgorithms—Idon'tknow—Ithinkhalfadozentimesadayormaybeadozentimesadayormore,andoftenwithoutknowingit.
So,forexample,everytimeyousendmailviatheUSPostalSystem,turnsoutthere'sanalgorithmthattriestoautomaticallyreadthezipcodeyouwroteonyourenvelope,andthat'sdonebyalearningalgorithm.
SoeverytimeyousendUSmail,youareusingalearningalgorithm,perhapswithoutevenbeingawareofit.
Similarly,everytimeyouwriteacheck,Iactuallydon'tknowthenumberforthis,butasignificantfractionofchecksthatyouwriteareprocessedbyalearningalgorithmthat'slearnedtoreadthedigits,sothedollaramountthatyouwrotedownonyourcheck.
Soeverytimeyouwriteacheck,there'sanotherlearningalgorithmthatyou'reprobablyusingwithoutevenbeingawareofit.
Ifyouuseacreditcard,orIknowatleastonephonecompanywasdoingthis,andlotsofcompanieslikeeBayaswellthatdoelectronictransactions,there'sagoodchancethatthere'salearningalgorithminthebackgroundtryingtofigureoutif,say,yourcreditcard'sbeenstolenorifsomeone'sengaginginafraudulenttransaction.
IfyouuseawebsitelikeAmazonorNetflixthatwilloftenrecommendbooksforyoutobuyormoviesforyoutorentorwhatever,theseareotherexamplesoflearningalgorithmsthathavelearnedwhatsortsofthingsyouliketobuyorwhatsortsofmoviesyouliketowatchandcanthereforegivecustomizedrecommendationstoyou.
Justaboutaweekago,Ihadmycarserviced,andeventhere,mycarmechanicwastryingtoexplaintomesomelearningalgorithmintheinnardsofmycarthat'ssortofdoingitsbesttooptimizemydrivingperformanceforfuelefficiencyorsomething.
So,see,mostofususelearningalgorithmshalfadozen,adozen,maybedozensoftimeswithoutevenknowingit.
Andofcourse,learningalgorithmsarealsodoingthingslikegivingusagrowingunderstandingofthehumangenome.
Soifsomedayweeverfindacureforcancer,Ibetlearningalgorithmswillhavehadalargeroleinthat.
That'ssortofthethingthatTomworkson,yesSointeachingthisclass,Isortofhavethreegoals.
OneofthemisjusttoIhopeconveysomeofmyownexcitementaboutmachinelearningtoyou.
Thesecondgoalisbytheendofthisclass,Ihopeallofyouwillbeabletoapplystate-of-the-artmachinelearningalgorithmstowhateverproblemsyou'reinterestedin.
Andifyoueverneedtobuildasystemforreadingzipcodes,you'llknowhowtodothatbytheendofthisclass.
Andlastly,bytheendofthisclass,Irealizethatonlyasubsetofyouareinterestedindoingresearchinmachinelearning,butbytheconclusionofthisclass,Ihopethatallofyouwillactuallybewellqualifiedtostartdoingresearchinmachinelearning,okaySolet'ssayafewwordsaboutlogistics.
Theprerequisitesofthisclassarewrittenononeofthehandouts,areasfollows:Inthisclass,I'mgoingtoassumethatallofyouhavesortofbasicknowledgeofcomputerscienceandknowledgeofthebasiccomputerskillsandprinciples.
SoIassumeallofyouknowwhatbigOnotation,thatallofyouknowaboutsortofdatastructureslikequeues,stacks,binarytrees,andthatallofyouknowenoughprogrammingskillsto,like,writeasimplecomputerprogram.
Anditturnsoutthatmostofthisclasswillnotbeveryprogrammingintensive,althoughwewilldosomeprogramming,mostlyineitherMATLABorOctave.
I'llsayabitmoreaboutthatlater.
Ialsoassumefamiliaritywithbasicprobabilityandstatistics.
Somostundergraduatestatisticsclass,likeStat116taughthereatStanford,willbemorethanenough.
I'mgonnaassumeallofyouknowwhatrandomvariablesare,thatallofyouknowwhatexpectationis,whatavarianceorarandomvariableis.
Andincaseofsomeofyou,it'sbeenawhilesinceyou'veseensomeofthismaterial.
Atsomeofthediscussionsections,we'llactuallygooversomeoftheprerequisites,sortofasarefreshercourseunderprerequisiteclass.
I'llsayabitmoreaboutthatlateraswell.
Lastly,Ialsoassumefamiliaritywithbasiclinearalgebra.
Andagain,mostundergraduatelinearalgebracoursesaremorethanenough.
Soifyou'vetakencourseslikeMath51,103,Math113orCS205atStanford,thatwouldbemorethanenough.
Basically,I'mgonnaassumethatallofyouknowwhatmatrixesandvectorsare,thatyouknowhowtomultiplymatricesandvectorsandmultiplymatrixandmatrices,thatyouknowwhatamatrixinverseis.
Ifyouknowwhataneigenvectorofamatrixis,that'dbeevenbetter.
Butifyoudon'tquiteknoworifyou'renotquitesure,that'sfine,too.
We'llgooveritinthereviewsections.
SothereareacouplemorelogisticalthingsIshoulddealwithinthisclass.
Oneisthat,asmostofyouknow,CS229isatelevisedclass.
Andinfact,IguessmanyofyouareprobablywatchingthisathomeonTV,soI'mgonnasayhitoourhomeviewers.
Soearlierthisyear,IapproachedSCPD,whichtelevisestheseclasses,abouttryingtomakeasmallnumberofStanfordclassespubliclyavailableorpostingthevideosontheweb.
Andsothisyear,Stanfordisactuallystartingasmallpilotprograminwhichwe'llpostvideosofasmallnumberofclassesonline,soontheInternetinawaythatmakesitpubliclyaccessibletoeveryone.
I'mveryexcitedaboutthatbecausemachinelearninginschool,let'sgetthewordoutthere.
Oneoftheconsequencesofthisisthat—let'ssee—sovideosorpicturesofthestudentsinthisclassroomwillnotbepostedonline,soyourimages—sodon'tworryaboutbeingbyseeingyourownfaceappearonYouTubeoneday.
Butthemicrophonesmaypickupyourvoices,soIguesstheconsequenceofthatisthatbecausemicrophonesmaypickupyourvoices,nomatterhowirritatedyouareatme,don'tyelloutswearwordsinthemiddleofclass,butbecausetherewon'tbevideoyoucansafelysitthereandmakefacesatme,andthatwon'tshow,okayLet'ssee.
Ialsohandedoutthis—thereweretwohandoutsIhopemostofyouhave,courseinformationhandout.
Soletmejustsayafewwordsaboutpartsofthese.
Onthethirdpage,there'sasectionthatsaysOnlineResources.
Oh,okay.
LouderActually,couldyouturnupthevolumeTesting.
IsthisbetterTesting,testing.
Okay,cool.
Thanks.
Soallright,onlineresources.
Theclasshasahomepage,soit'sinonthehandouts.
Iwon'twriteonthechalkboard—http://cs229.
stanford.
edu.
Andsowhentherearehomeworkassignmentsorthingslikethat,weusuallywon'tsortof—inthemissionofsavingtrees,wewillusuallynotgiveoutmanyhandoutsinclass.
Sohomeworkassignments,homeworksolutionswillbepostedonlineatthecoursehomepage.
Asfarasthisclass,I'vealsowritten,andIguessI'vealsorevisedeveryyearasetoffairlydetailedlecturenotesthatcoverthetechnicalcontentofthisclass.
Andsoifyouvisitthecoursehomepage,you'llalsofindthedetailedlecturenotesthatgooverindetailallthemathandequationsandsoonthatI'llbedoinginclass.
There'salsoanewsgroup,su.
class.
cs229,alsowrittenonthehandout.
Thisisanewsgroupthat'ssortofaforumforpeopleintheclasstogettoknoweachotherandhavewhateverdiscussionsyouwanttohaveamongstyourselves.
SotheclassnewsgroupwillnotbemonitoredbytheTAsandme.
Butthisisaplaceforyoutoformstudygroupsorfindprojectpartnersordiscusshomeworkproblemsandsoon,andit'snotmonitoredbytheTAsandme.
Sofeelfreetotalktrashaboutthisclassthere.
Ifyouwanttocontacttheteachingstaff,pleaseusetheemailaddresswrittendownhere,cs229-qa@cs.
stanford.
edu.
Thisgoestoanaccountthat'sreadbyalltheTAsandme.
Soratherthansendingusemailindividually,ifyousendemailtothisaccount,itwillactuallyletusgetbacktoyoumaximallyquicklywithanswerstoyourquestions.
Ifyou'reaskingquestionsabouthomeworkproblems,pleasesayinthesubjectlinewhichassignmentandwhichquestiontheemailrefersto,sincethatwillalsohelpustorouteyourquestiontotheappropriateTAortomeappropriatelyandgettheresponsebacktoyouquickly.
Let'ssee.
Skippingahead—let'ssee—forhomework,onemidterm,oneopenandtermproject.
Noticeonthehonorcode.
SoonethingthatIthinkwillhelpyoutosucceedanddowellinthisclassandevenhelpyoutoenjoythisclassmoreisifyouformastudygroup.
Sostartlookingaroundwhereyou'resittingnoworattheendofclasstoday,minglealittlebitandgettoknowyourclassmates.
Istronglyencourageyoutoformstudygroupsandsortofhaveagroupofpeopletostudywithandhaveagroupofyourfellowstudentstotalkovertheseconceptswith.
Youcanalsopostontheclassnewsgroupifyouwanttousethattotrytoformastudygroup.
Butsomeoftheproblemssetsinthisclassarereasonablydifficult.
Peoplethathavetakentheclassbeforemaytellyoutheywereverydifficult.
AndjustIbetitwouldbemorefunforyou,andyou'dprobablyhaveabetterlearningexperienceifyouformastudygroupofpeopletoworkwith.
SoIdefinitelyencourageyoutodothat.
Andjusttosayawordonthehonorcode,whichisIdefinitelyencourageyoutoformastudygroupandworktogether,discusshomeworkproblemstogether.
Butifyoudiscusshomeworkproblemswithotherstudents,thenI'llaskyoutosortofgohomeandwritedownyourownsolutionsindependentlywithoutreferringtonotesthatweretakeninanyofyourjointstudysessions.
Soinotherwords,whenyouturninahomeworkproblem,whatyouturninshouldbesomethingthatwasreconstructedindependentlybyyourselfandwithoutreferringtonotesthatyoutookduringyourstudysessionswithotherpeople,okayAndobviously,showingyoursolutionstoothersorcopyingothersolutionsdirectlyisrightout.
Weoccasionallyalsoreuseproblemsetquestionsfrompreviousyearssothattheproblemsareabitmoredebuggedandworkmoresmoothly.
Andasaresultofthat,Ialsoaskyounottolookatsolutionsfrompreviousyears,andthisincludesbothsortofofficialsolutionsthatwe'vegivenouttopreviousgenerationsofthisclassandprevioussolutionsthatpeoplethathavetakenthisclassinpreviousyearsmayhavewrittenoutbythemselves,okaySadly,inthisclass,thereareusually—sadly,inpreviousyears,therehaveoftenbeenafewhonorcodeviolationsinthisclass.
Andlastyear,IthinkIprosecutedfivehonorcodeviolations,whichIthinkisaridiculouslylargenumber.
Andsojustdon'tworkwithoutsolutions,andhopefullythere'llbezerohonorcodeviolationsthisyear.
I'dloveforthattohappen.
Thesectionhereonthelatehomeworkpolicyifyoueverwanttohandinahomeworklate,I'llleaveyoutoreadthatyourself.
Wealsohaveamidterm,whichisscheduledforThursday,8thofNovemberat6:00p.
m.
,sopleasekeepthateveningfree.
Andlet'ssee.
AndonemoreadministrativethingIwantedtosayisabouttheclassproject.
Sopartofthegoalofthisclassistoleaveyouwellequippedtoapplymachinelearningalgorithmstoaproblemortodoresearchinmachinelearning.
Andsoaspartofthisclass,I'llaskyoutoexecuteasmallresearchprojectsortofasasmalltermproject.
Andwhatmoststudentsdoforthisiseitherapplymachinelearningtoaproblemthatyoufindinterestingorinvestigatesomeaspectofmachinelearning.
Sotothoseofyouthatareeitheralreadydoingresearchortothoseofyouwhoareinindustry,you'retakingthisfromacompany,onefantasticsortofwaytodoaclassprojectwouldbeifyouapplymachinelearningalgorithmstoaproblemthatyou'reinterestedin,toaproblemthatyou'realreadyworkingon,whetheritbeascienceresearchproblemorsortofaprobleminindustrywhereyou'retryingtogetasystemtoworkusingalearningalgorithm.
Tothoseofyouthatarenotcurrentlydoingresearch,onegreatwaytodoaprojectwouldbeifyouapplylearningalgorithmstojustpickaproblemthatyoucareabout.
Pickaproblemthatyoufindinteresting,andapplylearningalgorithmstothatandplaywiththeideasandseewhathappens.
Andlet'ssee.
Oh,andthegoaloftheprojectshouldreallybeforyoutodoapublishablepieceofresearchinmachinelearning,okayAndifyougotothecoursewebsite,you'llactuallyfindalistoftheprojectsthatstudentshaddonelastyear.
AndsoI'mholdingthelistinmyhand.
Youcangohomelaterandtakealookatitonline.
Butreadingdownthislist,Iseethatlastyear,therewerestudentsthatappliedlearningalgorithmstocontrolasnakerobot.
Therewasafewprojectsonimprovinglearningalgorithms.
There'saprojectonflyingautonomousaircraft.
TherewasaprojectactuallydonebyourTAPaulonimprovingcomputervisionalgorithmsusingmachinelearning.
ThereareacoupleofprojectsonNetflixrankingsusinglearningalgorithms;afewmedicalrobots;onesonsegmenting[inaudible]tosegmentingpiecesofthebodyusinglearningalgorithms;oneonmusicalinstrumentdetection;anotheronironysequencealignment;andafewalgorithmsonunderstandingthebrainneuroscience,actuallyquiteafewprojectsonneuroscience;acoupleofprojectsonundescendingfMRIdataonbrainscans,andsoon;anotherprojectonmarketmakings,thefinancialtrading.
Therewasaninterestingprojectontryingtouselearningalgorithmstodecidewhatisitthatmakesaperson'sfacephysicallyattractive.
There'salearningalgorithmonopticalillusions,andsoon.
Anditgoeson,solotsoffunprojects.
Andtakealook,thencomeupwithyourownideas.
Butwhateveryoufindcoolandinteresting,Ihopeyou'llbeabletomakemachinelearningaprojectoutofit.
Yeah,questionStudent:ArethesegroupprojectsInstructor(AndrewNg):Oh,yes,thankyou.
Student:SohowmanypeoplecanbeinagroupInstructor(AndrewNg):Right.
Soprojectscanbedoneingroupsofuptothreepeople.
Soaspartofformingstudygroups,latertodayasyougettoknowyourclassmates,Idefinitelyalsoencourageyoutograbtwootherpeopleandformagroupofuptothreepeopleforyourproject,okayAndjuststartbrainstormingideasfornowamongstyourselves.
YoucanalsocomeandtalktomeortheTAsifyouwanttobrainstormideaswithus.
Okay.
Soonemoreorganizationalquestion.
I'mcurious,howmanyofyouknowMATLABWow,cool,quitealot.
Okay.
Soaspartofthe—actuallyhowmanyofyouknowOctaveorhaveusedOctaveOh,okay,muchsmallernumber.
Soaspartofthisclass,especiallyinthehomeworks,we'llaskyoutoimplementafewprograms,afewmachinelearningalgorithmsaspartofthehomeworks.
AndmostofthosehomeworkswillbedoneineitherMATLABorinOctave,whichissortof—IknowsomepeoplecallitafreeversionofMATLAB,whichitsortofis,sortofisn't.
SoIguessforthoseofyouthathaven'tseenMATLABbefore,andIknowmostofyouhave,MATLABisIguesspartoftheprogramminglanguagethatmakesitveryeasytowritecodesusingmatrices,towritecodefornumericalroutines,tomovedataaround,toplotdata.
Andit'ssortofanextremelyeasytolearntooltouseforimplementingalotoflearningalgorithms.
Andincasesomeofyouwanttoworkonyourownhomecomputerorsomethingifyoudon'thaveaMATLABlicense,forthepurposesofthisclass,there'salso—[inaudible]writethatdown[inaudible]MATLAB—there'salsoasoftwarepackagecalledOctavethatyoucandownloadforfreeofftheInternet.
AndithassomewhatfewerfeaturesthanMATLAB,butit'sfree,andforthepurposesofthisclass,itwillworkforjustabouteverything.
SoactuallyI,well,soyeah,justasidecommentforthoseofyouthathaven'tseenMATLABbeforeIguess,onceacolleagueofmineatadifferentuniversity,notatStanford,actuallyteachesanothermachinelearningcourse.
He'staughtitformanyyears.
Sooneday,hewasinhisoffice,andanoldstudentofhisfrom,like,tenyearsagocameintohisofficeandhesaid,"Oh,professor,professor,thankyousomuchforyourmachinelearningclass.
Ilearnedsomuchfromit.
There'sthisstuffthatIlearnedinyourclass,andInowuseeveryday.
Andit'shelpedmemakelotsofmoney,andhere'sapictureofmybighouse.
"Somyfriendwasveryexcited.
Hesaid,"Wow.
That'sgreat.
I'mgladtohearthismachinelearningstuffwasactuallyuseful.
SowhatwasitthatyoulearnedWasitlogisticregressionWasitthePCAWasitthedatanetworksWhatwasitthatyoulearnedthatwassohelpful"Andthestudentsaid,"Oh,itwastheMATLAB.
"Soforthoseofyouthatdon'tknowMATLAByet,Ihopeyoudolearnit.
It'snothard,andwe'llactuallyhaveashortMATLABtutorialinoneofthediscussionsectionsforthoseofyouthatdon'tknowit.
Okay.
Theverylastpieceoflogisticalthingisthediscussionsections.
SodiscussionsectionswillbetaughtbytheTAs,andattendanceatdiscussionsectionsisoptional,althoughthey'llalsoberecordedandtelevised.
Andwe'llusethediscussionsectionsmainlyfortwothings.
Forthenexttwoorthreeweeks,we'llusethediscussionsectionstogoovertheprerequisitestothisclassorifsomeofyouhaven'tseenprobabilityorstatisticsforawhileormaybealgebra,we'llgooverthoseinthediscussionsectionsasarefresherforthoseofyouthatwantone.
Laterinthisquarter,we'llalsousethediscussionsectionstogooverextensionsforthematerialthatI'mteachinginthemainlectures.
Somachinelearningisahugefield,andthereareafewextensionsthatwereallywanttoteachbutdidn'thavetimeinthemainlecturesfor.
Solaterthisquarter,we'llusethediscussionsectionstotalkaboutthingslikeconvexoptimization,totalkalittlebitabouthiddenMarkovmodels,whichisatypeofmachinelearningalgorithmformodelingtimeseriesandafewotherthings,soextensionstothematerialsthatI'llbecoveringinthemainlectures.
Andattendanceatthediscussionsectionsisoptional,okaySothatwasallIhadfromlogistics.
Beforewemoveontostarttalkingabitaboutmachinelearning,letmecheckwhatquestionsyouhave.
YeahStudent:[Inaudible]RorsomethinglikethatInstructor(AndrewNg):Oh,yeah,let'ssee,right.
Soourpolicyhasbeenthatyou'rewelcometouseR,butIwouldstronglyadviseagainstit,mainlybecauseinthelastproblemset,weactuallysupplysomecodethatwillruninOctavebutthatwouldbesomewhatpainfulforyoutotranslateintoRyourself.
Soforyourotherassignments,ifyouwannasubmitasolutioninR,that'sfine.
ButIthinkMATLABisactuallytotallyworthlearning.
IknowRandMATLAB,andIpersonallyendupusingMATLABquiteabitmoreoftenforvariousreasons.
YeahStudent:Forthe[inaudible]project[inaudible]Instructor(AndrewNg):Soforthetermproject,you'rewelcometodoitinsmallergroupsofthree,oryou'rewelcometodoitbyyourselforingroupsoftwo.
Gradingisthesameregardlessofthegroupsize,sowithalargergroup,youprobably—Irecommendtryingtoformateam,butit'sactuallytotallyfinetodoitinasmallergroupifyouwant.
Student:[Inaudible]whatlanguage[inaudible]Instructor(AndrewNg):Solet'ssee.
ThereisnoCprogramminginthisclassotherthananythatyoumaychoosetodoyourselfinyourproject.
SoallthehomeworkscanbedoneinMATLABorOctave,andlet'ssee.
AndIguesstheprogramprerequisitesismoretheabilitytounderstandbigOnotationandknowledgeofwhatadatastructure,likealinkedlistoraqueueorbinarytreatments,moresothanyourknowledgeofCorJavaspecifically.
YeahStudent:Lookingattheendsemesterproject,Imean,whatexactlywillyoubetestingoverthere[Inaudible]Instructor(AndrewNg):OftheprojectStudent:Yeah.
Instructor(AndrewNg):Yeah,letmeanswerthatlater.
Inacoupleofweeks,Ishallgiveoutahandoutwithguidelinesfortheproject.
Butfornow,weshouldthinkofthegoalasbeingtodoacoolpieceofmachinelearningworkthatwillletyouexperiencethejoysofmachinelearningfirsthandandreallytrytothinkaboutdoingapublishablepieceofwork.
Somanystudentswilltrytobuildacoolmachinelearningapplication.
That'sprobablythemostcommonproject.
Somestudentswilltrytoimprovestate-of-the-artmachinelearning.
Someofthoseprojectsarealsoverysuccessful.
It'salittlebithardertodo.
Andthere'salsoasmallerminorityofstudentsthatwillsometimestrytoprove—developthetheoryofmachinelearningfurtherortrytoprovetheoremsaboutmachinelearning.
Sothey'reusuallygreatprojectsofallofthosetypeswithapplicationsandmachinelearningbeingthemostcommon.
AnythingelseOkay,cool.
Sothatwasitforlogistics.
Let'stalkaboutlearningalgorithms.
SocanIhavethelaptopdisplay,please,ortheprojectorActually,couldyoulowerthebigscreenCool.
Thisisamazingcustomerservice.
Thankyou.
Isee.
Okay,cool.
Okay.
No,that'sfine.
Isee.
Okay.
That'scool.
Thanks.
Okay.
Bigscreenisn'tworkingtoday,butIhopeyoucanreadthingsonthesmallerscreensoutthere.
Actually,[inaudible]Ithinkthisroomjustgotanewprojectorthat—someonesentyouanexcitedemail—wasitjustonFriday—sayingwejustgotanewprojectorandtheysaid4,000-to-1somethingorotherbrightnessratio.
Idon'tknow.
Someonewasveryexcitedaboutthenewprojectorinthisroom,butIguesswe'llseethatinoperationonWednesday.
Sostartbytalkingaboutwhatmachinelearningis.
WhatismachinelearningActually,canyoureadthetextoutthereRaiseyourhandifthetextonthesmallscreensislegible.
Oh,okay,cool,mostlylegible.
Okay.
SoI'lljustreaditout.
SowhatismachinelearningWaybackinabout1959,ArthurSamueldefinedmachinelearninginformallyasthe[inaudible]thatgivescomputerstolearn—[inaudible]thatgivescomputerstheabilitytolearnwithoutbeingexplicitlyprogrammed.
SoArthurSamuel,sowaybackinthehistoryofmachinelearning,actuallydidsomethingverycool,whichwashewroteacheckersprogram,whichwouldplaygamesofcheckersagainstitself.
Andsobecauseacomputercanplaythousandsofgamesagainstitselfrelativelyquickly,ArthurSamuelhadhisprogramplaythousandsofgamesagainstitself,andovertimeitwouldstarttolearntorecognizepatternswhichledtowinsandpatternswhichledtolosses.
Soovertimeitlearnedthingslikethat,"Gee,ifIgetalotofpiecestakenbytheopponent,thenI'mmorelikelytolosethanwin,"or,"Gee,ifIgetmypiecesintoacertainposition,thenI'mespeciallylikelytowinratherthanlose.
"Andsoovertime,ArthurSamuelhadacheckersprogramthatwouldactuallylearntoplaycheckersbylearningwhatarethesortofboardpositionsthattendtobeassociatedwithwinsandwhataretheboardpositionsthattendtobeassociatedwithlosses.
Andwaybackaround1959,theamazingthingaboutthiswasthathisprogramactuallylearnedtoplaycheckersmuchbetterthanArthurSamuelhimselfcould.
Soeventoday,therearesomepeoplethatsay,well,computerscan'tdoanythingthatthey'renotexplicitlyprogrammedto.
AndArthurSamuel'scheckersprogramwasmaybethefirstIthinkreallyconvincingrefutationofthisclaim.
Namely,ArthurSamuelmanagedtowriteacheckersprogramthatcouldplaycheckersmuchbetterthanhepersonallycould,andthisisaninstanceofmaybecomputerslearningtodothingsthattheywerenotprogrammedexplicitlytodo.
Here'samorerecent,amoremodern,moreformaldefinitionofmachinelearningduetoTomMitchell,whosaysthatawell-posedlearningproblemisdefinedasfollows:HesaysthatacomputerprogramissettolearnfromanexperienceEwithrespecttosometaskTandsomeperformancemeasurePifitsperformanceonTasmeasuredbyPimproveswithexperienceE.
Okay.
Sonotonlyisitadefinition,itevenrhymes.
So,forexample,inthecaseofcheckers,theexperienceEthataprogramhaswouldbetheexperienceofplayinglotsofgamesofcheckersagainstitself,say.
ThetaskTisthetaskofplayingcheckers,andtheperformancemeasurePwillbesomethinglikethefractionofgamesitwinsagainstacertainsetofhumanopponents.
Andbythisdefinition,we'llsaythatArthurSamuel'scheckersprogramhaslearnedtoplaycheckers,okaySoasanoverviewofwhatwe'regoingtodointhisclass,thisclassissortoforganizedintofourmajorsections.
We'regonnatalkaboutfourmajortopicsinthisclass,thefirstofwhichissupervisedlearning.
Soletmegiveyouanexampleofthat.
Sosupposeyoucollectadatasetofhousingprices.
AndoneoftheTAs,DanRamage,actuallycollectedadatasetformelastweektouseintheexamplelater.
Butsupposethatyougotocollectstatisticsabouthowmuchhousescostinacertaingeographicarea.
AndDan,theTA,collecteddatafromhousingpricesinPortland,Oregon.
Sowhatyoucandoislet'ssayplotthesquarefootageofthehouseagainstthelistpriceofthehouse,right,soyoucollectdataonabunchofhouses.
Andlet'ssayyougetadatasetlikethiswithhousesofdifferentsizesthatarelistedfordifferentamountsofmoney.
Now,let'ssaythatI'mtryingtosellahouseinthesameareaasPortland,Oregonaswherethedatacomesfrom.
Let'ssayIhaveahousethat'sthissizeinsquarefootage,andIwantanalgorithmtotellmeabouthowmuchshouldIexpectmyhousetosellfor.
Sotherearelotsofwaystodothis,andsomeofyoumayhaveseenelementsofwhatI'mabouttosaybefore.
Soonethingyoucoulddoislookatthisdataandmaybeputastraightlinetoit.
Andthenifthisismyhouse,youmaythenlookatthestraightlineandpredictthatmyhouseisgonnagoforaboutthatmuchmoney,rightThereareotherdecisionsthatwecanmake,whichwe'lltalkaboutlater,whichis,well,whatifIdon'twannaputastraightlineMaybeIshouldputaquadraticfunctiontoit.
Maybethatfitsthedataalittlebitbetter.
Younoticeifyoudothat,thepriceofmyhousegoesupabit,sothat'dbenice.
Andthissortoflearningproblemoflearningtopredicthousingpricesisanexampleofwhat'scalledasupervisedlearningproblem.
Andthereasonthatit'scalledsupervisedlearningisbecausewe'reprovidingthealgorithmadatasetofabunchofsquarefootages,abunchofhousingsizes,andaswellassortoftherightanswerofwhattheactualpricesofanumberofhouseswere,rightSowecallthissupervisedlearningbecausewe'resupervisingthealgorithmor,inotherwords,we'regivingthealgorithmthe,quote,rightanswerforanumberofhouses.
Andthenwewantthealgorithmtolearntheassociationbetweentheinputsandtheoutputsandtosortofgiveusmoreoftherightanswers,okayItturnsoutthisspecificexamplethatIdrewhereisanexampleofsomethingcalledaregressionproblem.
Andthetermregressionsortofreferstothefactthatthevariableyou'retryingtopredictisacontinuousvalueandprice.
There'sanotherclassofsupervisedlearningproblemswhichwe'lltalkabout,whichareclassificationproblems.
Andso,inaclassificationproblem,thevariableyou'retryingtopredictisdiscreetratherthancontinuous.
Soasonespecificexample—soactuallyastandarddatasetyoucandownloadonline[inaudible]thatlotsofmachinelearningpeoplehaveplayedwith.
Let'ssayyoucollectadatasetonbreastcancertumors,andyouwanttolearnthealgorithmtopredictwhetherornotacertaintumorismalignant.
Malignantistheoppositeofbenign,right,somalignancyisasortofharmful,badtumor.
Sowecollectsomenumberoffeatures,somenumberofpropertiesofthesetumors,andforthesakeofsortofhavingasimple[inaudible]explanation,let'sjustsaythatwe'regoingtolookatthesizeofthetumoranddependingonthesizeofthetumor,we'lltrytofigureoutwhetherornotthetumorismalignantorbenign.
Sothetumoriseithermalignantorbenign,andsothevariableintheYaxisiseitherzeroor1,andsoyourdatasetmaylooksomethinglikethat,rightAndthat's1andthat'szero,okayAndsothisisanexampleofaclassificationproblemwherethevariableyou'retryingtopredictisadiscreetvalue.
It'seitherzeroor1.
Andinfact,moregenerally,therewillbemanylearningproblemswherewe'llhavemorethanoneinputvariable,morethanoneinputfeatureandusemorethanonevariabletotrytopredict,say,whetheratumorismalignantorbenign.
So,forexample,continuingwiththis,youmayinsteadhaveadatasetthatlookslikethis.
I'mgonnapartthisdatasetinaslightlydifferentwaynow.
AndI'mmakingthisdatasetlookmuchcleanerthanitreallyisinrealityforillustration,okayForexample,maybethecrossesindicatemalignanttumorsandthe"O"smayindicatebenigntumors.
Andsoyoumayhaveadatasetcomprisingpatientsofdifferentagesandwhohavedifferenttumorsizesandwhereacrossindicatesamalignanttumor,andan"O"indicatesabenigntumor.
Andyoumaywantanalgorithmtolearntopredict,givenanewpatient,whethertheirtumorismalignantorbenign.
So,forexample,whatalearningalgorithmmaydoismaybecomeinanddecidethatastraightlinelikethatseparatesthetwoclassesoftumorsreallywell,andsoifyouhaveanewpatientwho'sageandtumorsizefalloverthere,thenthealgorithmmaypredictthatthetumorisbenignratherthanmalignant,okaySothisisjustanotherexampleofanothersupervisedlearningproblemandanotherclassificationproblem.
Andsoitturnsoutthatoneoftheissueswe'lltalkaboutlaterinthisclassisinthisspecificexample,we'regoingtotrytopredictwhetheratumorismalignantorbenignbasedontwofeaturesorbasedontwoinputs,namelytheageofthepatientandthetumorsize.
Itturnsoutthatwhenyoulookatarealdataset,youfindthatlearningalgorithmsoftenuseothersetsoffeatures.
Inthebreastcancerdataexample,youalsousepropertiesofthetumors,likeclumpthickness,uniformityofcellsize,uniformityofcellshape,[inaudible]adhesionandsoon,sovariousothermedicalproperties.
Andoneofthemostinterestingthingswe'lltalkaboutlaterthisquarteriswhatifyourdatadoesn'tlieinatwo-dimensionalorthree-dimensionalorsortofevenafinitedimensionalspace,butisitpossible—whatifyourdataactuallyliesinaninfinitedimensionalspaceOurplotsherearetwo-dimensionalspace.
Ican'tplotyouaninfinitedimensionalspace,rightAndsoitturnsoutthatoneofthemostsuccessfulclassesofmachinelearningalgorithms—somemaycallsupportvectormachines—actuallytakesdataandmapsdatatoaninfinitedimensionalspaceandthendoesclassificationusingnottwofeatureslikeI'vedonehere,butaninfinitenumberoffeatures.
Andthatwillactuallybeoneofthemostfascinatingthingswetalkaboutwhenwegodeeplyintoclassificationalgorithms.
Andit'sactuallyaninterestingquestion,right,sothinkabouthowdoyouevenrepresentaninfinitedimensionalvectorincomputermemoryYoudon'thaveaninfiniteamountofcomputers.
HowdoyouevenrepresentapointthatliesinaninfinitedimensionalspaceWe'lltalkaboutthatwhenwegettosupportvectormachines,okaySolet'ssee.
Sothatwassupervisedlearning.
Thesecondofthefourmajortopicsofthisclasswillbelearningtheory.
SoIhaveafriendwhoteachesmathatadifferentuniversity,notatStanford,andwhenyoutalktohimabouthisworkandwhathe'sreallyouttodo,thisfriendofminewill—he'samathprofessor,right—thisfriendofminewillsortofgetthelookofwonderinhiseyes,andhe'lltellyouabouthowinhismathematicalwork,hefeelslikehe'sdiscoveringtruthandbeautyintheuniverse.
Andhesaysitinsortofareallytouching,sincereway,andthenhehasthis—youcanseeitinhiseyes—hehasthisdeepappreciationofthetruthandbeautyintheuniverseasrevealedtohimbythemathhedoes.
Inthisclass,I'mnotgonnadoanytruthandbeauty.
Inthisclass,I'mgonnatalkaboutlearningtheorytotrytoconveytoyouanunderstandingofhowandwhylearningalgorithmsworksothatwecanapplytheselearningalgorithmsaseffectivelyaspossible.
So,forexample,itturnsoutyoucanprovesurprisinglydeeptheoremsonwhenyoucanguaranteethatalearningalgorithmwillwork,allrightSothinkaboutalearningalgorithmforreadingzipcodes.
Whencanyouproveatheoremguaranteeingthatalearningalgorithmwillbeatleast99.
9percentaccurateonreadingzipcodesThisisactuallysomewhatsurprising.
Weactuallyprovetheoremsshowingwhenyoucanexpectthattohold.
We'llalsosortofdelveintolearningtheorytotrytounderstandwhatalgorithmscanapproximatedifferentfunctionswellandalsotrytounderstandthingslikehowmuchtrainingdatadoyouneedSohowmanyexamplesofhousesdoIneedinorderforyourlearningalgorithmtorecognizethepatternbetweenthesquarefootageofahouseanditshousingpriceAndthiswillhelpusanswerquestionslikeifyou'retryingtodesignalearningalgorithm,shouldyoubespendingmoretimecollectingmoredataorisitacasethatyoualreadyhaveenoughdata;itwouldbeawasteoftimetotrytocollectmore.
OkaySoIthinklearningalgorithmsareaverypowerfultoolthatasIwalkaroundsortofindustryinSiliconValleyorasIworkwithvariousbusinessesinCSandoutsideCS,Ifindthatthere'softenahugedifferencebetweenhowwellsomeonewhoreallyunderstandsthisstuffcanapplyalearningalgorithmversussomeonewhosortofgetsitbutsortofdoesn't.
TheanalogyIliketothinkofisimagineyouweregoingtoacarpentryschoolinsteadofamachinelearningclass,rightIfyougotoacarpentryschool,theycangiveyouthetoolsofcarpentry.
They'llgiveyouahammer,abunchofnails,ascrewdriverorwhatever.
Butamastercarpenterwillbeabletousethosetoolsfarbetterthanmostofusinthisroom.
IknowacarpentercandothingswithahammerandnailthatIcouldn'tpossibly.
Andit'sactuallyalittlebitlikethatinmachinelearning,too.
Onethingthat'ssadlynottaughtinmanycoursesonmachinelearningishowtotakethetoolsofmachinelearningandreally,reallyapplythemwell.
Sointhesameway,sothetoolsofmachinelearningareIwannasayquiteabitmoreadvancedthanthetoolsofcarpentry.
Maybeacarpenterwilldisagree.
Butalargepartofthisclasswillbejustgivingyoutherawtoolsofmachinelearning,justthealgorithmsandsoon.
ButwhatIplantodothroughoutthisentirequarter,notjustinthesegmentoflearningtheory,butactuallyasathemerunningthrougheverythingIdothisquarter,willbetotrytoconveytoyoutheskillstoreallytakethelearningalgorithmideasandreallytogetthemtoworkonaproblem.
It'ssortofhardformetostandhereandsayhowbigadealthatis,butwhenIwalkaroundcompaniesinSiliconValley,it'scompletelynotuncommonformetoseesomeoneusingsomemachinelearningalgorithmandthenexplaintomewhatthey'vebeendoingforthelastsixmonths,andIgo,oh,gee,itshouldhavebeenobviousfromthestartthatthelastsixmonths,you'vebeenwastingyourtime,rightAndsomygoalinthisclass,runningthroughtheentirequarter,notjustonlearningtheory,isactuallynotonlytogiveyouthetoolsofmachinelearning,buttoteachyouhowtousethemwell.
AndI'venoticedthisissomethingthatreallynotmanyotherclassesteach.
AndthisissomethingI'mreallyconvincedisahugedeal,andsobytheendofthisclass,Ihopeallofyouwillbemastercarpenters.
Ihopeallofyouwillbereallygoodatapplyingtheselearningalgorithmsandgettingthemtoworkamazinglywellinmanyproblems.
OkayLet'ssee.
So[inaudible]theboard.
Afterlearningtheory,there'sanotherclassoflearningalgorithmsthatIthenwanttoteachyouabout,andthat'sunsupervisedlearning.
Soyourecall,right,alittleearlierIdrewanexamplelikethis,right,whereyouhaveacoupleoffeatures,acoupleofinputvariablesandsortofmalignanttumorsandbenigntumorsorwhatever.
Andthatwasanexampleofasupervisedlearningproblembecausethedatayouhavegivesyoutherightanswerforeachofyourpatients.
Thedatatellsyouthispatienthasamalignanttumor;thispatienthasabenigntumor.
Soithadtherightanswers,andyouwantedthealgorithmtojustproducemoreofthesame.
Incontrast,inanunsupervisedlearningproblem,thisisthesortofdatayouget,okayWherespeakingloosely,you'regivenadataset,andI'mnotgonnatellyouwhattherightanswerisonanyofyourdata.
I'mjustgonnagiveyouadatasetandI'mgonnasay,"Wouldyoupleasefindinterestingstructureinthisdataset"Sothat'stheunsupervisedlearningproblemwhereyou'resortofnotgiventherightanswerforeverything.
So,forexample,analgorithmmayfindstructureinthedataintheformofthedatabeingpartitionedintotwoclusters,orclusteringissortofoneexampleofanunsupervisedlearningproblem.
SoIhopeyoucanseethis.
Itturnsoutthatthesesortofunsupervisedlearningalgorithmsarealsousedinmanyproblems.
Thisisascreenshot—thisisapictureIgotfromSueEmvee,who'saPhDstudenthere,whoisapplyingunsupervisedlearningalgorithmstotrytounderstandgenedata,soistryingtolookatgenesasindividualsandgroupthemintoclustersbasedonpropertiesofwhatgenestheyrespondto—basedonpropertiesofhowthegenesrespondtodifferentexperiments.
Anotherinterestingapplicationof[inaudible]sortsofclusteringalgorithmsisactuallyimageprocessing,thiswhichIgotfromSteveGules,who'sanotherPhDstudent.
Itturnsoutwhatyoucandoisifyougivethissortofdata,sayanimage,tocertainunsupervisedlearningalgorithms,theywillthenlearntogrouppixelstogetherandsay,gee,thissortofpixelseemstobelongtogether,andthatsortofpixelseemstobelongtogether.
Andsotheimagesyouseeonthebottom—Iguessyoucanjustbarelyseethemonthere—sotheimagesyouseeonthebottomaregroupings—arewhatthealgorithmhasdonetogroupcertainpixelstogether.
Onasmalldisplay,itmightbeeasiertojustlookattheimageontheright.
Thetwoimagesonthebottomaretwosortofidenticalvisualizationsofthesamegroupingofthepixelsinto[inaudible]regions.
Andsoitturnsoutthatthissortofclusteringalgorithmorthissortofunsupervisedlearningalgorithm,whichlearnstogrouppixelstogether,itturnsouttobeusefulformanyapplicationsinvision,incomputervisionimageprocessing.
I'lljustshowyouoneexample,andthisisarathercoolonethattwostudents,AshutoshSaxenaandMinSunheredid,whichisgivenanimagelikethis,rightThisisactuallyapicturetakenoftheStanfordcampus.
Youcanapplythatsortofclusteringalgorithmandgroupthepictureintoregions.
Letmeactuallyblowthatupsothatyoucanseeitmoreclearly.
Okay.
Sointhemiddle,youseethelinessortofgroupingtheimagetogether,groupingtheimageinto[inaudible]regions.
AndwhatAshutoshandMindidwastheythenappliedthelearningalgorithmtosaycanwetakethisclusteringanduseittobuilda3DmodeloftheworldAndsousingtheclustering,theythenhadalearningalgorithmtrytolearnwhatthe3Dstructureoftheworldlookslikesothattheycouldcomeupwitha3Dmodelthatyoucansortofflythrough,okayAlthoughmanypeopleusedtothinkit'snotpossibletotakeasingleimageandbuilda3Dmodel,butusingalearningalgorithmandthatsortofclusteringalgorithmisthefirststep.
Theywereableto.
I'lljustshowyouonemoreexample.
Ilikethisbecauseit'sapictureofStanfordwithourbeautifulStanfordcampus.
Soagain,takingthesamesortofclusteringalgorithms,takingthesamesortofunsupervisedlearningalgorithm,youcangroupthepixelsintodifferentregions.
Andusingthatasapre-processingstep,theyeventuallybuiltthissortof3DmodelofStanfordcampusinasinglepicture.
Youcansortofwalkintotheceiling,lookaroundthecampus.
OkayThisactuallyturnedouttobeamixofsupervisedandunsupervisedlearning,buttheunsupervisedlearning,thissortofclusteringwasthefirststep.
Soitturnsoutthesesortsofunsupervised—clusteringalgorithmsareactuallyroutinelyusedformanydifferentproblems,thingslikeorganizingcomputingclusters,socialnetworkanalysis,marketsegmentation,soifyou'reamarketerandyouwanttodivideyourmarketintodifferentsegmentsordifferentgroupsofpeopletomarkettothemseparately;evenforastronomicaldataanalysisandunderstandinghowgalaxiesareformed.
Thesearejustasortofsmallsampleoftheapplicationsofunsupervisedlearningalgorithmsandclusteringalgorithmsthatwe'lltalkaboutlaterinthisclass.
JustoneparticularlycoolexampleofanunsupervisedlearningalgorithmthatIwanttotellyouabout.
Andtomotivatethat,I'mgonnatellyouaboutwhat'scalledthecocktailpartyproblem,whichisimaginethatyou'reatsomecocktailpartyandtherearelotsofpeoplestandingallover.
Andyouknowhowitis,right,ifyou'reatalargeparty,everyone'stalking,itcanbesometimesveryhardtoheareventhepersoninfrontofyou.
Soimaginealargecocktailpartywithlotsofpeople.
Sotheproblemis,isthatallofthesepeopletalking,canyouseparateoutthevoiceofjustthepersonyou'reinterestedintalkingtowithallthisloudbackgroundnoiseSoI'llshowyouaspecificexampleinasecond,buthere'sacocktailpartythat'sIguessrathersparselyattendedbyjusttwopeople.
Butwhatwe'regonnadoiswe'llputtwomicrophonesintheroom,okayAndsobecausethemicrophonesarejustatslightlydifferentdistancestothetwopeople,andthetwopeoplemayspeakinslightlydifferentvolumes,eachmicrophonewillpickupanoverlappingcombinationofthesetwopeople'svoices,soslightlydifferentoverlappingvoices.
SoSpeaker1'svoicemaybemoreloudonMicrophone1,andSpeaker2'svoicemaybelouderonMicrophone2,whatever.
Butthequestionis,giventhesemicrophonerecordings,canyouseparateouttheoriginalspeaker'svoicesSoI'mgonnaplaysomeaudioclipsthatwerecollectedbyTaiYuanLeeatUCSD.
I'mgonnaactuallyplayforyoutheoriginalrawmicrophonerecordingsfromthiscocktailparty.
SothisistheMicrophone1:Microphone1:One,two,three,four,five,six,seven,eight,nine,ten.
Microphone2:Uno,dos,tres,cuatro,cinco,seis,siete,ocho,nueve,diez.
Instructor(AndrewNg):Soit'safascinatingcocktailpartywithpeoplecountingfromonetoten.
Thisisthesecondmicrophone:Microphone1:One,two,three,four,five,six,seven,eight,nine,ten.
Microphone2:Uno,dos,tres,cuatro,cinco,seis,siete,ocho,nueve,diez.
Instructor(AndrewNg):Okay.
Soinsupervisedlearning,wedon'tknowwhattherightansweris,rightSowhatwe'regoingtodoistakeexactlythetwomicrophonerecordingsyoujustheardandgiveittoanunsupervisedlearningalgorithmandtellthealgorithmwhichofthesediscoverstructureinthedata[inaudible]orwhatstructureisthereinthisdataAndweactuallydon'tknowwhattherightanswerisoffhand.
Sogivethisdatatoanunsupervisedlearningalgorithm,andwhatthealgorithmdoesinthiscase,itwilldiscoverthatthisdatacanactuallybeexplainedbytwoindependentspeakersspeakingatthesametime,anditcanfurtherseparateoutthetwospeakersforyou.
Sohere'sOutput1ofthealgorithm:Microphone1:One,two,three,four,five,six,seven,eight,nine,ten.
Instructor(AndrewNg):Andthere'sthesecondalgorithm:Microphone2:Uno,dos,tres,cuatro,cinco,seis,siete,ocho,nueve,diez.
Instructor(AndrewNg):Andsothealgorithmdiscoversthat,gee,thestructureunderlyingthedataisreallythattherearetwosourcesofsound,andheretheyare.
I'llshowyouonemoreexample.
Thisisa,well,thisisasecondsortofdifferentpairofmicrophonerecordings:Microphone1:One,two,three,four,five,six,seven,eight,nine,ten.
Microphone2:[Musicplaying.
]Instructor(AndrewNg):Sothepoorguyisnotatacocktailparty.
He'stalkingtohisradio.
There'sthesecondrecording:Microphone1:One,two,three,four,five,six,seven,eight,nine,ten.
Microphone2:[Musicplaying.
]Instructor(AndrewNg):Right.
Andwegetthisdata.
It'sthesameunsupervisedlearningalgorithm.
Thealgorithmisactuallycalledindependentcomponentanalysis,andlaterinthisquarter,you'llseewhy.
Andthenoutput'sthefollowing:Microphone1:One,two,three,four,five,six,seven,eight,nine,ten.
Instructor(AndrewNg):Andthat'sthesecondone:Microphone2:[Musicplaying.
]Instructor(AndrewNg):Okay.
Soitturnsoutthatbeyondsolvingthecocktailpartyalgorithm,thisspecificclassofunsupervisedlearningalgorithmsarealsoappliedtoabunchofotherproblems,likeintextprocessingorunderstandingfunctionalgradingandmachinedata,likethemagneto-encephalogramwouldbeanEEGdata.
We'lltalkaboutthatmorewhenwegoanddescribeICAorindependentcomponentanalysisalgorithms,whichiswhatyoujustsaw.
Andasanaside,thisalgorithmIjustshowedyou,itseemslikeitmustbeaprettycomplicatedalgorithm,right,totakethisoverlappingaudiostreamsandseparatethemout.
Itsoundslikeaprettycomplicatedthingtodo.
Soyou'regonnaaskhowcomplicatedisitreallytoimplementanalgorithmlikethisItturnsoutifyoudoitinMATLAB,youcandoitinonelineofcode.
SoIgotthisfromSamuelWyseatToronto,UofToronto,andtheexampleIshowedyouactuallyusedamorecomplicatedICAalgorithmthanthis.
Butnonetheless,IguessthisiswhyforthisclassI'mgoingtoaskyoutodomostofyourprogramminginMATLABandOctavebecauseifyoutrytoimplementthesamealgorithminCorJavaorsomething,Icantellyoufrompersonal,painfulexperience,youendupwritingpagesandpagesofcoderatherthanrelativelyfewlinesofcode.
I'llalsomentionthatitdidtakeresearchersmany,manyyearstocomeupwiththatonelineofcode,sothisisnoteasy.
Sothatwasunsupervisedlearning,andthenthelastofthefourmajortopicsIwannatellyouaboutisreinforcementlearning.
Andthisreferstoproblemswhereyoudon'tdoone-shotdecision-making.
So,forexample,inthesupervisedlearningcancerpredictionproblem,youhaveapatientcomein,youpredictthatthecancerismalignantorbenign.
Andthenbasedonyourprediction,maybethepatientlivesordies,andthenthat'sit,rightSoyoumakeadecisionandthenthere'saconsequence.
Youeithergotitrightorwrong.
Inreinforcementlearningproblems,youareusuallyaskedtomakeasequenceofdecisionsovertime.
So,forexample,thisissomethingthatmystudentsandIworkon.
IfIgiveyouthekeystoanautonomoushelicopter—weactuallyhavethishelicopterhereatStanford,—howdoyouwriteaprogramtomakeitfly,rightYounoticethatifyoumakeawrongdecisiononahelicopter,theconsequenceofcrashingitmaynothappenuntilmuchlater.
Andinfact,usuallyyouneedtomakeawholesequenceofbaddecisionstocrashahelicopter.
Butconversely,youalsoneedtomakeawholesequenceofgooddecisionsinordertoflyahelicopterreallywell.
SoI'mgonnashowyousomefunvideosoflearningalgorithmsflyinghelicopters.
ThisisavideoofourhelicopteratStanfordflyingusingacontrollerthatwaslearnedusingareinforcementlearningalgorithm.
SothiswasdoneontheStanfordfootballfield,andwe'llzoomoutthecamerainasecond.
You'llsortofseethetreesplantedinthesky.
Somaybethisisoneofthemostdifficultaerobaticmaneuversflownonanyhelicopterundercomputercontrol.
Andthiscontroller,whichisvery,veryhardforahumantositdownandwriteout,waslearnedusingoneofthesereinforcementlearningalgorithms.
Justawordaboutthat:Thebasicideabehindareinforcementlearningalgorithmisthisideaofwhat'scalledarewardfunction.
Whatwehavetothinkaboutisimagineyou'retryingtotrainadog.
Soeverytimeyourdogdoessomethinggood,yousay,"Gooddog,"andyourewardthedog.
Everytimeyourdogdoessomethingbad,yougo,"Baddog,"rightAndhopefully,overtime,yourdogwilllearntodotherightthingstogetmoreofthepositiverewards,togetmoreofthe"Gooddogs"andtogetfewerofthe"Baddogs.
"Sothewayweteachahelicoptertoflyoranyoftheserobotsissortofthesamething.
Everytimethehelicoptercrashes,wego,"Badhelicopter,"andeverytimeitdoestherightthing,wego,"Goodhelicopter,"andovertimeitlearnshowtocontrolitselfsoastogetmoreofthesepositiverewards.
Soreinforcementlearningis—Ithinkofitasawayforyoutospecifywhatyouwantdone,soyouhavetospecifywhatisa"gooddog"andwhatisa"baddog"behavior.
Andthenit'suptothelearningalgorithmtofigureouthowtomaximizethe"gooddog"rewardsignalsandminimizethe"baddog"punishments.
Soitturnsoutreinforcementlearningisappliedtootherproblemsinrobotics.
It'sappliedtothingsinwebcrawlingandsoon.
Butit'sjustcooltoshowvideos,soletmejustshowabunchofthem.
ThislearningalgorithmwasactuallyimplementedbyourheadTA,Zico,ofprogrammingafour-leggeddog.
IguessSamShriverinthisclassalsoworkedontheprojectandPeterRenfrewandMikeandafewothers.
ButIguessthisreallyisagooddog/baddogsinceit'sarobotdog.
Thesecondvideoontheright,someofthestudents,IguessPeter,Zico,Toncaworkingonaroboticsnake,againusinglearningalgorithmstoteachasnakerobottoclimboverobstacles.
Belowthat,thisiskindofafunexample.
AshutoshSaxenaandJeffMichaelsusedlearningalgorithmstoteachacarhowtodriveatreasonablyhighspeedsoffroadsavoidingobstacles.
Andonthelowerright,that'sarobotprogrammedbyPhDstudentEvaRoshentoteachasortofsomewhatstrangelyconfiguredrobothowtogetontopofanobstacle,howtogetoveranobstacle.
Sorry.
Iknowthevideo'skindofsmall.
Ihopeyoucansortofseeit.
OkaySoIthinkallofthesearerobotsthatIthinkareverydifficulttohand-codeacontrollerforbylearningthesesortsoflearningalgorithms.
Youcaninrelativelyshortordergetarobottodooftenprettyamazingthings.
Okay.
SothatwasmostofwhatIwantedtosaytoday.
Justacouplemorelastthings,butletmejustcheckwhatquestionsyouhaverightnow.
Soiftherearenoquestions,I'lljustclosewithtworeminders,whichareafterclasstodayorasyoustarttotalkwithotherpeopleinthisclass,Ijustencourageyouagaintostarttoformprojectpartners,totrytofindprojectpartnerstodoyourprojectwith.
Andalso,thisisagoodtimetostartformingstudygroups,soeithertalktoyourfriendsorpostinthenewsgroup,butwejustencourageyoutotrytostarttodobothofthosetoday,okayFormstudygroups,andtrytofindtwootherprojectpartners.
Sothankyou.
I'mlookingforwardtoteachingthisclass,andI'llseeyouinacoupleofdays.
[EndofAudio]Duration:69minutes