neutralletgo

PaperDeepfakes–anemergingrisktoindividualsandsocietiesalikebyTündeFaragó(TilburgUniversity)December2019ThisworkislicensedunderaCreativeCommonsAttribution-NoDerivatives4.
0InternationalLicense.
Toviewacopyofthislicense,visithttp://creativecommons.
org/licenses/by-nd/4.
0/Deepfakes–anemergingrisktoindividualsandsocietiesalikeMAThesisNameofauthor:TündeFaragóStudentnumber:2031520MAtrack:OnlineCultureMajor:Art,MediaandSocietyDepartmentofCultureStudiesSchoolofHumanitiesandDigitalSciencesDate:August2019Supervisor:dr.
PiiaVarisSecondreader:prof.
dr.
JanBlommaertDEEPFAKES–ANEMERGINGRISK2Contents1.
Introduction32.
Framework.
62.
1.
Post-truth.
62.
2.
Deepfakesandmoralpanic123.
Methodology.
184.
Casestudyanalyses194.
1.
Deepfakesandtheirpotentialtoharmindividuals.
194.
2.
Deepfakesandtheirpotentialtoharmsocieties.
305.
Conclusion.
446.
References53DEEPFAKES–ANEMERGINGRISK31.
IntroductionDigitalizationandtheriseoftheinternethavebroughtaboutarevolutioninthewaypeoplecanaccessinformationandshareitwithothers.
Italsoresultedincitizensthemselvesbecomingproducersofknowledge,aprivilegethatoncemorestrictlybelongedonlytogovernments,mediaoutletsandtheacademia.
(HanellandSal,2017:pp.
155-156)Butwithtraditionalinstitutionslosingauthorityoverfacts,fakenews,misinformationandlieshavebecomeacommonplace,sideliningthetruthintheprocess.
(Llorente,2017:p.
9)Althoughfakenews,propagandaandliesaretalesasoldastime,itwasn'tuntiltheyear2016whenmajoronlinemisinformationcampaignsthatseemtohavecontributedtoDonaldTrumpbecomingthepresidentoftheUnitedStatesandBritishpeoplevotingtoleavetheEuropeanUnion,thatthesephenomenabecameacenterofattentioninWesternsocieties.
These,formanyshockingevents,resultedinOxfordDictionariestoname"post-truth"thewordoftheyearin2016.
(OxfordDictionaries,n.
d.
)Thispost-trutherathatweliveinnowissaidtobeguidedbyouremotionsasobjectivefactsandthetrutharecherry-pickedandshapedtofitourownversionsofreality.
(McIntyre,2018:chapter1,paragraph12)Buthumanity'sabilitytotwisttherealityhastakenanevenabiggerstepforwardinrecentyearswiththecreationofdeepfaketechnology.
Withthistechnologyonecancreatedeepfakes:videosofrealpeoplesayinganddoingthingstheyhaveneversaidordoneinreallife.
(ChesneyandCitron,2018:n.
p.
)In2017thefirstdeepfakesemergedinwhichfacesofyoungHollywoodactresses,suchasScarletJohanssonandothers,weresuperimposedintopornmovies.
Thetechnologythatwasonceonlyinthehandsofexpertswassoondemocratizedandwebpagesandfreeappspromisingtocreatevideoforgeriesofcelebrities,ex-girlfriends,colleaguesandevenstrangerspoppedupallaroundtheworld.
Thishasledtoasituationwheresuddenlyeverybodycouldfallvictimtoadeepfake.
(Schwartz,2018)Moreover,themachinelearningabilityofArtificialIntelligence,whichwasDEEPFAKES–ANEMERGINGRISK4usedtocreatedeepfakes,isdevelopingsofastthatitbecomesveryhardforsoftware,nottomentionareveryownhumansenses,todetecttheforgeries.
(ChesneyandCitron,2018:n.
p.
)Andexactlythisinabilityofpeopletotellthemapartfromrealityiswheretheirtruedangerlies.
Sincedeepfakesarearelativelynewthing,besidesthereportdonebyLawprofessorsBobbyChesneyandDanielleCitronin2018abouttheloomingchallengesofdeepfakes,therehavenotbeenanyothermajorstudiesdealingwiththisphenomenon.
(ChesneyandCitron,2018)Therefore,thisthesisrepresentsoneofthefirstattemptstoexploretheirpotentialsocial,culturalandpoliticalimplicationsthroughtheanalysesofafewcarefullyselectedcasestudies.
Theaimofthisthesisistoanalyzethesocial,culturalandpoliticalramificationsofdeepfakesinrelationtoindividualsandsocietiesalike.
Thefirsttwochaptersprovideatheoreticalbackground,analysesofcurrentsocietaldevelopmentsandexplanationofthedeepfaketechnology.
Inthethirdchapteranexplanationwillbegivenaboutthemethodologychosentoanalyzethephenomenonofdeepfakes.
Thefollowingchapterswillprovideananalysisofthecasesutilizingtheaforementionedframework.
Thecasestudieschosenforthisthesisrepresentexampleswheredeepfakeshadsomeseriousconsequencesonindividualsandsocieties.
Forthechapteraboutthepotentialofdeepfakestoharmindividuals,tworecentcasestudieswerechosenwherethevictimssufferedpsychologicalandreputationaldamagescausedbyforgedvideos.
Thefirstcasestudyfrom2018isaboutayoungMuslimfreelancejournalistfromIndia,RanaAyyub,whowasharassedandbulliedbyapornographicdeepfakeintendedtosilenceheractivism.
Thesecondcaseisfrom2012andfocusesonanAustralianwoman,calledNoelleMartin,whosepicturesweredoctoredintoadultmoviesandsexuallyexplicitimagesanduploadedontofamouspornwebsites.
Theseparticularcaseswerechosenbecausethereisplentyofinformationavailableaboutthemandatthesametime,theyprovidegoodinsightsDEEPFAKES–ANEMERGINGRISK5tothevariousconsequencesdeepfakesmighthaveonindividuals.
Ontheotherhand,toanalyzewhatkindofpotentialrisksdeepfakesposeforsocieties,aBelgiandeepfakeexamplewaschosenwhereapoliticalpartycommissionedadeepfakeofDonaldTrumpinordertoshapethepublic'sopinionaboutapoliticalmatter.
Althoughthisspecificcasewasonlysmall-scale,neverthelessitshowedthepotentialcapacityofsuchtechnologytocausesomeseriousdisruptioninasociety.
Sincetherehasnotbeenanyothermajorincidentwhichinvolvesdeepfakes,otherfurtherexamplesofdoctoredvideos,fakenewsandmisinformationcampaignsarealsousedtoexplainastowhatwemightexpectwhenitcomestodeepfakes.
Thefinalchaptersummarizesthecentralpointsoftheanalysesoftheabove-mentionedcasestudiesandatthesametimeprovidessomesuggestionsforfuturestudiesinthisfield.
DEEPFAKES–ANEMERGINGRISK62.
Framework2.
1.
Post-truthInMarch2018followingthehorrificParklandhighschoolshootingintheU.
S.
,aGIFofoneoftheshootingsurvivors,EmmaGonzález,tearingupacopyoftheU.
S.
Constitutionstartedcirculatingonsocialmedia.
Eventhoughtheanimationwasfakeitstillstirredupalotofnegativeemotions,especiallyamongconservativesupporters.
(Mezzofiore,2018)GonzálezfirstbecameknowntothepublicasshegaveapassionatespeechfollowedbyaprolongedmomentofsilenceataguncontrolrallyinWashingtononlyfourdaysafteragunmankilled17peopleatherschoolon14thofFebruary2018.
Injustafewweeksshebecameoneofthefacesofthecampaign#NeverAgainwhichattractedalotofsupportallacrossAmerica.
(Mikkelson,2018;Mezzofiore,2018)However,justhoursaftertheConstitutionrippingGIFwentviral,manythatpreviouslyhadsupportedhernowturnedagainsther.
ShortlyafterthefakeGIFappearedontheinternet,multiplesourcespointedoutthatitwasdoctored;however,itdidnotstopitfromspreadingonsocialmediaplatformsandwebsiteslikeawildfire.
TheoriginalvideodepictedGonzálezrippingintwoashootingtargetaspartoftheTeenVoguestoryaboutthesurvivorsreleasedjustdaysbeforethefakeGIFstartedcirculatingtheinternet.
(Mezzofiore,2018)SoonafterthefakeGIFappeared,theshootingsurvivorandvocalactivistquicklybecameatargetofsocialmediaattacks,manycallingheratraitor.
JesseHughes,thefrontmanofthebandEaglesofDeathMetal,wasamongthefirstwhoaccusedGonzálezinaTweetfortreason.
(Snapes,2018)Prominentright-wingfigures,suchasthefamousactor,AdamBaldwin,whoisknownforhisconservativecomments,immediatelyjumpedtothechancetosharetheimageonTwitter.
AlthoughhelaterdeletedhisTwitterposthestilldefendedthefakeGIFas"apoliticalsatire".
(Mezzofiore,2018)DEEPFAKES–ANEMERGINGRISK7Themoralofthisstoryisclear:thetruthbecomesirrelevantintheheatofthemomentwhilefeelingsandopinionsdictatetheperspectiveonreality.
(Llorente,2017:p.
9)As"[e]xpertsandfactsnolongerseemcapableofsettlingargumentstotheextentthattheyoncedid"(Davies,2018:p.
xiv),wenowliveinwhatmanycallthepost-truthera,"whereobjectivityandrationalitygivewaytoemotions,ortoawillingnesstoupholdbeliefseventhoughthefactsshowotherwise.
"(Llorente,2017:p.
9)In2016OxfordDictionarynamed"post-truth"asthewordoftheyearfollowingcontroversialeventsandsurprisesonthepoliticalstagelikeDonaldTrumpwinningtheU.
S.
presidentialelectionsorBritain'sdecisiontoleavetheEU.
(Llorente,2017:p.
9)Theydefinedthistermas"[r]elatingtoordenotingcircumstancesinwhichobjectivefactsarelessinfluentialinshapingpublicopinionthanappealstoemotionandpersonalbelief.
"(OxfordDictionaries,n.
d.
)However,thisdoesnotmeanthatwearepastthetruthorthatbeforewehadmoretruthfulperiods;whatthisdefinitionreferstoisthatweliveinanerathatisindifferenttothetruth.
Thisisnottosaythatpeopledonotcareaboutthetruthanymore;itonlymeansthatthetruthbecomesamatterofopinion.
Whateverisinlinewithpeople'spersonalbeliefswillmorelikelybeconsideredtobetrue,regardlessoftheobjectivefacts.
ThisistheerawherefakenewsanddoctoredGIFshavebecomecommonplace,sideliningtheobjectivefactsandexperts.
Andveryrecentlyfakevideos,calleddeepfakes,emergedpromisingnewwaysinwhichrealitycanbebentandalteredtofavorsomeopinionsoverothers.
Inthelightofallthis,"[t]hevalueorcredibilityofthemediahassomewhatfadedincomparisontopersonalopinions.
Thefactsthemselvestakesecondplace,while'how'astoryistoldtakesprecedenceover'what'.
Itisthereforenotaboutwhathashappened,butratheraboutlistening,seeingandreadingtheversionoffactswhichmorecloselyfitswitheachperson'sideology.
"(Llorente,2017:p.
9)DEEPFAKES–ANEMERGINGRISK8AccordingtoDavies(2018:p.
7),appealingtopeoplewithobjectivefactsorexpertopinionsrarelytriggersthemtoactinacertainway.
Instead,peopleareeasilysweptawayifoneplaysontheiremotions.
Therefore,imagesorstorieswhichchangethewaypeoplefeelwillbeexchangedveryquicklyamongpeoplewhosharethesamesentiments.
Apublicthatissusceptibletoemotionalappealsisstirredandshapedquiteeasilybyfakenews,rumorsorlies.
However,weneedtobecarefulwhenwetalkaboutthepost-truthandlies.
Thesetwomaybeconnectedbuttheyarenotthesame;post-truthcanbemoreaccuratelyunderstoodastherelativizationofthetruth.
(Zarzalejos,2017:p.
11)AccordingtoMcIntyrethewordpost-truthis"irreduciblynormative"(McIntyre,2018:chapter1,paragraph6).
"[T]herearemanywaysonecanfitunderneaththepost-truthumbrella.
"(McIntyre,2018:chapter1,paragraph11)Onecanbewillfullyindifferenttothetruth,notknowinglyspreadalie,orevenintentionallylieandfalsifysomething.
(Ibid:chapter1,paragraph8-10)However,thisisnotsomethingnew.
Whatisnewaboutthepost-trutheraisthe"existenceofrealityitself".
(Ibid:chapter1,paragraph11)Thereasonforthisissimple:ifthemajorityofsocietyoritsleadersareindisputeoverthebasicfacts,ourwholedemocraticsystemsareatrisk.
(Ibid)AsMcIntyre(2018:chapter1,paragraph12)pointsout,"therealproblemhere…isnotmerelythecontentofanyparticular(outrageous)belief,buttheoverarchingideathat–dependingonwhatonewantstobetrue–somefactsmattermorethanothers.
"Thismeansthatformanypeoplewhatcorrespondswiththeirbeliefandopinionsistrue.
Thefactsareinthissensenotdisregarded;theyarejustcherry-pickedandshapedtofitaperson'sbeliefaboutreality.
Andthisisatthecoreoftheproblem,sinceitdirectly"underminestheideathatsomethingsaretrueirrespectiveofhowwefeelaboutthem.
"(Ibid,emphasisoriginal)ForinstancetheideathattheEarthisnotflat,orthatourplanetrevolvesaroundtheSun:thesescientificfactswillstillbetrueevenifconspiracytheoristsdonotbelieveinthem.
However,DEEPFAKES–ANEMERGINGRISK9whenthemajorityofthepopulationbelievesinconspiracytheoriesorfakenewsthenitbecomesaseriousthreattothesharedunderstandingofthenatureofreality.
Wethusliveinaworldwhich"revolvesaroundpassionsandbeliefs;wheretruthisnolongerneeded"(Madeiros,2017:p.
24),becauseeachpersonhashisorherownversionofthetruththatcorrespondstotheirpersonalideology.
Andwhenfactsareshapedanddictatedbyone'spoliticalideologythenwehaveaclearrecipeforpoliticaldominance.
(McIntyre,2018:chapter1,paragraph13)Thisisalsonotanewphenomenonassuch.
Whatispost-truthtodaywascalledpropagandainothertimes.
(Medrán,2017:p.
33)Peoplewhoparticipatedinthepowerstructuresofasocietywereabletoproduceandlegitimizecertaintypesofknowledgeandintroducethemascredible.
(HanellandSal,2017:p.
155)"Knowledge,thus,isastratifiedconstruct,interest-ladenandhistoricallysaturatedwithpower,and…somepeoplehavemorepowertospeakaboutsomesubjectsthanothers.
"(Ibid)Accordingtothis,notalltypesofknowledgeareequallycredibleorequallyvisibleforthatmatter.
However,whatisdifferentnowadaysisthateverybodycanhaveaccesstoallkindsofdifferentsourcesofinformation,createtheirownfactsandsharethemonline.
Withtheriseoftheinternet,theauthorityovertheproductionofknowledgeanditslegitimizationnolongerisinthehandsofthestate,theacademiaortraditionalmedia.
Consideringthefact,thatnowadaysanyonecanproduceandshareideasonline,knowledgeofdifferentordersofvisibilityisintertwinedontheinternet.
(Ibid:pp.
155-156)"Today,accesstoinformativecontent,aswellasitsimmediacyandvolume,hasnoprecedents.
Theimpactofdigitalizationintheworldofcommunicationshasbroughtaboutarevolutioninawaythatpeoplecanproduceinformationthemselves…Similarly,ithaschangedthewaytheyconsumeandassimilateit.
"(Gooch,2017:p.
14)DEEPFAKES–ANEMERGINGRISK10Atthesametime,whileeverybodycanclaimthatwhattheyknowistrue,thetrustinexpertopinionhasdrasticallydeclined.
"Journalists,judges,expertsandvariousother'elites'areunderfiretoday.
"(Davies,2018:p.
26)Thisiswhathappenswhentheworldgetstakenoverbyfeelings.
(Ibid:pp.
xvi-xvii)Whiledigitalizationallowedpeopletoaccess,produceandshareallkindsofinformationonline,ithasalsocontributedtothecreationoffilterbubbles:places"wherewefindonlythenewsweexpectandthepoliticalperspectiveswealreadyholddear.
"(Gillespie,2014:p.
188)ChesneyandCitron(2018:p.
13)arguethatevenwithouttechnology,peopletendtosurroundthemselveswithideasthatgohandinhandwiththeirownpersonalbeliefs.
Socialmediaplatformsfurtherexacerbatethistendencybyallowingtheiruserstocreateandsharecontenttotheirownliking.
(Ibid)Moreover,"[p]latforms'algorithmshighlightpopularinformation,especiallyifithasbeensharedbyfriends,surroundinguswithcontentfromrelativelyhomogenousgroups.
Asendorsementsandsharesaccumulate,thechancesforanalgorithmicboostincreases.
Afterseeingfriends'recommendationsonline,individualstendtosharethemwiththeirnetworks.
"(Ibid)Thus,informationthatissharedmoreoftenbyusersbecomesmorevisibleandatthesametimeisconsideredmorecredible.
WhatIalreadymentionedaboveregardingthestatusoffactsorthetruthinthe'post-truth'eraalsonowadaysgoesforinstancepolitics.
Thatis,itdoesnotnecessarilymeanthatthefactsorthetrutharecompletelydisregarded,"butinsteadaconvictionthatfactscanalwaysbeshaded,selected,andpresentedwithinapoliticalcontextthatfavorsoneinterpretationoftruthoveranother.
"(McIntyre,2018:chapter1,paragraph4)ThisisveryevidentwhenwetakealookattheTrumpadministration,whichisregardedasthefirsteverDEEPFAKES–ANEMERGINGRISK11Twitter-basedpresidencyintheworld.
(Hollinger,2017)Trumpandhisassociatesveryoftenplaywithfactsor"alternativefacts",astheysometimesrefertoit,topresenttheirownversionofevents.
(McIntyre,2018:chapter1,paragraph4)Forinstance,backinJanuary2017Trump'steamdefendedhispresssecretary'scommentsabouthowmanypeopleattendedtheinauguration,sayingthatthesecretarydidnotliebutoffered"alternativefacts".
(Swaine,2017)Itseemsthat"[i]tdoesn'tmatterif[Trump's]commentsaretrue--andmultiplefact-checkingsiteslikePolitiFact,FactCheck.
organdtheWashingtonPost'sFactCheckerbloghaveshownthatmanyoftheassertionshetweetsarefalse.
Trump's140-characteroutburstsarejustwhatmanyamonghis41.
5milliononlinefollowerswanttohear.
"(Collins,2018)However,thisatmosphereisnotonlyU.
S.
-specific;itispresentallovertheworld.
Traditionalmediaisbenchedwhilealternativemediaisseenasmoretrustworthy.
Thisisnosurpriseas,asitwassaidbefore,traditionalmediaandexpertshavelosttheircredibilityinthepublic'seye.
AndtheU.
S.
presidentcallingcertainmediahousesthecreatorsof"fakenews"and"thetrueenemyofthepeople"justaddsadditionalfueltothefire.
(Bell,2019;Wagner,2018)Insuchaclimatepeoplewillturntosomeonetheycantrustthemost–tothemselvesandtheirownconvictions.
Itisnotsurprisingperhapsthattheystopquestioningthecredibilityofthefactspresentedtothem.
Afterall,everybodythinksofthemselvesandtheirfriendsastrustworthysources.
(Sundar,2016)Thismakespeopleveryvulnerabletofakenews,liesandpropaganda.
Intheabsenceofan"institutiontoestablishfilters,separatethewheatfromthechaffandputdifferentviewsintoperspective"allweareleftwithisourownbeliefsandopinionstoguideus.
(Madeiros,2017:p.
23)Veryoftenthismeansthatpeoplewilltrytofittheworldintotheirownperspectiveandnottheotherwayaround.
DEEPFAKES–ANEMERGINGRISK12Inhearingwhattheywanttohear,whatmovespeoplemorethananythingisagoodjuicystory.
Objectivefactsandwellresearchedtopicsmayfallondeafears,butarumororahoaxthathasanemotionalappealandconfirmsexistingbiaseswillbesharedamonglike-mindedpeople.
Andifafamouspersonwithabiggersocialmediavisibilitycirculatessuchastory,likeTrumporAdamBaldwin,itwillbesharedinstantlyamongmillionsoffollowers,increasingthestory'scredibilityaswellintheprocess.
(Sundar,2016)However,itisnotthatwearesogullibleastobelieveeverythingwearepresentedwith;weareratherjustincapableofadmittingthatwearewrong.
Whenpeopleareconfrontedwithatruththatcontradictstheirbeliefsitcreatesapsychologicaltension.
Ifapersonisconfrontedwithevidencethatproveshisorherbeliefiswrong,thelogicalreactionwouldbetochangeone'smistakenbelief.
Butthisisnotwhathappensinreality.
Peoplearereluctanttoletgooftheirownconfirmationalbiasesandveryoftenwhenconfrontedwithcontradictingevidencetheyarenotthinkingrationally.
(McIntyre,2018:chapter3,paragraph2-3)Inpoliticsforinstancethisplaysoutbypickingasideandstickingtoitevenifthefactssayotherwise.
Suchapoliticalclimatewherepersonaltruthsmattermorethanobjectiveonescreatesadangertoasocietyasawhole.
Howdoesanemotionallysusceptiblesocietyevenstandachanceagainstdeepfakes:fakevideosthatarecreatedtolookrealbutareactuallynot2.
2.
DeepfakesandmoralpanicUntilrecentlythetechnologyofdeepfakeswaslimitedmostlytothecommunitydevelopingArtificialIntelligence;however,in2017aReddituserknownbyhisusername"Deepfakes"developedthesametechnologywiththehelpofGoogle'sfreeopensourcemachinelearningsoftwarecalledTensorFlow.
Heusedhiscreationtoputcelebrityfacesontothebodiesofactressesinpornmovies.
Anumberofmediaoutletsreportedaboutthesefakepornmovies,callingthem"deepfakes".
AlthoughRedditbannedthevideosfortheirviolationofprivacyDEEPFAKES–ANEMERGINGRISK13andinvoluntarypornography,thedamagewasalreadydone.
ThecreatorofthedeepfakessoonreleasedafreeapplicationcalledFakeAppwhichallowedanyonewhohadaccesstotheinternetandphotosofpeopletocreatedeepfakesforallkindsofuses.
(Schwartz,2018)Programsandappswiththeabilitytoalterandtweakimages,audiosandvideosarenotanewphenomenonassuch.
Inourdailylives,manyofususefilterstosmoothouttheimperfectionsofpicturesorvideosweintendtoposttooursocialmediaaccounts.
Someofthesetoolsarequiteimpressive,yetpeoplecanusuallytellifanimageofapersonisalteredornot.
Andwhatourhumaneyesmiss,forensictechnologiesarestillabletodetect.
However,deepfaketechnologyisexpectedtorevolutionizeallthis.
HeavilyrelyingonAItechnologies,deepfakesuseneuralnetworksandmorerecentlyGANtechnologies(generativeadversarialnetworks)tocreaterealisticforgeriesthatarenearlyimpossibletodetect.
Thesenetworksworkbymimickingthehumanbrain:"[m]uchasexperiencerefinesthebrain'sneuralnodes,examplestraintheneuralnetworksystem.
"(ChesneyandCitron,2018:p.
5)Withenoughexamplestheneuralnetworksareabletorecreateveryaccurateimpersonations.
TheGANtechnologyisevenmoresophisticated.
ItwascreatedbyIanGoodfellow,aresearcheratGoogle.
(Ibid)"AGANconsistsoftwoneuralnetworksplayingagamewitheachother.
Thediscriminatortriestodeterminewhetherinformationisrealorfake.
Theotherneuralnetwork,calledagenerator,triestocreatedatathatthediscriminatorthinksisreal.
"(Hergott,2019)Thediscriminatorisabletoveryaccuratelydeterminefakesbutgivenenoughtimeandpractice,thegeneratorcancreateforgeriesthatwillfooleventhediscriminatornetwork,nottomentionourhumansenses.
DEEPFAKES–ANEMERGINGRISK14Sincethe2017deepfakeRedditincidentthismind-bendingtechnologyhasbecomewidelyavailabletoeveryoneandmanydeepfakewebsitesandappshaveemerged,offeringtocreateforgedvideosofcelebrities,ex-girlfriendsoranyoneyoudesireforaslittleas20dollarsorsometimesevenfreeofcharge.
(Harwell,2018)Allyouneedtoprovideisafewhundredphotosofapersonorashortvideo.
(Solsman,2019a)Nowadaysitisofcoursealsoextremelyeasytofindphotosofpeopleonline,asafterallweliveourlivesmostlyinthepubliceye,bypostingregularlyonsocialmediaplatforms.
Also,withtherapidlyadvancingAItechnologies,nowevenjustahandfulofpictureswillbeenoughtomakeforgeries.
ThedeepfaketechnologydevelopedbySamsungresearchersin2019enablesitsuserstocreatevideoimpersonationsofanyonebasedonjustasinglepicture.
(Solsman,2019b)Andsincethistechnologyrequiresonlyoneimageofaperson,peoplewhodonotnecessarilyhavesomanypicturesofthemselvesonlinearealsoatriskofbecomingtargetsofdeepfakemanipulations.
Upuntilnow,thepublicperceptionofdeepfakeshasbeenmostlynegative.
Newsmediaoutletsandexpertssuperchargethisnegativitywitharticlesandreportswarningabouttheirpotentialtocreatenightmarishscenariosforbothindividualsandgovernments.
(ChesneyandCitron,2018;Harwell,2018;Schwartz,2018)Asboyd(2012)argues,"[m]oralpanicsemergewheneversomethingnewhappensthatdisruptsthesocialorderinawaythatmakespeopleanxiousandafraid.
"Thereisacertainamountofmoralpaniceverytimenewtechnologyappears(Ibid),anddeepfakesarenoexceptiontothis.
However,thisisnottosaythatdeepfaketechnologyisinherentlybad,farfromit.
AccordingtoKranzberg'slawweshouldnotlabelanytechnologyasgoodorbadorevenneutral:"[a]technology'svalueisshapedbyitssocialconstruction—howdesignerscreateitandhowpeopleuseit,interpretit,andreconfigureit.
Itisnotanoutcomeofthetechnologyaloneoritspotential.
"(boyd,2008:p.
12)HowacertaintechnologyisregardedorvaluedverymuchdependsonsocietalDEEPFAKES–ANEMERGINGRISK15developments.
Adoctoredvideotwodecadesago,whenYouTubewasinitsearlystages,wouldhavebeenregardedasafunprank,buttodaywhenpeopleareunabletodistinguishbetweenwhatisrealandwhatisfake,adecentdeepfakecanpassasarealthing.
Anditisnotonlybecausethetechnologyissoadvancedthatforgeriescaneasilyfoolpeople.
Nowadays,evenacrudelydoctoredvideowilldothat.
Forinstance,apoorlyalteredvideoofU.
S.
HouseSpeakerNancyPelositomakehersounddrunkgainedmorethan2millionviewsonthefirstdayitwaspostedonline.
1(Harwell,2019)Inthismediaera,fakenewscanveryquicklybecomewidelyknownandaccepted,duetoalgorithmicboostingandfilterbubbleswhichhelpspreadthiskindofcontentextremelyfast.
TheNancyPelosicasealsoshowsthatthethreatofdeepfakesisreal,especiallyinthispost-trutherawherepeople'semotionstakeprecedentoverthetruthitself.
Thatis,peoplerelyontheiremotionsandpersonalbeliefsratherthantherealfactswhendecidingwhatistrueandwhatnot.
Inanenvironmentlikethis,agoodfakestorythatplaysonpeople'semotionsisinfactaverypowerfultoolthatcouldleadanarmedmanontoacrusadetosavechildrenfromanundergroundsex-traffickingringfromaWashingtonpizzaplace.
Backin2016,amidtheU.
S.
presidentialelectionscampaigns,WikiLeakspublishedhackedemailsbyHillaryClintonandherstaffmemberswhichsparkedawholerangeofconspiracytheories.
Thehackedemailsamongotherthingscontainedtheword"pizza"anddinnerplansbetweenClinton'stopaides.
ItdidnottakelongforTrumpenthusiasts,whoweredesperatelylookingforsomecluesofwrongdoingsintheleakeddocuments,tocreateaconnectionbetweenthephrase"cheesepizza"andchildpornography.
ThisconnectionsoonledtheconspiracytheoriststobelievethatHillaryClintonandheradministrationarerunninganundergroundchild-traffickingringfromaWashingtonpizzaparlorcalledCometPingPong.
(Aisch,Huangand1ThedoctoredvideoofNancyPelosicanbeaccessedthroughthislink:https://www.
washingtonpost.
com/technology/2019/05/23/faked-pelosi-videos-slowed-make-her-appear-drunk-spread-across-social-media/noredirect=on&utm_term=.
66ae49201634DEEPFAKES–ANEMERGINGRISK16Kang,2016)"Thetheorystartedsnowballing,takingonthememe#PizzaGate.
FakenewsarticlesemergedandwerespreadonTwitterandFacebook.
"(Ibid)ThefakestoriesrangedfromkillroomsandSatanismallthewaytotheoriesaboutcannibalismtakingplaceinthesupposedundergroundtunnelsofthepizzaparlor.
Thisiswhena28-year-oldNorthCarolinaman,calledEdgarM.
Welch,decidedtotakemattersintohisownhandsandsavetheday.
ArmedwithmilitaryriflesandothergunshearrivedattheCometon28thofDecember2016andfiredthreewarningshotsbeforesurrenderingtothepoliceafterhavingfoundzeroevidencetosupporthistheoriesofsex-traffickingandpedophilia.
However,thisdidnotputanendtotheconspiracytheories.
Onthecontrary,supportersofthetheoriesrefusedtoletgooftheirpersonalbeliefsandwentsofarastoaccusethemainstreammediaofhelpingtocoverupthesecretcrimeorganization.
(Ibid)Wecanonlyimaginewhatevenalow-techdeepfakemightdoinsuchaclimatewhereonlyahandfulofemailsabout"cheesepizza"managedtoconjureupaconspiracytheorythatisacceptedbythousandsastrue.
Again,thisisnotbecausepeoplearesogullibleornaveastobelievewhateveryouservetothem.
Instead,fakenewsandconspiracytheoriesworkonpeople,becausetheyarewillingtoacceptallkindsof(mis)informationthatfitswiththeirownworldviews.
ThisenvironmentiswhatDaviescallsanervousstate,"astateofconstantandheightenedalertness"ofindividualsandgovernmentsalike.
(Davies,2018:p.
xii)Hedescribesthisasastatewherepeopleoftenmakedecisionsinaheatofthemoment,whilerationalthinkingisbenched,andemotionsaretheonesstirringthewheel.
(Ibid)Socialmediafurtheraggravatesthisnervousenvironment.
(Davies,2018:p.
6)Ontheinternet"informationmoveslikeavirusthroughanetwork",includingmisinformation.
(Ibid)Insuchaclimateimagesandwordsareusedastoolstoengageandmobilizepeople,veryoftenignoringthevalidityoftheinformationtheyprovideoritscorrespondencewiththeobjectivereality.
(Ibid:p.
7)"Thisistheanxietythatnowsurrounds'fakenews'andDEEPFAKES–ANEMERGINGRISK17propaganda.
"(Ibid)ItisthisnervousstatethatalsomadepeoplebelieveallkindsofhoaxesandfakenewsfollowingadeadlytsunamithathitIndonesiain2018.
WhilerecoveringfromthedevastatingearthquakeandtsunamithathittheIndonesianSulawesiislandinSeptember2018doctoredimages,videosandfakenewsallegingthatanothermuchdeadlierearthquakeandvolcaniceruptionswilloccurinthenextfewdaysstartedspreadingonlineandinmanyplaceswheretheelectricitywascutduetothesevereweather,bywordofthemouth.
Therumorscausedmajorpublicpanicandshock.
Thesituationescalatedsomuchthatthegovernmenthadtoreleaseofficialstatementsurgingthepublicnottofallforsuchmisinformationandtoreportitimmediatelytothepolice.
(LyonsandLamb,2018;TheStarOnline,2018)Whilethingsreturnedtonormalcysincethegovernmenttookextrameasurestodebunkthefakes,"Indonesia'scommunicationsministryhasannouncedplanstoholdweeklybriefingsonfakenews,inanefforttoeducatethepublicaboutthespreadofdisinformation.
"(Lamb,2018)Nowadays,thispublicanxietyisindeedmuchheightenedallaroundtheworld.
Andthiscreatesanexcellentbreedinggroundformisinformation,fakenewsandmostrecentlydeepfakes.
Althoughdeepfakesarearelativelynewphenomenon,expertsalreadywarnabouttheriskstheycanposetoindividuals,organizationsortoentirecountries.
Nobodyissafefromdeepfakes.
Oursocietyalready"suffersfromtruthdecayasournetworkedinformationenvironmentinteractsintoxicwayswithourcognitivebiases.
"(ChesneyandCitron,2018:n.
p.
,emphasisoriginal)Addtothisthenewtechnologyofdeepfakesandwhatwegetisnewwaysinwhichpeople,governmentsanddemocraticinstitutionscanbeexploited,sabotagedandaltogetherendangered.
Therefore,inthisthesis,thesocial,culturalandpoliticalramificationsofdeepfakeswillbeanalyzedinrelationtoindividualsandsocietiesalike.
DEEPFAKES–ANEMERGINGRISK183.
MethodologyInthisthesisacasestudymethodwillbeusedtoanalyzethesocial,culturalandpoliticalramificationsofdeepfakesinconnectiontoindividualsandsocieties.
Thismethodisaverypopularchoiceinsocialsciencesespecially"whenaholistic,in-depthinvestigationisrequired.
"(Zainal,2007:p.
1)Casestudiesaredesignedtoprovideanin-depthanalysisofacontemporaryphenomenoninitsowncontextandenvironmentthroughalimitedamountofcases.
(Ibid:pp.
1-2)Inotherwordsacasestudyis"anempiricalinquirythatinvestigatesacontemporaryphenomenonwithinitsreal-lifecontext;whentheboundariesbetweenphenomenonandcontextarenotclearlyevident;andinwhichmultiplesourcesofevidenceareused.
"(Yin,1984ascitedinZainal,2007:p.
2)Sincedeepfakesarearelativelynewphenomenonandconstantlyevolving,acasestudymethodseemedasthebestchoice,asitofferedauniquepossibilityofanalyzingasmallnumberofcasesingreatdetailonamicrolevel.
Thelackofrelevantresearchaboutdeepfakespresentsyetanotherreasonwhythisapproachwaschosen.
Moreover,thismethodprovidedanopportunityformultiplesourcesofdataregardingspecificdeepfakecasestobeexploredandinvestigated,givinginsightstoapreviouslylessknownphenomenon.
Asdeepfaketechnologyisrapidlyevolving,therearemanycasestobeexploredinconnectionstothistechnology.
However,giventhelimitedscopeofthisthesisIhadtodrawthelinesomewhere.
IamawarethatthislimitsmyresearchinthesensethatIamunabletoprovidegeneralconclusionsbasedonthelimitednumberofcasesanalyzed.
Nonetheless,thespecificcasesinthisthesiswerechosensoastobeabletoexploredifferentsocial,culturalandpoliticalaspectsofdeepfakesandtoanalyzethepotentialriskstheyposetobothindividualsandsocieties.
Intheupcomingchapters,thecasesselectedforthisthesiswillbediscussed.
DEEPFAKES–ANEMERGINGRISK194.
Casestudyanalyses4.
1.
DeepfakesandtheirpotentialtoharmindividualsPeoplehaveofcourseliedto,cheatedandframedotherpeoplesincethebeginningoftime,and"[a]llofthiswillbetruewithdeepfakesaswell,onlymoresoduetotheirinherentcredibilityandthemannerinwhichtheyhidetheliar'screativerole.
Deepfakeswillemergeaspowerfulmechanismsforsometoexploitandsabotageothers.
"(ChesneyandCitron,2018:p.
16)Thespecificsofthesemechanismsinusingdeepfakestoharmotherpeopleareyettobelargelyseen.
Possibleusesincludejustajoketoembarrassacolleague,apornvideoforsomeone'sowngratification,identitytheftoreventospurviolence.
(ChesneyandCitron,2018:p.
17)Onlytheskyisthelimitforthemanyusesofdeepfakes,orinotherwordstheinfinitecreativityofone'simagination.
Althoughnobodyissafefromdeepfakes,somepeoplearemorevulnerablethanothers.
InDecember2018ScarletJohansson,oneofthecelebrityvictimsofpornographicdeepfakes,statedthatalthoughthesefakesaredemeaning,theydonotnecessarilyaffecther,becausepeoplemostlyknowitisnotherinthepornvideos.
Butwhilehercelebritystatushelpsherbattlethesefakesmucheasier,shestressesthatitisatotallydifferentstoryforunknownpeoplewhocanlosetheirjobsovertheirimagebeingdoctoredtofakevideos.
(Gordon,2019)However,insomecasesevenbeingpubliclyknownwillnotsaveonefromintentionalreputationalsabotage.
SomesayRanaAyyubisoneofthemostabusedwomeninIndiainrecenthistory.
Besidesbeingawell-knowninvestigativefemalejournalistinacountrywherethegendergapDEEPFAKES–ANEMERGINGRISK20isstillveryhigh2,sheisalsoMuslimandisevenconsideredtobepartoftheanti-establishment.
(Ayyub,2018;WorldEconomicForum,2018)Asshesaysofherself,she"ticksalltheboxes.
"(Ayyub,2018)Yetdespiteyearsofconstanttrollingandhatredonsocialmediawhichwasintendedtosilenceher,shekeptreportinganywayaboutsensitivecasesregardinghumanrightsviolations.
Thesecaseswereusuallyinvolvinghigh-profileindividualsveryofteningovernmentalpositions.
Althoughovertheyearsshemanagedtoignoreandenduremostoftheonlineviolenceandhatredtargetedtowardsher,inApril2018allthatchanged.
Afterahorrificsexualassaultonan8-year-oldgirlinKashmirin2018,AyyubwasinvitedbytheBBCandAlJazeeratocommentonIndia'sstancetowardschildmolestingandthecountry'sshamefulbehaviorinprotectingtheperpetrators.
ThissparkedanoutrageamongthesupportersoftherulingHindunationalistparty,BharatiyaJanataParty(BJPinshort),whichdecidedtoteachheralesson.
Intheupcomingdays,seriesofoffensivefaketweetsappearedclaimingtobewrittenbyAyyub.
AmongotherstheystatedhatredtowardsIndiaandthepeopleofIndia.
(Ibid)Oneevenread"IlovechildrapistsandiftheyaredoingitinthenameofIslamIsupportthem.
"(ascitedbyAyyub,2018)Thefaketweetsweresharedbythousandsandneedlesstosay,soonhersocialmediawasfloodedwithabuse.
Eventually,Ayyubreleasedanofficialstatementthatshewasnottheauthoroftheallegedtweets;however,theabusedidnotstopthere.
Notlongafter,adeepfakeemergedwithAyyub'sfacedoctoredontoayoungpornactress'sbody.
TheBJPfanssharedthefakevideoontheirofficialwebsiteandthingsescalatedreallyquicklyfromthereonwards.
Instantaneouslyitwasviewedandsharedbythousandsofpeople,manypartofIndia'spoliticalestablishment.
Herprivatecontactinformationwasalsopublishedbytheunknown2In2018Indiaranked108thamong149countriesonthegenderequalityscaleaccordingtotheWorldEconomicForumreportfrom2018.
DEEPFAKES–ANEMERGINGRISK21perpetratorsandtheinappropriateoffersforsexsoonstartedarrivingonherprivatephoneandsocialmediaaccounts.
(Ayyub,2018)Itgoeswithoutsayingthatshewasbeyonddevastated.
Injustameretwodaysherentirepublicreputationwasdestroyed.
Peoplewerecallingherandtextinghertoaskifthevideowasreal,whileotherswereshamingheronline,referringtoherasaJihadistwhofavorschildrapists,someevencalledforherdeportationtoPakistan.
Tomakethematterevenworse,whenshetriedtofileacomplaintaboutthedeepfakevideotothepolice,theofficersrefusedtoreportit.
(Ayyub,2018)Sheremembersherstruggleswiththepolicelikethis:"Icouldn'tbelieveit.
Iwasawomanstandinginfrontofthemwhohadmusteredupthecouragetofileacomplaintandtheyweretryingtododgeit.
Ithreatenedthem.
Itoldthemiftheydidn'twanttoregisteracomplaintthenIwouldwriteaboutthemonsocialmedia.
Finally,aftermylawyertoldthemwewouldgotothemedia,theyfiledthereport.
"(Ayyub,2018)Thecoordinatedmisinformationcampaigntosmearhernameseemedtoworkjustasitwasintended.
People'sownconvictionsandconfirmationalbiasesweredeterminingAyyub'spublicnarrative.
Supportersoftheestablishmentandeventheauthoritiesshapedtheiropinionaboutherevenbeforereviewingtheevidence.
Therefore,itisnosurprisethatthecomplaintshefiledwiththepolicefellondeafears.
FormonthsAyyubdidnothearbackfromthematall.
Thingsonlystartedchangingwhenfinally,theUnitedNationsintervened.
"SixteenspecialrapporteurswrotetotheIndiangovernment"requestingfromthemtoprotectAyyub.
(Ayyub,2018)DavidKaye,oneoftheUNspecialrapporteursonfreedomofopinionandexpression,statedthattheUNwasespeciallyworriedaboutAyyub'ssafety,sincetheverbalabusestoherreportingwentbeyondjustdislikingherwritingandturned"sexualandviolentinnature".
(O'Grady,2018)HeaddedthattheUNfearedforAyyub'slifeduetotheDEEPFAKES–ANEMERGINGRISK22factthatherjournalistcolleagueandfriendGauriLankeshwaskilledinfrontofherhouseinBangalore,IndiabyahelmetedbikerinSeptember2017,justayearbeforethehighlycoordinatedonlineattacksstartedtargetingAyyub.
LankeshwasjustlikeAyyub,anoutspokencriticoftheHindunationalistgovernmentanditsPrimeMinisterModi.
Inherlasteditorialbeforeshewaskilled,shealleged"thatspreadingfakenewshadcontributedtothesuccessofMr.
Modiandhisparty.
"(Mondal,2017)However,shewasnottheonlyvictim.
InthelastdecademanyotherjournalistswhoattemptedtocriticizethecurrentIndianpoliticalestablishmentwerealsosilencedinsimilarwaysasLankesh.
YetthesekillingsandassaultsarenotonlypresentinIndia.
Nowadaysbeingajournalistoranoutspokenintellectualanywhereintheworldisadangerousprofession.
Accordingtoareportfrom2017"[m]orethan800journalists,mediaworkersandsocialmediaproducershavebeenkilledduringthepasttenyears.
"(CarlssonandPyhtri,2017:p.
12)Therearehowever,manymorenon-lethalattacksthatcanbringharmtoapersonrangingfrom"intimidation,harassmentandarbitrarydetentiontomisogynisticattacksdirectedagainstwomenjournalists"andveryrecentlydeepfakesaswell.
(Ibid)Whatthesestatisticsindicateisthatexpertsarenotjustlosingthetrustofthepublic,buttheyarealsospecificallytargetediftheyattempttoquestionthepopularnarratives,normsandvaluesinasociety.
NotlongaftertheUNfiledreportstotheIndiangovernment,theonlineandofflineabusesagainstAyyubdecreasedrapidly,butthedamagewasalreadydone.
AsAyyubrecollects,sheisnotthesamepersonanymoreafterthishighlycoordinatedmisinformationcampaignagainsther.
Shebecamelesspresentonlineandmorecautiousofexpressingheropiniononsocialmedia.
(Ayyub,2018)Whatweseehereisaclearretreatofavictim.
Risksandthreatstoaperson'sdigitalsecuritycanaffectthewaysinwhichonewillusetechnologyinthefuture.
(SinghandRoseen,2018)Ayyubforinstancedecidedtolimitheronlinepresenceandbelessoutspokenonsocialmedia.
(Ayyub,2018)ThisjustproveshowourDEEPFAKES–ANEMERGINGRISK23onlineworldisinfactreinforcingtheembeddednormsandvaluesofoursociety.
Groupswhicharealreadymarginalizedinourphysicalworldhaveahigherchanceofbeingvictimizedonlineaswell.
People,likeAyyub,"whoseintersectingidentitiescovermultiplemarginalizedgroupsfaceamongtheworstformsofonlineabuse.
"(SinghandRoseen,2018)BeingaMuslimwomanandafreelancejournalistnotafraidtospeakoutagainstthegovernmentputsadirecttargetonherback.
"Andatatimewhenthetechnologylandscapeisevolvinginincreasinglyintrusiveandunpredictableways,thesethreatscanoftentranslateintoofflineharms,too,asthelinesbetweenourdigitalandreallivesarecontinuouslyblurred.
"(SinghandRoseen,2018)WhileAyyubdecidedtoonlylimithervoiceonsocialmediaplatforms,othervictimsmightchoosetodisappearfromtheonlineworldaltogetherforfearofbeingtargetedandabusedagain.
"Asaresult,whenthinkingaboutthecollectiveoutcomethatsilencingindividualvoiceshasinademocraticsociety,theeffectsbecomeparticularlyworrisome.
"(SinghandRoseen,2018)Inthisenvironmentwomenareespeciallyvulnerable.
Pornographicdeepfakesintendedtosilence,intimidateorexploitwomenarebecominganewonlinephenomenonallovertheworldandnotjustinIndia.
Backin2012whenathen18-year-oldAustralianwoman,namedNoelleMartin,googledherownpicturesshewashorrifiedwiththeresults.
HerpicturestakenfromherprivateFacebookaccountweresuperimposedontopornographicimagesandvideosandsharedmultipletimesonfamouspornwebsitesandforums.
HerfacewasalsousedasacoverforpornDVD's.
Butitiswhenshetriedtospeakoutaboutthefakeimagesandvideoswhentherealharassmentstarted.
Shewasdoxxedbyharassers,whopublishedherprivateaddressandcontactinformationinordertoterrorizeandsilenceher.
Somepornsitesandforumsevenrefusedtotakedownthedeepfakes,unlessshewillsendthemnudephotosofherself.
Herstruggleswithharassmentcontinuedformorethan6years.
(Curtis,2018)DEEPFAKES–ANEMERGINGRISK24Aswesee,eveninsocietieswhichareintermsofgendermuchmoreequal,forinstanceAustralia(ranked39ontheglobalscalein2018accordingtotheWorldEconomicForum,2018)womenarefindingitveryhardtofightagainstpornographicdeepfakes.
Forinstance,lawsagainstnon-consensualusageofpicturesorvideosforpornographicpurposeswereonlyintroducedintotheAustralianjuridicalsystemin2018aftermultiplevictimshavespokenoutabouttheirexperience.
Undertheselegislationsconvictedindividualscouldfacejailtimeuptofiveyearsfordistributingnon-consensualintimateimagesanduptosevenyearsofimprisonmentiftheyaremorethanthreetimesoffenders.
Also,accordingtocivilpenaltiesthesecrimeswillbepunishabletouptoAU$105,000forindividualsandAU$525,000forcorporations.
(Reichert,2018)But,whilethesenewlawsmighthelppersecuteoffendersinAustralia,theywillnotbeapplicabletoperpetratorswhoareoverseas.
Itseemsthatonceagainnothingcanstopperpetratorsfromcountrieswithoutsuchcopyrightlawsfromacquiringsomeone'sphotosorvideosonlineevenifthatpersonhasaclosedandprivateaccount.
Addingtothattheunwillingnessoflawenforcementandjuridicalinstitutionstohelpthevictims,likeinthecaseofAyyub,thenwehaveanenvironmentwhereperpetratorswillbeabletooperatefreelyandwithoutconsequence,helpingtheuseofdeepfakestobecomecommonplace.
Whenitcomestothevictimsitisquitealarmingtoseethatmostoftheseinvoluntaryfakevideosareofwomenknownandunknownalike.
Thisisspecificallyevidentwhenwetakeintoconsiderationtheenormousamountofpornographicdeepfakesofcelebritiesandunknownpersonsinrecentyears.
Whilemenarenotanexceptionwhenitcomestodeepfakeexploitation,theymostlyneverendupinafakepornvideo.
Theirfacesareusuallydoctoredintoprankvideosorvideoswhichintendtosabotagetheirreputationinotherways.
Althoughtheseallcanbeextremelyharmfultoaperson,neverthelessitseemsthattheareaoffakepornisreservedforwomenonly.
(Harwell,2018)Thesevideoshavebeen"weaponizedDEEPFAKES–ANEMERGINGRISK25disproportionatelyagainstwomen,representinganewanddegradingmeansofhumiliation,harassmentandabuse.
Thefakesareexplicitlydetailed,postedonpopularpornsitesandincreasinglychallengingtodetect.
"(Ibid)Theseexampleshighlightthegendereddimensionofthephenomenon:"[t]hishasbeenthecaseforcyberstalkingandnon-consensualpornography,andlikelywillbethecasefordeepsexfakes.
"(ChesneyandCitron,2018:pp.
17-18)Thereisagrowingnumberofwebsitesandcloseddiscussionforumswhichofferdeepfakesfortheirusersforthestartingrateof$20perfake.
(Harwell,2018)Thetargetedvictimsaremostly"womenfarfromthepubliceye"referredtobyanonymoususersas"co-workers,classmatesandfriends.
"(Ibid)ChesneyandCitron(2018)stressthatwhenvictimsfindoutthattheirfaceshavebeensuperimposedintopornographicimagesandvideos,thepsychologicalpressurecanbequitedestructive.
Anditdoesnotevenmatterwhatthecreatorintendedtodowithsuchadeepfake.
Victimscanfeelanentirerangeofemotionsfrombeingscared,tobeinghumiliatedorevenpsychologicallyabused.
Fakepornvideostreatwomenlikesexualobjectsforcingthemintonon-consensualvirtualsex.
(ChesneyandCitron,2018:p.
18)Amorechillingeffectofdeepfakepornvideosisthattheycanalso"transformrapethreatsintoaterrifyingvirtualreality.
Theysendthemessagethatvictimscanbesexuallyabusedatwhim.
"(Ibid)Whentalkingaboutherfirstreactiontoseeingthefakepornvideoaboutherself,Ayyubsaiditwasaltogetheradevastatingexperience:"Ijustcouldn'tshowmyface.
Youcancallyourselfajournalist,youcancallyourselfafeministbutinthatmoment,Ijustcouldn'tseethroughthehumiliation…Ifeltsoembarrassed.
TheentirecountrywaswatchingapornvideothatclaimedtobemeandIjustcouldn'tbringmyselftodoanything.
"(Ayyub,2018)DEEPFAKES–ANEMERGINGRISK26Ontheotherhand,NoelleMartinwhosefacewassuperimposedtodozensofpornvideosandimagesovertheyearssaysthatshefeltlikeasifsomeonewasstrippingawayherdignityandhumanity.
(Laschon,2018)JusthowvulnerablewomenarewhenitcomestoonlinesexualexploitationbydeepfakeswasdemonstratedbyyetanotherfakeappcalledDeepNudereleasedinJune2019.
Thislatestapp"undresses"women'spicturesandbydoingsoitcreatesextremelyrealisticnudeimagesofthevictims.
Thetargetsareonlywomen.
Thecreator,whoremainsanonymous,claimsheusedmorethan10000picturesofnakedwomentotraintheapp'salgorithms,becausetheywereeasiertofindonline.
Atthispointtheappwillgeneratenudebodiesofwomenevenifyouprovidethealgorithmapictureofaman.
However,thecreatordoesnotruleoutthepossibilityofcreatinganappthatwillundressmenaswell.
EventhoughtheDeepNudedidnotcreaterealnudephotosofactualwomen,thepicturescaneasilybemistakenforarealthingandcauseirreversibledamagestothevictims.
AswehaveseenalreadyinthecaseofMartinandAyyub,fakepornisareallypowerfulmechanismtointimidate,harassandsilencewomen.
(Hao,2019)Duetothehighlyvocalonlinebacklashthe"DeepNudehasnowbeentakenoffline,butitwon'tbethelasttimesuchtechnologyisusedtotargetvulnerablepopulations.
"(Ibid)Onlinesurveillancecanhavequiteadeterringeffectonvictims.
Knowingthattheirfakepornvideosorimagesweresharedandviewedbythousandsofpeoplewillgiveadditionalreasonsforthevictimstoretract,feelingdefeated,humiliatedandviolated.
Thisisespeciallytrueofvulnerablegroups,whoareonlineevenmorevulnerable.
(SinghandRoseen,2018)Althoughnowadayseveryonecanparticipateintheproductionofknowledge,inequalitiesstillpersistinsocietyaswellaswithinthedigitalenvironment.
Thus,resultingincertainpeoplehavingmorepowertoshareinformationaboutacertainmatterthanothersdo.
Thismeansthatnotallinformationisequallyvisibleoraccessiblebyeveryone.
(HanellandDEEPFAKES–ANEMERGINGRISK27Sal,2017)Whilebeingvisiblehassomeadvantages.
visibilityisalsoconsideredtobe"adouble-edgedsword:itcanbeempoweringaswellasdisempowering.
"(Brighenti,2007:p.
335)Vulnerablegroupswhoarenotinpowerfulpositionsinasocietyarealsolessabletoparticipateintheproductionofknowledge.
Thus,theyareconsideredinvisible,whichmeanstheyare"beingdeprivedofrecognition.
"(Ibid:p.
329)However,thelinkbetweenrecognitionandvisibilityisnotalwaysstraightforward.
AccordingtoBrighenti,thresholdsofvisibilityplayanimportantroleindeterminingone'svisibilityandrecognitioninasociety.
This"fairvisibility",asBrighenticallsit,canrangefromminimumtomaximum,regardlessofthenatureofthecriteriaweuse.
Somepeoplewhofindthemselvesbelowthelowerthresholdoffairvisibilityareconsidered"sociallyexcluded".
(Ibid:pp.
329-330)Likeforinstance,illegalimmigrantsorhomelesspeople.
"Ontheotherhand,asyoupushyourself–orarepushed–overtheupperthresholdoffairvisibility,youenterazoneofsupra-visibility,orsuper-visibility,whereeverythingyoudobecomesgigantictothepointthatitparalysesyou.
Itisaconditionofparadoxicaldoublebindthatforbidsyoutodowhatyouaresimultaneouslyrequiredtodobythewholeensembleofsocialconstraints…Distortionsinvisibilityleadtodistortionsinsocialrepresentations,distortionsthroughvisibility.
"(Brighenti,2007:p.
330,emphasisoriginal)Therefore,vulnerableindividuals,likeAyyubandMartin,whoare'pushed'intothezoneofsuper-visibilityareunabletoinfluenceontheirowntermstheironlinenarratives.
Anotherterribleoutcomeforthevictimsofsuchpornographicdeepfakesisthehugeamountofinappropriateandoffensivesexualofferstheygetfromstrangersandsexualpredatorsalike.
NoelleMartinforinstancenotedthatthefirstdoctoredsexuallyexplicitpornographyofherswhichappearedontheinternetwasmadeusingapictureshetookwhenshewasonly17.
Shesaysshewasspecificallytargetedbecausepeoplefetishizedher.
DEEPFAKES–ANEMERGINGRISK28(Laschon,2018)RanaAyyubhadasimilarexperience.
SherecallsmanypeopleleavingherdistastefulcommentsonFacebooktalkingabouther"stunningbody"whileothersdidn'tevenshyawayofopenlyaskingherhowmuchmoneyshechargesforsex.
(Ayyub,2018)Evenifsuchdeepfakeswillbedebunkeditmightbetoolittletoolateforthevictimsdepictedinthem.
Apartfromtheinflictedpsychologicaldamageandonlineabuse,thesedeepfakescancauseindividualstolosetheirjobsoraltogethertodestroyanyprospectfortheircurrentorfuturecareers.
Thepost-truthclimatethatweliveinonlyincreasesthechancesofsuchdeepfakestobedistributedbythousandsofpeopleonlineregardlessoftheirfactuality,causingirreversiblereputationaldamagetoindividualsintheprocess.
(ChesneyandCitron,2018:p.
19)AndonceoutthereontheWorldWideWebsharedbythousandsandthousandsofpeople,thesedeepfakeswillbehardifnotimpossibletoremove.
In2018,"Googleadded'involuntarysyntheticpornographicimagery'toitsbanlist,allowinganyonetorequestthesearchengine[to]blockresultsthatfalselydepictthemas'nudeorinasexuallyexplicitsituation.
'Butthere'snoeasyfixtotheircreationandspread.
"(Harwell,2018)Martinknowsexactlyhowthisfeels;evenafteryearsoffightingforjusticeshestillencountersdozensofdeepfakeswithherimagedoctoredintothemjustbygooglinghername.
(Laschon,2018)Tomakethematterevenworse,people'sconfirmationalbiasesandalgorithmicprocessingraisethepotentialrisksofthefakesbeingencounteredbytheirfutureemployers,colleagues,acquaintancesandevenromanticinterests.
(ChesneyandCitron,2018:p.
19)InthecaseofAyyub,thepornographicdeepfakeaboutherwassharedbymanymembersoftheHindunationalistpoliticalparty,BJP,andwaspostedontheirofficialfanpage,boostingitsvisibilitysomuchsothatinjustmerefewdaysthevideowassharedmorethan40,000times.
DEEPFAKES–ANEMERGINGRISK29Thewholecountryhadaccesstoherdeepfakeporn.
(Ayyub,2018)AsitwasalreadymentionedinthePost-truthsection,thehigherthevisibilityofthestory,themorecredibleitisamongpeoplewhoencounterit.
Thefactthatnowadaysinformationisdistributedinanalgorithmicmanner,andthatpeopleshareitwithlike-mindedpeopleonsocialmediaplatforms,haschangedthewayinwhichthingsbecomevisibletous.
(HanellandSal,2017:pp.
154-156;ChesneyandCitron,2018:p.
13)Moreover,ifcertainmisinformationorfakenewstargetinganindividualbecomesaviralphenomenon,itonlygainsmorevisibilityandthuspotentiallycausingevenfurtherharm.
Yetitisnotonlyhighvisibilitythatmakesthesevideosbelievable.
Asmentioned,itispeople'ssusceptibilitytobeingeasilysweptawaybyrumors,liesandmisinformationwhichreinforcetheirpre-existingbeliefsandopinionsthatposearealthreat.
(Davies,2018:p.
7;Prego,2017:pp.
20-21)Ayyubforinstance,inoneofherrecentarticlesfrom2019,talksaboutthiskindoftoxicenvironmentpresentinIndia.
Shearguesthatthehatredandviolenceagainstethnicminorities,especiallyMuslims,isspreadinglikeadiseasebothonlineandofflineaswell.
Inaddition,therulingBharatiyaJanataPartyanditspresident,AmitShah,encouragethiskindofbehaviorbypubliclyexpressingtheirlackofrespectandhatredtowardsMuslimseverychancetheyget.
InatweetfromApril2019,theystatedthattheywouldgetridofalltheinfiltratorsofthecountrywhoarenotBuddhists,HindusorSikhs.
Lateron,Shahtookthispromiseastepfurtherbyvowinghewillgetridofallthe'termites'bythrowingthemintotheBengaliBay.
However,besidesthegrowingnationalismandincreasedhostilitytowardsMuslimsandotherminorities,whatAyyubfindsevenmorealarmingisthat"manycitizenshavefoundthisnewlanguageofhateliberatingandacceptable.
"(Ayyub,2019)Anditisexactlythisclimate,whichcelebratesnarrativesthatareinlinewiththerulingpoliticalideologythathasenabledAyyub'sdeepfakeporntobecomesopopularandcredibleamongthepublic.
(Ibid)SuchapublicthatDEEPFAKES–ANEMERGINGRISK30allowsitselfto"beblindedpermanently"willthreatentodestroythecountry'sdemocraticsystemaltogether.
(Ibid)4.
2.
DeepfakesandtheirpotentialtoharmsocietiesOurtechnologicalcapacitiestocreatevideosthatcanmimicrealityquiteaccuratelycomeatatimewhensocietyisalreadyvulnerabletomisinformationandfakenews.
Theproductionofknowledgeisnolongerinthehandsof(trusted)mediacompanies,theacademiaandthegovernment.
Nowadayseverybodycancreatetheirownversionsofthetruthandfactsanddistributeittotherestoftheworld.
Ourconfirmationalbiasesplayaroleintheacceptanceofmisinformation,andalgorithmshelpspreadthemfurther.
(ChesneyandCitron,2018:p.
9)Thisalsomeansthat"[d]eepfakesarenotjustathreattospecificindividualsorentities.
Theyhavethecapacitytoharmsocietyinavarietyofways.
"(ChesneyandCitron,2018:p.
20)Thefactthattheyhavenotbeenutilizedonamuchgreaterscaleyetdoesnotmeanthatpotentialharmcausedbythemisanunrealisticscenario.
Sincetheyareindeedaverynewphenomenon,wehaveyettounderstandwhatspecificriskstheypresenttooursociety.
However,giventheenormousamountoffakenewsandmisinformationcampaignsin2016,whichcontributedtotheBritishpeoplevotingforBrexitandTrumpbeingelectedforpresidentgivesussomeinsightintowhatwemightexpectwhenitcomestodeepfakesaswell.
ChesneyandCitron(2018)wereamongthefirsttotrytooutlinethepotentialrisksanddangersdeepfakesposetosocieties.
Theyarguethattherearealotofbeneficialoutcomesaswellfromthecreationofdeepfakes,especiallyinarts,educationandgaming.
Forinstance,Hollywoodalreadyutilizeddeepfaketechnologytobringtolifedeceasedactorsforcertainroles.
ThemostfamousexamplesofthisareintheStarWarsfranchise,wherethecharactersplayedbythelateCarrieFisherandPeterCushingwererecreatedwiththehelpofthistechnology.
GamingindustryisyetanotherareawheredeepfakesmightDEEPFAKES–ANEMERGINGRISK31comeinhandy.
Gamingplatforms,suchasNintendoWii,alreadydevelopedcertaingameswhereuserscancreatetheirowncustomizableavatars.
Whenitcomestoeducation,ChesneyandCitronarguethatthistechnologycouldalsobeusedtoalterexistingfilms,documentariesorshowsforpedagogicalpurposes.
(ChesneyandCitron,2018:pp.
14-16)Forinstance"[w]ithdeepfakes,itwillbepossibletomanufacturevideosofhistoricalfiguresspeakingdirectlytostudents,givinganotherwiseunappealinglectureanewleaseonlife.
"(Ibid:p.
14)Despiteallthebeneficialpossibilitiesofdeepfaketechnology,itistheirpotentialharmthatweshouldbefocusingoninstead.
Thisshouldbedoneconsideringouralreadyvulnerabledigitalenvironmentandcurrentsocietaldevelopments.
ChesneyandCitron(2018:pp.
16-21)pointoutthatthethreatpresentedbydeepfakestosocietyhasfirstofallsystematiccharacteristics,sincetheyhaveapotentialtoreachandharmalllevelsofsociety.
Forinstance,"[t]hedamagemayextendto,amongotherthings,distortionofdemocraticdiscourseonimportantpolicyquestions;manipulationofelections;erosionoftrustinsignificantpublicandprivateinstitutions;enhancementandexploitationofsocialdivisions;harmtospecificmilitaryorintelligenceoperationsorcapabilities;threatstotheeconomy;anddamagetointernationalrelations.
"(Ibid:p.
21)However,thereisnowayofaccuratelypredictingwhataspecificdeepfakemightdo.
Thepotentialharmdependsonthecontextofitscreationandcirculation,anditisonlypossibletounderstanditsfullimpactonceithadtimetoreachpeople.
Bythattimeofcoursesomedamagescouldbeirreversible.
InMay2018adeepfakeofDonaldTrumpaddressingtheBelgianpublicemerged.
3InthiscrudelydoctoredvideothefakeTrumpisseencallingoutontheBelgianpublicto3ThedeepfakeofDonaldTrumpaddressingtheBelgianpubliccanbeaccessedthroughthislink:https://hoax-alert.
leadstories.
com/3469396-fake-news-spa-on-twitter.
htmlDEEPFAKES–ANEMERGINGRISK32urgetheirgovernmenttowithdrawfromtheParisclimateagreement.
ThevideowascreatedbyaBelgianleft-wingpoliticalpartycalledSocialistischePartijAnders(orsp.
a)andpostedontheirofficialsocialmediaaccounts.
Thevideo,withtheintentiontosparkpeople'sinterestinissuesofclimatechange,receivedhundredsofangrycomments,manystatingthatTrumphasnorightatallinexpressinghisopinionaboutaBelgianpoliticalmatter.
(Schwartz,2018)Oneoutragedtweetread:"HumpyTrumpneedstolookathisowncountrywithhisderangedchildkillerswhojustendupwiththeheaviestweaponsinschools.
"(ascitedbySchwartz,2018)AnotherwentevenfurtherthrowinginsultsontheentireAmericannation:"Trumpshouldn'tblowsohighfromthetowerbecausetheAmericansarethemselvesasdumb.
"(Ibid)Needlesstosay,thedeepfakemanagedtoprovokeandangerpartsoftheBelgianpublicquitealot.
Eventhefactthatthedeepfakewasreallybadlydoctored–thefakeTrump'slipsandthesoundswereoffsyncandthewholevideowasofverylowquality–didnothelpthesituationatall.
ThepeoplewhowatcheditweregenuinelyconvincedthatTrumpreallydidsaythosethingsandunderstandablytheydirectedtheirangertowardshim.
Butthatwasamistake.
(Schwartz,2018)Ifonewatchesthevideoverycarefully,attheendthefakeTrumphimselfadmitsthatheisactuallyfeaturinginafakevideo.
However,attheexactmomentwhenhesaysthosewordsthesoundleveldropsdrasticallyandtheDutchsubtitleswhichwerepresentduringtheentirevideodisappear,makingitverydifficultforthoseBelgianpeoplewhodonotunderstandEnglishtorealizethatwhattheyhaveseenisnotactualfootageofTrump.
Also,thosehavingbeenpayingattentiononlytothesubtitlesmaybeforgivenforhavingmissedthisfinalpart.
Asitwaslaterrevealed,thesp.
arequestedthehi-techforgeryfromaproductionhousewhichusesAItogeneratehighlyrealisticfakevideosofpeople.
However,whenthevideodidnotendupmakingtheexpectedoutcomes,thepartywaslefttomanageDEEPFAKES–ANEMERGINGRISK33thesituation.
Theirsocialmediateamwasforcedtoexplainoverandoveragaintotheiroutragedfollowersthatitwasjustasillyprankvideoandnothingmore.
(Schwartz,2018)AlthoughtheintentionsofthecreatorsoftheTrumpdeepfakeweretoenlightentheBelgianpublicabouttheimmediatethreatofglobalwarmingandtoinspirethemtoactonit,theeffectsweresomethingtheydidnotanticipate.
Andthatistheproblem,becauseonceoutthereontheWorldWideWebdeepfakescanhavealifeoftheirown.
Themembersofsp.
adefendedtheiractionsbystatingthat,giventhelow-techqualityofthevideo,theyassumedthattheirfollowerswillinstantlyknowitisafakeandunderstandthehiddenmessageinit,whichwilleventuallyleadthemtosigntheclimatechangepetitionthatthefakevideowasmeanttopopularize.
(Schwartz,2018)Thisiswhereoneofthetruedangersofsuchdeepfakeslies;theinabilityofcreatorstopredicthowpeoplewillreactonthemandwhatkindofconsequencestheymighthaveonthewidersociety.
Whatwearedealingwithhereisacleardiscrepancybetweenthecreators'intentionwithsuchatechnologyanditseverydayusageandperceptionbythepublic.
Forinstance,theBelgiandeepfakestirredupsomenegativeemotionsamongthefollowersofsp.
a.
ManythoughtthatTrumpwasdisrespectingtheirnationandminglingwiththeircountry'spolitics.
Byplayingonpeople'semotions,adeepfakeisabletomotivatethemto(re)act.
Negativeemotionssuchasfearandthefeelingofinsecurityareincrediblystrongmotivators.
Theycancreateinstabilityindiversesocieties,erodingthetrustincivicanddemocraticinstitutionsresultinginastateofnervousness.
(Davies,2018:p.
20)AccordingtoDavies,"[m]uchofthisnervousnessthatinfluencesdemocracytodayisnotsimplybecausefeelingshaveinvadedaspacepreviouslyoccupiedbyreason,butbecausethelikelysourcesandnatureofviolencehavebecomehardertospecify.
"(Ibid:p.
17)Inthissenseviolencedoesnotnecessarilymeanphysicalviolence;itcanalsomeanjustathreatofviolence,asenseofdangerorthefeelingofinsecurity.
Nonetheless,thisnervousstatecanshapethepublic'sopinionsomuchsothatevenifthedeepfakeisprovedDEEPFAKES–ANEMERGINGRISK34tobefalsebyoutsiders,the"beliefinitbecomesanarticleoffaith,alitmustestofone'sadherencetothatcommunity'sidiosyncraticworldview.
"(Donath,2016ascitedinZuckerman,2017)Peopleingeneralfallforfakenewsandsharethemmuchmoreoftenthanaccuratenews.
Thisisduetothefactthatinformationtravelsfasteronsocialmediaplatforms"ifitlooksandfeelstrueonavisualandemotionallevel.
"(Davies,2018:p.
15)Anotherideatoconsideristhathumanstendtocaremorefornegativeandnovelinformation,whichcanevokestrongeremotionslikesurpriseanddisgust.
(ChesneyandCitron,2018:p.
12)"Negativeinformationnotonlyistemptingtoshare,butitisalsorelatively'sticky.
'Associalscienceresearchshows,peopletendtocredit—andremember—negativeinformationfarmorethanpositiveinformation.
"(Ibid)Moreover,humansarepredisposedtopaycloseattentiontothingsthatwillstimulatethem,forinstancethingsthatareviolent,sexual,disgusting,embarrassingandevenhumiliating.
(Ibid:p.
13)Therefore,itisnosurprisethatmanyBelgianswhosawthevideowereattentivetothefakeTrump'sdisrespectfulbehaviorandmistookhimfortherealTrump,atthesametimecompletelyignoringthefactthatthevideowasoflowqualityorthatthefakeTrumphimselfstatedattheendthatthevideoisjustafake.
ReportsaboutthisTrumpvideomadeinternationalheadlinesaswell,butitdidnotcauseanymajorreactionsontheglobalscaleanditwaswidelyignoredbytheTrumpadministrationaswell.
However,thismightsimplybebecausebythetimethenewsaboutthevideoreachedtherestoftheworldithadalreadybeendebunkedandnewsoutletswerestatingitclearlythatitwasacaseofafakevideomeanttobeapracticaljoke.
4Anotherexplanationastowhythisvideodidnothavesomebiggerimpactontheglobalscaleis4TheBelgiandeepfakefeaturingTrumpwasfirstdebunkedbytheonlinenewswebsitecalledLeadStorieson20thMay2018.
(Schenk,2018)DEEPFAKES–ANEMERGINGRISK35maybethefactthatthefakeTrumpwasverysimilarincharactertotherealTrump.
EspeciallywhenitcomestodeclaringclimatechangeasnothingmorebutahoaxorcallingonothernationstofollowtheAmericanexampleindealingwithpoliticalissues.
ManypeoplearealreadyusedtothisrealTrump;therefore,encounteringanothervideoofhimrantingabouthowclimatechangeisonlyfakenewswillnotbeanythingnewtothem.
Anotherthingtoconsiderisperhapsthecontentofthevideo:climatechange.
Peopleareverymotivatedtoavoidanyimmediatethreats,likeabarkingdog,oradangerousstreet.
Yetwhenitcomestoclimatechange,peoplearemoredifficulttobemoved.
Thistopicdoesnothavealotofemotionalappealforindividualswhohaveonadailybasistheirownpersonalproblemstodealwithanddonotnecessarilyfeelthedirectimpactofglobalwarming.
(Markman,2018)Infact,"manyeffectsofclimatechangearedistantfrommostpeople.
"(Ibid)Therefore,itisnosurprisethattheBelgianpublicwasmorefocusedon(thefake)Trumpaddressingtheirnationandcommentingontheirpolitics,ratherthantheissueofclimatechange.
Butnonetheless,evenifthiswasjustasmall-scaledeepfakeincident,thefactthatitwascreatedbyapoliticalpartyinordertopersuadepeopletochangetheirmindsaboutacertainpoliticalissuemakesthisscenarioquitealarming.
Especiallyifweaddtothis"thenatureoftoday'scommunicationsenvironment"incombinationwithourconfirmationalbiasesandalgorithmicnetworks–thenwehaveaclearrecipeforapotentialdisasterevenonamoreglobalscale.
(ChesneyandCitron,2018:p.
19)Theresultsofafakevideocirculatingcanbemoresignificantincaseswherethestakesarehigher,ortheemotionalappealstronger,forexampleifinsteadoftryingtoinspirepeopletocareabouttheenvironment,theintentofthevideowouldbetomakethembelievetherehasbeenaterroristattack.
AsDavies(2018:p.
123)pointsout,evenjust"[s]mallactsoftransgressioncanhavemajorpoliticaleffects,iftherighttoolandtargetarecarefullyselected.
"DEEPFAKES–ANEMERGINGRISK36TheBelgiandeepfakecameoutjustamonthafterBuzzfeedpublishedanarticlewarningaboutthepossiblerisksofusingdeepfakesinpoliticalcampaigns.
Inordertodemonstratethecapacityofsuchtechnologyandforthesakeoftheargument,theycreatedadeepfake,featuringfakeBarackObamatrashtalkingaboutTrump.
5ForthistheyutilizedafreeapplicationcalledFakeApp,whichwasreleasedbytheReddituser,"Deepfakes",whoalsocreatedthefakepornvideosofcelebritiesin2017mentionedabove.
Intheirarticle,whichalsoshowcasedtheforgery,theyarguedthatalthoughthetechnologytocreatedoctoredvideosisinitsinfantstage,requiringafairamountofITskills,timeandfastcomputers,thepotentialforittobecomemoresophisticatedanddemocratizedisjustaroundthecorner.
Thearticlemakestheclaimthatifsoonanyonecanmakeafakevideothatdefiesreality,thenthereareprettyperiloustimesaheadofus.
(Silverman,2018)Atthesametimetherearestillsomewhoopposetheseapocalypticpredictions,claimingthatthetechnologyisnotsofaradvancedthatdeepfakescouldactuallyposearealthreattooursociety.
AnarticlewhichappearedinTheVergeinMarch2019statedthat"deepfakepropagandaisnotarealproblem"anditwon'tbecomeanissueinournearestfuture.
(Brandom,2019)Theauthorofthetextarguesthatthepredictionsofdeepfakesbecomingathreattoourdemocraticsystemshavenotmaterializedyet.
Headdsthatthemainreasonforthisissimple:it'sjustnotworththetrouble.
Heclaimsthatthealgorithmsthatcandetecttheforgeriesarewidelyavailableonline;therefore,itcanbeeasilyprovedthatthedoctoredvideosarefakes.
Oneofhisargumentsisthatdoctoredvideosarenotusefulenoughtotheextentthattrollcampaignsare.
(Ibid)"Mosttrollcampaignsfocusedonaffiliationsratherthaninformation,drivingaudiencesintoevermorefactionalcamps.
Videodoesn'thelpwiththat;ifanything,ithurtsbygroundingtheconversationindisprovablefacts.
"(Ibid)5ThedeepfakecreatedbyBuzzfeedfeaturingBarackObamatalkingbadlyaboutDonaldTrumpcanbeaccessedthroughthislink:https://www.
buzzfeed.
com/craigsilverman/obama-jordan-peele-deepfake-video-debunk-buzzfeedutm_term=.
bgWEL9pQjg#.
fs3VQGmReqDEEPFAKES–ANEMERGINGRISK37Accordingtohimdeepfakesareingeneralmoredangerousforindividuals,giventhehugeamountoffakepornofcelebritiesandunknownpeoplealikecirculatingtheinternet.
Nonetheless,heremainsskepticalaboutthembecomingarealthreattooursocietyanytimesoon.
(Ibid)However,onemightdisagreewiththis.
IfthepublicreactionstothebadlydoctoredvideoofNancyPelosiandtheBelgiandeepfakeareofanyindication,wehavesomethingtoworryabout.
Thefactthattherehasnotbeenyetanymajorsocietaldisruptionbyadeepfakedoesnotmeanthatitcannothappentomorrow,aweekfromnow,amonthfromnoworinthefuture.
Itseemsthat"[e]venatthisearlystage[oftechnologicaladvancementofdeepfakes]it'sprovingdifficultforhumanstoconsistentlyseparate"themfromreality.
(Silverman,2018)Ifapicturesaysmorethanathousandwords,thenwhataboutavideo"Insomeinstances,theemotionalpunchofafakevideooraudiomightaccomplishadegreeofmobilizationto-actionthatwrittenwordsalonecouldnot.
"(ChesneyandCitron,2018:p.
24)Whatwehaveseensofararedoctoredimages,alteredvideosandfakenewsintendedtodistorttherealityandchangepeople'sopinionaboutcertainthings.
Wehaveyettoexperienceadeepfakewiththecapacitytodisruptoursensesofrealitythatwillhavebiggersocialandpoliticalramifications.
Thisisapossibility,asevenifouralgorithmswouldbeabletodetectthefake,itwillnotbeenough.
(Vincent,2019)OnepieceofevidenceforthisisthatalreadysomebadlydoctoredvideoshavemanagedtocreatequitealotoffrictionontheAmericanpoliticalscene.
ThebestrecentexamplesaretheNancyPelosicasefrom2019mentionedabove,andthedoctoredvideoofaCNNreporter,JimAcosta,seeminglyhittingaWhiteHouseassistantin2018.
6Thelatterbeingevenamorechillingepisode,consideringthefactthattheWhiteHousePressSecretary6ThealteredvideoofJimAcostacanbeaccessedthroughthefollowinglink:https://www.
forbes.
com/sites/laurenaratani/2018/11/08/altered-video-of-cnn-reporter-jim-acosta-heralds-a-future-filled-with-deep-fakes/#d063663f6c1eDEEPFAKES–ANEMERGINGRISK38SarahSanderssharedtheforgedvideoonTwitter,whiledefendingtheWhiteHouse'sdecisiontopermanentlyrevokeAcosta'spresspass.
ThepartoftheoriginalvideowheretheCNNreporterisseentakingthemicrophonebackfromtheWhiteHouseassistant,whowasdeterminedtostophimfromfurtherquestioningthepresident,wasacceleratedtoappearthatAcostaactuallyhittheassistant.
TheforgerywascreatedbyInfowars,anonlineconspiracywebsitebestknownforits2012SandyHookElementarySchoolshootingconspiracy,inwhichtheyclaimedthattheshootingthatledto28peoplebeingkilled,actuallyneverhappened.
Keepingthisinmind,thefactthattheWhiteHousedecidedtobackitsdecisionofbanningtheCNNreporterbyusingaforgedvideocreatedbyaconspiracywebsiteraisessomeseriousconcerns.
Atthesametime,italsodemonstratesthegrowingrisksofdeepfakesbeingweaponizedinthenameofpolitics.
(Aratani,2018)Whattheseexamplesalsoillustrateisjusthowmuchonesocietycanbecomeeasilypolarizedbyfakenews,doctoredvideosandveryrecentlydeepfakesaswell.
Theseliesandmisinformationthathavecreptintoourinformationnetworksanddemocraticsystemshavemanagedtodestabilizeinstitutionsandcreatefearandmutualsuspicionamongthepopulation.
(Davies,2018:p.
22)Fakenewsandmisinformationarespreadingwithenormousspeedonline,duetopeople'sconfirmationalbiases,algorithmicboostingandfilterbubbles.
Thesefilterbubblesinparticularcontributetothesocietalpolarizationevenmore.
Socialmediaplatformscreateanenvironmentwhereindividualsfindthemselvesenclosedwithinaninformativebubblethatreinforcestheirpersonalbeliefsandopinionswhileatthesametimesuffocatinganyothernarrativesthatmightchallengethestatusquo.
(Prego,2017:p.
20)AsChesneyandCitron(2018:p.
13)mention,"[f]ilterbubblescanbepowerfulinsulatorsagainsttheinfluenceofcontraryinformation.
"Opposingnarrativesthatenterthesebubblesareimmediatelydiscredited,leavingthefilterbubbleintactintheprocess.
(Prego,2017:p.
20)"Inthisatomizedworldthatisself-strengthening,itisactuallyahugeweaknessDEEPFAKES–ANEMERGINGRISK39becauseitistheperfectbreedinggroundforspreadingfakenews.
"(Ibid:p.
21)Peoplewillnotconsiderfactcheckingtheinformationtheyreceiveintheirbubblesincetheygenuinelybelieveittobetrue,becauseitconfirmstheirownbeliefsaboutacertaintopic.
(Ibid)However,"[o]neoftheprerequisitesfordemocraticdiscourseisashareduniverseoffactsandtruthssupportedbyempiricalevidence.
Intheabsenceofanagreeduponreality,effortstosolvenationalandglobalproblemswillbecomeenmeshedinneedlessfirstorderquestionslikewhetherclimatechangeisreal.
Thelargescaleerosionofpublicfaithindataandstatisticshasledustoapointwherethesimpleintroductionofempiricalevidencecanalienatethosewhohavecometoviewstatisticsaselitist.
"(ChesneyandCitron,2018:p.
21)Onepossiblesolutionforthisproblemmightlieineducatingthepublichowtotellapartfactsfromfictionandincreasingmedialiteracy.
Accordingtodanahboyd,inordertoachievethis,weneedtobeverycreativeanddevelopastructuralbaseforpeopletocommunicatewitheachotheracrossdivisionsinameaningfulway.
(boyd,2017a;boyd,2017b)Thatis,"[w]eneedtoenablepeopletoheardifferentperspectivesandmakesenseofaverycomplicated—andinmanyways,overwhelming—informationlandscape.
Wecannotfallbackonstandardeducationalapproachesbecausethesocietalcontexthasshifted.
Wealsocannotsimplyassumethatinformationintermediariescanfixtheproblemforus,whethertheybetraditionalnewsmediaorsocialmedia.
"(boyd,2017a)Apartfromthisthereisalsothelegalsideofdeepfakestoconsider.
LawmakersandexpertsalreadywarnthatdeepfakesmighthinderanddisrupttheupcomingU.
S.
electionsinDEEPFAKES–ANEMERGINGRISK40theyear2020.
DuringaHouseIntelligenceCommitteehearinginWashingtononthe13thofJune2019,expertsfromvariousfieldsaswellaspoliticianswerediscussingthepotentialdangersofdeepfakesandideashowtopreventthemfromcreatingwidespreaddamages.
(George,2019)UsingtherecentdoctoredvideoofNancyPelosiasanexample,theyarguedthattheeraofdeepfakeswillhave"thecapacitytodisruptentirecampaigns,includingthatforthepresidency.
"(Ibid)Thismightbepossiblegiventhefactthatthepublicisalreadystrugglingtoseparatefactsfromfiction.
Oneexpertsuggestedthatprivatetechcompaniesneedtostepuptheirgameandbansuchcontentfromtheirplatforms.
However,givingthesecompanies'freedomtodecideontheirownwhatkindofcontentshouldberemovedwasseenastooriskyamove.
DanielleCitron,aUniversityofMarylandLawprofessorwhoalsoattendedthemeeting,toldthelawmakersthatmostofthelegislationregulatingtheusageofonlinevideosaredecadesoldandneedurgentrevision.
(Ibid)However,evenwithnewlawsrestrictingthecreationanddistributionofsuchforgeriesitwillnotbeenoughtocombattheentireproblem,notwhenthoselawshavenojurisdictionoutsidetheircountryoforigin.
Forinstance,"U.
S.
officialsdeterminedRussiacarriedoutasweepingpoliticaldisinformationcampaignonU.
S.
socialmediatoinfluencethe2016election.
"(Ibid)Itisnotunimaginablethattheyoranyoneelsewillnottryin2020aswelltomeddlewiththepoliticalcampaignsofpresidentialcandidates.
Forinstance,adeepfakeincriminatingapoliticalcandidatefavoredbythepublicreleasedanightbeforetheelectionscouldpotentiallyfliptheresultsinfavoroftheothercandidate.
Evenifthefakevideowouldbedebunkedandprovenfalse,itmightbetoolate.
"Wheneventsareunfoldingrapidlyandemotionsareridinghigh,thereisasuddenabsenceofanyauthoritativeperspectiveonreality.
"(Davies,2018:p.
xi)Intheheatofthemomentpeoplemightnotstoptoconsiderthefactsbeforeacting.
(Ibid)Deepfakescanindeedfunctionlikeanewwayoffalselyshoutingfireinacrowdedtheater.
Allittakesisjustawell-timeddeepfaketotiptheDEEPFAKES–ANEMERGINGRISK41electioninfavorofoneofthecandidates,"particularlyiftheattackerisabletotimethedistributionsuchthattherewillbeenoughwindowforthefaketocirculatebutnotenoughwindowforthevictimtodebunkiteffectively.
"(ChesneyandCitron,2018:p.
22)Thisisalsobecause"[m]orethananythingelse,thedynamicsthatdefinetheweb—frictionlesssharingandthemonetizationofattention—meanthatdeepfakeswillalwaysfindanaudience.
"(Vincent,2019)Moreover,dependingonthecontextoftheshareddeepfake,thewillingnessofpeopletoacceptfactsthatareinlinewiththeirownopinionsandbeliefswillfurthercreditvideoforgeries.
Afterall,ifthereisalreadysomedoubtamongthepublic,deepfakeswilldeepenthemistrustevenmore.
Thisiswhathappenswithbelieversinconspiracytheories:theyarevulnerabletobelieveinmanyiftheybelieveinone.
(Barkun,2016:p.
2)Ontheotherhand,ifthetargetaudienceisnotchosencarefully,eitherthecontentofthevideowillnotbeveryrelevantforthepopulationorthetimingisnotright,inwhichcasethevideocanbeforgottenwithin24hours.
Thismeansthatapartfromthecontentofthedeepfake,itspublicationandcirculationstrategyisalsokeyindeterminingitspotentialeffects.
However,theeffectsofdeepfakesdonotnecessarilyhavetohappenovernight,asonlineattacksareoftenaformof"slowviolence".
(Varis,2018)"[A]violencethatoccursgraduallyandoutofsight,aviolenceofdelayeddestructionthatisdispersedacrosstimeandspace,anattritionalviolencethatistypicallynotviewedasviolenceatall.
"(Nixon,2013ascitedinVaris,2018)Inotherwords,smalltriggersthatoccurhereandtherecangraduallyovertimecreateanavalancheofmistrustandmutualsuspiciousnessamongthepublic.
Thesecontinuousonlinepersuasionseventuallywearpeopledownandonalargescalehelpstirthepublic'sopiniononacertainpoliticalmatter.
(Varis,2018)Modernpoliticalcampaignersdeploythisstrategyveryoften.
Theyareveryawarethatpublicopinioncanbebestswayedwithsmall-scaleinterventions,whichsometimesgounnoticed,ratherthanbigformalpublicstatements.
(Davies,2018:p.
13)Therefore,acarefullyexecutedmisinformationcampaignDEEPFAKES–ANEMERGINGRISK42overacertainperiodoftimewilleventuallyleadtoamuch-desiredresult.
Andonceitmanagestopolarizeasociety,thethreatstothedemocraticsystemsstarttobecomevisible.
(ChesneyandCitron,2018:p.
29)"Ifthepubliclosesfaithinwhattheyhearandseeandtruthbecomesamatterofopinion,thenpowerflowstothosewhoseopinionsaremostprominent—empoweringauthoritiesalongtheway.
"(Ibid)Insuchavulnerablestatewhenoursocietyissufferingfromtruthandtrustdecay,authoritarianleaderswillstrivetoexploitthepublic'sopinionevenfurther.
(Ibid)ThisisalreadyhappeningallaroundEurope.
Whatwearewitnessingistheriseofpopulism,whichhasmanagedtodivideaoncemoreintegratedunionofEuropeancountrieswithopenbordersandasharedcurrencyintosmallerconflictedislets.
Populistleaders,suchasViktorOrbaninHungary,usethevulnerabilityoftheEuropeanUnionandspecificallytherefugeecrisistowidenthedivideevenmore.
Hismisinformationcampaignspecificallytargetspoliticalopponents,theacademia,themedia,NGO's,prominentindividualsandbasicallyeveryonewhodoesnotsupporthispopulistideasandwhoareinpositiontohaveaninfluenceonsociety.
(Shattuck,2019)Ifthecredibilityofindividualsandinstitutionswhohavethecapacitytoproduceandverifyknowledgeandinformationareundermined,thepublicisleftwithnochoicebuttobelievethosewhosewhoholdthepower,resultingintheerosionofdemocraticsystems.
(ChesneyandCitron,2018:p.
29)Inthispost-truthclimate,deepfakesareexceptionallydangerous.
Toputitinperspective:thisclimategavebirthtovariousmisinformationcampaignswhichcontributedtoBrexitandTrumpbecomingapresidentinthefirstplace.
Andthiswasallpossiblebecausethepublicisalreadystrugglingtomakeadifferencebetweenfactsandfiction.
Thisenvironmentisespeciallyvulnerabletoallkindsoffakenews,conspiracytheoriesandmisinformation,becausepeoplewhoencounterthemwillactfirstandstartaskingquestionslateror,intheworstcasescenario,neveratall.
YetthecapacityofdeepfakestomimicDEEPFAKES–ANEMERGINGRISK43realityandfoolnotjustoursensesbutalsoourtechnologicaltoolscreatedtocounterthemgobeyondanyotherformsoffakenews,misinformationcampaignsorlies.
Evenjustonerealisticvideoforgerycouldhavemind-blowingeffectsforwhichourdemocraticsystemsarejustsimplynotpreparedfor.
DEEPFAKES–ANEMERGINGRISK445.
ConclusionTherealwaystendstobeageneralmoralpanicaboutnewtechnologies,butveryoftenitisamisplacedworry.
(boyd,2012)However,whenitcomestodeepfakesthepublicanxietythatfollowsisactuallywell-based.
Atechnologythatiscapabletofooloursensesandevensoftwareandalgorithmscreatedtocounteritissomethingtobeworriedabout.
(ChesneyandCitron,2018:n.
p.
)Andsinceweliveinapost-trutherawherepeopleareunabletotellapartfactsfromfiction,andwhereemotionsandpersonalbeliefsoftendictateourknowledgeabouttheworld,evenacrudelydoctoredvideocouldtrickpeopleintobelievingitisreal.
(OxfordDictionaries,n.
d.
;McIntyre,2018:chapter1,paragraph12;Llorente,2017:p.
9)Despitebeingarelativelynewphenomenon,deepfakesalreadymanagedtogainquiteanegativereputationinWesternsocieties.
(ChesneyandCitron,2018)Thisisduetothefactthatsomanyknownandunknownpeoplehavefallenvictimstothistechnology.
Whilethereisnodenyingthatdeepfakescanhavesomebeneficialuseslikeforinstanceineducation,Hollywoodorinthegamingindustryasitwasmentionedabove,stillthedifferentpossibilitiesofcausingharmtoindividualsandsocietiesalikeiswhatweshouldbefocusingoninstead.
Afterall,givenourweakinformationnetworksandcurrentsocietaldevelopmentswhichgaverisetomisinformationcampaignsandmutualmistrustamongthepublic,ourdemocraticsystemsaresimplynotpreparedfordeepfakesatall.
(Ibid:16)Yetsurprisinglytherehavenotbeenanymajoracademicstudiesfocusingonthisparticularphenomenon.
Therefore,theaimofthisthesiswastooutlinethepossiblesocial,culturalandpoliticalramificationsdeepfakesmighthavebothonsocietalandindividuallevelsthroughadiscussionofspecificcasestudies.
ThisfinalchapterpresentsthefindingsoftheanalysesDEEPFAKES–ANEMERGINGRISK45basedonthesecasesbysummarizingthecentralpointsandprovidingadditionalsuggestionsregardingfuturedirections.
Whenitcomestotheirpotentialtoharmindividuals,weneedtotalkaboutthedifferentmotivationsbehinddeepfakesandtheirvariousoutcomesforvictimsandtheirenvironments.
Themotivationsbehindeachdeepfakecanbevarious,dependingmainlyonthecreativeimaginationofitscreator(s).
However,basedonthecasesreviewedinthisthesiswecanconcludethatthusfarthemajorityofdeepfakeshasbeencreatedwiththeintentiontoshameandsilenceindividuals,totakerevengeonsomeone,forbothpersonalandpoliticalreasons;andveryoftentoupholdthevaluesandnormsinasociety.
Thislatterisspecificallythecasewhenitcomestoshamingandsilencingwomeninsociety.
Althougheveryonecanbecomeavictimofsuchtechnologicalforgery,somearemorevulnerablethanothers.
(SinghandRoseen,2018)Thecasesofmanyknownandunknownwomenwhofallvictimtofakepornvideosindicatethatthereisagendereddimensionofsexualexploitationbydeepfakeswhichisveryworrisome.
(ChesneyandCitron,2018:p.
17)Womenareamongthemostvulnerablegroups,incertainsocietiesalsomorethaninothers.
Thisisalsotruefortheonlineenvironment.
Vulnerablegroupswhoarealreadybulliedofflinehaveahigherchanceofbecomingvictimsofonlineabusesaswell.
Sincethelinesbetweenouronlineandofflinelivesareblurred,duetothefactthatdigitalizationhascreatedanunpredictableandeverevolvingtechnologicallandscape,onlinethreatscaneasilybetransferredtotheofflineworldaswell.
(SinghandRoseen,2018)Andgiventhenatureofoursocialmediaplatforms,whereeverybodycanuploadandshareallkindsofinformationveryoftenwithoutanyorwithfewrestrictions,thisenvironmentisratheridealfororganizedandsystematicbullyingandabuse.
(Ibid)Thisclimatealsoallowsperpetratorstobeanonymousandstayhidden,makingitevenharderforthevictimtosucceedinseekingjusticefortheharmthathasbeendonetoher.
Additionally,ourlegalsystemswhicharelaggingfarbehindourcurrenttechnologicalDEEPFAKES–ANEMERGINGRISK46developmentmakethismatterevenworse,sincethevictimsareleftalonetodealwiththeirownproblemswhiletheperpetratorsareneverevenpersecuted.
(George,2019;Curtis,2018)Ontheotherhand,theoutcomesofdeepfakescanbealsodifferentforeachindividual.
Deepfakescancausevictimstoexperiencevariouspsychologicalharms,suchasfear,anxietyorwhenitcomestofakepornvideos,victimscanbealsosubjectedtovisualizationoftheirownsexualharassment,whichcanbealsodescribedasnon-consensualvirtualsex.
(ChesneyandCitron,2018:pp.
16-20)Oneofthecommonoutcomesofdeepfakesistheretreatofvictimsfromsocialmediaplatforms.
(SinghandRoseen,2018)Thisisspecificallythecaseforpornographicdeepfakeswheretheintentionsareveryoftentoshameandsilencewomenandtopreservethestatusquoinasociety.
Likeotherformsofonlineabusewhichtargetwomen,deepfakebullyingcanresultinwomenrefusingtospeakoutabouttheirexperienceandinsomecasestheychoosetoremovethemselvesfromtheonlineenvironmentaltogether.
Sadly,thisisalsonotspecifictowomenalone.
Othervulnerablegroupswhomightbetargetedbydeepfakescanhavethesameorsimilarexperiences.
(Ibid)Althoughnowadayseveryonecanhaveaccessandshareinformationonline,duetothedigitalrevolutionthatdemocratizedtheproductionofknowledge,noteverybodyhasthesameamountofpowertotalkaboutallkindsoftopics.
(HanellandSal,2017)Somepeoplewhoaremarginalizedofflinecanalsobeinvisibleonlineaswell.
However,beingvisibledoesnotalwaysgohandinhandwithbenefits.
Thisisbecausevisibilitycanbeempoweringasmuchasdisempowering.
Thelatterisespeciallytrueincaseswheremarginalizedindividualsarepushedoverthefairvisibilitythresholdintothezoneofsupra-orsuper-visibility.
Inthisenvironmentvulnerableindividualsdonotgettochoosehowtheyarerepresentedonlineleadingtodistortionsintheirdigitalvisibility.
(Brighenti,2007:pp.
330-335)DEEPFAKES–ANEMERGINGRISK47Reputationalsabotageisanotheroutcomethatcanbeattributedtodeepfakeabuse.
Visibilityofallkindsofinformationhasdrasticallychangedduetodigitalizationthatledtoinformationonlinebeingdistributedinanalgorithmicmannerandsharedonsocialmediaplatforms,oftenshapedbyfilterbubbles.
(HanellandSal,2017:pp.
154-156;ChesneyandCitron,2018:p.
13)Yet,thisenvironmentalsoincreasedthechancesofcertainmisinformationandfakenewstobecomeaviralphenomenon,whichcouldpotentiallycauseharmtobothindividualsandsocieties.
Whilesomedeepfakesmightbeshort-livedandforgottenwithin24hours,othersmighthavelong-termeffectsonpeople.
Especiallyreputationalharmcanbeverydangeroussinceitcanhavemajorlong-termramificationsindifferentdomainsofone'slife.
Anindividualwhofallsvictimtoadeepfakemightlosehisorherjob,losethesupportandtrustoffriendsandfamilymembers,losepartnersandspouses,orbecastoutofthecommunityaltogether.
Inaworstcasescenariovictimsmightfeelentirelyhumiliatedandleftallalone,blamingthemselvesforthesituationstherearein.
Thiscanmakeitevenharderforthemtorecoverandrestorenormalcyintheireverydaylife.
(ChesneyandCitron,2018:pp.
18-20)Moreover,itisnotnecessarilyonlythevictimthatwillbeaffectedbyadeepfake.
Insomecases,deepfakesthattargetspecificindividualscanhavewiderimplicationstothepublicsphereaswell,suchasdividingsocietyintofractionsandevenposingathreattothehealthofourdemocraticsystems.
Peoplewhoareclosetothevictim,likefamilyandfriendsortheircommunity,mightsufferaswell.
Insomeinstances,theaftermathofadeepfakecanbealsofeltonasocietallevel.
Adeepfakeintendedtoharmanindividualcantriggerfrictionsinsociety,furtherdeepeningpossibleexistingsocietaldivisionandcausingpotentiallyevenmajordisturbancestoitsdemocraticsystems.
(ChesneyandCitron,2018)Deepfakes'potentialtoharmsocietieshavefirstofallsystematiccharacteristics,duetothefactthattheireffectscanbefeltonalllevelsofsociety.
HereaswellwecantalkaboutDEEPFAKES–ANEMERGINGRISK48bothvariousmotivationsonbehalfofthecreatorsaswellastypesofoutcomesdeepfakesmighthave.
Themotivationsbehinddeepfakescanvaryendlessly.
Aforgerymightbecreatedtoshapepublicopinionaboutacertainpoliticalmatter,tocreateinstabilityinsociety,todeepenthemistrustofpeopleinprivateandpublicinstitutions,ortodisruptentiresocieties.
Somedeepfakesarecreatedwiththeintentiontostrengthenapublic'sbeliefinaconspiracytheoryortoconfirmtheirdoubtsaboutacertainpoliticalissue,whileothersareusedtodiscreditprominentindividualsormediaoutlets.
Andagain,certaindeepfakescanbesimplycreatedtodrawthepublic'sattentiontocertainproblemswhichareofoutmostimportancetothem.
(ChesneyandCitron,2018:p.
21)Whenitcomestoperpetrators,theycanremainanonymous,buttheycanalsobeannouncingtheirinvolvementinthecreationand/ordistributionofdeepfakes.
Thisisverytrueincaseswhenthecreatorshavesomethingtogainbyrevealingtheirtrueidentity.
Ontheotherhand,deepfakesmightbecreatedbyindividuals,buttheycanalsobeapartofstate-organizedcampaigns.
Theconsequencesofdeepfakesonsocietallevelalsovary.
Deepfakesamongotherthingscanleadtomajorchangesinthecourseofelections,damagestointernationalrelations,beingusedasweaponsforwarorterrorism,threatstonationalsecurity,disruptionsofeconomicdevelopments,lossoftrustinpublicinstitutions,polarizationofsocietyandthreattothedemocraticsystemaltogether.
(ChesneyandCitron,2018:p.
21)Whilesomeeffectsofdeepfakesmighthappenovernight,likeforinstanceadeepfakeintendedtodiscreditapoliticalfigurepostedadaybeforeelectionspotentiallyimpactingthewaypeoplevote,othersmighthavelong-termeffects.
Theselong-termeffectscanbedescribedas"slowviolence"tothepublicsphereordemocracy,sincetheyresultinminordisruptionsoveralongerperiodoftime,whichisthecaseforinstancewiththepolarizationofsociety,erodingDEEPFAKES–ANEMERGINGRISK49trustininstitutionsanddisruptionstodemocraticsystems.
(Varis,2018;ChesneyandCitron,2018:p.
29)Deepfakesintendedtodisruptthenormalcyofdemocraticsystemsmightcauseastateofnervousnessinasociety.
(Davies,2018:p.
xii)Asdigitalizationresultedintherevolutionofhowinformationisproducedandsharedamongpeople(HanellandSal,2017:pp.
155-156),italsoresultedincreationofastateofcontinuousandheightenedalertnessofindividualsandsocietiesalike.
Informationonlinespreadswithenormousspeed,especiallymisinformationandfakenews.
Thisisduetothefactthat,inthisclimatepeoplerelymostlyontheirfeelingsandpersonalbeliefsdisregardingtheobjectivefactsintheprocess.
Thispublicanxietycanresultinpeoplebeingscared,feelinginsecure,allthewaytoangryoutburstsofcertainindividualsorverbalaggression.
Thisnervousnessiswhatfollowsfakenewsandmisinformationcampaignsonline.
(Davies,2018:pp.
7-17)Intheworstcasescenariopeoplewhowerefooledbyafakevideomightbemotivatedtoactoutandtakethematterintotheirownhands,whichofcoursecanleadtophysicalviolenceaswell.
Polarizationofsocietyisoftenseenasamajorblowtothehealthofdemocraticsystems.
Divisionsthatalreadyexistinasocietycanbedeepenedbydeepfakesevenmore.
(ChesneyandCitron,2018:p.
29)Filterbubblescanfurtheraggravatesuchdivisions.
Bycreatingenclosedspaceswhereinformationisusedtoreinforcepeople'sownopinionsandbeliefs,theypresentaperfectbreedinggroundforspreadingmisinformation,fakenewsanddeepfakes.
Veryoftenopposinginformationdoesnotappearinsuchfilterbubblesorevenifitdoes,itistodiscreditit,whileatthesametimestrengtheningtheexistingconfirmationalbiasesofpeople.
(ChesneyandCitron,2018:p.
13;Prego,2017:pp.
20-21)Thisisespeciallytrueduetothefactthathumansaremoreinterestedinnegativeandnovelinformationratherthanrealfacts.
(ChesneyandCitron,2018:p.
12)Moreover,informationthatfeelsandlookstruewilltravelfasteronline,furthercontributingtotheproblem.
(Davies,2018:p.
15)AndifDEEPFAKES–ANEMERGINGRISK50membersofacertainsocietyfailtoagreeuponwhatisrealandwhatisfake,theninsteadoffindingsolutionstoproblemsthefocuswillbeondecidingwhetherthoseproblemsarearealissueinthefirstplace.
Apublicthatisunabletoagreeuponrealityitselfisveryvulnerabletoexploitationsandmanipulations.
Inthistoxicenvironment,authoritarianfiguresholdingthepowerwillbeabletodictatewhatistrueandwhatnot,resultinginthecreationofauthoritarianismthatpresentsarisktoourdemocraticsystemsasweknowthem.
(ChesneyandCitron,2018:pp.
21-29)Andwhiletheoutcomesofusingdeepfakescanalsobeverydifferent,theycanalsobeunpredictableandarenotalwayssomethingthatthecreatorshadintendedtoachieve.
Thisisduetothefactthatcreatorsofdeepfakescannotalwaysanticipateaccuratelyhowpeoplemightperceiveandreactontheseforgeries,orhowtheycirculateforinstanceonline.
Inorderforacertaindeepfaketohavethedesiredeffectonsocietyitscontent,thetargetedaudienceandthetimingneedstobecarefullychosenandeventhen,thereisnoguaranteethatthingsmightturnoutthewaythecreatorswantedto.
Tobeabletodeterminethepotentialimplicationsofdeepfakesweneedtoexploretheircontent,theirpublicationandcirculationstrategyaswellasthereactionsofpeoplewhoencounteredthem.
Whenitcomestopossiblesolutionsthatmightstopdeepfakesfromcausingharm,thefactthattherearenolawsyetinmanycountriesaroundtheworldthatallowtorecognizedeepfakesasaformof(serious)crimemeansthatbothindividualsandourdemocraticsystemsareleftdefenselessagainstsuchattacks.
However,evenifmorecountrieswouldintroducelegislationtacklingthemisuseofprivatephotosofpeopleforthepurposesofcreatingforgedimagesandvideos,theproblemwouldnotbecompletelysolved.
Thereasonforthisiseitherthatperpetratorscangounidentifiedorcanoperatefromcountrieswherelawsagainstdeepfakesdonotexistyet.
(Reichert,2018;Curtis,2018)Ontheotherhand,onceoutontheinternet,deepfakeswouldbehardifnotimpossibletoremove,giventhefactDEEPFAKES–ANEMERGINGRISK51thatmanypeoplecanpotentiallyviewthemandsharethemfurtherwithininaveryshortamountoftime.
Thelackofsupervisionandgatekeepersonsocialmediaplatformsalsoaggravatesthisissue.
(Madeiros,2017:p.
23)However,thequestionis,shouldsocialmediaplatformsbetheonesdecidingwhatkindofcontentneedstoberemovedinthefirstplaceCanwetrustcompaniessuchasFacebookorGoogletomakeobjectiveandunbiasedjudgmentsaboutthismatterOntheotherhand,appointingthirdpartiestosuperviseonlinecontentsuchasgovernments,expertsinAIorinternationalorganizationsmightjustleadtounwantedcensorshiporbiaseddecisions,whichagainaredangerousoutcomes.
Intheendthequestionstillremainswhoseresponsibilityitisthentostopdeepfakesfromspreadingonline.
Apossiblesolutionmightincludeajointactiontotackletheproblem.
Thismeansrestrictionsonsocialmediaplatforms,legalframeworkstopersecuteperpetratorsandmostimportantlyincreasingmedialiteracyinconnectionwithpropereducationofthepubliconmakingacleardistinctionbetweenfactsandfiction,whileatthesametimefocusingonbuilding"socialinfrastructures"wherepeoplewouldbeabletomeaningfullyengagewitheachother.
(boyd,2017a)Thesenewinfrastructureswouldbelikebridgesconnectingdiversepartsofsocietyandatthesametimereducingsocialpolarization.
(boyd,2017b)Andtheselastthreesolutionsmightjustbethemostimportantincounteringthepotentialeffectsofdeepfakes.
However,despitetheabove-mentionedconclusionsandfindings,certainlimitationsandflawsremainregardingthisresearch.
Asitwasalreadypointedout,itisimpossibletomakedefinitegeneralizationsabouttherisksdeepfakesposetoindividualsandsocietiesbasedonsolelythecasestudiesexploredinthisthesis.
Althoughthecasesgiveadetailedinsightintospecificaspectsofdeepfakes,theyarestilllimitedtotheirspecificcontext.
Whileinthisthesisitwasarguedthatweshouldbemainlyfocusingontheharmfuleffectsofdeepfakes,thisdoesnotmeanthattheirbeneficialusesshouldbeignored.
Onthecontrary,futurestudiesmightfinditinterestingtoexploretheseaspectsofdeepfakesaswell.
GivenDEEPFAKES–ANEMERGINGRISK52themanysocial,culturalandpoliticalramificationsofdeepfakes,futurestudiesinthisfieldneedtofurtherexplorethemotivationbehindsuchvideoforgeriesandthepotentialoutcomesinordertofindwaystocounterthemoratleasttominimizethedamagetheywillcreate.
Sincethisthesismostlyfocusedonwomenastheindividualvictimsofdeepfakes,futureresearchpaperscouldexploreinmoredetailothervulnerablegroupsinconnectiontodeepfakes.
Apossiblecomparisonbetweenthegendersisalsoanotherinterestingwaytohighlightevenmoretheissueofthegendereddimensionofthistechnology.
Anotherinterestingaspecttoexploremightbetheconsequencesdeepfakescanhaveonthevictim'sfamilymembers,friends,community,andonthepublicsphere.
Morecasestudiesaboutthepossibleimplicationsofdeepfakesconcerningsocietywouldalsowidenourunderstandingofthisphenomenon.
Also,aninterestingtopictoconsideristhesocial,culturalandpoliticalramificationsoftechnologicalsolutions,suchassoftwareoralgorithms,designedtocounterdeepfakes.
Finally,futurestudiescouldfocusonfindingsolutionstotackletheloomingchallengesofdeepfakes.
Thismightincludeexploringcurrentandproposedlegislativemeasures,theresponsibilitiesofsocialmediaplatforms,governmentsandtechgiantsindecidingwhatkindofcontentshouldberemovedfromtheinternet,possibilitiesofeducatingthepublicabouttheharmfuleffectofdeepfakes,increasingmedialiteracy,andfindingnewwaystobridgethedifferencesinasocietybybuildingamoreconnectedandfunctionalsocialfabric.
DEEPFAKES–ANEMERGINGRISK536.
ReferencesAisch,G.
,Huang,J.
andC.
Kang.
(2016).
Dissectingthe#PizzaGateconspiracytheories.
Retrievedfromhttps://www.
nytimes.
com/interactive/2016/12/10/business/media/pizzagate.
htmlAratani,L.
(2018).
AlteredvideoofCNNreporterJimAcostaheraldsafuturefilledwith'deepfakes'.
Retrievedfromhttps://www.
forbes.
com/sites/laurenaratani/2018/11/08/altered-video-of-cnn-reporter-jim-acosta-heralds-a-future-filled-with-deep-fakes/#56ebf9053f6cAyyub,R.
(2018).
Iwasthevictimofadeepfakepornplotintendedtosilenceme.
Retrievedfromhttps://www.
huffingtonpost.
co.
uk/entry/deepfake-porn_uk_5bf2c126e4b0f32bd58ba316guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAAKGJ5jRAo6u7sRJLGcx3zlwLIo3vLyHj3PkFE_MotDg_Ls0n3XPxmauv7B9PP_vn8-U3mZn_recRZCommm0QA-2iJxpEFPJ9NLFxTYFwVMr81GqijH7mucQ2cFxplSBxkF2lyTbi5FeBotUaX8R2e9IDHAcPYY9SJ0-QuZtE9JIxAyyub,R.
(2019).
YouknowIndia'sdemocracyisbrokenwhenmillionswaitforelectionresultsinfear.
Retrievedfromhttps://www.
dailymaverick.
co.
za/article/2019-05-17-you-know-indias-democracy-is-broken-when-millions-wait-for-election-results-in-fear/DEEPFAKES–ANEMERGINGRISK54Barkun,M.
(2016).
Conspiracytheoriesasstigmatizedknowledge.
Diogenes:pp.
1-7.
Bell,E.
(2019).
AtWhiteHousepressconferences,noquestionsallowed.
Retrievedfromhttps://www.
theatlantic.
com/politics/archive/2019/01/donald-trump-continues-to-call-the-media-fake-news/579670/boyd,m.
d.
(2008).
Takenoutofcontext:Americanteensocialityinnetworkedpublics[PDFversion].
Retrievedfromhttps://papers.
ssrn.
com/sol3/papers.
cfmabstract_id=1344756boyd,d.
(2012).
Thepoweroffearinnetworkedpublics[transcriptionofatalk].
SXSW.
Austin,TexasRetrievedfromhttp://www.
danah.
org/papers/talks/2012/SXSW2012.
htmlboyd,d.
(2017a).
DidmedialiteracybackfireRetrievedfromhttps://points.
datasociety.
net/did-media-literacy-backfire-7418c084d88dboyd,d.
(2017b).
WhyAmericaisself-segregating.
Retrievedfromhttps://points.
datasociety.
net/why-america-is-self-segregating-d881a39273abBrandom,R.
(2019).
Deepfakepropagandaisnotarealproblem.
Retrievedfromhttps://www.
theverge.
com/2019/3/5/18251736/deepfake-propaganda-misinformation-troll-video-hoaxBrighenti,A.
(2007).
Visibility:AcategoryfortheSocialSciences.
CurrentSociology.
55(3):pp.
323–342.
Carlsson,U.
andR.
Pyhtri(eds.
).
(2017).
Theassaultonjournalism:Buildingknowledgetoprotectfreedomofexpression[PDFversion].
Retrievedfromhttp://www.
unesco.
se/wp-content/uploads/2017/04/The-Assault-on-Journalism.
pdfDEEPFAKES–ANEMERGINGRISK55Chesney,R.
andD.
K.
Citron.
(2018).
DeepFakes:Aloomingchallengeforprivacy,democracy,andnationalsecurity[Draftversion].
Retrievedfrom:https://ssrn.
com/abstract=3213954Collins,T.
(2018).
Trump'sitchyTwitterthumbshaveredefinedpolitics.
Retrievedfromhttps://www.
cnet.
com/news/donald-trump-twitter-redefines-presidency-politics/Curtis,C.
(2018).
Deepfakesarebeingweaponizedtosilencewomen—butthiswomanisfightingback.
Retrievedfromhttps://thenextweb.
com/code-word/2018/10/05/deepfakes-are-being-weaponized-to-silence-women-but-this-woman-is-fighting-back/Davies,W.
(2018).
Nervousstates:Howfeelingtookovertheworld.
London:JonathanCape.
George,S.
(2019).
'Deepfakes'callednewelectionthreat,withnoeasyfix.
Retrievedfromhttps://www.
apnews.
com/4b8ec588bf5047a981bb6f7ac4acb5a7Gillespie,T.
(2014).
Therelevanceofalgorithms.
InGillespie,Tarleton,PabloJ.
BoczkowskiandKirstenA.
Foot(eds.
)Mediatechnologies:Essaysoncommunication,materialityandsociety.
MITScholarshipOnline:pp.
167-193.
Gooch,E.
(2017).
Inpursuitofthetruth.
UNO.
ThePost-truthEra:Realityvs.
Perception.
17:pp.
14-15.
Gordon,N.
(2019).
ScarlettJohanssononhowhelplessshefeelsaboutherimagebeingusedinfakepornvideos.
Retrievedfromhttps://www.
cosmopolitan.
com/uk/reports/a25722524/scarlett-johansson-helpless-face-fake-porn-videos/Hanell,L.
andL.
Sal(2017).
Ninemonthsofentextualizations.
Discourseandknowledgeinanonlinediscussionforumthreadforexpectingparents.
InKerfoot,CarolineandDEEPFAKES–ANEMERGINGRISK56KennethHyltenstam(eds.
)Entangleddiscourses.
South-Northordersofvisibility.
London:Routledge:pp.
154-170.
Hao,K.
(2019).
AnAIappthat"undressed"womenshowshowdeepfakesharmthemostvulnerable.
Retrievedfromhttps://www.
technologyreview.
com/s/613898/an-ai-app-that-undressed-women-shows-how-deepfakes-harm-the-most-vulnerable/Harwell,D.
(2018).
Fake-pornvideosarebeingweaponizedtoharassandhumiliatewomen:'Everybodyisapotentialtarget'.
Retrievedfromhttps://www.
washingtonpost.
com/technology/2018/12/30/fake-porn-videos-are-being-weaponized-harass-humiliate-women-everybody-is-potential-target/utm_term=.
5c9026480758Harwell,D.
(2019).
FakedPelosivideos,slowedtomakeherappeardrunk,spreadacrosssocialmedia.
Retrievedfromhttps://www.
washingtonpost.
com/technology/2019/05/23/faked-pelosi-videos-slowed-make-her-appear-drunk-spread-across-social-media/noredirect=on&utm_term=.
e54e8a1b6041Hergott,M.
(2019).
Aleapintothefuture:Generativeadversarialnetworks.
Retrievedfromhttps://medium.
com/datadriveninvestor/a-leap-into-the-future-generative-adversarial-networks-96a780ed8ee6Hollinger,J.
(2017).
Trump,socialmediaandthefirstTwitter-basedpresidency.
Retrievedfromhttps://www.
diggitmagazine.
com/articles/Trump-Twitter-Based-PresidencyLamb,K.
(2018).
Indonesiangovernmenttoholdweekly'fakenews'briefings.
Retrievedfromhttps://www.
theguardian.
com/world/2018/sep/27/indonesian-government-to-hold-weekly-fake-news-briefingsDEEPFAKES–ANEMERGINGRISK57Laschon,E.
(2018).
RevengeporncrackdownannouncedbyWAGovernmentoffershopeforvictims.
Retrievedfromhttps://www.
abc.
net.
au/news/2018-06-28/revenge-porn-crackdown-announced-by-wa-government/9920560Llorente.
A.
J.
(2017).
Thepost-truthera:Realityvs.
perception.
UNO.
ThePost-truthEra:Realityvs.
Perception.
17:p.
9.
Lyons,K.
andK.
Lamb.
(2018).
Sulawesitsunami:Indonesiabattlesfakenewsashoaxersspreadpanic.
Retrievedfromhttps://www.
theguardian.
com/world/2018/oct/04/sulawesi-tsunami-indonesia-battles-fake-news-as-hoaxers-spread-panicMadeiros,A.
(2017).
Thedangerofindifferencetotruth.
UNO.
ThePost-truthEra:Realityvs.
Perception.
17:pp.
23-25.
Markman,A.
(2018).
Whypeoplearen'tmotivatedtoaddressclimatechange.
Retrievedfromhttps://hbr.
org/2018/10/why-people-arent-motivated-to-address-climate-changeMcIntyre,L.
(2018).
Post-truth[KoboAuraversion].
Retrievedfromhttps://www.
kobo.
com/nl/enMedrán,A.
(2017).
Inthekingdomofpost-truth,irrelevanceisthepunishment.
UNO.
ThePost-truthEra:Realityvs.
Perception.
17:pp.
33-35.
Mezzofiore,G.
(2018).
No,EmmaGonzalezdidnottearupaphotooftheConstitution.
Retrievedfromhttps://edition.
cnn.
com/2018/03/26/us/emma-gonzalez-photo-doctored-trnd/index.
htmlMikkelson,D.
(2018).
WasEmmaGonzálezfilmedrippinguptheU.
S.
ConstitutionRetrievedfromhttps://www.
snopes.
com/fact-check/emma-gonzalez-ripping-up-constitution/DEEPFAKES–ANEMERGINGRISK58Mondal,S.
(2017).
WhywasGauriLankeshkilledRetrievedfromhttps://www.
nytimes.
com/2017/09/13/opinion/gauri-lankesh-india-dead.
htmlO'Grady,S.
(2018).
AnIndianjournalisthasbeentrolledforyears.
NowU.
N.
expertssayherlifecouldbeatrisk.
Retrievedfromhttps://www.
washingtonpost.
com/news/worldviews/wp/2018/05/26/an-indian-journalist-has-been-trolled-for-years-now-u-n-experts-say-her-life-could-be-at-risk/utm_term=.
fddc4b5773e0OxfordDictionaries,(n.
d.
).
Post-truth.
Retrievedfromhttps://en.
oxforddictionaries.
com/definition/post-truthPrego,V.
(2017).
Informativebubbles.
UNO.
ThePost-truthEra:Realityvs.
Perception.
17:pp.
20-21.
Reichert,C.
(2018).
Australiapasses'revengeporn'legislation.
Retrievedfromhttps://www.
zdnet.
com/article/australia-passes-revenge-porn-legislation/Schenk,M.
(2018).
Fakenews:BelgiansocialdemocratpartyusesfakedTrumpvideoinclimatechangecampaign.
Retrievedfromhttps://hoax-alert.
leadstories.
com/3469396-fake-news-spa-on-twitter.
htmlSchwartz,O.
(2018).
YouthoughtfakenewswasbadDeepfakesarewheretruthgoestodie.
Retrievedfromhttps://www.
theguardian.
com/technology/2018/nov/12/deep-fakes-fake-news-truthShattuck,J.
(2019).
HowViktorOrbandegradedHungary'sweakdemocracy.
Retrievedfromhttps://theconversation.
com/how-viktor-orban-degraded-hungarys-weak-democracy-109046DEEPFAKES–ANEMERGINGRISK59Silverman,C.
(2018).
HowtospotadeepfakeliketheBarackObama–JordanPeelevideo.
Retrievedfromhttps://www.
buzzfeed.
com/craigsilverman/obama-jordan-peele-deepfake-video-debunk-buzzfeedutm_term=.
bgWEL9pQjg#.
fs3VQGmReqSingh,SandD.
Roseen.
(2018).
Online,vulnerablegroupsonlybecomemorevulnerable.
Retrievedfromhttps://www.
newamerica.
org/weekly/edition-229/online-vulnerable-groups-only-become-more-vulnerable/Snapes,L.
(2018).
BataclansurvivorJesseHughescallsMarchforOurLives'pathetic'.
Retrievedfromhttps://www.
theguardian.
com/music/2018/mar/26/eagles-of-death-metal-jesse-hughes-march-for-our-lives-bataclanutm_term=Autofeed&CMP=twt_b-gdnnews#link_time=1522064213Solsman,E.
J.
(2019a).
Deepfakesmayruintheworld.
Andtheycancomeforyou,too.
Retrievedfromhttps://www.
cnet.
com/news/deepfakes-may-try-to-ruin-the-world-but-they-can-come-for-you-too/Solsman,E.
J.
(2019b).
SamsungdeepfakeAIcouldfabricateavideoofyoufromasingleprofilepic.
Retrievedfromhttps://www.
cnet.
com/news/samsung-ai-deepfake-can-fabricate-a-video-of-you-from-a-single-photo-mona-lisa-cheapfake-dumbfake/Sundar,S.
(2016).
WhydowefallforfakenewsRetrievedfromhttps://theconversation.
com/why-do-we-fall-for-fake-news-69829Swaine,J.
(2017).
DonaldTrump'steamdefends'alternativefacts'afterwidespreadprotests.
Retrievedfromhttps://www.
theguardian.
com/us-news/2017/jan/22/donald-trump-kellyanne-conway-inauguration-alternative-factsTheStarOnline.
(2018).
Policeurgepublicnottofallforfakenews.
Retrievedfromhttps://www.
thestar.
com.
my/metro/metro-news/2019/03/22/police-urge-public-not-to-fall-for-fake-news/DEEPFAKES–ANEMERGINGRISK60Varis,P.
(2018).
WhatisthewholesomeinternetHowwholesomememesbecameatrend.
Retrievedfromhttps://www.
diggitmagazine.
com/column/wholesome-internet-memesVincent,J.
(2019).
Deepfakedetectionalgorithmswillneverbeenough.
Retrievedfromhttps://www.
theverge.
com/2019/6/27/18715235/deepfake-detection-ai-algorithms-accuracy-will-they-ever-workWagner,J.
(2018).
Trumprenewsattacksonmediaas'thetrueEnemyofthePeople'.
Retrievedfromhttps://www.
washingtonpost.
com/politics/trump-renews-attacks-on-media-as-the-true-enemy-of-the-people/2018/10/29/9ebc62ee-db60-11e8-85df-7a6b4d25cfbb_story.
htmlutm_term=.
77979ed9510aWorldEconomicForum,(2018).
Theglobalgendergapreport2018[PDFversion].
Retrievedfromhttp://www3.
weforum.
org/docs/WEF_GGGR_2018.
pdfZainal,Z.
(2007).
Casestudyasaresearchmethod.
Retrievedfromhttps://www.
researchgate.
net/publication/41822817_Case_study_as_a_research_methodZarzalejos,A.
J.
(2017).
Communication,JournalismandFact-checking.
UNO.
ThePost-truthEra:Realityvs.
Perception.
17:pp.
11-13.
Zuckerman,E.
(2017).
Fakenewsisaredherring.
Retrievedfromhttps://www.
dw.
com/en/fake-news-is-a-red-herring/a-37269377