217.320traceroute

AcuriouscaseofbrokenDNSresponsesBabakFarrokhiRIPE75AboutmeUnixSA(FreeBSD,Solaris,Linux)since1996IPNetworkingsince1997FreeBSDPortsTeamsince2004Enthusiasticcoder@farrokhiPrologueWhenitcomestonetwork,IalwayshavetrustissuesMostpeopleignorethosestrangenetworkbehaviorsOnlyafewpeopletaketheredpillandgodowntherabbithole.
.
.
ObservationOutgoingSMTPfailsduetoMXlookupfailuresonlycertaindomains(e.
g.
twitter.
com)Localresolverreturns"incorrect"responsePublicResolver(e.
g.
Google)alsoreturnedincorrectresponseIneededtolookdeeperintothisDowntherabbithole.
.
.
StrangeresponsesfrompublicresolversThisisnotwhatIexpectedtogetfromapublicresolver:%dig+short-tAtwitter.
com@8.
8.
8.
810.
10.
34.
34%dig+short-tAripe.
net@8.
8.
8.
8193.
0.
6.
139%dig-tMXtwitter.
com@8.
8.
8.
8;;Gotbadpacket:badlabeltype45bytes40108180000100010000000007747769twi7474657203636f6d00000c0001c00c00tter.
com.
.
.
.
.
.
.
.
0c000100000379000341414100.
.
.
.
.
.
y.
.
AAA.
NeedtotakeacloserlookNotobviousatfirstglance,buttimedeltaisstrange.
.
.
%tcpdump-ttt-c2-nqrripe.
pcapreadingfromPCAP-NGfileripe.
pcap00:00:00.
000000IP192.
168.
0.
132.
53425>8.
8.
8.
8.
53:UDP,length2700:00:00.
138933IP8.
8.
8.
8.
53>192.
168.
0.
132.
53425:UDP,length75%tcpdump-ttt-c2-nqrtwitter.
pcapreadingfromPCAP-NGfiletwitter.
pcap00:00:00.
000000IP192.
168.
0.
132.
58418>8.
8.
8.
8.
53:UDP,length2900:00:00.
028077IP8.
8.
8.
8.
53>192.
168.
0.
132.
58418:UDP,length45dnsping-AnewtoolisbornStartedasahumblePythonscripttosolvemyownproblemNotintendedtore-inventthewheelSimilaruserexperienceasthelegacyPING%.
/dnsping.
py-s8.
8.
8.
8-c3ripe.
netdnsping.
pyDNS:8.
8.
8.
8:53,hostname:ripe.
net,rdatatype:A32bytesfrom8.
8.
8.
8:seq=0time=200.
371ms32bytesfrom8.
8.
8.
8:seq=1time=217.
320ms32bytesfrom8.
8.
8.
8:seq=2time=236.
644ms---8.
8.
8.
8dnspingstatistics---3requeststransmitted,3responsesreceived,0%lostmin=200.
371ms,avg=218.
112ms,max=236.
644ms,stddev=18.
149msICMPvsDNSResponseTimes%ping-q-c108.
8.
8.
8PING8.
8.
8.
8(8.
8.
8.
8):56databytes---8.
8.
8.
8pingstatistics---10packetstransmitted,10packetsreceived,0.
0%packetlossround-tripmin/avg/max/stddev=124.
237/155.
044/227.
499/31.
464ms%.
/dnsping.
py-q-s8.
8.
8.
8-c3twitter.
comdnsping.
pyDNS:8.
8.
8.
8:53,hostname:twitter.
com,rdatatype:A---8.
8.
8.
8dnspingstatistics---3requeststransmitted,3responsesreceived,0%lostmin=12.
934ms,avg=21.
355ms,max=29.
425ms,stddev=8.
251msFirstanomalyDifferentdomainnamesbeingtreateddifferentlyAroguenameserverimpersonatingasGoogleResolverTheroguenameserverisclosetome(givenresponsetimes)WhereisitAndhowcanIfindoutdnstraceroute:TraceroutetoolforDNSprotocolSimilartolegacytraceroute,butforDNSprotocolSendoutactualDNSqueriesandexpectaresponseUsingTTLtricktomapthejourneyCouldnotuselegacytraceroutewithUDPprobesThetrafficredirectionisbasedonDNS"payload"RealvsRogueDNSServers%.
/dnstraceroute.
py-s8.
8.
8.
8ripe.
netdnstraceroute.
pyDNS:8.
8.
8.
8:53,hostname:ripe.
net,rdatatype:A1192.
168.
0.
1(192.
168.
0.
1)3.
912ms2*3192.
168.
10.
105(192.
168.
10.
105)15.
792ms4172.
17.
2.
1(172.
17.
2.
1)17.
063ms5172.
17.
2.
9(172.
17.
2.
9)11.
245ms6172.
19.
18.
5(172.
19.
18.
5)24.
862ms7172.
19.
17.
2(172.
19.
17.
2)18.
972ms810.
201.
177.
41(10.
201.
177.
41)13.
261ms910.
10.
53.
190(10.
10.
53.
190)14.
240ms10185.
100.
209.
117(185.
100.
209.
117)176.
592ms11*12de-cix.
fra.
google.
com(80.
81.
192.
108)152.
757ms13108.
170.
251.
193(108.
170.
251.
193)90.
347ms14google-public-dns-a.
google.
com(8.
8.
8.
8)185.
401ms%.
/dnstraceroute.
py-s8.
8.
8.
8twitter.
comdnstraceroute.
pyDNS:8.
8.
8.
8:53,hostname:twitter.
com,rdatatype:A1192.
168.
0.
1(192.
168.
0.
1)3.
160ms2*3192.
168.
10.
105(192.
168.
10.
105)5.
985ms4172.
17.
2.
1(172.
17.
2.
1)8.
535ms5172.
17.
2.
9(172.
17.
2.
9)20.
617ms6172.
19.
18.
5(172.
19.
18.
5)7.
823ms7*8*9google-public-dns-a.
google.
com(8.
8.
8.
8)19.
557msBacktothecaseofbrokenMXRoguenameserveralsoreturns"broken"responsesOnlytocertaintypeofqueries(e.
g.
MX)%dig-tMXtwitter.
com@8.
8.
8.
8;;Gotbadpacket:badlabeltype45bytes40108180000100010000000007747769twi7474657203636f6d00000c0001c00c00tter.
com.
.
.
.
.
.
.
.
0c000100000379000341414100.
.
.
.
.
.
y.
.
AAA.
Lookingatpacketsagain.
.
.
ResponsetoMXrequestismalformedServerrespondedwithPTRresponseRDLENGTHis3butRDATAfieldcontains4bytesAdditionalbytewasalwaysNULLSeemslikeabugincodeQueriedtop10,000domainnames[1],received139brokenresponses[2]UncoveringtherougeresolveraddressQueryTXTrecordfrommaxmind.
test-ipv6.
comIttellsyouthepublicaddressofyourresolverTheaddressshouldbelongtoGoogle(AS15169)AnythingelsemeansMITMUsingRIPEAtlastoseeifthisisaregularpractice%dig+short-tTXTmaxmind.
test-ipv6.
com@8.
8.
4.
4"ip='74.
125.
74.
14'as='15169'isp='Google'country='FI'"IsitjustmeLet'saskRIPEAtlas500Probesworldwide-484Replied(DNS/UDP/IPv4)475Good(Req.
fromGoogleaddressspace)~98%9Bad(Req.
fromnon-Googleaddressspace)~2%Same500probes-484Replied(DNS/TCP/IPv4)479Good~99%5Bad~1%ThelogicbehindDNStrafficredirectionWhatisthemotivationTherearemainlytworeasons:1.
Privacyprotection(TheGood)Preventsendingrequesttousehaveyourend-pointIPaddressasitssourceaddressFilteroutmalwareslookingfortheirC&C2.
DNSbasedserviceredirection(TheEvil)RestrictyouraccessRedirectyourtrafficCountermeasuresForcelocalresolvertouseTCPDNSCrypt(dnscrypt.
org)OpenDNSsupportsit,desktopclientsavailableDNSoverTLS/DTLS(RFC7858and8094)DNSPrivacyProject(dnsprivacy.
org)offerstutorials,tools,recommendationsandtestserversDNSSECFromtop100domainnamesonly2ofthemaresigned[3]WhatiftheroguenameserverdoesnotvalidateDNSSECFinalwordsDon'ttrustapublicDNSresolver,useyourownThereain'tnosuchthingasafreelunch(TANSTAAFL)Stubresolversareeasytosetupanduse(e.
g.
Stubby)Don'ttrustyourupstream,encryptasmuchaspossibleDNScontainsimportantinformationAsperRFC7258:"PervasiveMonitoringIsanAttack"Toolsofthetradednsping,dnstracerouteanddnsevalarepartof"dnsdiagtoolkit"onGitHub[4]LookingforfeedbackandideasfromcommunityCombiningwithothertools(e.
g.
RIPEAtlas)toperformmorecomplexbehavioranalysisSuggestion:dnstracerouteinRIPEAtlasprobesQuestionsResources[1]https://github.
com/opendns/public-domain-lists[2]https://gist.
github.
com/farrokhi/0b56ae06813391be9164[3]https://gist.
github.
com/farrokhi/1d9de9df5877aaf9c42fc14412a4b0f8[4]https://github.
com/farrokhi/dnsdiagAlsomyarticlesonRIPELabsdiscussionthesameissue:https://labs.
ripe.
net/Members/babak_farrokhi