子网dns故障引发子网流量异常（DNS fault raises subnet traffic anomalies）

dns故障引发子网流量异常DNS fault raises subnet trafficanomal ies

This is a friend of network fault, fault is typical,troubleshooting ideas more desirable. So the journey to findthe solution, all told the world, hope everyone after use.

1. Symptom description

Customers call report center network is basically normal, buta subnet suddenly slows down. This is the local railway networkservices company, the company provides Web services andInternet access service for ordinary users. A few days ago, thearea of service users reflect the network speed is very slow,Email also need towait more than60 seconds over time toUnicom.This area is divided into a sub network, network managementsystem from the host housing observation found that in additionto the area (subnet) router traffic is very high (test 97%) ,interactive traffic center network routers and other subnetworks was below 40%. In addition, no other specialphenomenon.

2 、 diagnostic process

Railway maintenance personnel conducted their own networkdebugging but did not find the fault, unable to disconnect thenetwork user service stop to check, and turn to us, I was sentout. It should be said that judging from the symptoms of thisfault is relatively simple, as long as the found routing subnettraffic sources can quickly determine the direction of faultfurther, it can find out the source of traffic immediately.

From the network topology, fault sub network and Network Centerfor the E1 link. There is a hall fault sub network below, onlyinteract with some business data center network should not havetoo much traffic. In addition, the number of Web server underthe subnet to 45 units, the network management system reportcenter 97% the traffic is certainly too high.

I consider the effective flow only in one situation can moreoccupy the E1 channel, that is the fault of multimedia documentsbetween the website and the subnet network site or servertransmission or download servicewill cause this situation. Butthe management staff asked that the network does not providesuch as multimedia video playback and download service. It canonly use tools to detect.

Because of the relatively small size of the network faultmanagement system, network support only to router levelmanagement, switch and server etc. using the desktop switchescheap, it can not support the network management. The networkaccess switch tester for testing, started carrying the networkmanagement function, you can see the router observation flowand network management system the flow is the same, are around97%. (ylmf skills)

This view is connected with the router traffic center network,is about 997%, indicating that the channel router linkperformance is basically normal. But this will inevitably leadto high flow channel router congestion and packet loss, so fromthe perspective of flow is not normal. Now need to understandis that the routing traffic is so high where it came from. The

packets arriving at the router and later to. So you can quicklylocate to the data source and the source channel trafficcongestion so high.

The router channel network traffic analyzer access networkmonitoring and analysis, results showed that 95%of the trafficf low data server, and the majority of HTTP and Email. Among them,Internet traffic accounted for 88%, traffic sourcesdistribution of local traffic accounted for 7%. instructionsto check the flow of the analyzer, no traffic concentrationapplication discovery,

IP address distribution is balanced, the highest rate accountedfor only 0.5%. . These data suggest that the applicationproportion of user equilibrium, the cause of the malfunctionshould be in the application process rather than a centralizeduser "bombardment" such as hackers. That is to say, the processand application of channel should be out of the question. Thereason is that these flow at channel design should not reachthe business hall network service server, but should enter theInternet directly from the Internet router center network. So,the flow is to be guided to the direction of the business hallserver?

Here we carry out further analysis, we know that the IP datapacket in the transmission process will address resolution inthe router (ARP) , or domain name analysis in the local DNS. Ifthe path analysis problems, the IP transmission of data packetand exchange will be a problem. According to the trafficanalyzer' s instructions, the arbitrary choose 10 IP addressrouting tracking test results for tracking network tester is

that they must pass through a DNS server. And imitate thebusiness hall of known members of the network are local andforeign users ICMP monitoring and route tracking test, the datapacket redirection ICMP monitoring target accounted for 82%.Not up to the number of data packets for 13%. which show thatonly about 2%of the users can access the normal route to thetarget site, the remaining 95% of the IP data packet to gothrough competition or re sent to the routing part Theopportunity arrives at the destination

This transformation can focus on examination of the main routerrouting table and DNS table. Since the majority of Internettraffic is directed to the business server, so you can focuson the server. Check the DNS query to the DNS server by networktester, observation results showed that DNS conversion tablehas a considerable proportion to business hall network serviceserver. I suspect the DNS server is out of the question!So the notification center network management personnel willrestart the DNS server and quickly set up a network managementnetwork business report later returned to normal. Using networkanalyzer Internet toolkit querying the DNS server, you can seeto business server data has disappeared, which indicates thatthe network has been fully restored to normal work but goodtimes don't last long. , about 3 minutes after the fault appearsagain, still have 97% of the channel flow is directed to asubnet.

Because the DNS server set only one, no backup server, and hadto immediately came to the center of network computer room, tocheck the DNS server and its peripheral equipment. The test

server adapter and cable and router. In order not to interruptthe normal service, the author makes network managementpersonnel set up a temporary installation of DNS server inanother backup server. After a brief interruption of business,the replacement of a new DNS server application started. Seethe subnet router traffic immediately reduced to 1.5%. after30 minutes of work after all users were restored to the normalworking state, fault elimination.

3, the cause of the failure

As we all know, the DNS server for the user domain names intoIP addresses, generally does not appear what problem. But forsome reason, causing all point to the business office networkservice server address translation in this case. The similarbusiness server does not have the routing function,

IP packets that are sent either are rejected, collocated,ignored, or returned to unreachable or redirected packets. Thisis what we often observe when monitoring ICMP

The number of users of local railway is not much, but with highernetwork bandwidth for the ATM link 155M, a large surplus, soInternet users access to the Internet speed is mainly affectedby the subnet bandwidth. Because many users through E1 invalidlink congestion, routing redirection and cause serious delayof IP data. A large number of packets to hold only 2M bandwidthof the subnet router, traffic reached 97%, resulting in subnetwork speed suddenly slow, serious congestion routerphenomenon.

4, two suggestions

(1) the.DNS server should have a regular medical examination"Based on DNS service in order to prevent instability caused bybusiness interruption or error, many network administratorsare installed in the alternate DNS server set up DNS server,which is not only the installation of a DNS server. But it alsoposes a potential danger, which is the main DNS server, backupserver automatically put into operation, it will sacrifice thenetwork bandwidth, the overall performance of the systemdecreased. The danger is that the decline in performance isoften to imperceptibly. So, in order to ensure that the networkis often in good working condition, the conversion networkmanagers need to periodically check the DNS server.

The fault at the DNS error led to the user' s IP data packetson the subnet server, but if the alignment is not a server inthe local network of network center instead of a machine, thenthe fault strength will be weakened, the user will not feel veryobviously slower. It may not feel obvious "discomfort" whichmakes the network for a long time to stop the operation. Likepeople, regular physical examination is necessary for timelydetection of disease and risk. And how to discover the problemsof routing optimization, and network test in the regularproject content on a large network, it is necessary, we mustadhere to the regular maintenance and testing.

(2) real time monitoring of network status

Many network devices such as routers, switches, hubs, can only

support SNMP network management function, but in order tomonitor the network channel function, network equipment alsoneed to support full RMON and RMON2. use this equipment set upthe network management and fault diagnosis function is verygood. But the real problem is that such a network device theprice of ordinary network equipment 6 ~ 10 times, it isdifficult for users to accept. Therefore, in order to monitorthe service flow and the proportion of application and networksources, unpack analysis records and when necessary,suggestions for users to install monitoring interface in theserver channel or channel routing. If necessary at any time willflow analyzer, network analyzer access monitoring and analysis.In this way, the fault detection time can be shortened to 20minutes or so. Of course, if the money. Xu, you can also flowanalyzer long-term access channel for a number of importantnetwork devices at full speed, transparent traffic monitoring,so that you can reduce the fault location time to less than 1m i nu t e s

This "home visit" generally works well. In fact, every visitis a chance to learn and improve. Maybe the above case is justa case. You may not meet,

But troubleshooting ideas or worth learning. In addition, Isuggest that the hope can cause everybody' s attention at theend of the two.