Sponsored Content
Special Forums IP Networking Serious un-pingable stumper of a problem... Post 302161471 by jjinno on Thursday 24th of January 2008 07:02:13 PM
Old 01-24-2008
Serious un-pingable stumper of a problem...

I have been busting my head over a network issue at work recently. I believe the problem to be in the L2 domain, but "the powers that be" believe that it looks more like a server port related problem. And the biggest problem of all is that EVERYBODY in the Engineering Department uses this file-server...

The symptoms are as follows:
  • A samba connection is shared out from "FileServ_1" to my desktop. While having a file open for read/write, I will lose the file (aka. the persistence of connection), and will be prompted by my App to save a local copy (lucky me).
  • From that point, I immediately (being prepared) switch to a shell in which I kick off a ping to "FileServ_1"... then another shell I bypass DNS & go straight for the IP... then another shell I have a remote connection from a totally different subnet, also pinging "FileServ_1"... and finally a trace-route running from both my desktop and the remote connection.
  • From ALL pings I receive timeouts & from all traces I find the last hop is the dead-zone.

Although "the powers that be" make a strong case for their point, I have noticed "network topology changes" being reported at the switch (indicating a loop?) and I have been able to serial-console "FileServ_1" and watch it while it is supposedly "down"... only problem is: It never thinks that it is down.
  • Eth1 (till last week was the only port plugged in) never reports any issues (at least not at any default log levels) and from what I can see there is no way to tell if the ICMP packets are dying on the way in or on the way out.

Finally, as if things were not bad enough, they decided last week to make Eth0 a redundant fail-over for Eth1... which amazingly seemed to lighten the problem from "a few minutes of un-ping" to "a few seconds of un-ping"... and now, instead of happening 10 times a day it happens only once or twice.

So first things first (unless you have better ideas), I am wondering how to turn up the logging of ICMP (thats kernel level right?) and possibly Eth* logging so that I don't have to resort to sniffing for the entire day till it happens. Cause if nothing else, I would like to diagnose this problem correctly and get something done about it.

Any Help?
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

SSH Problem auth problem

Hi, Just recently we seem to be getting the following error message relating to SSH when we run the UNIX script in background mode: warning: You have no controlling tty. Cannot read confirmation.^M warning: Authentication failed.^M Disconnected; key exchange or algorithm negotiation... (1 Reply)
Discussion started by: budrito
1 Replies

2. Shell Programming and Scripting

ssh script problem problem

Hi Please help me with the following problem with my script. The following block of code is not repeating in the while loop and exiting after searching for first message. input_file ========== host001-01 host001-02 2008-07-23 13:02:04,651 ConnectionFactory - Setting session state... (2 Replies)
Discussion started by: pcjandyala
2 Replies

3. Solaris

problem in finding a hardware problem

Hi I am right now facing a strange hardware problem. System get booted with the following error: Fatal Error Reset CPU 0000.0000.0000.0003 AFSR 0100.0000.0000.0000 SCE AFAR 0000.07c6.0000.1000 SC Alert: Host System has Reset It happen 4 or 5 times and get the same error every time.I... (8 Replies)
Discussion started by: girish.batra
8 Replies

4. Shell Programming and Scripting

need to check whether a sever is pingable or not inside the script

Hi, need to write a script which will check number of ip address are able to ping or not .. (2 Replies)
Discussion started by: mail2sant
2 Replies

5. AIX

user login problem & Files listing problem.

1) when user login to the server the session got colosed. How will resolve? 2) While firing the command ls -l we are not able to see the any files in the director. but over all view the file system using the command df -g it is showing 91% used. what will be the problem? Thanks in advance. (1 Reply)
Discussion started by: pernasivam
1 Replies

6. UNIX for Dummies Questions & Answers

host not booting, but is pingable

hi there. im having a problem with a host at the moment, i can ping the host and responds with host is alive. i cannot telnet, rsh or anything else to it... it tells me connection refused. when i run a ckport on it i get answers from : *** successful - smtp *** successful - sunrpc ... (6 Replies)
Discussion started by: brian112
6 Replies

7. Solaris

[Help] - 2 VM solaris pingable

Hi, I have 2 VM of Solaris ( 2nd one full clone ) 1st VM - 192.168.1.30 2nd VM - 192.168.1.31 My need : ping both VM from each other I have added host entry in /etc/hosts of both server but unable to ping each other from solaris console... Pls advice (4 Replies)
Discussion started by: saurabh84g
4 Replies

8. IP Networking

Problem with forwarding emails (SPF problem)

Hi, This is rather a question from a "user" than from a sys admin, but I think this forum is apropriate for the question. I have an adress with automatic email forwarding and for some senders (two hietherto), emails are bouncing. This has really created a lot of problems those two time so I... (0 Replies)
Discussion started by: carwe
0 Replies

9. Shell Programming and Scripting

validating(pingable or not) remote ip address in shell script

i need to verify whether the ip adress given as input to the shell script is pingable or not... that is whether the ip is alive and responding.. ping $ip_adress the above wont work in script because the execution is continuous... so the shell script keeps will dwell in this pinging process...... (8 Replies)
Discussion started by: vivek d r
8 Replies

10. UNIX for Dummies Questions & Answers

sed Or Grep Problem OR Terminal Problem?

I don't know if you guys get this problem sometimes at Terminal but I had been having this problem since yesterday :( Maybe I overdid the Terminal. Even the codes that used to work doesn't work anymore. Here is what 's happening: * I wanted to remove lines containing digits so I used this... (25 Replies)
Discussion started by: Nexeu
25 Replies
TRACEROUTE(1)                                                 General Commands Manual                                                TRACEROUTE(1)

NAME
traceroute - print the route packets take to network host SYNOPSIS
traceroute [-46FInrRTV] [-f first_ttl] [-p port] [-m max_hops] [-N concurrent_hops] [-t tos] [-w timeout] [-q nqueries] [-S source_addr] [-i interface] [-g gateway] host [packetlen] traceroute6 [options] DESCRIPTION
traceroute tracks the route packets take across a TCP/IP network on their way to a given host. It utilizes the IP protocol's time to live (TTL) field and attempts to elicit an ICMP TIME_EXCEEDED response from each gateway along the path to the host. traceroute6 is just another name for the same program, and is equivalent to invoking traceroute with the -6 option. Tracing with UDP The default mode of operation is to use UDP packets for path detection. This will send packets of a given size to a range of destination ports, usually in the high port range, and increment the destination port number for each probe packet sent). Intermediate gateways will return ICMP time exceeded errors when the packet's TTL reaches zero. When a packet reaches the remote host, and the chosen port is not in use, the host will return an ICMP port unreachable error. If the port is in use, the probe packet will be delivered to this port, more likely than not without eliciting a response. The port range is chosen so that it is rather unlikely that this port will be in use on the remote system. Tracing in UDP mode does not require super user privileges, as the Linux UDP implementation supports a so-called error queue, through which ICMP error packets related to a specific port can be delivered to the application. Tracing with ICMP In this mode, traceroute sends ICMP echo packets to the destination host, pretty much like the ping utility. ICMP tracing requires super user privilege, as it needs to open a raw socket to listen for incoming ICMP packets. Non-root users are not permitted to open such sockets for security reasons. ICMP traces are currently supported over IPv4 only. Tracing with TCP In TCP mode, traceroute will try to connect to the destination host. This causes the kernel to send a SYN packet to the remote host, again varying the TTL for each probe. The default port number used for TCP probes is 80 (http). When the probe reaches the destination host, it will either accept the connection (in case there's a service listening on this port), or respond with a TCP RST packet (in case there isn't). TCP tracing requires super user privilege, as it needs to open a raw socket to listen for incoming ICMP packets. Non-root users are not permitted to open such sockets for security reasons. ICMP traces are currently supported over IPv4 only. Caveats Many gateways are configured to limit the rate at which they will send ICMP error packets. Thus, when you run lots of traces in close suc- cession, an increasing number of gateways may "disappear", with probes timing out instead of eliciting an ICMP response. Rate limiting often happens at a granularity of 30 to 60 seconds, so do not expect things to work properly if you run more than two or three traces per minute. OPTIONS
The only required parameter is the name or IP address of the destination host. This parameter can be followed by the size of the probing packet sent to that host (40 by default). Varying the size of the packet in conjunction with the -F parameter can be used to obtain infor- mation about the MTU of individual network hops. This does not apply to TCP, which uses SYN packets to establish a path trace. Additional options are: -6, -4 Explicitly force IPv4 or IPv6 traceouting. By default, the program will try to resolve the name given, and choose the appropriate protocol automatically. If resolving a host name returns both IPv4 and IPv6 addresses, traceroute will use IPv4. Invoking the pro- gram as traceroute6 is the same as using the -6 option. -F Set the "Don't Fragment" bit. This tells intermediate routers not to fragment the packet when they find it's too big for a network hop's MTU. -f first_ttl Specifies with what TTL to start. Defaults to 1. -g gateway Tells traceroute to add an IP source routing option to the outgoing packet that tells the network to route the packet through the specified gateway. Not very useful, because most routers have disabled source routing for security reasons. -I Use ICMP ECHO instead of UDP packets when probing the route. This option is available to the super user only, as normal users do not have the privileges to create raw sockets. -i interface Specifies the interface through which to traceroute should send packets. By default, the interface is selected according to the routing table. NOTE: This option changed from previous versions of traceroute! Older versions used -I to specify the interface name. -m max_hops Specifies the maximum number of hops traceroute will probe. The default value is 30. -N concurrent_hops Specifies the number of probe packets sent out simultaneously. Sending several probes concurrently can speed up traceroute consid- erably. However, when specifying a number that's too large, the destination host will start to throw away random ICMP responses (if it implements ICMP rate throttling), and traceroute will be unable to detect the final hope reliably. The default value is 6. -n Do not try to map IP addresses to host names when displaying them. -p port Specifies the UDP destination port base traceroute will use. When sending its UDP probe packets, it will send them to port + hop - 1 for each hop. If there are ports in this range in use on the destination host, traceroute will not be able to identify reliably when it has reached the destination host (probes will appear to time out). The default port is 33434; you can use -p to change this to a different value. -q numqueries Sets the number of probe packets per hop. The default value is 3. -r Bypass the normal routing tables and send directly to a host on an attached network. If the host is not on a directly-attached net- work, an error is returned. This option can be used to ping a local host through an interface that has no route through it. -R Set the loose source route option on outgoing packets, asking intermediate routers to record their address as the packet passes. This can be useful if you want to find the address of an intermediate router that has been configured to not respond to traceroute packets. This feature hasn't been implemented yet. -S source_addr Chooses an alternative source address. Note that you must select the address of one of the interfaces. By default, the address of the outgoing interface is used. -T Use TCP instead of UDP packets when probing the route. This option is available to the super user only, as this requires a raw ICMP socket, which normal users are not permitted to create. -t tos Set the IP Type of Service (TOS) and Precedence value. Useful values are 16 (low delay) and 8 (high throughput). Note that in order to use some TOS precendence values, you have to be super user. -V Print the version and exit. -w sec Wait for sec seconds before sending the next probe packet. Note that unlike older traceroute versions, this implementation will transmit several probe packets in parallel, for different hop values. However, it will never send more than 1 packet per hop value at the same time. SEE ALSO
ping(8), netstat(8). AUTHOR
Concept and command line options based on the original LBL implementation of traceroute, written by Van Jacobson. This implentation is a complete rewrite and redesign, written and copyright (C) 2000-2008 Olaf Kirch <okir@suse.de>. 4.3 Berkeley Distribution 6 December 2000 TRACEROUTE(1)
All times are GMT -4. The time now is 04:41 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy