UNIX for Dummies Questions & Answers

Yes, my information wasn't too complete. I have to deal with a file of 78G and I'm using ssh to another computer, so when I try to run this

Code:

awk '$6==0.60003{print FNR}' MICECAT_v1.0_order_by_id.ssv

or this

Code:

grep -n '^[^ ][^ ][^ ][^ ][^ ][^ ]*  *3 ' MICECAT_v1.0_order_by_id.ssv | cut -d: -f1

I always get (one hour later):

OK, just to recap: you use "ssh" to connect from one host to another, then run your script on the other computer. After some long time you get the mentioned error.

First off: the error is not from the script, but from the - terminated - ssh-session. This is ssh informing you about this. You will have to investigate why this is the case but my first suspicion is the following: there is a timeout value for idle sessions which can be configured. Because you fire off this long job and then do nothing - as the script is running anyways - the shell thinks your session is idle and terminates it.

Log on to the other computer and issue the command

Code:

set | grep TMOUT

Eventually you might find something like:

Code:

# set | grep TMOUT
TMOUT=3600

This is the timeout value in seconds. Because this is usually a read-only variable you probably cannot "unset" it, therefore you need to get a systems admin to this for you. If you can become root at the system you can simply unset it yourself, which should switch off the timeout.

I hope this helps.

bakunin

Sorry again but is it possible that TMOUT is not defined? I can't find it in set, and echo doesn't answer anything

How about another approach:
ssh into that other computer, launch a script with the nohupped / disowned awk command, its output redirected to a file, and leave the session.
After a while, come back and collect the output file.

Thanks!

I've tried this:

Code:

nohup awk '$6==0.60003{print FNR; exit}'>MICEcarol.ssv MICECAT_v1.0_order_by_id.ssv &

But I just get nohup:

Code:

ignoring input and redirecting stderr to stdout

Yes, it is possible that TMOUT is not defined and the automatic logoff feature is configured in another way. What i told you was just the most wide-spread way of achieving this.

The method your systems administration employed is probably system-specific therefore without you telling us your operating system, the version of it and in general the specifics of your environment (which i asked you to do above - you didn't respond) we can't help you.

But with what we told you you should at least be able to approach the systems admin and tell them to disable the automatic logoff feature for this job. They should know how to do it because they implemented it.

I hope this helps.

bakunin

The OP didn't define what was too long (line length?) or what is "slow". Nor is there any information about the source file, what version of the O/S, what type of file system on what kind of volume. So my grep/cut suggestion may, or may not, be faster than awk. However, the OP would be wise to test multiple solutions to find the one works best for their environment -- if performance and running time are considerations.