UNIX for Advanced & Expert Users

Good day,

I am trying to run a ssh command on a remote server, but when I run the command the server takes FOREVER... and ends up giving me an error..
then the script also kicks server load up by 100% while it is running..
But if I run the same command on the server it goes pretty quickly and gives the correct std output.

Command on server:
readonly@VCRE02 ~# YEAR=2009;DAY=27;MONTH=02;LMONTH=Feb;for t in 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23; \
do nice -n 20 gunzip -c "/log-archive/access.log_$YEAR$MONTH$DAY_"*$t.gz |nice -n 20 nawk -v a=$t '{total[$8] += 1} END{for (i in total) print a,i, total[i]}' ;done
<std output below>
00 "200" 353061
00 "302" 5001
00 "404" 3689
00 "304" 20030
00 "500" 592
(this of course is what I want..)

ssh command:
Servername=cre02 ; ServerIP=10.113.98.17 ;nice -n 20 ssh readonly@$ServerIP 'YEAR=2009;DAY=27;MONTH=02;LMONTH=Feb;for t in 00 01 02 03 04 05 \
06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23; do nice -n 20 gunzip -c "/log-archive/access.log_$YEAR$MONTH$DAY_"*$t.gz | \
nice -n 20 nawk -v a=$t '\''{total[$8] += 1} END{for (i in total) print a,i, total[i]}'\'' ;done'
<std error below>
gunzip: /log-archive/apache/access.log_200902*00.gz: No such file or directory
gunzip: /log-archive/apache/access.log_200902*01.gz: No such file or directory
gunzip: /log-archive/apache/access.log_200902*02.gz: No such file or directory
gunzip: /log-archive/apache/access.log_200902*03.gz: No such file or directory
gunzip: /log-archive/apache/access.log_200902*04.gz: No such file or directory
(this is not what I want)

I have tried quoting the filename for the "gunzip -c" command in various ways, but is does not really work.
ex. gunzip -c "/log-archive/access.log_$YEAR$MONTH$DAY_"*$t.gz
gunzip -c /log-archive/access.log_$YEAR$MONTH$DAY_*$t.gz
gunzip -c /log-archive/access.log_$YEAR$MONTH$DAY*$t.gz
no luck.

This is a sample filename of the files I am trying to gunzip:
/log-archive/access.log_20090101_00.gz

I noticed if I "cat" the files that have not yet been archived the whole process goes much smoother and there is no issues with the filenames..

So somewhere I am doing something wrong.. and for the life of me I cannot figure out what it is.

Any advise on this will be greatly appreciated.

Thanks.

I think you're running afoul of ' characters forcing * to be literal in your ssh command. Plus nice on your side does not mean it got niced over there.

You've got keys, right?
So we write a small script on your end using what works, you may want to make it use 4 parms instead of YEAR=xxxx, etc.

Code:

# script.sh
YEAR=2009
DAY=27
MONTH=02
LMONTH=Feb
for t in 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 
do 
  nice -n 20 gunzip -c "/log-archive/access.log_$YEAR$MONTH$DAY_"*$t.gz |
  nice -n 20 nawk -v a=$t '{total[$8] += 1} END{for (i in total) print a,i, total[i]}' 
done

next we write a tiny script that stays on your side:

Code:

# tiny.sh
sftp readonly@server1 << EOF
put script.sh
exit
EOF
ssh readonly@SERVER01 chmod +x ./script.sh && ./script.sh > output

You execute tiny.sh from home.

Thanks for the reply Jim.

The nice command does work very..um.. nicely..
The problem I have is that I am not allowed to place any files on the servers I have to monitor. I am sitting in development environment, they are all on production and I do not have enough privileges to do anything on production.
That is why I have the whole "string-of-infinite-headache" following the ssh command..

So what can I do without slapping files on the server itself?

This command for example works 100% (with dates substituted of course)
ssh readonly@$ServerIP 'YEAR=xx;DAY=xx;MONTH=xx;LMONTH=xx;for i in 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 \
16 17 18 19 20 21 22 23; do cat "/var/SP/log/cre/access.log_$YEAR$MONTH$DAY_"*$i | grep -c "HTTP/1.1\" \"500";done'

I figured it out! Just had to change the stupid quotes...

Now the script goes helluva fast and only uses around 0.6-0.8% Utilization, compared to 14% if the gunzip filename does not match properly...

ssh_function_CRE()
{
ssh readonly@$ServerIP 'YEAR=2009;DAY=28;MONTH=02;LMONTH=Mar;for t in 00 01 02 03 04 05 06 07 08 09 10 11 \
12 13 14 15 16 17 18 19 20 21 22 23; \
do nice -n 20 gunzip -c /log-archive/access.log_$YEAR$MONTH$DAY"_$t".gz |\
nice -n 20 nawk -v a=$t '\''{total[$8] += 1} END{for (i in total) print a,i, total[i]}'\'' ;done'
#echo "$Servername, $Date" > $EFCRE
sleep 1 ; cat $sshEF >> $EFCRE
}