Shell Programming and Scripting

I have been trying to search for a string from close to 200 *.gz file, But i get a error. Can someone suggest a bulletproof solution Please.

zgrep 20/Aug/2008:13:50:23 request.log.*.gz
-bash: /usr/bin/zgrep: /bin/sh: bad interpreter: Argument list too long

also

zgrep 20/Aug/2008:13:50:23 request.log*
-bash: /usr/bin/zgrep: /bin/sh: bad interpreter: Argument list too long


Thanks

Woah, do you keep that many rotated log generations?
You run into this kind of problem when the shell's globbing would expand to more chars than what your shell's max arg length is.
But this, at least on this Linux box of mine here, is quite a few chars anyway, viz.
If you still want to keep that many files then you would have to refrain from globbing
and use something like this

Try this:

for F in `ls ./request.log.*.gz`; do zgrep 20/Aug/2008:13:50:23 $F; done

or even

ls request.log.*.gz |xargs "zgrep 20/Aug/2008:13:50:23"

zgrep 'IN0800' eodlog.*.gz
i have 1000 of eodlog.*.gz
but this is not giving any errors..

Then your kernel's ARG_MAX is bigger.

i have encountered similar problems on servers keeping log files for 3 or 4 years
a new file is cerated evry 5 mins, so you can guess how many did i had to manage...

the proble is the expansion ( the * )
"ls *" is not the same as "ls"
ls * will be captuerd by the shell, and the * will replaced by the list of files in the current directory, separated by a whitespace.
when executing "ls" it will be ls the one to get the list.
keeping that in mind is all you need to work over that folder, but be aware that you are going to work mostly blind.

a few tips would be.
create a listing of the folder, soy ou dont have to re-list evrything all the time (my listing took more than 5 mins and made the server crawl)
ls > list-of-files
then, use other tools over that textfile, like grep or awk, ect

also, when you feel you are reado to work on the files, use a while, and read from that file

Code:

while read line
do
    echo "the var line has a line of the file, $line"
done  < list-of-files

and expect alot of cpu usage, so if that server is actually doing some other stuff, then try to coordinate some downtime or window
you are gona make the server be slow, and other prioceses in the server are ging to make your work slower

Thanks for the replies !

Unfortunately none of them seem to work

1.ls | grep \\.gz\$ | xargs zgrep 20/Aug/2008:13:50:23

Takes for ever.

2.zgrep 20/Aug/2008:13:50:23 *
-bash: /usr/bin/zgrep: /bin/sh: bad interpreter: Argument list too long

Please provide any inputs its quite critical !

I am searching a string FROM close to 200 .gz files each in the range of 2 - 3 MB.


Thanks