UNIX for Beginners Questions & Answers

Need help in piping commands using xargs

I have several .tar.gz files that I need to list the folder content in a subdirectory.
For example,

Code:

a.tar.gz
b.tar.gz
c.tar.gz

The following command works great for each .tar.gz file but it's a pain to run the tar command for each file.

Code:

tar -tf a.tar.gz|grep folder1/folder2

I tried this command and failed.

Code:

ls *.tar.gz|xargs tar -tf|grep folder1/folder2

I got this error message "tar: a.tar.gz: Not found in archive" for each gz file.

What do I need to fix to run this command successfully?
Or do I need to use a different command to process all the .tar.gz files?

Thank you.

Moderator's Comments:

Please use CODE tags as required by forum rules!

I'm afraid there are more than one erroneous approaches in above:
- the files seem to be gzipped (by their ending) - gunzip before presenting them to tar. In fact, I'm slightly alienated that your single manual command should run error free...
- Shouldn't there be a message like tar: This does not look like a tar archive?
- the -f option takes just one single file (except for a multi-volume archive which I doubt you have here)

You might want to try the -n1 option to xargs and see how far you get.

Perhaps something like this might do it:-

Code:

for targz in *.tar.gz
do
   tar -tf $targz
done | grep folder1/folder2

The problem you are seeing is because tar is seeing the input as -f [I]first_file_listed item1_to_extract item2_to_extract so you might get away with adding -n 1 to xargs like this:-

Code:

ls *.tar.gz|xargs -n 1 tar -tf|grep folder1/folder2

This will at least run tar for each file separately. I'm not sure how it will handle the pipe to grep, but it should work.



I hope that these help. If not, please run it with debug on your shell (i.e. set -x first) and paste the output in CODE tags. It would help if you have a small set of small (few members) tar files to keep the output manageable.



Kind regards,
Robin

Thank you, Robin.

I tried both examples you provided and they both list the folder content. The only issue is both examples only print out the results without displaying the input file name so I don't know which input file produces the results.

What needs to be modified to print out the input file name and the results?

Thank you again for your help.

You could try:-

Code:

for targz in *.tar.gz
do
   echo "$targz" >&2                    # Write to STDERR, so show up on the screen
   tar -tf $targz
done | grep folder1/folder2

...or...

Code:

ls *.tar.gz|xargs -tn 1 tar -tf|grep folder1/folder2

The -t flag for xargs shows you what it is executing each time.

Do either of these help?



Robin

Hi Robin,
With pipe buffering, I don't think there is any guarantee that the output from the echo or from xargs -t sent to STDERR won't appear on the screen before some output from grep of the previous archive. And the same thing could happen if you try to capture both STDOUT and STDERR and redirect them to a single output file.

To get the name of the archive being processed in the standard output stream and survive the grep in the pipeline, one might try something more like:

Code:

for targz in *.tar.gz
do
   echo "folder1/folder2 files in $targz..."   # Write to STDOUT so script output can be redirected
   tar -tf $targz
done | grep folder1/folder2

As RudiC mentioned, some versions of tar will get lost if given a zipped archive. But, assuming that the tar on april's system gunzips a file automatically if its name ends in .gz, this should work.

I tried both methods from Robin and they both worked. I see the file names and the grep results following each file name.

I tried Don's suggestion

Code:

echo "folder1/folder2 files in $targz..."

. That did not give me the file names. I only see the grep results.

Thank you all for your help.