Home Man
Search
Today's Posts
Register

If you're not sure where to post a Unix or Linux question, post it here. All unix and Linux beginners welcome in this forum!

Piping commands using xargs

Login to reply

 
Thread Tools Search this Thread
# 1  
Old 11-29-2017
Piping commands using xargs

Need help in piping commands using xargs

I have several .tar.gz files that I need to list the folder content in a subdirectory.
For example,
Code:
a.tar.gz
b.tar.gz
c.tar.gz

The following command works great for each .tar.gz file but it's a pain to run the tar command for each file.
Code:
tar -tf a.tar.gz|grep folder1/folder2

I tried this command and failed.
Code:
ls *.tar.gz|xargs tar -tf|grep folder1/folder2

I got this error message "tar: a.tar.gz: Not found in archive" for each gz file.

What do I need to fix to run this command successfully?
Or do I need to use a different command to process all the .tar.gz files?

Thank you.

Moderator's Comments:
Piping commands using xargs Please use CODE tags as required by forum rules!

Last edited by RudiC; 11-29-2017 at 08:20 AM.. Reason: Added CODE tags.
# 2  
Old 11-29-2017
I'm afraid there are more than one erroneous approaches in above:
- the files seem to be gzipped (by their ending) - gunzip before presenting them to tar. In fact, I'm slightly alienated that your single manual command should run error free...
- Shouldn't there be a message like tar: This does not look like a tar archive?
- the -f option takes just one single file (except for a multi-volume archive which I doubt you have here)

You might want to try the -n1 option to xargs and see how far you get.
# 3  
Old 11-29-2017
Perhaps something like this might do it:-
Code:
for targz in *.tar.gz
do
   tar -tf $targz
done | grep folder1/folder2

The problem you are seeing is because tar is seeing the input as -f [I]first_file_listed item1_to_extract item2_to_extract so you might get away with adding -n 1 to xargs like this:-
Code:
ls *.tar.gz|xargs -n 1 tar -tf|grep folder1/folder2

This will at least run tar for each file separately. I'm not sure how it will handle the pipe to grep, but it should work.



I hope that these help. If not, please run it with debug on your shell (i.e. set -x first) and paste the output in CODE tags. It would help if you have a small set of small (few members) tar files to keep the output manageable.



Kind regards,
Robin
# 4  
Old 11-29-2017
Piping commands using xargs

Thank you, Robin.

I tried both examples you provided and they both list the folder content. The only issue is both examples only print out the results without displaying the input file name so I don't know which input file produces the results.

What needs to be modified to print out the input file name and the results?

Thank you again for your help.
# 5  
Old 11-29-2017
You could try:-
Code:
for targz in *.tar.gz
do
   echo "$targz" >&2                    # Write to STDERR, so show up on the screen
   tar -tf $targz
done | grep folder1/folder2

...or...
Code:
ls *.tar.gz|xargs -tn 1 tar -tf|grep folder1/folder2

The -t flag for xargs shows you what it is executing each time.

Do either of these help?



Robin
# 6  
Old 11-29-2017
Quote:
Originally Posted by rbatte1
You could try:-
Code:
for targz in *.tar.gz
do
   echo "$targz" >&2                    # Write to STDERR, so show up on the screen
   tar -tf $targz
done | grep folder1/folder2

...or...
Code:
ls *.tar.gz|xargs -tn 1 tar -tf|grep folder1/folder2

The -t flag for xargs shows you what it is executing each time.

Do either of these help?



Robin
Hi Robin,
With pipe buffering, I don't think there is any guarantee that the output from the echo or from xargs -t sent to STDERR won't appear on the screen before some output from grep of the previous archive. And the same thing could happen if you try to capture both STDOUT and STDERR and redirect them to a single output file.

To get the name of the archive being processed in the standard output stream and survive the grep in the pipeline, one might try something more like:
Code:
for targz in *.tar.gz
do
   echo "folder1/folder2 files in $targz..."   # Write to STDOUT so script output can be redirected
   tar -tf $targz
done | grep folder1/folder2

As RudiC mentioned, some versions of tar will get lost if given a zipped archive. But, assuming that the tar on april's system gunzips a file automatically if its name ends in .gz, this should work.
The Following User Says Thank You to Don Cragun For This Useful Post:
rbatte1 (11-29-2017)
# 7  
Old 11-29-2017
I tried both methods from Robin and they both worked. I see the file names and the grep results following each file name.

I tried Don's suggestion
Code:
echo "folder1/folder2 files in $targz..."

. That did not give me the file names. I only see the grep results.

Thank you all for your help.
Login to reply

« Previous Thread | Next Thread »
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Proper chaining and piping of commands SkySmart Shell Programming and Scripting 2 01-08-2016 01:46 AM
Xargs: multiple commands to each argument urello Shell Programming and Scripting 1 03-02-2015 12:25 AM
Piping commands iliya24 UNIX for Dummies Questions & Answers 7 03-28-2013 10:24 PM
Piping multiple commands jrymer UNIX for Dummies Questions & Answers 3 08-06-2012 03:48 PM
piping problem with xargs ivpz Shell Programming and Scripting 7 08-09-2011 11:30 AM
Piping results of 'ls' to 'find' using 'xargs' MartynAbbott Solaris 4 08-21-2009 09:00 AM
Piping through commands read as variables from file fitzwilliam Shell Programming and Scripting 1 07-20-2009 08:19 AM
use of xargs and prune piping with find command. venkatesht UNIX for Dummies Questions & Answers 1 01-28-2009 08:29 AM
Can Xargs execute multiple commands of evry input file nilesrex Shell Programming and Scripting 4 08-30-2006 08:39 AM
Executing commands with xargs bmopal UNIX for Dummies Questions & Answers 1 08-13-2001 04:05 PM


All times are GMT -4. The time now is 05:35 AM.

Unix & Linux Forums Content Copyrightę1993-2018. All Rights Reserved.
UNIX.COM Login
Username:
Password:  
Show Password