Piping commands using xargs

Login to Reply

 
Thread Tools Search this Thread
# 1  
Old 11-29-2017
Piping commands using xargs

Need help in piping commands using xargs

I have several .tar.gz files that I need to list the folder content in a subdirectory.
For example,
Code:
a.tar.gz
b.tar.gz
c.tar.gz

The following command works great for each .tar.gz file but it's a pain to run the tar command for each file.
Code:
tar -tf a.tar.gz|grep folder1/folder2

I tried this command and failed.
Code:
ls *.tar.gz|xargs tar -tf|grep folder1/folder2

I got this error message "tar: a.tar.gz: Not found in archive" for each gz file.

What do I need to fix to run this command successfully?
Or do I need to use a different command to process all the .tar.gz files?

Thank you.

Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 11-29-2017 at 09:20 AM.. Reason: Added CODE tags.
# 2  
Old 11-29-2017
I'm afraid there are more than one erroneous approaches in above:
- the files seem to be gzipped (by their ending) - gunzip before presenting them to tar. In fact, I'm slightly alienated that your single manual command should run error free...
- Shouldn't there be a message like tar: This does not look like a tar archive?
- the -f option takes just one single file (except for a multi-volume archive which I doubt you have here)

You might want to try the -n1 option to xargs and see how far you get.
# 3  
Old 11-29-2017
Perhaps something like this might do it:-
Code:
for targz in *.tar.gz
do
   tar -tf $targz
done | grep folder1/folder2

The problem you are seeing is because tar is seeing the input as -f [I]first_file_listed item1_to_extract item2_to_extract so you might get away with adding -n 1 to xargs like this:-
Code:
ls *.tar.gz|xargs -n 1 tar -tf|grep folder1/folder2

This will at least run tar for each file separately. I'm not sure how it will handle the pipe to grep, but it should work.



I hope that these help. If not, please run it with debug on your shell (i.e. set -x first) and paste the output in CODE tags. It would help if you have a small set of small (few members) tar files to keep the output manageable.



Kind regards,
Robin
# 4  
Old 11-29-2017
Piping commands using xargs

Thank you, Robin.

I tried both examples you provided and they both list the folder content. The only issue is both examples only print out the results without displaying the input file name so I don't know which input file produces the results.

What needs to be modified to print out the input file name and the results?

Thank you again for your help.
# 5  
Old 11-29-2017
You could try:-
Code:
for targz in *.tar.gz
do
   echo "$targz" >&2                    # Write to STDERR, so show up on the screen
   tar -tf $targz
done | grep folder1/folder2

...or...
Code:
ls *.tar.gz|xargs -tn 1 tar -tf|grep folder1/folder2

The -t flag for xargs shows you what it is executing each time.

Do either of these help?



Robin
# 6  
Old 11-29-2017
Quote:
Originally Posted by rbatte1
You could try:-
Code:
for targz in *.tar.gz
do
   echo "$targz" >&2                    # Write to STDERR, so show up on the screen
   tar -tf $targz
done | grep folder1/folder2

...or...
Code:
ls *.tar.gz|xargs -tn 1 tar -tf|grep folder1/folder2

The -t flag for xargs shows you what it is executing each time.

Do either of these help?



Robin
Hi Robin,
With pipe buffering, I don't think there is any guarantee that the output from the echo or from xargs -t sent to STDERR won't appear on the screen before some output from grep of the previous archive. And the same thing could happen if you try to capture both STDOUT and STDERR and redirect them to a single output file.

To get the name of the archive being processed in the standard output stream and survive the grep in the pipeline, one might try something more like:
Code:
for targz in *.tar.gz
do
   echo "folder1/folder2 files in $targz..."   # Write to STDOUT so script output can be redirected
   tar -tf $targz
done | grep folder1/folder2

As RudiC mentioned, some versions of tar will get lost if given a zipped archive. But, assuming that the tar on april's system gunzips a file automatically if its name ends in .gz, this should work.
This User Gave Thanks to Don Cragun For This Post:
rbatte1 (11-29-2017)
# 7  
Old 11-29-2017
I tried both methods from Robin and they both worked. I see the file names and the grep results following each file name.

I tried Don's suggestion
Code:
echo "folder1/folder2 files in $targz..."

. That did not give me the file names. I only see the grep results.

Thank you all for your help.
Login to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Similar Threads More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Proper chaining and piping of commands SkySmart Shell Programming and Scripting 2 01-08-2016 02:46 AM
Xargs: multiple commands to each argument urello Shell Programming and Scripting 1 03-02-2015 01:25 AM
Piping commands iliya24 UNIX for Dummies Questions & Answers 7 03-28-2013 11:24 PM
Piping multiple commands jrymer UNIX for Dummies Questions & Answers 3 08-06-2012 04:48 PM
piping problem with xargs ivpz Shell Programming and Scripting 7 08-09-2011 12:30 PM
Piping results of 'ls' to 'find' using 'xargs' MartynAbbott Solaris 4 08-21-2009 10:00 AM
Piping through commands read as variables from file fitzwilliam Shell Programming and Scripting 1 07-20-2009 09:19 AM
use of xargs and prune piping with find command. venkatesht UNIX for Dummies Questions & Answers 1 01-28-2009 09:29 AM
Piping in Perl Raynon Shell Programming and Scripting 11 09-07-2008 10:54 PM
Piping from device? gary101 Shell Programming and Scripting 0 03-31-2007 07:21 AM
Can Xargs execute multiple commands of evry input file nilesrex Shell Programming and Scripting 4 08-30-2006 09:39 AM
piping lnatz Shell Programming and Scripting 1 07-14-2006 03:30 AM
Help (Piping ls, tr, cut) scan Shell Programming and Scripting 2 02-11-2006 08:40 AM
redirecting/piping crashnburn UNIX for Advanced & Expert Users 3 03-25-2002 03:07 PM
Executing commands with xargs bmopal UNIX for Dummies Questions & Answers 1 08-13-2001 05:05 PM
All times are GMT -4. The time now is 05:25 PM.

Unix & Linux Forums Content Copyright 1993-2018. All Rights Reserved.