Unix/Linux Go Back    


UNIX for Beginners Questions & Answers If you're not sure where to post a Unix or Linux question, post it here. All unix and Linux beginners welcome in this forum!

Piping commands using xargs

UNIX for Beginners Questions & Answers


Reply    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 11-29-2017   -   Original Discussion by april
april's Unix or Linux Image
april april is offline
Registered User
 
Join Date: Apr 2007
Last Activity: 8 December 2017, 12:01 PM EST
Posts: 16
Thanks: 0
Thanked 0 Times in 0 Posts
Piping commands using xargs

Need help in piping commands using xargs

I have several .tar.gz files that I need to list the folder content in a subdirectory.
For example,


Code:
a.tar.gz
b.tar.gz
c.tar.gz

The following command works great for each .tar.gz file but it's a pain to run the tar command for each file.


Code:
tar -tf a.tar.gz|grep folder1/folder2

I tried this command and failed.


Code:
ls *.tar.gz|xargs tar -tf|grep folder1/folder2

I got this error message "tar: a.tar.gz: Not found in archive" for each gz file.

What do I need to fix to run this command successfully?
Or do I need to use a different command to process all the .tar.gz files?

Thank you.

Moderator's Comments:
Piping commands using xargs Please use CODE tags as required by forum rules!

Last edited by RudiC; 11-29-2017 at 09:20 AM.. Reason: Added CODE tags.
Sponsored Links
    #2  
Old Unix and Linux 11-29-2017   -   Original Discussion by april
RudiC's Unix or Linux Image
RudiC RudiC is offline Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 17 January 2018, 2:48 PM EST
Location: Aachen, Germany
Posts: 11,940
Thanks: 352
Thanked 3,678 Times in 3,376 Posts
I'm afraid there are more than one erroneous approaches in above:
- the files seem to be gzipped (by their ending) - gunzip before presenting them to tar. In fact, I'm slightly alienated that your single manual command should run error free...
- Shouldn't there be a message like tar: This does not look like a tar archive?
- the -f option takes just one single file (except for a multi-volume archive which I doubt you have here)

You might want to try the -n1 option to xargs and see how far you get.
Sponsored Links
    #3  
Old Unix and Linux 11-29-2017   -   Original Discussion by april
rbatte1's Unix or Linux Image
rbatte1 rbatte1 is offline Forum Staff  
Root armed
 
Join Date: Jun 2007
Last Activity: 17 January 2018, 12:01 PM EST
Location: Lancashire, UK
Posts: 3,437
Thanks: 1,492
Thanked 671 Times in 604 Posts
Perhaps something like this might do it:-

Code:
for targz in *.tar.gz
do
   tar -tf $targz
done | grep folder1/folder2

The problem you are seeing is because tar is seeing the input as -f [I]first_file_listed item1_to_extract item2_to_extract so you might get away with adding -n 1 to xargs like this:-

Code:
ls *.tar.gz|xargs -n 1 tar -tf|grep folder1/folder2

This will at least run tar for each file separately. I'm not sure how it will handle the pipe to grep, but it should work.



I hope that these help. If not, please run it with debug on your shell (i.e. set -x first) and paste the output in CODE tags. It would help if you have a small set of small (few members) tar files to keep the output manageable.



Kind regards,
Robin
    #4  
Old Unix and Linux 11-29-2017   -   Original Discussion by april
april's Unix or Linux Image
april april is offline
Registered User
 
Join Date: Apr 2007
Last Activity: 8 December 2017, 12:01 PM EST
Posts: 16
Thanks: 0
Thanked 0 Times in 0 Posts
Piping commands using xargs

Thank you, Robin.

I tried both examples you provided and they both list the folder content. The only issue is both examples only print out the results without displaying the input file name so I don't know which input file produces the results.

What needs to be modified to print out the input file name and the results?

Thank you again for your help.
Sponsored Links
    #5  
Old Unix and Linux 11-29-2017   -   Original Discussion by april
rbatte1's Unix or Linux Image
rbatte1 rbatte1 is offline Forum Staff  
Root armed
 
Join Date: Jun 2007
Last Activity: 17 January 2018, 12:01 PM EST
Location: Lancashire, UK
Posts: 3,437
Thanks: 1,492
Thanked 671 Times in 604 Posts
You could try:-

Code:
for targz in *.tar.gz
do
   echo "$targz" >&2                    # Write to STDERR, so show up on the screen
   tar -tf $targz
done | grep folder1/folder2

...or...


Code:
ls *.tar.gz|xargs -tn 1 tar -tf|grep folder1/folder2

The -t flag for xargs shows you what it is executing each time.

Do either of these help?



Robin
Sponsored Links
    #6  
Old Unix and Linux 11-29-2017   -   Original Discussion by april
Don Cragun's Unix or Linux Image
Don Cragun Don Cragun is offline Forum Staff  
Administrator
 
Join Date: Jul 2012
Last Activity: 17 January 2018, 1:40 PM EST
Location: San Jose, CA, USA
Posts: 10,913
Thanks: 608
Thanked 3,817 Times in 3,261 Posts
Quote:
Originally Posted by rbatte1 View Post
You could try:-

Code:
for targz in *.tar.gz
do
   echo "$targz" >&2                    # Write to STDERR, so show up on the screen
   tar -tf $targz
done | grep folder1/folder2

...or...


Code:
ls *.tar.gz|xargs -tn 1 tar -tf|grep folder1/folder2

The -t flag for xargs shows you what it is executing each time.

Do either of these help?



Robin
Hi Robin,
With pipe buffering, I don't think there is any guarantee that the output from the echo or from xargs -t sent to STDERR won't appear on the screen before some output from grep of the previous archive. And the same thing could happen if you try to capture both STDOUT and STDERR and redirect them to a single output file.

To get the name of the archive being processed in the standard output stream and survive the grep in the pipeline, one might try something more like:


Code:
for targz in *.tar.gz
do
   echo "folder1/folder2 files in $targz..."   # Write to STDOUT so script output can be redirected
   tar -tf $targz
done | grep folder1/folder2

As RudiC mentioned, some versions of tar will get lost if given a zipped archive. But, assuming that the tar on april's system gunzips a file automatically if its name ends in .gz, this should work.
The Following User Says Thank You to Don Cragun For This Useful Post:
rbatte1 (11-29-2017)
Sponsored Links
    #7  
Old Unix and Linux 11-29-2017   -   Original Discussion by april
april's Unix or Linux Image
april april is offline
Registered User
 
Join Date: Apr 2007
Last Activity: 8 December 2017, 12:01 PM EST
Posts: 16
Thanks: 0
Thanked 0 Times in 0 Posts
I tried both methods from Robin and they both worked. I see the file names and the grep results following each file name.

I tried Don's suggestion

Code:
echo "folder1/folder2 files in $targz..."

. That did not give me the file names. I only see the grep results.

Thank you all for your help.
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Piping commands iliya24 UNIX for Dummies Questions & Answers 7 03-28-2013 11:24 PM
Piping multiple commands jrymer UNIX for Dummies Questions & Answers 3 08-06-2012 04:48 PM
piping problem with xargs ivpz Shell Programming and Scripting 7 08-09-2011 12:30 PM
Piping results of 'ls' to 'find' using 'xargs' MartynAbbott Solaris 4 08-21-2009 10:00 AM
use of xargs and prune piping with find command. venkatesht UNIX for Dummies Questions & Answers 1 01-28-2009 09:29 AM



All times are GMT -4. The time now is 04:18 PM.