Pipelining with tar


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Pipelining with tar
# 8  
Old 10-21-2010
Usually, you can find relative paths and control where it picks up and drops with cd on each end!

If your net is so fast and your cpu is so slow, compress is faster than gzip -1.

I feel like I am overhearing half of a phone conversation. Can you summarize all your needs in one list?



---------- Post updated at 03:01 PM ---------- Previous update was at 02:55 PM ----------




Quote:
Originally Posted by Corona688
It's trying to extract it in /opt/ which won't work unless you're root. And if you're still testing don't do it as root! Stick in your home dir and test in there.

Your parameters don't look quite right.

Code:
tar cvf - usr/local/ | gzip -c - | gunzip -c - | tar -C opt/ -xvf -

By doing 'opt/' instead of '/opt/' you can test in an opt dir inside your current dir instead of stomping over /opt/ itself.
The '-c -' on gzip/gunzip actually do nothing, and the point of bothering to compress is to put rsh or ssh in the chain before gunzip and so keep the network from being such a bottleneck. If that was no concern, rcp -rp would move it all.
# 9  
Old 10-22-2010
Alright here are my updated needs...

I need to gather all the directories and files (using tar) from one machine's usr/local/ directory, compress the file, then using ssh, transfer it to another machine, uncompress it, and extract it into that machine's opt/ directory.

So as far as I can tell the first half will remain the same...
tar cf - usr/local/ | gzip -c - | #now I am assuming I will be using cd to change to the local machine... and then continuing with the last half of the code (note: this is assuming that I am already connected to the remote machine through ssh)

Last edited by huntreilly25; 10-22-2010 at 04:52 PM.. Reason: update
# 10  
Old 10-22-2010
Code:
(
 cd source-relative-dir
 Find and archive to stdout
)| compression-tool-of-your-choice | remote-shell-tool-of-your-choice  host-and-user-options  "(
 cd target-relative-dir
 matching-decompress-tool | matching-dearchiver-from-stdin
)"

or just:

scp -rpC local-dir remote-id@remote-host:remote-drop-parent-dir

What's your affection for tar due to, sea chanteys? Smilie
This User Gave Thanks to DGPickett For This Post:
# 11  
Old 10-22-2010
Sweet!! I think I figured it out!

Code:
ssh userB@domainB
tar cf - usr/local/ | gzip -c - | ssh userA@domainA " gunzip -c - | tar -C opt/ -xf -"

using some practice directories this has seemed to work so far Smilie
# 12  
Old 10-22-2010
Try gzip -9, gzip -1 and compress (gzcat handles all these)!

Try 2 at least, or 3 at once if there is CPU left over. If two is better, three means there are 2 while you are starting another.
# 13  
Old 10-22-2010
will do!
and as far as my affection with tar goes...its when my supervisor is telling me to use, so I haven't really been looking at any other archiving tools yet.

Thanks a lot for the help!
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. AIX

Tar - pre-checking before making the Tar file

Coming from this thread, just wondering if there is an option to check if the Tar of the files/directory will be without any file-errors without actually making the tar. Scenario: Let's say you have a directory of 20GB, but you don't have the space to make Tar file at the moment, and you want... (14 Replies)
Discussion started by: filosophizer
14 Replies

2. UNIX for Dummies Questions & Answers

Do I need to extract the entire tar file to confirm the tar folder is fine?

I would like to confirm my file.tar is been tar-ed correctly before I remove them. But I have very limited disc space to untar it. Can I just do the listing instead of actual extract it? Can I say confirm folder integrity if the listing is sucessful without problem? tar tvf file1.tar ... (1 Reply)
Discussion started by: vivien_chu
1 Replies

3. Shell Programming and Scripting

tar command to explore multiple layers of tar and tar.gz files

Hi all, I have a tar file and inside that tar file is a folder with additional tar.gz files. What I want to do is look inside the first tar file and then find the second tar file I'm looking for, look inside that tar.gz file to find a certain directory. I'm encountering issues by trying to... (1 Reply)
Discussion started by: bashnewbee
1 Replies

4. Shell Programming and Scripting

Bash pipelining

Hello, There is a symbolic link in a folder. I would like to read destination of this link and get base name of pointed file. Let's say, there is symlink : symlink -> ../file.txt. I can do that what I want by this script: readlink symlink | while read var do echo `basename $var` done but... (1 Reply)
Discussion started by: scdmb
1 Replies

5. UNIX for Dummies Questions & Answers

tar -cvf test.tar `find . -mtime -1 -type f` only tar 1 file

Hi all, 4 files are returned when i issue 'find . -mtime -1 -type f -ls'. ./ora_475244.aud ./ora_671958.aud ./ora_934052.aud ./ora_934050.aud However, when I issued the below command: tar -cvf test.tar `find . -mtime -1 -type f`, the tar file only contains the 1st file -... (2 Replies)
Discussion started by: ahSher
2 Replies

6. UNIX for Advanced & Expert Users

How to create a Tar of multiple Files in Unix and FTP the tar to Windows.

Hi, On my Unix Server in my directory, I have 70 files distributed in the following directories (which have several other files too). These files include C Source Files, Shell Script Source Files, Binary Files, Object Files. a) /usr/users/oracle/bin b) /usr/users/oracle... (1 Reply)
Discussion started by: marconi
1 Replies

7. UNIX for Advanced & Expert Users

Tar utility (untar a .tar file) on VxWorks

Hi All Can someone pls guide me if there any utility to compress file on windows & uncompress on vxworks I tried as - - compressed some folders on windows ... i created .tar ( to maintain directory structure ) and compressed to .gz format. - on VxWorks i have uncompressed it to .tar... (1 Reply)
Discussion started by: uday_01
1 Replies

8. Shell Programming and Scripting

Pipelining

My problem is more a question of how to do it more elegantly than how to do it at all. The problem: I have a pipeline which has to write to the screen AND to a logfile: proc1 | tee -a <logfile> What makes things difficult is i also need the return code of proc1. But proc1 | tee -a... (6 Replies)
Discussion started by: bakunin
6 Replies

9. UNIX for Advanced & Expert Users

Does tar do crc checking on a tape or tar file?

Trying to answer a question about whether tar table-of-contents is a good tool for verifying tape data. (1 Reply)
Discussion started by: tjlst15
1 Replies

10. Programming

Pipelining Processes

I am doing a program that will calculate the 199th fibonacci number using pipelines and multiple processes for each calculation. I have figured everything out except why none of the processes before the last one never execute the findfib() function; Our teacher gave us the pipeline code to... (3 Replies)
Discussion started by: Trivialnight
3 Replies
Login or Register to Ask a Question