Curl and Tar


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Curl and Tar
# 1  
Old 04-11-2013
Curl and Tar

Hi Experts,

I have a situation where I need to extract the code from a code base using curl which is returning it as zipped file.

1. I want the contents of this tar to be untared in the current directory i.e. the dir from where I am running this curl and tar command.

Below is the command I used -
Code:
$ curl -LsS "http://xyz.com/nexus/service/local/" | tar xz

But this is creating a directory under which has all the files untarred. I am going to put this command in a script, and in the current directory there will be other dirs so I wont know by script which dir to look inside for the code. So I want the code to be extracted to the current dir.

2. I have tried -C option of tar but looks like the tar is created using absolute path and its not untaring in current dir.

3. I tried couple of options using wget but its downloading it as .aar file and tar az errors out
Code:
$ wget --quiet "http://xyz.com/nexus/service/local/" | tar xz
gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error exit delayed from previous errors

Please help on how to extract the file in the current directory either using curl/wget and which options of tar

Last edited by Franklin52; 04-12-2013 at 03:04 AM.. Reason: Please use code tags for data and code samples
# 2  
Old 04-11-2013
Is there possibly a problem that tar is expecting names of files to archive, but is instead getting the output from curl (or wget)?
# 3  
Old 04-11-2013
Usually, for tar like this, need f - for stdin as tar file and ( cd ... ; tar ) to lay it down in the right subtree:
Code:
curl-gets-remote-tgz | ( cd head_dir ; tar xzf - )

# 4  
Old 04-12-2013
Thanks both for the response but the below command is still extracting the files into a new directory

curl-gets-remote-tgz | ( cd head_dir ; tar xzf - )
Is it because of how its been tared by the repository? Guys please help, I am stuck on this.

Otherwise is there a way to find the tar file returned by the curl command, for ex say i run wget or curl command it returns a tar file, how do i know the name of the file?
On the same lines when the tar is creating a new dir and extracting the files, is there a way to know the name of the directory the tar command created to extract the files.If i get to know the name of the directory then I can pull the files from it to the current directory.
# 5  
Old 04-12-2013
What is "curl-gets-remote-tgz" command? Is that a standard UNIX command? I don't find it on a search.

Quote:
ex say i run wget or curl command it returns a tar file,
Do wget or curl return a tar file? I'm not aware that they do.
# 6  
Old 04-12-2013
Hanson, sorry there is no such command, it meant the curl command with the url...

Yes looks like curl command is returning the tar file.

Any help please.
# 7  
Old 04-12-2013
If what you're trying to do is strip a leading directory component from every path in a gzip'd tar archive, the following untested pipeline should do it:
Code:
curl ... | gzip -dc | pax -r -s '#^[^/]*/##'

Needless to say, this can cause collisions if there is more than one archive member at the top level.

If this is not what you're trying to accomplish, then you should properly document the situation with a sample listing from the archive as it exists, the exact command used to unpack it, a listing of the resulting filesystem hierarchy, and a simulated listing of the desired filesystem hierarchy.

Regards,
Alister

---------- Post updated at 12:06 PM ---------- Previous update was at 12:02 PM ----------

Quote:
Originally Posted by whoami191
Otherwise is there a way to find the tar file returned by the curl command, for ex say i run wget or curl command it returns a tar file, how do i know the name of the file?
On the same lines when the tar is creating a new dir and extracting the files, is there a way to know the name of the directory the tar command created to extract the files.If i get to know the name of the directory then I can pull the files from it to the current directory.
Create an empty temp directory to work with. Anything that's put there is what you're looking for. You don't need to know its name; just use sh pattern matching wildcards, e.g. for f in *; do ... done.

Regards,
Alister
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. AIX

Tar - pre-checking before making the Tar file

Coming from this thread, just wondering if there is an option to check if the Tar of the files/directory will be without any file-errors without actually making the tar. Scenario: Let's say you have a directory of 20GB, but you don't have the space to make Tar file at the moment, and you want... (14 Replies)
Discussion started by: filosophizer
14 Replies

2. Linux

How to install *.pkg.tar.tar on Linux?

Hi Guys, I need some suggestion as how to install openssl-1.0.1.g-1-i686.pkg.tar.tar openssl-1.0.1.g-1-x86_64.pkg.tar.tar on Linux system? is for 32bit system and other is for 64 bit system linux machine. How to untar them? I tried few commands but did not work. I need to upgrade to... (5 Replies)
Discussion started by: manalisharmabe
5 Replies

3. UNIX for Dummies Questions & Answers

Do I need to extract the entire tar file to confirm the tar folder is fine?

I would like to confirm my file.tar is been tar-ed correctly before I remove them. But I have very limited disc space to untar it. Can I just do the listing instead of actual extract it? Can I say confirm folder integrity if the listing is sucessful without problem? tar tvf file1.tar ... (1 Reply)
Discussion started by: vivien_chu
1 Replies

4. Shell Programming and Scripting

tar command to explore multiple layers of tar and tar.gz files

Hi all, I have a tar file and inside that tar file is a folder with additional tar.gz files. What I want to do is look inside the first tar file and then find the second tar file I'm looking for, look inside that tar.gz file to find a certain directory. I'm encountering issues by trying to... (1 Reply)
Discussion started by: bashnewbee
1 Replies

5. Shell Programming and Scripting

tar command dont tar to original directory

HI, if I have a tarfile called pmapdata.tar that contains tar -tvf pmapdata.tar -rw-r--r-- 0/0 21 Oct 15 11:00 2009 /var/tmp/pmapdata/pmap4628.txt -rw-r--r-- 0/0 21 Oct 14 20:00 2009 /var/tmp/pmapdata/pmap23752.txt -rw-r--r-- 0/0 1625 Oct 13 20:00 2009... (1 Reply)
Discussion started by: borderblaster
1 Replies

6. UNIX for Dummies Questions & Answers

tar -cvf test.tar `find . -mtime -1 -type f` only tar 1 file

Hi all, 4 files are returned when i issue 'find . -mtime -1 -type f -ls'. ./ora_475244.aud ./ora_671958.aud ./ora_934052.aud ./ora_934050.aud However, when I issued the below command: tar -cvf test.tar `find . -mtime -1 -type f`, the tar file only contains the 1st file -... (2 Replies)
Discussion started by: ahSher
2 Replies

7. UNIX for Advanced & Expert Users

How to create a Tar of multiple Files in Unix and FTP the tar to Windows.

Hi, On my Unix Server in my directory, I have 70 files distributed in the following directories (which have several other files too). These files include C Source Files, Shell Script Source Files, Binary Files, Object Files. a) /usr/users/oracle/bin b) /usr/users/oracle... (1 Reply)
Discussion started by: marconi
1 Replies

8. UNIX for Advanced & Expert Users

Tar utility (untar a .tar file) on VxWorks

Hi All Can someone pls guide me if there any utility to compress file on windows & uncompress on vxworks I tried as - - compressed some folders on windows ... i created .tar ( to maintain directory structure ) and compressed to .gz format. - on VxWorks i have uncompressed it to .tar... (1 Reply)
Discussion started by: uday_01
1 Replies

9. UNIX for Advanced & Expert Users

Does tar do crc checking on a tape or tar file?

Trying to answer a question about whether tar table-of-contents is a good tool for verifying tape data. (1 Reply)
Discussion started by: tjlst15
1 Replies
Login or Register to Ask a Question