Uncompress several tar.gz files inside several folders


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Uncompress several tar.gz files inside several folders
# 1  
Old 07-10-2018
Uncompress several tar.gz files inside several folders

Hi,

I need to uncompress sevral tar.gz files that are located insides sevral folders using a shell script.


Folder tree looks like this :
Code:
/folder/001
/folder/002
/folder/003

Inside each folder dossier (001,002,003) we can find 1 or several tar.gz files.


I've scripted something like this :
Code:
!/bin/bash
liste_fichier="/folder/*"
for fichier in $liste_fichier do
tar -xzvf $fichier/*.tar.gz --directory $fichier/
done
exit

Issue:
This scripts only works with one tar.gz file inside the folder.
When it finds more then one tar.gz file it crashes the message tar : <path/file.tar.gz> : not found in archive


Some other tests:
-I've tested the code below inside one folder(001) containing several tar.gz.file :
Code:
for i in *.tar.gz 
do 
tar -xzvf $i 
done

It works.


I try to run it inside my script without success. To many loops I guess

Any "good" ideas are welcome.
Thank you

Last edited by shellX; 07-10-2018 at 11:08 AM.. Reason: Added CODE & ICODE tags
# 2  
Old 07-10-2018
Wold you be better with the find command? You can ask it to locate files of the name format you need and call a command for each one found.

Perhaps this will get you started:-
Code:
find /base/path/to/files -name "*.tar.gz" -print -exec tar -tzvf {} \;

Beware that if the files found contain full path archives, you might overwrite any file on your server. Additionally, if you are extracting, you might find that your current directory gets all the extracted files into it if they use relative paths. I have set the command to just list the contents for now so you can check what it will try to do.



I hope that this helps,
Robin
# 3  
Old 07-10-2018
Hi,

That's the point, I must extract the file in the same folders.
Besides it doesn't work.
eRR mEssage : missing parameters for "-exec"

Last edited by shellX; 07-10-2018 at 11:52 AM..
# 4  
Old 07-10-2018
What is your operating system and shell used ?
Gnu find printf option can be of help if you operating system supports it.

Following should fill you requirement regardless of the paths used in archives (absolute or relative).

Code:
find /folder/00[1-3] -name '*.tar.gz' -printf 'tar -xvzf %h/%f -C %h\n' | sh

You can match folders with regex or input them along each other.
Code:
find /folder001 /folder002 /folder003 -name '*.tar.gz' -printf 'tar -xvzf %h/%f -C %h\n' | sh

More portable solution could include a awk program which will generate those commands instead of gnu find printf or some xargs magic possibly.
Be sure to get back, if you do not have gnu find.


Hope that helps.
Regards
Peasant.
# 5  
Old 07-11-2018
@PEASANT

Hi,

This solution would be great for a small number of folders.
But unfortunnatelly "for me" I've something like 1000 sub-folders with tar.gz files to uncompress.

I cannot type the names of 1000 folders and subfolders in a script.

That's why my idea in the beggining was to accept a variable "$1"to store the path to the folders containing the tar.gz files
This way the script could be executed automatically for each folder,
(e.g.) below

Code:
!/bin/bash 
liste_fichier="/folder/$1/*" 
for fichier in $liste_fichier do 
tar -xzvf $fichier/*.tar.gz --directory $fichier/ 
done 
exit

Moderator's Comments:
Mod Comment
Please wrap all code, files, input & output/errors in CODE tags (or ICODE tags for in-line snippets)
It makes it far easier to read and preservers spaces for indenting and/or fixed-width data.

Last edited by rbatte1; 07-11-2018 at 12:53 PM.. Reason: Added CODE tags
# 6  
Old 07-11-2018
Assuming that your code actually starts #!/bin/bash then you can actually simplify this to:-
Code:
#!/bin/bash

[ -z "$1" ] && echo "On doit donner un dossier." && exit

for ficher in /folder/$1/*
do
   echo ; echo "$ficher" ; sleep 1                                 # Remove this if you like, it is just to illustrate the loop
   tar -xzvf $fichier/*.tar.gz --directory $fichier/
done

The shell will expand the directory name with wildcard to make a long list for your for loop.


I've assumed French language for output from your use of ficher. Apologies if that is incorrect.



Does that help?
Robin

Last edited by rbatte1; 07-11-2018 at 01:01 PM.. Reason: Added a debug line to show progress through the file list
# 7  
Old 07-12-2018
Hi Robin

That code is very good.

The issue is it doesn't work when there are more then one tar.gz file in the folder.

--> Lets suposse $1 = folder
--> The content of my folder "folder" is
Code:
[shellx] $ ls -l folder
total 16
drwxr-xr-x 2 shellx mrv 4096 11 juil. 14:36 001
drwxr-xr-x 2 shellx mrv 4096 11 juil. 14:37 002
drwxr-xr-x 2 shellx mrv 4096 11 juil. 14:37 003
drwxr-xr-x 2 shellx mrv 4096 11 juil. 14:34 004

--> sub-folders (001, 002, 003, 004) content
Code:
[shellx] $ ls  folder/*
folder/001:
CARDV022A0.tar.gz  CARDV02976.tar.gz  CARDV02A0.tar.gz  CARDV976.tar.gz

folder/002:
CARDV02976.tar.gz  CARDV976.tar.gz

folder/003:
CARDV022A0.tar.gz  CARDV02976.tar.gz  CARDV976.tar.gz

folder/004:
CARDV02976.tar.gz  CARDV976.tar.gz



-->I've set 2 loops in the new script

Code:
#!/bin/bash
liste_rep="folder/*"
folder="$(ls folder)"
echo "LISTE REP :" $liste_rep
        for rep in "$liste_rep"
        do
                echo "AFFICHE REP :"$rep
                #for fichier in "$(ls /folder/*)" # test
                for fichier in $liste_rep/*.tar.gz
                do
                        #cd $liste_rep
                        echo "FICHIER :" $fichier
                        tar -zxvf $fichier   --directory $liste_rep/
                done
        done
exit


-->Variables content verified
Code:
[presta2] $ echo $liste_rep
folder/001 folder/002 folder/003 folder/004
-(jeu. juil. 12 10:13:41)--(mrvpgsa001:~)-
[presta2] $ echo $folder
001 002 003 004


Resultat d'execution
Code:
LISTE REP : folder/001 folder/002 folder/003 folder/004  --->> folder liste correct
AFFICHE REP :folder/*  --->> because $rep = "$list_rep"??? and should be $list_rep without " " ?
FICHIER : folder/001/CARDV022A0.tar.gz  --> Var fichier is correctly set
tar: folder/002 : non trouvé dans l'archive ---> How is it ? if  --> tar -zxvf $fichier
tar: folder/003 : non trouvé dans l'archive
tar: folder/004 : non trouvé dans l'archive
tar: Arrêt avec code d'échec à cause des erreurs précédentes
FICHIER : folder/001/CARDV02976.tar.gz
tar: folder/002 : non trouvé dans l'archive
tar: folder/003 : non trouvé dans l'archive
tar: folder/004 : non trouvé dans l'archive
tar: Arrêt avec code d'échec à cause des erreurs précédentes
FICHIER : folder/001/CARDV02A0.tar.gz
tar: folder/002 : non trouvé dans l'archive
tar: folder/003 : non trouvé dans l'archive
tar: folder/004 : non trouvé dans l'archive
tar: Arrêt avec code d'échec à cause des erreurs précédentes
FICHIER : folder/001/CARDV976.tar.gz

--> If uncomment line "#cd $liste_rep" inside second loop
--> we gett error message below

Code:
[presta2] $ ./decomp2.sh
LISTE REP : folder/001 folder/002 folder/003 folder/004
AFFICHE REP :folder/*
FICHIER : folder/001/CARDV022A0.tar.gz
tar (child): folder/001/CARDV022A0.tar.gz : la fonction open a échoué: Aucun fichier ou dossier de ce type
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
FICHIER : folder/001/CARDV02976.tar.gz
tar (child): folder/001/CARDV02976.tar.gz : la fonction open a échoué: Aucun fichier ou dossier de ce type
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now


Any ideas are welcome




Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 07-12-2018 at 12:59 PM.. Reason: Added CODE tags.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to delete all the files and folders inside all the directories except some specific directory?

hi, i have a requirement to delete all the files from all the directories except some specific directories like archive and log. for example: there are following directories such as A B C D Archive E Log F which contains some sub directories and files. The requirement is to delete all the... (7 Replies)
Discussion started by: Little
7 Replies

2. Shell Programming and Scripting

Editing all files inside folders and updating pos 88-94 with continuous numbering.

Here is expected output:- For each file with following file name pattern we need to look at position 1 inside first file matching our search criteria if first letter of the line is 5 then position 88-94 will be 0000001 then look for line immediately after 5 which starts with i.e. position 1 = 8... (6 Replies)
Discussion started by: lancesunny
6 Replies

3. Shell Programming and Scripting

Listing folders that have a file inside them

Hi guys, I'm new to the forums and putting my foot in the door with SED and AWK. I was wondering if someone could help me as I think I'm making this harder than it needs to be... I have a list of folders named as urls, inside these are log files and possibly a 'status' file. I'm trying to get... (6 Replies)
Discussion started by: KakersUK
6 Replies

4. Shell Programming and Scripting

How to tar files inside a script?

Hi , I have a file which contains few file names. I need to tar those files but am unable to do inside the script. Any help will be useful. cat /tmp/test aa.txt bb.txt cc.txt I have tried the below code but its not working. for i in `cat /tmp/test';do tar -cvf TEST.tar $i;done (9 Replies)
Discussion started by: rogerben
9 Replies

5. Shell Programming and Scripting

tar command to explore multiple layers of tar and tar.gz files

Hi all, I have a tar file and inside that tar file is a folder with additional tar.gz files. What I want to do is look inside the first tar file and then find the second tar file I'm looking for, look inside that tar.gz file to find a certain directory. I'm encountering issues by trying to... (1 Reply)
Discussion started by: bashnewbee
1 Replies

6. Shell Programming and Scripting

uncompress tar.gz files

i Have 150 tar.gz files and i need to uncompress and extract all the files from those 150 tar.gz and i will have 150 files on daily basis. could any one help me out with the script to uncompress tar.gz files. (5 Replies)
Discussion started by: gaddamshashank
5 Replies

7. Solaris

Cannot uncompress the tar.gz

I've downloaded the tomcat from http://archive.apache.org/dist/jakarta/tomcat-5/v5.0.30/bin/ at window 2000 and then ftp to Solaris 5.9 in binary mode. gunzip -c jakarta-tomcat-5.0.30.tar.gz | tar -xvf - tar: directory checksum error if try this one, gzip -d jakarta-tomcat-5.0.30.tar.gz... (2 Replies)
Discussion started by: sbox
2 Replies

8. UNIX Desktop Questions & Answers

file zip,rar,tar,compress,uncompress,unzip,unrar

i want know how to compress and uncompress file using unix, compress uncompress,zip,unzip,rar,unrar,how its work and more about this.:confused: (1 Reply)
Discussion started by: ismael xavier
1 Replies

9. Shell Programming and Scripting

rsh and uncompress and tar

Hello, how can I send a compressed file (file.tar.Z) on another server ans uncompresse and extract it there ? My code which is like this uncompress *tar.Z | rsh serveurB -l user1 "cd /d15/store/; tar xf -" does not function. Many thanks before. (4 Replies)
Discussion started by: big123456
4 Replies

10. UNIX for Dummies Questions & Answers

uncompress files

I get a compressed file for linux and I want to uncompress it in Unix. Is that possible? I try with te command uncompress but it didn´t work. T.hanks (3 Replies)
Discussion started by: diegoe
3 Replies
Login or Register to Ask a Question