Unzip in bash script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Unzip in bash script
# 1  
Old 02-09-2010
Unzip in bash script

My script works like this.
1. First for loop - checks if it can access zip parameters in a directory
2. If unzip fails for any of the file in that directory, then it goes inside the second for loop - to list which file is that
3. My doubt is..Is there any way I can tell status for every 5000 files in the first for loop. like processed 5000 files...10000 files etc...for monitoring
4.I can very well keep a counter and do this logic in second for loop, since it loops every file.but this will execute only when there are corrupted files

Code:
for j in `find $i -name "abc*" -type d`; do
              unzip -qT $j/\* 2>>/dev/null;
             if [ $? -ne 0 ]; then
                for k in `find $j -type f`; do
                   unzip -qT $k 2>>/dev/null;
                done
             fi
done


Any inputs pls...

Thanks

Last edited by pludi; 02-09-2010 at 06:54 AM.. Reason: code tags, please...
# 2  
Old 02-09-2010
At the first line of your loop :
Code:
((i++)); (($i%5000)) || echo "$i files processed"

# 3  
Old 02-09-2010
display progress details - for every 5000 files- when unzipping the files in directory

Frans,

Code:
for j in `find $i -name "abc*" -type d`; do 
// Here $i is directory.
((i++)); (($i%5000)) || echo "$i files processed"
// Here it will display the count of sub-directories. I need to display list of
files in sub-directories.

Pls help.

Thanks

Last edited by Scott; 02-09-2010 at 11:39 AM.. Reason: Code tags, PLEASE!
# 4  
Old 02-09-2010
Apologize : Replace i with another unused letter !
I think you could try to avoid the first check (unzip -qT $j/\*), believe that it should'nt slow-down the script to much and you don't need to look twice if there is an error with unzip.
I suggest (some modif. to make it more readable)
Code:
for FILE in $(ls abc*/*) # This should work
do
    ((n++)); (($n%5000)) || echo "$n files processed"
    unzip -qT $FILE 2>>/dev/null || echo "Error in file $FILE"
done

# 5  
Old 02-10-2010
I get error saying all the files are corrupted.

Code:
 for FILE in $(ls $j) # This should work
// $j - last level subdirectory which has list of zipfiles without .zip extension
do
   ((n++)); (($n%5000)) || echo "$n files processed" 
    
    unzip -qT $FILE 2>>/dev/null 
    if [ $? -ne 0 ]; then
           echo $FILE is BAD 
    fi
   
done

Instead if I use as I mentioned in the below code it gives the correct output.

Code:
 for k in `find $j -type f`; do

performance :

if I run the code with the 2 loops - 1. scan all the files in the directory with one unzip system call
2. If any of the dir has corrupted files, loop the second directory and list the files
It takes 0m5.186s this time to execute 1.3 lakhs files

If i remove the outer loop, it takes 1m46.280s to execute 1.3 lakh files

Our server will hold data close to 1.4 TB. There is a performance hit in these 2 ways of implementation. Thats the reason, I wanted to count the files in the outer for loop (while testing the unzip for all the files in one sub-directory) Smilie

Frans, your help pls.

Thanks,
Vidhya.

Last edited by Scott; 02-10-2010 at 04:42 AM.. Reason: Code tags
# 6  
Old 02-10-2010
Can you tell me how many directories and how many files you have in each one.
a command like
Code:
(( n+=$(find $j -type f | wc -l) ))

would increment n by the number of files in the dir.
Please put code tags around code
# 7  
Old 02-10-2010
Quote:
Originally Posted by frans
Apologize : Replace i with another unused letter !
I think you could try to avoid the first check (unzip -qT $j/\*), believe that it should'nt slow-down the script to much and you don't need to look twice if there is an error with unzip.
I suggest (some modif. to make it more readable)
Code:
for FILE in $(ls abc*/*) # This should work


Not only is ls unnecessary, but it will break your script if any filenames contain whitespace or other pathological characters.

Use wildcard expansion directly:
Code:
for FILE in abc*/*

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Unzip the .zip file without using unzip utility in UNIX

I have .zip file, i want to list all the files archived in the zip file. unzip utility is not working for me in unix. Please help me resolve this issue Thanks ganesh. (3 Replies)
Discussion started by: Ganesh L
3 Replies

2. Shell Programming and Scripting

Using Shell Script in place of Perl script to Unzip the zip files.

Hi Expert, We have some shell scripts which Internally uses Perl Script to Unzip the source zip files which comes to inbound directory. So now our requirement is to avoid the dependency on Perl Script and us Shell Script to unzip the files. I have the Perl script with me attached can some one... (3 Replies)
Discussion started by: naveen.dasu
3 Replies

3. Shell Programming and Scripting

How to Unzip a .ZIP file in Unix without using unzip cmd..?????

Hi All I have ftped a .ZIP file (zipped using WinZip in Windows) to my Unix server (HP-UX). I don't have unzip cmd available in my curent Unix version Please let me know any cmd in UNIX (other than unzip) using which I can unzip this .ZIP file . Please elaborate on the commands aval and... (5 Replies)
Discussion started by: sureshg_sampat
5 Replies

4. Shell Programming and Scripting

SFTP / UNZIP script issues

Hi everyone, i'm having a little trouble wih my first shell script ever. So the point of that script is to: -Archive Zip files in a directory -Remove txt files from that directory -connect through sftp and a rsa key to a remote server -download a couple of files -unzip downloaded files ... (0 Replies)
Discussion started by: Peanutz
0 Replies

5. Shell Programming and Scripting

unzip via bash startup script problems

i have two lines in my rc.local file that are wget -O/<path>/<file>.zip url://domain.com unzip -o /<path>/<file>.zip the wget works fine, but the unzip won't work. when i copy/pase the unzip line to the prompt it works fine. i thought that maybe the unzip was running before the wget... (0 Replies)
Discussion started by: easysnowboards
0 Replies

6. Shell Programming and Scripting

Script to unzip specific files

As part of an audit at work I need to review a lot of Windows workstations. One thing I need to do is review all of the .doc files. My problem is that I guessed a number of these files would be in zip archives. This is the case but they are mixed in with a lot of other file types that I am not... (1 Reply)
Discussion started by: stumpyuk
1 Replies

7. Shell Programming and Scripting

Shell script for unzip

All, I'm having 4 .zip files that are coming from FTP. I need to unzip those files and put that files into another folder. Can anyone help me how to write a shell script to check wether .zip files are located in FTP folder, if condition true then how to unzip and put it in another folder. Thanks in... (2 Replies)
Discussion started by: nvkuriseti
2 Replies

8. HP-UX

How to Unzip a .ZIP file in Unix without using unzip cmd..?????

Hi All I have ftped a .ZIP file (zipped using WinZip in Windows) to my Unix server (HP-UX). I don't have unzip cmd available in my curent Unix version Please let me know any cmd in UNIX (other than unzip) using which I can unzip this .ZIP file . Please elaborate on the commands aval and... (5 Replies)
Discussion started by: sureshg_sampat
5 Replies

9. Shell Programming and Scripting

Problems writing bash script to unzip files

I'm getting the following errors when I try to write a script to unzip some zip files. When I use the free trial copy of the commerical winzip program, however, they work fine. When I use -l or -t on unzip it indicates no errors. When I use the -o switch interactively from the bash command line it... (1 Reply)
Discussion started by: siegfried
1 Replies

10. Shell Programming and Scripting

unzip in script using rsh

Hi I have searched usr/bin and usr/sbin to try to find unzip but I cannot find it. My script is using the ksh #!/usr/bin/ksh but when I rsh to another box it cannot find unzip to unzip to a new archive, but I can unzip files when I am using a telnet session on the box. The error message... (4 Replies)
Discussion started by: speedieB
4 Replies
Login or Register to Ask a Question