Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

File name manipulation when extracting from gzip

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File name manipulation when extracting from gzip
# 1  
Old 04-08-2015
File name manipulation when extracting from gzip
Hi,

Below is the description of my problem.

I am trying to loop through the below file names sitting in the file - FileNames.txt, and trying to decompress them

FileNames.txt
Code:
20150408_MGAC_[0-9][0-9][0-9][0-9].txt.gz
20150408_MGCC_[0-9][0-9][0-9][0-9].txt.gz
20150408_MGSH_[0-9][0-9][0-9][0-9]_[0-9][0-9][0-9][0-9].txt.gz
20150408_MGSL_[0-9][0-9][0-9][0-9]_[0-9][0-9][0-9][0-9].txt.gz

filename="20150408_MGAC_[0-9][0-9][0-9][0-9].txt.gz"

But when I use the below command -
Code:
gzip -c  ${filename} > ${filename%.*}

I get the output file name like this -
Code:
20150408_MGAC_[0-9][0-9][0-9][0-9].txt

How can I format the output file name to something like the below?
Code:
20150408_MGAC_1111.txt
20150408_MGCC_1111.txt
20150408_MGSH_1111_1111.txt
20150408_MGSL_1111_1111.txt

I have tried multiple options while extracting the files but I am not able to get the desired result.

Please help.
Last edited by Scrutinizer; 04-08-2015 at 12:52 PM..
# 2  
Old 04-08-2015
Something like this perhaps:

Code:
while read FILENAME
do
        for FILE in $FILENAME
        do
                echo gzip -c "${FILE} ">" "${FILE%.*}"
        done
done < filenames.txt

The glob expands in $FILENAME, and FILE becomes the actual name for each matching file in turn.

Change echo gzip -c "${FILE} ">" "${FILE%.*}" into gzip -c "${FILE} > "${FILE%.*}" once you've tested it and are sure it does what you want.
# 3  
Old 04-08-2015
Hi Corona688,

Thanks for your quick response.

I was able to loop through the files, and I was also able to extract only the ".txt" section using the string manipulation ${FILE%.*}.

But after I rename, I need the files
Code:
20150408_MGAC_[0-9][0-9][0-9][0-9].txt
20150408_MGCC_[0-9][0-9][0-9][0-9].txt
20150408_MGSH_[0-9][0-9][0-9][0-9]_[0-9][0-9][0-9][0-9].txt
20150408_MGSL_[0-9][0-9][0-9][0-9]_[0-9][0-9][0-9][0-9].txt

to be renamed as
Code:
20150408_MGAC_1111.txt
20150408_MGCC_1111.txt
20150408_MGSH_1111_1111.txt
20150408_MGSL_1111_1111.txt

While decompressing, I also want to rename all occurrences of [0-9] to 1.

How could I achieve that?

Thanks
Last edited by Scrutinizer; 04-08-2015 at 12:51 PM.. Reason: code tags
# 4  
Old 04-08-2015
That sounds likely to overwrite many of your files! Did you mean to append perhaps?

Does your filenames.txt literally contain 20150408_MGAC_[0-9][0-9][0-9][0-9].txt, or does it just contain numbers?
This User Gave Thanks to Corona688 For This Post:
Richmond_CS
# 5  
Old 04-08-2015
It literally contains the characters 20150408_MGAC_[0-9][0-9][0-9][0-9].txt.

A little description about the root of the problem - There is a process that extracts the file names present in an SFTP server and updates a text file. I then have to search the files, with the above pattern (where the the data in the regex [0-9] can be any random number between 0 to 9), and make sure the file exists in the extracted list of files.

Once I confirm the file is present in the server, I loop through the list of files that I was expecting, which is nothing but the files with the wildcards (20150408_MGAC_[0-9][0-9][0-9][0-9].txt), I scp the file to my local, do a "gzip -c" to a new filename on the .txt format, and then I wish to rename the files, which I am unable to do..

I am sorry if I am confusing you, I have can give you an example if you want more information.

Thank you so much for offering to help!
# 6  
Old 04-08-2015
So there is never more than one MGAC, or more than one MGCC, etc?

Code:
while read FILENAME
do
        for FILE in $FILENAME
        do
                NEWFILE="${FILE%.*}"
                echo gzip -c "${FILE} ">" "${NEWFILE//\[0-9]/1}"
        done
done < filenames.txt

This User Gave Thanks to Corona688 For This Post:
Richmond_CS
# 7  
Old 04-08-2015
Did you try
Code:
gzip -k ${filename}

?
Let gzip do the output file(s) and -keep the original(s).
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

MIME type for sending gzip file as attachment in email

Hello, I am trying to send a gzip file on email using below command but the zipped file received on email is corrupt. mailsend -smtp $smtpip -content-type 'application/x-gzip' -mime-type "application/x-gzip" -t $receiver -f $sender -sub "$subject" -M "$MSG" -attach $file file name is ... (1 Reply)
Discussion started by: tushar.modgil
1 Replies

2. Shell Programming and Scripting

Extracting logs using gunzip awk and gzip

Hi All I am trying to use a hard coded script into shell scripting but I am unable to . Kindly find the Script below along with the command Please help gunzip -c FilePath/FileName_*.gz | awk '$0 > "" && $0 < ""'|\ gzip >> FilePath/Outputfile.log.gz I Am trying to use this... (9 Replies)
Discussion started by: pulkitbahl
9 Replies

3. Shell Programming and Scripting

Shell script not able to zip the file using gzip

Hi all, I am calling Temp.sh and it is has simple line $gpath=`which gzip` $gpath $FilePath/My_temp.log if I run this script, logging to server then its works fine. But when I send this script over the SSH it does not work at it. gzip is command is not execute. I am using gzip 1.6... (2 Replies)
Discussion started by: girijajoshi
2 Replies

4. Shell Programming and Scripting

Check if file is locked otherwise gzip it

Dear community, I've a continuos tcpdump on redhat that close the dumped file every 100000 captured packets. To avoid disk full I would like to gzip the closed *.cap file. But how can I check if the file is currently opened by tcpdump and skip it from gzip? Thanks! EDIT: Just to post an... (9 Replies)
Discussion started by: Lord Spectre
9 Replies

5. UNIX for Dummies Questions & Answers

gzip of a file in the same folder

I want gzip of a file in the same folder where it is kept now $filename = '/var/dev/test.txt' /opt/home/>> gzip -c $filename > test.txt.gz however command creates it in the folder in /opt/home/ How to gzip a file in the same directory where it is now , no matter from where we execute and also... (2 Replies)
Discussion started by: lalitpct
2 Replies

6. UNIX for Advanced & Expert Users

gzip vs pipe gzip: produce different file size

Hi All, I have a random test file: test.txt, size: 146 $ ll test.txt $ 146 test.txt Take 1: $ cat test.txt | gzip > test.txt.gz $ ll test.txt.gz $ 124 test.txt.gz Take 2: $ gzip test.txt $ ll test.txt.gz $ 133 test.txt.gz As you can see, gzipping a file and piping into gzip... (1 Reply)
Discussion started by: hanfresco
1 Replies

7. Shell Programming and Scripting

Appending a GZIP File

Guys, I just want to know the difference in following (core difference) zcat a.gz b.gz c.gz |gzip >d.gz And zcat a.gz >>d.gz zcat b.gz >>d.gz zcat c.gz >>d.gz do we have 3 gzip header in 1st and only one in second case. please let me know this in detail Thanks,... (2 Replies)
Discussion started by: mohan_xunil
2 Replies

8. Solaris

gzip a file and append creation date stamp to file

I want to gzip a file and append the creation date to the end of the file. How can I accomplish this task. Basically they are log files which need a creation date stamp appended to make sure they do not overwrite other log files. -jack (3 Replies)
Discussion started by: jacktravine
3 Replies

9. Solaris

script for Gzip thousands of file

Hi experts, I have thousands of file (data file and Gziped file) in same directory like below-- bash-2.05$ pwd /home/mmc bash-2.05$ file PP023149200709270546 TT023149200709270546: gzip compressed data - deflate method bash-2.05$ file PP027443200711242320 TT027443200711242320: ... (10 Replies)
Discussion started by: thepurple
10 Replies

10. UNIX for Dummies Questions & Answers

How do I send a file as an attachment (gzip file) on a Unix system

Hi, How do I send a file as an attachment (gzip file) on a Unix system ? Using sendmail. Please help me. :confused: (3 Replies)
Discussion started by: lacca
3 Replies
Login or Register to Ask a Question