File name manipulation when extracting from gzip


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File name manipulation when extracting from gzip
# 1  
Old 04-08-2015
File name manipulation when extracting from gzip

Hi,

Below is the description of my problem.

I am trying to loop through the below file names sitting in the file - FileNames.txt, and trying to decompress them

FileNames.txt
Code:
20150408_MGAC_[0-9][0-9][0-9][0-9].txt.gz
20150408_MGCC_[0-9][0-9][0-9][0-9].txt.gz
20150408_MGSH_[0-9][0-9][0-9][0-9]_[0-9][0-9][0-9][0-9].txt.gz
20150408_MGSL_[0-9][0-9][0-9][0-9]_[0-9][0-9][0-9][0-9].txt.gz

filename="20150408_MGAC_[0-9][0-9][0-9][0-9].txt.gz"

But when I use the below command -
Code:
gzip -c  ${filename} > ${filename%.*}

I get the output file name like this -
Code:
20150408_MGAC_[0-9][0-9][0-9][0-9].txt

How can I format the output file name to something like the below?
Code:
20150408_MGAC_1111.txt
20150408_MGCC_1111.txt
20150408_MGSH_1111_1111.txt
20150408_MGSL_1111_1111.txt

I have tried multiple options while extracting the files but I am not able to get the desired result.

Please help.

Last edited by Scrutinizer; 04-08-2015 at 01:52 PM..
# 2  
Old 04-08-2015
Something like this perhaps:

Code:
while read FILENAME
do
        for FILE in $FILENAME
        do
                echo gzip -c "${FILE} ">" "${FILE%.*}"
        done
done < filenames.txt

The glob expands in $FILENAME, and FILE becomes the actual name for each matching file in turn.

Change echo gzip -c "${FILE} ">" "${FILE%.*}" into gzip -c "${FILE} > "${FILE%.*}" once you've tested it and are sure it does what you want.
# 3  
Old 04-08-2015
Hi Corona688,

Thanks for your quick response.

I was able to loop through the files, and I was also able to extract only the ".txt" section using the string manipulation ${FILE%.*}.

But after I rename, I need the files
Code:
20150408_MGAC_[0-9][0-9][0-9][0-9].txt
20150408_MGCC_[0-9][0-9][0-9][0-9].txt
20150408_MGSH_[0-9][0-9][0-9][0-9]_[0-9][0-9][0-9][0-9].txt
20150408_MGSL_[0-9][0-9][0-9][0-9]_[0-9][0-9][0-9][0-9].txt

to be renamed as
Code:
20150408_MGAC_1111.txt
20150408_MGCC_1111.txt
20150408_MGSH_1111_1111.txt
20150408_MGSL_1111_1111.txt

While decompressing, I also want to rename all occurrences of [0-9] to 1.

How could I achieve that?

Thanks

Last edited by Scrutinizer; 04-08-2015 at 01:51 PM.. Reason: code tags
# 4  
Old 04-08-2015
That sounds likely to overwrite many of your files! Did you mean to append perhaps?

Does your filenames.txt literally contain 20150408_MGAC_[0-9][0-9][0-9][0-9].txt, or does it just contain numbers?
This User Gave Thanks to Corona688 For This Post:
# 5  
Old 04-08-2015
It literally contains the characters 20150408_MGAC_[0-9][0-9][0-9][0-9].txt.

A little description about the root of the problem - There is a process that extracts the file names present in an SFTP server and updates a text file. I then have to search the files, with the above pattern (where the the data in the regex [0-9] can be any random number between 0 to 9), and make sure the file exists in the extracted list of files.

Once I confirm the file is present in the server, I loop through the list of files that I was expecting, which is nothing but the files with the wildcards (20150408_MGAC_[0-9][0-9][0-9][0-9].txt), I scp the file to my local, do a "gzip -c" to a new filename on the .txt format, and then I wish to rename the files, which I am unable to do..

I am sorry if I am confusing you, I have can give you an example if you want more information.

Thank you so much for offering to help!
# 6  
Old 04-08-2015
So there is never more than one MGAC, or more than one MGCC, etc?

Code:
while read FILENAME
do
        for FILE in $FILENAME
        do
                NEWFILE="${FILE%.*}"
                echo gzip -c "${FILE} ">" "${NEWFILE//\[0-9]/1}"
        done
done < filenames.txt

This User Gave Thanks to Corona688 For This Post:
# 7  
Old 04-08-2015
Did you try
Code:
gzip -k ${filename}

?
Let gzip do the output file(s) and -keep the original(s).
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

MIME type for sending gzip file as attachment in email

Hello, I am trying to send a gzip file on email using below command but the zipped file received on email is corrupt. mailsend -smtp $smtpip -content-type 'application/x-gzip' -mime-type "application/x-gzip" -t $receiver -f $sender -sub "$subject" -M "$MSG" -attach $file file name is ... (1 Reply)
Discussion started by: tushar.modgil
1 Replies

2. Shell Programming and Scripting

Extracting logs using gunzip awk and gzip

Hi All I am trying to use a hard coded script into shell scripting but I am unable to . Kindly find the Script below along with the command Please help gunzip -c FilePath/FileName_*.gz | awk '$0 > "" && $0 < ""'|\ gzip >> FilePath/Outputfile.log.gz I Am trying to use this... (9 Replies)
Discussion started by: pulkitbahl
9 Replies

3. Shell Programming and Scripting

Shell script not able to zip the file using gzip

Hi all, I am calling Temp.sh and it is has simple line $gpath=`which gzip` $gpath $FilePath/My_temp.log if I run this script, logging to server then its works fine. But when I send this script over the SSH it does not work at it. gzip is command is not execute. I am using gzip 1.6... (2 Replies)
Discussion started by: girijajoshi
2 Replies

4. Shell Programming and Scripting

Check if file is locked otherwise gzip it

Dear community, I've a continuos tcpdump on redhat that close the dumped file every 100000 captured packets. To avoid disk full I would like to gzip the closed *.cap file. But how can I check if the file is currently opened by tcpdump and skip it from gzip? Thanks! EDIT: Just to post an... (9 Replies)
Discussion started by: Lord Spectre
9 Replies

5. UNIX for Dummies Questions & Answers

gzip of a file in the same folder

I want gzip of a file in the same folder where it is kept now $filename = '/var/dev/test.txt' /opt/home/>> gzip -c $filename > test.txt.gz however command creates it in the folder in /opt/home/ How to gzip a file in the same directory where it is now , no matter from where we execute and also... (2 Replies)
Discussion started by: lalitpct
2 Replies

6. UNIX for Advanced & Expert Users

gzip vs pipe gzip: produce different file size

Hi All, I have a random test file: test.txt, size: 146 $ ll test.txt $ 146 test.txt Take 1: $ cat test.txt | gzip > test.txt.gz $ ll test.txt.gz $ 124 test.txt.gz Take 2: $ gzip test.txt $ ll test.txt.gz $ 133 test.txt.gz As you can see, gzipping a file and piping into gzip... (1 Reply)
Discussion started by: hanfresco
1 Replies

7. Shell Programming and Scripting

Appending a GZIP File

Guys, I just want to know the difference in following (core difference) zcat a.gz b.gz c.gz |gzip >d.gz And zcat a.gz >>d.gz zcat b.gz >>d.gz zcat c.gz >>d.gz do we have 3 gzip header in 1st and only one in second case. please let me know this in detail Thanks,... (2 Replies)
Discussion started by: mohan_xunil
2 Replies

8. Solaris

gzip a file and append creation date stamp to file

I want to gzip a file and append the creation date to the end of the file. How can I accomplish this task. Basically they are log files which need a creation date stamp appended to make sure they do not overwrite other log files. -jack (3 Replies)
Discussion started by: jacktravine
3 Replies

9. Solaris

script for Gzip thousands of file

Hi experts, I have thousands of file (data file and Gziped file) in same directory like below-- bash-2.05$ pwd /home/mmc bash-2.05$ file PP023149200709270546 TT023149200709270546: gzip compressed data - deflate method bash-2.05$ file PP027443200711242320 TT027443200711242320: ... (10 Replies)
Discussion started by: thepurple
10 Replies

10. UNIX for Dummies Questions & Answers

How do I send a file as an attachment (gzip file) on a Unix system

Hi, How do I send a file as an attachment (gzip file) on a Unix system ? Using sendmail. Please help me. :confused: (3 Replies)
Discussion started by: lacca
3 Replies
Login or Register to Ask a Question

Featured Tech Videos