How to remove files with different release number?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to remove files with different release number?
# 8  
Old 04-07-2013
Quote:
Originally Posted by Don Cragun
OK. That helps, but one more point is crucial. Will any part of the versionNumber or releaseNumber ever be more than one digit (such as 2.1.10, 1.11, or 3.0.12.5)?
Yes, it will.

Thanks Don.
# 9  
Old 04-07-2013
What about a set of meaningful samples so we don't have to dream them up ourselves?
# 10  
Old 04-07-2013
Quote:
Originally Posted by RudiC
What about a set of meaningful samples so we don't have to dream them up ourselves?
Thanks RudiC, please see following files, Files with the highest release number such as graphicData1-1.2.3-2.0.2.mel.au.data and graphicData2-1.0-1.2.mel.au.data should be kept. And files of graphicData1-1.2.3-2.0.0.mel.au.data, graphicData1-1.2.3-2.0.1.mel.au.data, graphicData2-1.0-1.0.mel.au.data and graphicData2-1.0-1.1.mel.au.data should be removed.

graphicData1-1.2.3-2.0.0.mel.au.data
graphicData1-1.2.3-2.0.1.mel.au.data
graphicData1-1.2.3-2.0.2.mel.au.data

graphicData2-1.0-1.0.mel.au.data
graphicData2-1.0-1.1.mel.au.data
graphicData2-1.0-1.2.mel.au.data

Thanks.
# 11  
Old 04-07-2013
You did not provide
- four field release numbers
- two digit release fields
Try this concoction of commands 'n pipes, for a set of sample files that I dreamed up it worked as desired:
Code:
$ awk -F- '{X=$3; sub (/(\.[A-z]*)*$/, "", X); gsub (/\./," ",X);print $0, $1, $2, X}' file |sort  -k2,3 -k4,4nr -k5,5nr -k6,6nr | sort -u -t- -k1,2 | cut -d" " -f1
alphaData1-1.2.3-2.0.2.mel.au.data
graphicData1-1.2.3-2.0.2.mel.au.data
graphicData1-1.2.5-2.0.2.22.mel.au.data
graphicData2-1.0-1.2.mel.au.data
numericData1-1.2.3-2.20.2.mel.au.data

Use it as a starting point for further refinement from your side if it does not fit your needs. Or, provide a really really representative set of samples!
# 12  
Old 04-07-2013
Quote:
Originally Posted by RudiC
You did not provide
- four field release numbers
- two digit release fields
Try this concoction of commands 'n pipes, for a set of sample files that I dreamed up it worked as desired:
Code:
$ awk -F- '{X=$3; sub (/(\.[A-z]*)*$/, "", X); gsub (/\./," ",X);print $0, $1, $2, X}' file |sort  -k2,3 -k4,4nr -k5,5nr -k6,6nr | sort -u -t- -k1,2 | cut -d" " -f1
alphaData1-1.2.3-2.0.2.mel.au.data
graphicData1-1.2.3-2.0.2.mel.au.data
graphicData1-1.2.5-2.0.2.22.mel.au.data
graphicData2-1.0-1.2.mel.au.data
numericData1-1.2.3-2.20.2.mel.au.data

Use it as a starting point for further refinement from your side if it does not fit your needs. Or, provide a really really representative set of samples!
That is very close. Actually your first statement can also display all the files I need to keep:
$ ls
graphicData1-1.2.3-2.0.0.mel.au.data graphicData2-1.0-1.0.mel.au.data
graphicData1-1.2.3-2.0.1.mel.au.data graphicData2-1.0-1.1.mel.au.data
graphicData1-1.2.3-2.0.2.mel.au.data graphicData2-1.0-1.2.mel.au.data
graphicData1-1.2.3-2.0.3.mel.au.data


$ ls | sort -r | sort -u -t- -k2,2
graphicData2-1.0-1.2.mel.au.data
graphicData1-1.2.3-2.0.3.mel.au.data

How can I display all the files I need to delete instead I need to keep? Then I might be able to use following command if the following command syntax can be fixed:

$ find . -type f | sort -r | sort -u -t- -k2,2 -exec rm -f {} \;

Thanks RudiC.
# 13  
Old 04-07-2013
a) assuming your shell is bash (in your Linux system), use the "extended pattern matching operator" !(pattern-list).
b) use grep -v to eliminate the files found from the total file list.
c) ...
z) use your creativity/fantasy/imagination to dream up other solutions!
# 14  
Old 04-07-2013
Here is a much more verbose awk script that I think does what you want.
Code:
#!/bin/ksh
tmp="rm_tmp1.$$"        # tmp file to hold sorted ouput list of normalized
                        # dataName-versionNumber-releaseNumber-inputLine records
tmp2="rm_tmp2.$$"       # tmp file to hold rm commands to be executed
ls *.data | awk -F- -v tmp="$tmp" -v tmp2="$tmp2" '
BEGIN { # Create the sort command to be used to sort in reverse order with
        # keys: dataName, versionNumber, and releaseNumber
        cmd = "sort -t- -r -k1,3 > " tmp
}
{       # Split the versionNumber into its components:
        vc = split($2, v, /[.]/)
        # Split the releaseNumber into its components:
        rc = split($3, r, /[.]/)
        # Save the input filename
        il[NR] = $0
        # Create the normalized sort keys:
        #       dataName-versionNumber-releaseNumber-inputLine
        # where each component of versionNumber and releaseNumber has been
        # expnded to 5 decimal digits with leading zero fill, and feed them
        # into the sort command to be sorted into reverse order.
        o = sprintf("%s-%05d", $1, v[1])
        for(i = 2; i <= vc; i++)
                o = sprintf("%s.%05d", o, v[i])
        o = sprintf("%s-%05d", o, r[1])
        for(i = 2; i <= rc - 3; i++) # -3 skips ".cityCode.countryCode.data"
                o = sprintf("%s.%05d", o, r[i])
        printf("%s-%d\n", o, NR) | cmd
}
END {   # Close output to the sort command:
        close(cmd)
        # Read the sorted list of normalized filenames:
        while((rc = (getline < tmp)) == 1) {
                if($1"-"$2 != last) {
                        # This is a new dataName-versionNumber combination.
                        if(nnl) {
                                # Add a newline to the command file.
                                printf("\n") > tmp2
                                nnl = 0
                        }
                        # Save the new dataName-versionNumber and report the
                        # combination we are now processing
                        last = $1"-"$2
                        printf("Processing %s:\nkeeping \"%s\"\n", last, il[$4])
                } else {# We have seen this dataName-versionNumber before
                        # Report that the corresponding file will be removed
                        printf("discarding \"%s\"\n", il[$4])
                        if(nnl) # Add the corresponding file to the list to be
                                # removed for this dataName-versionNumber pair.
                                printf(" \"%s\"", il[$4]) > tmp2
                        else {  # This is the first file that needs to be
                                # removed for this dataName-versionNumber pair
                                printf("echo rm \"%s\"", il[$4]) > tmp2
                                # Note that we will need to add a newline after
                                # we have added the last entry for this pair
                                nnl = 1
                        }
                }
        }
        # Add final newline, if needed
        if(nnl) printf("\n") > tmp2
        # If we hit EOF on sorted temp file, we are done
        if(rc == 0) exit 0
        # Otherwise, print a diagnostic and exit with a non-zero exit status
        printf("I/O error while reading sorted list from %s\n", tmp)
        exit 1
}'
if [ $? -eq 0 ]
then    # awk completed successfully, run the commands to remove old
        # releaseNumber files
        if [ ! -e $tmp2 ]
        then    # No files found to be removed.
                printf "No unneeded versionNumber files found.\n"
                exit 0
        fi
        if sh < $tmp2
        then    # Successful completion, remove temp files and exit
                rm $tmp $tmp2
                printf "Discarded files soccessfully, temp files removed.\n"
                exit 0
        fi
fi
# Otherwise, print diagnostic and quit (saving temp files in place
printf "Some files not removed; temp files:\n\t%s\n\t%s\nsaved.\n" $tmp $tmp2 >&2
exit 1

As always, if you're using a Solaris/SunOS system, use /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk instead of awk.

This script was tested using the Korn shell and bash; it should work with any POSIX conforming shell. It will work with one or more components in the versionNumber and releaseNumber fields (as long as all of the components are entirely numeric values with each component consisting of no more than 5 digits. It will not work if any dataName contains a hyphen character (-).

If you don't want the play by play report of what will be kept and what will be removed, delete the lines in blue above. Once you have convinced yourself that it will remove the files you want removed, delete the echo in red above.

Enjoy...
These 2 Users Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Remove string between number and character

hello ! I have to remove string between a number and set of characters. For example, 35818 -stress - - -stress - - - - - - DB-3754 44412 caul kid notify DB-3747 54432 roberto -, notify DB-3725 55522 aws _ _int _ _classified 2_a _a 2_m _m 2_classified 2_search... (7 Replies)
Discussion started by: ManoharMa
7 Replies

2. Shell Programming and Scripting

List files with number to select based on number

Hi experts, I am using KSH and I am need to display file with number in front of file names and user can select it by entering the number. I am trying to use following command to display list with numbers. but I do not know how to capture number and identify what file it is to be used for... (5 Replies)
Discussion started by: mysocks
5 Replies

3. Solaris

Remove number in file name

Hi , i have a file name with date and time append like test_SEC_AES_V1_T03_2016031404306 i want to remove the date and time after _ and rename to current date and time,can any one please let me know as i tried some options but din't help:( Thanks, Please use code tags as required... (10 Replies)
Discussion started by: caba_jones
10 Replies

4. Shell Programming and Scripting

Remove trailing number

I have some strings such as ABC1 ABC2 TYFASDD12 They will only have letters and numbers. In each case I want to remove the last digit? The lengths will vary. So a hard coded substr won't work. What do I do? if it doesn't end in a number, I don't want to remove any characters. (6 Replies)
Discussion started by: guessingo
6 Replies

5. UNIX for Dummies Questions & Answers

Remove char if not a number

I need to write a BASH script that takes a 2 character string and removes the second character if it is not a digit e.g. If the string is numberical value >9 e.g. string1 = '34' then leave string1 = '34'. However if the string is <10 e.g. string1 = '3X' then remove the second char (which... (7 Replies)
Discussion started by: millsy5
7 Replies

6. Shell Programming and Scripting

Remove last number from a file

Hi, I need to delete last number ( i.e 126) from the file which has below contents. ABC DEF "XYZ" 2837.5 3852.5 126 ABC DEF "XYZ" 2897.5 3852.5 126 ABC DEF "XYZ" 2957.5 3852.5 126 Please help. Thanks in advance. (17 Replies)
Discussion started by: gujrathinr
17 Replies

7. Shell Programming and Scripting

How to count number of files in directory and write to new file with number of files and their name?

Hi! I just want to count number of files in a directory, and write to new text file, with number of files and their name output should look like this,, assume that below one is a new file created by script Number of files in directory = 25 1. a.txt 2. abc.txt 3. asd.dat... (20 Replies)
Discussion started by: Akshay Hegde
20 Replies

8. Shell Programming and Scripting

script for remove descending order number

hi all i want to remove some descending order number example : 1 100 200 135.00 Gk_wirs 1 1 100 200 136.00 Gk_wirs 50 1 110 210 138.00 Gk_wirs 60 1 100 200 136.00 Gk_wirs 57 ----> how to remove... (6 Replies)
Discussion started by: nithyanandan
6 Replies

9. Shell Programming and Scripting

remove the number from the String

I have string like 20091112_File_Name_40301.txt and i have a set of files in the directory with the same format . i want to write the ksh to rename the file ..... like eg 20091112_File_Name_40301.txt to File_Name.txt 20091112_abc_2343.txt to abc.txt... (6 Replies)
Discussion started by: gavemani
6 Replies

10. AIX

Remove APAR number from aix 5.3

Hi... Can i remove one APAR number from aix5.3.. If it is possible how to do.. Thanks.. (3 Replies)
Discussion started by: sumathi.k
3 Replies
Login or Register to Ask a Question