Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Using awk to append incremental numbers to the end of duplicate file names.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using awk to append incremental numbers to the end of duplicate file names.
# 1  
Old 06-20-2012
Using awk to append incremental numbers to the end of duplicate file names.
I'm currently working on a script that extracts files from a .zip, runs an sha1sum against them and then uses awk to pre-format them into zomething more readable thusly:

Code:
Z 69 89e013b0d8aa2f9a79fcec4f2d71c6a469222c07 File1
Z 69 6c3aea28ce22b495e68e022a1578204a9de908ed File2
Z 69 54122c1890d94ff76578076018d00a7c16e86e6d File4
Z 69 e66b24788b24a1733778d983d8163bb02a5c7201 File5
Z 69 c738aae20b50658ada2e0d5144302b9b695f98f3 File6
Z 69 40a63fe5078f5481737d4c0a7cbdca775cf9af28 File4
Z 69 4754cfd9f38d8a0beba4c8ebbf83569736c7ebfc File4
Z 69 721e29ad87bc1fccb3e0cae987b81a21a941e408 File2
Z 47 ca926c99b0394a8783152c19a7ffc6d686dcbcb3 File1
Z 47 1d66b78cc8b67e848aba4eecc3cb0f32e81b4004 File2
Z 47 114e0653503340c7325b345642fb61afddf4f339 File3
Z 47 4754cfd9f38d8a0beba4c8ebbf83569736c7ebfc File4
Z 47 86a09ee704fde229b0b3b25fe6190179e334364c File3
Z 47 7ba8ae667e48f53ccffb52a308340726e446712a File5


The problem I am having is that since this file operates on zips sometimes it will result in files with duplicate names appearing in a specific subset of files. For example subset 69 has File4 appear three times and File2 twice, while subset 47 has File3 appear twice. what I am trying to accomplish is to make awk check for duplicate occurances of $4 within a $2 subset and then use '{ count=1, count++ }' to append incrementally increasing numbers to the end of those files, thus making them uniqe, Any help or hints on how I might accomplish this would be appreciated since I'm pulling my hair out over this.
# 2  
Old 06-20-2012
Can you post desired output for this sample data?
# 3  
Old 06-20-2012
The code below is pre-formatted with awk I am jsut struggling with the rename part, but ideally $count would reset to 1 every time a new value is picked up in $2 thusly:
Code:
Z 69 89e013b0d8aa2f9a79fcec4f2d71c6a469222c07 File1
Z 69 6c3aea28ce22b495e68e022a1578204a9de908ed File2_1
Z 69 54122c1890d94ff76578076018d00a7c16e86e6d File4_1
Z 69 e66b24788b24a1733778d983d8163bb02a5c7201 File5
Z 69 c738aae20b50658ada2e0d5144302b9b695f98f3 File6
Z 69 40a63fe5078f5481737d4c0a7cbdca775cf9af28 File4_2
Z 69 4754cfd9f38d8a0beba4c8ebbf83569736c7ebfc File4_3
Z 69 721e29ad87bc1fccb3e0cae987b81a21a941e408 File2_2
Z 47 ca926c99b0394a8783152c19a7ffc6d686dcbcb3 File1
Z 47 1d66b78cc8b67e848aba4eecc3cb0f32e81b4004 File2
Z 47 114e0653503340c7325b345642fb61afddf4f339 File3_1
Z 47 4754cfd9f38d8a0beba4c8ebbf83569736c7ebfc File4
Z 47 86a09ee704fde229b0b3b25fe6190179e334364c File3_2
Z 47 7ba8ae667e48f53ccffb52a308340726e446712a File5

Failing that count could just increment indefinately so you end up with File3_55 File3_56 ect...
Last edited by Ethereal; 06-20-2012 at 10:08 AM.. Reason: Typos
# 4  
Old 06-20-2012
Try:
Code:
awk '{$4=$4"_"++a[$4]}1' file

# 5  
Old 06-20-2012
Something like this?

Code:
awk '{if(++a[$2,$4]>1)$4=$4"_"a[$2,$4]}1' inputfile

Last edited by elixir_sinari; 06-20-2012 at 10:25 AM..
This User Gave Thanks to elixir_sinari For This Post:
Ethereal
# 6  
Old 06-20-2012
Thanks for the quick response guys elixirs is a closer to what I need, but about 3x as complex Smilie. As near as I can tell though you basically did what I wasted about half a day trying to make work by putting the variables into an associative array, but you seem to have used a multi-dimensional array which I'm not overly familiar with. To me it looks like you're creating a numbered array, incrementing it with ++, adding the values of $2 and $4 into it, then stating that if that array is > 1 let $4 = $4 appended with _ and the number value of the array, is that right?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with awk script to append seq num at end of record

Hi Unix forum. I have the following requirement to add a sequence value to each record in a file but only if it meets certain conditions. Field value in pos. 1 and 2 must be '0B' or 'OA' else leave as is. Sequence value must be preserved for each OB and OA pair. Data Before: 123 456... (5 Replies)
Discussion started by: pchang
5 Replies

2. Shell Programming and Scripting

Print numbers along with file names.

Hi All, I have some thousand files with names like 1.syl, 2.syl, 5.syl etc. These files contain one sentence each. I want to store all those sentences along with the file ID that is 1, 2, 5 with the sentences they contain. For example, 1.syl has this is a test line 2.syl has ... (3 Replies)
Discussion started by: shoaibjameel123
3 Replies

3. Shell Programming and Scripting

Getting file names which does not end with .log

Hi All, I have a directory in which all log files will be generated. Apart from the log files, there are a few other files also generated. I want the names of all files which does not end with .LOG. consider the dir has the following files I want only ebaf02012009.ERR file. Unix... (4 Replies)
Discussion started by: suresh_kb211
4 Replies

4. UNIX for Advanced & Expert Users

How to select only those file names whose name contains only numbers

Hi Guru's, Before writing to this forum I have searched extensively on this forum about my problem. I have to write a shell script which takes out only those file names from the given directory which contains only numbers. For example, In the given directory these files are present: ... (4 Replies)
Discussion started by: spranm
4 Replies

5. UNIX for Dummies Questions & Answers

How to select only those file names whose name contains only numbers.

Hi Guru's, Before writing to this forum I have searched extensively on this forum about my problem. I have to write a shell script which takes out only those file names from the given directory which contains only numbers. For example, In the given directory these files are present: ... (0 Replies)
Discussion started by: spranm
0 Replies

6. Shell Programming and Scripting

append a record at the end of a file

Hi all, i have to append a record at the end of the file(a file which is already with some records).how do i do?please help me? is there any way of doing this with "SED" command.i am not sure.plz help me on this. would appreciate your ideas!!!! bye rao. (3 Replies)
Discussion started by: raoscb
3 Replies

7. Shell Programming and Scripting

Append newline at the file end

Hi All, Is there any way to append a newline character at the end of a file(coma-separated file), through shell script? I need to check whether newline character exists at the end of a file, if it does not then append it. Regards, Krishna (1 Reply)
Discussion started by: KrishnaSaran
1 Replies

8. Shell Programming and Scripting

Bash Script duplicate file names

I am trying to write a housekeeping bash script. Part of it involves searching all of my attached storage media for photographs and moving them into a single directory. The problem occurs when files have duplicate names, obviously a file called 001.jpg will get overwritten with another file... (6 Replies)
Discussion started by: stumpyuk
6 Replies

9. Shell Programming and Scripting

to append few chars at the end of a file

hi i want to open a file at runtime append few chars at the end of each line all these i want to have done automatically how to do it (2 Replies)
Discussion started by: trichyselva
2 Replies

10. Shell Programming and Scripting

echo, append to end of file

I need the line printed with echo to append to eof of to exactly line, am i able to do that? i mean echo "sysctl -w lalala=1" > to end of file /etc/sysctl.conf or to the 21st line, if the line exist, open new line and insert text there. Thx.maybe i'm in wrong topic but anyway... (2 Replies)
Discussion started by: hachik
2 Replies
Login or Register to Ask a Question