07-06-2017
Many thanks for all the replies. I'm likely going with the solution from Mannu2525 which worked fine on my Solaris server once I changed awk to nawk.
I'll try to find time to try the others & let you all know how it goes.
I consider myself a reasonable shell scripter but these awk constructs are well beyond my abilities. Thanks again.
Chris
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have a input file with formating:
6000000901 ;36200103 ;h3a01f496 ;
2000123605 ;36218982 ;heefa1328 ;
2000273132 ;36246985 ;h08c5cb71 ;
2000041207 ;36246985 ;heef75497 ;
Each fields is seperated by semi-comma. Sometime, the second files is... (6 Replies)
Discussion started by: anhtt
6 Replies
2. Shell Programming and Scripting
I have searched the internet for duplicate row extracting.
All I have seen is extracting good rows or eliminating duplicate rows.
How do I extract duplicate rows from a flat file in unix.
I'm using Korn shell on HP Unix.
For.eg.
FlatFile.txt
========
123:456:678
123:456:678
123:456:876... (5 Replies)
Discussion started by: bobbygsk
5 Replies
3. Shell Programming and Scripting
I want to duplicate a row if found two or more values in a particular column for corresponding row which is delimitted by comma.
Input
abc,line one,value1
abc,line two, value1, value2
abc,line three,value1
needs to converted to
abc,line one,value1
abc,line two, value1
abc,line... (8 Replies)
Discussion started by: Incrediblian
8 Replies
4. UNIX for Dummies Questions & Answers
if the key (A or B or ...others) has 4 in its 3rd column the 1st A row has to form 4 dupicates along with the all the values of A in 4th column (2.9, 3.8, 4.2) .
Hope I explain the question clearly.
Cheers
Ruby
input
"A" 1 4 2.9
"A" 2 5 ... (7 Replies)
Discussion started by: ruby_sgp
7 Replies
5. Shell Programming and Scripting
Hi! I have a file as below:
line1
line2
line2
line3
line3
line3
line4
line4
line4
line4
I would like to extract duplicate lines (not unique, triplicate or quadruplicate lines). Output will be as below:
line2
line2
I would appreciate if anyone can help. Thanks. (4 Replies)
Discussion started by: chromatin
4 Replies
6. Shell Programming and Scripting
I am new to this forum and this is my first post.
I am looking at an old post with exactly the same name. Can not paste URL because I do not have 5 posts
My requirement is exactly opposite.
I want to get rid of duplicate rows and try to append the values of columns in those rows
... (10 Replies)
Discussion started by: vbhonde11
10 Replies
7. Shell Programming and Scripting
Hi All,
I need to extract duplicate rows from a file and write these bad records into another file. And need to have a count of these bad records.
i have a command
awk '
{s++}
END {
for(i in s) {
if(s>1) {
print i
}
}
}' ${TMP_DUPE_RECS}>>${TMP_BAD_DATA_DUPE_RECS}... (5 Replies)
Discussion started by: Arun Mishra
5 Replies
8. Shell Programming and Scripting
Dear members, I need to filter a file based on the 8th column (that is id), and does not mather the other columns, because I want just one id (1 line of each id) and remove the duplicates lines based on this id (8th column), and does not matter wich duplicate will be removed.
example of my file... (3 Replies)
Discussion started by: clarissab
3 Replies
9. Shell Programming and Scripting
I have this file input.txt. I want to take average column-wise for the rows having duplicate gene names.
Gene Sample_1 Sample_2 Sample_3
gene_A 2 4 5
gene_B 1 2 3
gene_A 0 5 7
gene_B 4 5 6
gene_A 11 12 13
gene_C 2 3 4
Desired output:
gene_A 4.3 7 8.3
gene_B 2.5 3.5 4.5
gene_C 2 3 4... (6 Replies)
Discussion started by: Sanchari
6 Replies
10. Shell Programming and Scripting
Gents
Can you help please.
Input file
5490921425 1 7 1310342 54909214251
5490921425 2 1 1 54909214252
5491120937 1 1 3 54911209371
5491120937 3 1 1 54911209373
5491320785 1 ... (4 Replies)
Discussion started by: jiam912
4 Replies
LEARN ABOUT MOJAVE
ucblinks
ucblinks(1B) SunOS/BSD Compatibility Package Commands ucblinks(1B)
NAME
ucblinks - adds /dev entries to give SunOS 4.x compatible names to SunOS 5.x devices
SYNOPSIS
/usr/ucb/ucblinks [-e rulebase] [-r rootdir]
DESCRIPTION
ucblinks creates symbolic links under the /dev directory for devices whose SunOS 5.x names differ from their SunOS 4.x names. Where possi-
ble, these symbolic links point to the device's SunOS 5.x name rather than to the actual /devices entry.
ucblinks does not remove unneeded compatibility links; these must be removed by hand.
ucblinks should be called each time the system is reconfiguration-booted, after any new SunOS 5.x links that are needed have been created,
since the reconfiguration may have resulted in more compatibility names being needed.
In releases prior to SunOS 5.4, ucblinks used a nawk rule-base to construct the SunOS 4.x compatible names. ucblinks no longer uses nawk
for the default operation, although nawk rule-bases can still be specifed with the -e option. The nawk rule-base equivalent to the SunOS
5.4 default operation can be found in /usr/ucblib/ucblinks.awk.
OPTIONS
-e rulebase Specify rulebase as the file containing nawk(1) pattern-action statements.
-r rootdir Specify rootdir as the directory under which dev and devices will be found, rather than the standard root directory /.
FILES
/usr/ucblib/ucblinks.awk sample rule-base for compatibility links
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+-----------------------------+-----------------------------+
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+-----------------------------+-----------------------------+
|Availability |SUNWscpu |
+-----------------------------+-----------------------------+
SEE ALSO
devlinks(1M), disks(1M), ports(1M), tapes(1M), attributes(5)
SunOS 5.10 13 Apr 1994 ucblinks(1B)