Make (and test) the combination of $1, $2, and $3 unique:
Code:
awk '
FNR == 1 {next
}
!T[$1,$2,$3]++ {v_array[$1 OFS $2]=v_array[$1 OFS $2] ? v_array[$1 OFS $2] "," $3 : $3
}
END {for (v in v_array) print v, v_array[v]
}
' file
abc def xyz,qrs
abc efg pqr
stu vwx yz
While some awk implementations provide a sort function, and you could build one yourself in others, piping through sort might be the easiest way to get what you want:
Code:
. . . | sort -k3
abc efg pqr
abc def xyz,qrs
stu vwx yz
Hi All,
I have an array that contains duplicates as well unique numbers.
ex- (21, 33, 35, 21, 33, 70, 33, 35, 50)
I need to arrange it in such a way that all the duplicates will come up first followed by unique numbers.
Result for the given example should be:
(21, 21, 33, 33, 35, 35, 70,... (4 Replies)
Hello Experts,
I have two files named old and new. Below are my example files. I need to compare and print the records that only exist in my new file. I tried the below awk script, this script works perfectly well if the records have exact match, the issue I have is my old file has got extra... (4 Replies)
I need to use a bash script to remove duplicate files from a download list, but I cannot use uniq because the urls are different.
I need to go from this:
http://***/fae78fe/file1.wmv
http://***/39du7si/file1.wmv
http://***/d8el2hd/file2.wmv
http://***/h893js3/file2.wmv
to this:
... (2 Replies)
I have the following code for removing duplicate records based on fields in inputfile file & moves the duplicate records in duplicates file(1st Awk) & in 2nd awk i fetch the non duplicate entries in inputfile to tmp file and use move to update the original file.
Requirement:
Can both the awk... (4 Replies)
Hi All,
I have searched many threads for possible close solution. But I was unable to get simlar scenario.
I would like to print all duplicate based on 3rd column except the first occurance. Also would like to print if it is single entry(non-duplicate).
i/P file
12 NIL ABD LON
11 NIL ABC... (6 Replies)
Hi all,
I need some help to remove duplicates from a file before merging.
I have got 2 files:
file1 has data in format
4300 23456
4301 2357
the 4 byte values on the right hand side is uniq, and are not repeated anywhere in the file
file 2 has data in same format but is not in... (10 Replies)
I have a file with the following format:
fields seperated by "|"
title1|something class|long...content1|keys
title2|somhing class|log...content1|kes
title1|sothing class|lon...content1|kes
title3|shing cls|log...content1|ks
I want to remove all duplicates with the same "title field"(the... (3 Replies)
Hi I have a below file structure.
200,1245,E1,1,E1,,7611068,KWH,30, ,,,,,,,,
200,1245,E1,1,E1,,7611070,KWH,30, ,,,,,,,,
300,20140223,0.001,0.001,0.001,0.001,0.001
300,20140224,0.001,0.001,0.001,0.001,0.001
300,20140225,0.001,0.001,0.001,0.001,0.001
300,20140226,0.001,0.001,0.001,0.001,0.001... (1 Reply)
Please help me on this
My script name is uniqueArray.csh
#!/bin/csh
set ARRAY = ( one teo three one three )
set ARRAY = ( $ARRAY one five three five )
How to remove the duplicates in this array ,sort and save those in the same variable or different variable.
Thanks in the advance
... (3 Replies)
Discussion started by: SA_Palani
3 Replies
LEARN ABOUT MOJAVE
subst
subst(n) Tcl Built-In Commands subst(n)
__________________________________________________________________________________________________________________________________________________NAME
subst - Perform backslash, command, and variable substitutions
SYNOPSIS
subst ?-nobackslashes? ?-nocommands? ?-novariables? string
_________________________________________________________________DESCRIPTION
This command performs variable substitutions, command substitutions, and backslash substitutions on its string argument and returns the
fully-substituted result. The substitutions are performed in exactly the same way as for Tcl commands. As a result, the string argument
is actually substituted twice, once by the Tcl parser in the usual fashion for Tcl commands, and again by the subst command.
If any of the -nobackslashes, -nocommands, or -novariables are specified, then the corresponding substitutions are not performed. For
example, if -nocommands is specified, command substitution is not performed: open and close brackets are treated as ordinary characters
with no special interpretation.
Note that the substitution of one kind can include substitution of other kinds. For example, even when the -novariables option is speci-
fied, command substitution is performed without restriction. This means that any variable substitution necessary to complete the command
substitution will still take place. Likewise, any command substitution necessary to complete a variable substitution will take place, even
when -nocommands is specified. See the EXAMPLES below.
If an error occurs during substitution, then subst will return that error. If a break exception occurs during command or variable substi-
tution, the result of the whole substitution will be the string (as substituted) up to the start of the substitution that raised the excep-
tion. If a continue exception occurs during the evaluation of a command or variable substitution, an empty string will be substituted for
that entire command or variable substitution (as long as it is well-formed Tcl.) If a return exception occurs, or any other return code is
returned during command or variable substitution, then the returned value is substituted for that substitution. See the EXAMPLES below.
In this way, all exceptional return codes are "caught" by subst. The subst command itself will either return an error, or will complete
successfully.
EXAMPLES
When it performs its substitutions, subst does not give any special treatment to double quotes or curly braces (except within command sub-
stitutions) so the script
set a 44
subst {xyz {$a}}
returns "xyz {44}", not "xyz {$a}" and the script
set a "p} q {r"
subst {xyz {$a}}
returns "xyz {p} q {r}", not "xyz {p} q {r}".
When command substitution is performed, it includes any variable substitution necessary to evaluate the script.
set a 44
subst -novariables {$a [format $a]}
returns "$a 44", not "$a $a". Similarly, when variable substitution is performed, it includes any command substitution necessary to
retrieve the value of the variable.
proc b {} {return c}
array set a {c c [b] tricky}
subst -nocommands {[b] $a([b])}
returns "[b] c", not "[b] tricky".
The continue and break exceptions allow command substitutions to prevent substitution of the rest of the command substitution and the rest
of string respectively, giving script authors more options when processing text using subst. For example, the script
subst {abc,[break],def}
returns "abc,", not "abc,,def" and the script
subst {abc,[continue;expr {1+2}],def}
returns "abc,,def", not "abc,3,def".
Other exceptional return codes substitute the returned value
subst {abc,[return foo;expr {1+2}],def}
returns "abc,foo,def", not "abc,3,def" and
subst {abc,[return -code 10 foo;expr {1+2}],def}
also returns "abc,foo,def", not "abc,3,def".
SEE ALSO
Tcl(n), eval(n), break(n), continue(n)
KEYWORDS
backslash substitution, command substitution, variable substitution
Tcl 7.4 subst(n)