Code used to find duplicate with regard to 2nd column
Code:
awk 'NR == 1 {p=$2; next} p == $2 { print "Line" NR "$2 is duplicated"} {p=$2}' FS="," ./input.csv
Now is there a wise way to de-duplicate the entire line (remove the duplicate) based on the criteria found within this one liner or wrapped around additional logic?
Moderator's Comments:
Please use CODE tags (not ICODE tags) for full-line or multi-line code segments AND for sample input and sample output.
Your thread title says you are trying to find duplicates in the 1st field; your code prints lines in which the 2nd field on the line has been seen before. Note that it prints duplicates; it does not remove duplicates.
And, since there are no lines in your sample input where the 1st field is duplicated on any other line, I have no idea what you are trying to do. What additional logic are you talking about? What output are you hoping to produce from this sample input?
Hello experts,
I have a requirement where I have to implement two checks on a csv file:
1. Check to see if the value in first column is duplicate, if any value is duplicate script should exit.
2. Check to verify if the value at second column is between "yes" or "no", if it is anything else... (4 Replies)
I have a .CSV file with the below format:
"column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10
"12310","42324564756","a simple string with a , comma","string with or, without commas","string 1","USD","12","70%","08/01/2013",""... (2 Replies)
input.csv:
Field1,Field2,Field3,Field4,Field4
abc ,123 ,xyz ,000 ,pqr
mno ,123 ,dfr ,111 ,bbb
output:
Field2,Field4
123 ,000
123 ,111
how to fetch the values of Field4 where Field2='123'
I don't want to fetch the values based on column position. Instead want to... (10 Replies)
Hello,
I have a script that is generating a tab delimited output file.
num Name PCA_A1 PCA_A2 PCA_A3
0 compound_00 -3.5054 -1.1207 -2.4372
1 compound_01 -2.2641 0.4287 -1.6120
3 compound_03 -1.3053 1.8495 ... (3 Replies)
Hi Gurus,
I have a file(weblog) as below
abc|xyz|123|agentcode=sample code abcdeeess,agentcode=sample code abcdeeess,agentcode=sample code abcdeeess|agentadd=abcd stereet 23343,agentadd=abcd stereet 23343
sss|wwq|999|agentcode=sample1 code wqwdeeess,gentcode=sample1 code... (4 Replies)
Dear folks
I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
I have csv file with 30, 40 columns
Pasting just three column for problem description
I want to filter record if column 1 matches CN or DN then,
check for values in column 2 if column contain 1235, 1235 then in column 3 values must be sequence of 2345, 2345
and if column 2 contains 6789, 6789... (5 Replies)
Hi Experts,
I have csv file with 30, 40 columns
Pasting just 2 column for problem description.
Need to print error if below combination is not present in file
check for column-1 (DocumentNumber) and filter columns where value in DocumentNumber field is same.
For all such rows, the field... (7 Replies)
Discussion started by: as7951
7 Replies
LEARN ABOUT OPENSOLARIS
wscol
wstring(3C) Standard C Library Functions wstring(3C)NAME
wstring, wscasecmp, wsncasecmp, wsdup, wscol - Process Code string operations
SYNOPSIS
#include <widec.h>
int wscasecmp(const wchar_t *s1, const wchar_t *s2);
int wsncasecmp(const wchar_t *s1, const wchar_t *s2, int n);
wchar_t *wsdup(const wchar_t *s);
int wscol(const wchar_t *s);
DESCRIPTION
These functions operate on Process Code strings terminated by wchar_t null characters. During appending or copying, these routines do not
check for an overflow condition of the receiving string. In the following, s, s1, and s2 point to Process Code strings terminated by a
wchar_t null.
wscasecmp(), wsncasecmp()
The wscasecmp() function compares its arguments, ignoring case, and returns an integer greater than, equal to, or less than 0, depending
upon whether s1 is lexicographically greater than, equal to, or less than s2. It makes the same comparison but compares at most n Process
Code characters. The four Extended Unix Code (EUC) codesets are ordered from lowest to highest as 0, 2, 3, 1 when characters from different
codesets are compared.
wsdup()
The wsdup() function returns a pointer to a new Process Code string, which is a duplicate of the string pointed to by s. The space for the
new string is obtained using malloc(3C). If the new string cannot be created, a null pointer is returned.
wscol()
The wscol() function returns the screen display width (in columns) of the Process Code string s.
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+-----------------------------------------------------------+
|ATTRIBUTE TYPE ATTRIBUTE VALUE |
|MT-Level MT-Safe |
+-----------------------------------------------------------+
SEE ALSO malloc(3C), string(3C), wcstring(3C), attributes(5)SunOS 5.11 29 Dec 1996 wstring(3C)
10-16-2015
