Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: CSV File:Filter duplicate records from column1 & another column having unique record
Top Forums Shell Programming and Scripting CSV File:Filter duplicate records from column1 & another column having unique record Post 303010467 by RudiC on Tuesday 2nd of January 2018 05:46:02 AM
Old 01-02-2018
No apologies needed as people in these fora are here to help. Posts don't disturb anybody - what IS disturbing is if people don't learn, be it to comply to forum rules, how to resonably specify a problem, or to apply / adapt coding hints to actual problerms.

Your code sample doesn't word with the sample in post#1 as the field separator in the data is a <TAB> followed by multiple spaces (matched by the default awk FS) and the code has | . Try
Code:
awk 'C[$1,$2]++ {print "error line", NR}' file

and report back the results.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

return a list of unique values of a column from csv format file

Hi all, I have a huge csv file with the following format of data, Num SNPs, 549997 Total SNPs,555352 Num Samples, 157 SNP, SampleID, Allele1, Allele2 A001,AB1,A,A A002,AB1,A,A A003,AB1,A,A ... ... ... I would like to write out a list of unique SNP (column 1). Could you... (3 Replies)
Discussion started by: phoeberunner
3 Replies

2. Shell Programming and Scripting

Find Duplicate records in first Column in File

Hi, Need to find a duplicate records on the first column, ANU4501710430989 0000000W20389390 ANU4501710430989 0000000W67065483 ANU4501130050520 0000000W80838713 ANU4501210170685 0000000W69246611... (3 Replies)
Discussion started by: Murugesh
3 Replies

3. UNIX for Dummies Questions & Answers

CSV file:Find duplicates, save original and duplicate records in a new file

Hi Unix gurus, Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me. File format: CSV file File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies

4. Shell Programming and Scripting

Removing duplicate records in a file based on single column

Hi, I want to remove duplicate records including the first line based on column1. For example inputfile(filer.txt): ------------- 1,3000,5000 1,4000,6000 2,4000,600 2,5000,700 3,60000,4000 4,7000,7777 5,999,8888 expected output: ---------------- 3,60000,4000 4,7000,7777... (5 Replies)
Discussion started by: G.K.K
5 Replies

5. Shell Programming and Scripting

Removing duplicate records in a file based on single column explanation

I was reading this thread. It looks like a simpler way to say this is to only keep uniq lines based on field or column 1. https://www.unix.com/shell-programming-scripting/165717-removing-duplicate-records-file-based-single-column.html Can someone explain this command please? How are there no... (5 Replies)
Discussion started by: cokedude
5 Replies

6. Linux

Filter a .CSV file based on the 5th column values

I have a .CSV file with the below format: "column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10 "12310","42324564756","a simple string with a , comma","string with or, without commas","string 1","USD","12","70%","08/01/2013",""... (2 Replies)
Discussion started by: dhruuv369
2 Replies

7. Shell Programming and Scripting

Output first unique record in csv file

Hi, I have to output a new csv file from an input csv file with first unique value in the first column. input csv file color product id status green 102 pass yellow 201 hold yellow 202 keep green 101 ok green 103 hold yellow 203 ... (5 Replies)
Discussion started by: Chris LAU
5 Replies

8. Linux

To get all the columns in a CSV file based on unique values of particular column

cat sample.csv ID,Name,no 1,AAA,1 2,BBB,1 3,AAA,1 4,BBB,1 cut -d',' -f2 sample.csv | sort | uniq this gives only the 2nd column values Name AAA BBB How to I get all the columns of CSV along with this? (1 Reply)
Discussion started by: sanvel
1 Replies

9. Shell Programming and Scripting

Filter file to remove duplicate values in first column

Hello, I have a script that is generating a tab delimited output file. num Name PCA_A1 PCA_A2 PCA_A3 0 compound_00 -3.5054 -1.1207 -2.4372 1 compound_01 -2.2641 0.4287 -1.6120 3 compound_03 -1.3053 1.8495 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies

10. Shell Programming and Scripting

Filter duplicate records from csv file with condition on one column

I have csv file with 30, 40 columns Pasting just three column for problem description I want to filter record if column 1 matches CN or DN then, check for values in column 2 if column contain 1235, 1235 then in column 3 values must be sequence of 2345, 2345 and if column 2 contains 6789, 6789... (5 Replies)
Discussion started by: as7951
5 Replies

LEARN ABOUT CENTOS

glsamplemaski

GLSAMPLEMASKI(3G)						   OpenGL Manual						 GLSAMPLEMASKI(3G)

NAME

       glSampleMaski - set the value of a sub-word of the sample mask

C SPECIFICATION

       void glSampleMaski(GLuint maskNumber, GLbitfield mask);

PARAMETERS

       maskNumber
	   Specifies which 32-bit sub-word of the sample mask to update.

       mask
	   Specifies the new value of the mask sub-word.

DESCRIPTION

       glSampleMaski sets one 32-bit sub-word of the multi-word sample mask, GL_SAMPLE_MASK_VALUE.

       maskIndex specifies which 32-bit sub-word of the sample mask to update, and mask specifies the new value to use for that sub-word.
       maskIndex must be less than the value of GL_MAX_SAMPLE_MASK_WORDS. Bit B of mask word M corresponds to sample 32 x M + B.

NOTES

       glSampleMaski is available only if the GL version is 3.2 or greater, or if the ARB_texture_multisample extension is supported.

ERRORS

       GL_INVALID_VALUE is generated if maskIndex is greater than or equal to the value of GL_MAX_SAMPLE_MASK_WORDS.

SEE ALSO

       glGenRenderbuffers(), glBindRenderbuffer(), glRenderbufferStorageMultisample(), glFramebufferRenderbuffer(), glDeleteRenderbuffers()

COPYRIGHT

       Copyright (C) 2010 Khronos Group. This material may be distributed subject to the terms and conditions set forth in the Open Publication
       License, v 1.0, 8 June 1999.  http://opencontent.org/openpub/.

AUTHORS

       opengl.org

opengl.org							    06/10/2014							 GLSAMPLEMASKI(3G)
All times are GMT -4. The time now is 12:28 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy