Removing duplicate records in a file based on single column
Hi,
I want to remove duplicate records including the first line based on column1. For example
inputfile(filer.txt):
-------------
expected output:
----------------
Is it possible to achieve this using awk command ??
I tried below awk command , it is working but i dont want to give two times file name(filer.txt) in the command. I am allowed to give file name only one time.
Please suggest me how to achieve this.
Thanks in advance
Last edited by Franklin52; 08-22-2011 at 03:57 AM..
Reason: Please use code tags for code and data samples, thank you
Hi all,
I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g.
COL1 COL2 COL3 ............... COL47
1 XX 45 ... (4 Replies)
Hi,
Need to find a duplicate records on the first column,
ANU4501710430989 0000000W20389390
ANU4501710430989 0000000W67065483
ANU4501130050520 0000000W80838713
ANU4501210170685 0000000W69246611... (3 Replies)
Can anyone help me to removing duplicate records from 2 separate files in UNIX?
Please find the sample records for both the files
cat Monday.dat
3FAHP0JA1AR319226MOHMED ATEK 966504453742 SAU2010DE
3LNHL2GC6AR636361HEA DEUK CHOI 821057314531 KOR2010LE
3MEHM0JG7AR652083MUTLAB NAL-NAFISAH... (4 Replies)
I am a newbie to shell scripting ..
I have a .csv file. It has 1000 some rows and about 7 columns...
but before I insert this data to a table I have to parse it and clean it ..basing on the value of the first column..which a string of phone number type...
example below..
column 1 ... (2 Replies)
Hello, I'm trying to delete duplicates when there are more than 10 duplicates, based on the value of the first column.
e.g.
a 1
a 2
a 3
b 1
c 1
gives
b 1
c 1
but requires 11 duplicates before it deletes.
Thanks for the help
Video tutorial on how to use code tags in The UNIX... (11 Replies)
I was reading this thread. It looks like a simpler way to say this is to only keep uniq lines based on field or column 1.
https://www.unix.com/shell-programming-scripting/165717-removing-duplicate-records-file-based-single-column.html
Can someone explain this command please? How are there no... (5 Replies)
Hi,
I have tried to remove dublicate lines based on first column with pipe delimiter . but i ma not able to get some uniqu lines
Command : sort -t'|' -nuk1 file.txt
Input :
38376KZ|09/25/15|1.057
38376KZ|09/25/15|1.057
02006YB|09/25/15|0.859
12593PS|09/25/15|2.803... (2 Replies)
I have csv file with 30, 40 columns
Pasting just three column for problem description
I want to filter record if column 1 matches CN or DN then,
check for values in column 2 if column contain 1235, 1235 then in column 3 values must be sequence of 2345, 2345
and if column 2 contains 6789, 6789... (5 Replies)
Hi Experts,
I have csv file with 30, 40 columns
Pasting just 2 column for problem description.
Need to print error if below combination is not present in file
check for column-1 (DocumentNumber) and filter columns where value in DocumentNumber field is same.
For all such rows, the field... (7 Replies)
Discussion started by: as7951
7 Replies
LEARN ABOUT DEBIAN
otfdump
OTFDUMP(1) User Commands OTFDUMP(1)NAME
otfdump - otfdump
DESCRIPTION
otfdump - convert otf traces or parts of it into a human readable, long
version
Options:
-h, --help
show this help message
-V show OTF version
-f <n> set max number of filehandles available (default: 50)
-o <file>
output file if the ouput file is unspecified the stdout will be used
--num <a> <b>
output only records no. [a,b]
--time <a> <b> output only records with time stamp in [a,b]
--nodef
omit definition records
--noevent
omit event records
--nostat
omit statistic records
--nosnap
omit snapshot records
--nomarker
omit marker records
--nokeyvalue
omit key-value pairs
--fullkeyvalue show key-value pairs including the contents
of byte-arrays
--procs <a>
show only processes <a> <a> is a space-seperated list of process-tokens
--records <a>
show only records <a> <a> is a space-seperated list of record-type-numbers record-type-numbers can be found in OTF_Definitions.h
(OTF_*_RECORD)
-s, --silent
do not display anything except the time otfdump needed to read the tracefile
otfdump 1.10.2 May 2012 OTFDUMP(1)