10-17-2008
How to find Duplicate Records in a text file
Hi all
pls help me by providing soln for my problem
I'm having a text file which contains duplicate records .
Example:
abc 1000 3452 2463 2343 2176 7654 3452 8765 5643 3452
abc 1000 3452 2463 2343 2176 7654 3452 8765 5643 3452
tas 3420 3562 2123 1343 2176 7654 3252 8765 5643 3452
aer 1000 3452 2463 2343 2176 7654 3452 8765 5643 3452
tas 3420 3562 2123 1343 2176 7654 3252 8765 5643 3452
I want to compare each and every column of every row with rest columns of file
Im doing by following method:
two for loops
first[$i]=`head -$i filename.txt | tail -1 | cut -d" " -f$num1-$num2`
in same way im extracting entire columns using another foer loop and comparing with "if" condition. Its showing wrong value
can any one help me.
By
G.Aavudai Nayagam.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi experts,
I am new to scripting. I have a requirement as below.
File1:
A|123|NAME1
A|123|NAME2
B|123|NAME3
File2:
C|123|NAME4
C|123|NAME5
D|123|NAME6
1) I have 2 merge both the files.
2) need to do a sort ( key fields are first and second field)
3) remove all the instances... (3 Replies)
Discussion started by: vukkusila
3 Replies
2. UNIX for Advanced & Expert Users
Hi,
I want to fetch duplicate records from an external table to a text file.
Pls suggest me.
Thanks (1 Reply)
Discussion started by: shilendrajadon
1 Replies
3. Shell Programming and Scripting
Hi all:
Let's suppose I have a file like this (but with many more records).
XX ME 342 8688 2006 7 6 3c 60.029 -38.568 2901 0001 74 4 7603 8
969.8 958.4 3.6320 34.8630
985.5 973.9 3.6130 34.8600
998.7 986.9 3.6070 34.8610
1003.6 991.7 ... (4 Replies)
Discussion started by: rleal
4 Replies
4. Shell Programming and Scripting
Hi:
I've been searching the net but didnt find a clue. I have a file in which, for some records, some fields coincide. I want to compare one (or more) of the dissimilar fields and retain the one record that fulfills a certain condition. For example, on this file:
99 TR 1991 5 06 ... (1 Reply)
Discussion started by: rleal
1 Replies
5. Shell Programming and Scripting
Dear All,
I have one file which looks like :
account1:passwd1
account2:passwd2
account3:passwd3
account1:passwd4
account5:passwd5
account6:passwd6
you can see there're two records for account1. and is there any shell command which can find out : account1 is the duplicate record in... (3 Replies)
Discussion started by: tiger2000
3 Replies
6. Shell Programming and Scripting
Hi,
Need to find a duplicate records on the first column,
ANU4501710430989 0000000W20389390
ANU4501710430989 0000000W67065483
ANU4501130050520 0000000W80838713
ANU4501210170685 0000000W69246611... (3 Replies)
Discussion started by: Murugesh
3 Replies
7. UNIX for Dummies Questions & Answers
Hi Unix gurus,
Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me.
File format: CSV file
File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies
8. Shell Programming and Scripting
I have a flat file that contains records similar to the following two lines;
1984/11/08 7 700000 123456789 2
1984/11/08 1941/05/19 7 700000 123456789 2
The 123456789 2 represents an account number, this is how I identify the duplicate record.
The ### signs represent... (4 Replies)
Discussion started by: jolney
4 Replies
9. Shell Programming and Scripting
I have 2 files
"File 1" is delimited by ";" and "File 2" is delimited by "|".
File 1 below (3 record shown):
Doc1;03/01/2012;New York;6 Main Street;Mr. Smith 1;Mr. Jones
Doc2;03/01/2012;Syracuse;876 Broadway;John Davis;Barbara Lull
Doc3;03/01/2012;Buffalo;779 Old Windy Road;Charles... (2 Replies)
Discussion started by: vestport
2 Replies
10. Shell Programming and Scripting
Hello,
I have some text data that is in the form of multi-line records. Each record ends with the string $$$$ and the next record starts on the next line.
RDKit 2D
15 14 0 0 0 0 0 0 0 0999 V2000
5.4596 2.1267 0.0000 C 0 0 0 0 0 0 0 0 0 0 0 ... (5 Replies)
Discussion started by: LMHmedchem
5 Replies
LEARN ABOUT FREEBSD
nfscbd
NFSCBD(8) BSD System Manager's Manual NFSCBD(8)
NAME
nfscbd -- NFSv4 client side callback daemon
SYNOPSIS
nfscbd [-p port_number] [-P client_principal]
DESCRIPTION
nfscbd runs on a client using NFSv4 to handle callback requests from the NFSv4 server. If no nfscbd is running, NFSv4 mounts will still
work, but the server will never issue Open Delegations to the client.
One callback server and one master server are always started.
The following options are available:
-p port_number
Specifies what port# the callback server should use.
-P client_principal
Specifies the host based principal name to be used as the target for callbacks over RPCSEC_GSS. For KerberosV, it must be in the
client's default keytab file. This client_principal should be the same one specified by the gssname argument being used by nfsv4
mounts. If you do not specify this argument, callbacks will still work over AUTH_SYS, which is what many extant servers use even for
RPCSEC_GSS mounts, as of 2009.
For example, ``nfscbd -p 7654 -P root'' starts the daemon to handle callbacks on port# 7654 and is using the host based principal
root@<client-host>.<dns-domain> as the callback target.
nfscbd listens for service requests at the port defined by NFSV4_CBPORT in /usr/include/fs/nfs/nfs.h, unless -p has been specified. For more
information on what callbacks and Open Delegations do, see Network File System (NFS) Version 4 Protocol, RFC3530.
EXIT STATUS
The nfscbd utility exits 0 on success, and >0 if an error occurs.
SEE ALSO
nfsv4(4), mount_nfs(8)
HISTORY
First introduced with the experimental nfs client for NFSv4 support in 2009.
BSD
April 25, 2009 BSD