Sponsored Content
Full Discussion: report duplicate
Top Forums UNIX for Dummies Questions & Answers report duplicate Post 302539480 by dennis.jacob on Monday 18th of July 2011 12:43:29 AM
Old 07-18-2011
Try:

PHP Code:
awk '!_[$1]++' filename 
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove duplicate

Hi all, I have a text file fileA.txt DXRV|02/28/2006 11:36:49.049|SAC||||CDxAcct=2420991350 DXRV|02/28/2006 11:37:06.404|SAC||||CDxAcct=6070970034 DXRV|02/28/2006 11:37:25.740|SAC||||CDxAcct=2420991350 DXRV|02/28/2006 11:38:32.633|SAC||||CDxAcct=6070970034 DXRV|02/28/2006... (2 Replies)
Discussion started by: sabercats
2 Replies

2. HP-UX

Disk duplicate in 10.20

HI: I know this topic already exist in this forum but not exactly with my problem. I want to duplicate a disk , my source disk is like 2gb size, while the new disk is like 36 gb size. The problems: When I use the command dd it fails, I think because the disk sizes, and the sizes of the... (13 Replies)
Discussion started by: pmoren
13 Replies

3. UNIX for Dummies Questions & Answers

Report of duplicate files based on part of the filename

I have the files logged in the file system with names in the format of : filename_ordernumber_date_time eg: file_1_12012007_1101.txt file_2_12022007_1101.txt file_1_12032007_1101.txt I need to find out all the files that are logged multiple times with same order number. In the above eg, I... (1 Reply)
Discussion started by: sudheshnaiyer
1 Replies

4. Shell Programming and Scripting

duplicate directories

Hi, I have file which users like filename ->"readfile", following entries peter john alaska abcd xyz and i have directory /var/ i want to do first cat of "readfile" line by line and first read peter in variable and also cross check with /var/ how many directories are avaialble... (8 Replies)
Discussion started by: learnbash
8 Replies

5. Shell Programming and Scripting

Duplicate

I am looking for a way to delete duplicate entries in a VERY large file (approx 2gb) However I need to compare several fields before determining if this is a duplicate. I setup a hash in perl but it seems to not function correctly. Any help appreciated. of the 19 comma separated fields I... (2 Replies)
Discussion started by: Goyde
2 Replies

6. Shell Programming and Scripting

Duplicate Line Report per Section

I've been working on a script (/bin/sh) in which I have requested and received help here (in which I am very grateful for!). The client has modified their requirements (a tad), so without messing up the script to much, I come once again for assistance. Here are the file.dat contents: ABC1... (4 Replies)
Discussion started by: petersf
4 Replies

7. Shell Programming and Scripting

Find duplicate based on 'n' fields and mark the duplicate as 'D'

Hi, In a file, I have to mark duplicate records as 'D' and the latest record alone as 'C'. In the below file, I have to identify if duplicate records are there or not based on Man_ID, Man_DT, Ship_ID and I have to mark the record with latest Ship_DT as "C" and other as "D" (I have to create... (7 Replies)
Discussion started by: machomaddy
7 Replies

8. Shell Programming and Scripting

Duplicate value

Hi All, i have file like ID|Indiv_ID 12345|10001 |10001 |10001 23456|10002 |10002 |10002 |10002 |10003 |10004 if indiv_id having duplicate values and corresponding ID column is null then copy the id. I need output like: ID|Indiv_ID 12345|10001... (11 Replies)
Discussion started by: bmk
11 Replies

9. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies

10. UNIX for Beginners Questions & Answers

Iterate through a list - checking for a duplicate then report it ot

I have a job that produces a file of barcodes that gets added to every time the job runs I want to check the list to see if the barcode is already in the list and report it out if it is. (3 Replies)
Discussion started by: worky
3 Replies
UNIDUP(1)						      General Commands Manual							 UNIDUP(1)

NAME
unidup - Scan through a sorted .hex file and report duplicate code points. SYNOPSIS
unidup < input_font.hex DESCRIPTION
unidup reads a sorted GNU Unifont .hex file (sorted with the Unix sort utility) from stdin and prints notification of any duplicate code points on stdout. FILES
*.hex GNU Unifont font files SEE ALSO
bdfimplode(1), hex2bdf(1), hex2bdf-split(1), hex2sfd(1), hexbraille(1), hexdraw(1), hexmerge(1), johab2ucs2(1), unibmp2hex(1), unicover- age(1), unihex2bmp(1), unipagecount(1), uniunmask(1) AUTHOR
unidup was written by Paul Hardy. LICENSE
unidup is Copyright (C) 2007 Paul Hardy, and is released under version 2 of the GNU General Public License, or (at your option) a later version. BUGS
No known bugs exist. 2007 Dec 31 UNIDUP(1)
All times are GMT -4. The time now is 11:29 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy