Sponsored Content
Full Discussion: report duplicate
Top Forums UNIX for Dummies Questions & Answers report duplicate Post 302539243 by neutronscott on Friday 15th of July 2011 09:35:05 PM
Old 07-15-2011
Using awk arrays...

Code:
[mute@geek ~/test]$ awk '{l[$0]++} END {for (i in l) if (l[i]>1) print i }' file.txt
hi
rag

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove duplicate

Hi all, I have a text file fileA.txt DXRV|02/28/2006 11:36:49.049|SAC||||CDxAcct=2420991350 DXRV|02/28/2006 11:37:06.404|SAC||||CDxAcct=6070970034 DXRV|02/28/2006 11:37:25.740|SAC||||CDxAcct=2420991350 DXRV|02/28/2006 11:38:32.633|SAC||||CDxAcct=6070970034 DXRV|02/28/2006... (2 Replies)
Discussion started by: sabercats
2 Replies

2. HP-UX

Disk duplicate in 10.20

HI: I know this topic already exist in this forum but not exactly with my problem. I want to duplicate a disk , my source disk is like 2gb size, while the new disk is like 36 gb size. The problems: When I use the command dd it fails, I think because the disk sizes, and the sizes of the... (13 Replies)
Discussion started by: pmoren
13 Replies

3. UNIX for Dummies Questions & Answers

Report of duplicate files based on part of the filename

I have the files logged in the file system with names in the format of : filename_ordernumber_date_time eg: file_1_12012007_1101.txt file_2_12022007_1101.txt file_1_12032007_1101.txt I need to find out all the files that are logged multiple times with same order number. In the above eg, I... (1 Reply)
Discussion started by: sudheshnaiyer
1 Replies

4. Shell Programming and Scripting

duplicate directories

Hi, I have file which users like filename ->"readfile", following entries peter john alaska abcd xyz and i have directory /var/ i want to do first cat of "readfile" line by line and first read peter in variable and also cross check with /var/ how many directories are avaialble... (8 Replies)
Discussion started by: learnbash
8 Replies

5. Shell Programming and Scripting

Duplicate

I am looking for a way to delete duplicate entries in a VERY large file (approx 2gb) However I need to compare several fields before determining if this is a duplicate. I setup a hash in perl but it seems to not function correctly. Any help appreciated. of the 19 comma separated fields I... (2 Replies)
Discussion started by: Goyde
2 Replies

6. Shell Programming and Scripting

Duplicate Line Report per Section

I've been working on a script (/bin/sh) in which I have requested and received help here (in which I am very grateful for!). The client has modified their requirements (a tad), so without messing up the script to much, I come once again for assistance. Here are the file.dat contents: ABC1... (4 Replies)
Discussion started by: petersf
4 Replies

7. Shell Programming and Scripting

Find duplicate based on 'n' fields and mark the duplicate as 'D'

Hi, In a file, I have to mark duplicate records as 'D' and the latest record alone as 'C'. In the below file, I have to identify if duplicate records are there or not based on Man_ID, Man_DT, Ship_ID and I have to mark the record with latest Ship_DT as "C" and other as "D" (I have to create... (7 Replies)
Discussion started by: machomaddy
7 Replies

8. Shell Programming and Scripting

Duplicate value

Hi All, i have file like ID|Indiv_ID 12345|10001 |10001 |10001 23456|10002 |10002 |10002 |10002 |10003 |10004 if indiv_id having duplicate values and corresponding ID column is null then copy the id. I need output like: ID|Indiv_ID 12345|10001... (11 Replies)
Discussion started by: bmk
11 Replies

9. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies

10. UNIX for Beginners Questions & Answers

Iterate through a list - checking for a duplicate then report it ot

I have a job that produces a file of barcodes that gets added to every time the job runs I want to check the list to see if the barcode is already in the list and report it out if it is. (3 Replies)
Discussion started by: worky
3 Replies
explain_strdup_or_die(3)				     Library Functions Manual					  explain_strdup_or_die(3)

NAME
explain_strdup_or_die - duplicate a string and report errors SYNOPSIS
#include <libexplain/strdup.h> char *explain_strdup_or_die(const char *data); char *explain_strdup_on_error(const char *data); DESCRIPTION
The explain_strdup_or_die function is used to call the strdup(3) system call. On failure an explanation will be printed to stderr, obtained from the explain_strdup(3) function, and then the process terminates by calling exit(EXIT_FAILURE). The explain_strdup_on_error function is used to call the strdup(3) system call. On failure an explanation will be printed to stderr, obtained from the explain_strdup(3) function, but still returns to the caller. data The data, exactly as to be passed to the strdup(3) system call. RETURN VALUE
The explain_strdup_or_die function only returns on success, see strdup(3) for more information. On failure, prints an explanation and exits, it does not return. The explain_strdup_on_error function always returns the value return by the wrapped strdup(3) system call. EXAMPLE
The explain_strdup_or_die function is intended to be used in a fashion similar to the following example: char *result = explain_strdup_or_die(data); SEE ALSO
strdup(3) duplicate a string explain_strdup(3) explain strdup(3) errors exit(2) terminate the calling process COPYRIGHT
libexplain version 0.52 Copyright (C) 2009 Peter Miller explain_strdup_or_die(3)
All times are GMT -4. The time now is 01:21 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy