Sponsored Content
Full Discussion: List Duplicate
Top Forums Shell Programming and Scripting List Duplicate Post 302123856 by vgersh99 on Wednesday 27th of June 2007 01:34:04 PM
Old 06-27-2007
Code:
nawk '{
  idx= $2 SUBSEP $3
  arr[idx] = (idx in arr) ? arr[idx] ORS $0 : $0
  arrCnt[idx]++
}
END {
  for (i in arr)
     if (arrCnt[i] > 1) print arr[i]
}' myInputFile

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Get a none duplicate list file

Dear sir i got a file like following format with the duplicate line: AAA AAA AAA AAA AAA BBB BBB BBB BBB CCC CCC CCC CCC ... (5 Replies)
Discussion started by: trynew
5 Replies

2. Shell Programming and Scripting

Removing duplicate files from list with different path

I have a list which contains all the jar files shipped with the product I am involved with. Now, in this list I have some jar files which appear again and again. But these jar files are present in different folders. My input file looks like this /path/1/to a.jar /path/2/to a.jar /path/1/to... (10 Replies)
Discussion started by: vino
10 Replies

3. Shell Programming and Scripting

Splitting a list @list by space delimiter so i can access it by using $list[0 ..1..2]

EDIT : This is for perl @data2 = grep(/$data/, @list_now); This gives me @data2 as Printing data2 11 testzone1 running /zones/testzone1 ***-*****-****-*****-***** native shared But I really cant access data2 by its individual elements. $data2 is the entire list, while $data,2,3...... (1 Reply)
Discussion started by: shriyer
1 Replies

4. Shell Programming and Scripting

Find duplicate based on 'n' fields and mark the duplicate as 'D'

Hi, In a file, I have to mark duplicate records as 'D' and the latest record alone as 'C'. In the below file, I have to identify if duplicate records are there or not based on Man_ID, Man_DT, Ship_ID and I have to mark the record with latest Ship_DT as "C" and other as "D" (I have to create... (7 Replies)
Discussion started by: machomaddy
7 Replies

5. Shell Programming and Scripting

Duplicate value

Hi All, i have file like ID|Indiv_ID 12345|10001 |10001 |10001 23456|10002 |10002 |10002 |10002 |10003 |10004 if indiv_id having duplicate values and corresponding ID column is null then copy the id. I need output like: ID|Indiv_ID 12345|10001... (11 Replies)
Discussion started by: bmk
11 Replies

6. Shell Programming and Scripting

Duplicate files and output list

Gents, I have a file like this. 1 1 1 2 2 3 2 4 2 5 3 6 3 7 4 8 5 9 I would like to get something like it 1 1 2 2 3 4 5 3 6 7 Thanks in advance for your support :b: (8 Replies)
Discussion started by: jiam912
8 Replies

7. Shell Programming and Scripting

Find and remove duplicate record and print list

Gents, I needs to delete duplicate values and only get uniq values based in columns 2-27 Always we should keep the last record found... I need to store one clean file and other with the duplicate values removed. Input : S3033.0 7305.01 0 420123.8... (18 Replies)
Discussion started by: jiam912
18 Replies

8. Shell Programming and Scripting

List duplicate files based on Name and size

Hello, I have a huge directory (with millions of files) and need to find out duplicates based on BOTH file name and File size. I know fdupes but it calculates MD5 which is very time-consuming and especially it takes forever as I have millions of files. Can anyone please suggest a script or... (7 Replies)
Discussion started by: prvnrk
7 Replies

9. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies

10. UNIX for Beginners Questions & Answers

Iterate through a list - checking for a duplicate then report it ot

I have a job that produces a file of barcodes that gets added to every time the job runs I want to check the list to see if the barcode is already in the list and report it out if it is. (3 Replies)
Discussion started by: worky
3 Replies
GET_DEFINED_FUNCTIONS(3)						 1						  GET_DEFINED_FUNCTIONS(3)

get_defined_functions - Returns an array of all defined functions

SYNOPSIS
array get_defined_functions (void ) DESCRIPTION
Gets an array of all defined functions. RETURN VALUES
Returns a multidimensional array containing a list of all defined functions, both built-in (internal) and user-defined. The internal func- tions will be accessible via $arr["internal"], and the user defined ones using $arr["user"] (see example below). EXAMPLES
Example #1 get_defined_functions(3) example <?php function myrow($id, $data) { return "<tr><th>$id</th><td>$data</td></tr> "; } $arr = get_defined_functions(); print_r($arr); ?> The above example will output something similar to: Array ( [internal] => Array ( [0] => zend_version [1] => func_num_args [2] => func_get_arg [3] => func_get_args [4] => strlen [5] => strcmp [6] => strncmp ... [750] => bcscale [751] => bccomp ) [user] => Array ( [0] => myrow ) ) SEE ALSO
function_exists(3), get_defined_vars(3), get_defined_constants(3), get_declared_classes(3). PHP Documentation Group GET_DEFINED_FUNCTIONS(3)
All times are GMT -4. The time now is 06:02 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy