Sponsored Content
Top Forums Shell Programming and Scripting removing duplicates based on key Post 302197758 by jim mcnamara on Wednesday 21st of May 2008 03:50:41 PM
Old 05-21-2008
This gives the last occurrence only of the key field:
Code:
awk '{ arr[substr($0,1,4)]=$0} END {for (i in arr ){print arr[i]}}' inputfile

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing duplicates

Hi, I've been trying to removed duplicates lines with similar columns in a fixed width file and it's not working. I've search the forum but nothing comes close. I have a sample file: 27147140631203RA CCD * 27147140631203RA PPN * 37147140631207RD AAA 47147140631203RD JNA... (12 Replies)
Discussion started by: giannicello
12 Replies

2. Shell Programming and Scripting

removing duplicates

Hi I have a file that are a list of people & their credentials i recieve frequently The issue is that whne I catnet this list that duplicat entries exists & are NOT CONSECUTIVE (i.e. uniq -1 may not weork here ) I'm trying to write a scrip that will remove duplicate entries the script can... (5 Replies)
Discussion started by: stevie_velvet
5 Replies

3. Shell Programming and Scripting

Removing duplicates

Hi, I have a file in the below format., test test (10) to to (25) see see (45) and i need the output in the format of test 10 to 25 see 45 Some one help me? (6 Replies)
Discussion started by: imdadulla
6 Replies

4. Shell Programming and Scripting

Remove duplicates based on the two key columns

Hi All, I needs to fetch unique records based on a keycolumn(ie., first column1) and also I needs to get the records which are having max value on column2 in sorted manner... and duplicates have to store in another output file. Input : Input.txt 1234,0,x 1234,1,y 5678,10,z 9999,10,k... (7 Replies)
Discussion started by: kmsekhar
7 Replies

5. Shell Programming and Scripting

need to remove duplicates based on key in first column and pattern in last column

Given a file such as this I need to remove the duplicates. 00060011 PAUL BOWSTEIN ad_waq3_921_20100826_010517.txt 00060011 PAUL BOWSTEIN ad_waq3_921_20100827_010528.txt 0624-01 RUT CORPORATION ad_sade3_10_20100827_010528.txt 0624-01 RUT CORPORATION ... (13 Replies)
Discussion started by: script_op2a
13 Replies

6. UNIX for Dummies Questions & Answers

Removing duplicates based on key

Hi, I have the input file with the below data: 12345|12|34 12345|13|23 3456|12|90 15670|12|13 12345|10|14 3456|12|13 I need to remove the duplicates based on the first field only. I need the output like: 12345|12|34 3456|12|90 15670|12|13 The first field needs to be unique . (4 Replies)
Discussion started by: pandeesh
4 Replies

7. Emergency UNIX and Linux Support

Removing all the duplicates

i want to remove all the duplictaes in a file.I dont want even a single entry. For the input data: 12345|12|34 12345|13|23 3456|12|90 15670|12|13 12345|10|14 3456|12|13 i need the below data in one file 15670|12|13 and the below data in another file (9 Replies)
Discussion started by: pandeesh
9 Replies

8. Shell Programming and Scripting

finding duplicates in csv based on key columns

Hi team, I have 20 columns csv files. i want to find the duplicates in that file based on the column1 column10 column4 column6 coulnn8 coulunm2 . if those columns have same values . then it should be a duplicate record. can one help me on finding the duplicates, Thanks in advance. ... (2 Replies)
Discussion started by: baskivs
2 Replies

9. Shell Programming and Scripting

Removing duplicates in fixed width file which has multiple key columns

Hi All , I have a requirement where I need to remove duplicates from a fixed width file which has multiple key columns .Also , need to capture the duplicate records into another file . File has 8 columns. Key columns are col1 and col2. Col1 has the length of 8 col 2 has the length of 3. ... (5 Replies)
Discussion started by: saj
5 Replies

10. Shell Programming and Scripting

Removing duplicates from delimited file based on 2 columns

Hi guys,Got a bit of a bind I'm in. I'm looking to remove duplicates from a pipe delimited file, but do so based on 2 columns. Sounds easy enough, but here's the kicker... Column #1 is a simple ID, which is used to identify the duplicate. Once dups are identified, I need to only keep the one... (2 Replies)
Discussion started by: kevinprood
2 Replies
DATEFMT_LOCALTIME(3)							 1						      DATEFMT_LOCALTIME(3)

IntlDateFormatter::localtime - Parse string to a field-based time value

	Object oriented style

SYNOPSIS
public array IntlDateFormatter::localtime (string $value, [int &$position]) DESCRIPTION
Procedural style array datefmt_localtime (IntlDateFormatter $fmt, string $value, [int &$position]) Converts string $value to a field-based time value ( an array of various fields), starting at $parse_pos and consuming as much of the input value as possible. PARAMETERS
o $fmt - The formatter resource o $value - string to convert to a time o $position - Position at which to start the parsing in $value (zero-based). If no error occurs before $value is consumed, $parse_pos will contain -1 otherwise it will contain the position at which parsing ended . If $parse_pos > strlen($value), the parse fails immedi- ately. RETURN VALUES
Localtime compatible array of integers : contains 24 hour clock value in tm_hour field EXAMPLES
Example #1 datefmt_localtime(3) example <?php $fmt = datefmt_create( 'en_US', IntlDateFormatter::FULL, IntlDateFormatter::FULL, 'America/Los_Angeles', IntlDateFormatter::GREGORIAN ); $arr = datefmt_localtime($fmt, 'Wednesday, December 31, 1969 4:00:00 PM PT', 0); echo 'First parsed output is '; if ($arr) { foreach ($arr as $key => $value) { echo "$key : $value , "; } } ?> Example #2 OO example <?php $fmt = new IntlDateFormatter( 'en_US', IntlDateFormatter::FULL, IntlDateFormatter::FULL, 'America/Los_Angeles', IntlDateFormatter::GREGORIAN ); $arr = $fmt->localtime('Wednesday, December 31, 1969 4:00:00 PM PT', 0); echo 'First parsed output is '; if ($arr) { foreach ($arr as $key => $value) { echo "$key : $value , "; } } ?> The above example will output: First parsed output is tm_sec : 0 , tm_min : 0 , tm_hour : 16 , tm_year : 1969 , tm_mday : 31 , tm_wday : 4 , tm_yday : 365 , tm_mon : 11 , tm_isdst : 0 , SEE ALSO
datefmt_create(3), datefmt_format(3), datefmt_parse(3), datefmt_get_error_code(3), datefmt_get_error_message(3). PHP Documentation Group DATEFMT_LOCALTIME(3)
All times are GMT -4. The time now is 12:35 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy