Sponsored Content
Top Forums Shell Programming and Scripting Find duplicate based on 'n' fields and mark the duplicate as 'D' Post 302593743 by machomaddy on Saturday 28th of January 2012 12:25:44 PM
Old 01-28-2012
oh!!! am sorry...I missed it...My bad. Yes the code works as what parthmittal says!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract duplicate fields in rows

I have a input file with formating: 6000000901 ;36200103 ;h3a01f496 ; 2000123605 ;36218982 ;heefa1328 ; 2000273132 ;36246985 ;h08c5cb71 ; 2000041207 ;36246985 ;heef75497 ; Each fields is seperated by semi-comma. Sometime, the second files is... (6 Replies)
Discussion started by: anhtt
6 Replies

2. Shell Programming and Scripting

compare fields in a file with duplicate records

Hi: I've been searching the net but didnt find a clue. I have a file in which, for some records, some fields coincide. I want to compare one (or more) of the dissimilar fields and retain the one record that fulfills a certain condition. For example, on this file: 99 TR 1991 5 06 ... (1 Reply)
Discussion started by: rleal
1 Replies

3. Shell Programming and Scripting

awk 2 fields duplicate and 1 different

I have file that I need to remove the duplicates. The problem is, I need to only keep the one which has a unique 3rd field. Here is a sample file: xxx.xxx:x:CISCO1.CLEVE61W:ERIE.NET:x:x:x:x: xxx.xxx:x:CISCO2.CLEVE62W:OHIO.NET:x:x:x:x: xxx.xxx:x:CISCO2.CLEVE62W:NORTH.NET:x:x:x:x:... (1 Reply)
Discussion started by: numele
1 Replies

4. Shell Programming and Scripting

Filter or remove duplicate block of text without distinguishing marks or fields

Hello, Although I have found similar questions, I could not find advice that could help with our problem. The issue: We have several hundreds text files containing repeated blocks of text (I guess back at the time they were prepared like that to optmize printing). The block of texts... (13 Replies)
Discussion started by: samask
13 Replies

5. Shell Programming and Scripting

Remove duplicate based on Group

Hi, How can I remove duplicates from a file based on group on other column? for example: Test1|Test2|Test3|Test4|Test5 Test1|Test6|Test7|Test8|Test5 Test1|Test9|Test10|Test11|Test12 Test1|Test13|Test14|Test15|Test16 Test17|Test18|Test19|Test20|Test21 Test17|Test22|Test23|Test24|Test5 ... (2 Replies)
Discussion started by: yale_work
2 Replies

6. Shell Programming and Scripting

Join fields from files with duplicate lines

I have two files, file1.txt: 1 abc 2 def 2 dgh 3 ijk 4 lmn file2.txt 1 opq 2 rst 3 uvw My desired output is: 1 abc opq 2 def rst 2 dgh rst 3 ijk uvw (2 Replies)
Discussion started by: xan.amini
2 Replies

7. Shell Programming and Scripting

How To Remove Duplicate Based on the Value?

Hi , Some time i got duplicated value in my files , bundle_identifier= B Sometext=ABC bundle_identifier= A bundle_unit=500 Sometext123=ABCD bundle_unit=400 i need to check if there is a duplicated values or not if yes , i need to check if the value is A or B when Bundle_Identified ,... (2 Replies)
Discussion started by: OTNA
2 Replies

8. Shell Programming and Scripting

Remove duplicate lines from file based on fields

Dear community, I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns The data are like this: Region 23/11/2014 09:11:36 41752 Medio 23/11/2014 03:11:38 4132 Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies

9. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies

10. UNIX for Beginners Questions & Answers

Discarding records with duplicate fields

Hi, My input looks like this (tab-delimited): grp1 name2 firstname M 55 item1 item1.0 grp1 name2 firstname F 55 item1 item1.0 grp2 name1 firstname M 55 item1 item1.0 grp2 name2 firstname M 55 item1 item1.0 Using awk, I am trying to discard the records with common fields 2, 4, 5, 6, 7... (4 Replies)
Discussion started by: beca123456
4 Replies
lp-set-dup(1)						      General Commands Manual						     lp-set-dup(1)

NAME
lp-set-dup - mark one or more bugs as duplicate of another bug SYNOPSIS
lp-set-dup [-f] <main bug> <duplicate bug> [<duplicate bug> ...] lp-set-dup --help DESCRIPTION
lp-set-dup allow to easily mark one or more bug as duplicate of another bug. It checks for permission to operate on a given bug first, then perform required tasks on Launchpad. OPTIONS
Listed below are the command line options for lp-set-dup: -h, --help Display a help message and exit. -f Skip confirmation prompt. -l INSTANCE, --lpinstance=INSTANCE Use the specified instance of Launchpad (e.g. "staging"), instead of the default of "production". --no-conf Do not read any configuration files, or configuration from environment variables. ENVIRONMENT
All of the CONFIGURATION VARIABLES below are also supported as environment variables. Variables in the environment take precedence to those in configuration files. CONFIGURATION VARIABLES
The following variables can be set in the environment or in ubuntu-dev-tools(5) configuration files. In each case, the script-specific variable takes precedence over the package-wide variable. LP_SET_DUP_LPINSTANCE, UBUNTUTOOLS_LPINSTANCE The default value for --lpinstance. SEE ALSO
ubuntu-dev-tools(5) AUTHORS
lp-set-dup was written by Loic Minier <lool@dooz.org>, and this manual page was written by Luca Falavigna <dktrkranz@debian.org>. Both are released under the terms of the GNU General Public License, version 2. lptools March 6 2010 lp-set-dup(1)
All times are GMT -4. The time now is 08:05 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy