Sponsored Content
Top Forums Shell Programming and Scripting Find duplicate based on 'n' fields and mark the duplicate as 'D' Post 302593722 by machomaddy on Saturday 28th of January 2012 08:32:47 AM
Old 01-28-2012
Thanks, rdcwayx!! Smilie Could please explain the code? It will be very helpful
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract duplicate fields in rows

I have a input file with formating: 6000000901 ;36200103 ;h3a01f496 ; 2000123605 ;36218982 ;heefa1328 ; 2000273132 ;36246985 ;h08c5cb71 ; 2000041207 ;36246985 ;heef75497 ; Each fields is seperated by semi-comma. Sometime, the second files is... (6 Replies)
Discussion started by: anhtt
6 Replies

2. Shell Programming and Scripting

compare fields in a file with duplicate records

Hi: I've been searching the net but didnt find a clue. I have a file in which, for some records, some fields coincide. I want to compare one (or more) of the dissimilar fields and retain the one record that fulfills a certain condition. For example, on this file: 99 TR 1991 5 06 ... (1 Reply)
Discussion started by: rleal
1 Replies

3. Shell Programming and Scripting

awk 2 fields duplicate and 1 different

I have file that I need to remove the duplicates. The problem is, I need to only keep the one which has a unique 3rd field. Here is a sample file: xxx.xxx:x:CISCO1.CLEVE61W:ERIE.NET:x:x:x:x: xxx.xxx:x:CISCO2.CLEVE62W:OHIO.NET:x:x:x:x: xxx.xxx:x:CISCO2.CLEVE62W:NORTH.NET:x:x:x:x:... (1 Reply)
Discussion started by: numele
1 Replies

4. Shell Programming and Scripting

Filter or remove duplicate block of text without distinguishing marks or fields

Hello, Although I have found similar questions, I could not find advice that could help with our problem. The issue: We have several hundreds text files containing repeated blocks of text (I guess back at the time they were prepared like that to optmize printing). The block of texts... (13 Replies)
Discussion started by: samask
13 Replies

5. Shell Programming and Scripting

Remove duplicate based on Group

Hi, How can I remove duplicates from a file based on group on other column? for example: Test1|Test2|Test3|Test4|Test5 Test1|Test6|Test7|Test8|Test5 Test1|Test9|Test10|Test11|Test12 Test1|Test13|Test14|Test15|Test16 Test17|Test18|Test19|Test20|Test21 Test17|Test22|Test23|Test24|Test5 ... (2 Replies)
Discussion started by: yale_work
2 Replies

6. Shell Programming and Scripting

Join fields from files with duplicate lines

I have two files, file1.txt: 1 abc 2 def 2 dgh 3 ijk 4 lmn file2.txt 1 opq 2 rst 3 uvw My desired output is: 1 abc opq 2 def rst 2 dgh rst 3 ijk uvw (2 Replies)
Discussion started by: xan.amini
2 Replies

7. Shell Programming and Scripting

How To Remove Duplicate Based on the Value?

Hi , Some time i got duplicated value in my files , bundle_identifier= B Sometext=ABC bundle_identifier= A bundle_unit=500 Sometext123=ABCD bundle_unit=400 i need to check if there is a duplicated values or not if yes , i need to check if the value is A or B when Bundle_Identified ,... (2 Replies)
Discussion started by: OTNA
2 Replies

8. Shell Programming and Scripting

Remove duplicate lines from file based on fields

Dear community, I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns The data are like this: Region 23/11/2014 09:11:36 41752 Medio 23/11/2014 03:11:38 4132 Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies

9. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies

10. UNIX for Beginners Questions & Answers

Discarding records with duplicate fields

Hi, My input looks like this (tab-delimited): grp1 name2 firstname M 55 item1 item1.0 grp1 name2 firstname F 55 item1 item1.0 grp2 name1 firstname M 55 item1 item1.0 grp2 name2 firstname M 55 item1 item1.0 Using awk, I am trying to discard the records with common fields 2, 4, 5, 6, 7... (4 Replies)
Discussion started by: beca123456
4 Replies
EDOS-DEBCHECK(1)					      General Commands Manual						  EDOS-DEBCHECK(1)

NAME
Edos-debcheck - Check satisfiability of Debian package dependencies SYNOPSIS
edos-debcheck [option] ... [package] ... DESCRIPTION
edos-debcheck reads from standard input a set of Debian package descriptions, each of which is in the format of deb-control(5). For instance, the Packages files as found on a Debian mirror server, or in the directory /var/lib/apt/lists/ of a Debian system, are suitable as input to edos-debcheck. Packages may appear with multiple versions in this set. We say that a package is identified in this set by the pair consisting of the pack- age name and of the package version. A package (called goal) is called installable with respect to a set of packages if there exists a subset of the packages that - contains at most one version for each package name - contains the goal package (in any version, or a specific version) - all packages of the subset have their dependencies and conflicts satisfied in this subset. Pre-depends are treated like Depends, and Breaks are treated like Conflicts. Debcheck checks whether every package given as argument is installable with respect to the input set of packages. Specific versions can be specified by following the package name with the sign '=' and the version of the package (for instance, 'xemacs21=21.4.17-1'). If no pack- age name is given as argument then all packages of the input set are checked for satisfiability. The constraint solving algorithm is complete, that is it finds a solution whenever there exists one, even for multiple disjunctive depen- dencies and deep package conflicts. This problem is computationally infeasible in theory (that is, NP-complete), but can in practice be solved very efficiently for Packages files occurring actually in Debian. OPTIONS
-i file Specify an additional input file containing stanzas in deb-control(5) format. These packages are checked for installability the same way as packages read from standard input. -I file Specify an additional input file containing stanzas in deb-control(5) format. These packages that are not checked for installabil- ity but only used for resolving dependencies. -explain Explain the results -rules Print generated rules -failures Only show failures -successes Only show successes -checkonly list where list is a comma-separated list of package names. Check only for installability of packages with name in list (of any avail- able version). This means that if a package does not exist in the input pool then the test passes (since all available versions are in that case installable). However, a warning is issued in case a package mentionend in list is not available. In the context of this option, a package of the form src:p expands to the list of all packages that stem from the source package p. -quiet Supress warnings and progress/timing messages -xml Output results in XML format -help, --help Display this list of options EXIT CODE
The exit code is 0 when all packages that were asked to be checked (either explicitly with the -checkonly option, or because they exist in the package pool file) are available and found to be installable, it is 1 when a least one of these packages is available but not instal- lable, and it is 2 when at least one of the packages is not available but all available packages are found to be installable. Hence, the exit code 2 can occur only when the -checkonly option is used. EXAMPLE
Check which packages in a particular distribution are not installable and why: edos-debcheck -failures -explain < Packages where Packages is the file pertaining to that distribution, as for instance found in the directory /var/lib/apt/lists. Check whether the xemacs21 and debhelper packages are installable in a distribution described by the file Packages: edos-debcheck -explain -checkonly xemacs21,debhelper < Packages Check whether the package bibtex2html, and all binary packages stemming from the ocaml source package, are installable edos-debcheck -explain -checkonly bibtex2html,src:ocaml < Packages AUTHOR
Edos-debcheck has been written by Jerome Vouillon for the EDOS project. This man-page has been compiled by Ralf Treinen. SEE ALSO
deb-control(5) <http://www.edos-project.org> is the home page of the EDOS project. edos-rpmcheck(1) is the analogous tool for rpm packages. EDOS
2009-08-18 EDOS-DEBCHECK(1)
All times are GMT -4. The time now is 04:50 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy