Sponsored Content
Full Discussion: Remove Duplicate Records
Top Forums Shell Programming and Scripting Remove Duplicate Records Post 302401056 by murugaperumal on Thursday 4th of March 2010 10:19:02 PM
Old 03-04-2010
You can use the following code using Perl script

Code:
 
open (FH,"<file") or die "Can't open";
my $val=1;
my $value;
 while($value=<FH>)
 {
   if($value=~/^$val/)
   {
       $val=$val+1;
       print "$value";
   }
 }

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Records Duplicate

Hi Everyone, I have a flat file of 1000 unique records like following : For eg Andy,Flower,201-987-0000,12/23/01 Andrew,Smith,101-387-3400,11/12/01 Ani,Ross,401-757-8640,10/4/01 Rich,Finny,245-308-0000,2/27/06 Craig,Ford,842-094-8740,1/3/04 . . . . . . Now I want to duplicate... (9 Replies)
Discussion started by: ganesh123
9 Replies

2. Shell Programming and Scripting

Remove all instances of duplicate records from the file

Hi experts, I am new to scripting. I have a requirement as below. File1: A|123|NAME1 A|123|NAME2 B|123|NAME3 File2: C|123|NAME4 C|123|NAME5 D|123|NAME6 1) I have 2 merge both the files. 2) need to do a sort ( key fields are first and second field) 3) remove all the instances... (3 Replies)
Discussion started by: vukkusila
3 Replies

3. Solaris

How to remove duplicate records with out sort

Can any one give me command How to delete duplicate records with out sort. Suppose if the records like below: 345,bcd,789 123,abc,456 234,abc,456 712,bcd,789 out tput should be 345,bcd,789 123,abc,456 Key for the records is 2nd and 3rd fields.fields are seperated by colon(,). (2 Replies)
Discussion started by: svenkatareddy
2 Replies

4. Shell Programming and Scripting

How to remove duplicate records with out sort

Can any one give me command How to delete duplicate records with out sort. Suppose if the records like below: 345,bcd,789 123,abc,456 234,abc,456 712,bcd,789 out tput should be 345,bcd,789 123,abc,456 Key for the records is 2nd and 3rd fields.fields are seperated by colon(,). (19 Replies)
Discussion started by: svenkatareddy
19 Replies

5. Shell Programming and Scripting

Remove duplicate records

I want to remove the records based on duplicate. I want to remove if two or more records exists with combination fields. Those records should not come once also file abc.txt ABC;123;XYB;HELLO; ABC;123;HKL;HELLO; CDE;123;LLKJ;HELLO; ABC;123;LSDK;HELLO; CDF;344;SLK;TEST key fields are... (7 Replies)
Discussion started by: svenkatareddy
7 Replies

6. Shell Programming and Scripting

Remove somewhat Duplicate records from a flat file

I have a flat file that contains records similar to the following two lines; 1984/11/08 7 700000 123456789 2 1984/11/08 1941/05/19 7 700000 123456789 2 The 123456789 2 represents an account number, this is how I identify the duplicate record. The ### signs represent... (4 Replies)
Discussion started by: jolney
4 Replies

7. UNIX for Dummies Questions & Answers

Need to keep duplicate records

Consider my input is 10 10 20 then, uniq -u will give 20 and uniq -dwill return 10. But i need the output as , 10 10 How we can achieve this? Thanks (4 Replies)
Discussion started by: pandeesh
4 Replies

8. Shell Programming and Scripting

Deleting duplicate records from file 1 if records from file 2 match

I have 2 files "File 1" is delimited by ";" and "File 2" is delimited by "|". File 1 below (3 record shown): Doc1;03/01/2012;New York;6 Main Street;Mr. Smith 1;Mr. Jones Doc2;03/01/2012;Syracuse;876 Broadway;John Davis;Barbara Lull Doc3;03/01/2012;Buffalo;779 Old Windy Road;Charles... (2 Replies)
Discussion started by: vestport
2 Replies

9. Shell Programming and Scripting

Remove duplicate records

Hi, i am working on a script that would remove records or lines in a flat file. The only difference in the file is the "NOT NULL" word. Please see below example of the input file. INPUT FILE:> CREATE a ( TRIAL_CLIENT NOT NULL VARCHAR2(60), TRIAL_FUND NOT NULL... (3 Replies)
Discussion started by: reignangel2003
3 Replies

10. Shell Programming and Scripting

Duplicate records

Gents, Please give a help file --BAD STATUS NOT RESHOOTED-- *** VP 41255/51341 in sw 2973 *** VP 41679/51521 in sw 2973 *** VP 41687/51653 in sw 2973 *** VP 41719/51629 in sw 2976 --BAD COG NOT RESHOOTED-- *** VP 41689/51497 in sw 2974 *** VP 41699/51677 in sw 2974 *** VP... (18 Replies)
Discussion started by: jiam912
18 Replies
catch(n)						       Tcl Built-In Commands							  catch(n)

__________________________________________________________________________________________________________________________________________________

NAME
catch - Evaluate script and trap exceptional returns SYNOPSIS
catch script ?resultVarName? ?optionsVarName? _________________________________________________________________ DESCRIPTION
The catch command may be used to prevent errors from aborting command interpretation. The catch command calls the Tcl interpreter recur- sively to execute script, and always returns without raising an error, regardless of any errors that might occur while executing script. If script raises an error, catch will return a non-zero integer value corresponding to the exceptional return code returned by evaluation of script. Tcl defines the normal return code from script evaluation to be zero (0), or TCL_OK. Tcl also defines four exceptional return codes: 1 (TCL_ERROR), 2 (TCL_RETURN), 3 (TCL_BREAK), and 4 (TCL_CONTINUE). Errors during evaluation of a script are indicated by a return code of TCL_ERROR. The other exceptional return codes are returned by the return, break, and continue commands and in other special situa- tions as documented. Tcl packages can define new commands that return other integer values as return codes as well, and scripts that make use of the return -code command can also have return codes other than the five defined by Tcl. If the resultVarName argument is given, then the variable it names is set to the result of the script evaluation. When the return code from the script is 1 (TCL_ERROR), the value stored in resultVarName is an error message. When the return code from the script is 0 (TCL_OK), the value stored in resultVarName is the value returned from script. If the optionsVarName argument is given, then the variable it names is set to a dictionary of return options returned by evaluation of | script. Tcl specifies two entries that are always defined in the dictionary: -code and -level. When the return code from evaluation of | script is not TCL_RETURN, the value of the -level entry will be 0, and the value of the -code entry will be the same as the return code. | Only when the return code is TCL_RETURN will the values of the -level and -code entries be something else, as further described in the doc- | umentation for the return command. | When the return code from evaluation of script is TCL_ERROR, three additional entries are defined in the dictionary of return options | stored in optionsVarName: -errorinfo, -errorcode, and -errorline. The value of the -errorinfo entry is a formatted stack trace containing | more information about the context in which the error happened. The formatted stack trace is meant to be read by a person. The value of | the -errorcode entry is additional information about the error stored as a list. The -errorcode value is meant to be further processed by | programs, and may not be particularly readable by people. The value of the -errorline entry is an integer indicating which line of script | was being evaluated when the error occurred. The values of the -errorinfo and -errorcode entries of the most recent error are also avail- | able as values of the global variables ::errorInfo and ::errorCode respectively. | Tcl packages may provide commands that set other entries in the dictionary of return options, and the return command may be used by scripts | to set return options in addition to those defined above. EXAMPLES
The catch command may be used in an if to branch based on the success of a script. if { [catch {open $someFile w} fid] } { puts stderr "Could not open $someFile for writing $fid" exit 1 } There are more complex examples of catch usage in the documentation for the return command. SEE ALSO
break(n), continue(n), dict(n), error(n), return(n), tclvars(n) KEYWORDS
catch, error Tcl 8.5 catch(n)
All times are GMT -4. The time now is 08:06 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy