Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Remove somewhat Duplicate records from a flat file
Top Forums Shell Programming and Scripting Remove somewhat Duplicate records from a flat file Post 302560394 by durden_tyler on Thursday 29th of September 2011 04:01:51 PM
Old 09-29-2011
Code:
$
$ cat f29
1984/11/08            7 700000 123456789 2
1984/11/08 1941/05/19 7 700000 123456789 2
1999/06/08            8 800000 234567891 5
1999/06/08 1956/11/23 8 800000 234567891 5
$
$ # print only those lines that have 5 fields
$
$ perl -lane 'print if $#F==5' f29
1984/11/08 1941/05/19 7 700000 123456789 2
1999/06/08 1956/11/23 8 800000 234567891 5
$
$ # print only those lines that do have 2 dates at the beginning
$
$ perl -lne 'print if /^(\s*\d{4}\/\d\d\/\d\d){2}/' f29
1984/11/08 1941/05/19 7 700000 123456789 2
1999/06/08 1956/11/23 8 800000 234567891 5
$
$ # If the file is fixed-format, then you could try the following two approaches
$ # print only those lines whose column positions 12 through 21 are not blank spaces
$
$ perl -lne 'print if substr($_,11,10) !~ /^\s+$/' f29
1984/11/08 1941/05/19 7 700000 123456789 2
1999/06/08 1956/11/23 8 800000 234567891 5
$
$ # print only those lines whose column position 12 is not a blank space
$
$ perl -lne 'print if substr($_,11,1) ne " "' f29
1984/11/08 1941/05/19 7 700000 123456789 2
1999/06/08 1956/11/23 8 800000 234567891 5
$
$

tyler_durden
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Inserting records from flat file to db table

I have 20000 numbers present in a file in each line like 25663, 65465, 74579, 56446, .. .. I have created a table in db with single number column in it. create table testhari (no number(9)); I want to insert all these numbers into that table. how can i do it? can anybody please... (4 Replies)
Discussion started by: Hara
4 Replies

2. Shell Programming and Scripting

Remove all instances of duplicate records from the file

Hi experts, I am new to scripting. I have a requirement as below. File1: A|123|NAME1 A|123|NAME2 B|123|NAME3 File2: C|123|NAME4 C|123|NAME5 D|123|NAME6 1) I have 2 merge both the files. 2) need to do a sort ( key fields are first and second field) 3) remove all the instances... (3 Replies)
Discussion started by: vukkusila
3 Replies

3. Solaris

How to remove duplicate records with out sort

Can any one give me command How to delete duplicate records with out sort. Suppose if the records like below: 345,bcd,789 123,abc,456 234,abc,456 712,bcd,789 out tput should be 345,bcd,789 123,abc,456 Key for the records is 2nd and 3rd fields.fields are seperated by colon(,). (2 Replies)
Discussion started by: svenkatareddy
2 Replies

4. Shell Programming and Scripting

How to remove duplicate records with out sort

Can any one give me command How to delete duplicate records with out sort. Suppose if the records like below: 345,bcd,789 123,abc,456 234,abc,456 712,bcd,789 out tput should be 345,bcd,789 123,abc,456 Key for the records is 2nd and 3rd fields.fields are seperated by colon(,). (19 Replies)
Discussion started by: svenkatareddy
19 Replies

5. Shell Programming and Scripting

Sorting the records in the Flat file

Hi all, I am using this command "sort -d -u -k1 IMSTEST.74E -o tmp.txt" to the records in the flat. Can any tell me how to sort the file except first line in the file For ex: i/p First line: DXYZ Second line : jumy third : cmhk fourth : andy Output should... (5 Replies)
Discussion started by: sudhir_barker
5 Replies

6. Shell Programming and Scripting

How do I load records from table to a flat file!

Hi All, I need to load records from oracle table XYZ to a flat file say ABC.dat. could any one tell me how do i do this in UNXI, Regards Ann (1 Reply)
Discussion started by: Haque123
1 Replies

7. Shell Programming and Scripting

Remove duplicate records

I want to remove the records based on duplicate. I want to remove if two or more records exists with combination fields. Those records should not come once also file abc.txt ABC;123;XYB;HELLO; ABC;123;HKL;HELLO; CDE;123;LLKJ;HELLO; ABC;123;LSDK;HELLO; CDF;344;SLK;TEST key fields are... (7 Replies)
Discussion started by: svenkatareddy
7 Replies

8. Shell Programming and Scripting

Remove Duplicate Records

Hi frinds, Need your help. item , color ,desc ==== ======= ==== 1,red ,abc 1,red , a b c 2,blue,x 3,black,y 4,brown,xv 4,brown,x v 4,brown, x v I have to elemnet the duplicate rows on the basis of item. the final out put will be 1,red ,abc (6 Replies)
Discussion started by: imipsita.rath
6 Replies

9. Shell Programming and Scripting

Deleting duplicate records from file 1 if records from file 2 match

I have 2 files "File 1" is delimited by ";" and "File 2" is delimited by "|". File 1 below (3 record shown): Doc1;03/01/2012;New York;6 Main Street;Mr. Smith 1;Mr. Jones Doc2;03/01/2012;Syracuse;876 Broadway;John Davis;Barbara Lull Doc3;03/01/2012;Buffalo;779 Old Windy Road;Charles... (2 Replies)
Discussion started by: vestport
2 Replies

10. Shell Programming and Scripting

Remove duplicate records

Hi, i am working on a script that would remove records or lines in a flat file. The only difference in the file is the "NOT NULL" word. Please see below example of the input file. INPUT FILE:> CREATE a ( TRIAL_CLIENT NOT NULL VARCHAR2(60), TRIAL_FUND NOT NULL... (3 Replies)
Discussion started by: reignangel2003
3 Replies

LEARN ABOUT DEBIAN

getds

GETDS(1p)						User Contributed Perl Documentation						 GETDS(1p)

NAME

       getds - Create a DS record from DNSKEYing information

SYNOPSIS

	 getds <domain>

DESCRIPTION

       getds will create a DS record from DNSKEYs for the specified DNS domain.  It does this by converting DNSKEYs to DS records using the
       specified hashing algorithm.  The results can then be passed to upstream DNSSEC-supporting parents or to DLV registries.

       getds will also pull the parent's published DS records and compare them against the existing keys.  It will then list any DS records not
       published in the parent, as well as any DS records that are published in the parent but which don't match an existing key.

OPTIONS

       getds takes the following options:

       -a ALGORITHMS
       --hash-algorithm algorithm ALGORITHMS
	   This option specifies the hash algorithm to use when converting DNSKEYs to DS records.  It may be a comma-separated list if multiple
	   algorithms are desired.  The algorithms to choose from may be either SHA256 or SHA1.

	   The default is SHA256,SHA1

       -z
       --print-zsks
	   This option causes getds to print ZSK DS records, as well as KSK records.

       -p
       --dont-check-parent
	   Instructs getds to not check the records in the parent for their published DS records.

       -q
       --quiet
	   Produces quiet output with no explanatory headers.  In other words, it only prints the DS records generated from the DNSKEYs.

	   Note: Running with -q implies -p.

SECURITY CONSIDERATIONS

       By default, getds pulls data from the live DNS.	If your DNS resolver isn't configured so that this is pulled securely, then the results
       can't be trusted.

COPYRIGHT

       Copyright 2008-2012 SPARTA, Inc.  All rights reserved.  See the COPYING file included with the DNSSEC-Tools package for details.

AUTHOR

       Wes Hardaker, hardaker AT AT AT users.sourceforge.net

perl v5.14.2							    2012-06-21								 GETDS(1p)
All times are GMT -4. The time now is 12:38 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy