How to remove the specific lines from file using perl


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to remove the specific lines from file using perl
# 1  
Old 06-04-2008
How to remove the specific lines from file using perl

Can anyone tell me what could be the solution to following :

I have one .txt file which contains some "seed" information. This seed may appear multiple time in the file so what I want do is if this seed appears again in the file then that line should be removed.

Please provide the script code also:

Here in I am attaching the text file.
Thanks in advance..
# 2  
Old 06-05-2008
So if the tenth field is identical to one we have seen before, remove the whole line?

Code:
perl -ane 'print unless $seen{$F[9]}++' my_log.txt

(Perl arrays are numbered from zero, so $F[9] is the tenth field. The -a option causes Perl to split the input line into the array @F, somewhat similarly to how awk works.)

The input line is printed unless the hash %seen already has an entry for the tenth field. We also add one to its value, which causes it to be set (to one) if it didn't exist before. Thus, the %seen value for the current seed will be set the next time it is encountered.

For the sample file you posted, this reduces 1,274 lines to just 72 lines.
# 3  
Old 06-05-2008
if you want to remove the lines which is having the word seed, execute the following command :

grep -v "seed" my_file.txt
# 4  
Old 06-05-2008
this is the same question posted here
# 5  
Old 06-11-2008
Hi era,

Thank you very much for your help.

--Dipak
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove duplicate consecutive lines with specific string

Hello, I'm trying to remove the duplicate consecutive lines with specific string "WARNING". File.txt abc; WARNING 2345 WARNING 2345 WARNING 2345 WARNING 2345 WARNING 2345 bcd; abc; 123 123 123 WARNING 1234 WARNING 2345 WARNING 2345 efgh; (6 Replies)
Discussion started by: Mannu2525
6 Replies

2. Shell Programming and Scripting

Remove lines matching a substring in a specific column

Dear group, I have following input text file: Brit 2016 11 18 12 00 10 1.485,00 EUR Brit 2016 11 18 12 00 10 142,64 EUR Brit 2016 11 18 12 00 10 19,80 EUR Brit 2016 11 18 12 00 10 545,00 EUR Brit 2016 11 18 12 00 10 6.450,00 EUR... (3 Replies)
Discussion started by: gfhsd
3 Replies

3. Shell Programming and Scripting

awk to remove lines in file if specific field matches

I am trying to remove lines in the target.txt file if $5 before the - in that file matches sorted_list. I have tried grep and awk. Thank you :). grep grep -v -F -f targets.bed sort_list grep -vFf sort_list targets awk awk -F, ' > FILENAME == ARGV {to_remove=1; next} > ! ($5 in... (2 Replies)
Discussion started by: cmccabe
2 Replies

4. Shell Programming and Scripting

NAWK to remove lines that matches a specific pattern

Hi, I have requirement that I need to split my input file into two files based on a search pattern "abc" For eg. my input file has below content abc defgh zyx I need file 1 with abc and file2 with defgh zyx I can use grep command to acheive this. But with grep I need... (8 Replies)
Discussion started by: sbhuvana20
8 Replies

5. Shell Programming and Scripting

Sed or Awk to remove specific lines

I have searched the forum for this - forgive me if I missed a previous post. I have the following file: blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah alter table "informix".esc_acct add constraint (foreign key (fi_id) references "informix".fi ... (5 Replies)
Discussion started by: Shoeless_Mike
5 Replies

6. Shell Programming and Scripting

how to remove specific lines from a file (reprise)

Hello, I've to change the shell in /etc/passwd for some users . I've the list of users but I'm not able to modify the file with scripting . I'm working on a Sol10 . Can anyone help me ? tnks (7 Replies)
Discussion started by: gogol_bordello
7 Replies

7. Shell Programming and Scripting

How to remove the lines from file using perl

Can anyone tell me what could be the solution to following : I have one .txt file which contains some seed information. This seed may appear multiple time in the file so what I want do is if this seed appears again in the file then that line should be removed. here is the contents of .txt... (5 Replies)
Discussion started by: dipakg
5 Replies

8. Shell Programming and Scripting

remove specific lines from flat file using perl

Hi, Here is wat im looking for.. i have a flat file which looks like this.. 00 * * * * .. .. * * text text text COL1 COL2 ----- ----- 1 a (12 Replies)
Discussion started by: meghana
12 Replies

9. Shell Programming and Scripting

how to remove specific lines from a file

When restoring a file in my uninstall program I need to remove the lines I added to a file during the install. In between the file can be modified by the users. Assume file1 is as follow: xxx str2 xxxx ..... ...The Following lines containing str* have to be removed... xxx str1 xxxx xxx ... (17 Replies)
Discussion started by: bluemoon1
17 Replies

10. Shell Programming and Scripting

remove specific lines from a file

Hi there I have a file with a variable amount of rows but the 45th, 46th and 47th charachter of each line is the status field which is a three digit code ie 001, 002, 003 etc. My question is this..I need to strip all the records/lines with 002's out of the file completely and put them into... (14 Replies)
Discussion started by: hcclnoodles
14 Replies
Login or Register to Ask a Question