Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Need a script or one-liner to purge lines from a file.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need a script or one-liner to purge lines from a file.
# 1  
Old 10-12-2010
Need a script or one-liner to purge lines from a file.
i all.

This one sounds so simple, but I can't get it to work. I need to delete lines with certain keywords from a file.

I have a file called defaultRules, with keywords:

Code:
IPSEC_AH
IKE_UDP
IPMP_TEST_IFACE2

Then, I have another file called rules.txt with some rules:

Code:
'344.','IPSEC_ESP','','','','','1.','0.','0.','0.','0.','5.','1.','Allow IPSec ESP Packets on all IP interfaces'
'345.','IPSEC_AH','','','','','1.','0.','0.','0.','0.','6.','1.','Allow IPSec AH Packets on all IP interfaces'
'346.','IKE_TCP','','','','','1.','500.','0.','0.','0.','3.','1.','Allow IKE Negotiation on all IP interfaces'
'347.','IKE_UDP','','','','','1.','500.','0.','0.','0.','2.','1.','Allow IKE Negotiation on all IP interfaces'
'348.','CLUSTER_IC_NODE1','clusternode1-priv','','','','1.','0.','0.','0.','0.','4.','1.','Allow cluster inter-connect packets from node 1'
'349.','CLUSTER_IC_NODE2','clusternode2-priv','','','','1.','0.','0.','0.','0.','4.','1.','Allow cluster inter-connect packets from node 2'
'350.','ICMP_NODE_IP','ipmp_node_alias','','','','1.','0.','0.','0.','0.','1.','1.','Allow ICMP to Node IP'
'351.','ICMP_BMS','ipmp_bms','','','','1.','0.','0.','0.','0.','1.','1.','Allow ICMP to BMS IP'
'352.','IPMP_TEST_IFACE1','ipmp_test_alias_1','','','','1.','0.','0.','0.','0.','1.','1.','Allow IPMP Test Packets on 1st test interface'
'353.','IPMP_TEST_IFACE2','ipmp_test_alias_2','','','','1.','0.','0.','0.','0.','1.','1.','Allow IPMP Test Packets on 2nd test interface'

The objective is to remove all lines in rules.txt that contain the keywords in defaultRules, so I tried this by first renaming the keywords to 'unused' and then removing all lines that contain 'unused':

Code:
for a in `cat defaultRules` ; do sed -e 's/$a/unused/' rules.txt | grep -v unused > rules.out ; done

It did nothing, rules.txt and rules.out are identical. Smilie

Any ideas?

Thanks in advance,
BRH
# 2  
Old 10-12-2010
Code:
grep -vf defaultRules rules.txt

# 3  
Old 10-12-2010
I've only used Perl for scripting, so I'll give you some advice for Perl.

Read rules.txt and read each line.
If the line has any of those keywords in defaultRules, then do nothing.
Else, put that line into an output file rules.out

Code:
open(INPUTFILE, "<rules.txt");
open(OUTPUTFILE, ">rules.out");

while (<INPUTFILE>) {
    my $line = $_;
    if ($line =~ m/IPSEC_AH/i || $line =~ m/IKE_UDP/i || $line =~ m/IPMP_TEST_IFACE2/i) {
             print OUTPUTFILE "$line";
    }
}

close(INPUTFILE);
close(OUTPUTFILE);

# 4  
Old 10-12-2010
Quote:
Originally Posted by danmero
Code:
grep -vf defaultRules rules.txt

Hi danmero, which grep are you using, I'm on a Solaris 10 system:

Code:
/usr/bin/grep -vf defaultRules rules.txt
/usr/bin/grep: illegal option -- f
Usage: grep -hblcnsviw pattern file . . .

Regards,
BRH

---------- Post updated at 03:58 PM ---------- Previous update was at 03:56 PM ----------
Quote:
Originally Posted by kooshi
I've only used Perl for scripting, so I'll give you some advice for Perl.

Read rules.txt and read each line.
If the line has any of those keywords in defaultRules, then do nothing.
Else, put that line into an output file rules.out

Code:
open(INPUTFILE, "<rules.txt");
open(OUTPUTFILE, ">rules.out");

while (<INPUTFILE>) {
    my $line = $_;
    if ($line =~ m/IPSEC_AH/i || $line =~ m/IKE_UDP/i || $line =~ m/IPMP_TEST_IFACE2/i) {
             print OUTPUTFILE "$line";
    }
}

close(INPUTFILE);
close(OUTPUTFILE);

Thanks for the reply, kooshi, however, this won't workin my case. The full defaultRules is over 100 lines.

Regards,
BRH
# 5  
Old 10-12-2010
Try /usr/xpg4/bin/grep on Solaris ?!
# 6  
Old 10-12-2010
Quote:
Originally Posted by danmero
Try /usr/xpg4/bin/grep on Solaris ?!
HI dan, no luck with this one either, it didn't complain, but it did nothing.

Regards,
BRH
# 7  
Old 10-12-2010
is there one in /usr/ucb/grep ? or give a try with egrep ?
Last edited by ctsgnb; 10-12-2010 at 06:25 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed one liner to Delete blank lines - Help required

Hi, cat test.txt BlankLine BlankLine BlankLine BlankLine ello hi helo BlankLine BlankLine heylo BlankLine BlankLine BlankLine done BlankLine BlankLine BlankLine (1 Reply)
Discussion started by: TomG
1 Replies

2. Shell Programming and Scripting

Perl one liner to wrap comment lines

Greetings fellow scripters. I find myself editing multiple files, sometimes with the same bits of information. My bash script, a changelog, and a plist file (OS X). Once I realized this, I thought why not script part of this process (and so it begins). In any case, I've solved several of the... (1 Reply)
Discussion started by: reid
1 Replies

3. Shell Programming and Scripting

To create a script and schedule which purge 30 days old files

Hi Friends, Very new in Unix and i got a requirement like writing a script and schedule it, so that it removes 30 days old files from all the log locations of a unix box. Suppose i have a unix server ltbamdev1 and in this server i have a mount point opt/bam. In this mount point i have 3... (1 Reply)
Discussion started by: duos
1 Replies

4. Shell Programming and Scripting

Search & Replace regex Perl one liner to AWK one liner

Thanks for giving your time and effort to answer questions and helping newbies like me understand awk. I have a huge file, millions of lines, so perl takes quite a bit of time, I'd like to convert these perl one liners to awk. Basically I'd like all lines with ISA sandwiched between... (9 Replies)
Discussion started by: verge
9 Replies

5. UNIX for Advanced & Expert Users

Purge MAil file

Hi, merry christmas. on AIX 6.1, the file /var/spool/mail/user, should/can be purged manually ? Any commande line to purge it ? Thanks. (2 Replies)
Discussion started by: big123456
2 Replies

6. Shell Programming and Scripting

Shell script to check the files hourly and purge

I'm new to shell scripting... i have been given a task.. can any one help in this regard.... 1) Check hourly for files in <destination-path><destination-file-template><destination-file-suffix> for files older than <destination-file-retention> days and purge. It should then check... (1 Reply)
Discussion started by: satishpabba
1 Replies

7. Shell Programming and Scripting

Sed one-liner to print specific lines?

I need to print specific lines from a file, say 2-5, 8, 12-15, 17, 19, 21-27. How do I achieve this? (2 Replies)
Discussion started by: Ilja
2 Replies

8. Shell Programming and Scripting

script to archive and purge

Hi, I am writing a shell script for archive data from a table. The design is as follows. Step 1: Execute the select query and extract the data into a text file. Step 2: The primary key for this table is TRACKING_NUM, TRACKING_NUM_SUFFIX, TIMESTAMP_UPDATED. So These three fields will be read... (1 Reply)
Discussion started by: kmanivan82
1 Replies

9. Shell Programming and Scripting

shell script for archive purge

I am writing a shell script for Archive Purge for the table having rows < 1 year. The shell script has to extract the rows from the table and write those extracted rows to a text file. Then from the text file, each rows will be read and deleted by means of delete query one by one. The fields will... (5 Replies)
Discussion started by: regnumber
5 Replies

10. Shell Programming and Scripting

script for purge

Hi , I want to purge 7 days older data from a list of data sorted on date in a log file... Can anyone provide me with the shell script for the same.. Thanks, Jaz (1 Reply)
Discussion started by: JP003
1 Replies
Login or Register to Ask a Question