Shell Programming and Scripting

Hi there,
I need help about using sed. Iam using sed to delete and print lines that match the port number as listed in sedfile. I am using -d and -p command for delete match port and print them respectively. However, the output is not synchonize where the total deleted lines is not similar with the printed lines. I found that sed print lines that match port 4513 although I just want exactly the 4513.
Here is my code

Code:

sed -n -f print.sed tcpfile >> tcpfile.out
sed -f delete.sed tcpfile >>tcpfile.delete

Example of the print.sed

Code:

/451/p
/46[6-7][1-4]/p
...

Example lines in tcpfile

Code:

src=78.136.25.50 dst=192.168.133.202 bytes=610 proto=6 sport=80 dport=3159
src=89.1.65.142 dst=192.168.133.202 bytes=184 proto=6 sport=43954 dport=1153
src=72.14.217.189 dst=192.168.133.202 bytes=270 proto=6 sport=80 dport=3145
src=78.136.25.50 dst=192.168.133.202 bytes=610 proto=6 sport=80 dport=3158
src=208.70.8.23 dst=192.168.133.202 bytes=2472 proto=6 sport=80 dport=1447
src=208.70.8.23 dst=192.168.133.202 bytes=2526 proto=6 sport=80 dport=1811

Really need help. Thanks

Please be a little more clear as to your expectations.

The way you have it written, the "print" operation should print every line that matches a pattern in file print.sed.

Without knowing what is in file delete.sed, it's hard to say what result you will get. If file delete.sed has something like /pattern/p then it will print every line but those with matching patterns will be printed twice. If, however, file delete.sed has something like /pattern/d then it will print every line that does NOT match the pattern. You say you are using -d so that would seem to imply the second case. If your delete.sed file has /pattern/d then what you will have in tcpfile.delete is everything that remains after having already deleted the matching lines. If you want tcpfile.delete to contain the lines that should be deleted, use !d.


Are you saying that you only want it to print the port number and not the whole line? I'm not clear what you're saying here. If all you want is the port number, and not the entire line, you need more than sed. You also need to use a little caution the way you are specifying your port numbers. The pattern /451/ will match every port that has 451 anywhere in it (451, 1451, 4513, etc). If you make your intentions clearer, it will be easier to help.

Thank you for reply and sorry for my unclear explanation.

Actually, I wanted to have both output which have the only match lines and deleted lines for specified port number in print.sed and delete.sed copied into different files. Port numbers defined in delete.sed and print.sed are similar and only different in term of pattern/d and pattern/p. I tried to use pattern/!d instead of /p but I could not get the output. I could not open the output file and make my disk usage reach 100%.
Apart from that, the problem is the output from both command. It should have similar lines copied and lines deleted but as far as I see, it is not. Then, I figure out that the problem is from the way I defined the port number.

Code:

How to define the port number if I want only lines that match 451, 4661,4662 and so on.
/port=451/p
/port=46[6-7][1-4]/p
...

I hope that I have make a clearer explanation. Thanks.

Sorry for the delay in getting back to you. Work 4-on/4-off and some days I just choose to leave the computer off.


Ok, you want your tcpfile.delete to actually contain the lines that are (to be) deleted. That's the problem. For this to work, using the !d for the delete, you would have to build a single regular expression that matches every line to be deleted. That is no small task.

A couple of suggestions.

1. Since nothing is really being deleted, and tcpfile.delete really just identifies those lines that need to be deleted, you could use the /p to identify them.

Assuming you're not doing it this way because you want to use the /d, then I have two other suggestions.

2. If you use /d in delete.sed, tcpfile.delete ends up containing the contents of tcpfile minus those lines which would be deleted by the delete expressions. You could then use comm, or some other utility, to compare tcpfile and tcpfile.delete to produce a list of the lines which were deleted. I ran a quick test with comm and, on my system, anyway, it was producing one extra line of output that it shouldn't have been.

3. To generate a file tcpfile.delete that contains the (to be) deleted lines, and using the !d form for delete, you could use multiple delete.sed files.

Code:

 
delete1.sed
/=315[89]$/!d
 
delete2.sed
/=1447$/!d
 
for i in delete*.sed;do sed -f $i tcpfile >> tcpfile.delete;done

There are probably other ways, as well. Just need to decide what's easiest for you. Note the $ in the expressions. That matches the end-of-line in the input. In your last example for matching port 451, you still have the potential to match more than you want (451, 4510, 4511, etc).