Shell Programming and Scripting

try putting '[0-2][0-9][0-5][0-9] TMU [^W][^X][^:]' (no quotes) as the 'SOMETHING' in the delete function of sed.

This worked in my tests.

Basically you eliminate the four digits followed by the 'TMU' unless it follows that up with 'WX:'

[edit]

Well scratch that, "duh" moment for me, doesn't work if you have more than one entry to search for. It will eventually pull in another '1911 TMP WX:' line as the last line to the previous entry.

Is there nothing in the actual last line (with the 'FWD:' in it) that you can key in on to make that the last line of your sed command?

Otherwise I think but have not tested, that you could pipe the output from what I've done above through 'uniq -d'

Yeah, I was thinking of the FWD, but looking at complete log files I see that not every entry is forwarded to someone else. Unfortunately, the only constant between entries is that the next entry starts with the 4 digit time and position (which for this facility is always TMU). The only other constant is the keyword that starts the entry, such as WX: or METERING:. Even to colon isn't constant after the keyword.

I've asked the development team that works on the program to add keyword search to the reports we can generate from the command line, but even if they do it, it'll take over a year to get it in place. The government moves extremely slow when tinkering with stuff they consider "operational" to the National Airspace System, even when it doesn't have any effect on controlling aircraft!

Thanks again for the suggestions. I'll keep tinkering, but scripting isn't my strong point. I can program in Visual Basic, but that won't work on Linux

If you can do it in VB, you can do it in Linux. It will probably best be done with awk, rather than as a sed one-liner.

What algorithm would you use in VB?

Hadn't given it too much thought. Since each line in the entry is a fixed length (including spaces, it looks like it's 80) maybe I could save it into a variable, then trim so many characters from the end. With sed I don't know how to trim the end of each individual return, and then copy the variable into the new file and continue the sed. Wish I understood this better!

That's a job for awk, not sed.

Code:

awk '/WX: /,/[0-2][0-9][0-5][0-9] TMU/ {
   if ( last ) print last
   last = $0
   next
 }

{ last = "" }

' 15.txt

If you can play with the input file a little, I can do this in ksh.

Code:

#!/bin/ksh

FILE1=$1         
FILE2=tmp.txt

cat $FILE1 | tr -s '\012' ' ' > $FILE2

print >> $FILE2

cat $FILE2 | sed 's/ \([0-2][0-9][0-5][0-9] TMU\)/\
\1/g' | sed -n '/[0-2][0-9][0-5][0-9] TMU [MW][EX]/P'

rm $FILE2

usage:

thisscript.sh 15.txt

Notes: I'm commenting here to keep the clutter in the code down.

Script takes your input file as parameter '1' and asigns that to FILE1.
FILE2 is a temp file we'll use for output.

cat FILE1 to 'tr' to change all newlines to spaces and store the result in FILE2

'print' then appends a newline to the end of FILE2. This is necessary or 'sed' will ignore the input of FILE2 as it must see a newline.

cat FILE2 into two distinct sed process. The first inserts a new line before the specified pattern of ' #### TMU' (I use # instead of the actual numerical pattern for brevity here). Note there is a space before the first number.

In the first sed process note that there is a REQUIRED newline after the '/\' so that the rest of the command resumes on the next line with "\1/g' "

Now you have records that are separated by a newline w/o any newlines in the records themselves as there was before.

So the second sed process can now identify each record you want printed with only 1 address, and print the full record, w/o printing the record(s) you don't want.

The [WM][EX] will, unfortunately find 'MX' and 'WE' as well as 'WX' and 'ME', so you may have to play with this depending upon your actual data.

why exactly do you need to 'cat' into 'tr' and 'sed'?