Shell Programming and Scripting

One substantive issue with Jimbo's post of 6-14-2003: If you use substr($1,1,1) == "T", you will pick up lines where the first non-blank character is a "T", which is in general a superset of the lines the OP asked for (lines with "T" in column 1), though it may be sufficient for this particular application. Better would be substr($0,1,1)=="T" or /^T/.

That's not my code. I was just fixing the major problems while retaining as much as possible of the original code.

When the OP says "column 1", some people may think column 1 of the entire line, but many people call the fields "columns", while some say "fields" and yet others say "words". Therefore, the OP actually might want to check character 1 of $1 rather than character 1 of $0.

Thanks for the hint. It almost worked. Now the program runs in 1-3 seconds (as opposed to 5-10 minutes)...however, it did not seem to preserve the spaces in the record that was updated...even though I said to print out from 1 - 14 then 24 - to end of record... There are many blanks in this fixed formatted file and I need to preserve everything as is...other than the date.

Also, how do I set variables in the awk statement so that I can echo out line number, incrementing line count, record type, etc of the updated record as they are being processed? My attempt to do it resulted in my code abending again and again....

Thanks.

Gian

It would be helpful to see the code you have at this point. If you take substr($0,1,14), that takes the first 14 characters of the line, multiple spaces and all.

Set variables with

myvar=(some expression)

See "man awk" for predefined expressions. For example, NR is a predefined variable representing the current record number being processed from 1 thru n across all files, and FNR is a file-relative current record nuimber that begins over at 1 when awk is processing multiple input files.

If you want to see additional info being processed such as record numbers etc, you will want to have two outputs: your newly formatted output file and your "processing log output". This is easily done with awk:

print substr($0,1,14) substr($0,24)
print NR, rectype, amount > "mylogfile"

You don't need a double right-angle bracket here for append because awk will open mylogfile only once, not once for each record being processed. But if you want mylogfile to be appended across multiple runs, then use the double.

Just a side note about your code from above, giannicello. It didn't seem (at least to me when I was reading) that there were spaces in your file, but now you said there are. Awk uses the space as a delimiter by default. What this means is that if a typical line in your file looks like:

abcdefghijklmn12345678some other stuff

then $1 is abcdefghijklmn12345678some, $2 is other and $3 is stuff.
$0 stores the whole line.

So this piece of code you entered - print substr ($1, 1, 14) '\"${proc_dt}\"' substr ($1, 24) - will print the first 14 characters of $1, or abcdefghijklmn, then the value of $proc_dt, then the rest of $1, which is just some. other and stuff get left out completely, so you need to use $0.

As you can tell, I didn't know what I was doing that's why I'm asking the experts. I think I understand awk a little better now. I started out with $0 and somewhere along the line it because $1. I made the change and now everything is fine. The code is some much simpler and faster too.

Thanks.