Shell Programming and Scripting

I want to replace columns 15 thru 22 with a date in this format mmddyyyy in a file that has fixed record length of 110 columns. Only lines with column 1 = "T" will be changed but I can't seem to get it to work.

I saw several postings and tried to work with them but they don't work for me...

If I had 20 lines, any lines not 'T' is echoed out as is and when there's a 'T', it is supposed to substitute 15-22 with, say 06062003, but awk command substr won't do it:

echo "$LINE" | awk '{print substr ($0, 1, 14) $proc_dt substr ($0, 24, length ($0) - 24)}' >> newfile

it's literally printing $proc_dt in the newfile...when it should say 06062003 in col 15-22 (I've already set the variable...)

What's going on? I tried enclosing $proc_dt in (, {, etc and nothing works...

Gianni

You have something like this:

'blah blah blah ${variable} more stuff'

The whole idea of single quotes is that "what you see is what you get". The shell will not make any substitutions inside single quotes.

So turn that into two single quoted strings with the variable in between. Like this:

'blah blah blah '${variable}' more stuff'

The brackets are not strictly needed, but I think they improve the readability of the code.

Awesome. I thought I tried this already and it didn't work before...

In any case. this works as expected, so thank you.

The other question comes back to calling the external awk program which seems really slow when trying to substitute 20K records (could take 10-15 minutes) depending on how many users are on the system. Does anyone have any other ideas on how this could be achieved quicker? sed?

Gianni

If you have a shell while loop, and within it you are doing:

echo "$LINE" | awk ...

then you are calling awk 20,000 times, opening your output file 20,000 times etc. Instead, do away with the shell loop, and do it with awk:

Code:

awk '{
if ($1 != "T")
   print
else
   print substr ...
}' infile > newfile

A bit more flexible than the other solutions:

Code:

awk -v newdate=06062003 '/^T/ {
        $0 = substr($0,1,14) newdate substr($0,23,length($0)-22)
    }
    { print $0 }'

This can be put in a shell script to automatically get the date from the 'date' command (if you want today's date), or your script can check the date to be sure it's valid (or you can check the validity in the awk ...) ...

BTW, your original solution dropped a character from the beginning of the line after what comes after the date and another from the end of the line in the expression "substr ($0, 24, length ($0) - 24)".

awk '{
if (substr($1, 1, 1) != 'T' )
print
else
print substr ($1, 1, 14) '${proc_dt}' substr ($1, 24, length ($1))}'
}' infile > newfile

What's wrong with this? I'm not good w/awk..
I got it to work with the while statement but not this way...

Gianni

That's not bad, actually - just a couple of issues ...

Code:

awk '{
if (substr($1, 1, 1) != "T" )
print
else
print substr ($1, 1, 14) '\"${proc_dt}\"' substr ($1, 24)
}' infile > newfile

Your code terminates with two rightbrace-quotemarks, so it hangs waiting on stdin because it does not see infile beyond that second quote.

There are several ways to get a variable into awk. The approach you used by allowing it to be substituted when the command line is parsed is a good way. However, when awk sees the unprotected 14-JUN-2003, that is evaluated as a numeric expression. JUN, as an undefined variable, evaluates to zero, so that makes 14-0-2003 = -1989. You want awk to see that as a string constant. So I surrounded it with double quotes and protected with backslashes so they don't get removed by the shell.

And a small point: When you want the tail-end of a word, such as beginning with character 24 for the remainder of that word, you can just say substr($1,24). With no third operand, it takes the remainder of the expression being substringed.