Shell Programming and Scripting

Hi,

I have a shell script which has a for loop that scans list of files and do find and replace few variables using sed command. While doing this, it deletes the last line of all input file which is something wrong. how to fix this. please suggest. When i add an empty line in all my input file, it fixes the issue. but i look for any other suggestion that will avoid adding this empty line.

my code snippet is:

Code:

for j in `cat $tmp_tilda_var`

do
for i in `cat $tmpfile`
do

        tilda_value1=`grep ^${j} ${ENV_FILE}`
        echo $tilda_value1
        tilda_value=`echo "${tilda_value1}" | cut -d"=" -f2`
        tilda_var1=~${j}~
        sed -e 's#'"$tilda_var1"'#'"$tilda_value"'#g' $i >$i.tmp && mv $i.tmp $i
done
done

where i - is the input file
tilda_var1 - is the pattern to be replaced
tilda_value - is the final string will go in place of variable tilda_var1

Without more detailed, representative info on input files' and variables' contents it is very difficult for me - and mayhap others in here - to reproduce and understand (and even less comment on possible improvements of) the problem.
There's no obvious bug in your code snippet, so only generic suspicions can be uttered: non-*nix text files (DOS line terminators, missing terminator at end-of-file), unexpected behaviour of the for loops, regex special characters in the respective shell variables, ...

RudiC,

Thanks for your response.

I digged into this issue, the input files are coming from accurev SCM tool to my unix location. somehow it is removing the last line when it processes all the input files through this for loop and sed. But if i edit any file in vi mode before running my script and saving back after adding an empty line and removing it again (means no actual change to the file) and saving. now when i run the script, that particular file contents are good. It is not removing the last line now. My input files are with extension .txt, .cfg, .properties, .ksh.

As a workaround what i did is, added the below code which is adding an empty line at the end of all my input files before it hits the for loop and sed command.

Code:

for k in `cat $tmpfile`
do
echo >> $k
done

I know it is not actual fix. Still i added this to move ahead.

For your comments,
i can say, my input file call it as a.txt contain some variables with ~, say ~variableA~ in some places.

a.txt:

Code:

line1
line2
~variableA~
line4

my other file call it as b.txt, contains this list of variables with values, say
b.txt:

Code:

variableA=/proj/apps/sun
variableB=/tmp

For loop will read all variables in b.txt one by one, and go to other for loop to read all input files one by one, find replace the variables with values.

Let me know if anything else i can try. Thanks again.

Moderator's Comments:

To keep the forums high quality for all users, please take the time to format your posts correctly. First of all, use Code Tags when you post any code or data samples so others can easily read your code. You can easily do this by highlighting your code and then clicking on the # in the editing menu. (You can also type code tags [code] and [/code] by hand.) Continued refusal to properly format your posts may result in you being placed in read-only mode for a few days or, in extreme cases, being permanently banned from this site.

Can't. The specification is too sparse (for me) to analyse the error you describe, and the samples don't allow to reproduce it. So don't expect to learn what is going wrong.

Let me open my magic box, dig out my soft skills, and try to paraphrase what I infer from your data and what you say:
There is a file with replacement definitions in the shape varA=ValA, and a (or some) file(s) in which to-be-replaced entries are enclosed in ~ . Entries found are to be substituted by the defined new values.

SHOULD this be close to what you want, consider this proposal:

Code:

sed 's:^:s#~:; s:=:~#:; s:$:#:; ' b.txt | sed -f- a.txt
line1
line2
/proj/apps/sun
line4

RudiC,

I get error with that sed command. once we fix this command, i need to put it in a for loop for processing list of all my input files and see if i do not get my original issue here too.

and other part is, my input file may contain other ~variables~ which are not part of a.txt. so those should be remain same.

Code:

$more b.txt
line1
line2
~variableA~
line4

$more a.txt
variableA=/proj/apps/sun
variableB=/tmp

$sed 's:^:s#~:; s:=:~#:; s:$:#:; ' b.txt | sed -f- a.txt
Cannot open pattern-file: -

OS version:
SunOS ahs2518 5.10 Generic_150400-27 sun4v sparc sun4v

Thanks

Moderator's Comments:

Seriously: Please use CODE tags as required by forum rules!

Please use code tags in your posts!
--
If you want all variable=value pairs from a.txt then you better process it in a while loop.

Code:

filenames=`cat $tmpfile`
while IFS="=" read var value
do
  [ -n "$value" ] || continue
  for f in $filenames
  do
    echo "replacing '~$var~' by '$value' in '$f'"
    sed 's#~'"$var"'~#'"$value"'#g' "$f" >"$f.tmp" && mv -f "$f.tmp" "$f"
  done
done < a.txt