Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Sed, replace variables in file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sed, replace variables in file
# 8  
Old 05-01-2013
It turned out that the problem was caused by slashes in HTML tags.
Anyway, I've found a sed command that escapes all of the special characters.

Now, my code looks like this:
Code:
fileA=`cat ./fileA | tr "\n" "\f" | sed 's:[]\[\^\$\.\*\/]:\\\\&:g'`
fileB=`cat ./fileB | tr "\n" "\f" | sed 's:[]\[\^\$\.\*\/]:\\\\&:g'`

sed -e 's/\$a/'"$fileA"'/g' -e 's/\$b/'"$fileB"'/g' -e 's/$\$/$/g' ./template | tr "\f" "\n"

... and it seems to work with my test files. Smilie

Thank you so much for your help.
# 9  
Old 05-01-2013
Quote:
Originally Posted by csarli2006
Anyway, I've found a sed command that escapes all of the special characters.

Now, my code looks like this:
Code:
fileA=`cat ./fileA | tr "\n" "\f" | sed 's:[]\[\^\$\.\*\/]:\\\\&:g'`
fileB=`cat ./fileB | tr "\n" "\f" | sed 's:[]\[\^\$\.\*\/]:\\\\&:g'`

sed -e 's/\$a/'"$fileA"'/g' -e 's/\$b/'"$fileB"'/g' -e 's/$\$/$/g' ./template | tr "\f" "\n"

... and it seems to work with my test files. Smilie
In its present form, there a couple of serious flaws in your code, along with a couple of minor ones.

You are not adding a backslash before ampersands, which have a special meaning in the replacement text. Similarly, you neglect to backslash backslashes, which are also special in replacement text. Ironically, except for the forward-slash, none of the characters which you are trying to protect against are a threat; they are not special in replacement text, only in regular expressions.

While it's not causing a malfunction, you should be aware that none of the backslashes in the bracket expression, []\[\^\$\.\*\/] are necessary. None of the characters within the brackets would be treated specially if its preceding backslash were deleted. In fact, the backslash before the dollar sign is consumed by the command substitution operation and never seen by sed. If sed needed it, this would be another bug.

You could simplify the quoting situation by abandoning the old, deprecated form of command substitution. Instead, use the modern syntax: $( ... ). An example of the benefit: \\\\& becomes \\&.

I would suggest using AWK. The code may be longer, but it's simpler and safer.

If your replacement text files have an undesirable trailing newline, and if you can remove it, then you can use this simple script:
Code:
BEGIN {
        files["a"] = "./a"
        files["b"] = "./b"
}

! /\$/ {
        print
        next
}

{
        for (spos = 1; mpos = index(substr($0, spos), "$"); spos += 2) {
                printf "%s", substr($0, spos, mpos-1)
                spos = spos+mpos-1
                key = substr($0, spos+1, 1)
                if (key in files)
                        system("cat " files[key])
                else
                        printf "%s", substr($0, spos, 2)
        }
        printf "%s%s", substr($0, spos), ORS
}

Alternatively, for the price of increased memory consumption, you can cache the files. The following modification caches the file contents minus any trailing newline:
Code:
function slurpfile(path, key) {
        while (ret = getline t < path) {
                if (ret == 0) {
                        close(path)
                        break
                }
                if (ret == -1) {
                        printf "%s: Error reading file: %s\n", ARGV[0], path
                        exit 1
                }
                s = s t ORS
        }
        files[key] = substr(s, 1, length(s)-1)
        s = t = ""
}

BEGIN {
        slurpfile("./a", "a")
        slurpfile("./b", "b")
}

! /\$/ {
        print
        next
}

{
        for (spos = 1; mpos = index(substr($0, spos), "$"); spos += 2) {
                printf "%s", substr($0, spos, mpos-1)
                spos = spos+mpos-1
                key = substr($0, spos+1, 1)
                if (key in files)
                        printf "%s", files[key]
                else
                        printf "%s", substr($0, spos, 2)
        }
        printf "%s%s", substr($0, spos), ORS
}

It's possible that there's an entirely unrelated method to achieve your goal that is much simpler than any of these, but that would require knowing the "why", not just the "what".

Regards,
Alister
Last edited by alister; 05-01-2013 at 02:07 PM..
# 10  
Old 05-01-2013
This is using the ` in the outer sed as discussed before.
The inner sed puts \ at the end of all lines but the last line, so the outer sed gets its multi-line format.
Code:
sed '
s`\$a`'"$(sed '$!s/$/\\/' fileA)"'`g
s`\$b`'"$(sed '$!s/$/\\/' fileB)"'`g
s/\$\$/$/g
' template

Last edited by MadeInGermany; 05-01-2013 at 04:17 PM.. Reason: Added last step - replace $$ and corrected a$ to $a
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk search/replace specific field, using variables for regexp & subsitution then overwrite file

Hello, I'm trying the solve the following problem. I have a file which I intend to use as a csv called master.csv The columns are separated by commas. I want to change the text on a specific row in either column 3,4,5 or 6 from xxx to yyy depending upon if column 1 matches a specified pattern.... (3 Replies)
Discussion started by: cyphex
3 Replies

2. Shell Programming and Scripting

environment variables in a sed script file

Hello Everyone I need to create a script file which must append some lines to a target text file, I'm using sed for windows, the script file look like this: { a\ STRINGTABLE DISCARDABLE\ BEGIN\ 5, 150 {a\ #define RC_SHELL, "%ID_SHELL%"\ #define RC_NAME, "%ID_NAME%"\ END } ... (1 Reply)
Discussion started by: edgarvm
1 Replies

3. UNIX for Dummies Questions & Answers

How to use sed to copy specific lines from a file using shell variables?

hello! I am trying to use sed to copy specific set of lines from a file for which the starting and ending line numbers of the lines to be copied are stored in shell variables. How can i copy those lines? if the input_file is something like this and if the following is the script a=2 b=4... (4 Replies)
Discussion started by: a_ba
4 Replies

4. Shell Programming and Scripting

Read file and for each line replace two variables, add strings and save output in another file

Hi All, I have a file, let's call it "info.tmp" that contains data like this .. ABC123456 PCX333445 BCD789833 I need to read "info.tmp" and for each line add strings in a way that the final output is put /logs/ua/dummy.trigger 'AAA00001.FTP.XXX.BLA03A01.xxxxxx(+1)' where XXX... (5 Replies)
Discussion started by: Andy_ARG
5 Replies

5. Shell Programming and Scripting

replace two character strings by two variables with sed command

Hello, I want to writte a script that replace two character strings by two variables with the command sed butmy solution doesn't work. I'm written this: sed "s/TTFactivevent/$TTFav/g && s/switchSLL/$SLL/g" templatefile. I want to replace TTFactivevent by the variable $TTFav, that is a... (4 Replies)
Discussion started by: POPO10
4 Replies

6. Shell Programming and Scripting

How to use sed to replace the a string in the same file using sed?

How do i replace a string using sed into the same file without creating a intermediate file? (7 Replies)
Discussion started by: gomes1333
7 Replies

7. Shell Programming and Scripting

using sed to replace a specific string on a specific line number using variables

using sed to replace a specific string on a specific line number using variables this is where i am at grep -v WARNING output | grep -v spawn | grep -v Passphrase | grep -v Authentication | grep -v '/sbin/tfadmin netguard -C'| grep -v 'NETWORK>' >> output.clean grep -n Destination... (2 Replies)
Discussion started by: todd.cutting
2 Replies

8. Shell Programming and Scripting

Replace environment variables with sed

Hi, I need to write a sed that replaces the value of all environment variables that have PASS in the name with ***** . I have a log file who prints the value of all environment variables ,including those who hold passwords. In the name of these variables I found always the PASS string, ex:... (16 Replies)
Discussion started by: liorfe
16 Replies

9. UNIX for Dummies Questions & Answers

Problems with sed and flat file variables

Hello All, It has been a loooooooooooong time since I had last used sed but decided to use it for a simple task I have . My goal is to use sed to read variables from a flat file then use those same variables in order to make some subsitutions. However what I am finding is that when the... (1 Reply)
Discussion started by: icalderus
1 Replies

10. UNIX for Dummies Questions & Answers

variables use upper case? sed : output to the same file?

Hi, Q1: are the variables in shell script usually UPPER CASE? Q2: can sed output to the same file that it's using it? eg. cat sameFile | sed 's/here/there/g' > sameFile ? I expect the sed replace all "here" to "there" and change it in sameFile. i tried that one, the sameFile... (1 Reply)
Discussion started by: gusla
1 Replies
Login or Register to Ask a Question