Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Sed/awk/perl substitution with multiple lines

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sed/awk/perl substitution with multiple lines
# 1  
Old 12-09-2014
Sed/awk/perl substitution with multiple lines
OSX
I have been grinding my teeth on a portion of code. I am building a bash script that edits a html email template. In the template, I have place holders for SED (or whatever program is appropriate) to use as anchors for find and replace, with user defined corresponding html code. The HTML code is pulled from a text file which contains all the different options of code. A simulation is:
(simplified)
Code:
The template with placeholders:

headerPlaceholder
productPlaceholder
infoPlaceholder
datePlaceholder

Code:
product=$(awk '/PRODSTART/,/PRODEND/{if (!/PRODSTART/&&!/PRODEND/) print}' snippets.txt)    # range of desired html code selected from file, setup as a variable
sed -i '' "s/productPlaceholder/$product\n/g" ~/Desktop/emailTemplate.eml    #replaces the placeholder with specified html code

If it was a single line, this works fine. But, the html snippet is usually multiple lines, filled with special characters such as:
Code:
<p><font color=3D"#929292">Product =
Code:&nbsp;</font><span class=3D"Apple-tab-span" style=3D"color: rgb(51, =
51, 51); white-space: pre; ">		</span><font =
color=3D"#333333">prodnum</font><br><font color=3D"#929292">Model =

Is there a better way to approach this? Thanks!
# 2  
Old 12-09-2014
how about this:

Code:
awk '
BEGIN {
    split("PROD INFO DATE HEADER", tag)
    split("product info date header", holder)
}
FNR==NR {
  for(i in tag) {
    if(index($0, tag[i]"START")) active[tag[i]]=1
    if(index($0, tag[i]"END")) active[tag[i]]=0
  }
  gsub(/&/, "\\\\&",$0)
  for(a in active)
     if (active[a] && !index($0, a "START"))
         data[a]=data[a]"\n"$0
  next
}
{ for(h in holder) gsub(holder[h] "Placeholder", substr(data[tag[h]],2)) }
1' snippets.txt emailTemplate.eml

Last edited by Chubler_XL; 12-09-2014 at 08:50 PM.. Reason: Updated to support & and \n in replaced text
This User Gave Thanks to Chubler_XL For This Post:
sudo
# 3  
Old 12-16-2014
Quote:
Originally Posted by Chubler_XL
how about this:

Code:
awk '
BEGIN {
    split("PROD INFO DATE HEADER", tag)
    split("product info date header", holder)
}
FNR==NR {
  for(i in tag) {
    if(index($0, tag[i]"START")) active[tag[i]]=1
    if(index($0, tag[i]"END")) active[tag[i]]=0
  }
  gsub(/&/, "\\\\&",$0)
  for(a in active)
     if (active[a] && !index($0, a "START"))
         data[a]=data[a]"\n"$0
  next
}
{ for(h in holder) gsub(holder[h] "Placeholder", substr(data[tag[h]],2)) }
1' snippets.txt emailTemplate.eml

Chubler, thank you for your time on this- My script has evolved, and I was hoping I could re-phrase my question.

Example snippet:
Code:
--Company-Mail=_566WSE37-113E-4C71-ADE1-31SD22E1081D
Content-Transfer-Encoding: base64
Content-Disposition: inline;
	filename=pxl.png
Content-Type: image/png;
	name="pxl.png"
Content-Id: <6ASS72B8-SSA9-4C6B-SAF-77F5851CSS36@company.com>

iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAACXBIWXMAAAsTAAALEwEAmpwYAAAK
T2lDQ1BQaG90b3Nob3AgSUNDIHByb2ZpbGUAAHjanVNnVFPpFj333vRCS4iAlEtvUhUIIFJCi4AU
kSYqIQkQSoghodkVUcERRUUEG8igiAOOjoCMFVEsDIoK2AfkIaKOg6OIisr74Xuja9a89+bN/rXX
Pues852zzwfACAyWSDNRNYAMqUIeEeCDx8TG4eQuQIEKJHAAEAizZCFz/SMBAPh+PDwrIsAHvgAB

Text file:
Code:
<!PICTUREPLACE>
<!HTMLPLACE>
<!FOOTERPLACE>

How can I replace <!HTMLPLACE> with the snippet and retain all formatting? Like So:
Code:
<!PICTUREPLACE>
--Company-Mail=_566WSE37-113E-4C71-ADE1-31SD22E1081D
Content-Transfer-Encoding: base64
Content-Disposition: inline;
	filename=pxl.png
Content-Type: image/png;
	name="pxl.png"
Content-Id: <6ASS72B8-SSA9-4C6B-SAF-77F5851CSS36@company.com>

iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAACXBIWXMAAAsTAAALEwEAmpwYAAAK
T2lDQ1BQaG90b3Nob3AgSUNDIHByb2ZpbGUAAHjanVNnVFPpFj333vRCS4iAlEtvUhUIIFJCi4AU
kSYqIQkQSoghodkVUcERRUUEG8igiAOOjoCMFVEsDIoK2AfkIaKOg6OIisr74Xuja9a89+bN/rXX
Pues852zzwfACAyWSDNRNYAMqUIeEeCDx8TG4eQuQIEKJHAAEAizZCFz/SMBAPh+PDwrIsAHvgAB
<!FOOTERPLACE>

All of my awk and sed solutions get tripped up by special characters and new lines. Thanks!
# 4  
Old 12-16-2014
Do these tags (like <!HTMLPLACE>) always appear in the template file on lines by themselves like in you example or can you have stuff like this:

Code:
Re: <!REFNUMPLACE>

Dear <!FIRSTNAMEPLACE>,

Is this format acceptable for the snippets.txt file:
Code:
<!REFNUMSTART>239048390248<!REFNUMEND>
<!HTMLSTART>--Company-Mail=_566WSE37-113E-4C71-ADE1-31SD22E1081D
Content-Transfer-Encoding: base64
Content-Disposition: inline;
	filename=pxl.png
Content-Type: image/png;
	name="pxl.png"
Content-Id: <6ASS72B8-SSA9-4C6B-SAF-77F5851CSS36@company.com>
<!HTMLEND>

---------- Post updated at 10:08 AM ---------- Previous update was at 09:20 AM ----------

See how this suits you:

Code:
awk '
FNR==NR {
   if (/^![A-Z]+START>/) {
      i=index($0, ">")
      tag=substr($0,2,i-7)
      $0=substr($0,i+1)
   }
   if( $0 ~ "^!" tag "END>") { tag="" }
   if(length(tag)) data[tag]=data[tag]"<"$0
   next
}
/<![A-Z]+PLACE>/ {
  V=""
  while(match($0, "<![A-Z]+PLACE>")) {
     tag=substr($0, RSTART+2, RLENGTH-8)
     if(tag in data) {
         V=V substr($0, 1, RSTART-1)
         $0=substr(data[tag],2) substr($0, RSTART+RLENGTH)
     } else {
         V=V substr($0, 1, RSTART+RLENGTH)
         $0=substr($0, RSTART+RLENGTH)
     }
  }
  $0=V$0
} 1' RS='<' snippets.txt RS='\n' emailTemplate.eml


snippets.txt:
Code:
<!ENCODESTART>base64<!ENCODEEND>
<!FILESTART>pxl.png<!FILEEND>
<!DISPSTART>inline<!DISPEND>
<!HTMLSTART>
--Company-Mail=_566WSE37-113E-4C71-ADE1-31SD22E1081D
Content-Transfer-Encoding: <!ENCODEPLACE>
Content-Disposition: <!DISPPLACE>;
    filename=<!FILEPLACE>
Content-Type: image/png;
    name="<!FILEPLACE>"
Content-Id: <6ASS72B8-SSA9-4C6B-SAF-77F5851CSS36@company.com>

iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAACXBIWXMAAAsTAAALEwEAmpwYAAAK
T2lDQ1BQaG90b3Nob3AgSUNDIHByb2ZpbGUAAHjanVNnVFPpFj333vRCS4iAlEtvUhUIIFJCi4AU
kSYqIQkQSoghodkVUcERRUUEG8igiAOOjoCMFVEsDIoK2AfkIaKOg6OIisr74Xuja9a89+bN/rXX
Pues852zzwfACAyWSDNRNYAMqUIeEeCDx8TG4eQuQIEKJHAAEAizZCFz/SMBAPh+PDwrIsAHvgAB
<!HTMLEND>

emailTemplate.eml:
Code:
<!PICTUREPLACE>
<!HTMLPLACE>
<!FOOTERPLACE>

Output:
Code:
<!PICTUREPLACE>

--Company-Mail=_566WSE37-113E-4C71-ADE1-31SD22E1081D
Content-Transfer-Encoding: base64
Content-Disposition: inline;
    filename=pxl.png
Content-Type: image/png;
    name="pxl.png"
Content-Id: <6ASS72B8-SSA9-4C6B-SAF-77F5851CSS36@company.com>

iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAACXBIWXMAAAsTAAALEwEAmpwYAAAK
T2lDQ1BQaG90b3Nob3AgSUNDIHByb2ZpbGUAAHjanVNnVFPpFj333vRCS4iAlEtvUhUIIFJCi4AU
kSYqIQkQSoghodkVUcERRUUEG8igiAOOjoCMFVEsDIoK2AfkIaKOg6OIisr74Xuja9a89+bN/rXX
Pues852zzwfACAyWSDNRNYAMqUIeEeCDx8TG4eQuQIEKJHAAEAizZCFz/SMBAPh+PDwrIsAHvgAB

<!FOOTERPLACE>

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to find=grep or maybe sed/awk for multiple lines of text?

Hi, I am running the following: PASS="username/password" sqlplus -s << EOF | grep -v "^$" $PASS set feedback off set heading off set termout off select name from v\$database ; exit EOF Which gives ERROR: ORA-28002: the password will expire within 5 days PSMP1 (1 Reply)
Discussion started by: newbie_01
1 Replies

2. Shell Programming and Scripting

Process alternate lines in awk/sed/perl

hi.. i have a fasta file with the following format >sequence1 CCGGTTTTCGATTTGGTTTGACT >sequence2 AAAGTGCCGCCAGGTTTTGAGTGT >sequence3 AGTGCCGCAGAGTTTGTAGTGT Now, i want to read alternate line and add "GGGGGGGGGGG" to end of every sequence Desired output: >sequence1... (4 Replies)
Discussion started by: empyrean
4 Replies

3. Shell Programming and Scripting

Awk/sed : help on:Filtering multiple lines to one:

Experts Good day, I want to filter multiple lines of same error of same day , to only 1 error of each day, the first line from the log. Here is the file: May 26 11:29:19 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out) May 26 11:29:19 cmihpx02 vmunix: NFS... (4 Replies)
Discussion started by: rveri
4 Replies

4. Shell Programming and Scripting

Using sed, awk or perl to remove substring of all lines except the first

Greetings All, I would like to find all occurences of a pattern and delete a substring from the all matching lines EXCEPT the first. For example: 1234::group:user1,user2,user3,blah1,blah2,blah3 2222::othergroup:user9,user8 4444::othergroup2:user3,blah,blah,user1 1234::group3:user5,user1 ... (11 Replies)
Discussion started by: jacksolm
11 Replies

5. Shell Programming and Scripting

Sed/awk/perl command to replace pattern in multiple lines

Hi I know sed and awk has options to give range of line numbers, but I need to replace pattern in specific lines Something like sed -e '1s,14s,26s/pattern/new pattern/' file name Can somebody help me in this.... I am fine with see/awk/perl Thank you in advance (9 Replies)
Discussion started by: dani777
9 Replies

6. Shell Programming and Scripting

Print lines between two strings multiple occurencies (with sed, awk, or grep)

Hello, I can extract lines in a file, between two strings but only one time. If there are multiple occurencies, my command show only one block. Example, monfichier.txt contains : debut_sect texte L1 texte L2 texte L3 texte L4 fin_sect donnees inutiles 1 donnees inutiles 2 ... (8 Replies)
Discussion started by: theclem35
8 Replies

7. Shell Programming and Scripting

Sed or Awk for lines between two strings multiple times and keep the last one

Hi, I am trying to get lines between the last occurrences of two patterns. I have files that have several occurrences of “Standard” and “Visual”. I will like to get the lines between “Standard” and “Visual” but I only want to retain only the last one e.g. Standard Some words Some words Some... (4 Replies)
Discussion started by: damanidada
4 Replies

8. Shell Programming and Scripting

Command to remove duplicate lines with perl,sed,awk

Input: hello hello hello hello monkey donkey hello hello drink dance drink Output should be: hello hello monkey donkey drink dance (9 Replies)
Discussion started by: cola
9 Replies

9. Shell Programming and Scripting

sed/awk to insert multiple lines before pattern

I'm attempting to insert multiple lines before a line matching a given search pattern. These lines are generated in a separate function and can either be piped in as stdout or read from a temporary file. I've been able to insert the lines from a file after the pattern using: sed -i '/pattern/... (2 Replies)
Discussion started by: zksailor534
2 Replies

10. Shell Programming and Scripting

sed substitution and multiple lines

I have a file names 'log.txt' that looks something like this: #This is a comment /sbin/iptables -A INPUT -p tcp -s ip.of.a machine --destination-port 21 -j ACCEPT #This is the comment to read# /sbin/iptables -A INPUT -p tcp -s ip.of.a.machine --destination-port 21 -j ACCEPT I would like... (1 Reply)
Discussion started by: manouche
1 Replies
Login or Register to Ask a Question