Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

inserting and replacing lines with awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting inserting and replacing lines with awk
# 1  
Old 06-11-2011
inserting and replacing lines with awk
Hello,

I need to insert varying lines (i.e. these lines are an output of another script) between lines starting with certain fields.
An example to make it more clear.

This is the file where I wanna insert lines:
(save it as "input.txt")
Code:
ContrInMi_c_mir 	 2 	 10066	181014 	 200750
ContrMidl_y_cut 	 2 	 12345	201085 	 220988
ContrMiRi_c_mir 	 2 	 13206	221155 	 240891
ContrIdex_p_mir 	 1 	 765	241225 	 260962
ContrMidl_b_cut 	 1 	 2686	261296 	 281032

This is the file ("insert.txt") containing the lines that should be inserted:
Code:
texta: ContrInMi_c_mir
textb: ContrMidl_y_cut
textins: sec_buttonpress 	 3 	 15147
texta: ContrIdex_p_mir
textb: ContrMidl_b_cut
textins: sec_buttonpress 	 2 	 14605

And this is the script which calls at the end the .awk file "dothis.awk"
Code:
#!/bin/bash
declare -a Arraytexta
declare -a Arraytextb
declare -a Arraytextins
nrLines=`awk 'END {print NR}' insert.txt`
nrLines=`expr ${nrLines} - 2`

for ((i=1; i<=$nrLines; i++)); 
do 
countb=`expr ${i} + 1`
countins=`expr ${i} + 2`
Arraytexta[i]=`sed -n "${i} p" insert.txt | awk 'BEGIN{FS=":"}{print $2}'`
Arraytextb[i]=`sed -n "${countb} p" insert.txt | awk 'BEGIN{FS=":"}{print $2}'`
Arraytextins[i]=`sed -n "${countins} p" insert.txt | awk 'BEGIN{FS=":"}{print $2}'`
gawk -v textins="${Arraytextins[i]}" -v texta="${Arraytexta[i]}" -v textb="${Arraytextb[i]}" -f dothis.awk input.txt" > output.txt
done

Here is dothis.awk
Code:
BEGIN{
	flg=0
}
{
	if (flg==1 && $1==textb || $1=="sec_buttonpress"){
		if($1==textb  || $1=="sec_buttonpress"){
			print textins
		}
	}
	if ($1==texta){
		flg=1
	}
	print $0
}
END{
	
}

So, basically what I try to do is: I search for the lines starting with "texta" and check if the following line starts either with "textb" or - if there has been a line previously inserted (which can happen) - if this starts with "sec_buttonpress". If I found these lines then the "textins" should inserted.
Actually, an extension to cover all needs would be to check also if the second field in the file "input.txt" contains "1". If so then the line (only fields 2 and 3) to be inserted after should actually replace the fields 2 and 3 of this line.

For some reason, "dothis.awk" is not working: it inserts at the end of teh "input.txt" file always the same 2 fields: containing a 6digit number and a "0".

What am I doing wrong?

Thanks a lot in advance for help and advices!!!


tempestas
Last edited by tempestas; 06-12-2011 at 08:17 AM.. Reason: correcting a file name in the example code
# 2  
Old 06-11-2011
Hi,

I don't know what is wrong, but:

1.- Which file is untitled3.txt?
2.- Try comment a little bit your code to know what you want to do.
3.- Try one awk program instead using several 'sed' & 'awk' commands.
4.- Input file of 'dothis.awk' is 'input.txt' while in the 'awk' program you
search 'texta' and 'textb' fields which are from 'insert.txt' file. Is this
correct?
5.- Paste the output.txt you are looking for.

Regards,
Birei
Last edited by birei; 06-11-2011 at 06:12 PM..
# 3  
Old 06-11-2011
Hi birei,

thanks a lot for your comments! Sorry, if it's not really clear what I meant. I'll try to improve that:

Quote:
Originally Posted by birei
1.- Which file is untitled3.txt?
Sorry, my mistake: it should be called "insert.txt". I changed that in my first post.
Quote:
Originally Posted by birei
2.- Try comment a little bit your code to know what you want to do.
Here the code commented:
Code:
#!/bin/bash
### ok, this is only for creating the example files and try to make it look like the original. 
### My original code is much longer since I process also some other parts of the original input.
declare -a Arraytexta
declare -a Arraytextb
declare -a Arraytextins
nrLines=`awk 'END {print NR}' insert.txt`
nrLines=`expr ${nrLines} - 2`

for ((i=1; i<=$nrLines; i++)); 
do 
countb=`expr ${i} + 1`
countins=`expr ${i} + 2`
### Following, I create the arrays that contain the informations that are needed for the "gawk" command:
### First, the pattern from the first line of the match.
Arraytexta[i]=`sed -n "${i} p" insert.txt | awk 'BEGIN{FS=":"}{print $2}'`
### Second, the pattern with which the following line should start.
Arraytextb[i]=`sed -n "${countb} p" insert.txt | awk 'BEGIN{FS=":"}{print $2}'`
### Third, the text that should be inserted.
Arraytextins[i]=`sed -n "${countins} p" insert.txt | awk 'BEGIN{FS=":"}{print $2}'`
### Here, I'll try to pass on the informations from the arrays above into the "dothis.awk" file.
gawk -v textins="${Arraytextins[i]}" -v texta="${Arraytexta[i]}" -v textb="${Arraytextb[i]}" -f dothis.awk input.txt > output.txt
done

Quote:
Originally Posted by birei
3.- Try one awk program instead using several 'sed' & 'awk' commands.
Sorry, but I am a beginner with awk. So some things seem for me at this point easier or more familiar with sed. Smilie
Quote:
Originally Posted by birei
4.- Input file of 'dothis.awk' is 'input.txt' while in the 'awk' program you
search 'texta' and 'textb' fields which are from 'insert.txt' file. Is this
correct?
Yes, that's correct.
Quote:
Originally Posted by birei
5.- Paste the output.txt you are looking for.
Here is an example for the "output.txt" file.
Code:
ContrInMi_c_mir 	 2 	 10066	181014 	 200750
sec_buttonpress 	 3 	 15147
ContrMidl_y_cut 	 2 	 12345	201085 	 220988
ContrMiRi_c_mir 	 2 	 13206	221155 	 240891
ContrIdex_p_mir 	 1 	 765	241225 	 260962
sec_buttonpress 	 2 	 14605
ContrMidl_b_cut 	 1 	 2686	261296 	 281032

It would be even better if lines that contain in $2 the number "1", can be replaced, i.e. only $2 and $3 should be replaced. Like in this case:

instead of:
Code:
ContrIdex_p_mir 	 1 	 765	241225 	 260962
sec_buttonpress 	 2 	 14605

rather:
Code:
ContrIdex_p_mir 	 2 	 14605	241225 	 260962

I hope it becomes more clear what I would like to do.

Thanks a lot for help!

tempestas
Last edited by tempestas; 06-12-2011 at 08:18 AM..
# 4  
Old 06-11-2011
The example you posted are very long and the lines does not correspond between input file, replacement file , and output file (it looks as if they were screwed).
I mean i don't see the logic that should be followed.

I think you should post a shorter an cleaner example of infile, insert file and output expected so we could

1) quickly understand the logic of formatting
2) provide you a more accurate answer
# 5  
Old 06-12-2011
Hi,

sorry, I thought the longer examples help to better understand the structure, because the input files are a mess, but they are given to me like that.
I shortened the examples in my former posts. I hope now it becomes more clear what I wanna do.

Thanks a lot!

tempestas
# 6  
Old 06-12-2011
Code:
$ cat f1
ContrInMi_c_mir          2       10066  181014   200750
ContrMidl_y_cut          2       12345  201085   220988
ContrMiRi_c_mir          2       13206  221155   240891
ContrIdex_p_mir          1       765    241225   260962
ContrMidl_b_cut          1       2686   261296   281032

Code:
$ cat f2
texta: ContrInMi_c_mir
textb: ContrMidl_y_cut
textins: sec_buttonpress         3       15147
texta: ContrIdex_p_mir
textb: ContrMidl_b_cut
textins: sec_buttonpress         2       14605

Code:
$ cat myawk
BEGIN{i=1}
NR==FNR{p[NR]=$1;a[NR]=$0;next}
/^texta/{x=$2}
x&&/^textins/{sub(".*"$3,$3);y=$0}!y{next}
{do {
        if(p[i]!=x){
                print a[i]
        }else{
                split(a[i],b)
                print p[i]" \t "y"\t"b[4]" \t "b[5]
                x=y=z;delete b;i++;next
        }
        delete a[i];i++
}while(length(a[i]))
}END{do{print a[i];i++}while(length(a[i]))}

Code:
$ awk -f myawk f1 f2
ContrInMi_c_mir          3       15147  181014   200750
ContrMidl_y_cut          2       12345  201085   220988
ContrMiRi_c_mir          2       13206  221155   240891
ContrIdex_p_mir          2       14605  241225   260962
ContrMidl_b_cut          1       2686   261296   281032
$

Last edited by ctsgnb; 06-12-2011 at 02:46 PM..
# 7  
Old 06-12-2011
Hi ctsgnb,

thanks a lot for your effort! The code seems quite complicated. Actually, I do not understand what is done. Can you explain it a little bit so that I learn something?
And one question: the reason why I explicitly marked not only "texta" but the following line as "textb" is that "texta" and also "textb" can be more than once in the original input file. But they will most likely not appear again in consecutive lines.
As far as I understand you code, it looks only for "texta" and ignores "textb". Is that correct?


Thanks a lot!!!

tempestas
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merging multiple lines to columns with awk, while inserting commas for missing lines

Hello all, I have a large csv file where there are four types of rows I need to merge into one row per person, where there is a column for each possible code / type of row, even if that code/row isn't there for that person. In the csv, a person may be listed from one to four times... (9 Replies)
Discussion started by: RalphNY
9 Replies

2. Shell Programming and Scripting

Replacing lines matching a multi-line pattern (sed/perl/awk)

Dear Unix Forums, I am hoping you can help me with a pattern matching problem. What am I trying to do? I want to replace multiple lines of a text file (that match a multi-line pattern) with a single line of text. These patterns can span several lines and do not always have the same number of... (10 Replies)
Discussion started by: thefang
10 Replies

3. Shell Programming and Scripting

Summing over specific lines and replacing the lines with the sum using sed, awk

Hi friends, This is sed & awk type question. I have a text file which has numbers spread all over the file. I want to sum the series of numbers whenever i find it and produce an output file with the sum. For example ###start of input text file #### abc def ghi 1 2 3 4 kjld random... (3 Replies)
Discussion started by: kaaliakahn
3 Replies

4. UNIX for Dummies Questions & Answers

Finding lines with a regular expression, replacing them with blank lines

So the tag for this forum says all newbies welcome... All I want to do is go through my file and find lines which contain a given string of characters then replace these with a blank line. I really tried to find a simple command to do this but failed. Here's what I did come up with though: ... (2 Replies)
Discussion started by: Golpette
2 Replies

5. Shell Programming and Scripting

Help in replacing two blank lines with two lines of diff data

Hi.. I'm facing a trouble in replacing two blank lines in a file using shell script... I used sed to search a line and insert two blank lines after the searchd line using the following sed command. sed "/data/{G;G;}/" filename . In the file, after data tag, two lines got inserted blank lines..... (4 Replies)
Discussion started by: arjun_arippa
4 Replies

6. Shell Programming and Scripting

inserting multiple lines with awk

awk '/<login-module code="com.nlayers.seneca.security.LdapLogin" flag="sufficient">/{p++} /<login-module code="com.nlayers.seneca.security.LdapLogin" flag="sufficient">/ && p==1 {$0="Mulitple lines here\n"$0}1' login-config.xml In the above awk code inside shell script, i am having problems when... (1 Reply)
Discussion started by: sunrexstar
1 Replies

7. Shell Programming and Scripting

Replacing lines between two files with awk

Hello Masters, I have two subtitles file with different language like below First file : 1 00:00:41,136 --> 00:00:43,900 2 00:00:55,383 --> 00:00:58,477 <i> Ladies and gentlemen,</i> <i>this is Simon Barsinister,</i> 3 00:00:58,553 --> 00:01:00,521 <i>the wickedest man in the... (8 Replies)
Discussion started by: rk4k
8 Replies

8. Shell Programming and Scripting

inserting lines

Hi I would like to add lines to a file at specific locations. eg: If I have a file containing data... ABC DEF GHIJKLKMNOP RSTUVWXYZ and I have a requirement to insert a lines a) "LINE_FIRST" before first line in the file b) "LINE_X" before third line in the file c) "LINE_LAST"... (2 Replies)
Discussion started by: tostay2003
2 Replies

9. Shell Programming and Scripting

inserting a lines in a file

Hi folks, i need to insert the same set of lines between each line input lines 111111 aaaaaa 333333 output should be 111111 1 2 3 aaaaaa 1 2 3 333333 1 (2 Replies)
Discussion started by: Balaji Sukumara
2 Replies

10. UNIX for Dummies Questions & Answers

Inserting specific lines in vi

How can I insert, say lines 500 - 700 from another file into the current file on the current line (cursor) that I am editing while in vi (AIX). I know how to insert the entire file but how do you do it when you only need certain lines from a huge file? I've referenced my Unix Unleash book but... (2 Replies)
Discussion started by: giannicello
2 Replies
Login or Register to Ask a Question