Trying to use a sed command in a while loop


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Trying to use a sed command in a while loop
# 1  
Old 01-04-2010
Trying to use a sed command in a while loop

I have a file with just over 1 million lines. I am trying to write a script that will find 1400 instances of two particular patterns and insert a url on the 22nd line after the second pattern is matched. Each line in the text file corresponds to a particular check box or dropdown in the software so it is important that when the url is inserted no additional lines are added to the file or the software that uses the text file will be all messed up. When I manage to get the sed command to work it adds a line. When i get it to do what i want, it only replaces one instance. I've gone through a couple of tutorials and the man page but for some reason I can't wrap my head around what i'm doing wrong. Hope someone can help. Thanks in advance!

This is the script:
Code:
#!/bin/bash

cnt=0
while true
do
read -r fplan
if [[ -f "$fplan" || -f ../"$fplan" ]]
then
echo "File exists"
break
else
echo "Try again"
fi

((cnt+=1))
if [[ $cnt -eq 5 ]]
then
echo "exceeded 5 tries. quitting"
exit
fi
done
echo
echo "Please wait while the task completes"


sleep 5                                           

while read -r LINE
do                                                              
    if [[ "${LINE}" =~ "SystemDeclarationData" ]]               #  find the string SystemDeclarationData
    then
    echo "$LINE"
        read -r LINE  
        if [[ ${LINE} == 21* ]]                                 #  find the string that matches 21*.  If the matching string is the 
    then                                                    #+ very next line after SystemDeclarationData then insert the generic url
        echo "$LINE"                                    #+ on the 22nd line from the second matched pattern.  Repeat this for 1        
            sed '22 i\http:\/\/generic_url\g'                   #+ million plus lines.
    else echo "$LINE"
        fi
    else echo "$LINE"
    fi
done < "$fplan" > $PWD/complete                                                       
        



sleep 5
echo
echo
echo  "backing up original fplan"

cp "$fplan" "$fplan".bak

sleep 5
echo
echo "renaming the changed file to work with jcats"

mv $PWD/complete "$fplan"

sleep 3
echo
echo "operation completed"
echo
echo

This is a portion of the text file:

Code:
Type: OTHGoldUnitInfo;0
UNK

UNK

SEC

UNK

end;

8
Type: SystemDeclarationData;7
216120





U0010002372

DETAINEE MALE

















1
1
2385
-1
0
0
0
0
0
Type: SupplyLoad;4
0
0
end;

end;

Type: SystemDeclarationData;7
216063





U0010002373

DETAINEE MALE

















1
1
2386
-1
0
0
0
0
0
Type: SupplyLoad;4
0
0
end;

end;

Type: SystemDeclarationData;7
216061





U0010002374

DETAINEE MALE

















1
1
2387
-1
0
0
0
0
0
Type: SupplyLoad;4
0
0
end;

end;

Type: SystemDeclarationData;7
216060





U0010002375

DETAINEE MALE

















1
1
2388
-1
0
0
0
0
0
Type: SupplyLoad;4
0
0
end;

end;

Type: SystemDeclarationData;7
216059





U0010002376

DETAINEE MALE

















1
1
2389
-1
0
0
0
0
0
Type: SupplyLoad;4
0
0
end;

end;

Type: SystemDeclarationData;7
216058





U0010002377

DETAINEE MALE

















1
1
2390
-1
0
0
0
0
0
Type: SupplyLoad;4
0
0
end;

end;

Type: SystemDeclarationData;7
216057





U0010002378

DETAINEE MALE

















1
1
2391
-1
0
0
0
0
0

# 2  
Old 01-04-2010
Hi,

You seem to be mixing two things up. This:
Code:
sed '22 i\http:\/\/generic_url\g'

means insert the string before line 22 of the input file. 22 is an absolute number. Also there is no input to this command since you did not use a pipe into this command or specify a file name. Instead you should do 22 reads and echos and then I think you can just use:
Code:
echo 'http://generic_url'

instead.

Last edited by Scrutinizer; 01-04-2010 at 03:11 PM..
# 3  
Old 01-04-2010
You sir, are correct. I was so determined to get sed to work that I never even thought about that. Either way, as I understand it, the 22, being an absolute number, is what caused sed to quit working after the first change? It makes sense. I was thinking that since it was in a loop that it would replace the 22nd line after each found instance.
Thanks for taking the time to help me out!
# 4  
Old 01-04-2010
Could you please provide part of sample file, I have the feeling that your request can be replaced by one liner.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed and awk giving error ./sample.sh: line 13: sed: command not found

Hi, I am running a script sample.sh in bash environment .In the script i am using sed and awk commands which when executed individually from terminal they are getting executed normally but when i give these sed and awk commands in the script it is giving the below errors :- ./sample.sh: line... (12 Replies)
Discussion started by: satishmallidi
12 Replies

2. Shell Programming and Scripting

For loop with sed command

Hi Expert, I am working on one script which needs to read the input from the file and replace the words using sed command. For each line in the input file It should create one file. Below is the code. I can see the for loop is working fine but instead of three files it is creating one file ... (3 Replies)
Discussion started by: sharsour
3 Replies

3. UNIX for Dummies Questions & Answers

Using sed in a for loop

Hi, I have a file called 1.txt, I want to create 2.txt through 100.txt using a for loop. In each instance of the loop, I want to change all "1"'s in the text file to "2"s and so on... I tried the code below to do this: for i in {2..100}; do sed 's/1/$i/g' 1.txt > $i.txt; done but it... (1 Reply)
Discussion started by: evelibertine
1 Replies

4. UNIX for Dummies Questions & Answers

Using sed in a for loop

Hello I have a group of files a1.profile a2.profile a3.profile a4.profile b1.profile b2.profile b3.profile b4.profile These files all have the same first line with a value s1 atop the columns s1_context s1_ref s1_sample s1_% etc I am trying to use sed in a for loop to replace the s1 in the... (2 Replies)
Discussion started by: plumb_r
2 Replies

5. UNIX for Dummies Questions & Answers

grep sed and a loop

:wall: I have a requirement to search a log file that never rotates for certain values. If I find them I pipe them to a another file. To log file is constanyl being appened with new lines and never rotating Easy so far. The problem is I dont want to pipe out matches already seen before. ... (3 Replies)
Discussion started by: gunnahafta
3 Replies

6. Shell Programming and Scripting

Using sed in for loop

I have a file MAT.txt which contains the following data: mat1.txt mat2.txt . . . . mat100.txt I want to remove the '.txt' from every line and have an output file with the following data: mat1 mat2 . . . . mat100 I know this can be done with sed easily for each line, but I do... (8 Replies)
Discussion started by: ursaan
8 Replies

7. UNIX for Dummies Questions & Answers

SED inside while loop

Hi, im having problem creating a loop using my code: aside from the fact that the 1st variable (VAR) does not increment, it loops more than the expected output. for sample purposes, test csv contains 3 lines. #get number of lines in the file lines=$( wc -l < test.csv ) ... (5 Replies)
Discussion started by: paoie
5 Replies

8. Shell Programming and Scripting

For loop with sed

Hi there, I have 1 file with different 144 lines and 144 files that I want to change with a sed. What I want to do is to go trough the file with the 144 different lines take the line a replace a certain pattern with said in the first file of the folder where the 144 files are. Take the second... (3 Replies)
Discussion started by: sickboy
3 Replies

9. Shell Programming and Scripting

Loop with sed command to replace line with sed command in it

Okay, title is kind of confusion, but basically, I have a lot of scripts on a server that I need to replace a ps command, however, the new ps command I'm trying to replace the current one with pipes to sed at one point. So now I am attempting to create another script that replaces that line. ... (1 Reply)
Discussion started by: cbo0485
1 Replies

10. UNIX for Dummies Questions & Answers

sed in a for loop

dear guys, I want to replace the beginning of some lines in a file by (,) The sequence number of lines is located in a file named numbers so, i used this simple script to do so, but it didn't work. it does'nt recognize the $j in the command sed #! /bin/sh for j in `cat numbers` do sed '$j... (6 Replies)
Discussion started by: marwan
6 Replies
Login or Register to Ask a Question