I'm going madder than I was this morning :)


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting I'm going madder than I was this morning :)
# 1  
Old 06-03-2008
I'm going madder than I was this morning :)

I'm creating a lot of test data for some performance testing coming up. The vendor product I used to create the file had a slight bug in it and got some times wrong so I decided to use sed to fix it.

This works on the command line but not in a shell script and I can't work out why.

File to be changed. I've added the $ sign as the termnal isn't wide enough.

I want to change 039914, for eg, to 100001 and say 039915 to 102000

so as expected sed ''s/039914/100001/'' file.dat > file1.dat works and I get what I want.

In the shell script , given I have about 2,000 substitutions to make , I cat a file with the substitution
eg
039914 100001
039915 102000

Then the script, again dead simple
Code:
 
#!/bin/sh 
rm b1.dat
touch b1.dat
j=`wc -l cnt1.txt|awk '{FS=" "};{print $1}'`

cnt=1
while [ $cnt -le $j ] ; do
   match=`sed -n "$cnt p" cnt1.txt |awk '{FS=" "};{print $1}'`
   replace=`sed -n "$cnt p" cnt1.txt |awk '{FS=" "};{print $2}'`
   `sed -e "s/$match/$replace/g" file.dat >> b1.dat` 
#(I've tried a bazillion different ways to do this line and even setenv at the #command prompt works)
   cnt=`expr $cnt + 1`
done

As expected I should be running sed "s/039914/100001/" at the bold line but for some reason it removes everything up to ANZ rather than substituting . A -xv gives a weird sed command which is what is confusing me.

eg
+ sed -n 1 p cnt1.txt
replace=100100
echo 039914 100100
echo "Replacing 039914 with 100100"
+
/g file.dat 9914/100100

Code:
if I use single quote , eg , sed -e 's/${match}/${replace}/g' file.dat I get the correct command echo but th evariables don't expand

Any thoughts?

Data sample.
Code:
"000000" "ZZ"$
"0000"$
"000001" "TA"$
"0" "1" "039914" "ANZ" "   " "01" "0" "2103" "7777" "022000000" "000001000" "000002200000" "0490" "20080528" "43133
3" "EP              " "        " "20080602" "          " "N" "          " "          " "000000000000"$
"000002" "TA"$
"0" "1" "039915" "CBA" "   " "01" "0" "2103" "7777" "047000000" "000011000" "000051700000" "0490" "20080528" "43134
6" "EP              " "        " "20080602" "          " "N" "          " "          " "000000000000"$
... for a half million rows.

# 2  
Old 06-03-2008
If I got it right, you have problems with the variable substitution inside sed.
You can write it like this:
Code:
VAR="just"; echo "this is just a test" | sed 's/'$VAR'/not just/g'

# 3  
Old 06-03-2008
Quote:
Originally Posted by zaxxon
If I got it right, you have problems with the variable substitution inside sed.
You can write it like this:
Code:
VAR="just"; echo "this is just a test" | sed 's/'$VAR'/not just/g'

That wasn't quite the answer but got me thinking thanks Smilie

for some reason , not sure why, the backticks in the variable creation were interfering with the sed. Removing the backticks and suddenly it works.

I have no idea why though.
[code]
#!/bin/sh
rm b1.dat
touch b1.dat
j=`wc -l cnt1.txt|awk '{FS=" "};{print $1}'`

cnt=1
while [ $cnt -le $j ] ; do
match=`sed -n "$cnt p" cnt1.txt |awk '{FS=" "};{print $1}'`
replace=`sed -n "$cnt p" cnt1.txt |awk '{FS=" "};{print $2}'`
sed -e "s/$match/$replace/g" file.dat >> b1.dat
#(I've tried a bazillion different ways to do this line and even setenv at the #command prompt works)
cnt=`expr $cnt + 1`
done
Login or Register to Ask a Question

Previous Thread | Next Thread

5 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How do you schedule a job to run at 3:45 am every morning in UNIX?

give the suggestion (3 Replies)
Discussion started by: sonu pandey
3 Replies

2. Shell Programming and Scripting

How do you schedule a job to run at 3:45 am every morning?

help me (2 Replies)
Discussion started by: sonu pandey
2 Replies

3. UNIX for Dummies Questions & Answers

Slow UNIX system this morning

Hi, what would be the first things to check on a system that normally works fine, and is not so fine this morning ? Its accessing menus and various other screens 100x slower than normal. Version: UnixWare 5 7.1.3 i386 SCO UNIX_SVR5 I have tried this pf -ef|grep paulc and found a huge list of... (6 Replies)
Discussion started by: Mick_Dundee
6 Replies

4. IP Networking

Hi Good Morning

What do u mean by gateway not reachable? :confused: Please explain me with example Regards -Meti (3 Replies)
Discussion started by: ashokmeti
3 Replies

5. UNIX for Dummies Questions & Answers

How do you schedule a command to run at 4:00 every morning?

How do you schedule a command to run at 4:00 every morning? (1 Reply)
Discussion started by: JosephGerard
1 Replies
Login or Register to Ask a Question