Using sed with a foreach loop


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using sed with a foreach loop
# 1  
Old 01-23-2009
Using sed with a foreach loop

So I am back again beating my head against the wall with a shell script and getting a headache! I want to change each year in a file (1980, 1981, 1982, 1983, etc.) to the same year followed by a tab.

The input is "blah blah (1980) blah blah".

I want to get "blah blah (1980 ) blah blah".

Code:
foreach year (1980 1981 1982)
sed -e 's/$year/$year     )/g' input_file > output_file

only changes the last year. Does sed need something special with foreach?

Thanks much to anyone in advance, Peggy

Last edited by bakunin; 01-23-2009 at 08:04 PM.. Reason: added code-tags. Please use them when posting code
# 2  
Old 01-23-2009
I don't know which shell you are using but this way it shouldn't work at all:

First, a "for"-loop looks like this:

Code:
for year in 1980 1981 1982 ; do
     sed ....
done

You might have a shell which can understand what you have written, but i suggest writing shell scripts which are really really portable. You might want to use a script you have written before for system x on system y too only to find that what works on system x will not work on system y because some specialities of system x you have used.

Don't get me wrong, this is a trap almost all beginners fall into, especially because they cannot distinguish between system-specific extensions and commonly available tools. This is not meant to embarrass you, just to help you starting proper scripting habits as soon as possible - you can't start too early doing it the right way.


Second, you repeatedly read from "inputfile" and write to "outputfile". You are overwriting in every pass what you have written in the last pass - this is why only the last change is visible to you. To have all the changes in your output use the following logic:

Code:
lastyear="input_file"
for year in 1980 1981 1982 ; do
     sed ..... "$lastyear" > "output_file.$year"
     lastyear="$year"
done
mv "output_file.$year" output_file      # <- remove intermediate files
rm -f output_file.198[012]              # <-

To better understand the logic go through it manually and write down the content of the "$year" and "$lastyear" variables as they change throughout the execution.

Third, you use single quotes to surround your sed script. While this is usually very sound practice because sed scripts tend to contain all sorts of characters which are special to the shell and need to be protected by single quotes in this case it is "protecting" the variable from being expanded as well. If you write

$year

the shell will expand this to the content of the variable "year", but if you write

'$year'

(note the single quotes) this is just literally the string "$year". Again, your shell might tolerate this and still expand "$year" to its value, but the standard shells will not. To be portable you better do not rely on this mechanism but use the following:

Code:
sed 's/'"$year"'/'"$year"' )/g' input_file > output_file

I know, this looks a bit awkward, but it will work in every standard shell (Bourne, Korn, Bourne Again, etc.) there is.

There are two last points, which are more optional: if you match some string in sed and want to use the complete matched string in the replacement you can use the "&" metacharacter. A second point is you could phrase the regular expression in a way that you would not need a loop at all. The following variant of the sed script will do the same as yours, but without the need of a shell loop (replace the "<tab>" with literal tab characters, of course):

Code:
sed 's/198[012]/&<tab>)/g' input_file > output_file

Even if you can't pack it into one regex you can execute several commands at once in a sed script:

Code:
sed 's/1980/&<tab>)/g
     s/1981/&<tab>)/g
     s/1982/&<tab>)/g' input_file > output_file


I hope this helps.

bakunin
# 3  
Old 01-24-2009
Using sed with a foreach loop

THANKS! Absolutely wonderful! Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Foreach loop with two variables

I need to put together a script that will take the contents of two different files (database name and database owner) and put them in two variables within a line: foreach x (`cat /local/hd3/dba/tools/build_db_scripts/dbs`) foreach z (`cat /local/hd3/dba/tools/build_db_scripts/dbas`)... (6 Replies)
Discussion started by: deneuve01
6 Replies

2. UNIX for Dummies Questions & Answers

foreach loop in csh

Hi everyone I'm new to unix and encountered a small problem i couldnt find out a reason why it doesn't work..please help.. in my csh script when i tried to use the foreach loop like this: foreach x ( ls ) echo $x end when i tried to run it, it printed out 'ls' to the std out instead of... (3 Replies)
Discussion started by: ymc1g11
3 Replies

3. Shell Programming and Scripting

foreach loop problem

Dear all, I wrote a script to download files and move files in directories according to their name. Now here is the problem: Both p101 and p360 data download successfully, but when I move them according to the year and month, only p101 data can be placed at the right location, p360,... (1 Reply)
Discussion started by: handsonzhao
1 Replies

4. UNIX for Dummies Questions & Answers

Using the Foreach loop, Needing help

I am trying to make a script for my Counter-Strike: Source servers. What i am wanting it to do is for it to restart each server, the only way i can think of doing this in through for each. Years what i have at the moment. server_start() { START=`ps x | grep SCREEN | grep $SRV | cut -d '?' -f... (5 Replies)
Discussion started by: grahamn95
5 Replies

5. UNIX for Advanced & Expert Users

Problem with foreach loop

Hi All, Is there any problem with the below 'foreach' loop? foreach risk_factor ($(cat "$rf_list")) where "rf_list=$SCRIPT/Utility/rflist.txt " I'm wondering, it is throwing below error message: syntax error at line 34: `(' unexpected Any idea/suggestions ? Thanks in advance /... (7 Replies)
Discussion started by: ganapati
7 Replies

6. Shell Programming and Scripting

foreach loop

Hi everyone Does anyone know what is wrong with this script. i keep getting errors foreach filename (`cat testing1`) set string=$filename set depth=`echo "$string" echo $depth end the error is the following testing: line 1: syntax error near unexpected token `(' testing: line 1:... (3 Replies)
Discussion started by: ROOZ
3 Replies

7. Shell Programming and Scripting

foreach loop + 2 variables

In a foreach loop, is it possible for the loop to go through 2 arguments instead of one i.e. instead of foreach i (do stuff for i), we have foreach i j(do stuff for i; do stuff for j) I am working under BASH and TCSH shell environments cheers (3 Replies)
Discussion started by: JamesGoh
3 Replies

8. Shell Programming and Scripting

foreach loop

Hi Guys, I have a loop which uses a wildcard i.e. foreach f (*) but when I execute the tcsh file in unix then it gives me an error ->>>>>>>foreach: words not parenthesized<<<<<<<<<<- Any help. (1 Reply)
Discussion started by: abch624
1 Replies

9. Shell Programming and Scripting

Foreach loop

What am I doing wrong with this foreach loop? foreach var ($argv) @sum = $sum + $var (4 Replies)
Discussion started by: haze21
4 Replies

10. UNIX for Dummies Questions & Answers

foreach loop question

Hello, I am new at this forum so please bare with me on this. Within a given directory, I have a list of files in which in each file, I would like to do a substitution. I would like to substitute the string mlcl to mll in each file using the foreach command. I dont quite get how to do that. If... (7 Replies)
Discussion started by: clipski
7 Replies
Login or Register to Ask a Question