Use of Variables in a sed/awk script


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Use of Variables in a sed/awk script
# 1  
Old 11-18-2013
Use of Variables in a sed/awk script

Hi,


After looking at the differents post on this forum, I am convinced that I will benefit from the experience of advanced Unix user on some script I have already done for an aeronautical study. Here is one of them :

Step 1 :


Code:
sed -e "s/??/00/g" Base_Awk.txt > Awk_Cut_00.txt4;
sed -e "s/??/01/g" Base_Awk.txt > Awk_Cut_01.txt4;
sed -e "s/??/02/g" Base_Awk.txt > Awk_Cut_02.txt4;
cat *.txt4 > Awk_Cut_All.sh;rm *txt4;

with Base_Awk.txt :

Code:
awk 'NR==1{system("cat Cut_1??.txt")}1' Fichier_Base.txt >   Fichier_Base_1.txt;
awk 'NR==5001{system("cat Cut_2??.txt")}1' Fichier_Base_1.txt > Fichier_Base_2.txt;
...
awk 'NR==30001{system("cat Cut_7??.txt")}1' Fichier_Base_6.txt > Statique_6oc_Cut_??.txt;

Step 2
Code:
./Awk_Cut_All.sh

To summarize, I have 7 x 70 files Cut_1??.txt,Cut_2??.txt,...,Cut_7??.txt and the output is 70 files ordered with the awk command (line 1 file Cut_101.txt line 5001 Cut_2??.txt and so on to Cut_7??.txt line 30001).

I have obtained this results with the code above, I was wondering if i could bypass the script with the sed replacement and using variables in the awk command, something like that (I know it won(t work just for the idea) :

Code:
i=[1-70];
for i in [1-70];
do 
awk 'NR==1{system("cat Cut_1$I.txt")}1' Fichier_Base.txt > Fichier_Base_1$i.txt
...
awk 'NR==1{system("cat Cut_7$I.txt")}1' Fichier_Base.txt > Fichier_Base_7$i.txt
cat Fichier_Base_?$i.txt > output_$i.txt5;

I have seen on this forum the use of variable in Awk but have been unable to use it for this script.

If anyone can lend me a hand on this one, I would be much grateful.

Marc
# 2  
Old 11-18-2013
The easiest way to use shell variables in an awk script is to pass them in:
Code:
export SHELLVAR="Pie"; echo | awk -vAWKVAR="$SHELLVAR" '{print AWKVAR}'


Last edited by CarloM; 11-18-2013 at 10:15 AM..
This User Gave Thanks to CarloM For This Post:
# 3  
Old 11-18-2013
Why don't you run the loop within awk to be more efficient?

I'm sorry I don't absolutely understand what you are heading for, but I'd bet there would be more efficient ways to achieve your goals. We could help if you'd post sample input and desired output files and the logics that connect them.
This User Gave Thanks to RudiC For This Post:
# 4  
Old 11-19-2013
I have attached the maximum of 5 files per post but it will be enough I hope.

The Cut_?0?.txt files are the input, as output for this example let's say I want the Desired_Output_Cut_01(Cut_101_Cut_201).txt file and another which would correspond to Cut_02 with Cut_102.txt and Cut_202.txt at line 50 concatenated.

In fact, the first digit match an envelop of loadcase (?01), the last two digit correspond to a cut to obtain the flux in the area (1??). On the real study, I have obtained the correct output with the scripts enclosed in the first post but as RudiC pointed out a loop like the one described in the end of my first post is the code I would like to see working. It might spare me a lot of time. I have not been able to use variable in the loop (Thanks CarloM for the syntax).
# 5  
Old 11-19-2013
Sorry, still not clear. Your Desired_Output_Cut_01(Cut_101_Cut_201).txt is just a concatenation of two of the input files. You don't need awk for that.
If I read your approaches in post#1 correctly, you have a file and want the cut- files inserted every 5000 lines? How's all that connected to Base_Awk.txt and Fichier_Base.txt?
# 6  
Old 11-19-2013
Re-reading myself, I reckon I have been difficult to understand.
As Input, let's say I have 90 files named Cut_??.txt (with ?? matching the cut from 01 ->10 and matching the envelop from 1 ->9).

I want as output; 10 files one per cut and in each one a cat of the different envelop with the awk command to have the different envelop at the correct line.

I did that with the sed script and the Base_Awk.txt of Step 1, the Fichier_Base.txt file is an empty file with 50 000 line because I did not manage to insert a file at line 25000 for example when the input file was not 25000 line long.
# 7  
Old 11-19-2013
If you are using bash, try this (set -vx options to see what happens):
Code:
for i in {1..9}; do cat cut_${i}{01..10}.txt > output_$i.txt; done

This User Gave Thanks to RudiC For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Updating variables using sed or awk

Hi, I have a file(testfile.txt) that contains list of variables as shown below. T $$FirstName=James $$LastName=Fox $$Dateofbirth=1980-02-04 ……and so on there are 50 different variables. I am writing a script(script1.sh) that will update the above three variable one by one with the values... (6 Replies)
Discussion started by: Saanvi1
6 Replies

2. Shell Programming and Scripting

Variables into SED or AWK and multiple commands

Hello I am hoping you may help. I am not sure how to go about this exactly, I know the tools but not sure how to make them work together. I have two SED commands that I would like to run in a shell script. I would like to take the manual input of a user (types in when prompted) to be used... (4 Replies)
Discussion started by: lostincashe
4 Replies

3. Shell Programming and Scripting

print pattern between two variables awk sed

I am trying to print text between two variables in a file I have tried the following things but none seem to work: awk ' /'$a'/ {flag=1;next} /'$b'/{flag=0} flag { print }' file and also sed "/$a/,/$b/p" file But none seem to work Any Ideas? Thanks in Advance (5 Replies)
Discussion started by: forumbaba
5 Replies

4. Shell Programming and Scripting

sed command using variables in shell script

hi guys, The following command doesn't seem to work in my shell script: tag=$(sed -n '/${line}/ s/.*\.*/\1/p' myfile.txt) When i replace the ${line} with an actual value, it works fine. So, how do i use the ${line} in this sed command? Thanks in advance, Zaff (2 Replies)
Discussion started by: zaff
2 Replies

5. Shell Programming and Scripting

Using variables within awk/sed commands

Can I use my own variables within awk and sed for example: I've written a while loop with a counter $i and I want to use the value of $i within sed and awk to edit certain lines of text within a data file. I want to use : sed '1s/$/texthere/g' data.csv Like this: sed '$is/$/$age/g' data.csv... (5 Replies)
Discussion started by: mustaine85
5 Replies

6. Shell Programming and Scripting

Accessing Shell Variables in awk or sed

Hello, I wonder if it is possible to pass and use variables from shell environment into sed or awk. I am trying to achieve something similar to the following using sed or awk: var=some_regular_expression grep "$var" filename # Will extract lines from filename The following code,... (3 Replies)
Discussion started by: nasersh
3 Replies

7. Shell Programming and Scripting

using sed on bash variables (or maybe awk?)

Hi all- I've been fooling with this for a few days, but I'm rather new at this... I have a bash variable containing a long string of various characters, for instance: JUNK=this that the other xyz 1234 56 789 I don't know what "xyz" actually is, but I know that: START=he other and ... (2 Replies)
Discussion started by: rev66
2 Replies

8. Shell Programming and Scripting

passing variables to sed function in a script ....

Hello , I have a script named testscript.sh wherein I have two variables $var and $final (both of which contain a number) I have a sed write function inside this script as follows: sed '1,2 w somefile.txt' fromfile.txt Now , in the above i want to pass $var and $final instead of... (2 Replies)
Discussion started by: shweta_d
2 Replies

9. Shell Programming and Scripting

Manipulating awk $variables using sed?

I have been searching around the forums here trying to find a solution to my problem but not getting anywhere but closer to baldness. I have a 20 column pipe "|" seperated text file. The 14th variable doesnt always exist, but will have the format of YYYYMM or YYYY if it does. I need to take... (2 Replies)
Discussion started by: r0sc0
2 Replies

10. Shell Programming and Scripting

passing variables to sed inside script

I am trying to pass a regular expression variable from a simple script to sed to remove entries from a text file e.g. a='aaaa bbbb cccc ...|...:' then executing sed from the script sed s'/"'$a"'//g <$FILE > $FILE"_"1 my output file is always the same as the input file !! any... (5 Replies)
Discussion started by: Daniel234
5 Replies
Login or Register to Ask a Question