Use of Variables in a sed/awk script


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Use of Variables in a sed/awk script
# 8  
Old 11-19-2013
Thanks that was exactly what I needed for the loop.

Code:
for i in {01..31}; do cat Cut_{1..6}${i}.txt > output_$i.txt; done

Is it possible to include several Awak command to have the cat ordered with the files every 5000 lines in the same loop in the same spirit as :

Code:
for i in {01..31}; do
awk 'NR==1{system("cat Cut_1$i.txt")}1' Fichier_Base.txt >   Fichier_Base_1.txt;
awk 'NR==5001{system("cat Cut_2$i.txt")}1' Fichier_Base_1.txt > Fichier_Base_2.txt;
...
awk 'NR==30001{system("cat Cut_7$i.txt")}1' Fichier_Base_6.txt > Statique_6oc_Cut_$i.txt;
rm Fichier_Base_{1..6};

Thanks for the
Code:
bash -vx

command it will greatly help me in my future script, I started using Unix 1 month ago only.
# 9  
Old 11-19-2013
If you want your cut... files to go to one single file, each starting at multiples of 5000 lines, above awk won't do the job. The system command adds them to the output file without altering the record counter, so you end up with cut(n+1) starting at line (5000 + cut(n)'s length).

See my post#3. Showing us your (failed?) attempts is fine, but to enable people to find good solutions, show where you come from and where you're going to, by what logic means.

EDIT: After some thought I need to revise my statement: As there are different awk instances running working on different files, your cut... files will be at positions (n x 5000) +1 in the final file. Still I think there will be a more efficient solution. Can't you make the cut... files 5000 lines in size when creating them?

---------- Post updated at 21:23 ---------- Previous update was at 18:23 ----------

You may want to try this:
Code:
for i in {1..9}     
  do
   awk  'FNR==1 && R    {for (i=L; i<TL; i++) print x}
                        {R=FNR}
         1
         END            {for (i=L; i<TL; i++) print x}
        ' TL=5000 cut_${i}{01..10}.txt > output_$i.txt
  done


Last edited by RudiC; 11-19-2013 at 04:01 PM..
This User Gave Thanks to RudiC For This Post:
# 10  
Old 11-20-2013
Making the input files 5000 lines long could make the trick. Then the
Code:
for i in {01..31}; do cat Cut_{1..6}${i}.txt > output_$i.txt; done

you gave me will match perfectly to obtain the good output.

I try to run your awk loop but I am getting syntax error or empty file, maybe I do not run with the correct package. I will look into awk command more deeply, it seems to be quite powerful especially when combined with a loop and condition.

Thanks a lot anyway for your helpful advice RudiC.
# 11  
Old 11-20-2013
Please post the syntax errors so we can help. What's your system (OS, awk version)?
# 12  
Old 11-20-2013
If i run :
Code:
for i in {1..3}     
  do
   awk  'FNR==1 && R    {for (i=L; i<TL; i++) print x}
                        {R=FNR}
         1
         END            {for (i=L; i<TL; i++) print x}
        ' TL=5000 Fatigue_Cut_${i}{00..02}.txt > output_$i.txt
  done

I get
Code:
$ ./Awk_Script.sh
./Awk_Script.sh: line 2: Erreur de syntaxe près du symbole inattendu « $'do\r' »
'/Awk_Script.sh: line 2: `  do

As for the version, I am running GNU Awk 4.0.1 on Cygwin.175. I installed a rxvt package with it.

Edit :
I managed to run it with
Code:
for i in {1..3};do awk  'FNR==1 && R    {for (i=L; i<TL; i++) print x}
                        {R=FNR}
         1
         END            {for (i=L; i<TL; i++) print x}
        ' TL=5000 Fatigue_Cut_${i}{00..02}.txt > output_$i.txt
  done

It seems to put 5000 ligns between two input files, the file length may vary but I can work it out under excel.

Last edited by Marc_Camoc; 11-20-2013 at 08:57 AM..
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Updating variables using sed or awk

Hi, I have a file(testfile.txt) that contains list of variables as shown below. T $$FirstName=James $$LastName=Fox $$Dateofbirth=1980-02-04 ……and so on there are 50 different variables. I am writing a script(script1.sh) that will update the above three variable one by one with the values... (6 Replies)
Discussion started by: Saanvi1
6 Replies

2. Shell Programming and Scripting

Variables into SED or AWK and multiple commands

Hello I am hoping you may help. I am not sure how to go about this exactly, I know the tools but not sure how to make them work together. I have two SED commands that I would like to run in a shell script. I would like to take the manual input of a user (types in when prompted) to be used... (4 Replies)
Discussion started by: lostincashe
4 Replies

3. Shell Programming and Scripting

print pattern between two variables awk sed

I am trying to print text between two variables in a file I have tried the following things but none seem to work: awk ' /'$a'/ {flag=1;next} /'$b'/{flag=0} flag { print }' file and also sed "/$a/,/$b/p" file But none seem to work Any Ideas? Thanks in Advance (5 Replies)
Discussion started by: forumbaba
5 Replies

4. Shell Programming and Scripting

sed command using variables in shell script

hi guys, The following command doesn't seem to work in my shell script: tag=$(sed -n '/${line}/ s/.*\.*/\1/p' myfile.txt) When i replace the ${line} with an actual value, it works fine. So, how do i use the ${line} in this sed command? Thanks in advance, Zaff (2 Replies)
Discussion started by: zaff
2 Replies

5. Shell Programming and Scripting

Using variables within awk/sed commands

Can I use my own variables within awk and sed for example: I've written a while loop with a counter $i and I want to use the value of $i within sed and awk to edit certain lines of text within a data file. I want to use : sed '1s/$/texthere/g' data.csv Like this: sed '$is/$/$age/g' data.csv... (5 Replies)
Discussion started by: mustaine85
5 Replies

6. Shell Programming and Scripting

Accessing Shell Variables in awk or sed

Hello, I wonder if it is possible to pass and use variables from shell environment into sed or awk. I am trying to achieve something similar to the following using sed or awk: var=some_regular_expression grep "$var" filename # Will extract lines from filename The following code,... (3 Replies)
Discussion started by: nasersh
3 Replies

7. Shell Programming and Scripting

using sed on bash variables (or maybe awk?)

Hi all- I've been fooling with this for a few days, but I'm rather new at this... I have a bash variable containing a long string of various characters, for instance: JUNK=this that the other xyz 1234 56 789 I don't know what "xyz" actually is, but I know that: START=he other and ... (2 Replies)
Discussion started by: rev66
2 Replies

8. Shell Programming and Scripting

passing variables to sed function in a script ....

Hello , I have a script named testscript.sh wherein I have two variables $var and $final (both of which contain a number) I have a sed write function inside this script as follows: sed '1,2 w somefile.txt' fromfile.txt Now , in the above i want to pass $var and $final instead of... (2 Replies)
Discussion started by: shweta_d
2 Replies

9. Shell Programming and Scripting

Manipulating awk $variables using sed?

I have been searching around the forums here trying to find a solution to my problem but not getting anywhere but closer to baldness. I have a 20 column pipe "|" seperated text file. The 14th variable doesnt always exist, but will have the format of YYYYMM or YYYY if it does. I need to take... (2 Replies)
Discussion started by: r0sc0
2 Replies

10. Shell Programming and Scripting

passing variables to sed inside script

I am trying to pass a regular expression variable from a simple script to sed to remove entries from a text file e.g. a='aaaa bbbb cccc ...|...:' then executing sed from the script sed s'/"'$a"'//g <$FILE > $FILE"_"1 my output file is always the same as the input file !! any... (5 Replies)
Discussion started by: Daniel234
5 Replies
Login or Register to Ask a Question