AWK looping over 2 variables


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting AWK looping over 2 variables
# 1  
Old 09-12-2012
AWK looping over 2 variables

I would like to loop over variables i and j consecutively,
Code:
{ a = -6.7
  b =  7.0
  c =0.1
{ for (i = 0; i<=(b-a)/c; i++)
  for (j = 1; j<=(b-a)/c; j++)
  '$1<=(a+j*c)&&$1>=(a+i*c)' FILENAME > output_j
  '{print $2}' output_j > output_j_f
}

I essentially want to print the range of $1 when {i=0,j=1}, {i=1,j=2} etc.

Is this sort of loop possible? I am assuming FILENAME is the in-built variable for the file:
Code:
awk '{script}' file > output

# 2  
Old 09-12-2012
Maybe you had better tell us in English what you're trying to do.

Your awk statement:
Code:
awk '{script}' file > output

doesn't make use of the script you shown in the first section you have marked with code tags.

Assuming you intend for the 1st code segment to be used as your awk script,
you have a missing "}", a couple of very strange single quoted or unqouted strings (depending on how you intended to merge this code into your awk statement in the 2nd code segment, and some string comparisons against the undefined variables output_j and output_j_f that look like they may have been intended to redirect output instead of be used as string comparison operators.

Your combination of for statements without "{" and "}" means that the statement:
Code:
'{print $2}' output_j > output_j_f

is to be executed only once after the for loops terminate rather than being performed in the loop.

I can't make out what you were trying to do from these code segments, so I can't correct it for you without a description of what you're trying to do.
# 3  
Old 09-12-2012
I want to restrict the values in $1 to only those in interval (a+i*c) to (a+j*c).
For those values in that interval, print the output file with the name output_j where j denotes the value in the loop.

Then each output file I would like to open, read and print only $2.
Sorry, it's a bit hard to explain.

Code:
BEGIN { 
  a = -6.7
  b =  7.0
  c =0.1 
  output_j = "junk"
  output_j_f = "junk2"
}
{ for (i = 0; i<=(b-a)/c; i++)&&(j = 1; j<=(b-a)/c; j++) $1<=(a+j*c)&&$1>=(a+i*c) FILENAME > output_j } 
 print $2 output_j > output_j_f


Alternatively, discard the previous and just consider a loop of i = 0 to (7+6.7)/0.1 =137. Over this loop, I want to restrict $1 in interval (a+i*c) to (a+j*c) so essentially (-6.7 to -6.6), (-6.6 to -6.5) ... (6.9 to 7.0)
I want to print the restricted data for each iteration in loop.

Code:
BEGIN { 
  a = -6.7
  b =  7.0
  c =0.1 
  }
{ for (i = 0; i<=(b-a)/c; i++)&&$1<=(a+j*c)&&$1>=(a+i*c) {print $2 FILENAME > output }}


Last edited by chrisjorg; 09-12-2012 at 06:30 PM..
# 4  
Old 09-12-2012
There are still several syntax errors in your awk script and I'm still having trouble trying to guess what you want to get out of this script.

Please show us the contents of file and the results that you want to find in the files junk and junk2 when the script completes. If junk and junk2 aren't the files that you want to be produced as output from this script, please give actual filenames that you want to be produced when this script is run and exact output that should go into each file for the contents you give for file in your example.
# 5  
Old 09-12-2012
File

Code:
6.89659599594770e+00 -1.95793448063334e+03 3.26595536915786e+01
6.88084179683748e+00 -7.36059815620241e+02 3.26477026965960e+01
6.86470044063108e+00 -2.40724743710743e+02 3.26237500062772e+01
6.84925664789780e+00 -7.85069828802755e+01 3.26019223256409e+01
6.83607768359705e+00 -7.29586106449428e+01 3.25775558352855e+01
6.82530025473873e+00 -5.87717858943014e+01 3.25584906563908e+01
6.81468368213696e+00 -1.27968776811500e+02 3.25438302259661e+01
6.80314534950459e+00 -1.27432104807358e+02 3.25261886105316e+01
6.79141390234897e+00 -5.76431075044758e+01 3.25011722018034e+01
6.78070847032493e+00 -1.64218539638179e+01 3.24705013258269e+01
6.77015810502113e+00 -4.10975277987160e+01 3.24355393423734e+01
6.75993512746526e+00 -6.27501206897462e+01 3.24026880631920e+01
6.74979233306058e+00 -1.02027857429646e+01 3.23736095023893e+01
6.73810795452939e+00 -2.48659952136555e+00 3.23465573412226e+01
6.72647567793150e+00 -8.49339494040116e+01 3.23252288095938e+01
6.71270526837741e+00 -1.27698372249139e+02 3.23005211100514e+01
6.69833221734466e+00 -1.33127816927270e+01 3.22725313583402e+01
6.68380505032079e+00 5.97696297306984e+01 3.22415811510352e+01
6.66814935560922e+00 2.93466451379411e+01 3.22084406647545e+01
6.65250684944586e+00 6.09172433305864e+00 3.21856350819541e+01
6.63833676369360e+00 2.74886749397098e+01 3.21761595025029e+01
6.62480209051477e+00 6.96238677844523e+01 3.21697071299579e+01
6.61194487706185e+00 3.40170949789293e+01 3.21669839363518e+01
6.60136334858594e+00 -3.96531642581863e+01 3.21637735185563e+01
6.59353014806799e+00 -7.64867068182449e+01 3.21509977201048e+01
....
.....

.....

I want to awk
Code:
awk '$1<=-6.6&&$1>=-6.7' file > output_1

Code:
awk '$1<=-6.5&&$1>=-6.6' file > output_2

...
Code:
awk '$1<=7.0&&$1>=6.9' file > output_173

Then, for all these files, I would like to
Code:
awk '{print $2}' file_1 > file_1_f

Code:
awk '{print $2}' file_2 > file_2_f

Code:
awk '{print $2}' file_3 > file_3_f

...
Code:
awk '{print $2}' file_173 > file_173_f

That's why I set up the loop in the hope of doing this automated.

---------- Post updated at 06:43 PM ---------- Previous update was at 06:37 PM ----------

Code:
BEGIN {
  a = -6.7
  b =  7.0
  c =0.1
  output = "junk"
  }
END {for (i = 0; i<=(b-a)/c; i++); $1<=(a+j*c)&&$1>=(a+i*c); print $2 > output }

This script has no syntax errors, but I would like to output the data I have selected in each iteration of the loop and direct it to a unique output. How could I do that? At the moment, I only get one output file.
# 6  
Old 09-12-2012
Try this:

Code:
awk -F" " '{
  bucket=int($1*10+68)
  v[bucket]++
  l[bucket,v[bucket]]=$0
  f[bucket,v[bucket]]=$2
}
END {
  for(i=1;i<=137;i++) { 
    loutput="file_"i
    foutput="file_"i"_f"
    if(i in v)
      for(j=1;j<=v[i];j++) {
        print l[i,j] > loutput
        print f[i,j] > foutput
      }   
    close(loutput)
    close(foutput)
}}' infile

Edit: Typo above maximum bucket is 137 not 173

Last edited by Chubler_XL; 09-12-2012 at 10:26 PM..
This User Gave Thanks to Chubler_XL For This Post:
# 7  
Old 09-12-2012
Quote:
Originally Posted by chrisjorg
File

Code:
6.89659599594770e+00 -1.95793448063334e+03 3.26595536915786e+01
6.88084179683748e+00 -7.36059815620241e+02 3.26477026965960e+01
6.86470044063108e+00 -2.40724743710743e+02 3.26237500062772e+01
...

I want to awk
Code:
awk '$1<=-6.6&&$1>=-6.7' file > output_1
awk '$1<=-6.5&&$1>=-6.6' file > output_2
...
awk '$1<=7.0&&$1>=6.9' file > output_173

Then, for all these files, I would like to
Code:
awk '{print $2}' file_1 > file_1_f
awk '{print $2}' file_2 > file_2_f
awk '{print $2}' file_3 > file_3_f
...
awk '{print $2}' file_173 > file_173_f

That's why I set up the loop in the hope of doing this automated.

---------- Post updated at 06:43 PM ---------- Previous update was at 06:37 PM ----------

Code:
BEGIN {
  a = -6.7
  b =  7.0
  c =0.1
  output = "junk"
  }
END {for (i = 0; i<=(b-a)/c; i++); $1<=(a+j*c)&&$1>=(a+i*c); print $2 > output }

This script has no syntax errors, but I would like to output the data I have selected in each iteration of the loop and direct it to a unique output. How could I do that? At the moment, I only get one output file.
Unfortunately, the script above is functionally equivalent to:
Code:
END {print $2 > "junk" }

which only writes the 2nd field from the last line in your input file into a file named junk. But, I think understand what you're trying to do now.

To be sure that I do understand what you want, please confirm or correct the following statements:
  1. You want files named file_x for 1 <= x <= 173 which contain the copies of the lines from the file named file where the value in the first field in the line is in corresponding range.
  2. Then for each file named file_x you want a file named file_x_f that contains the same number of lines as the file_x file, but only contains the contents of the 2nd field of each line instead of the entire line.
  3. In your description above you sometimes talk about files named file_x and at other times talk about files named output_x. Am I correct in assuming that "output_" was a typo and you meant "file_"?
Is this correct?

Note that since you're creating up to 346 output files from this script, the script is going to have to open and close files while it is running rather than opening everything and letting awk automatically close them when the script terminates.

Please also answer the following questions:
  1. Do you want empty files created for files that don't have any lines that will be directed to those files?
  2. Do existing file_x and file_x_f files need to be removed when this script starts?
  3. If not, should lines to be written by this script replace the contents of existing files or append lines to them?
I'm hoping that you either want all existing files to be removed or overwritten by the script rather than appending to existing files. The file handling logic is much more difficult in an awk script if you want to portably append to existing files. Given the script: print >> file_x some systems will create file_x if it doesn't already exist. Others will only create a file when using print > file_x and will give an error if you try print >> file_x when file_x doesn't already exist.

The script Chubler_XL provided in the message before this should work fine as long as you don't care about the order in which lines appear in the output files and don't want to append to existing files. If you want to append rather than replace, or if you want to have all entries in the output files be in the same order that they appeared in the input file, but script will be more complex.

================
I apologize. Chubler_XL's script does indeed maintain order, and (as he said) you can just replace > with >> if you want to append rather than overwrite. (It is w >> file in ex that may fail if file doesn't already exist. In awk >> file is guaranteed to create the file if it didn't exist and append to it if it did exist.)

Last edited by Don Cragun; 09-13-2012 at 01:27 AM.. Reason: Corrected mistaken comment
This User Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk nested looping?

I am trying to parse a text file and send its output to another file but I am having trouble conceptualizing how I am supposed to do this in awk. The text file has a organization like so: Name Date Status Location (city, state, zip fields) Where each of these is on a separate line in... (1 Reply)
Discussion started by: kellyanneghj
1 Replies

2. Shell Programming and Scripting

Help on looping using awk

I have the data like this: PONUMBER,SUPPLIER,LINEITEM,SPLITLINE,LINEAMOUNT,CURRENCY IR5555,Supplier1,1,1,83.1,USD IR5555,Supplier1,1,3,40.4,USD IR5555,Supplier1,1,6,54.1,USD IR5555,Supplier1,1,8,75.1,USD IR5556,Supplier2,1,1,41.1,USD IR5556,Supplier2,1,3,43.1,USD ... (3 Replies)
Discussion started by: jeffreybsu
3 Replies

3. Shell Programming and Scripting

looping in awk

How do I remove last comma? echo "xx yy zz" | awk 'BEGIN{FS=" "}{for (i=1; i<=NF; i++) printf "%s,", $i}'output: xx,yy,zz, required output: xx,yy,zz or (ideally!): xx, yy & zz many thanks in advance! (4 Replies)
Discussion started by: euval
4 Replies

4. UNIX for Dummies Questions & Answers

Help with AWK looping

I'm trying to parse a configuration text file using awk. The following is a sample from the file I'm searching. I can retrieve the formula and recipe names easily but now I want to take it one step farther. In addition to the formula name, I would like to also get the value of the attribute... (6 Replies)
Discussion started by: new2awk
6 Replies

5. Shell Programming and Scripting

Urgent - Looping using AWK

Hi I have a file which is having following text. The file is in a tabular form with 5 fields. i.e field1, field2 ..... field5 are its columns and there are many rows in it say COUNT is the number of rows Field 1 Field2 Field3 Field4 Field5 ------- ------- ... (8 Replies)
Discussion started by: skyineyes
8 Replies

6. Shell Programming and Scripting

Looping script with variables

If I have a file with a bunch of various numbers in one column, how can I make a script to take each number in the file and put in into a command line? Example: cat number_file 2 5 8 11 13 34 55 I need a loop to extract each of these numbers and put them into a command line... (1 Reply)
Discussion started by: jojojmac5
1 Replies

7. Shell Programming and Scripting

looping and awk/sed help

I am pretty new to this, but imagine what I am trying to do is possible iI am trying to make an automated DB comparison tool that selects all columns in all tables and compares them to the same thing in another DB. anyway I have created 2 files to help with this the first file is a... (13 Replies)
Discussion started by: Zelp
13 Replies

8. Shell Programming and Scripting

Awk: looping problem!

I am having a problem with awk when I run it with a loop. It works perfectly when I echo a single line from the commandline. For example: echo 'MFG009 9153852832' | awk '$2 ~ /^0-9]$/{print $2}' The Awk command above will print field 2 if field 2 matches 10 digits, but when I run the loop... (5 Replies)
Discussion started by: cstovall
5 Replies

9. Shell Programming and Scripting

looping through variables

Hi, I tired to do this in a korn shell on an HP-UX 9000/800... var1="a b c" var2="d e f" vars="var1 var2" for i in $vars do for j in $i do echo $i $j done done When run, this would output var1 var1 (1 Reply)
Discussion started by: andyfaeglasgow
1 Replies

10. UNIX for Advanced & Expert Users

Looping in awk

Can somebody give me a cleaner way of writing the following script. I was thinking that I could use a loop in the awk statement. It works fine the way it is but I just want the script to be cleaner. #!/usr/bin/sh for r in 0 1 2 3 4 5 6 do DAY=`gdate --date="${r} days ago" +%m\/%d\/%y`... (3 Replies)
Discussion started by: keelba
3 Replies
Login or Register to Ask a Question