Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Using a variable to select records with awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using a variable to select records with awk
# 1  
Old 09-26-2008
Power Using a variable to select records with awk
As part of a bigger task, I had to read thru a file and separate records into various batches based on a field. Specifically, separate records based on the value in the batch field as defined below. The batch field left-justified numbers.
The datafile is here
Code:
> cat infile
12345  1 John Smith   *
109    2 Joe Doe      *
1-9089 13Harry Can    *
14-12  5 Mary Will    *
43-4-5 13Kerry Lay    *
50505  1 Wanda Round  *

My script is here (and includes info on file layout)
Code:
> cat anal_bat
#! /usr/bin/bash
#
# analyze infile
#   acct_no 01-07
#   batch   08-09
#   name    10-22
#   extra   23

cnt=1
max=15
while [ $cnt -le $max ]
   do
   cnta=$(printf "%-2s" $cnt)
   echo $cnta

   awk 'substr($0,8,2)=="$cnta" {print}' <infile >outfile_${cnt}
   cnt=$((cnt+1))
done

This is all greatly simplified from my overall task which was then to perform data manipulations on the separated files. However, my small script above with the awk command merely creates 15 files that are empty.
So, what is wrong with the syntax?
# 2  
Old 09-26-2008
joeyg,

You need to pass the input variable value (cnta) to awk and use cnta instead of $cnta in awk,
for eg:
Code:
 awk -v cnta=$cnta 'substr($0,8,2)==cnta {print}' <infile >outfile_${cnt}

Last edited by manosubsulo; 09-26-2008 at 10:48 AM..
# 3  
Old 09-26-2008
Try this one, replace this line:

Code:
awk 'substr($0,8,2)=="$cnta" {print}' <infile >outfile_${cnt}

with:
Code:
awk -v var="$cnta" 'substr($0,8,2)==var {print}' <infile >outfile_${cnt}

Regards
# 4  
Old 09-26-2008
Another solution to this..
Code:
awk -v cnta=$cnta '$2==cnta {print $0}' infile 
 
instead of 
 
awk -v var="$cnta" 'substr($0,8,2)==var {print}' <infile

# 5  
Old 09-26-2008
Power I think Franklin52 got it
I tried the first suggestion, and it too did not quite separate the records. But the 2nd suggestion (thanks Franklin52) seemed to do the trick. I also fixed output filename to make the names easier to read (instead of _1 thru _15 I now create _01 thru _15)
Final code=

Code:
cnt=1
max=15
while [ $cnt -le $max ]
   do
   cnta=$(printf "%-2s" $cnt)
   cntt=$(printf "%.2d" $cnt)
   echo $cnta

#   awk 'substr($0,8,2)=="$cnta" {print}' <infile >outfile_${cnt}
#   awk -v cnta=$cnta 'substr($0,8,2)=="$cnta" {print}' <infile >outfile_${cnt}
   awk -v var="$cnta" 'substr($0,8,2)==var {print}' <infile >outfile_${cntt}
   cnt=$((cnt+1))
done

Finally, a look at the output shows that the records were put into the correct output files:
Code:
> grep "*" outfile*
outfile_01:12345  1 John Smith   *
outfile_01:50505  1 Wanda Round  *
outfile_02:109    2 Joe Doe      *
outfile_05:14-12  5 Mary Will    *
outfile_13:1-9089 13Harry Can    *
outfile_13:43-4-5 13Kerry Lay    *

which displays filename: and data records (remember, I placed a * at the end of every data record)
# 6  
Old 09-26-2008
Using egrep
Code:
egrep "^.{7}$cnta" infile > outfile_${cntt}

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Quick way to select many records from a large file

I have a file, named records.txt, containing large number of records, around 0.5 million records in format below: 28433005 1 1 3 2 2 2 2 2 2 2 2 2 2 2 28433004 0 2 3 2 2 2 2 2 2 1 2 2 2 2 ... Another file is a key file, named key.txt, which is the list of some numbers in the first column of... (5 Replies)
Discussion started by: zenongz
5 Replies

2. Shell Programming and Scripting

Select records and fields

Hi All I would like to modify a file like this: >antax gioq21 tris notes abcdefghij klmnopqrs >betax gion32 ter notes2 tuvzabcdef ahgskslsooin this: >tris abcdefghij klmnopqrs >ter tuvzabcdef ahgskslsoo So, I would like to remove the first two fields(and output field 3) in record... (4 Replies)
Discussion started by: giuliangiuseppe
4 Replies

3. Shell Programming and Scripting

To select non-duplicate records using awk

Friends, I have data sorted on id like this id addressl 1 abc 2 abc 2 abc 2 abc 3 aabc 4 abc 4 abc I want to pick all ids with addressesses leaving out duplicate records. Desired output would be id address 1 abc 2 abc 3 abc 4 abc (5 Replies)
Discussion started by: paresh n doshi
5 Replies

4. UNIX for Dummies Questions & Answers

Turning to SED to select specific records

Hi All, I am looking for a simple concise solution most likely using sed to process the following 4 rows of data from the same record and only keeps it if the second record satisfy certain critea such as surname matches up to smith or jackson: John (firstname) Smith (surname) ... (21 Replies)
Discussion started by: gjackson123
21 Replies

5. Shell Programming and Scripting

awk print only select records from file2

Print only records from file 2 that do not match file 1 based on criteria of comparing column 1 and column 6 Was trying to play around with following code I found on other threads but not too successful Code: awk 'NR==FNR{p=$1;$1=x;A=$0;next}{$2=$2(A?A:",,,")}1' FS=~ OFS=~ file1 FS="*"... (11 Replies)
Discussion started by: sigh2010
11 Replies

6. Shell Programming and Scripting

Block of records to select from a file

Hello: I am new to shell script programming. Now I would like to select specific records block from a file. For example, current file "xyz.txt" is containing 1million records and want to select the block of records from line number 50000 to 100000 and save into a file. Can anyone suggest me how... (3 Replies)
Discussion started by: nvkuriseti
3 Replies

7. UNIX for Dummies Questions & Answers

using unix variable in select column in awk

Hi, I have file on below pattern, and i want to write a generic awk to handle all cases. input_file: col1~col2~col3~col4~col5~col6 I need to generate 4 files something like this File1: col1~~col2~~col3 File2: col1~~col2~~col4 File3: col1~~col2~~col5 File4: col1~~col2~~col6 (1 Reply)
Discussion started by: luckybalaji
1 Replies

8. Shell Programming and Scripting

Automatically select records from several files and then run a C executable file inside the script

Dear list its my first post and i would like to greet everyone What i would like to do is select records 7 and 11 from each files in a folder then run an executable inside the script for the selected parameters. The file format is something like this 7 100 200 7 100 250 7 100 300 ... (1 Reply)
Discussion started by: Gtolis
1 Replies

9. Shell Programming and Scripting

Using awk to search variable length records

New to awk and need some help. I have a script that I would like to make more compact. I want to read a file and grab every field, from every record, except the last field. The records are variable length and have varying number of fields. A record will have at least two fields, but can have... (9 Replies)
Discussion started by: synergy_texas
9 Replies

10. UNIX for Dummies Questions & Answers

Select records based on search criteria on first column

Hi All, I need to select only those records having a non zero record in the first column of a comma delimited file. Suppose my input file is having data like: "0","01/08/2005 07:11:15",1,1,"Created",,"01/08/2005" "0","01/08/2005 07:12:40",1,1,"Created",,"01/08/2005"... (2 Replies)
Discussion started by: shashi_kiran_v
2 Replies
Login or Register to Ask a Question