Shell Variables passed to awk to return certain rows


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell Variables passed to awk to return certain rows
# 1  
Old 06-29-2015
Shell Variables passed to awk to return certain rows

Hi Forum.

I have the following test.txt file and need to extract certain rows based on "starting position", "length of string" and "string to search for":

Code:
1a2b3d
2a3c4d
.....

My script accepts 3 parameters: (starting col pos, length to search for, string to search for) and would like to pass these parameters for awk to extract the records.

Running the script as:

Code:
script 2 3 a2b test.txt

would return the first record from the file:
Code:
1a2b3d


Code:
 awk  -v search_col_pos=$search_col_pos search_str_len=$search_str_len string_to_search=$string_to_search 'substr($0, search_col_pos, search_str_len) == "string_to_search"' test.txt

where $search_col_pos, $search_str_len, $string_to_search are passed as input parameters to the script.

but code is not working and is returning the following error:

Code:
awk: fatal: cannot open file

Please help.

Thanks.
Moderator's Comments:
Mod Comment Please use CODE tags for all sample input, sample output, and sample code segments.

Last edited by Don Cragun; 06-29-2015 at 08:04 PM.. Reason: Add CODE tags.
# 2  
Old 06-29-2015
I am quite sure there would be an easier way of doing what you are trying to do, if you would have provide some real input and output files. Nevertheless, I would like to point out the following:

Code:
-v search_col_pos=$search_col_pos -v search_str_len=$search_str_len -v string_to_search=$string_to_search

Code:
 'substr($0, search_col_pos, search_str_len) == "string_to_search"' test.txt

What you are saying here is: if that evaluates to true (any value not a 0) print the default, which is $0
# 3  
Old 06-29-2015
Quote:
Originally Posted by Aia
I am quite sure there would be an easier way of doing what you are trying to do, if you would have provide some real input and output files. Nevertheless, I would like to point out the following:

Code:
-v search_col_pos=$search_col_pos -v search_str_len=$search_str_len -v string_to_search=$string_to_search

Code:
 'substr($0, search_col_pos, search_str_len) == "string_to_search"' test.txt

What you are saying here is: if that evaluates to true (any value not a 0) print the default, which is $0
This is a sample of what the data looks like:
Code:
H2015051303:52:46CUSTOMER LEVEL DETAIL   8987DET





B2015051303:52:463570





D3570                          1057            3570 000000000000040000C5360885903549753         00189DEBIT CASH BAL-REVERSAL                  0189DEBITCASHBAL-REVERSA 75360885132132
000004319CAD20150512201505128987CTPC NE0000000012  89878888880060              03                      CAD0000000000000400000000000245100




D3570                          1057            3570 000000000000040000D5360884924480379         00184CREDIT TO PURCHASE BAL-REVERSAL          CREDITTOPURCHASEBAL-REVE 75360885132132
000004137CAD20150512201505128987CTPC NE0000000012  89878888880060              03                      CAD0000000000000400000000000245300

...

thanks for pointing out the missing -v, this is what the updated code looks like but it still doesn't work as coded.

Code:
awk -v search_col_pos=$search_col_pos -v search_str_len=$search_str_len -v segment_type=$segment_type 'substr($0, search_col_pos, search_str_len) == "segment_type"' test.txt

If I hard code "D" as segment_type in the awk command, it works but not when I use the variable segment_type.

Essentially I want to extract all records with a "D" in certain column position for a length of one in this example.

I have other files that the search string can be in any position and be 4 characters long.
# 4  
Old 06-29-2015
You need a -v option for each variable you're setting; not just the first one.

If there is any chance that string_to_search could contain any whitespace characters or characters that have special meaning to the shell, it will also need to be quoted.

It is generally a good idea to quote all shell variable expansions, but I will assume for now that your script has already verified that $search_col_pos and $search_str_len expand to numeric strings.

And, inside an awk script, putting a variable name in quotes will try to match against the name of the variable instead of the contents of the variable.

Try:
Code:
awk -v search_col_pos=$search_col_pos -v search_str_len=$search_str_len -v string_to_search="$string_to_search" 'substr($0, search_col_pos, search_str_len) == string_to_search' test.txt

If that still doesn't work, show us what happens with the above corrections.

Last edited by Don Cragun; 06-29-2015 at 09:55 PM.. Reason: Fix typos.
This User Gave Thanks to Don Cragun For This Post:
# 5  
Old 06-29-2015
Hi Don.

Your solution worked perfectly!!!

thanks.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script to create runtime variables based on the number of parameters passed in the script

Hi All, I have a script which intends to create as many variables at runtime, as the number of parameters passed to it. The script needs to save these parameter values in the variables created and print them abc.sh ---------- export Numbr_Parms=$# export a=1 while do export... (3 Replies)
Discussion started by: dev.devil.1983
3 Replies

2. UNIX for Dummies Questions & Answers

Multiple variables to be passed in a loop

Hi, I need to pass the multiple values of src1 to another variable. I managed to print it but not sure how to assign it to a variable in a loop. src1=01,02,03 echo $src1|awk 'BEGIN {FS=","} {for(i=1;i<=NF;i++) print $i}' I need to pass the value as src2=01 src2=02 src2=03 Thanks... (4 Replies)
Discussion started by: shash
4 Replies

3. Shell Programming and Scripting

Multiple bash variables passed into nawk

I have a file that has 2 fields called b_file: 11977 DAR.V3.20150209.1.CSV 3295 DAR.V3.20150209.1.CSV 1721 DAR.V2.20150210.1.CSV I need to search a sftplog using the field 1, but want to maintain the relationship between field 1 and 2. I am passing field 1 as a parameter in a bash loop. ... (14 Replies)
Discussion started by: smenago
14 Replies

4. Shell Programming and Scripting

awk - Why can't value of awk variables be passed to external functions ?

I wrote a very simple script to understand how to call user-defined functions from within awk after reading this post. function my_func_local { echo "In func $1" } export -f my_func_local echo $1 | awk -F"/" '{for (k=1;k<=NF;k++) { if ($k == "a" ) { system("my_local_func $k") } else{... (19 Replies)
Discussion started by: sreyan32
19 Replies

5. Shell Programming and Scripting

awk processing of passed variables

Currently have this: set current=192.168.0.5 set servicehost = `echo $current | awk -F. '{print $4}'` echo $numberoffields 5 ..but would like to reduce # of variables and eliminate echo to have something like this: set servicehost = `awk -v s="$current" -F. 'BEGIN{print $2}'`But... (3 Replies)
Discussion started by: Mid Ocean
3 Replies

6. Shell Programming and Scripting

awk - take variables out to shell

Hi, How could we take the value of awk variables out to shell? I know the following methods 1. awk '{print $1}' < file | read a echo $a 2. a=`awk '{print $1}' < file` echo $a Please let me know if there are any other methods. Also, how do we take more than 1 variable value... (4 Replies)
Discussion started by: Thumban
4 Replies

7. Shell Programming and Scripting

AWK: Retrieving names of variables passed with -v

I'm an experienced awk user, but this one has me stumped. I have an awk script which is called from a UNIX command line as you'd expect: myscript.awk -v foo=$1 -v bar=$2 filename My question is this: is there a mechanism for determining the names of the -v variables within a script? ... (3 Replies)
Discussion started by: John Mac
3 Replies

8. Shell Programming and Scripting

Return Awk Variable to Shell

I'm a bit stuck in getting variable from awk to shell. I tried searching but most of them showing to assign to shell variable via.. VAR=`echo $line | awk -F: '{print $1}'` which is correct ofcourse My problem is multiple assignments of variable like this one. The above solution will give... (10 Replies)
Discussion started by: ryandegreat25
10 Replies

9. Shell Programming and Scripting

Awk, shell variables

Hello, I've been trying to figure out how to use variables inside the AWK command and use it back in the korn shell sript. in my script I have lots of awk commands like this grep Listen /etc/ssh/sshd_config | \ awk '{ if ($2 == "22" ) print "OK"; else print "not OK" }' ... (3 Replies)
Discussion started by: mirusko
3 Replies

10. Shell Programming and Scripting

Using shell variables In awk

Oh its not my day for syntax... cat gzipsize.txt | awk '{print "echo",$1,$2} > master.txt I have read a lot about the awk -v but haven't been able to get it to work. I have a variable in my script and I'm looking just to push it into the awk after the $2 (or anywhere would do)!!! Every... (11 Replies)
Discussion started by: nortypig
11 Replies
Login or Register to Ask a Question

Featured Tech Videos