"Bash variable expansion in awk script"

Post #303036301 by yifangt on Friday 21st of June 2019 04:59:54 PM

Bash variable expansion in awk script

Hello,
I need to split a file into two of different locations by re-direction in awk.
Code:
cat infle
aaa     1 3
bbb     2 4
aaa     3 3
bbb     4 4
aaa     5 3
bbb     6 4

cat /storage/tmp/group_a.gtf
aaa     1 3
aaa     3 3
aaa     5 3

cat /storage/tmp/group_b.gtf
bbb     2 4
bbb     4 4
bbb     6 4

The path of the resulted files should be passed in with shell variable, and I have tried:
Code:
MY_DIR="/storage/tmp"
awk -v MY_DIR="${MY_DIR}" '{if ($1 == "aaa")  {print $0 >> "${MY_DIR}/group_a.gtf"} else if ($1 == "bbb") {print  $0 >> "${MY_DIR}/group_b.gtf"}}' infle  #A typo was $7=="bbb"
MY_DIR="/storage/tmp";  awk '{if ($1 == "aaa") {print $0 >>  ENVIRON[\"MY_DIR\"]/group_a.gtf"} else if ($1 == "bbb") {print $0  >> "ENVIRON[\"MY_DIR\"]/group_b.gtf"}}' infle
awk -v  awk_var=${MY_DIR} '{if ($1 == "aaa") {print $0 >>  "awk_var/group_a.gtf"} else if ($1 == "bbb") {print $0 >>  "awk_var/group_b.gtf"}}' infle

but none worked with error like: cmd. line:1: (FILENAME=aaa FNR=1) fatal: can't redirect to `awk_var/group_a.gtf' (No such file or directory)

I checked the threads in this forum:
How to pass a variable from shell to awk
Pass shell Variable to awk
but they did not help me out probably due to my expansion is related to file directory. However, the it worked if the path is hard-coded, which is not thru shell variable along with other part of the shell script.
Code:
awk  '{if ($1 == "aaa") {print $0 >> "/storage/tmp/group_a.gtf"} else  if ($1 == "bbb") {print $0 >> "/storeage/tmp/group_b.gtf"}}'  infle

Also It is interesting with following oneliner, in which the expansion worked at every other hit of ENTER. Could not understand why.
Code:
MY_DIR="/storage/tmp"; awk -v awk_var="$MY_DIR" '{print awk_var}'

/storage/tmp

/storage/tmp

/storage/tmp

 ......

1) What are the rules to expand shell (bash) variables (especialy related to directory) inside awk script?
2) Why my last oneliner worked every other time with ENTER? Thanks a lot!

Last edited by yifangt; 06-24-2019 at 01:22 PM..
This User Gave Thanks to yifangt For This Post:
 
Test Your Knowledge in Computers #180
Difficulty: Medium
The Apple II was invented by Steve Jobs.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

globbing, $# is too high after wildcard expansion in bash script

How can I pass in an argument such as "*.k" to a bash script without having to double-quote *.k and not having *.k `glob` to match all files in the pattern? I tried using noglob in my script but this didn't work the way I thought it would.. expansion is still occuring, $# is higher than I... (3 Replies)
Discussion started by: zoo591
3 Replies

2. Shell Programming and Scripting

bash - delay expansion of variable

Hello - I have a bash script which does some logging, and I'd like to include the line number of the echo statement that pipes into $LOGGER: MYPID=$$ MYNAME=`basename $0` LOGGER="/usr/bin/logger -t $MYNAME($LINENO) -p daemon.error" ... echo 'this is an entry into the log file' | $LOGGER ... (3 Replies)
Discussion started by: scandora
3 Replies

3. Shell Programming and Scripting

Brace expansion problem in Bash

I have a script that takes an option for server pools to run the script against. The option is given as a comma separated list (ie, -p 201,204,301). I'm using eval and brace expansion to get those pool numbers into an array. It works fine unless only 1 pool number is given. Here's the code: ... (5 Replies)
Discussion started by: mglenney
5 Replies

4. Shell Programming and Scripting

Bash variable delayed expansion?

i write a batch file , here is the content. dirname='date +%Y-%m-%d' mkdir dirname but it doen's work, it just create a folder named date and +%Y-%m-%d. i have tried run the command seperately in the bash prompt. after the first sentence executed , i use $dirname to watch the value of... (4 Replies)
Discussion started by: premotheus
4 Replies

5. Shell Programming and Scripting

Bash shell expansion help

This is what I have in my directory. $ ls test1.txt test2.txt test3.txt test4.txt test5.txt test_script.sh This is what my shellscript looks like. #!/bin/bash for filename in /shell_expansion/*.txt; do for ((i=0; i<=3; i++)); do echo "$filename" ... (5 Replies)
Discussion started by: cokedude
5 Replies

6. Shell Programming and Scripting

Shell variable expansion in awk

I want to split one file input.tab into two separate ones, odd lines to input_reads1.txt, even lines to input_reads2.txt for a serial of files with similar name pattern. Also I want to "match" input/output file names to keep consistency of file name: CSEL_02.0_input.tab CSEL_03.4_input.tab... (2 Replies)
Discussion started by: yifangt
2 Replies

7. Shell Programming and Scripting

Bash expansion

Hello. I cannot write a command without using eval. Any help is welcome Note 1 : What does the function SOMETHING has no importance. Note 2 : What does the command find has no importance. It is an expansion variable problem : where to put or or or anythings else What works (FILTRE_1... (8 Replies)
Discussion started by: jcdole
8 Replies

8. Shell Programming and Scripting

Bash variable expansion

Hello. The file /etc/fstab contains UUID=957c3295-9944-1593-82e2-2b90dede4312 / ext4 noatime,discard,acl,user_xattr 1 1 I fill a variable SOME_LINE=$( cat /etc/fstab | grep \/\..*ext4 | grep noatime,discard )echo $SOME_LINE... (3 Replies)
Discussion started by: jcdole
3 Replies

9. Shell Programming and Scripting

Bash Parameter Expansion

I have made the following examples that print various parameter expansions text: iv-hhz-sac/hpac/hhz.d/iv.hpac..hhz.d.2016.250.070018.sac (text%.*): iv-hhz-sac/hpac/hhz.d/iv.hpac..hhz.d.2016.250.070018 (text%%.*): iv-hhz-sac/hpac/hhz (text#*.): d/iv.hpac..hhz.d.2016.250.070018.sac... (2 Replies)
Discussion started by: kristinu
2 Replies

10. Shell Programming and Scripting

Bash Parameter Expansion

#!/bin/bash SNMPW='/usr/bin/snmpwalk' while read h i do loc=$($SNMPW -v3 -u 'Myusername' -l authPriv -a SHA -A 'Password1' -x AES -X 'Password2' $i sysLocation.0 2>/dev/null) loc=${loc:-" is not snmpable."} loc=${loc##*: } loc=${loc//,/} echo "$i,$h,$loc" done < $1 My question is ... ... (1 Reply)
Discussion started by: sumguy
1 Replies
IGAWK(1)							 Utility Commands							  IGAWK(1)

NAME
igawk - gawk with include files SYNOPSIS
igawk [ all gawk options ] -f program-file [ -- ] file ... igawk [ all gawk options ] [ -- ] program-text file ... DESCRIPTION
Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1). AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like @include getopt.awk in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path. OPTIONS
See gawk(1) for a full description of the AWK language and the options that gawk supports. EXAMPLES
cat << EOF > test.awk @include getopt.awk BEGIN { while (getopt(ARGC, ARGV, "am:q") != -1) ... } EOF igawk -f test.awk SEE ALSO
gawk(1) Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995. AUTHOR
Arnold Robbins (arnold@skeeve.com). ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +--------------------+-----------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +--------------------+-----------------+ |Availability | SUNWgawk | +--------------------+-----------------+ |Interface Stability | Volatile | +--------------------+-----------------+ NOTES
Source for gawk is available on http://opensolaris.org. Free Software Foundation Nov 3 1999 IGAWK(1)

Featured Tech Videos

All times are GMT -4. The time now is 06:25 AM.
Unix & Linux Forums Content Copyright 1993-2019. All Rights Reserved.
Privacy Policy