Sponsored Content
Top Forums Shell Programming and Scripting Bash variable expansion in awk script Post 303036301 by yifangt on Friday 21st of June 2019 04:59:54 PM
Old 06-21-2019
Bash variable expansion in awk script

Hello,
I need to split a file into two of different locations by re-direction in awk.
Code:
cat infle
aaa     1 3
bbb     2 4
aaa     3 3
bbb     4 4
aaa     5 3
bbb     6 4

cat /storage/tmp/group_a.gtf
aaa     1 3
aaa     3 3
aaa     5 3

cat /storage/tmp/group_b.gtf
bbb     2 4
bbb     4 4
bbb     6 4

The path of the resulted files should be passed in with shell variable, and I have tried:
Code:
MY_DIR="/storage/tmp"
awk -v MY_DIR="${MY_DIR}" '{if ($1 == "aaa")  {print $0 >> "${MY_DIR}/group_a.gtf"} else if ($1 == "bbb") {print  $0 >> "${MY_DIR}/group_b.gtf"}}' infle  #A typo was $7=="bbb"
MY_DIR="/storage/tmp";  awk '{if ($1 == "aaa") {print $0 >>  ENVIRON[\"MY_DIR\"]/group_a.gtf"} else if ($1 == "bbb") {print $0  >> "ENVIRON[\"MY_DIR\"]/group_b.gtf"}}' infle
awk -v  awk_var=${MY_DIR} '{if ($1 == "aaa") {print $0 >>  "awk_var/group_a.gtf"} else if ($1 == "bbb") {print $0 >>  "awk_var/group_b.gtf"}}' infle

but none worked with error like: cmd. line:1: (FILENAME=aaa FNR=1) fatal: can't redirect to `awk_var/group_a.gtf' (No such file or directory)

I checked the threads in this forum:
How to pass a variable from shell to awk
Pass shell Variable to awk
but they did not help me out probably due to my expansion is related to file directory. However, the it worked if the path is hard-coded, which is not thru shell variable along with other part of the shell script.
Code:
awk  '{if ($1 == "aaa") {print $0 >> "/storage/tmp/group_a.gtf"} else  if ($1 == "bbb") {print $0 >> "/storeage/tmp/group_b.gtf"}}'  infle

Also It is interesting with following oneliner, in which the expansion worked at every other hit of ENTER. Could not understand why.
Code:
MY_DIR="/storage/tmp"; awk -v awk_var="$MY_DIR" '{print awk_var}'

/storage/tmp

/storage/tmp

/storage/tmp

 ......

1) What are the rules to expand shell (bash) variables (especialy related to directory) inside awk script?
2) Why my last oneliner worked every other time with ENTER? Thanks a lot!

Last edited by yifangt; 06-24-2019 at 12:22 PM..
This User Gave Thanks to yifangt For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

globbing, $# is too high after wildcard expansion in bash script

How can I pass in an argument such as "*.k" to a bash script without having to double-quote *.k and not having *.k `glob` to match all files in the pattern? I tried using noglob in my script but this didn't work the way I thought it would.. expansion is still occuring, $# is higher than I... (3 Replies)
Discussion started by: zoo591
3 Replies

2. Shell Programming and Scripting

bash - delay expansion of variable

Hello - I have a bash script which does some logging, and I'd like to include the line number of the echo statement that pipes into $LOGGER: MYPID=$$ MYNAME=`basename $0` LOGGER="/usr/bin/logger -t $MYNAME($LINENO) -p daemon.error" ... echo 'this is an entry into the log file' | $LOGGER ... (3 Replies)
Discussion started by: scandora
3 Replies

3. Shell Programming and Scripting

Bash variable delayed expansion?

i write a batch file , here is the content. dirname='date +%Y-%m-%d' mkdir dirname but it doen's work, it just create a folder named date and +%Y-%m-%d. i have tried run the command seperately in the bash prompt. after the first sentence executed , i use $dirname to watch the value of... (4 Replies)
Discussion started by: premotheus
4 Replies

4. Shell Programming and Scripting

Bash shell expansion help

This is what I have in my directory. $ ls test1.txt test2.txt test3.txt test4.txt test5.txt test_script.sh This is what my shellscript looks like. #!/bin/bash for filename in /shell_expansion/*.txt; do for ((i=0; i<=3; i++)); do echo "$filename" ... (5 Replies)
Discussion started by: cokedude
5 Replies

5. Shell Programming and Scripting

Shell variable expansion in awk

I want to split one file input.tab into two separate ones, odd lines to input_reads1.txt, even lines to input_reads2.txt for a serial of files with similar name pattern. Also I want to "match" input/output file names to keep consistency of file name: CSEL_02.0_input.tab CSEL_03.4_input.tab... (2 Replies)
Discussion started by: yifangt
2 Replies

6. Shell Programming and Scripting

Bash expansion

Hello. I cannot write a command without using eval. Any help is welcome Note 1 : What does the function SOMETHING has no importance. Note 2 : What does the command find has no importance. It is an expansion variable problem : where to put or or or anythings else What works (FILTRE_1... (8 Replies)
Discussion started by: jcdole
8 Replies

7. Shell Programming and Scripting

Bash variable expansion

Hello. The file /etc/fstab contains UUID=957c3295-9944-1593-82e2-2b90dede4312 / ext4 noatime,discard,acl,user_xattr 1 1 I fill a variable SOME_LINE=$( cat /etc/fstab | grep \/\..*ext4 | grep noatime,discard )echo $SOME_LINE... (3 Replies)
Discussion started by: jcdole
3 Replies

8. Shell Programming and Scripting

Bash Parameter Expansion

I have made the following examples that print various parameter expansions text: iv-hhz-sac/hpac/hhz.d/iv.hpac..hhz.d.2016.250.070018.sac (text%.*): iv-hhz-sac/hpac/hhz.d/iv.hpac..hhz.d.2016.250.070018 (text%%.*): iv-hhz-sac/hpac/hhz (text#*.): d/iv.hpac..hhz.d.2016.250.070018.sac... (2 Replies)
Discussion started by: kristinu
2 Replies

9. Shell Programming and Scripting

Bash Parameter Expansion

#!/bin/bash SNMPW='/usr/bin/snmpwalk' while read h i do loc=$($SNMPW -v3 -u 'Myusername' -l authPriv -a SHA -A 'Password1' -x AES -X 'Password2' $i sysLocation.0 2>/dev/null) loc=${loc:-" is not snmpable."} loc=${loc##*: } loc=${loc//,/} echo "$i,$h,$loc" done < $1 My question is ... ... (1 Reply)
Discussion started by: sumguy
1 Replies

10. Shell Programming and Scripting

Use parameter expansion over a parameter expansion in bash.

Hello All, Could you please do help me here as I would like to perform parameter expansion in shell over a parameter expansion. Let's say I have following variable. path="/var/talend/nat/cdc" Now to get only nat I could do following. path1="${path%/*}" path1="${path1##*/}" Here... (8 Replies)
Discussion started by: RavinderSingh13
8 Replies
All times are GMT -4. The time now is 10:46 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy