Shell variable expansion in awk


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
# 1  
Shell variable expansion in awk

I want to split one file input.tab into two separate ones, odd lines to input_reads1.txt, even lines to input_reads2.txt for a serial of files with similar name pattern. Also I want to "match" input/output file names to keep consistency of file name:
Code:
CSEL_02.0_input.tab 
CSEL_03.4_input.tab
CSEL_04.9_input.tab
CSEL_05.6_input.tab
CSEL_06.6_input.tab
CSEL_11.0_input.tab
CSEL_18.0_input.tab
......

One example:
CSEL_02.0_input.tab
Code:
seq1_description CAATCGATCGACTAG
seq2_description GTCGATCGACTAGGA
seq3_description CTCGATCGACTAGCT
seq4_description CGATCGCGCGGGGAA

Outputs file names and contents are:
CSEL_02.0_reads_R1.txt
Code:
seq1_description 
CAATCGATCGACTAG
seq3_description 
CTCGATCGACTAGCT

CSEL_02.0_reads_R2.txt
Code:
seq2_description 
GTCGATCGACTAGGA
seq4_description 
CGATCGCGCGGGGAA

I have tried:
Code:
for i in 02.0 03.4 04.9 05.6 06.6 11.0 18.0; do 
awk -v stem=CSEL_${i}_reads_R '{print $1"\n"$2 >> "stem"(NR%2?1:2)".txt"}' CSEL_${i}_input.tab
done

But I got:
Code:
stem1.txt
stem2.txt

Shell variable "stem" did not expand as I want to match input/output file names.
What did I miss?

Last edited by yifangt; 10-26-2015 at 06:09 PM..
# 2  
It didn't expand because you gave awk the quoted string literal "stem", not the unquoted variable name stem.

awk sets FILENAME for your convenience, however, might be easier to use that.

Also, > is a little different in awk in that it keeps the file open. The only difference between > and >> is that the very first write of > will truncate the file, but >> never truncates at all.


Code:
awk 'FNR==1 { F=FILENAME ; sub(/_input[.]tab/, "", F); } { print $1"\n"$2 > F "_reads_R" (NR%2)+1 }' inputfile

This User Gave Thanks to Corona688 For This Post:
# 3  
quoted string literal "stem"
Ahh, it's not so simple until you pointed out.
Thanks for pointing out the difference between ">" and ">>", using ">>" is easy to cause trouble when the script is re-run unless the file is deleted each time before start.
Also I like the FILENAME version. The print part should be:
Code:
{ print $1"\n"$2 > F "_reads_R" (NR%2)+1 ".txt"}

according to file name requirement in the example, right? Thanks a lot!

Last edited by yifangt; 10-26-2015 at 06:44 PM..
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #262
Difficulty: Easy
During the Second World War, Alan Turing worked for the Government Code and Cypher School (GC&CS) at Bletchley Park, Britain's codebreaking centre that produced Ultra intelligence.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash variable expansion in awk script

Hello, I need to split a file into two of different locations by re-direction in awk. cat infle aaa 1 3 bbb 2 4 aaa 3 3 bbb 4 4 aaa 5 3 bbb 6 4 cat /storage/tmp/group_a.gtf aaa 1 3 aaa 3 3 aaa 5 3 cat /storage/tmp/group_b.gtf bbb 2 4 bbb ... (2 Replies)
Discussion started by: yifangt
2 Replies

2. Shell Programming and Scripting

Bash variable expansion

Hello. The file /etc/fstab contains UUID=957c3295-9944-1593-82e2-2b90dede4312 / ext4 noatime,discard,acl,user_xattr 1 1 I fill a variable SOME_LINE=$( cat /etc/fstab | grep \/\..*ext4 | grep noatime,discard )echo $SOME_LINE... (3 Replies)
Discussion started by: jcdole
3 Replies

3. Shell Programming and Scripting

Help required regarding variable expansion in UNIX

Hello, I have one variable coming from one file: abc=$xyz/filename.txt where $xyz is defined in .profile file as say, /usr/dev/src i am passing abc variable to one perl script as input parameter. perl 123.pl -s $abc But inside the perl script execution, the variable $abc is just... (1 Reply)
Discussion started by: vikas_trl
1 Replies

4. Shell Programming and Scripting

Protecting variable indicator ($) from expansion

Hello, I use a lot this command to edit a bunch of files at once find . -name filename" | xargs -ifoo sh -c 'echo foo ; sed "s/pattern1/pattern2/" foo > ./tmp ; mv -f ./tmp foo' I'm trying to put a function on my .bashrc file. function loopSed() { local filename=$1 local... (2 Replies)
Discussion started by: phollox
2 Replies

5. UNIX for Dummies Questions & Answers

Can I use a variable with brace expansion?

So, I was bored on the train today, and was thinking of ways to loop through elements of an array. I came up with the following simple script, but it doesn't work as brace expansion doesn't seem to work with variables. Is there something I'm missing, or does the shell just not work like this? ... (4 Replies)
Discussion started by: DeCoTwc
4 Replies

6. Shell Programming and Scripting

delay variable expansion

Hi forum, in my bash script I've many lines executing commands with redirection to log files. ... xyz_cmd 2>&1 > $BASENAME.$LINENO The trailing part of these lines doesn't look nice and I like to put it into a variable. The (not working) idea is something like that ... (3 Replies)
Discussion started by: wolfi089
3 Replies

7. Shell Programming and Scripting

Variable expansion in sed

The objective of the code below is to create sed script to be later executed. However, it bonks because $ARCHIVENAME expands to a directory specification so the forward slashes cause problems. I can think of a few solutions that would involve redesigning the process, but I'm hoping there might be... (4 Replies)
Discussion started by: tiggyboo
4 Replies

8. UNIX for Dummies Questions & Answers

Variable brace expansion

I'm in the habit of using the following type of loop structure: for num in `seq $1 $2` do command doneWhile `seq $1 $2` is not exactly a huge resource hog, I would like to learn a better way. It seems that brace expansion is a good way to go: for num in {3..10}The problem, though, is... (2 Replies)
Discussion started by: treesloth
2 Replies

9. Shell Programming and Scripting

Basic variable expansion not working...

#!/usr/bin/bash if then echo "Not valid arguments entered. Just username should be entered." else USER_NAME=$1 FILE_NAME=$USER_NAME.info UNN=STUDIN\\\\$1 echo $UNN last STUDIN\\\\$1 last UNN If I type `last STUDIN\\eip060` it works but if I try to expand it with variable it is... (5 Replies)
Discussion started by: Zammy_bg
5 Replies

10. UNIX for Dummies Questions & Answers

ksh on HP-UX -- variable expansion

We have a script that runs in ksh on HP-UX 11.11. It takes three arguments. The last argument can be a filename or wildcard character. For example: script -s hello -t goodbye '*.d*' In a case such as this, I would wrap single quotes around the final argument because I dont want the shell to... (4 Replies)
Discussion started by: dangral
4 Replies

Featured Tech Videos