Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Shell variable expansion in awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Shell variable expansion in awk
# 1  
Old 10-26-2015
Shell variable expansion in awk
I want to split one file input.tab into two separate ones, odd lines to input_reads1.txt, even lines to input_reads2.txt for a serial of files with similar name pattern. Also I want to "match" input/output file names to keep consistency of file name:
Code:
CSEL_02.0_input.tab 
CSEL_03.4_input.tab
CSEL_04.9_input.tab
CSEL_05.6_input.tab
CSEL_06.6_input.tab
CSEL_11.0_input.tab
CSEL_18.0_input.tab
......

One example:
CSEL_02.0_input.tab
Code:
seq1_description CAATCGATCGACTAG
seq2_description GTCGATCGACTAGGA
seq3_description CTCGATCGACTAGCT
seq4_description CGATCGCGCGGGGAA

Outputs file names and contents are:
CSEL_02.0_reads_R1.txt
Code:
seq1_description 
CAATCGATCGACTAG
seq3_description 
CTCGATCGACTAGCT

CSEL_02.0_reads_R2.txt
Code:
seq2_description 
GTCGATCGACTAGGA
seq4_description 
CGATCGCGCGGGGAA

I have tried:
Code:
for i in 02.0 03.4 04.9 05.6 06.6 11.0 18.0; do 
awk -v stem=CSEL_${i}_reads_R '{print $1"\n"$2 >> "stem"(NR%2?1:2)".txt"}' CSEL_${i}_input.tab
done

But I got:
Code:
stem1.txt
stem2.txt

Shell variable "stem" did not expand as I want to match input/output file names.
What did I miss?
Last edited by yifangt; 10-26-2015 at 05:09 PM..
# 2  
Old 10-26-2015
It didn't expand because you gave awk the quoted string literal "stem", not the unquoted variable name stem.

awk sets FILENAME for your convenience, however, might be easier to use that.

Also, > is a little different in awk in that it keeps the file open. The only difference between > and >> is that the very first write of > will truncate the file, but >> never truncates at all.


Code:
awk 'FNR==1 { F=FILENAME ; sub(/_input[.]tab/, "", F); } { print $1"\n"$2 > F "_reads_R" (NR%2)+1 }' inputfile

This User Gave Thanks to Corona688 For This Post:
yifangt
# 3  
Old 10-26-2015
quoted string literal "stem"
Ahh, it's not so simple until you pointed out.
Thanks for pointing out the difference between ">" and ">>", using ">>" is easy to cause trouble when the script is re-run unless the file is deleted each time before start.
Also I like the FILENAME version. The print part should be:
Code:
{ print $1"\n"$2 > F "_reads_R" (NR%2)+1 ".txt"}

according to file name requirement in the example, right? Thanks a lot!
Last edited by yifangt; 10-26-2015 at 05:44 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash variable expansion in awk script

Hello, I need to split a file into two of different locations by re-direction in awk. cat infle aaa 1 3 bbb 2 4 aaa 3 3 bbb 4 4 aaa 5 3 bbb 6 4 cat /storage/tmp/group_a.gtf aaa 1 3 aaa 3 3 aaa 5 3 cat /storage/tmp/group_b.gtf bbb 2 4 bbb ... (2 Replies)
Discussion started by: yifangt
2 Replies

2. Shell Programming and Scripting

Bash variable expansion

Hello. The file /etc/fstab contains UUID=957c3295-9944-1593-82e2-2b90dede4312 / ext4 noatime,discard,acl,user_xattr 1 1 I fill a variable SOME_LINE=$( cat /etc/fstab | grep \/\..*ext4 | grep noatime,discard )echo $SOME_LINE... (3 Replies)
Discussion started by: jcdole
3 Replies

3. Shell Programming and Scripting

Protecting variable indicator ($) from expansion

Hello, I use a lot this command to edit a bunch of files at once find . -name filename" | xargs -ifoo sh -c 'echo foo ; sed "s/pattern1/pattern2/" foo > ./tmp ; mv -f ./tmp foo' I'm trying to put a function on my .bashrc file. function loopSed() { local filename=$1 local... (2 Replies)
Discussion started by: phollox
2 Replies

4. UNIX for Dummies Questions & Answers

Can I use a variable with brace expansion?

So, I was bored on the train today, and was thinking of ways to loop through elements of an array. I came up with the following simple script, but it doesn't work as brace expansion doesn't seem to work with variables. Is there something I'm missing, or does the shell just not work like this? ... (4 Replies)
Discussion started by: DeCoTwc
4 Replies

5. Shell Programming and Scripting

delay variable expansion

Hi forum, in my bash script I've many lines executing commands with redirection to log files. ... xyz_cmd 2>&1 > $BASENAME.$LINENO The trailing part of these lines doesn't look nice and I like to put it into a variable. The (not working) idea is something like that ... (3 Replies)
Discussion started by: wolfi089
3 Replies

6. Shell Programming and Scripting

Variable expansion in sed

The objective of the code below is to create sed script to be later executed. However, it bonks because $ARCHIVENAME expands to a directory specification so the forward slashes cause problems. I can think of a few solutions that would involve redesigning the process, but I'm hoping there might be... (4 Replies)
Discussion started by: tiggyboo
4 Replies

7. Shell Programming and Scripting

Bash variable delayed expansion?

i write a batch file , here is the content. dirname='date +%Y-%m-%d' mkdir dirname but it doen's work, it just create a folder named date and +%Y-%m-%d. i have tried run the command seperately in the bash prompt. after the first sentence executed , i use $dirname to watch the value of... (4 Replies)
Discussion started by: premotheus
4 Replies

8. UNIX for Dummies Questions & Answers

Variable brace expansion

I'm in the habit of using the following type of loop structure: for num in `seq $1 $2` do command doneWhile `seq $1 $2` is not exactly a huge resource hog, I would like to learn a better way. It seems that brace expansion is a good way to go: for num in {3..10}The problem, though, is... (2 Replies)
Discussion started by: treesloth
2 Replies

9. Shell Programming and Scripting

Basic variable expansion not working...

#!/usr/bin/bash if then echo "Not valid arguments entered. Just username should be entered." else USER_NAME=$1 FILE_NAME=$USER_NAME.info UNN=STUDIN\\\\$1 echo $UNN last STUDIN\\\\$1 last UNN If I type `last STUDIN\\eip060` it works but if I try to expand it with variable it is... (5 Replies)
Discussion started by: Zammy_bg
5 Replies

10. UNIX for Dummies Questions & Answers

ksh on HP-UX -- variable expansion

We have a script that runs in ksh on HP-UX 11.11. It takes three arguments. The last argument can be a filename or wildcard character. For example: script -s hello -t goodbye '*.d*' In a case such as this, I would wrap single quotes around the final argument because I dont want the shell to... (4 Replies)
Discussion started by: dangral
4 Replies
Login or Register to Ask a Question