Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Use while loop to read file and use ${file} for both filename input into awk and as string to print

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Use while loop to read file and use ${file} for both filename input into awk and as string to print
# 1  
Old 05-16-2017
Use while loop to read file and use ${file} for both filename input into awk and as string to print
I have files named with different prefixes. From each I want to extract the first line containing a specific string, and then print that line along with the prefix.

I've tried to do this with a while loop, but instead of printing the prefix I print the first line of the file twice.

Files:
Code:
cat prefixes.txt 
N2_A
N2_O

cat N2_A_ko.txt 
K02234  6.24
K02588  14.971
K02588|unclassified     14.971

cat N2_O_ko.txt 
K02588  2.647
K02588|unclassified     2.647
K02233 3.45

Here's what I've tried:
Code:
while read file
do 
var=${file}
awk -OFS'\t' '/K02588/ {print $var,$1,$2; exit}' ${file}_ko.txt > ${file}_nifh.txt
done < prefixes.txt

But, again, instead of printing the prefix as the first column, it prints the first line of the file.
Code:
cat N2_A_nifh.txt 
K02588  14.971 K02588   14.971 

cat N2_O_nifh.txt 
K02588  2.647 K02588    2.647

# 2  
Old 05-16-2017
You're close, but shell variables aren't known in an awk script unless you explicitly pass them in. And, in awk $var prints the field named by the field number contained in the awk variable var (and since var hasn't been defined in your awk script it expanded to $0 which is the contents of the current line). Try:
Code:
while read file
do	awk -F'\t' -v var="$file" '/K02588/ {print var,$1,$2; exit}' "${file}_ko.txt" > "${file}_nifh.txt"
done < prefixes.txt

This User Gave Thanks to Don Cragun For This Post:
pathunkathunk
# 3  
Old 05-17-2017
Can be elegantly done with shell builtins
Code:
while read file
do 
  while read id w wr
  do
    case $id in
    K02588 )
      printf "%s\t%s\t%s\n" "$file" "$id" "$w"
      break
    ;;
    esac
  done <"$file"_ko.txt >"$file"_nifh.txt
done < prefixes.txt

--
EDIT: just seeing it is not always found in the first line, therefore added another while loop (and actually the previous awk solution does this loop)
Last edited by MadeInGermany; 05-17-2017 at 04:54 AM..
# 4  
Old 05-17-2017
You didn't say which awk you are using. If your awk is linked to nawk or gawk, there is also an alternative to the solution posted by Don Cragun, in that you could export your shell variable (so that it becomes an environment variable) and use awk's built-in ENVIRON array to access it. If you follow this route, I strongly recommend to write the variable in all-uppercase.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Read file input in while loop does not work on AIX system

I'm working on Aix 6.1 and using ksh shell. The below works fine on Linux bash or ksh shell . while IFS= read -r dirpath ; do echo "Hi" done <<<"$var" However, any such while loop that reads the input from file or variable using <<< fails on Aix system with the below error: Below... (2 Replies)
Discussion started by: mohtashims
2 Replies

2. UNIX for Beginners Questions & Answers

How to make a loop to read the input from a file part by part?

Hi All, We've a VDI infrastructure in AWS (AWS workspaces) and we're planning to automate the process of provisioning workspaces. Instead of going to GUI console, and launching workspaces by selecting individual users is little time consuming. Thus, I want to create them in bunches from AWS CLI... (6 Replies)
Discussion started by: arun_adm
6 Replies

3. Shell Programming and Scripting

Failure using regex with awk in 'while read file' loop

I have a file1.txt with several 100k lines, each of which has a column 9 containing one of 60 "label" identifiers. Using an labels.txt file containing a list of labels, I'd like to extract 200 random lines from file1.txt for each of the labels in index.txt. Using a contrived mini-example: $ cat... (8 Replies)
Discussion started by: pathunkathunk
8 Replies

4. Shell Programming and Scripting

Read input file with in awk script not through command line

Hi All, Do we know how to read input file within awk script and send output toanother log file. All this needs to be in awk script, not in command line. I am running this awk through crontab. Cat my.awk #!/bin/awk -f function test(var){ some code} { } END { print"test code" } (5 Replies)
Discussion started by: random_thoughts
5 Replies

5. Shell Programming and Scripting

For loop inside awk to read and print contents of files

Hello, I have a set of files Xfile0001 - Xfile0021, and the content of this files (one at a time) needs to be printed between some line (lines start with word "Generated") that I am extracting from another file called file7.txt and all the output goes into output.txt. First I tried creating a for... (5 Replies)
Discussion started by: jaldo0805
5 Replies

6. Shell Programming and Scripting

Using awk instead of while loop to read file

Hello, I have a huge file, I am currently using while loop to read and do some calculation on it, but it is taking a lot of time. I want to use AWK to read and do those calculations. Please suggest. currently doing: cat input2 | while read var1 do num=`echo $var1 | awk... (6 Replies)
Discussion started by: anand2308
6 Replies

7. Shell Programming and Scripting

Take input from read and place it a string in another file

Hi, This is most likely a dumb question but I could not find answer to it elsewhere. I'm building a simple menu with case /esac and want to read user's input: Please enter XYZ ; read XYZ How do I take the value of XYZ and insert it as a variable $XYZ in file file.txt ? I may need to... (9 Replies)
Discussion started by: svetoslav_sj
9 Replies

8. Shell Programming and Scripting

Print only Filename based on given String on file name

Hi, I am new to this unix world. Any ways, I would like to write a shell script that can print the file name. Ex : directory will have 5 files with different name.No matter what are contents are. Now I need to find the file which will have particular name (sub string ).Please do not... (5 Replies)
Discussion started by: akb2010
5 Replies

9. Shell Programming and Scripting

Awk to print on condition in input file

I need only those records which has $2 equal to "DEF" independent of case (i.e upper or lower) nawk -F"," '$2 ~ //{print $0}' file This returns 3rd record also which i dont want I tried this but this doesnt work as expected. nawk -F"," '$2 == ""{print $0}' file i dont... (3 Replies)
Discussion started by: pinnacle
3 Replies

10. Shell Programming and Scripting

Awk+Grep Input file needs to match a column and print the entire line

I'm having problems since few days ago, and i'm not able to make it works with a simple awk+grep script (or other way to do this). For example, i have a input file1.txt: cat inputfile1.txt 218299910417 1172051195 1172070231 1172073514 1183135117 1183135118 1183135119 1281440202 ... (3 Replies)
Discussion started by: poliver
3 Replies
Login or Register to Ask a Question