Hello,

I have to fish out some specific columns from a file based on the header value. I have the list of columns I need in a different file. I thought I could read in the list of headers I need,

Code:

# file with header names of required columns in required order
headers_file=$2

# read contents of headers_file into array
IFS=$'\n' read -a headers_list < $headers_file

and then loop through the list to pick out each column I need,

Code:

# loop on header list
for header_value in "${headers_list[@]}"
do
   # print current input file
   echo $header_value

   # look for the column in the input file
   awk -v OFS='\t' -v column_header="$header_value" 'NR==1{for (i=1; i<=NF; i++) if ($i==column_header){p=i; break}; next} {print $p}' $input_file > $output_file

done

The above awk does not work and even if it did it would overwrite the data from each previous column found. How do I find all the columns I need and then print all of them in the right order so they all end up in the output file?

The only thing I could think of was to read the header line from $input_file into another array and then loop through $headers_list making a note of the numerical position of the columns I need. In theory, I could use the list of numerical positions to cobble together a cut argument to get the columns I need. That seems like it would be horribly messy syntax and could probably be done with one line of awk from someone who knows what they are doing.

That means it's time to post and ask for help. I found allot of topics like this one, but most of them seemed to find one column by the header value and print it.

In case that makes a difference, the input files I am working have < 200 columns but may have almost any number of rows. The input file is space delimited and the output should be tab delimited, though I could replace space with tab after the fact if necessary.

Suggestions would be greatly appreciated.

LMHmedchem