At one point, I had this set up to process the first few files from

Code:

FILES='./'$FOLD'/'$FOLD'_continue/'$OPTIMIZE_ON'/'*'out.txt'

but when the file names looked like,

Code:

99.82_E100_85.20_E100_26_ri_OA_f2_S2A_v8_47.46.1_ON_0.25lr.out.txt
99.88_E100_86.71_E100_17_ri_OA_f2_S2A_v8_47.46.1_ON_0.25lr.out.txt
100.22_E100_86.31_E100_39_ri_OA_f2_S2A_v8_47.46.1_ON_0.25lr.out.txt
100.24_E200_87.52_E100_47_ri_OA_f2_S2A_v8_47.46.1_ON_0.25lr.out.txt

The file starting with 100.22_E100 got processed first, presumably since 1 is smaller than 9. As I think about it, I described it wrong. It is the first field (the real number) that I would wand to sort on and then retrieve the other numbers. For the files above, if I wanted the top two results I would first look at the number in the first field,

Code:

99.82
99.88
100.22
100.24

Based on the values, I would want to extract,

Code:

MAE_EPOCH=100, RAND_SET=26 for the top file
MAE_EPOCH=100, RAND_SET=17 for the second file

These are the numbers I need to pass to the next program.

Is there some reason to use grep to filter the name instead of doing ls *.out.txt?

I guess I would be using some combination of -t -k -n with sort, like

ls *.out.txt | sort -t_ -k 1 -n | head -n 2 | awk

This would give me the top two files in the list above?

I'm sure I will have to play around with this, but thanks for the head start. I am a bit unclear on how to pass the result of the pipe into my loop. Do you have a link for an example of something like that?

LMHmedchem

---------- Post updated 02-07-14 at 02:00 PM ---------- Previous update was 02-06-14 at 10:13 PM ----------

Well I have it working with this,

Code:

#!/bin/bash

NUMBER_TO_PROCESS=2
BACKUP=100

cd ./the_folder_with_the_files

# find all files .out.txt and sort on the real number in the first position
# return the file names for the top $NUMBER_TO_PROCESS in sorted list
FILES=$(ls *.out.txt | sort -t_ -k 1 -n | head -n $NUMBER_TO_PROCESS)

# loop on all file names returned
for INPUT in $FILES
do

   # current file
   echo $INPUT

   # store random ini set
   RAND_SET=`echo $INPUT | awk 'BEGIN {FS="_"} {print $5}'`
   echo "RAND_SET"  $RAND_SET

   # find MAE min epoch value
   MAE_EPOCH=`echo $INPUT | awk 'BEGIN {FS="_"} {print $2}'`
   # remove leading E
   MAE_EPOCH=${MAE_EPOCH#"E"}
   # backup if specified
   let "MAE_EPOCH= $MAE_EPOCH - $BACKUP"
   echo "MAE_EPOCH"  $MAE_EPOCH

done

This gives the behavior that I am looking for.

I left the awk stuff in the loop since there are a couple of items in the file name string that need to be retrieved and assigned to variables. Does this make sense?

Is there some reason to filter the file list with grepinstead of using the glob?

LMHmedchem