Shell Programming and Scripting

Hello,

I have a simple script that runs an application,

Code:

# these arguments have the same value for all splits
ARCH=12.11.1
BATCHES=50
EPOCHS=5000
LEARN_MODE=ONLINE
LEARN_RATE=0.25
PROJ=02_BT_12.11.1.proj
   
echo "processing split A on hex"
cd A/
DATA_SET=S2A_v1_12.1.1_1
./00_parallel-batch_hex.sh  $DATA_SET  $ARCH  $BATCHES  $EPOCHS  $LEARN_MODE  $LEARN_RATE  $PROJ
cd ..

echo "processing split B on hex"
... more like this

This application can run for days at a time before it is finished and creates several thousand output files.

One problem I am having is that there are times when I look at some of the early output and decide that I need to change some of that parameters above, such as that values for EPOCHS. As it is, the value is read in and then the script runs about 2000 iterations using that value. There is no way to change the value without stopping the script, changing the value, and starting again.

It would be nice if I could have the script read in the values from a file before beginning a split.

Code:

echo "processing split A on hex"

# read values from file
ARCH=
BATCHES=
EPOCHS=
LEARN_MODE=
LEARN_RATE=
PROJ=

cd A/
DATA_SET=S2A_v1_12.1.1_1
./00_parallel-batch_hex.sh  $DATA_SET  $ARCH  $BATCHES  $EPOCHS  $LEARN_MODE  $LEARN_RATE  $PROJ
cd ..

echo "processing split B on hex"

# read values from file
ARCH=
BATCHES=
EPOCHS=
LEARN_MODE=
LEARN_RATE=
PROJ=

cd B/
DATA_SET=S2B_v1_12.1.1_1
./00_parallel-batch_hex.sh  $DATA_SET  $ARCH  $BATCHES  $EPOCHS  $LEARN_MODE  $LEARN_RATE  $PROJ
cd ..

... more like this

If the values were read in before each split, then at least I could change the values in the control file and the revised values would get used for the next split. This is much better than constantly having to start over and clean up the partial output, etc.

I guess I could hack this up with sed or awk, but I have never opened and read the contents of a file from bash. The only similar things I have done are with file names, and I think it would be overly goofy to read in the values from a file name. (overly goofy even for me)

I thought I would ask if there was a standard way to do such a thing.

Thanks for the advice,

LMHmedchem

Put the commands you want to set the values of these variables in a file, such as set_values:

Code:

ARCH=arch_val
BATCHES=batches_val
EPOCHS=epochs_val
LEARN_MODE=learn_mode_val
LEARN_RATE=learn_rate_val
PROJ=proj_val

and then when you want your script to reset the values of those variables, issue the command:

Code:

. set_values

Sorry, I haven't got back to this yet.

There are a couple of reasons I have for wanting to read parameters from a file. One is that there are a number of them and the list is getting longer. It would be tedious to always have to enter in the values I want. I also need to keep track of these things to some extent and the combination of a script and something like an ini file gives documentation of what I did.

Overall, there seem to be some good reasons to have the values read in from a file and I am wondering how it could be done in bash.

LMHmedchem

In what way does Don's solution not work for you? It does exactly what you asked for.

Once again it is demonstrated that reading is a virtue. When I first read that post, I somehow understood that the commands like ARCH=arch_val went in the original script and you would somehow be prompted for the new values when you entered . set_values. So much for reading. I got sidetracked with some other problems and didn't get to test the posted code.

I will play around with this and see if I can get the hang of how it works.

LMHmedchem