Sponsored Content
Top Forums Shell Programming and Scripting Read in shell variable values from a file Post 302883126 by LMHmedchem on Saturday 11th of January 2014 01:37:16 AM
Old 01-11-2014
Read in shell variable values from a file

Hello,

I have a simple script that runs an application,
Code:
# these arguments have the same value for all splits
ARCH=12.11.1
BATCHES=50
EPOCHS=5000
LEARN_MODE=ONLINE
LEARN_RATE=0.25
PROJ=02_BT_12.11.1.proj
   
echo "processing split A on hex"
cd A/
DATA_SET=S2A_v1_12.1.1_1
./00_parallel-batch_hex.sh  $DATA_SET  $ARCH  $BATCHES  $EPOCHS  $LEARN_MODE  $LEARN_RATE  $PROJ
cd ..

echo "processing split B on hex"
... more like this

This application can run for days at a time before it is finished and creates several thousand output files.

One problem I am having is that there are times when I look at some of the early output and decide that I need to change some of that parameters above, such as that values for EPOCHS. As it is, the value is read in and then the script runs about 2000 iterations using that value. There is no way to change the value without stopping the script, changing the value, and starting again.

It would be nice if I could have the script read in the values from a file before beginning a split.

Code:
echo "processing split A on hex"

# read values from file
ARCH=
BATCHES=
EPOCHS=
LEARN_MODE=
LEARN_RATE=
PROJ=

cd A/
DATA_SET=S2A_v1_12.1.1_1
./00_parallel-batch_hex.sh  $DATA_SET  $ARCH  $BATCHES  $EPOCHS  $LEARN_MODE  $LEARN_RATE  $PROJ
cd ..

echo "processing split B on hex"

# read values from file
ARCH=
BATCHES=
EPOCHS=
LEARN_MODE=
LEARN_RATE=
PROJ=

cd B/
DATA_SET=S2B_v1_12.1.1_1
./00_parallel-batch_hex.sh  $DATA_SET  $ARCH  $BATCHES  $EPOCHS  $LEARN_MODE  $LEARN_RATE  $PROJ
cd ..

... more like this

If the values were read in before each split, then at least I could change the values in the control file and the revised values would get used for the next split. This is much better than constantly having to start over and clean up the partial output, etc.

I guess I could hack this up with sed or awk, but I have never opened and read the contents of a file from bash. The only similar things I have done are with file names, and I think it would be overly goofy to read in the values from a file name. (overly goofy even for me)

I thought I would ask if there was a standard way to do such a thing.

Thanks for the advice,

LMHmedchem
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How can I read variable values from file?

Hi, I want to pratmeterze my scripts like, my confRsync file contains varibale values for 1. host 2. Destination and 3. source like this, I want to read this values from this and assing to my makeRsyn.sh file's varibales. how to do this? (1 Reply)
Discussion started by: redlotus72
1 Replies

2. Shell Programming and Scripting

Korn Shell Script - Read File & Search On Values

I am attempting to itterate through a file that has multiple lines and for each one read the entire line and use the value then to search in other files. The problem is that instead of an entire line I am getting each word in the file set as the value I am searching for. For example in File 1... (2 Replies)
Discussion started by: run_unx_novice
2 Replies

3. Shell Programming and Scripting

Read variable from file in a C shell script

Hi, I have a 1-line file which looks like " First second third 4 five". I need to extract the number (here 4) in that line and put it in a variable. I will use the variable later to make few tests in my C shell script. Can somebody help me? (2 Replies)
Discussion started by: haouesse
2 Replies

4. Shell Programming and Scripting

how can i read text file and assign its values to variables using shell

Hello, I have a cat.dat file, i would like shell to read each 3 lines and set this 3 lines to 3 different variables. my cat.dat is: 11 12 +380486461001 12 13 +380486461002 13 14 +380486461003 i want shell to make a loop and assign 1st line to student_id, 2nd line to... (4 Replies)
Discussion started by: rosalinda
4 Replies

5. Shell Programming and Scripting

Read a file and assign the values to a variable

i have a file in this format curyymm PRVYYMM CDDMmmYY bddMmmyy eddMmmyy --------- ------- ------------ ---------- ----------- 0906 0905 09Jun09 01Jun09 30Jun09 ----------- --------- ------------ ------------ ----------- i need to read the... (5 Replies)
Discussion started by: depakjan
5 Replies

6. Shell Programming and Scripting

Read the csv file and assign the values in to variable

I have a csv file with the values seperated by commas.I want to extract these values one by one and assign to a variable using shell script.Any ideas or code? (11 Replies)
Discussion started by: rajbal
11 Replies

7. Shell Programming and Scripting

How can I read values from a CSV file using Shell?

SHELL SCRIPT Hi, I have a file in which contents are as follows: 9999,abdc,123 9988,aba_12,2323 and so on I want to read the contents of this file such that i can do echo "This is $a followed by $b an then $c" I tried the following but id did not work cat test | cut -d ',' -f1|... (7 Replies)
Discussion started by: mayanksargoch
7 Replies

8. Fedora

How to read a text file and assign the values in the same to a variable in loop

Hi, I have a text file with multiple lines, each having data in the below format <DOB>,<ADDRESS> I have to write a script which reads each line in the text file in loop, assign the values to these variables and do some further processing in it. Using the following code prints the... (1 Reply)
Discussion started by: manishab00
1 Replies

9. UNIX for Advanced & Expert Users

How to read a text file and assign the values in the same to a variable in loop

Hi, I have a text file with multiple lines, each having data in the below format <DOB>,<ADDRESS> I have to write a script which reads each line in the text file in loop, assign the values to these variables and do some further processing in it. Using the following code prints the values... (12 Replies)
Discussion started by: manishab00
12 Replies

10. Shell Programming and Scripting

Store values from a file into an array variable in Shell

Dear All, I have been trying to do a simple task of extracting 2 fields from the file (3 rows) and store it in an array variable. I tried with: #! /bin/bash ch=`cut -f10 tmp.txt` counter=0 for p in $pid do c=${ch} echo "$c ..$counter" counter=$((counter+1))... (2 Replies)
Discussion started by: ezhil01
2 Replies
MPB(1)							    MIT Photonic-Bands Package							    MPB(1)

NAME
mpb-split - compute eigenmodes with MPB using multiple processes SYNOPSIS
mpb-split NUM-SPLIT [DEFINITION]... [CTLFILE]... DESCRIPTION
mpb-split is a parallelizing front-end to MIT Photonic Bands (MPB). For a computation with several k points, it splits the list of k points over multiple processes. Of course, this will only benefit you on a system where different processes will run on different proces- sors, such as an SMP or a cluster with automatic process migration (e.g. MOSIX). mpb-split is actually a trivial shell script, though, so you can easily modify it if you need to use a special command to launch processes on other processors/machines. MIT Photonic Bands (MPB) is a free program to compute the band structures (dispersion relations) and electromagnetic modes of periodic dielectric structures, and is applicable both to photonic crystals (photonic band-gap materials) and a wide range of other optical prob- lems. More information on MPB, including a detailed manual, can be found online at the MPB home page: http://ab-initio.mit.edu/mpb/ A typical invocation of mpb-split looks like: mpb-split num-split foo.ctl >& foo.out This causes mpb-split to process the control file foo.ctl, divide the k points into num-split equal chunks, run each list in a separate process with MPB, and redirect the output (in order) to foo.out. (One typically redirects output to a file, as the output is verbose and contains a number of comma-delimited datasets that one can extract by grepping.) Overall, the behavior and arguments are the same as for mpb except that the first argument must be the integer num-split. What mpb-split technically does is to set the MPB variable k-split-num to num-split and k-split-index to the index (starting with 0) of the chunk for each process. If you want, you can use these variables to divide the problem in some other way and then reset them to 1 and 0, respectively. BUGS
Send bug reports to S. G. Johnson, stevenj@alum.mit.edu. AUTHORS
Written by Steven G. Johnson. Copyright (c) 1999, 2000, 2001, 2002 by the Massachusetts Institute of Technology. SEE ALSO
mpb(1), mpb-data(1) MPB
March 13, 2002 MPB(1)
All times are GMT -4. The time now is 10:15 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy