Trying to loop through folders and execute an existing python script.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Trying to loop through folders and execute an existing python script.
# 1  
Old 03-07-2018
Trying to loop through folders and execute an existing python script.

I am trying to loop through lots and lots of folders and use the names of the folders to run a Python script which has parameters.
E.g.
-- setup_refs -n John -f England/London/Hackney/John -c con/con.cnf

Normally to run `setup_refs` once from command line it's: `python setup_refs.py -n John -f England/London/Hackney/ref -c con/con.cnf`

The program folder (/home/program/) contains:
- setup.refs.py (script)
- ref (folder)
- con (folder) - con.cnf (file)


Code:

cd /home/program/ ;

for fldr in /home/program/ref/* ; do

    echo "Setting up ---> $fldr" ;

    basenm=${fldr##*/} ; new_fldr=${basenm%%_*} ; 

    python /home/program/setup_refs.py -f "$new_$fldr" -n /home/program/ref/"$new_$fldr" -c /home/program/config/config.cnf ; 

    echo "$fldr setup complete" ;

done

echo "Automated setup complete." ;

Can this be done?
# 2  
Old 03-07-2018
Quote:
Originally Posted by Mr_Keystrokes
I am trying to loop through lots and lots of folders and use the names of the folders to run a Python script which has parameters.
E.g.
-- setup_refs -n John -f England/London/Hackney/John -c con/con.cnf

Normally to run `setup_refs` once from command line it's: `python setup_refs.py -n John -f England/London/Hackney/ref -c con/con.cnf`

The program folder (/home/program/) contains:
- setup.refs.py (script)
- ref (folder)
- con (folder) - con.cnf (file)


Code:

cd /home/program/ ;

for fldr in /home/program/ref/* ; do

    echo "Setting up ---> $fldr" ;

    basenm=${fldr##*/} ; new_fldr=${basenm%%_*} ; 

    python /home/program/setup_refs.py -f "$new_$fldr" -n /home/program/ref/"$new_$fldr" -c /home/program/config/config.cnf ; 

    echo "$fldr setup complete" ;

done

echo "Automated setup complete." ;

Can this be done?
Observations:
You use the -n and -f switches inconsistently in the code compared to your example.

In your code you use /home/program/config/config.cnf rather than your required con/con.cnf

You don't need the trailing semicolons at the end of each line in bash or other bourne-derived shellscripts.

If you use:
Code:
cd /home/program/ref
for fldr in *

you don't need the extra step of removing the directory part of the filename. You will, however be in a different directory than the one you start out in (this may be a problem, but you are using full pathnames anyway).

Stupid question: In the above code if there is a folder (file?) named john_smith your code will set the $new_fldr variable to john. What happens to a folder named john?

Another stupid question: Have you tried running that code?

Andrew
# 3  
Old 03-07-2018
Quote:
Originally Posted by apmcd47
If you use:
Code:
cd /home/program/ref
for fldr in *

you don't need the extra step of removing the directory part of the filename. You will, however be in a different directory than the one you start out in (this may be a problem, but you are using full pathnames anyway).

Stupid question: In the above code if there is a folder (file?) named john_smith your code will set the $new_fldr variable to john. What happens to a folder named john?

Another stupid question: Have you tried running that code?

Andrew
  1. I change directory into cd /home/p995824/programs/kmerid-master/ because the setup.py script is in there where I run it.
  2. Good point, I only need new_fldr=${fldr##*/} because the folder names are one word.
  3. When I run the script it returns:


Code:
 file "/home/programs/setup_refs.py", line 7, in <module>
    import ConfigParser
ModuleNotFoundError: No module named 'ConfigParser'

Moderator's Comments:
Mod Comment Please use CODE (not ICODE) tags as required by forum rules!

Last edited by RudiC; 03-08-2018 at 05:46 AM.. Reason: Changed ICODE to CODE tags.
# 4  
Old 03-08-2018
Please use code tags, not icode tags.

ConfigParser.py is not in your python path. Is it installed? If not, install it. Is it perhaps in a python virtualenv that you forgot to "enable"?

Andrew
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Unable to print python array in shell script loop.

I am unable to loop print a python string array in my unix shell script: ~/readarr.sh '{{ myarr }}' more readarr.sh echo "Parameter 1:"$1 MYARRAY= $1 IFS= MYARRAY=`python <<< "print ' '.join($MYARRAY)"` for a in "$MYARRAY"; do echo "Printing Array: $a" done Can you... (10 Replies)
Discussion started by: mohtashims
10 Replies

2. Windows & DOS: Issues & Discussions

How to execute python script on remote with python way..?

Hi all, I am trying to run below python code for connecting remote windows machine from unix to run an python file exist on that remote windows machine.. Below is the code I am trying: #!/usr/bin/env python import wmi c = wmi.WMI("xxxxx", user="xxxx", password="xxxxxxx")... (1 Reply)
Discussion started by: onenessboy
1 Replies

3. Shell Programming and Scripting

Python DictWriter header - not writing in first line of existing file

Hello I am facing a very unique problem and not able to understand why. I have written a function which will check header of the file. If header is present good else it will write the header on top def writeHeaderOutputCSV(fileName): # See if the file exist already try: ... (0 Replies)
Discussion started by: radioactive9
0 Replies

4. Shell Programming and Scripting

Execute python file, FTP output to another server

Greetings all, We are implementing a new tool called URLwatch which is a python utility. Here are the requirements. 1) Run every 10 seconds 2) Execute the python script 3) Output file gets generated, FTP it to a differernt server I gave no idea how to do this and management needs a demo... (3 Replies)
Discussion started by: jeffs42885
3 Replies

5. Shell Programming and Scripting

Exit while loop on execute script

Hi, I have first script which on IR remote command event execute the second script. If the second script is executed, it display echo "timeout expired" after 10s. This works as expected. But I also want to reset timer (increase time) in case if the second script is executed again within 10s. ... (8 Replies)
Discussion started by: armatron
8 Replies

6. Shell Programming and Scripting

Execute script in series of folders

I'm trying to get a script to iterate of an array in bash run a series of commands in those folders. #Folder names folders=(folder1 folder2 folder3) #Common path for folders fold_path="/Users/Rich/Projects/" # For each element array copy script to folder and execute for f in... (2 Replies)
Discussion started by: 3therk1ll
2 Replies

7. Shell Programming and Scripting

Help modifying script to loop through all folders

I have this script someone very kindly help me write last year which loops through all files in a folder and does a command. I need to modify it to loop through all sub-folders of a main folder and only perform the command on files modified after Jan 1st 2008. And I need the command to place the... (3 Replies)
Discussion started by: Fred Goldman
3 Replies

8. Shell Programming and Scripting

Need a script find the folders and execute a code

I need a script which can search for two files in location /share/point/ which has year and month stamps example 1) Extract200806.txt 2)file_new200805.csv If these files exists then it should run a code named abc.sas (3 Replies)
Discussion started by: reachsarath
3 Replies

9. Shell Programming and Scripting

loop does not execute in bash script?

I have a very basic bash shell script, which has many "while... done; for .... done" loop clauses, like the following ~~ #!/bin/bash while blablalba; do .... done < /tmp/file for line in `cat blablabla`; do grep $line /tmp/raw ; done > /tmp/1; while blablalba2; do .... done <... (2 Replies)
Discussion started by: fedora
2 Replies

10. Shell Programming and Scripting

script to loop around folders

I have a folder called {homedata} Within this folder there are 12 subfolders 200601.......200612 Within each subfolder there are 8 sets of files Each filename commences with A B C D E F G or H, so {filename}* can be used. I am trying to write a script which will from the top level go... (2 Replies)
Discussion started by: grinder182533
2 Replies
Login or Register to Ask a Question