How to loop command over folders in a directory?

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers How to loop command over folders in a directory?
# 1  
Old 05-31-2016
How to loop command over folders in a directory?

Hello all,

I'm fairly new to scripting so please bear with me. I will try to explain as best as I can but if there's anything that is not clear, please let me know.

I have a directory (see below) with numerous folders and within each of those folders are various files. I would like to run a script that will go into each folder and then select the specified files and then repeat this process for all the folders that are in a given directory. Here is an example of how the path is set up:



The folders contain multiple files with various extensions. I would like the script to go into random#1 folder, look for the specified files, execute the command, and then move onto random#2 folder and do the same.

Inside each folder are the following files:

I have the following script that only works on one specific type of file inside each folder but I need to write a new one that works on specified files (above) located inside each folder in the directory.

How can I make it so that I can replace the above to make a list of all the folders and then run a script that goes inside each folder and looks for the specified files?

Basically, I need the script to go into the directory, into the subdirectory, into the folders, and look for specified files in those folders. And repeat for every folder there is.

I have exhausted all options of self-teaching/experimenting to figure this out on my own and therefore, I'm hoping I can find some solution here.

Thank you in advance.

Last edited by azurite; 06-15-2016 at 12:46 AM..
# 2  
Old 05-31-2016
Let us start with a description of your environment:

- which OS are you using (and which version of it)
- which shell are you using (and, maybe, which version of it)

the answers to these questions may well influence the final solution to your problem.

Now, the general solution to your problem is the find-command: basically it produces a list of files, which (see the -exec-clause) then can be subjected to some command. For a (short) introduction see here

I hope this helps.
# 3  
Old 05-31-2016
Quote:
Originally Posted by bakunin
Let us start with a description of your environment:

- which OS are you using (and which version of it)
- which shell are you using (and, maybe, which version of it)

the answers to these questions may well influence the final solution to your problem.

Now, the general solution to your problem is the find-command: basically it produces a list of files, which (see the -exec-clause) then can be subjected to some command. For a (short) introduction see here

I hope this helps.
I am using Ubuntu 12.04 and the shell is the default one that comes with it.


So I have a directory with about 100 folders inside and each of those folders has the following 4 files inside it. I want the script to go into the first folder, look for the files, run the analysis on those files, and then go through the 2nd folder and the 3rd and so on.


I am trying to input those files into this code to automate the analysis instead of having to manually input files using the gui for the program I am using.


Can I just put the script inside the directory with the folder and the text file (with list of folders) and run it from there?

Last edited by azurite; 06-15-2016 at 12:47 AM..
# 4  
Old 05-31-2016
Quote:
Originally Posted by azurite
I am using Ubuntu 12.04 and the shell is the default one that comes with it.
OK. If i am not mistaken Ubuntu uses dash as the default shell, which is (more or less, to be honest i don't know exactly) the same as bash. Perhaps there are some differences, but they are minor.

Quote:
Originally Posted by azurite
So I have a directory with about 100 folders inside and each of those folders has the following 4 files inside it. I want the script to go into the first folder, look for the files, run the analysis on those files, and then go through the 2nd folder and the 3rd and so on.
I understood that already. Now, such a task breaks down to two parts: first, write a script which takes a certain directory name and then executes the task you described below for that one directory (respectively the files in this directory). The second part is to create a list of these directories and then call the script with each of these directory names as arguments.

You probably have overseen my mentioning of the find-command. It has everything you need to do this second part (creating a list of directory names) and i suggest you follow the link i gave you and find out how it works. Once you get that covered we will find out how to write a script which does what you want for one directory supplied on the commandline.

Quote:
Originally Posted by azurite
Can I just put the script inside the directory with the folder and the text file (with list of folders) and run it from there?
No, it doesn't matter at all where the script resides. It is a good habit to put all your (personal) scripts into one directory - the common name being "bin" and located in your home directory. If there is no such directory i suggest you create it and then modify the file .profile in you home directory to include this line:

Code:
PATH="${PATH}:${HOME}/bin" ; export PATH

Now you can put all your scripts into this directory and call them without having to explicitly type the path. (You need to log off and log on again for this to take effect).

I hope this helps.

bakunin
# 5  
Old 05-31-2016
Instead of cd `echo $CASE` you simply do cd "$CASE".
Instead of "cd" you can use "pushd" and return to the previous directory with "popd".
Code:
foreach ...
  pushd "$CASE"
  dtifit ...
  popd
end

You are using csh; instead I recommend to go to bash or ksh. Then, instead of the pushd/popd you can use a ( sub shell ) where the "cd" happens only in the sub shell and the current directory in the main shell is unchanged
Code:
#!/bin/bash
for CASE in `cat "DWI_cases_image_path"`
do
  # this is the main shell
  (
     # this is the sub shell
     cd "$CASE" || exit  # on error exit the sub shell
     dtifit ...
  )
  # sub shell ended, we are back in the main shell
done

# 6  
Old 05-31-2016
Code:
-rwxr-xr-x 1 root root 929K 2011-05-18 03:00 /bin/bash 
-rwxr-xr-x 1 root root 108K 2011-05-03 08:04 /bin/dash

There is definitively some lack of features in dash which causes bash to use 800 K more in size.

But in the present case here i doubt it will make a any difference at all.
# 7  
Old 05-31-2016
Yes there is a lack of 50k features in dash. On the other hand bash has 750k features too many Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Trying to loop through folders and execute an existing python script.

I am trying to loop through lots and lots of folders and use the names of the folders to run a Python script which has parameters. E.g. -- setup_refs -n John -f England/London/Hackney/John -c con/con.cnf Normally to run `setup_refs` once from command line it's: `python setup_refs.py -n John... (3 Replies)
Discussion started by: Mr_Keystrokes
3 Replies

2. UNIX for Beginners Questions & Answers

Loop through the folders and search for particular string in files

Hello, Opearting System Environment : HP Unix B.11.31 U I look for script to On specific folders list On specific filelist Search for given string For Example : r48_buildlib.txt contains wpr480.0_20161027 wpr480.0_20161114 wpr481.0_20161208 wpr482.0_20161222... (4 Replies)
Discussion started by: Siva SQL
4 Replies

3. Shell Programming and Scripting

Bash directory loop, but only choose those folders with specific word in it

Hello, how in bash i can get directory loop, but only choose those folders with specific word in it, so it will only echo those with specific word #!/bin/bash for filename in /home/test/* do if ; then echo $filename; fithx! (4 Replies)
Discussion started by: ZerO13
4 Replies

4. Shell Programming and Scripting

Copy the files in directory and sub folders as it is to another directory.

How to copy files from one directory to another directory with the subfolders copied. If i have folder1/sub1/sub2/* it needs to copy files to folder2/sub1/sub2/*. I do not want to create sub folders in folder2. Can copy command create them automatically? I tried cp -a and cp -R but did... (4 Replies)
Discussion started by: santosh2626
4 Replies

5. UNIX for Dummies Questions & Answers

Counting number of folders in a Directory

Help Needed ! Can we count number of folders of specific date in a directory, even if directory has folders of different dates. Please reply as soon as possible. (1 Reply)
Discussion started by: vishal_215
1 Replies

6. Shell Programming and Scripting

Loop folders, delete files, copy new ones

Folks, I am hopeful that you may be able to help me out with writing a script that can be run nightly (as cron?) to loop through all subfolders within the "/media" directory, delete all of the files in each of them, and then copy in all of the files from the "/home//sansa" directory to each of... (6 Replies)
Discussion started by: acraig
6 Replies

7. Shell Programming and Scripting

Loop through folders and open them

Well the title says it all. I need a loop script that directs to a folder and opens a bash file. I just found out about loop files the other day and yeah, dont know if that can be done. (6 Replies)
Discussion started by: snoopy817
6 Replies

8. Shell Programming and Scripting

Help modifying script to loop through all folders

I have this script someone very kindly help me write last year which loops through all files in a folder and does a command. I need to modify it to loop through all sub-folders of a main folder and only perform the command on files modified after Jan 1st 2008. And I need the command to place the... (3 Replies)
Discussion started by: Fred Goldman
3 Replies

9. Shell Programming and Scripting

Prepend name of directory to children folders

Hi, I am a shell scripting newbie. I am in need of a shell script that will prepend the name of the parent directory to the child directory. For example if the shell script called rename.sh is invoked with ">rename.sh /home/foobar/Simple" and the structure of the folder Simple is : Simple... (7 Replies)
Discussion started by: kalichar
7 Replies

10. Shell Programming and Scripting

script to loop around folders

I have a folder called {homedata} Within this folder there are 12 subfolders 200601.......200612 Within each subfolder there are 8 sets of files Each filename commences with A B C D E F G or H, so {filename}* can be used. I am trying to write a script which will from the top level go... (2 Replies)
Discussion started by: grinder182533
2 Replies
Login or Register to Ask a Question