Pick one file from each subdirectory


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Pick one file from each subdirectory
# 1  
Old 01-27-2011
Pick one file from each subdirectory

Hi, I have a problem I am trying to solve with bash.

I need to search in a file system (data base) with hundreds of directories and thousands of subdirectories and millions of files. The files have a specific format with a header that gives the properties. Directories are organized so subdirectores have some identical header properties, thus you can rule out a whole directory by testing a single file.

My plan was to get one file from every single subdirectory and test its header. However I can't think of a neat way to do it. The bash command DIRS=`find . -mindepth 3 -maxdepth 3 -type d` and then looping over $DIRS with the ls command is effective but requires an ls command on every single subdirectory..

I was wondering if there is a way to pick any file from $DIR without doing the ls?

Thanks!
# 2  
Old 01-27-2011
Picks 1st file in alpha order:

Code:
for dir in *
do
   [ -d $dir ] || continue
   for sub in $dir/*
   do
       [ -d $sub ] || continue
       for file in $sub/*
       do
            # process $file header here
            break
       done
   done
done

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Search for specific file type in subdirectory with multiple folders

I have a directory that is in the below order (the --- is not part of the directory tree, only there to help illustrate: DATE --- main level Folder1 --- level under DATE plugin_out --- level under Folder1 variantCaller_out.40 --- level under plugin_out 001,002,003 --- level under... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. UNIX for Dummies Questions & Answers

List all File Content its subdirectory

Is there anyway to display the file content in a subdirectory that starts or ends with a certain filename, say .txt? or start with NTS $ ls * dos2unix.bat dos2unix.exe test.txt FilePermissions: filepermission.html NTSLA32_SystemDrive.txt NTSLA36_regedit.txt filepermission.sh ... (1 Reply)
Discussion started by: alvinoo
1 Replies

3. Shell Programming and Scripting

Searching File in Directory and all Subdirectory and Delete

Hi All, My directory structure is like Directory1 SubDirectory1 SubDirectory2 SubDirectory3 I have main directories and subdirectories underneath. I want to write a shell script where I will be passing file name as a parameter, Now I want to find all the files in Directory1... (19 Replies)
Discussion started by: John William
19 Replies

4. Shell Programming and Scripting

how to count how many subdirectory containing more than a certain number of specific file type

hi I want to write a script which count the number of subdirectories in the current root directory that contain more than a specified number of files of a specific type. Is there an easy way to do this? Thanks Robert (2 Replies)
Discussion started by: piynik
2 Replies

5. Shell Programming and Scripting

Move all files not in a directory into a subdirectory named for each given file

Hi Everyone! Looking for some help with a script that will take all files in any given root folder (which are not already in a folder) and put them into separate folders with the name of each given file. Any ideas? Thank you! (1 Reply)
Discussion started by: DanTheMan
1 Replies

6. Shell Programming and Scripting

How can i find a word inside a file from a directory and it's subdirectory ?

Suppose i have a word "mail". I have to search this word in all files inside a directory and it's sub-directories. It will also search in all hidden directory and sub-directories. If it finds this word in any file it will list that file. How can i do this with perl/ruby/awk/sed/bash or... (9 Replies)
Discussion started by: cola
9 Replies

7. Shell Programming and Scripting

Search for file and size subdirectory as well

Hi I made this code to search in directory for file and size How can I remodel it to seach in the sub direcotry as well Thanks #!/bin/bash echo -n "Enter: " read var if then echo "Directory exists: ${var}" size=`du -hs "${var}"` echo The size of the current folder is... (4 Replies)
Discussion started by: lio123
4 Replies

8. Shell Programming and Scripting

Sum of file size in directory / subdirectory

Hi , I am trying to write something to find the size of particular type of files in a directory & it's subdirectory and sum the size .. These types of file are found at directory level or its subdirectories level .. #!/bin/ksh FNAME='.pdf' S_PATH=/abc/def/xyz find $S_PATH -exec ls -lad... (4 Replies)
Discussion started by: Vaddadi
4 Replies

9. UNIX for Dummies Questions & Answers

Create Year directory, date subdirectory and archive the file

Hi, After checking all the UNIX threads, I am able to come up with a solution so far. I am working on a shell script where it moves the files to a certain directory. The conditions to check are 1) Check if the file exists in the current directory. 2) Check if the destination directory... (2 Replies)
Discussion started by: madhunk
2 Replies

10. Shell Programming and Scripting

How to calculate file's size in directory and subdirectory

Hi, I have written one script to calculate total space of all file in one directory, ignoring subdirectory, it works fine. Now, I've been trying to calculate all files which includes files in any subdirectories. I use recursive function to do this, but it can work only if there is only one... (4 Replies)
Discussion started by: KLL
4 Replies
Login or Register to Ask a Question