Cull a Directory List?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Cull a Directory List?
# 1  
Old 04-08-2014
Cull a Directory List?

Greetings!

I'm trying to take a directory list (such as nn.list in dpkg/info) and sort out the line content which furnishes only dirs from that which provides paths to actual files with a bash "one liner". From there, I want to pass the culled result to stdout for more pipework down the line...

So, here's the pseudo (broken) code which I've been playing around with to no avail. From a terminal in dpkg/info:
Code:
read < xx.list | find -type f  '*.*' | sort

(I know, it's incomplete and rather "Monty Python" -- but it was the best I could figure Smilie )

At any rate, is there a better builtin or a straightforward way of doing something like this in Bash?

Thanks a bunch!

Last edited by LinQ; 04-08-2014 at 12:01 PM.. Reason: Better title ;o)
# 2  
Old 04-08-2014
Show the input you have and the output you want please. Broken code which doesn't do what you want sadly isn't very instructive!
# 3  
Old 04-08-2014
Try:
Code:
sort xx.list | while read -r file
do      if [ -f "$file" ]
        then    printf "%s\n" "$file"
        fi
done

Oops. That -f should have been a -r. It is fixed now.

Last edited by Don Cragun; 04-08-2014 at 12:36 PM.. Reason: Fix typo.
This User Gave Thanks to Don Cragun For This Post:
# 4  
Old 04-08-2014
Thanks for dropping by, folks!

@Corona688:

Sorry 'bout that. Here's a quick rundown of the functionality I was looking at.

To start, we have a hypothetical file, xx.list, which consists of nothing but these three lines:
Code:
/foo
/foo/bar.txt
/foo/bar/yum.txt

Now, from there, I was attempting to process this file, xx.list, through 'read' and 'find' to simply pass
Code:
/foo/bar.txt
/foo/bar/yum.txt

along to 'sort' and off to the next pipe for more processing (via xargs, etc).

For simplicity's sake, just wanted a quick "one liner" of code to do the work Smilie


@Don Cragun:

Thanks for the snippet!

As an adaptation, would it be possible to pipe
Code:
sort xx.list | xargs

to a quick inline test of some sort to cull out the true file paths for even more downstream processing? Still getting a handle on the "quirks and handshakes" of Bash, and quite amazed at the hidden functionality contained therein...

Thanks again, all Smilie
# 5  
Old 04-08-2014
Quote:
Originally Posted by LinQ
Thanks for dropping by, folks!

@Corona688:

Sorry 'bout that. Here's a quick rundown of the functionality I was looking at.

To start, we have a hypothetical file, xx.list, which consists of nothing but these three lines:
Code:
/foo
/foo/bar.txt
/foo/bar/yum.txt

Now, from there, I was attempting to process this file, xx.list, through 'read' and 'find' to simply pass
Code:
/foo/bar.txt
/foo/bar/yum.txt

along to 'sort' and off to the next pipe for more processing (via xargs, etc).

For simplicity's sake, just wanted a quick "one liner" of code to do the work Smilie


@Don Cragun:

Thanks for the snippet!

As an adaptation, would it be possible to pipe
Code:
sort xx.list | xargs

to a quick inline test of some sort to cull out the true file paths for even more downstream processing? Still getting a handle on the "quirks and handshakes" of Bash, and quite amazed at the hidden functionality contained therein...

Thanks again, all Smilie
I'm lost. I thought you said you wanted a sorted list of regular files to be created from the unsorted list of all sorts of files found in a file named xx.list. The script I gave you does that and provides the output with one filename per line. The output of that script can be piped into anything you want.

If you're saying it has to be a one-liner, change it to:
Code:
sort xx.list|while read -r file;do if [ -f "$file" ];then printf "%s\n" "$file";fi;done

but I'll take readable over one-liner every time.

Code:
sort xx.list | xargs

produces a sorted list of all of the filenames (without checking for the file type) and uses a combination of spaces and newlines (depending on the number of files in the list and the lenghts of the filenames) as filename terminators. Why do you believe that xargs will weed out the names of files that are not regular files? In cases where one or more filenames spaces, feeding a list of those filenames through xargs will create a list that is much harder to process in anything it in a pipeline.
# 6  
Old 04-08-2014
@Don Cragun:

I do apologize if my additional question presented a difficulty. The call to xargs was not for filtering, but formatting for the next stage, if necessary.

The command will be entered at the terminal directly; hence the need for a brief, one-line segment. Needed something easy enough to remember and keypunch without creating too many typos.

Thank you again for the post and your assistance...
# 7  
Old 04-08-2014
Ignore this -- I should have Don's post more carefully....

You should consider testing each entry in the to see if it is a regular file, in case some directories exist below /foo - ex: /foo/path/anotherdir

Code:
[ -d $variable ]

or
Code:
[ ! -f $variable ]

if there no other file types at all except regular files and directories.

does that for you. I know that "right now" there may be none, but you seem to imply you want to keep this code around.

Code:
while read name
do
  [  -d "$name" ] && echo "$name"
done < inputfile |  ... rest of pipe stream


Last edited by jim mcnamara; 04-08-2014 at 06:20 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

List files with date, create directory, move to the created directory

Hi all, i have a folder, with tons of files containing as following, on /my/folder/jobs/ some_name_2016-01-17-22-38-58_some name_0_0.zip.done some_name_2016-01-17-22-40-30_some name_0_0.zip.done some_name_2016-01-17-22-48-50_some name_0_0.zip.done and these can be lots of similar files,... (6 Replies)
Discussion started by: charli1
6 Replies

2. Shell Programming and Scripting

Copy list of files from a keyword list to another directory

Hello, I have a folder with a massive amount of files, and I want to copy out a specific subset of the files to a new directory. I would like to use a text file with the filenames listed, but can't get it to work. The thing I'm hung up on is that the folder names in the path can and do have... (5 Replies)
Discussion started by: twjolson
5 Replies

3. UNIX for Dummies Questions & Answers

List the files in directory

hi, i have so many files in my directory. for example $: ls abc.12 abc.23 abc.45 abc.56 abc.123 abc.343 abc.567 abc.345 like this. from this list, how to display only the files which ends with two numbers. If i give abc.*, it displays all the files. (2 Replies)
Discussion started by: JSKOBS
2 Replies

4. Solaris

How list all directories in a directory?

Hi How can i list all the directories inside a directory? not only the directories in the present dir. also the directories inside directories present in the dirrectory. recursive. "find . -type d" will work with other Unix flavors but not with SunOS. Please help Robin (1 Reply)
Discussion started by: robinbannis
1 Replies

5. Shell Programming and Scripting

find list of files from a list and copy to a directory

I will be very grateful if someone can help me with bash shell script that does the following: I have a list of filenames: A01_155716 A05_155780 A07_155812 A09_155844 A11_155876 that are kept in different sub directories within my current directory. I want to find these files and copy... (3 Replies)
Discussion started by: manishabh
3 Replies

6. Homework & Coursework Questions

List Files and Directory

Use and complete the template provided. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data i need to list all files in date/time order that contain the word "alpha" but not the the word "beta". 2. Relevant equations find or ls ... (7 Replies)
Discussion started by: jeht
7 Replies

7. Shell Programming and Scripting

To list files only in a directory

Hi all, I want to list all the files in a directory but not present in the subdirectories. I want the name of each file with its full pathname. ls /opt/cems/log/cemsdbg/db/* /opt/cems/log/cemsdbg/db/securitydb_backup_2009_01_12_06:00:54.log /opt/cems/log/cemsdbg/db/standby_monitor.sql... (5 Replies)
Discussion started by: dipashre
5 Replies

8. UNIX for Dummies Questions & Answers

Directory list inside a directory

Hi Bosses! I have a directory name sih. that directory contains some more directories and some files. i just want to list (ls) the directories under this directory. What will be the command.I am using debian linux. Thanks bosses.will appreciate your help. sih (6 Replies)
Discussion started by: little_jhon
6 Replies

9. UNIX for Dummies Questions & Answers

list of files in one Directory

hi All, How do I get list of all the files in one directory. Thanks in advance, Venky. (3 Replies)
Discussion started by: venkyA
3 Replies

10. Programming

list directory without use ls command?

Hi i need C code to list direcoty without using ls command and it sould take the path of the directory from the user? "please help me it is so important" (3 Replies)
Discussion started by: ashamry
3 Replies
Login or Register to Ask a Question