Sponsored Content
Top Forums Shell Programming and Scripting Fastest way to list a file in a folder containing 800,000 files using wildcard Post 302213218 by jerardfjay on Wednesday 9th of July 2008 01:36:07 PM
Old 07-09-2008
ls -1 wins

Looks like ls -1 is faster than find command.
Here are the results.

Code:
time ls -1 *.00abcdefg.Z
ABC.00abcdefg.Z
AB.00abcdefg.Z

real    0m2.05s
user    0m0.88s
sys     0m0.96s

Code:
 time find . -name "*.00abcdefg.Z"
./ABC.00abcdefg.Z
./AB.00abcdefg.Z

real    33m43.95s
user    0m3.02s
sys     0m29.87s

I am a bit surprised, since I would have imagined the find to be faster. However if there is some other mechanism other than ls -1 available I would be open to trying that as well. Thanks for your input.

Jerardfjay
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Find wildcard .shtml files in wildcard directories and removing them- How's it done?

I'm trying to figure out how to build a small shell script that will find old .shtml files in every /tgp/ directory on the server and delete them if they are older than 10 days... The structure of the paths are like this: /home/domains/www.domain2.com/tgp/ /home/domains/www.domain3.com/tgp/... (1 Reply)
Discussion started by: Neko
1 Replies

2. UNIX for Dummies Questions & Answers

Basic Q: getting list of all files of type within folder & subfolders

A painfully rudimentary UNIX question for somebody. I've been puzzling over this for the last hour but can't find the right command. I'm simply trying to get a list of all files - and their full paths - within a folder & subfolders which have extension .php and .js. That's it! No amount of... (1 Reply)
Discussion started by: AtomicPenguin
1 Replies

3. Shell Programming and Scripting

How list of files apart from *.class files in certain folder

Hi Friends How to list of all files in a particular directory structure, with out listing a particular file extn. I mean to say in a folder structure I have many files ..I want to list them with out *.class files. Please help Thanks Joy (3 Replies)
Discussion started by: itsjoy2u
3 Replies

4. UNIX for Dummies Questions & Answers

List certain file in a folder and make list

Hi, im having problem that frustate me today. there are list of file in a folder that i want to grab the folder /subject/items/ in this folder there are this file CREATE_SUBxxxx.xml UPDATE_SUBxxxx.xml DELETE_SUBxxxx.xml loginresponsexxxxx.xml core how can i grab all the file... (1 Reply)
Discussion started by: andrisetia
1 Replies

5. UNIX for Dummies Questions & Answers

how to search and list file with wildcard character

hi, I want to search all files in the current working direcotry and to print in comma (,) seperated output. But I have two patterns to search for. Files will be in ABC20100508.DAT format. Search should happen on the format (ABC????????.DAT) along with date(20100508). I can do a ls... (2 Replies)
Discussion started by: anandapani
2 Replies

6. Shell Programming and Scripting

Check folder existence using wildcard

Hi I would like to know how I can check whether there is one or more folders in the current directory which begins with e.g. 2011-11. Initially I figured that this could easily be done simply by: if ; then ... However if there is more than one folder which begins with 2011-11 then it... (1 Reply)
Discussion started by: aknu
1 Replies

7. Shell Programming and Scripting

List of files in a folder inclusive subfolder

Hi, I need to list the names of existing files in a specific folder. I have written a script for that, but the problem is, it is also picking up name of a subfolder that is there in that folder. I need only the list of files and not that subfolder. How to go about that ? Can anyone plz help... (2 Replies)
Discussion started by: Subhasis
2 Replies

8. Shell Programming and Scripting

How to copy all the contents of a list of files present in a folder to a particular file?

Hi All, I want to copy all the contents of a list of files in a folder to a particular file. i am using following command: cat dir/* >> newFile.txtIt's not working. Could you please help? Thanks, Pranav (3 Replies)
Discussion started by: Pranav Bhasker
3 Replies

9. Shell Programming and Scripting

Parse through ~21,000 Database DDL statements -- Fastest way to perform search, replace and insert

Hello All: We are looking to search through 2000 files with around 21,000 statements where we have to search, replace and insert a pattern based on the following: 1) Parse through the file and check for CREATE MULTISET TABLE or CREATE SET TABLE statements.....and they always end with ON... (5 Replies)
Discussion started by: madhunk
5 Replies

10. Shell Programming and Scripting

How to list and move files with spaces and wildcard?

I am writing a code that can move and archve all the files in a directory except the latest file based on file pattern provided in a controlfile. The filename is in the form of pattern. So basically we find the all the files of the pattern provided and archive all of them, leaving one latest file.... (3 Replies)
Discussion started by: Saanvi1
3 Replies
filelib(3erl)						     Erlang Module Definition						     filelib(3erl)

NAME
filelib - File utilities, such as wildcard matching of filenames DESCRIPTION
This module contains utilities on a higher level than the file module. The module supports Unicode file names, so that it will match against regular expressions given in Unicode and that it will find and process raw file names (i.e. files named in a way that does not confirm to the expected encoding). If the VM operates in Unicode file naming mode on a machine with transparent file naming, the fun() provided to fold_files/5 needs to be prepared to handle binary file names. For more information about raw file names, see the file module. DATA TYPES
filename() = string() | atom() | DeepList | RawFilename DeepList = [char() | atom() | DeepList] RawFilename = binary() If VM is in unicode filename mode, string() and char() are allowed to be > 255. RawFilename is a filename not subject to Unicode translation, meaning that it can contain characters not conforming to the Unicode encoding expected from the filesystem (i.e. non-UTF-8 characters although the VM is started in Unicode filename mode). dirname() = filename() EXPORTS
ensure_dir(Name) -> ok | {error, Reason} Types Name = filename() | dirname() Reason = posix() -- see file(3erl) The ensure_dir/1 function ensures that all parent directories for the given file or directory name Name exist, trying to create them if necessary. Returns ok if all parent directories already exist or could be created, or {error, Reason} if some parent directory does not exist and could not be created for some reason. file_size(Filename) -> integer() The file_size function returns the size of the given file. fold_files(Dir, RegExp, Recursive, Fun, AccIn) -> AccOut Types Dir = dirname() RegExp = regular_expression_string() Recursive = true|false Fun = fun(F, AccIn) -> AccOut AccIn = AccOut = term() The fold_files/5 function folds the function Fun over all (regular) files F in the directory Dir that match the regular expression RegExp (see the re module for a description of the allowed regular expressions). If Recursive is true all sub-directories to Dir are processed. The regular expression matching is done on just the filename without the directory part. If Unicode file name translation is in effect and the file system is completely transparent, file names that cannot be interpreted as Unicode may be encountered, in which case the fun() must be prepared to handle raw file names (i.e. binaries). If the regular expression contains codepoints beyond 255, it will not match file names that do not conform to the expected character encoding (i.e. are not encoded in valid UTF-8). For more information about raw file names, see the file module. is_dir(Name) -> true | false Types Name = filename() | dirname() The is_dir/1 function returns true if Name refers to a directory, and false otherwise. is_file(Name) -> true | false Types Name = filename() | dirname() The is_file/1 function returns true if Name refers to a file or a directory, and false otherwise. is_regular(Name) -> true | false Types Name = filename() The is_regular/1 function returns true if Name refers to a file (regular file), and false otherwise. last_modified(Name) -> {{Year,Month,Day},{Hour,Min,Sec}} | 0 Types Name = filename() | dirname() The last_modified/1 function returns the date and time the given file or directory was last modified, or 0 if the file does not exist. wildcard(Wildcard) -> list() Types Wildcard = filename() | dirname() The wildcard/1 function returns a list of all files that match Unix-style wildcard-string Wildcard . The wildcard string looks like an ordinary filename, except that certain "wildcard characters" are interpreted in a special way. The following characters are special: ? : Matches one character. * : Matches any number of characters up to the end of the filename, the next dot, or the next slash. [Character1,Character2,...] : Matches any of the characters listed. Two characters separated by a hyphen will match a range of characters. Example: [A-Z] will match any uppercase letter. {Item,...} : Alternation. Matches one of the alternatives. Other characters represent themselves. Only filenames that have exactly the same character in the same position will match. (Match- ing is case-sensitive; i.e. "a" will not match "A"). Note that multiple "*" characters are allowed (as in Unix wildcards, but opposed to Windows/DOS wildcards). Examples: The following examples assume that the current directory is the top of an Erlang/OTP installation. To find all .beam files in all applications, the following line can be used: filelib:wildcard("lib/*/ebin/*.beam"). To find either .erl or .hrl in all applications src directories, the following filelib:wildcard("lib/*/src/*.?rl") or the following line filelib:wildcard("lib/*/src/*.{erl,hrl}") can be used. To find all .hrl files in either src or include directories, use: filelib:wildcard("lib/*/{src,include}/*.hrl"). To find all .erl or .hrl files in either src or include directories, use: filelib:wildcard("lib/*/{src,include}/*.{erl,hrl}") wildcard(Wildcard, Cwd) -> list() Types Wildcard = filename() | dirname() Cwd = dirname() The wildcard/2 function works like wildcard/1 , except that instead of the actual working directory, Cwd will be used. Ericsson AB stdlib 1.17.3 filelib(3erl)
All times are GMT -4. The time now is 05:12 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy