Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Unix File System performance with large directories Post 48560 by Perderabo on Wednesday 10th of March 2004 09:13:35 AM
Old 03-10-2004
Large directories devastate unix performance. With most file systems directories can grow but not shrink. So even a directory that once contained 30,000 files is a problem.

Directories are searched sequentially. They obviously must be searched to open a file. But they must also be scanned to find an empty slot during file creation. If the directory has subdirectories, a /bin/pwd in those subdirectories involves a search of the massive parent directory.

A large directory may be fragmented and scattered across the filesystem. This exacerbates the situation.

Unix commands such as ls are designed with smaller directories in mind. ls will read the entire directory into memory, sort it, then display the results. Even a ls executed in the directory will place a significant burden on the system.

uname -a
will tell you which version of unix you are using.

df -n /some/directory
will usually tell you the type of filesystem.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

list directories on a file system

how can i see the list of directories, mounted on a filesystem? example, to show the list of directories mounted on / for aix (8 Replies)
Discussion started by: yls177
8 Replies

2. AIX

Why is my file system cache so large

Hi I have a filesystem cache which is around 20G in size and I'm a bit perplexed as to what is in it. I'm running Sybase on the machine with the db on raw volumes and a tempdb on a ramdisk. My understanding is that raw volumes are not cached and I assumed that the ramdisk is not either. Am... (1 Reply)
Discussion started by: mgibbons
1 Replies

3. UNIX for Advanced & Expert Users

suggestions require for unix system performance on certain task

Dear all, On my UNIX server there is an apache web log file. The rate of logging of data in this file is very high. I want to extract user logging log from this file in run time. As soon as the user logging log logged in this file I want to redirect this user log into another file. I want to... (4 Replies)
Discussion started by: zing_foru
4 Replies

4. Shell Programming and Scripting

Performance issue in UNIX while generating .dat file from large text file

Hello Gurus, We are facing some performance issue in UNIX. If someone had faced such kind of issue in past please provide your suggestions on this . Problem Definition: /Few of load processes of our Finance Application are facing issue in UNIX when they uses a shell script having below... (19 Replies)
Discussion started by: KRAMA
19 Replies

5. Programming

question about empty directories in unix system

how is it possible for a directory to be empty and still have a size greater than 0 in bytes... i made a shell script that shows info about all files/directories and this is what came up the last one is the size, here its showing 1024 in the for loop i did something like for h in * .*; do ... (4 Replies)
Discussion started by: omega666
4 Replies

6. Red Hat

Empty directory, large size and performance

Hi, I've some directory that I used as working directory for a program. At the end of the procedure, the content is deleted. This directory, when I do a ls -l, appears to still take up some space. After a little research, I've seen on a another board of this forum that it's not really taking... (5 Replies)
Discussion started by: bdx
5 Replies

7. Red Hat

GFS file system performance is very slow

My code Hi All, I am having redhat linux 5.3 (Tikanga) with GFS file system and its very very slow for executing ls -ls command also.Please see the below for 2minits 12 second takes. Please help me to fix the issue. $ sudo time ls -la BadFiles |wc -l 0.01user 0.26system... (3 Replies)
Discussion started by: susindram
3 Replies

8. Shell Programming and Scripting

Performance issue in Grepping large files

I have around 300 files(*.rdf,*.fmb,*.pll,*.ctl,*.sh,*.sql,*.prog) which are of large size. Around 8000 keywords(which will be in the file $keywordfile) needed to be searched inside those files. If a keyword is found in a file..I have to insert the filename,extension,catagoery,keyword,occurrence... (8 Replies)
Discussion started by: millan
8 Replies

9. HP-UX

Test cases for file system mount/umount performance in HP

Hi Folks, Could anyone please assist me with the what could be the scenarios to test the file system mount/umount performance check in HPUX. Thanks in advance, Vaishey (5 Replies)
Discussion started by: Vaishey
5 Replies

10. Shell Programming and Scripting

Bash script search, improve performance with large files

Hello, For several of our scripts we are using awk to search patterns in files with data from other files. This works almost perfectly except that it takes ages to run on larger files. I am wondering if there is a way to speed up this process or have something else that is quicker with the... (15 Replies)
Discussion started by: SDohmen
15 Replies
uuaids(8c)																uuaids(8c)

Name
       uucompact, uumkspool, uurespool, uupoll - uucp utilities

Syntax
       uucompact -ssystem
       uumkspool system ...
       uurespool [ -t# ]
       uupoll system ...

Description
       All of the commands are located in

       The  command  compacts  uucp  system spool directories and associated subdirectories. If system is ALL, then all existing uucp system spool
       directories are compacted.  Otherwise, only the specified system spool directory is compacted.  If no system is	specified,  is	compacted.
       If  is stopped before it is finished, it can be restarted without reprocessing directories.  The command continues processing where it left
       off during it's previous instantiation.

       The command makes a per system spool directory and associated subdirectories for each of the specified systems.	For example, if system	is
       mk3 and if the local system name is penny, the following directories are created:
	       /usr/spool/uucp/sys/mk3
	       /usr/spool/uucp/sys/mk3/C.
	       /usr/spool/uucp/sys/mk3/X.
	       /usr/spool/uucp/sys/mk3/D.
	       /usr/spool/uucp/sys/mk3/D.penny
	       /usr/spool/uucp/sys/mk3/D.penny
       The  command  moves files from old spool directories to	new spool directories.	Because the structure of the spool directories has changed
       from older versions of it is necessary to respool old spooled files to new spool directories in at least two instances:

       o    When installing the current version of

       o    When creating a new system spool directory for each system.

       In the latter case, it is necessary to move files from to the new spool directories.  To ease this task, moves files that have been spooled
       in  one	of 4 formats and respools them under the new spooling structure.  The format is specified by the -t# option, where the number sign
       (#) can be any one of the following:

       o   Original spool - All files are in

       o   Split spool - Contains the subdirectories

       o   Modified split spool -  Contains all subdirectories listed in split spool, and

       o   Used when a new system directory has been created and spool files must be moved from the DEFAULT directory to the new system directory.

       The command forces a connect attempt to the named systems even if recent attempts have failed, but not if the file prohibits the call.  For
       example, the file will prohibit the call if it is the wrong time of day.  Thus, the should be monitored for messages about the connection.

Files
       Spool directory

       Logfile

See Also
       mail(1), uucp(1c), uux(1c)

																	uuaids(8c)
All times are GMT -4. The time now is 06:19 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy