Sponsored Content
Top Forums Shell Programming and Scripting Fastest way calculating directory Post 302867861 by jim mcnamara on Friday 25th of October 2013 06:39:35 AM
Old 10-25-2013
Code:
df -h

will run much faster.

When you have gigantic directories many kinds of filesystems perform very poorly. du reads information on a per file basis, in your case millions of file reads (calls to stat). df gets information stored in the kernel about whole filesystems. One read per filesystem.

At some point you should attempt to reorganize your directories so that you don't have what appears to me to be an unmanageable mess.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

fastest copy command

wich is the fastest command in HP-UX to copy an entire disk to dat tapes, or even disk to disk? thanks (0 Replies)
Discussion started by: vascobrito
0 Replies

2. Shell Programming and Scripting

Scripts for calculating size and remaining space of a directory automatically.

I would like to create a script for calculating size and remaining space of a directory automatically every 24 hours, then send an email to report to the admin. * POSIX and PERL are preferred. Can anyone help, please? (1 Reply)
Discussion started by: leonall
1 Replies

3. Shell Programming and Scripting

how to delete/remove directory in fastest way

hello i need help to remove directory . The directory is not empty ., it contains several sub directories and files inside that.. total number of files in one directory is 12,24,446 . rm -rf doesnt work . it is prompting for every file .. i want to delete without prompting and... (6 Replies)
Discussion started by: getdpg
6 Replies

4. UNIX for Advanced & Expert Users

Fastest way for searching the file

I want to search a file in fastest manner. Presently I am using 'find' command. But it is taking around 15min for searching. Is there any other method through which I can do it fast? (3 Replies)
Discussion started by: vaibhavbhat
3 Replies

5. AIX

Fastest way to count big amount of files in sub directory

Hi, what happened is we want to count all the files in a directory and inside this directory got many folders and so take long time to count it. Already run for about few minutes but still not done. The command we use to count is find . -type f | wc -l Just wondering if there is any other... (9 Replies)
Discussion started by: ngaisteve1
9 Replies

6. Solaris

The FASTEST copy method?

Hi Experts, I've been asked if there is a fast way to duplicate a file(10GB) and zip it at the same time. The zipped file would be FTP'd.....management is asking this. Maybe there is a better method all together? any ideas? CP will not cut it. Thanks in advance Harley (1 Reply)
Discussion started by: Harleyrci
1 Replies

7. Shell Programming and Scripting

calculating column summation in a directory of flat files

Hello Guru s I need your kind help to solve my below issue I have a directory of flat files and have to calculate sum of some columns from the flat file . Say for flat file 302 I need the column summation of 2 and 3 rd column For flat file 303 I need the column summation of 5 and... (2 Replies)
Discussion started by: Pratik4891
2 Replies

8. Shell Programming and Scripting

Calculating the epoch time from standard time using awk and calculating the duration

Hi All, I have the following time stamp data in 2 columns Date TimeStamp(also with milliseconds) 05/23/2012 08:30:11.250 05/23/2012 08:30:15.500 05/23/2012 08:31.15.500 . . etc From this data I need the following output. 0.00( row1-row1 in seconds) 04.25( row2-row1 in... (5 Replies)
Discussion started by: ks_reddy
5 Replies

9. Shell Programming and Scripting

How to Calculating space used in GB for any particular directory in UNIX?

How to Calculating space used in GB for any particular directory in unix Currently I am using : df -h which gives me space for each mout point ldndyn1:/vol/v01/dyn/sbcexp/dyn 1.1T 999G 29G 98% /sbcimp/dyn but I need for some internal particular directory... (3 Replies)
Discussion started by: RahulJoshi
3 Replies

10. Shell Programming and Scripting

Check fastest server and using it

hello we have upload some data in 15 servers in usa asia ... i consider to add new feature , script can detect download speed between localhost and destination and use fastest server, i have cut this part from a script which have this feature, download a xx MB file from all its source and... (0 Replies)
Discussion started by: nimafire
0 Replies
xfs_fsr(8)						      System Manager's Manual							xfs_fsr(8)

NAME
xfs_fsr - filesystem reorganizer for XFS SYNOPSIS
xfs_fsr [-v] [-t seconds] [-f leftoff] [-m mtab] xfs_fsr [-v] [xfsdev | file] ... DESCRIPTION
xfs_fsr is applicable only to XFS filesystems. xfs_fsr improves the organization of mounted filesystems. The reorganization algorithm operates on one file at a time, compacting or oth- erwise improving the layout of the file extents (contiguous blocks of file data). The following options are accepted by xfs_fsr. The -m, -t, and -f options have no meaning if any filesystems or files are specified on the command line. -m mtab Use this file for the list of filesystems to reorganize. The default is to use /etc/mtab. -t seconds How long to reorganize. The default is 7200 (2 hours). -f leftoff Use this file instead of /var/tmp/.fsrlast to read the state of where to start and as the file to store the state of where reorganization left off. -v Verbose. Print cryptic information about each file being reorganized. When invoked with no arguments xfs_fsr reorganizes all regular files in all mounted filesystems. xfs_fsr makes many cycles over /etc/mtab each time making a single pass over each XFS filesystem. Each pass goes through and selects files that have the largest number of extents. It attempts to defragment the top 10% of these files on each pass. It runs for up to two hours after which it records the filesystem where it left off, so it can start there the next time. This information is stored in the file /var/tmp/.fsrlast_xfs. If the information found here is somehow inconsistent or out of date it is ignored and reor- ganization starts at the beginning of the first filesystem found in /etc/mtab. xfs_fsr can be called with one or more arguments naming filesystems (block device name), and files to reorganize. In this mode xfs_fsr does not read or write /var/tmp/.fsrlast_xfs nor does it run for a fixed time interval. It makes one pass through each specified regular file and all regular files in each specified filesystem. A command line name referring to a symbolic link (except to a file system device), FIFO, or UNIX domain socket generates a warning message, but is otherwise ignored. While traversing the filesystem these types of files are silently skipped. FILES
/etc/mtab contains default list of filesystems to reorganize. /var/tmp/.fsrlast_xfs records the state where reorganization left off. SEE ALSO
xfs_fsr(8), mkfs.xfs(8), xfs_ncheck(8), xfs(5). NOTES
xfs_fsr improves the layout of extents for each file by copying the entire file to a temporary location and then interchanging the data extents of the target and temporary files in an atomic manner. This method requires that enough free disk space be available to copy any given file and that the space be less fragmented than the original file. It also requires the owner of the file to have enough remaining filespace quota to do the copy on systems running quotas. xfs_fsr generates a warning message if space is not sufficient to improve the target file. A temporary file used in improving a file given on the command line is created in the same parent directory of the target file and is pre- fixed by the string '.fsr'. The temporary files used in improving an entire XFS device are stored in a directory at the root of the target device and use the same naming scheme. The temporary files are unlinked upon creation so data will not be readable by any other process. xfs_fsr does not operate on files that are currently mapped in memory. A 'file busy' error can be seen for these files if the verbose flag (-v) is set. Files marked as no-defrag will be skipped. The xfs_io(8) chattr command with the f attribute can be used to set or clear this flag. Files and directories created in a directory with the no-defrag flag will inherit the attribute. An entry in /etc/mtab or the file specified using the -m option must have the rw option specified for read and write access. If this option is not present, then xfs_fsr skips the filesystem described by that line. See the fstab(5) reference page for more details. In general we do not foresee the need to run xfs_fsr on system partitions such as /, /boot and /usr as in general these will not suffer from fragmentation. There are also issues with defragmenting files lilo(8) uses to boot your system. It is recommended that these files should be flagged as no-defrag with the xfs_io(8) chattr command. Should these files be moved by xfs_fsr then you must rerun lilo before you reboot or you may have an unbootable system. xfs_fsr(8)
All times are GMT -4. The time now is 02:19 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy