10-25-2013
Any access to such a file system will tend to be slow because of the many necessary calls to stat(), not only the reading of the (basically) inode-structure by du. To minimize not only your problem at hand but all similar problems regarding this filesystem i suggest you move the vital FS information (that is: inodes and the like - all the metainformation) to some high-bandwidth storage, like an SSD.
I saw a similar problem (backup/restore of a huge GPFS with ~500TB of data) solved by introducing a 150GB SSD holding just the metadata. It reduced the necessary time from ~6 hours to ~90 minutes using the same hardware.
I hope this helps.
bakunin
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
wich is the fastest command in HP-UX to copy an entire disk to dat tapes, or even disk to disk?
thanks (0 Replies)
Discussion started by: vascobrito
0 Replies
2. Shell Programming and Scripting
I would like to create a script for calculating size and remaining space of a directory automatically every 24 hours, then send an email to report to the admin.
* POSIX and PERL are preferred.
Can anyone help, please? (1 Reply)
Discussion started by: leonall
1 Replies
3. Shell Programming and Scripting
hello
i need help to remove directory . The directory is not empty ., it contains
several sub directories and files inside that..
total number of files in one directory is 12,24,446 .
rm -rf doesnt work . it is prompting for every file ..
i want to delete without prompting and... (6 Replies)
Discussion started by: getdpg
6 Replies
4. UNIX for Advanced & Expert Users
I want to search a file in fastest manner.
Presently I am using 'find' command. But it is taking around 15min for searching. Is there any other method through which I can do it fast? (3 Replies)
Discussion started by: vaibhavbhat
3 Replies
5. AIX
Hi, what happened is we want to count all the files in a directory and inside this directory got many folders and so take long time to count it. Already run for about few minutes but still not done. The command we use to count is find . -type f | wc -l
Just wondering if there is any other... (9 Replies)
Discussion started by: ngaisteve1
9 Replies
6. Solaris
Hi Experts,
I've been asked if there is a fast way to duplicate a file(10GB) and zip it at the same time. The zipped file would be FTP'd.....management is asking this. Maybe there is a better method all together? any ideas? CP will not cut it.
Thanks in advance
Harley (1 Reply)
Discussion started by: Harleyrci
1 Replies
7. Shell Programming and Scripting
Hello Guru s
I need your kind help to solve my below issue
I have a directory of flat files and have to calculate sum of some columns from the flat file .
Say for flat file 302 I need the column summation of 2 and 3 rd column
For flat file 303 I need the column summation of 5 and... (2 Replies)
Discussion started by: Pratik4891
2 Replies
8. Shell Programming and Scripting
Hi All,
I have the following time stamp data in 2 columns
Date TimeStamp(also with milliseconds)
05/23/2012 08:30:11.250
05/23/2012 08:30:15.500
05/23/2012 08:31.15.500
.
.
etc
From this data I need the following output.
0.00( row1-row1 in seconds)
04.25( row2-row1 in... (5 Replies)
Discussion started by: ks_reddy
5 Replies
9. Shell Programming and Scripting
How to Calculating space used in GB for any particular directory in unix
Currently I am using : df -h which gives me space for each mout point
ldndyn1:/vol/v01/dyn/sbcexp/dyn
1.1T 999G 29G 98% /sbcimp/dyn
but I need for some internal particular directory... (3 Replies)
Discussion started by: RahulJoshi
3 Replies
10. Shell Programming and Scripting
hello
we have upload some data in 15 servers in usa asia ...
i consider to add new feature , script can detect download speed between localhost and destination and use fastest server,
i have cut this part from a script which have this feature, download a xx MB file from all its source and... (0 Replies)
Discussion started by: nimafire
0 Replies
cp(1) General Commands Manual cp(1)
Name
cp - copy file data
Syntax
cp [ -f ] [ -i ] [ -p ] file1 file2
cp [ -f ] [ -i ] [ -p ] [ -r ] file... directory
cp [ -f ] [ -i ] [ -p ] [ -r ] directory... directory
Description
The command copies file1 onto file2. The mode and owner of file2 are preserved if it already existed; the mode of file1 is used otherwise.
Note that the command will not copy a file onto itself.
In the second form, one or more files are copied into the directory with their original file names.
In the third form, one or more source directories are copied into the destination directory with their original file names.
Options
-f Forces existing destination pathnames to be removed before copying, without prompting for confirmation. The -i option is ignored if
the -f option is specified.
-i Prompts user with the name of file whenever the copy will cause an old file to be overwritten. A yes answer will cause to continue.
Any other answer will prevent it from overwriting the file.
-p Preserves (duplicates) in the copies the modification time, access time, file mode, user ID, and group ID as allowed by the permis-
sions of the source files, ignoring the present umask.
-r Copies directories. Entire directory trees, including their subtrees and the individual files they contain, are copied to the speci-
fied destination directory. The directory, its subtrees, and the individual files retain their original names. For example, to copy
the directory including all of its subtrees and files, into the directory enter the following command:
cp -r reports news
See Also
cat(1), pr(1), mv(1)
cp(1)