Sponsored Content
Operating Systems HP-UX Potential file system contention on directory Post 302469328 by DGPickett on Friday 5th of November 2010 01:35:56 PM
Old 11-05-2010
Huge directories with many users can be a big bottleneck. A basic UFS directory is a huge unsorted and unstructured pile of name-inode# pairs, so access is by brute force. Create a tree and move files down.

PS: You can only shrink a directory, once you thin it out, by recreating it: make a sibling directory new/ right beside it (same file system so links are legal), clone the contents by cpio pass with use links (less space, faster), make directories and preserve times and permissions, then rename to swap it out, and later remove orig/:
Code:
mkdir new
cd old_dir
find . |cpio -pdal ../new
cd ..
mv old_dir orig
mv new old_dir
#later
rm -rf orig

I tried this on my older PARISC HP-UX, and the rm was problematic because it was on an nfs mount and a (script) file was open (even though linked elsewhere, so this is a bug in rm or nfs), so it left a .nfs#### file until it was closed. I used:
Code:
fuser <file> <dir>
ps -fp <pid1>,<pid2>

to identify the processes holding the file open.

Last edited by DGPickett; 11-05-2010 at 02:43 PM..
 

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Distinction b/w file system and directory

Can you give a very basic definition of file system vs. directory in unix. The context is around monitoring for disk capacity maxing out. Why would /appl/sbe/inst_2/config/logs not be able to be because it is a directory but appl/cce can be monitored because it is a unix file system. They... (1 Reply)
Discussion started by: kd9626
1 Replies

2. UNIX for Dummies Questions & Answers

how many directory can be mounted on one file system

I have a question and seek help. How many directory can be mounted on one file system on UNIX with solaris 9? For example, I have one file system as /dev/dsk/cieit0a6. I have created one directory as /u01/app/oracle and mounted this directory to cieit06. It works. Then I create another directory as... (4 Replies)
Discussion started by: duke0001
4 Replies

3. UNIX for Dummies Questions & Answers

Potential new user of Unix

Hi all, Complete and utter virgin Unix person here (I don't even have the OS yet) As I'm doing a "looking into it" kinda thing before I move from MS I hope my questions are not inappropriate. 1. Should I get some kind off anti virus software. I know Unix is pretty good for not getting them... (2 Replies)
Discussion started by: dhula
2 Replies

4. Shell Programming and Scripting

Need to find the percentage of the directory in the file system.

Hi All, I want to find the percentage occupied by the directory in the file system. Say, i have the file system /home/arun/work under this file system i have the directories /home/arun/work/yesterday /home/arun/work/today /home/arun/work/tomorrow The size of the file system is... (5 Replies)
Discussion started by: Arunprasad
5 Replies

5. Shell Programming and Scripting

script to differenciate directory and file system

I need to write a script which will check all the directories in / and find out which is separate filesystem or a directory inside / suppose i have /application -->/dev/hdisk0 is a partitiom /abc --> is a directory /xyz-->is a directory /db-->/dev/hdisk1 is a partition actually my / gets... (2 Replies)
Discussion started by: pchangba
2 Replies

6. AIX

how to handle potential file contention

I need to change how a posting procedure currently works in order to improve load balancing but I am hitting a potential file contention problem that I was wondering if someone here could assist me with... In a directory called FilePool I would have a bunch of files that are constantly coming in... (3 Replies)
Discussion started by: philplasma
3 Replies

7. Cybersecurity

Attacking Potential of sh-scripts

Hey, I actually do have a question which seems rather easy for those you know more about this topic, since I am pretty new to bashscripting and don't know where it's limits are I have to ask you guys :) Imagine a system where all possible code execution methods (binary executables or... (15 Replies)
Discussion started by: disaster
15 Replies

8. Shell Programming and Scripting

Contention Identifier Script

Hi Experts, Need some help with a script which is definetly beyond my scripting skills. Here is flat file that I have with 4 Key Columns KEYCOLUMN1 KEYCOLUMN2 KEYCOLUMN3 KEYCOLUMN4 123ABC AEG MANCHESTER BIGBOX... (2 Replies)
Discussion started by: PG3
2 Replies
CPIO(1) 						    BSD General Commands Manual 						   CPIO(1)

NAME
cpio -- copy file archives in and out SYNOPSIS
cpio -o [-AaBcLvZz] [-C bytes] [-F archive] [-H format] [-O archive] < name-list [> archive] cpio -i [-6BbcdfmrSstuvZz] [-C bytes] [-E file] [-F archive] [-H format] [-I archive] [pattern ...] [< archive] cpio -p [-adLlmuv] destination-directory < name-list DESCRIPTION
The cpio command copies files to and from a cpio archive. If the archive is of the form: [[user@]host:]file then the archive will be pro- cessed using rmt(8). The options are as follows: -o, --create Create an archive. Reads the list of files to store in the archive from standard input, and writes the archive on standard output. -a, --reset-access-time Reset the access times on files that have been copied to the archive. -A, --append Append to the specified archive. -B Set block size of output to 5120 bytes. -c Use ASCII format for cpio header for portability. -C bytes Set the block size of output to bytes. -F archive -O archive Use the specified file name as the archive to write to. -H format Write the archive in the specified format. Recognized formats are: bcpio Old binary cpio format. cpio Old octal character cpio format. sv4cpio SVR4 hex cpio format. tar Old tar format. ustar POSIX ustar format. -L Follow symbolic links. -v Be verbose about operations. List filenames as they are written to the archive. --xz Compress/decompress archive using xz(1) format. -Z Compress archive using compress(1) format. -z Compress/decompress archive using gzip(1) format. -i, --extract Restore files from an archive. Reads the archive file from standard input and extracts files matching the patterns that were speci- fied on the command line. -b Do byte and word swapping after reading in data from the archive, for restoring archives created on systems with a different byte order. -B Set the block size of the archive being read to 5120 bytes. -c Expect the archive headers to be in ASCII format. -C bytes Read archive written with a block size of bytes. -d, --make-directories Create any intermediate directories as needed during restore. -E file, --pattern-file file Read list of file name patterns to extract or list from file. -f, --nonmatching Restore all files except those matching the patterns given on the command line. -F archive, --file archive -I archive Use the specified file as the input for the archive. -H format, --format format Read an archive of the specified format. Recognized formats are: bcpio Old binary cpio format. cpio Old octal character cpio format. sv4cpio SVR4 hex cpio format. tar Old tar format. ustar POSIX ustar format. -m Restore modification times on files. -r, --rename Rename restored files interactively. -s Swap bytes after reading data from the archive. -S, --swap-halfwords Swap words after reading data from the archive. -t, --list Only list the contents of the archive, no files or directories will be created. -u, --unconditional Overwrite files even when the file in the archive is older than the one that will be overwritten. -v, --verbose Be verbose about operations. List filenames as they are copied in from the archive. -z Uncompress archive using gzip(1) format. -Z Uncompress archive using compress(1) format. -6 Process old-style cpio format archives. -p, --pass-through Copy files from one location to another in a single pass. The list of files to copy are read from standard input and written out to a directory relative to the specified directory argument. -a Reset the access times on files that have been copied. -d Create any intermediate directories as needed to write the files at the new location. -l, --link When possible, link files rather than creating an extra copy. -L, --dereference Follow symbolic links. -m, --preserve-modification-time Restore modification times on files. -u, --unconditional Overwrite files even when the original file being copied is older than the one that will be overwritten. -v, --verbose Be verbose about operations. List filenames as they are copied. --force-local Do not interpret filenames that contain a ':' as remote files. --insecure Normally cpio ignores filenames that contain ``..'' as a path component. With this option, files that contain ``..'' can be processed. EXIT STATUS
cpio will exit with one of the following values: 0 All files were processed successfully. 1 An error occurred. Whenever cpio cannot create a file or a link when extracting an archive or cannot find a file while writing an archive, or cannot preserve the user ID, group ID, file mode, or access and modification times when the -p option is specified, a diagnostic message is written to stan- dard error and a non-zero exit value will be returned, but processing will continue. In the case where cpio cannot create a link to a file, cpio will not create a second copy of the file. If the extraction of a file from an archive is prematurely terminated by a signal or error, cpio may have only partially extracted the file the user wanted. Additionally, the file modes of extracted files and directories may have incorrect file bits, and the modification and access times may be wrong. If the creation of an archive is prematurely terminated by a signal or error, cpio may have only partially created the archive which may vio- late the specific archive format specification. SEE ALSO
pax(1), tar(1) AUTHORS
Keith Muller at the University of California, San Diego. BUGS
The -s and -S options are currently not implemented. BSD
June 18, 2011 BSD
All times are GMT -4. The time now is 06:53 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy