Sponsored Content
Top Forums Shell Programming and Scripting script to check large directory--help Post 302513925 by DGPickett on Thursday 14th of April 2011 10:43:04 AM
Old 04-14-2011
You processing might be simpler if you interrogate each file system, one at at time: "for fs in /AB/* . . . .", as then you can grep -q for % you dislike. For finding biggies, I like a mixed report of big dir/ and big file. You can get du to do most of it, and in ksh on /dev/fd/# systems or in bash, this runs in pipeline parallel, with about a 100Kb cutoff between sed pattern and file options. You need a few more options if you want to keep find and du on one filesystem, but you can read the man pages, too:
Code:
/usr/bin/bash
 
sort -nrm <(
  du -k $1 | sed '
    /^[0-9]\{3\}/d
    s/$/\//
   ' | sort -nr
 ) <(
  find $1 -type f -size +102400c |xargs -n999 du -k | sort -nr
 )

 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

check the file in multiple Directory in a script

hi, i want to write the script to list atleast one file inside that directory eg: /home/Log/amp01 /home/log/amp02 . . . /home/log/amp..N want to see atleast one file inside the /home/log/amp01 .... amp(N) if it not there.. need to give that no file exists inside... (3 Replies)
Discussion started by: mail2sant
3 Replies

2. Shell Programming and Scripting

script to check for a directory in /home for all users

Following on from this post: https://www.unix.com/shell-programming-scripting/150201-simple-script-mount-folder-all-users-home.html and getting told off for bumping the thread:( Please could someone help me with a short script to check is a certain directory is present in /home for all users... (8 Replies)
Discussion started by: barrydocks
8 Replies

3. Shell Programming and Scripting

Using find in a directory containing large number of files

Hi All, I have searched this forum for related posts but could not find one that fits mine. I have a shell script which removes all the XML tags including the text inside the tags from some 4 million XML files. The shell script looks like this (MODIFIED): find . "*.xml" -print | while read... (6 Replies)
Discussion started by: shoaibjameel123
6 Replies

4. Red Hat

Empty directory, large size and performance

Hi, I've some directory that I used as working directory for a program. At the end of the procedure, the content is deleted. This directory, when I do a ls -l, appears to still take up some space. After a little research, I've seen on a another board of this forum that it's not really taking... (5 Replies)
Discussion started by: bdx
5 Replies

5. UNIX for Dummies Questions & Answers

Is it better/possible to pause the rsyncing of a very large directory?

Possibly a dumb question, but I'm deciding how I'm going to do this. I'm currently rsyncing a 25TB directory (with several layers of sub directories most of which have video files ranging from 500 megs to 4-5 gigs), from one NAS to another using rsync -av. By the time I need to act ~15TB should... (3 Replies)
Discussion started by: DeCoTwc
3 Replies

6. Shell Programming and Scripting

How to copy very large directory trees

I have constant trouble with XCOPY/s for multi-gigabyte transfers. I need a utility like XCOPY/S that remembers where it left off if I reboot. Is there such a utility? How about a free utility (free as in free beer)? How about an md5sum sanity check too? I posted the above query in another... (3 Replies)
Discussion started by: siegfried
3 Replies

7. Shell Programming and Scripting

How to check whether directory has files in it or not in shell script?

hi, I am having script in which i want to check if directory has any file in it or not. If directory contains a single or more files then and only then it should proceed to further operations... P.S.: Directory might have thousand number of files. so there might be chance of getting error... (4 Replies)
Discussion started by: VSom007
4 Replies

8. UNIX for Dummies Questions & Answers

Shell script to check the file sitting in the directory more than 10 hours

Hi, I require shell script to check for any pending files which are sitting in the particular directory for more than 10 hours. Please help me on this...Thank you. (5 Replies)
Discussion started by: kiruthiish
5 Replies

9. Shell Programming and Scripting

Script to check the files existence inside a directory.

Hello Folks, On Below Script, I want to apply condition. Condition that it check the directory for files if not found keep checking. Once directory have files and founded do the calculation and finish the code. Script to check the files existence inside a directory, If not then keep... (16 Replies)
Discussion started by: sadique.manzar
16 Replies
XARGS(1L)																 XARGS(1L)

NAME
xargs - build and execute command lines from standard input SYNOPSIS
xargs [-0prtx] [-e[eof-str]] [-i[replace-str]] [-l[max-lines]] [-n max-args] [-s max-chars] [-P max-procs] [--null] [--eof[=eof-str]] [--replace[=replace-str]] [--max-lines[=max-lines]] [--interactive] [--max-chars=max-chars] [--verbose] [--exit] [--max-procs=max-procs] [--max-args=max-args] [--no-run-if-empty] [--version] [--help] [command [initial-arguments]] DESCRIPTION
This manual page documents the GNU version of xargs. xargs reads arguments from the standard input, delimited by blanks (which can be pro- tected with double or single quotes or a backslash) or newlines, and executes the command (default is /bin/echo) one or more times with any initial-arguments followed by arguments read from standard input. Blank lines on the standard input are ignored. xargs exits with the following status: 0 if it succeeds 123 if any invocation of the command exited with status 1-125 124 if the command exited with status 255 125 if the command is killed by a signal 126 if the command cannot be run 127 if the command is not found 1 if some other error occurred. OPTIONS --null, -0 Input filenames are terminated by a null character instead of by whitespace, and the quotes and backslash are not special (every character is taken literally). Disables the end of file string, which is treated like any other argument. Useful when arguments might contain white space, quote marks, or backslashes. The GNU find -print0 option produces input suitable for this mode. --eof[=eof-str], -e[eof-str] Set the end of file string to eof-str. If the end of file string occurs as a line of input, the rest of the input is ignored. If eof-str is omitted, there is no end of file string. If this option is not given, the end of file string defaults to "_". --help Print a summary of the options to xargs and exit. --replace[=replace-str], -i[replace-str] Replace occurences of replace-str in the initial arguments with names read from standard input. Also, unquoted blanks do not termi- nate arguments. If replace-str is omitted, it defaults to "{}" (like for `find -exec'). Implies -x and -l 1. --max-lines[=max-lines], -l[max-lines] Use at most max-lines nonblank input lines per command line; max-lines defaults to 1 if omitted. Trailing blanks cause an input line to be logically continued on the next input line. Implies -x. --max-args=max-args, -n max-args Use at most max-args arguments per command line. Fewer than max-args arguments will be used if the size (see the -s option) is exceeded, unless the -x option is given, in which case xargs will exit. --interactive, -p Prompt the user about whether to run each command line and read a line from the terminal. Only run the command line if the response starts with `y' or `Y'. Implies -t. --no-run-if-empty, -r If the standard input does not contain any nonblanks, do not run the command. Normally, the command is run once even if there is no input. --max-chars=max-chars, -s max-chars Use at most max-chars characters per command line, including the command and initial arguments and the terminating nulls at the ends of the argument strings. The default is as large as possible, up to 20k characters. --verbose, -t Print the command line on the standard error output before executing it. --version Print the version number of xargs and exit. --exit, -x Exit if the size (see the -s option) is exceeded. --max-procs=max-procs, -P max-procs Run up to max-procs processes at a time; the default is 1. If max-procs is 0, xargs will run as many processes as possible at a time. Use the -n option with -P; otherwise chances are that only one exec will be done. SEE ALSO
find(1L), locate(1L), locatedb(5L), updatedb(1) Finding Files (on-line in Info, or printed) XARGS(1L)
All times are GMT -4. The time now is 09:20 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy