GNU Parallel makes it often easy to generate command lines:
It will also run the jobs in parallel. If you jobs are multithreaded and you see 100% utilization of your cores all the time then you will not see a speed up using GNU Parallel.
If, however, your jobs have I/O and therefore sometimes wait for data, then it might be faster to have more processes running, that can utilize the CPU when other processes are waiting.
GNU Parallel can be made to look at the load average and not start more jobs if the load average is above a certain limit. Default (-j+0) is to start 1 job per CPU core regardless of load average.
If you want multiple instances of GNU Parallel to communicate, and not have more than N jobs in total - no matter how many times you start parallel, you should look at 'sem': http://www.gnu.org/software/parallel/sem.html
i am writing a script to perform some mysqldumps and gzip them. The problem I am running into is that if the user specifies a database that doesn't exist, the error the mysql engine produces is still piped into gzip, and the exit code returned is 0. If I don't pipe into gzip, an exit code... (4 Replies)
Can anyone interpret and tell me the way the below command works?
find * -name "*${msgType}" -mtime +${archiveDays} -prune -type f -print 2>/dev/null | xargs rm -f 2> /dev/null
Please tell me the usage of prune and xargs in the above command?
Looking forward your reply.
Thanks in... (1 Reply)
Hi
From the xargs man page (Solaris):
ls $1 | xargs -I {} -t mv $1/{} $2/{}
This would move all the files in directory $1 to directory $2
Problem 1:
In a shell script if I want to move files in d1 to d2
dir1=~/d1
dir2=~/d2
ls $dir1 | xargs -I {} -t mv $dir1/{} $dir2/{}
does not... (3 Replies)
I'm trying to get a count of all the files in a series of directories on a per directory basis. Directory structure is like (but with many more files):
/dir1/subdir1/file1.txt
/dir1/subdir1/file2.txt
/dir1/subdir2/file1.txt
/dir1/subdir2/file2.txt
/dir2/subdir1/file1.txt... (4 Replies)
Hi
I am trying to use "xargs" command to loop through each file, modify it and overwrite the old file with the modification but with the same file name.
I thought it is easy but I just can't get it to work
I tried the following
I thought {} would give me the current file name, but it... (1 Reply)
Hey,
I want to create a new file (devices) with the 39th and the 40th character of the line wich is in the array line and in the file drivers.
But unfortunately my try doesn't work:
sed -n '$linep' drivers | cut -c 39-40 | echo >>devices Perhaps one of you can help me. Thank you!
emoly
... (0 Replies)
Hey guys. I'm very new to Unix. I'm pretty fluent in Java and C, but I have never actually used Unix for anything. I am in an Operating Systems course now and I have an assignment to write a piece of code that involves forks and piping. I'm stuck.
1. The problem statement, all variables and... (6 Replies)
I entered the following <zzz.list xargs showtell |more which does a echo "<<<<<<<<<<<<<<<<<<<<<<<<< $1 >>>>>>>>>>>>>>>>>>"
head -20 $1 The file zzz.list contains 525 lines representing user scripts (1 per line), but only the first, 181st, and 399th lines were processed. What am I missing?
TIA (2 Replies)
Need help in piping commands using xargs
I have several .tar.gz files that I need to list the folder content in a subdirectory.
For example,
a.tar.gz
b.tar.gz
c.tar.gz
The following command works great for each .tar.gz file but it's a pain to run the tar command for each file.
tar -tf... (11 Replies)
Discussion started by: april
11 Replies
LEARN ABOUT REDHAT
xargs
XARGS(1L)XARGS(1L)NAME
xargs - build and execute command lines from standard input
SYNOPSIS
xargs [-0prtx] [-e[eof-str]] [-i[replace-str]] [-l[max-lines]] [-n max-args] [-s max-chars] [-P max-procs] [--null] [--eof[=eof-str]]
[--replace[=replace-str]] [--max-lines[=max-lines]] [--interactive] [--max-chars=max-chars] [--verbose] [--exit] [--max-procs=max-procs]
[--max-args=max-args] [--no-run-if-empty] [--version] [--help] [command [initial-arguments]]
DESCRIPTION
This manual page documents the GNU version of xargs. xargs reads arguments from the standard input, delimited by blanks (which can be pro-
tected with double or single quotes or a backslash) or newlines, and executes the command (default is /bin/echo) one or more times with any
initial-arguments followed by arguments read from standard input. Blank lines on the standard input are ignored.
xargs exits with the following status:
0 if it succeeds
123 if any invocation of the command exited with status 1-125
124 if the command exited with status 255
125 if the command is killed by a signal
126 if the command cannot be run
127 if the command is not found
1 if some other error occurred.
OPTIONS
--null, -0
Input filenames are terminated by a null character instead of by whitespace, and the quotes and backslash are not special (every
character is taken literally). Disables the end of file string, which is treated like any other argument. Useful when arguments
might contain white space, quote marks, or backslashes. The GNU find -print0 option produces input suitable for this mode.
--eof[=eof-str], -e[eof-str]
Set the end of file string to eof-str. If the end of file string occurs as a line of input, the rest of the input is ignored. If
eof-str is omitted, there is no end of file string. If this option is not given, the end of file string defaults to "_".
--help Print a summary of the options to xargs and exit.
--replace[=replace-str], -i[replace-str]
Replace occurences of replace-str in the initial arguments with names read from standard input. Also, unquoted blanks do not termi-
nate arguments. If replace-str is omitted, it defaults to "{}" (like for `find -exec'). Implies -x and -l 1.
--max-lines[=max-lines], -l[max-lines]
Use at most max-lines nonblank input lines per command line; max-lines defaults to 1 if omitted. Trailing blanks cause an input
line to be logically continued on the next input line. Implies -x.
--max-args=max-args, -n max-args
Use at most max-args arguments per command line. Fewer than max-args arguments will be used if the size (see the -s option) is
exceeded, unless the -x option is given, in which case xargs will exit.
--interactive, -p
Prompt the user about whether to run each command line and read a line from the terminal. Only run the command line if the response
starts with `y' or `Y'. Implies -t.
--no-run-if-empty, -r
If the standard input does not contain any nonblanks, do not run the command. Normally, the command is run once even if there is no
input.
--max-chars=max-chars, -s max-chars
Use at most max-chars characters per command line, including the command and initial arguments and the terminating nulls at the ends
of the argument strings. The default is as large as possible, up to 20k characters.
--verbose, -t
Print the command line on the standard error output before executing it.
--version
Print the version number of xargs and exit.
--exit, -x
Exit if the size (see the -s option) is exceeded.
--max-procs=max-procs, -P max-procs
Run up to max-procs processes at a time; the default is 1. If max-procs is 0, xargs will run as many processes as possible at a
time. Use the -n option with -P; otherwise chances are that only one exec will be done.
SEE ALSO find(1L), locate(1L), locatedb(5L), updatedb(1) Finding Files (on-line in Info, or printed)
XARGS(1L)