GNU Parallel makes it often easy to generate command lines:
It will also run the jobs in parallel. If you jobs are multithreaded and you see 100% utilization of your cores all the time then you will not see a speed up using GNU Parallel.
If, however, your jobs have I/O and therefore sometimes wait for data, then it might be faster to have more processes running, that can utilize the CPU when other processes are waiting.
GNU Parallel can be made to look at the load average and not start more jobs if the load average is above a certain limit. Default (-j+0) is to start 1 job per CPU core regardless of load average.
If you want multiple instances of GNU Parallel to communicate, and not have more than N jobs in total - no matter how many times you start parallel, you should look at 'sem': http://www.gnu.org/software/parallel/sem.html
i am writing a script to perform some mysqldumps and gzip them. The problem I am running into is that if the user specifies a database that doesn't exist, the error the mysql engine produces is still piped into gzip, and the exit code returned is 0. If I don't pipe into gzip, an exit code... (4 Replies)
Can anyone interpret and tell me the way the below command works?
find * -name "*${msgType}" -mtime +${archiveDays} -prune -type f -print 2>/dev/null | xargs rm -f 2> /dev/null
Please tell me the usage of prune and xargs in the above command?
Looking forward your reply.
Thanks in... (1 Reply)
Hi
From the xargs man page (Solaris):
ls $1 | xargs -I {} -t mv $1/{} $2/{}
This would move all the files in directory $1 to directory $2
Problem 1:
In a shell script if I want to move files in d1 to d2
dir1=~/d1
dir2=~/d2
ls $dir1 | xargs -I {} -t mv $dir1/{} $dir2/{}
does not... (3 Replies)
I'm trying to get a count of all the files in a series of directories on a per directory basis. Directory structure is like (but with many more files):
/dir1/subdir1/file1.txt
/dir1/subdir1/file2.txt
/dir1/subdir2/file1.txt
/dir1/subdir2/file2.txt
/dir2/subdir1/file1.txt... (4 Replies)
Hi
I am trying to use "xargs" command to loop through each file, modify it and overwrite the old file with the modification but with the same file name.
I thought it is easy but I just can't get it to work
I tried the following
I thought {} would give me the current file name, but it... (1 Reply)
Hey,
I want to create a new file (devices) with the 39th and the 40th character of the line wich is in the array line and in the file drivers.
But unfortunately my try doesn't work:
sed -n '$linep' drivers | cut -c 39-40 | echo >>devices Perhaps one of you can help me. Thank you!
emoly
... (0 Replies)
Hey guys. I'm very new to Unix. I'm pretty fluent in Java and C, but I have never actually used Unix for anything. I am in an Operating Systems course now and I have an assignment to write a piece of code that involves forks and piping. I'm stuck.
1. The problem statement, all variables and... (6 Replies)
I entered the following <zzz.list xargs showtell |more which does a echo "<<<<<<<<<<<<<<<<<<<<<<<<< $1 >>>>>>>>>>>>>>>>>>"
head -20 $1 The file zzz.list contains 525 lines representing user scripts (1 per line), but only the first, 181st, and 399th lines were processed. What am I missing?
TIA (2 Replies)
Need help in piping commands using xargs
I have several .tar.gz files that I need to list the folder content in a subdirectory.
For example,
a.tar.gz
b.tar.gz
c.tar.gz
The following command works great for each .tar.gz file but it's a pain to run the tar command for each file.
tar -tf... (11 Replies)
Discussion started by: april
11 Replies
LEARN ABOUT DEBIAN
queuedefs
queuedefs(4) File Formats queuedefs(4)NAME
queuedefs - queue description file for at, batch, and cron
SYNOPSIS
/etc/cron.d/queuedefs
DESCRIPTION
The queuedefs file describes the characteristics of the queues managed by cron(1M). Each non-comment line in this file describes one queue.
The format of the lines are as follows:
q.[njobj][nicen][nwaitw]
The fields in this line are:
q The name of the queue. a is the default queue for jobs started by at(1); b is the default queue for jobs started by batch (see
at(1)); c is the default queue for jobs run from a crontab(1) file.
njob The maximum number of jobs that can be run simultaneously in that queue; if more than njob jobs are ready to run, only the first
njob jobs will be run, and the others will be run as jobs that are currently running terminate. The default value is 100.
nice The nice(1) value to give to all jobs in that queue that are not run with a user ID of super-user. The default value is 2.
nwait The number of seconds to wait before rescheduling a job that was deferred because more than njob jobs were running in that job's
queue, or because the system-wide limit of jobs executing has been reached. The default value is 60.
Lines beginning with # are comments, and are ignored.
EXAMPLES
Example 1: A sample file.
#
#
a.4j1n
b.2j2n90w
This file specifies that the a queue, for at jobs, can have up to 4 jobs running simultaneously; those jobs will be run with a nice value
of 1. As no nwait value was given, if a job cannot be run because too many other jobs are running cron will wait 60 seconds before trying
again to run it.
The b queue, for batch(1) jobs, can have up to 2 jobs running simultaneously; those jobs will be run with a nice(1) value of 2. If a job
cannot be run because too many other jobs are running, cron(1M) will wait 90 seconds before trying again to run it. All other queues can
have up to 100 jobs running simultaneously; they will be run with a nice value of 2, and if a job cannot be run because too many other jobs
are running cron will wait 60 seconds before trying again to run it.
FILES
/etc/cron.d/queuedefs queue description file for at, batch, and cron.
SEE ALSO at(1), crontab(1), nice(1), cron(1M)SunOS 5.10 1 Mar 1994 queuedefs(4)