Sponsored Content
Full Discussion: Help in using xargs
Top Forums Shell Programming and Scripting Help in using xargs Post 302396248 by senthil3d on Thursday 18th of February 2010 04:35:43 AM
Old 02-18-2010
Hi,
I'm able to restrict the files one at a time using FOR loop. But still scp command is run serially for each files under $workset due to using FOR loop. I need to process scp in parallel for each workset and this parallelism should be taken dynamically based on the $setsize.

For example:

for workset 1
Code:
scp -rp ussfqaorc001:/home/oracle/senthil/a .
scp -rp ussfqaorc001:/home/oracle/senthil/b .

for workset 2
Code:
scp -rp ussfqaorc001:/home/oracle/senthil/c .
scp -rp ussfqaorc001:/home/oracle/senthil/d .

Code:
 
[oracle@usegainsqaorc001 senthil]$ ./pull_data.ksh
++ SOURCESERVER=ss01
++ SOURCEDIR=/home/oracle/senthil
++ TARGETDIR=/home/oracle/senthil
++ MAX_PARALLEL=2
+++ ssh ss01 'ls /home/oracle/senthil|wc -w'
++ nroffiles=4
++ echo 4
4
++ setsize=2
++ echo 2
2
++ ssh -n ss01 ls -1 /home/oracle/senthil/
++ xargs -n 2
++ read workset
++ for i in '$workset'
++ scp -rp ussfqaorc001:/home/oracle/senthil/a .
++ for i in '$workset'
++ scp -rp ussfqaorc001:/home/oracle/senthil/b .
++ read workset
++ for i in '$workset'
++ scp -rp ussfqaorc001:/home/oracle/senthil/c .
++ for i in '$workset'
++ scp -rp ussfqaorc001:/home/oracle/senthil/d .
++ read workset


Last edited by Scott; 02-18-2010 at 05:49 AM.. Reason: Code tags, PLEASE!
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Help with xargs

Hi there, I am trying to move around 3000 files from one directory to another. The mv command is complaining from too many arguments. I tried to use the xargs command but with no luck. Could some body provide help? Regards (4 Replies)
Discussion started by: JimJim
4 Replies

2. Shell Programming and Scripting

why we use xargs..

hi , can anyone help me by saying why we use xargs.. is it acing like a place holder..? thanks, Krips. (3 Replies)
Discussion started by: kripssmart
3 Replies

3. UNIX for Advanced & Expert Users

xargs -P

I discovered that GNU's xargs has a -P option to allow its processes to run in parallel. Great! Is this a GNU thing, or is it supported by other platforms as well? (4 Replies)
Discussion started by: otheus
4 Replies

4. Shell Programming and Scripting

Using xargs

hi i just want to know that how do we use xargs command to find files which are greater than specified memory in a given directory (6 Replies)
Discussion started by: sumit the cool
6 Replies

5. Shell Programming and Scripting

Xargs and

Hello there, Let me show you a simple example of what I am trying to achieve: 1) I have an input text file with some lines: 1 a 2 b 3 c 2) And I want to run a command with these lines as arguments (+ arbitrary extra arguments). For example: $ command "1 a" "2 b" "3 c" "bye" I... (7 Replies)
Discussion started by: tokland
7 Replies

6. Shell Programming and Scripting

Help with xargs

hi Could any one please tell me the option using which we can run multiple commands using xargs I have list of files, I want to run dos2unix and chmod at one shot on them I tried google n searched man pages but couldnt really find the solution , please help right now im doing this ls... (4 Replies)
Discussion started by: sunilmenhdiratt
4 Replies

7. Shell Programming and Scripting

xargs

Dear all , any suggest on xargs to combine from (1.txt and 2.txt) to output.txt ? thanks a lot. 1.txt 0123 BUM-5M BUM-5M 93490481 63839 0124 BUM-5M BUM-5M 112112 ... (3 Replies)
Discussion started by: samoptimus
3 Replies

8. Shell Programming and Scripting

Help with xargs

Using the bash shell I'm trying to either create a command for the command line or a script that will show netstat info for a given process name. Here is an example of what I'm trying to do:$ ps aux |grep catalina |grep -v grep | awk '{print $2}' 5132 $ netstat -nlp |grep 5132 (Not all processes... (11 Replies)
Discussion started by: axiopisty
11 Replies

9. Shell Programming and Scripting

Xargs

Hello, I need some help with xargs $ ls aaa bbb ccc ddd$ ls | xargs -I{} ls -la {} -rw-rw-r--. 1 xxx xx 0 May 30 20:04 aaa -rw-rw-r--. 1 xxx xx 0 May 30 20:04 bbb -rw-rw-r--. 1 xxx xx 0 May 30 20:04 ccc -rw-rw-r--. 1 xxx xx 0 May 30 20:04 dddit's possible to have output like this with... (3 Replies)
Discussion started by: vikus
3 Replies

10. UNIX for Dummies Questions & Answers

Xargs

Hi, can anyone tell me in detail ? what the following do in detail ? I am trying to get a largest number in a list Thanks Tao LARGEST=$(echo $* | xargs -n1 | sort -nr | tail -1) (3 Replies)
Discussion started by: ccp
3 Replies
BBHOSTGREP(1)						      General Commands Manual						     BBHOSTGREP(1)

NAME
bbhostgrep - pick out lines in bb-hosts SYNOPSIS
bbhostgrep --help bbhostgrep --version bbhostgrep [--noextras] [--test-untagged] [--bbdisp] [--bbnet] TAG [TAG...] DESCRIPTION
bbhostgrep(1) is for use by extension scripts that need to pick out the entries in a bb-hosts file that are relevant to the script. The utility accepts test names as parameters, and will then parse the bb-hosts file and print out the host entries that have at least one of the wanted tests specified. Tags may be given with a trailing asterisk '*', e.g. "bbhostgrep http*" is needed to find all http and https tags. The bbhostgrep utility supports the use of "include" directives inside the bb-hosts file, and will find matching tags in all included files. If the DOWNTIME or SLA tags are used in the bb-hosts(5) file, these are interpreted relative to the current time. bbhostgrep then outputs a "INSIDESLA" or "OUTSIDESLA" tag for easier use by scripts that want to check if the current time is inside or outside the expected uptime window. OPTIONS
--noextras Remove the "testip", "dialup", "INSIDESLA" and "OUTSIDESLA" tags from the output. --test-untagged When using the BBLOCATION environment variable to test only hosts on a particular network segment, bbtest-net will ignore hosts that do not have any "NET:x" tag. So only hosts that have a NET:$BBLOCATION tag will be tested. With this option, hosts with no NET: tag are included in the test, so that all hosts that either have a matching NET: tag, or no NET: tag at all are tested. --no-down[=TESTNAME] bbhostgrep will query the Xymon server for the current status of the "conn" test, and if TESTNAME is specified also for the current state of the specified test. If the status of the "conn" test for a host is non-green, or the status of the TESTNAME test is dis- abled, then this host is ignored and will not be included in the output. This can be used to ignore hosts that are down, or hosts where the custom test is disabled. --bbdisp Search the bb-hosts file following include statements as a BBDISPLAY server would. --bbnet Search the bb-hosts file following include statements as a BBNET server would. EXAMPLE
If your bb-hosts file looks like this 192.168.1.1 www.test.com # ftp telnet !oracle 192.168.1.2 db1.test.com # oracle 192.168.1.3 mail.test.com # smtp and you have a custom Xymon extension script that performs the "oracle" test, then running "bbhostgrep oracle" would yield 192.168.1.1 www.test.com # !oracle 192.168.1.2 db1.test.com # oracle so the script can quickly find the hosts that are of interest. Note that the reverse-test modifier - "!oracle" - is included in the output; this also applies to the other test modifiers defined by Xymon (the dialup and always-true modifiers). If your extension scripts use more than one tag, just list all of the interesting tags on the command line. bbhostgrep also supports the "NET:location" tag used by bbtest-net, so if your script performs network checks then it will see only the hosts that are relevant for the test location that the script currently executes on. USE IN EXTENSION SCRIPTS
To integrate bbhostgrep into an existing script, look for the line in the script that grep's in the $BBHOSTS file. Typically it will look somewhat like this: $GREP -i "^[0-9].*#.*TESTNAME" $BBHOSTS | ... code to handle test Instead of the grep, we will use bbhostgrep. It then becomes $BBHOME/bin/bbhostgrep TESTNAME | ... code to handle test which is simpler, less error-prone and more efficient. ENVIRONMENT VARIABLES
BBLOCATION If set, bbhostgrep outputs only lines from bb-hosts that have a matching NET:$BBLOCATION setting. BBHOSTS Filename for the Xymon bb-hosts(5) file. FILES
$BBHOSTS The Xymon bb-hosts file SEE ALSO
bb-hosts(5), hobbitserver.cfg(5) Xymon Version 4.2.3: 4 Feb 2009 BBHOSTGREP(1)
All times are GMT -4. The time now is 05:37 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy