Sponsored Content
Full Discussion: Help in using xargs
Top Forums Shell Programming and Scripting Help in using xargs Post 302395813 by senthil3d on Wednesday 17th of February 2010 04:42:49 AM
Old 02-17-2010
Help in using xargs

Hi,
I have a requirement to RCP the files from remote server to local server.
Also the RCP has to run in parallel. However using 'xargs' retrives 2 file names during each loop. How do we restrict to only one file name using xargs and loop till remaining files.


I use the below code for that..

Code:
set -x
SOURCESERVER=ss01
SOURCEDIR="/home/oracle/senthil"
TARGETDIR="/home/oracle/senthil"
MAX_PARALLEL=2
nroffiles=`ssh $SOURCESERVER "ls $SOURCEDIR|wc -w"`
echo $nroffiles
setsize=$(( nroffiles/MAX_PARALLEL ))
echo $setsize
ssh -n $SOURCESERVER ls -1 $SOURCEDIR/ | xargs -n $setsize | while read workset; do
ssh -n $SOURCESERVER scp -rp $SOURCESERVER:$SOURCEDIR/$workset . &
done

Here's the runtime log. As you can see there are two files passed -(a&b
c&d) during each loop processing. How can we restrict this to one file at a time.

Code:
 
[oracle@ss02 senthil]$ ./pull_data.ksh
++ SOURCESERVER=ss01
++ SOURCEDIR=/home/oracle/senthil
++ TARGETDIR=/home/oracle/senthil
++ MAX_PARALLEL=2
+++ ssh ss01 'ls /home/oracle/senthil|wc -w'
++ nroffiles=5
++ echo 5
5
++ setsize=2
++ echo 2
2
++ ssh -n ss01 ls -1 /home/oracle/senthil/
++ xargs -n 2
++ read workset
++ ssh -n ss01 scp -rp ss01:/home/oracle/senthil/a b .
++ read workset
++ ssh -n ss01 scp -rp ss01:/home/oracle/senthil/c d .
++ read workset


Last edited by zaxxon; 02-17-2010 at 05:51 AM.. Reason: use code tags please, ty
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Help with xargs

Hi there, I am trying to move around 3000 files from one directory to another. The mv command is complaining from too many arguments. I tried to use the xargs command but with no luck. Could some body provide help? Regards (4 Replies)
Discussion started by: JimJim
4 Replies

2. Shell Programming and Scripting

why we use xargs..

hi , can anyone help me by saying why we use xargs.. is it acing like a place holder..? thanks, Krips. (3 Replies)
Discussion started by: kripssmart
3 Replies

3. UNIX for Advanced & Expert Users

xargs -P

I discovered that GNU's xargs has a -P option to allow its processes to run in parallel. Great! Is this a GNU thing, or is it supported by other platforms as well? (4 Replies)
Discussion started by: otheus
4 Replies

4. Shell Programming and Scripting

Using xargs

hi i just want to know that how do we use xargs command to find files which are greater than specified memory in a given directory (6 Replies)
Discussion started by: sumit the cool
6 Replies

5. Shell Programming and Scripting

Xargs and

Hello there, Let me show you a simple example of what I am trying to achieve: 1) I have an input text file with some lines: 1 a 2 b 3 c 2) And I want to run a command with these lines as arguments (+ arbitrary extra arguments). For example: $ command "1 a" "2 b" "3 c" "bye" I... (7 Replies)
Discussion started by: tokland
7 Replies

6. Shell Programming and Scripting

Help with xargs

hi Could any one please tell me the option using which we can run multiple commands using xargs I have list of files, I want to run dos2unix and chmod at one shot on them I tried google n searched man pages but couldnt really find the solution , please help right now im doing this ls... (4 Replies)
Discussion started by: sunilmenhdiratt
4 Replies

7. Shell Programming and Scripting

xargs

Dear all , any suggest on xargs to combine from (1.txt and 2.txt) to output.txt ? thanks a lot. 1.txt 0123 BUM-5M BUM-5M 93490481 63839 0124 BUM-5M BUM-5M 112112 ... (3 Replies)
Discussion started by: samoptimus
3 Replies

8. Shell Programming and Scripting

Help with xargs

Using the bash shell I'm trying to either create a command for the command line or a script that will show netstat info for a given process name. Here is an example of what I'm trying to do:$ ps aux |grep catalina |grep -v grep | awk '{print $2}' 5132 $ netstat -nlp |grep 5132 (Not all processes... (11 Replies)
Discussion started by: axiopisty
11 Replies

9. Shell Programming and Scripting

Xargs

Hello, I need some help with xargs $ ls aaa bbb ccc ddd$ ls | xargs -I{} ls -la {} -rw-rw-r--. 1 xxx xx 0 May 30 20:04 aaa -rw-rw-r--. 1 xxx xx 0 May 30 20:04 bbb -rw-rw-r--. 1 xxx xx 0 May 30 20:04 ccc -rw-rw-r--. 1 xxx xx 0 May 30 20:04 dddit's possible to have output like this with... (3 Replies)
Discussion started by: vikus
3 Replies

10. UNIX for Dummies Questions & Answers

Xargs

Hi, can anyone tell me in detail ? what the following do in detail ? I am trying to get a largest number in a list Thanks Tao LARGEST=$(echo $* | xargs -n1 | sort -nr | tail -1) (3 Replies)
Discussion started by: ccp
3 Replies
scp(1)								   User Commands							    scp(1)

NAME
scp - secure copy (remote file copy program) SYNOPSIS
scp [-pqrvBC46] [-F ssh_config] [-S program] [-P port] [-c cipher] [-i identity_file] [-o ssh_option] [ [user@]host1:]file1 []... [ [user@]host2:]file2 DESCRIPTION
The scp utility copies files between hosts on a network. It uses ssh(1) for data transfer, and uses the same authentication and provides the same security as ssh(1). Unlike rcp(1), scp will ask for passwords or passphrases if they are needed for authentication. Any file name may contain a host and user specification to indicate that the file is to be copied to/from that host. Copies between two remote hosts are permitted. OPTIONS
The following options are supported: -4 Forces scp to use IPv4 addresses only. -6 Forces scp to use IPv6 addresses only. -B Selects batch mode. (Prevents asking for passwords or passphrases.) -c cipher Selects the cipher to use for encrypting the data transfer. This option is directly passed to ssh(1). -C Compression enable. Passes the -C flag to ssh(1) to enable compression. -F ssh_config Specifies an alternative per-user configuration file for ssh(1.). -i identity_file Selects the file from which the identity (private key) for RSA authentication is read. This option is directly passed to ssh(1). -o ssh_option The given option is directly passed to ssh(1). -p Preserves modification times, access times, and modes from the original file. -P port Specifies the port to connect to on the remote host. Notice that this option is written with a capital `P', because -p is already reserved for preserving the times and modes of the file in rcp(1). -q Disables the progress meter. -r Recursively copies entire directories. -S program Specifies the name of the program to use for the encrypted connection. The program must understand ssh(1) options. -v Verbose mode. Causes scp and ssh(1) to print debugging messages about their progress. This is helpful in debugging con- nection, authentication, and configuration problems. OPERANDS
The following operands are supported: host1, host2,... The name(s) of the host from or to which the file is to be copied. file1, file2,... The file(s) to be copied. EXIT STATUS
The following exit values are returned: 0 Successful completion. 1 An error occurred. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWsshu | +-----------------------------+-----------------------------+ |Interface Stability |Evolving | +-----------------------------+-----------------------------+ SEE ALSO
rcp(1), ssh(1), ssh-add(1), ssh-agent(1), ssh-keygen(1), sshd(1M), ssh_config(4), attributes(5) NOTES
Generally, use of scp with password or keyboard-interactive authentication method and two remote hosts does not work. It does work with either the pubkey, hostbased or gssapi-keyex authentication method. For the pubkey authentication method, either private keys not pro- tected by a passphrase, or an explicit ssh agent forwarding have to be used. The gssapi-keyex authentication method works with the ker- beros_v5 GSS-API mechanism, but only if the GSSAPIDelegateCredentials option is enabled. SunOS 5.11 22 Jun 2007 scp(1)
All times are GMT -4. The time now is 07:09 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy