Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Xargs and rsync not sending all files
Top Forums Shell Programming and Scripting Xargs and rsync not sending all files Post 303039517 by Chubler_XL on Tuesday 8th of October 2019 04:15:36 PM
Old 10-08-2019
It could be some sort of resource issue, or connection limit. I'd try less threads.

With disk and network I/O bottlenecks involved, I find it hard to believe that 40 threads would be quicker than 5, on my low spec system here 5 takes about twice as long as 1!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

MV files with xargs or -exec

Hi I need to move multiple (say 10 files) from one location to another location. My selection would be like this... ls -ltr *.arc | head ---> Need to move top 10 files with single command without iterating in loop. I know we can move files like this with find command but not sure if I can... (4 Replies)
Discussion started by: malaymaru
4 Replies

2. Shell Programming and Scripting

find with xargs to rm found files

I believe what is happening is rm is executing in the script on every directory and on failure of the first it stops although returns status 0. find $HOME -name /directory/filename | xargs -l rm This is the code I use but file remains. I am using sun solaris system which has way limited... (4 Replies)
Discussion started by: Ebodee
4 Replies

3. Shell Programming and Scripting

move files using xargs

Hi, I have many files like below in my currect dir test1.me test2.me test3.me I wanted them to rename without .me extention I tried below find . -name "*.me" | xargs -i mv {} `echo {} | sed 's/\.me//'` It thorws error that cant mv ./test1.me to ./test1.me as they are... (6 Replies)
Discussion started by: sunilmenhdiratt
6 Replies

4. Shell Programming and Scripting

help using find/xargs to apply mp3gain to files

I need to apply mp3gain (album mode) to all mp3 files in a given directory. Each album is in its own directory under /media/data/music/albums for example: /media/data/music/albums/foo /media/data/music/albums/bar /media/data/music/albums/more What needs to happen is: cd... (4 Replies)
Discussion started by: audiophile
4 Replies

5. Shell Programming and Scripting

Fast processing(mv command) of 1 million+ files using find, mv and xargs

Hi, I'd like to ask if anybody can help improve my code to move 1 million+ files from a directory to another: find /source/dir -name file* -type f | xargs -I '{}' mv {} /destination/dir I learned this line of code from this forum as well and it works fine. However, file movement is kinda... (6 Replies)
Discussion started by: agentgrecko
6 Replies

6. Shell Programming and Scripting

How to copy files from one location to another using xargs??

Hello Experts, I need to copy files from one location to another using xargs. Tried something like this (In Ubuntu & Solaris ). mkdir -p 1234; find /home/emd/Desktop/n007/M007/ -type f -name "A2014*" | xargs -0 cp -r {} /home/emd/Desktop/1234 But every time i run this, a weird error... (6 Replies)
Discussion started by: Saidul
6 Replies

7. Shell Programming and Scripting

How to delete directories and files with xargs?

Hello, i have an dynamical apache_cache that I need to clean all days (older tant one day) with an unix command : find /usr/IBM/HTTPServer/apache_cache/ -name '*' -mtime +1 -print0|xargs -0 rm -r -- but it didn't work. Could you explain me why. So I will put below all my script :... (13 Replies)
Discussion started by: steiner
13 Replies

8. Shell Programming and Scripting

Using rsync with xargs

I want to run parallelise rsync with xargs. The plan is to take separate directories and run rsync in parallel. find . -mindepth 1 -maxdepth 1 print0 | xargs -n1 -P0 rsync -avz Hoxever I am stuck on how to use rsync in this way ---------- Post updated at 04:44 AM ---------- Previous... (3 Replies)
Discussion started by: kristinu
3 Replies

9. Shell Programming and Scripting

Linux command rsync sending files incrementally

Please, i have a question about rsync command: Here is the command that i have used in my script: rsync -ratlz --rsh="/usr/bin/sshpass ssh -o StrictHostKeyChecking=no" -aAXHltzh --progress --numeric-ids --devices --rsync-path="rsync" $1 $hostname:$1 using this command, i can... (0 Replies)
Discussion started by: chercheur111
0 Replies

10. Shell Programming and Scripting

Xargs to call python executable to process multiple bam files

I am running the below loop that to process the 3 bam files (which isn't always the case). A .py executable is then called using | xargs sh to further process. If I just run it with echo the output is fine and expected, however when | xargs sh is added I get the error. I tried adding | xargs... (4 Replies)
Discussion started by: cmccabe
4 Replies

LEARN ABOUT PHP

posix_getrlimit

POSIX_GETRLIMIT(3)							 1							POSIX_GETRLIMIT(3)

posix_getrlimit - Return info about system resource limits

SYNOPSIS

       array posix_getrlimit (void  )

DESCRIPTION

       posix_getrlimit(3) returns an array of information about the current resource's soft and hard limits.

	Each  resource has an associated soft and hard limit. The soft limit is the value that the kernel enforces for the corresponding resource.
       The hard limit acts as a ceiling for the soft limit. An unprivileged process may only set its soft limit to a value  from  0  to  the  hard
       limit, and irreversibly lower its hard limit.

RETURN VALUES

	Returns an associative array of elements for each limit that is defined. Each limit has a soft and a hard limit.

       List of possible limits returned

       +-----------+---------------------------------------------------+
       |Limit name |						       |
       |	   |						       |
       |	   |		     Limit description		       |
       |	   |						       |
       +-----------+---------------------------------------------------+
       |   core    |						       |
       |	   |						       |
       |	   |  The  maximum  size of the core file. When 0, not |
       |	   | core files  are  created.	When  core  files  are |
       |	   | larger  than this size, they will be truncated at |
       |	   | this size. 				       |
       |	   |						       |
       | totalmem  |						       |
       |	   |						       |
       |	   |  The maximum size of the memory of  the  process, |
       |	   | in bytes.					       |
       |	   |						       |
       |virtualmem |						       |
       |	   |						       |
       |	   |  The  maximum  size of the virtual memory for the |
       |	   | process, in bytes. 			       |
       |	   |						       |
       |   data    |						       |
       |	   |						       |
       |	   |  The maximum size of the  data  segment  for  the |
       |	   | process, in bytes. 			       |
       |	   |						       |
       |  stack    |						       |
       |	   |						       |
       |	   |  The maximum size of the process stack, in bytes. |
       |	   |						       |
       |   rss	   |						       |
       |	   |						       |
       |	   |  The  maximum number of virtual pages resident in |
       |	   | RAM					       |
       |	   |						       |
       | maxproc   |						       |
       |	   |						       |
       |	   |  The maximum number of processes that can be cre- |
       |	   | ated for the real user ID of the calling process. |
       |	   |						       |
       | memlock   |						       |
       |	   |						       |
       |	   |  The  maximum  number of bytes of memory that may |
       |	   | be locked into RAM.			       |
       |	   |						       |
       |   cpu	   |						       |
       |	   |						       |
       |	   |  The amount of time the process is allowed to use |
       |	   | the CPU.					       |
       |	   |						       |
       | filesize  |						       |
       |	   |						       |
       |	   |  The  maximum  size  of  the data segment for the |
       |	   | process, in bytes. 			       |
       |	   |						       |
       |openfiles  |						       |
       |	   |						       |
       |	   |  One more than the maximum number	of  open  file |
       |	   | descriptors.				       |
       |	   |						       |
       +-----------+---------------------------------------------------+
EXAMPLES

       Example #1

	      Example use of posix_getrlimit(3)

	      <?php

	      $limits = posix_getrlimit();

	      print_r($limits);
	      ?>

	      The above example will output something similar to:

	      Array
	      (
		  [soft core] => 0
		  [hard core] => unlimited
		  [soft data] => unlimited
		  [hard data] => unlimited
		  [soft stack] => 8388608
		  [hard stack] => unlimited
		  [soft totalmem] => unlimited
		  [hard totalmem] => unlimited
		  [soft rss] => unlimited
		  [hard rss] => unlimited
		  [soft maxproc] => unlimited
		  [hard maxproc] => unlimited
		  [soft memlock] => unlimited
		  [hard memlock] => unlimited
		  [soft cpu] => unlimited
		  [hard cpu] => unlimited
		  [soft filesize] => unlimited
		  [hard filesize] => unlimited
		  [soft openfiles] => 1024
		  [hard openfiles] => 1024
	      )

NOTES

       Note

	       This is a not POSIX function, but is common on BSD and System V systems. If the system does not support this function, then it will
	      not be included at compile time. This may be checked with function_exists(3).

SEE ALSO

       man page GETRLIMIT(2).

PHP Documentation Group 													POSIX_GETRLIMIT(3)
All times are GMT -4. The time now is 08:42 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy