Solaris

The Solaris Operating System, usually known simply as Solaris, is a Unix-based operating system introduced by Sun Microsystems. The Solaris OS is now owned by Oracle.

Rsync quite slow (using very little cpu): how to improve its speed?


👤 Login to reply

    #8  
Old 06-05-2018
priyadarshan priyadarshan is offline
Registered User
 
Join Date: May 2018
Last Activity: 14 July 2018, 5:44 PM EDT
Posts: 6
Thanks: 10
Thanked 0 Times in 0 Posts
No, I have been running tests on a new system, with just OS install,
Code:
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
 rpool   476G   136G   340G        -         -     0%    28%  1.00x  ONLINE  -

After advice on this forum, I simplified testing, now only on local SSD.

I created a 10G file (with urandom, to avoid zfs caching), and had following results (tested many times, best result reported):

Code:
### cp to same SDD

root@omniosce:~# time cp random.bin random1.bin

real    0m5.671s
user    0m0.134s
sys     0m4.974s

root@omniosce:~# time rsync -a random.bin random2.bin

real    1m25.644s
user    2m24.261s
sys     0m14.273s

### rsync'ing to same SDD with Joyent pkgsrc rsync

root@omniosce:~# time /opt/local/bin/rsync -a random.bin random1.bin

real    0m31.302s
user    0m40.634s
sys     0m13.994s

The last result is very close to what I get with FreeBSD on exact same hardware.

So, it seems to me, a possible reason could lie in OmniOS core rsync.

Last edited by priyadarshan; 06-05-2018 at 10:16 AM..
Sponsored Links
    #9  
Old 06-15-2018
priyadarshan priyadarshan is offline
Registered User
 
Join Date: May 2018
Last Activity: 14 July 2018, 5:44 PM EDT
Posts: 6
Thanks: 10
Thanked 0 Times in 0 Posts
Mystery solved

The mystery has finally been solved, by switching OmniOS rsync to 64-bit and enable optimisation. That more than doubles the speed of the checksumming code.


Now, considering exact same hardware, rysnc on OmniOS is even a bit faster than FreeBSD's.


Thank you to all here, and thanks to the excellent OmniOS devs that helped so promptly and exhaustively.

Last edited by priyadarshan; 06-15-2018 at 08:48 AM..
Sponsored Links
    #10  
Old 06-15-2018
MadeInGermany MadeInGermany is offline Forum Staff  
Moderator
 
Join Date: May 2012
Last Activity: 21 July 2018, 3:53 AM EDT
Location: Simplicity
Posts: 4,156
Thanks: 365
Thanked 1,419 Times in 1,275 Posts
And thank you for updating us!
    #11  
Old 4 Weeks Ago
achenle achenle is offline Forum Advisor  
Registered User
 
Join Date: Jun 2009
Last Activity: 25 June 2018, 8:15 AM EDT
Posts: 1,015
Thanks: 3
Thanked 156 Times in 148 Posts
FWIW - on an x86-based server, processes running in 32-bit mode have access to eight 32-bit general purpose registers, three of which are PC (program counter), SP (stack pointer), and FP (frame pointer). So, unless the compilation process includes optimizations like '-fomit-frame-pointer', the process gets all of five general-purpose registers.

In 64-bit mode, processing have access to sixteen 64-bit general-purpose registers.

So an unoptimized 32-bit process gets to actually use five 32-bit registers, and an optimized 64-bit process gets to actually use fourteen 64-bit registers.

Guess which one's faster on the exact same hardware. ;-)
The Following User Says Thank You to achenle For This Useful Post:
priyadarshan (4 Weeks Ago)
Sponsored Links
    #12  
Old 4 Weeks Ago
priyadarshan priyadarshan is offline
Registered User
 
Join Date: May 2018
Last Activity: 14 July 2018, 5:44 PM EDT
Posts: 6
Thanks: 10
Thanked 0 Times in 0 Posts
Thank you for the explanation!
Sponsored Links
👤 Login to reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Improve script - slow process with big files jiam912 Shell Programming and Scripting 13 01-27-2017 10:02 AM
Help to improve speed of text processing script lorus Shell Programming and Scripting 19 07-27-2009 12:11 PM
egrep is very slow : How to improve performance hidnana Shell Programming and Scripting 7 02-12-2008 06:13 AM
network speed is slow q30 UNIX for Advanced & Expert Users 3 05-28-2003 05:16 AM



All times are GMT -4. The time now is 07:32 AM.

Unix & Linux Forums Content Copyrightę1993-2018. All Rights Reserved.
×
UNIX.COM Login
Username:
Password:  
Show Password





Not a Forum Member?
Forgot Password?