Looking at the performance hit on my server, does it matter wich command I run?
or
I think it doesn't really matter because both command strings involve a tar being run on the server and client. Both commandstrings do a read on the server and a write on the client.
Are there compelling reasons one would choose for option 1 or option 2?
They are identical and flawed. I like '*' or '.??* *' over '.'; all file names 2 bytes shorter!
Option 3 is to put in compression, and, although newer tar's have compression options, for speed and versatility outside tar so you can use more CPUs and try different compressors: compress (faster with most output), gzip, bzip2 (slowest with least output) with various options. Usually, bzip2 is too slow to be useful, but compress, gzip -1, gzip -7 or gzip -9 wins.
Here is with compress/zcat and tar:
Networks are rarely faster than CPUs, and the gap is widening.
I also like cpio over tar, but some tar's with some options have stopped padding every file to record length with nulls! Also, you can run parallel streams to get a little more out of the network by dividing the file name stream into cpio using xdemux.c: https://www.unix.com/shell-programming-scripting/146458-sort-big-data-file.html After all, what are 32 CPU systems for? Parallel streams means you can use slower, better compression and still max out the network, getting a higher total bandwidth. Of course, if one file is 90% of the bulk, not gonna help.
There is the more secure ssh, if security ever becomes a concern, but the encryption costs some speed. It has optional gzip-like -C compression option, if you want to save a few keystrokes and lose some speed.
Also, if there are already old files there, the -u can save you some writing of identical files. That might not be your bottleneck, and someone might play with the timestamps on files, or have updates you do not want to preserve, but sometimes updated files are worth keeping (like local configuration).
Finally, -n saves a few microseconds in forking an unused process for full duplex, when rsh reads nothing from stdin.
Last edited by DGPickett; 04-27-2011 at 03:14 PM..
Hello Coders
Some time ago i was asking about python and bash performances, and i was told i could post the regarding code, and someone would kindly help to make it faster (if possible).
If you have noted, i'm on the way to finalize, finish, stable TUI - Text(ual) User Interface.
It is a... (6 Replies)
A few of our machines need upgrading and we are looking into a selection of processors at present. There are suggestions on the vendor's websites that the L3 cache was specifically introduced for gamers. Is this true? Does having L1, L2 and/or L3 cache help at all in performance or are the... (0 Replies)
We have sun solaris server v440 and when we do netat -a | wc , it takes more than 5 minutes to complete. Do you think its a network problem or server load issue? (1 Reply)
Hello,
I run my pthread code on Linux with 4 processors. However, the speed up is only 2 times.
The code is about solving equation (G+s(i)C)z(i)=B*us(i), i=1,...,n. Here G,C are m*m matrix, B*us(i) is a m*1 vector and s(i) are n different numbers. I need to solve the equation n times to... (1 Reply)
I am trying to analyze the performance of an AIX system. I think I may have a disk I/O issue, but I am asking for help to validate or invalidate this assumption. I ran the commands below during a period of peak load.
Please help me to find any performance bottlenecks. Thanks in advance for your... (15 Replies)
About 4 years ago I wrote this tool inspired by Rob Urban's collect tool for DEC's Tru64 Unix. What makes this tool as different as collect was in its day is its ability to run at a low overhead and collect tons of stuff. I've expanded the general concept and even include data not available in... (0 Replies)
Running dedicated on AIX with 4 processors, creating 4 threads, each with equal work to do, only runs about 20% faster than 1 thread with all of the work. Test case has no blocking but does share memory for read access only.
Any ideas why I'm only seeing 20% gain? Is this typical on AIX? ... (1 Reply)
I am wracking my brains over this. I am trying to use a Korn Shell script to execute an Oracle PL/SQL procedure, using the Oracle command line interface (sqlplus). The script starts sqlplus in a coprocess, and the two processes communicate using a two-way pipe. The bgnice option is off, so both... (8 Replies)