May I advise against executing a script within a find command? This would include, for every file found, creating a shell to run the script, implying a huge overhead, esp. in this case with "Large Number of Files". Why not collect all filenames in a working file and then work on that?
The whole point of my suggestion to use a script was to use the script to rearrange arguments to be passed to a simple command thereby allowing find's -exec primary to exec the underlying UNIX command with a large number of file operands instead of with a single file operand. I tried an experiment using the following commands on my relatively old MacBook Pro running Mac OS X Version 10.7.5, which sets {ARG_MAX} to 262144 bytes (i.e., 256Kb). I used cp as the test command to copy all of the PDF files found in and under my home directory to /tmp/pdfdest. /tmp/pdfdest was a new directory when I started this test, but I did not empty and recreate the directory between tests.
I used the command:
three times and ignored the 1st time (which ran a lot slower than the other two as it cleared out the various caches and loaded my home directory's file hierarchy). The two remaining runs averaged 2 minutes 27.45 seconds wall clock time, 0.85 seconds user clock time, and 5.56 seconds system time to copy 881 files by invoking cp 881 times.
Adding the following Korn shell script (named CpDest1st):
and using the command:
three times and again ignored the first set of results and averaged the other two sets of results. Copying the same 881 files using this method invoking the shell script and the cp utility once each took 2 minutes 3.02 seconds wall time, 0.42 seconds user time, and 3.76 seconds system time.
This isn't a statistically valid comparison and your mileage will vary depending on the value of {ARG_MAX} on your system, the number and sizes of the PDF files being copied, ... . But, it does show that the overhead of using a shell script to reduce the number of invocations of cp (or mv or many other utilities) may actually reduce the elapsed time and the amount of system resources used. In this simple test, using the shell script reduced wall clock time 16%, reduced system time 50%, and reduced user time 32%.
hi all,
i am looking for ways to make ftp efficient by tuning the parameters
currently,
tcp_max_buf is 1 MB
tcp_xmit_hiwat is 48 KB
say to transmit multiple 2 gb files from unix server to mainframe sys,
will increasing the window size or the send buffer size of the current TCP/IP... (6 Replies)
Language: ksh
OS: SunOS
I have been getting the 'subscript out of range' error when the below array variable gets elements greater that 1024. I understand that 1024 is the default size for 'set -A' dynamic array, but is there a way to initialize it with a larger number?
set -A arr `grep... (6 Replies)
Hi,
Does anyone know if it is possible to tar files larger than 2GB? The reason being is they want me to dump a single file (which is around 20GB) to a tape drive and they will restore it on a Solaris box. I know the tar have a limitation of 2GB so I am thinking of a way how to overcome this.... (11 Replies)
Hi,
I am new to unix , I was planning to write a script that will FTP files to destination folder. , Please guide me what are the various networking commands that unix will help in this ftp process..?:confused: (1 Reply)
Hello!
This is my first post, and I just learned what UNIX was this week. For a JAVA programming class I am taking, I must be able to create a directory in UNIX, use the nano command to create a JAVA program, compile it, and then run it on the command prompt using the java command.
For some... (5 Replies)
My unzip command doesn't work for files that are greater than 4GB. Consider my file name is unzip -p -a filename.zip, the command doesn't work since the size of the file is larger. I need to know the corresponding 7z command for the same. This is my Unix shell script program:
if
then
... (14 Replies)
I need to backup my database but the files are very large and the TAR command will not let me. I searched aids and found that I could do something with the mknod, COMPRESS and TAR command using them together. I appreciate your help. (10 Replies)
Hi,
I have one large files of 100000 rows with header column.
Eg:
Emp Code, Emp Name
101,xxx
102,YYY
103,zzz
...
...
I want to split the files into smaller files with only 30000 rows each..File 1,2 and 3 must have 30000 rows and file 4 must contain 10000 rows.
But the column... (1 Reply)
I need help modifying these two scripts to do the following:
- print files in (MB) instead of (KB)
- only select files larger than 500MB -> these will be mailed out daily
- Select all files regardless of size all in (MB) -> these will be mailed out once a week
this is what i have so far and... (5 Replies)
I am new at developing EXPECT scripts. I'm trying to create a script that will automatically connect to a several UNIX (sun solaris and HPUX) database server via FTP and pull the sizes of the listener/alert log files from specified server directory on the remote machines.
1. I want the script... (7 Replies)
Discussion started by: mikebantor
7 Replies
LEARN ABOUT HPUX
time
time(1) General Commands Manual time(1)NAME
time - time a command
SYNOPSIS
command
utility [argument ...]
DESCRIPTION
When a specified command or utility completes execution, prints the elapsed time during the command or utility, the time spent in the sys-
tem, and the time spent executing the command or utility. Times are reported in seconds.
Execution time can depend on the performance of the memory in which the program is running.
The times are printed to standard error.
Note that the shell also has a keyword that times an entire pipeline if used anywhere in the pipeline. This action is different than the
command which times a particular command if used in a pipeline.
Options
recognizes the following options:
command The command to be executed and timed.
Writes the timing statistics to standard error.
utility The name of a utility to be invoked and timed. If the utility operand names any of the shell special built-in utilities,
the time results are undefined. See csh(1) and ksh(1) for information about special built-in utilities.
argument Any string that is an argument to the utility.
SEE ALSO csh(1), ksh(1), sh(1), timex(1), times(2).
STANDARDS CONFORMANCE time(1)