11-14-2006
You want a single process and this does that. A perl or ksh solution might beat this by a little bit, provided that they carefully use only built-in commands and never invoke anything external. Perl and ksh compile the script while awk does not. And a custom C program can beat anything else.
Your 5 stage pipeline will not be even close to a single process. Even if you have 5 CPU's available that can be dedicated to the pipeline, all of that reading and writing to pipes is expensive. (Anything is expensive when you do it many millions times.) And you probably do not have 5 CPU's available for the entire run. Without 5 dedicated CPU's you will need to context switch several million times as well.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
what's the most efficient and effective search for a file in a dir ?
I see many guys use this
# find - print
or something as such ? and sometimes pipe it to something else ?
Is there a better way of using "grep" in all of this ?
thanks
simon2000 (3 Replies)
Discussion started by: simon2000
3 Replies
2. Shell Programming and Scripting
Hi all,
I need to pull the (last section) from the header to the end of file. This logfile will have many sections appended, but I only need to capture the last one and email it to someone. Any ideas? Thanks Kathy
Example Logfile:
=============
= BOX : SAPCFI
= JOB : SAPCFI
=... (9 Replies)
Discussion started by: kburrows
9 Replies
3. Shell Programming and Scripting
Hi,
I need to make a script to edit a file. File is a large file in below format
Version: 2008120101
;$INCLUDE ./abc/xyz/Delhi
;$INCLUDE ./abc/xyz/London
$INCLUDE ./abc/xyz/New York
First line in the file is version number which is in year,month,date and serial number format. Each... (5 Replies)
Discussion started by: makkar4u
5 Replies
4. Red Hat
hi all,
We are trying to run a process in the background and in the process we call fork ;and wait for the child process to finish .We find that the died = wait(&status); happens after 10 seconds randomly and sometimes completes in time (within 1 sec)
This behavior is seen only when the... (0 Replies)
Discussion started by: vishnu.priya
0 Replies
5. Red Hat
hi all,
We are trying to run a process in the background and in the process we call fork ;and wait for the child process to finish .We find that the died = wait(&status); happens after 10 seconds randomly and sometimes completes in time (within 1 sec)
This behavior is seen only when the... (1 Reply)
Discussion started by: vishnu.priya
1 Replies
6. Shell Programming and Scripting
Hi Friendz,
I have 14 DB load scripts say 1,2,3....14. I want a script to call each script automatically, and after completion of every script, it needs to track the logfile and mail that log file to a group.and again it should run the next script in sequence 1,2,3...14
Please help me,need... (1 Reply)
Discussion started by: shirdi
1 Replies
7. UNIX for Dummies Questions & Answers
Can anyone explain me in details of Real and Effective IDs (6 Replies)
Discussion started by: kkalyan
6 Replies
8. Cybersecurity
I was looking for a good list of words to exclude people from using as passwords, i.e. those that could be guessed easily. I'm working through a whole bunch of suggestions from skullsecurity.org, but I managed to find this page that seems to suggest I have more options than I thought. :b:
I... (1 Reply)
Discussion started by: rbatte1
1 Replies
9. Shell Programming and Scripting
Bonjour,
I've wrote a script to monitor a logfile in realtime. It is working almost perfeclty except for two things.
The script use the following technique :
tail -fn0 $logfile | \
while read line ; do
... some stuff
done
First one, I'd like a way to end the monitoring script if a... (3 Replies)
Discussion started by: Warluck
3 Replies
10. Shell Programming and Scripting
Hi there, I'm camor and I'm trying to process huge files with bash scripting and awk.
I've got a dataset folder with 10 files (16 millions of row each one - 600MB), and I've got a sorted file with all keys inside.
For example:
a sample_1 200
a.b sample_2 10
a sample_3 10
a sample_1 10
a... (4 Replies)
Discussion started by: camor
4 Replies
LEARN ABOUT OPENDARWIN
times
times(1) User Commands times(1)
NAME
times - shell built-in function to report time usages of the current shell
SYNOPSIS
sh
times
ksh
times
DESCRIPTION
sh
Print the accumulated user and system times for processes run from the shell.
ksh
Print the accumulated user and system times for the shell and for processes run from the shell.
On this man page, ksh(1) commands that are preceded by one or two * (asterisks) are treated specially in the following ways:
1. Variable assignment lists preceding the command remain in effect when the command completes.
2. I/O redirections are processed after variable assignments.
3. Errors cause a script that contains them to abort.
4. Words, following a command preceded by ** that are in the format of a variable assignment, are expanded with the same rules as a vari-
able assignment. This means that tilde substitution is performed after the = sign and word splitting and file name generation are not
performed.
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+-----------------------------+-----------------------------+
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+-----------------------------+-----------------------------+
|Availability |SUNWcsu |
+-----------------------------+-----------------------------+
SEE ALSO
ksh(1), sh(1), time(1), attributes(5)
SunOS 5.10 15 Apr 1994 times(1)