12-26-2011
Help with Efficient Looping
Hello guys
My requirement is to read a file with parent-child relationship
we need to iterate through each row to find its latest child.
for eg. parent child
ABC PQR
PQR DEF
DEF XYZ
Expected Output
ABC XYZ
PQR XYZ
DEF XYZ
Script Logic :
read parent from file
seach child =parent in file if match found replace child with parent
else
go to next line
I have created a bash script to achive this and its working fine
My issue is I need to process a file with more than 2 million records.
My script is taking one and half hrs for 25000 records
Can anyone suggest more effecient approach
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
Does anyone know what's new with Efficient dispatching in the Solaris 2.8 release (vs Solaris 2.6) release?
Specifically, does anyone know of a good website to get detailed information on thread dispatching using efficient dispatching in solaris 2.8?
Thank you. (1 Reply)
Discussion started by: uchachra
1 Replies
2. Shell Programming and Scripting
I'm using korn shell to connect to oracle, retrieve certain values, put them in a list, and iterate through them. While this method works, I can't help but think there is an easier method.
If you know of one, please suggest a shorter, more efficient method.
############### FUNCTIONS ... (6 Replies)
Discussion started by: SelectSplat
6 Replies
3. Shell Programming and Scripting
Hi,
Can someone let me know if the below AWK can be made much simpler / efficient ?
I have 200 fields, I need to substr only the last fields.
So i'm printing awk -F~ 'print {$1, $2, $3....................................$196,$197 , susbstr($198,1,3999), substr($199,1,3999)..}'
Is there a... (4 Replies)
Discussion started by: braindrain
4 Replies
4. Shell Programming and Scripting
Hi
I have the following at the end of a service shutdown script used in part of an active-passive failover setup:
###
# Shutdown all primary Network Interfaces
# associated with failover
###
# get interface names based on IP's
# and shut them down to simulate loss of
# heartbeatd
... (1 Reply)
Discussion started by: mikie
1 Replies
5. Shell Programming and Scripting
Hello,
We have a directory with 15 sub-directories where each sub-directory contains 1.5 to 2 lakhs of files in it. Daily, around 300-500 files will be uploaded to each sub-directory.
Now, i need to get the list of files received today in most efficient way. I tried using "find with newer... (16 Replies)
Discussion started by: prvnrk
16 Replies
6. Shell Programming and Scripting
I have the following code.
printf "Test Message Report" > report.txt
while read line
do
msgid=$(printf "%n" "$line" | cut -c1-6000| sed -e 's///g' -e 's|.*ex:Msg\(.*\)ex:Msg.*|\1|')
putdate=$(printf "%n" "$line" | cut -c1-6000| sed -e 's///g' -e 's|.*PutDate\(.*\)PutTime.*|\1|')... (9 Replies)
Discussion started by: gugs
9 Replies
7. Shell Programming and Scripting
Hi,
i have 2 files each with 200K lines. Each line contains a number. Now, i need to get the list of numbers existing in one fine and NOT in other file. I'm doing this by reading each number from 1 file and grepping on other file. But this taking LOT of time. Is there any efficient way of doing... (14 Replies)
Discussion started by: prvnrk
14 Replies
8. UNIX for Advanced & Expert Users
some of the data i receive has been typed in manually due to which there are often places where i find 8 instead of ( and the incorrect use of case
what according to you is the best way to correct such data.
The data has around 20,000 records.
The value i want to change is in the 4th field.... (2 Replies)
Discussion started by: VGR
2 Replies
9. UNIX for Dummies Questions & Answers
I want to match the red portion:
9784323456787-Unknown Phrase with punctuation "Some other PhrASE."
Is this the best regex to match this?
'978\{10\}-*' (4 Replies)
Discussion started by: glev2005
4 Replies
10. UNIX for Advanced & Expert Users
Hi Experts,
I've been trying simple grep to search for a string in a huge number of files in a directory.
grep <pattern> *
this gives the search results as well as the following -
grep: <filename>: Permission denied
grep: <filename>: Permission denied
for files which I don't have... (4 Replies)
Discussion started by: sumoka
4 Replies
LEARN ABOUT OPENDARWIN
fork
FORK(2) BSD System Calls Manual FORK(2)
NAME
fork -- create a new process
SYNOPSIS
#include <sys/types.h>
#include <unistd.h>
pid_t
fork(void);
DESCRIPTION
Fork() causes creation of a new process. The new process (child process) is an exact copy of the calling process (parent process) except for
the following:
o The child process has a unique process ID.
o The child process has a different parent process ID (i.e., the process ID of the parent process).
o The child process has its own copy of the parent's descriptors. These descriptors reference the same underlying objects, so that,
for instance, file pointers in file objects are shared between the child and the parent, so that an lseek(2) on a descriptor in the
child process can affect a subsequent read or write by the parent. This descriptor copying is also used by the shell to establish
standard input and output for newly created processes as well as to set up pipes.
o The child processes resource utilizations are set to 0; see setrlimit(2).
RETURN VALUES
Upon successful completion, fork() returns a value of 0 to the child process and returns the process ID of the child process to the parent
process. Otherwise, a value of -1 is returned to the parent process, no child process is created, and the global variable errno is set to
indicate the error.
ERRORS
Fork() will fail and no child process will be created if:
[EAGAIN] The system-imposed limit on the total number of processes under execution would be exceeded. This limit is configuration-
dependent.
[EAGAIN] The system-imposed limit MAXUPRC (<sys/param.h>) on the total number of processes under execution by a single user would
be exceeded.
[ENOMEM] There is insufficient swap space for the new process.
SEE ALSO
execve(2), wait(2)
HISTORY
A fork() function call appeared in Version 6 AT&T UNIX.
4th Berkeley Distribution June 4, 1993 4th Berkeley Distribution