help - exec time too long


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting help - exec time too long
# 1  
Old 09-28-2009
help - exec time too long

Dear everyone...

thanks to this forum I am able to do everyday more and more complex scripts...but now I come up with problem with optimisation..


problem 1 - optimise:
here is my code:

Code:
        while read number
                        do
                             nawk -F "|" -v ms="$number" ' {  if ($2==ms) { print $0 ; } } ' AllnumbersFile | nawk -F "|" '{print $2"|"$14 }'  | nawk -F"|" '{print $1"|"substr($0,13,4)""substr($0,18,2)""substr($0,21,2)}' >> MergedOutput
        done < $dir/1000NumbersFile

and files:

head 1000NumbersFile :

32762879208
48762880032
48762880052
48763881141
48762882490
32761884631
42761884632


head MergedOutput :

38762879208|20090928
32762879208|20090928
48762880032|20090928
48762880052|20090928
48763881141|20090928
48762882490|20090928
32761884631|20090928
42761884632|20090928




AllnumbersFile :

1|38762879208|0|1000|2000-01-01 00:00:00.0|90|1|0|false|0|0|0|0|1070-00-01 01:00:00.0|0|0|1900-01-01 01:00:00.0|1
1|48762880052|0|1000|2000-01-01 00:00:00.0|90|1|0|false|0|0|0|0|1070-00-01 01:00:00.0|0|0|1900-01-01 01:00:00.0|1
1|42761884632|0|1000|2000-01-01 00:00:00.0|90|1|0|false|0|0|0|0|1070-00-01 01:00:00.0|0|0|1900-01-01 01:00:00.0|1
1|48763881141|0|1000|2000-01-01 00:00:00.0|90|1|0|false|0|0|0|0|1070-00-01 01:00:00.0|0|0|1900-01-01 01:00:00.0|1


time is tooo long..


problem 2:
also is it possible taht when system is too busy my script from cron is being stopped. (I notices it becouse it executes everyday from cron and one day additional files is being erased - as script say and the other those files stay - like the end of script is not executed ?!?!?)

---------- Post updated at 01:55 PM ---------- Previous update was at 11:48 AM ----------

anyone??
# 2  
Old 09-28-2009
Bumping up posts or double posting is not permitted in these forums.

Please read the rules, which you agreed to when you registered, if you have not already done so.

You may receive an infraction for this. If so, don't worry, just try to follow the rules more carefully. The infraction will expire in the near future

Thank You.

The UNIX and Linux Forums.
# 3  
Old 09-28-2009
Code:
nawk -F='|' ' FILENAME=="1000NumbersFile" {[arr[$0]++}
         FILENAME=="AllNumbersFile"
         {
           if($2 in arr)
           {
             tmp= $2 "|" $14
             print $2 "|" substr(tmp,13,4)  substr(tmp,18,2)  substr(tmp,21,2)
           }
         }                
        '  100NumberFiles  AllNumbersFile  >> MergedOutput

You get to tweak this to get what is needed. Some of your code makes no sense to me. The idea is to read all of the first file, then read thru the second file just one time processing matching records. Note: a space between variables concatenates them in awk. I think nawk is new enough to do that correctly.
# 4  
Old 09-28-2009
You can try this solution.
Code:
awk -F'|' 'NR==FNR{arr[$0]++}$2 in arr{gsub("-","",$14);print $2 FS substr($14,1,8)}' 1000NumbersFile AllNumbersFile >> MergedOutput

# 5  
Old 09-29-2009
thanks guys....danmero I use your code..it is awsame..but some explanation would be nice..hehehehe

jim thanks but I do have problem :

nawk: syntax error at source line 1
context is
FILENAME=="1000NumbersFile" >>> {[ <<<
nawk: illegal statement at source line 1
missing ]


what might me a problem
# 6  
Old 09-29-2009
Quote:
Originally Posted by abdulaziz
jim thanks but I do have problem :
Quote:
Originally Posted by jim mcnamara
You get to tweak this to get what is needed
Quote:
Originally Posted by abdulaziz
thanks guys....danmero I use your code..it is awsame..but some explanation would be nice..hehehehe
If you can tell me what part is not clear maybe I will try to explain.
Your solution was over complicated Smilie
# 7  
Old 09-29-2009
well it is little bit hard to undestand which file U access first...becouse both files are just listed after awk.

I do not get this part:

arr[$0]++}$2

Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

First script in a long time

I was wondering if I could get some feedback on my script to grab time from our MDM... I blocked out all of the important stuff. I really appreciate any guidance, since I am long out of practice. #!/bin/bash serial=$1 # get last seen value of ipad lastseen=$(curl -s -X "GET"... (11 Replies)
Discussion started by: andysensible
11 Replies

2. Shell Programming and Scripting

Expect script idles for a long time

the following code works sometimes. other times, it behaves mysteriously. when the script sshs to a box, it is suppose to automatically begin running the command it is told to run. but in this case, after this script logs into a host, it just sits there at the prompt and does not run the... (1 Reply)
Discussion started by: SkySmart
1 Replies

3. UNIX for Dummies Questions & Answers

ls is taking long time to list

Hi, All the data are kept on Netapp using NFS. some directories are so fast when doing ls but few of them are slow. After doing few times, it becomes fast. Then again after few minutes, it becomes slow again. Can you advise what's going on? This one directory I am very interested is giving... (3 Replies)
Discussion started by: samnyc
3 Replies

4. Shell Programming and Scripting

How long ago since time - Using perl

echo "1337124526" | perl -pe 's/(\d+)/easttime($1)/e' the above gives a date and time. how can i subtract the date and time given by this command, from the current present date? can this be a one liner or as close to a one-liner as possible? (1 Reply)
Discussion started by: SkySmart
1 Replies

5. Shell Programming and Scripting

sort takes a long time

Dear experts I have a 200MG text file in this format: text \tab number I try to sort using options -fd and it takes very long! is that normal or I can speed it up in some ways? I dont want to split the file since this one is already splitted. I use this command: sort -fd file >... (12 Replies)
Discussion started by: voolek
12 Replies

6. UNIX for Dummies Questions & Answers

Job is taking long time

Hi , We have 20 jobs are scheduled. In that one of our job is taking long time ,it's not completing. If we are not terminating it's running infinity time actually the job completion time is 5 minutes. The job is deleting some records from the table and two insert statements and one select... (7 Replies)
Discussion started by: ajaykumarkona
7 Replies

7. Red Hat

login process taking a long time

I'm having a bit of a login performance issue.. wondering if anyone has any ideas where I might look. Here's the scenario... Linux Red Hat ES 4 update 5 regardless of where I login from (ssh or on the text console) after providing the password the system seems to pause for between 30... (4 Replies)
Discussion started by: retlaw
4 Replies

8. Linux

it takes long time to login on server

Hi, I am trying to login using ssh on Red Hat Linux 5 server, The password appears immediately but after I enter the password it takes about 90 seconds to login completely. Please suggest what changes require? Regards, Manoj (4 Replies)
Discussion started by: manoj.solaris
4 Replies

9. Programming

Debug env for long time use

Hi, I'm pritty new to C, but a recent bug in a program i've been using has forced me to debug it. But I am unable to find a debugger that can act as a layer between the OS and the program to see whats going on.. The problem is that this piece of software makes a connection through localhost... (2 Replies)
Discussion started by: nephilimbe
2 Replies

10. AIX

rcp gets hung for long time

Every evening I run a script in AIX production box, which executes below command: rcp prod_bkup.tar prodapp@IP:/data/appl/prod This will rcp a backup of around 11 GB from production to another machine (runs every evening so overwrites previous one). Just to keep the backup safe. Since 2-3 days,... (0 Replies)
Discussion started by: panchpan
0 Replies
Login or Register to Ask a Question