Script Performance problem . urgent frnds


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script Performance problem . urgent frnds
# 8  
Old 08-03-2009
You opens files (for reading or writing) too many times :

Code:
total_lines=`wc -l $1 | awk '{print $1}'`

grep -n "ORA-" $1 |cut -d: -f1 >> ora_occ.txt
grep -n "Targ[[:space:]]Rowid" $1 | cut -d: -f1 >> targ_occ.txt
grep -n '^)$' $1 | cut -d: -f1 >> bracket_occ.txt

nr_of_occ=`wc -l targ_occ.txt | awk '{print $1}'`

   while [ "$i" -le "$nr_of_occ" ]
   do

      ora_line_nr=`sed -n \`echo $i\`p ora_occ.txt`
      targ_line_nr=`sed -n \`echo $i\`p  targ_occ.txt`
      bracket_line_nr=`sed -n \`echo $i\`p bracket_occ.txt`

      sed -n "$ora_line_nr p" $1 > ora_error_detail.txt

      while read ora_line
      do

      done <ora_error_detail.txt


      sed -n "$rec_start,$rec_end p " $1 > data_block.txt

      sed -e 's/(.*):/=/' data_block.txt |awk '{ line =line $0"|"} END { print line}' >>out_file.txt

   done

For an inputfile with 5000 rejected records (46 columns each) :
Code:
                          Open for  Open for
File            Records       Read     Write
--------------- -------   --------  --------
Inputfile             ?       5004
ora_occ.txt        5000       5000
targ_occ.txt       5000       5000
bracket_occ.txt    5000       5000
ora_error_deatail     1       5000      5000
data_block.txt       46       5000      5000 
--------------- -------   --------  --------
Total                        30004     10000

The awk solution reads the file only once, and doesn't use any other file.

Jean-Pierre.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

URGENT Reading a file and assessing the syntax shell script URGENT

I am trying to write a shell script which takes an input file as an arguement in the terminal e.g. bash shellscriptname.sh input.txt. I would like for the file to be read line by line each time checking if the .txt file contains certain words or letters(validating the syntax). If the line being... (1 Reply)
Discussion started by: Gurdza32
1 Replies

2. Shell Programming and Scripting

Performance problem in Shell Script

Hi, I am Shell script beginner. I wrote a shell programming that will take each line of a file1 and search for it in another file2 and give me the output of the lines that do not exist in the file2. I wrote it using do while nested loop but the problem here is its running for ever . Is there... (12 Replies)
Discussion started by: sakthisivi
12 Replies

3. UNIX for Advanced & Expert Users

Performance problem with bidirectional nc

Working on a simple, half duplex network diagnostic that will run anywhere using nc and dd. Performance is symmetrical with sink and source nc processes open as a server: nc -vkl 5000 > /dev/null & cat /dev/zero | nc -vkl 5001 & With this on the client: nc host0 5001 | dd of=/dev/null... (0 Replies)
Discussion started by: netdrx
0 Replies

4. Solaris

Performance problem

Hi All, There is a virtual user "ecoouk" which logs on to the server and runs some scripts. I want to know how much server performance can I gain if I put off all the scripts run by this user. Please tell me how to analyse how much resources a specific user is using. Regards, Abhishek (3 Replies)
Discussion started by: max29583
3 Replies

5. Shell Programming and Scripting

Getting Following problem when opening shell script (Very Urgent)

Hi All, Actually when i am trying to open my shell script file 1.sh then i am getting the following error. > vi 1.sh "/var/tmp/ExdNaarK" No space left on device Can anybody tell me that how to rectify it. It is very urgent. Because i am not able to do any work due to the above error. ... (1 Reply)
Discussion started by: sunitachoudhury
1 Replies

6. Shell Programming and Scripting

Performance problem with my script ...suggestions pls

Hi , I have included my script below, pls read thro this req. I want my script to run for every hour , the problem is I CANNOT USE CRONTAB which is prohibited inside the company. My script does what it is supposed to do (to determine the memory and then send a email if it crosses a certain... (2 Replies)
Discussion started by: vivsiv
2 Replies

7. Shell Programming and Scripting

shell script performance issues --Urgent

I need help in awk please help immediatly. This below function is taking lot of time Please help me to fine tune it so that it runs faster. The file count is around 3million records # Process Body processbody() { #set -x while read line do ... (18 Replies)
Discussion started by: icefish
18 Replies

8. Shell Programming and Scripting

urgent: script problem???

#!/usr/bin/sh read file #exec 3<&0 exec 0<$file while read line do if then BOUND=$line break fi count=`expr $count + 1` done #exec 0<&3 echo $BOUND sh check.sh (4 Replies)
Discussion started by: skyineyes
4 Replies

9. UNIX for Advanced & Expert Users

performance problem

Hello, I have a mail server (sendmail) with SUNOS 5.5.1. Just recently it began to respond very slowly. I used vmstat to check the performance data. Only interupt, system call and CPU context swiching are relatively high. Other statistics are normal, especially CPU utilization are very... (5 Replies)
Discussion started by: caoai
5 Replies
Login or Register to Ask a Question