Sponsored Content
Top Forums Shell Programming and Scripting Slow performance filtering file Post 302495437 by Miila on Thursday 10th of February 2011 08:50:33 AM
Old 02-10-2011
Slow performance filtering file

Please, I need help tuning my script. It works but it's too slow.
The code reads an acivity log file with 50.000 - 100.000 lines and filters error messages from it. The data in the actlog file look similar to this:
Code:
02/08/2011 00:25:01,ANR2034E QUERY MOUNT: No match found using this criteria. (SESSION: 211433)
or
02/08/2011 01:05:50,"ANE4991I (Session: 211857, Node: xxx) blablabla (SESSION: 211857)"

I was trying to do the same with awk but it seems it's over my head. Any ideas will be appreciated. Thanks!

here is the code:
Code:
while read LINE
do
    A=$(echo ${LINE} | cut -c 21)
    if [[ ${A} == "\"" ]];then  
        MSGDATE=$(echo ${LINE} | cut -c 1-10)
        MSGTIME=$(echo ${LINE} | cut -c 12-19)
        PREFIX=$(echo ${LINE} | cut -c 22-24)
        MSGNUM=$(echo ${LINE} | cut -c 25-28)
        MSGTYPE=$(echo ${LINE} | cut -c 29)
        MESSAGE=$(echo ${LINE} | cut -c 30-)
        if [[ ${MSGTYPE} == "E" ]];then
            echo "${MSGDATE},${MSGTIME},${PREFIX}${MSGNUM}${MSGTYPE},${MESSAGE}" >> ${TMPFILE}
        fi
    else  
        MSGDATE=$(echo ${LINE} | cut -c 1-10)
        MSGTIME=$(echo ${LINE} | cut -c 12-19)
        PREFIX=$(echo ${LINE} | cut -c 21-23)
        MSGNUM=$(echo ${LINE} | cut -c 24-27)
        MSGTYPE=$(echo ${LINE} | cut -c 28)
        MESSAGE=$(echo ${LINE} | cut -c 29-)
        if [[ ${MSGTYPE} == "E" ]];then
            echo "${MSGDATE},${MSGTIME},${PREFIX}${MSGNUM}${MSGTYPE},${MESSAGE}" >> ${TMPFILE}
        fi
    fi
done <${ACTLOGFILE}

 

8 More Discussions You Might Find Interesting

1. Post Here to Contact Site Administrators and Moderators

Help! Slow Performance

Is the performance now very, very slow (pages take a very long time to load)? Or is it just me? Neo (6 Replies)
Discussion started by: Neo
6 Replies

2. Shell Programming and Scripting

egrep is very slow : How to improve performance

We have an egrep search in a while loop. egrep -w "$key" ${PICKUP_DIR}/new_update >> ${PICKUP_DIR}/update_record_new ${PICKUP_DIR}/new_update is 210 MB file In each iteration, the egrep on an average takes around 50-60 seconds to search. Ther'es nothing significant in the loop other... (7 Replies)
Discussion started by: hidnana
7 Replies

3. Filesystems, Disks and Memory

Slow Copy(CP) performance

Hi all We have got issues with copying a 2.6 GB file from one folder to another folder. Well, this is not the first issue we are having on the box currently, i will try to explain everything we have done from the past 2 days. We got a message 2 days back saying that our Production is 98%... (3 Replies)
Discussion started by: b_sri
3 Replies

4. UNIX for Dummies Questions & Answers

Slow copy/performance... between volumes

hi guys We are seeing weird issues on my Linux Suse 10, it has lotus 8.5 and 1 filesystem for OS and another for Lotus Database. the issue is when the Lotus service starts wait on top is very high about 25% percent and in general CPU usage is very high we found that when this happens if we... (0 Replies)
Discussion started by: kopper
0 Replies

5. Infrastructure Monitoring

99% performance wa, slow server.

There is a big problem with the server (VPS based on OpenVZ, CentOS 5, 3GB RAM). The problem is the following. The first 15-20 minutes after starting the server is operating normally, the load average is less than or about 1.0, but then begins to increase sharply% wa, then hovers around 95-99%.... (2 Replies)
Discussion started by: draiphod
2 Replies

6. Red Hat

GFS file system performance is very slow

My code Hi All, I am having redhat linux 5.3 (Tikanga) with GFS file system and its very very slow for executing ls -ls command also.Please see the below for 2minits 12 second takes. Please help me to fix the issue. $ sudo time ls -la BadFiles |wc -l 0.01user 0.26system... (3 Replies)
Discussion started by: susindram
3 Replies

7. Solaris

Solaris 11.1 Slow Network Performance

I have identical M5000 machines that are needing to transfer very large amounts of data between them. These are fully loaded machines, and I've already checked IO, memory usage, etc... I get poor network performance even when the machines are idle or copying via loopback. The 10 GB NICs are... (7 Replies)
Discussion started by: christr
7 Replies

8. Filesystems, Disks and Memory

Slow copy (cp) performance when overwriting files

I have a lot of binary files I need to copy to a folder. The folder is already filled with files of the same name. Copying on top of the old files takes MUCH longer than if I were to delete the old files then copy the new files to the now-empty folder. This result is specific to one system -... (3 Replies)
Discussion started by: ces55
3 Replies
ROUND(3)								 1								  ROUND(3)

round - Rounds a float

SYNOPSIS
float round (float $val, [int $precision], [int $mode = PHP_ROUND_HALF_UP]) DESCRIPTION
Returns the rounded value of $val to specified $precision (number of digits after the decimal point). $precision can also be negative or zero (default). Note PHP doesn't handle strings like "12,300.2" correctly by default. See converting from strings. PARAMETERS
o $val - The value to round o $precision - The optional number of decimal digits to round to. o $mode - Use one of the following constants to specify the mode in which rounding occurs. +--------------------+---------------------------------------------------+ | Constant | | | | | | | Description | | | | +--------------------+---------------------------------------------------+ | | | | PHP_ROUND_HALF_UP | | | | | | | Round $val up to $precision decimal places away | | | from zero, when it is half way there. Making 1.5 | | | into 2 and -1.5 into -2. | | | | | | | |PHP_ROUND_HALF_DOWN | | | | | | | Round $val down to $precision decimal places | | | towards zero, when it is half way there. Making | | | 1.5 into 1 and -1.5 into -1. | | | | | | | |PHP_ROUND_HALF_EVEN | | | | | | | Round $val to $precision decimal places towards | | | the next even value. | | | | | | | |PHP_ROUND_HALF_ODD | | | | | | | Round $val to $precision decimal places towards | | | the next odd value. | | | | +--------------------+---------------------------------------------------+ RETURN VALUES
The rounded value EXAMPLES
Example #1 round(3) examples <?php echo round(3.4); // 3 echo round(3.5); // 4 echo round(3.6); // 4 echo round(3.6, 0); // 4 echo round(1.95583, 2); // 1.96 echo round(1241757, -3); // 1242000 echo round(5.045, 2); // 5.05 echo round(5.055, 2); // 5.06 ?> Example #2 $mode examples <?php echo round(9.5, 0, PHP_ROUND_HALF_UP); // 10 echo round(9.5, 0, PHP_ROUND_HALF_DOWN); // 9 echo round(9.5, 0, PHP_ROUND_HALF_EVEN); // 10 echo round(9.5, 0, PHP_ROUND_HALF_ODD); // 9 echo round(8.5, 0, PHP_ROUND_HALF_UP); // 9 echo round(8.5, 0, PHP_ROUND_HALF_DOWN); // 8 echo round(8.5, 0, PHP_ROUND_HALF_EVEN); // 8 echo round(8.5, 0, PHP_ROUND_HALF_ODD); // 9 ?> Example #3 $mode with precision examples <?php /* Using PHP_ROUND_HALF_UP with 1 decimal digit precision */ echo round( 1.55, 1, PHP_ROUND_HALF_UP); // 1.6 echo round( 1.54, 1, PHP_ROUND_HALF_UP); // 1.5 echo round(-1.55, 1, PHP_ROUND_HALF_UP); // -1.6 echo round(-1.54, 1, PHP_ROUND_HALF_UP); // -1.5 /* Using PHP_ROUND_HALF_DOWN with 1 decimal digit precision */ echo round( 1.55, 1, PHP_ROUND_HALF_DOWN); // 1.5 echo round( 1.54, 1, PHP_ROUND_HALF_DOWN); // 1.5 echo round(-1.55, 1, PHP_ROUND_HALF_DOWN); // -1.5 echo round(-1.54, 1, PHP_ROUND_HALF_DOWN); // -1.5 /* Using PHP_ROUND_HALF_EVEN with 1 decimal digit precision */ echo round( 1.55, 1, PHP_ROUND_HALF_EVEN); // 1.6 echo round( 1.54, 1, PHP_ROUND_HALF_EVEN); // 1.5 echo round(-1.55, 1, PHP_ROUND_HALF_EVEN); // -1.6 echo round(-1.54, 1, PHP_ROUND_HALF_EVEN); // -1.5 /* Using PHP_ROUND_HALF_ODD with 1 decimal digit precision */ echo round( 1.55, 1, PHP_ROUND_HALF_ODD); // 1.5 echo round( 1.54, 1, PHP_ROUND_HALF_ODD); // 1.5 echo round(-1.55, 1, PHP_ROUND_HALF_ODD); // -1.5 echo round(-1.54, 1, PHP_ROUND_HALF_ODD); // -1.5 ?> CHANGELOG
+--------+---------------------------------------------------+ |Version | | | | | | | Description | | | | +--------+---------------------------------------------------+ | 5.3.0 | | | | | | | The $mode parameter was introduced. | | | | | 5.2.7 | | | | | | | The inner workings of round(3) was changed to | | | conform to the C99 standard. | | | | +--------+---------------------------------------------------+ SEE ALSO
ceil(3), floor(3), number_format(3). PHP Documentation Group ROUND(3)
All times are GMT -4. The time now is 05:02 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy