Sponsored Content
Top Forums Programming File - reading - Performance improvement Post 302198431 by aamirglb on Friday 23rd of May 2008 01:24:49 AM
Old 05-23-2008
Hello!
What you can try out is: have a huge circular buffer for example say around 6144 (6KB) , you can experiment with the size!!
What i mean by circular is have two pointers, start_ptr and processed_ptr.

Code:
offset = 0;
read(fd, &buffer[offset], 3KB);
if(offset == 0)
{
    // next time read in the next chunk of buffer
    offset = 3KB;
    start_ptr = 0;
}
else
{    
    offset = 0;
    start_ptr = 3KB; 
}

bytes_read = start_ptr - processed_ptr;

//start processing it
while(bytes_read >= minimum_size_of_record)
{
     ret_val = check_for_complete_record(processed_ptr);
    // incomplete record
     if(ret_val == -1)
     {
             // don't modify processed_ptr since the record is not complete
             break; //without modifying the pointers 
     }
     else
     {
          // in this case check_for_complete_record will return the size of record
          bytes_read = bytes_read - ret_val;
          processed_ptr = processed_ptr + ret_val;
     }      
}

1) Have start_ptr and processed_ptr as global
2) You must take care of rollover of processed_ptr for every read

Code:
     if(processed_ptr >= MAX_BUFFER_SIZE) // in this case 6KB
               processed_ptr = 0;

Regards,
Aamir
 

7 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script ready but might need some improvement.

Hi All, I have written a script which does some editing in the files, based on user input.This might not be the most elegant way of doing it and there would be many improvements needed. Please go through it and let me know how it could be improved. Suggestions are welcome!! Thanks!... (2 Replies)
Discussion started by: nua7
2 Replies

2. Shell Programming and Scripting

Any improvement possible in this script

Hi! Thank you for the help yesterday This is the finished product There is one more thing I would like to do to it but I’m not to certain On how to proceed I would like to log all output to a log in order to Be able to roll back This script is meant to be used in repairing a... (4 Replies)
Discussion started by: Ex-Capsa
4 Replies

3. Infrastructure Monitoring

Possible performance improvement (Bash and flat file)

Hello, I am pretty new to shell scripts and I recently wrote one that seems to do what it should but I am exploring the possibility of improving its performance and would appreciate some help. Here is what it does - Its meant to monitor a bunch of systems (reads in IPs one at a time from a flat... (9 Replies)
Discussion started by: prafulnama
9 Replies

4. UNIX for Advanced & Expert Users

linux os improvement

can anyone help to share the knowledge on linux os improvement? 1) os account - use window AD authentication, such as ldap, but how to set /etc/passwd, where to put user home? 2) user account activity - how to log os user activity share the idea and what tools can do that...thx (5 Replies)
Discussion started by: goodbid
5 Replies

5. Shell Programming and Scripting

I need the improvement for my script

Hi All, Here is my script #! /bin/sh var1=some email id var2=some email id grep -i "FAILED FILE FORMAT VALIDATION" /opt >tmp2 diff tmp1 tmp2 | grep ">" >tmp3 if then cat tmp3 | mailx -s " Error Monitoring" $var2 else echo "Pattern NOt Found" | mailx -s " Error Monitoring" $var1... (1 Reply)
Discussion started by: Gopalak
1 Replies

6. UNIX for Dummies Questions & Answers

Improvement in shell script

Hi This is my Following code: #!/bin/sh echo "TOTAL_NO_OF_MAILS" read TOTAL_NO_OF_MAILS echo "TOTAL_NO_OF_TICKETS " read TOTAL_NO_OF_TICKETS echo "TICKETS_IN_QUEUE" read TICKETS_IN_QUEUE rm -rf `pwd`/Focus echo "Hi Team\nSTATS IN CLRS MAIL BOX\n\n==============================" >> Focus... (11 Replies)
Discussion started by: wasim999
11 Replies

7. Shell Programming and Scripting

Performance improvement in grep

Below script is used to search numeric data from around 400 files in a folder. I have 300 such folders. Need help in performance improvement in the script. Below Script searches 20 such folders ( 300 files in each folder) simultaneously. This increases cpu utilization upto 90% What changes... (3 Replies)
Discussion started by: vegasluxor
3 Replies
pthread_rwlock_wrlock(3C)												 pthread_rwlock_wrlock(3C)

NAME
pthread_rwlock_wrlock, pthread_rwlock_trywrlock - lock or attempt to lock read-write lock object for writing SYNOPSIS
cc -mt [ flag... ] file... -lpthread [ library... ] #include <pthread.h> int pthread_rwlock_wrlock(pthread_rwlock_t *rwlock); int pthread_rwlock_trywrlock(pthread_rwlock_t *rwlock); The pthread_rwlock_wrlock() function applies a write lock to the read-write lock referenced by rwlock. The calling thread acquires the write lock if no other thread (reader or writer) holds the read-write lock rwlock. Otherwise, the thread blocks until it can acquire the lock. The pthread_rwlock_trywrlock() function applies a write lock like the pthread_rwlock_wrlock() function, with the exception that the func- tion fails if any thread currently holds rwlock (for reading or writing). Writers are favored over readers of the same priority to avoid writer starvation. See pthread_rwlock_rdlock(3C). Results are undefined if any of these functions are called with an uninitialized read-write lock. If a signal is delivered to a thread waiting for a read-write lock for writing, upon return from the signal handler the thread resumes waiting for the read-write lock for writing as if it was not interrupted. If successful, the pthread_rwlock_wrlock() function returns 0. Otherwise, an error number is returned to indicate the error. The pthread_rwlock_trywrlock() function returns 0 if the lock for writing on the read-write lock object referenced by rwlock is acquired. Otherwise an error number is returned to indicate the error. The pthread_rwlock_wrlock() function will fail if: EDEADLK The current thread already owns the read-write lock for writing or reading. The pthread_rwlock_trywrlock() function will fail if: EBUSY The read-write lock could not be acquired for writing because it was already locked for reading or writing. See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ |MT-Level |MT-Safe | +-----------------------------+-----------------------------+ pthread_rwlock_init(3C), pthread_rwlock_unlock(3C), pthread_rwlockattr_init(3C), pthread_rwlock_rdlock(3C), attributes(5), standards(5) 23 Mar 2005 pthread_rwlock_wrlock(3C)
All times are GMT -4. The time now is 01:34 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy