File - reading - Performance improvement Post: 302198431

Sponsored Content

Top Forums Programming File - reading - Performance improvement Post 302198431 by aamirglb on Friday 23rd of May 2008 01:24:49 AM

05-23-2008

Registered User

Hello!
What you can try out is: have a huge circular buffer for example say around 6144 (6KB) , you can experiment with the size!!
What i mean by circular is have two pointers, start_ptr and processed_ptr.

Code:

offset = 0;
read(fd, &buffer[offset], 3KB);
if(offset == 0)
{
    // next time read in the next chunk of buffer
    offset = 3KB;
    start_ptr = 0;
}
else
{    
    offset = 0;
    start_ptr = 3KB; 
}

bytes_read = start_ptr - processed_ptr;

//start processing it
while(bytes_read >= minimum_size_of_record)
{
     ret_val = check_for_complete_record(processed_ptr);
    // incomplete record
     if(ret_val == -1)
     {
             // don't modify processed_ptr since the record is not complete
             break; //without modifying the pointers 
     }
     else
     {
          // in this case check_for_complete_record will return the size of record
          bytes_read = bytes_read - ret_val;
          processed_ptr = processed_ptr + ret_val;
     }      
}

1) Have start_ptr and processed_ptr as global
2) You must take care of rollover of processed_ptr for every read

Code:

     if(processed_ptr >= MAX_BUFFER_SIZE) // in this case 6KB
               processed_ptr = 0;

Regards,
Aamir

aamirglb

View Public Profile for aamirglb

Find all posts by aamirglb

7 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script ready but might need some improvement.

Hi All, I have written a script which does some editing in the files, based on user input.This might not be the most elegant way of doing it and there would be many improvements needed. Please go through it and let me know how it could be improved. Suggestions are welcome!! Thanks!...

2. Shell Programming and Scripting

Any improvement possible in this script

Hi! Thank you for the help yesterday This is the finished product There is one more thing I would like to do to it but I’m not to certain On how to proceed I would like to log all output to a log in order to Be able to roll back This script is meant to be used in repairing a...

3. Infrastructure Monitoring

Possible performance improvement (Bash and flat file)

Hello, I am pretty new to shell scripts and I recently wrote one that seems to do what it should but I am exploring the possibility of improving its performance and would appreciate some help. Here is what it does - Its meant to monitor a bunch of systems (reads in IPs one at a time from a flat...

4. UNIX for Advanced & Expert Users

linux os improvement

can anyone help to share the knowledge on linux os improvement? 1) os account - use window AD authentication, such as ldap, but how to set /etc/passwd, where to put user home? 2) user account activity - how to log os user activity share the idea and what tools can do that...thx

5. Shell Programming and Scripting

I need the improvement for my script

Hi All, Here is my script #! /bin/sh var1=some email id var2=some email id grep -i "FAILED FILE FORMAT VALIDATION" /opt >tmp2 diff tmp1 tmp2 | grep ">" >tmp3 if then cat tmp3 | mailx -s " Error Monitoring" $var2 else echo "Pattern NOt Found" | mailx -s " Error Monitoring" $var1...

6. UNIX for Dummies Questions & Answers

Improvement in shell script

Hi This is my Following code: #!/bin/sh echo "TOTAL_NO_OF_MAILS" read TOTAL_NO_OF_MAILS echo "TOTAL_NO_OF_TICKETS " read TOTAL_NO_OF_TICKETS echo "TICKETS_IN_QUEUE" read TICKETS_IN_QUEUE rm -rf `pwd`/Focus echo "Hi Team\nSTATS IN CLRS MAIL BOX\n\n==============================" >> Focus...

7. Shell Programming and Scripting

Performance improvement in grep

Below script is used to search numeric data from around 400 files in a folder. I have 300 such folders. Need help in performance improvement in the script. Below Script searches 20 such folders ( 300 files in each folder) simultaneously. This increases cpu utilization upto 90% What changes...

LEARN ABOUT REDHAT

krb5_appdefault

KRB5_APPDEFAULT(3)					   BSD Library Functions Manual 					KRB5_APPDEFAULT(3)

NAME

     krb5_appdefault_boolean, krb5_appdefault_string, krb5_appdefault_time -- get application configuration value

LIBRARY

     Kerberos 5 Library (libkrb5, -lkrb5)

SYNOPSIS

     #include <krb5.h>

     void
     krb5_appdefault_boolean(krb5_context context, const char *appname, krb5_realm realm, const char *option, krb5_boolean def_val,
	 krb5_boolean *ret_val);

     void
     krb5_appdefault_string(krb5_context context, const char *appname, krb5_realm realm, const char *option, const char *def_val, char **ret_val);

     void
     krb5_appdefault_time(krb5_context context, const char *appname, krb5_realm realm, const char *option, time_t def_val, time_t *ret_val);

DESCRIPTION

     These functions get application defaults from the appdefaults section of the krb5.conf(5) configuration file. These defaults can be specified
     per application, and/or per realm.

     These values will be looked for in krb5.conf(5), in order of descending importance.

	   [appdefaults]
		   appname = {
			   realm = {
				   option = value
			   }
		   }
		   appname = {
			   option = value
		   }
		   realm = {
			   option = value
		   }
		   option = value
     appname is the name of the application, and realm is the realm name. If the realm is omitted it will not be used for resolving values.
     def_val is the value to return if no value is found in krb5.conf(5).

SEE ALSO

     krb5_config(3), krb5.conf(5)

HEIMDAL 
							   July 25, 2000							   HEIMDAL