Possible performance improvement (Bash and flat file) Post: 302419485

Sponsored Content

Special Forums UNIX and Linux Applications Infrastructure Monitoring Possible performance improvement (Bash and flat file) Post 302419485 by prafulnama on Friday 7th of May 2010 10:05:34 AM

05-07-2010

Registered User

Possible performance improvement (Bash and flat file)

Hello,

I am pretty new to shell scripts and I recently wrote one that seems to do what it should but I am exploring the possibility of improving its performance and would appreciate some help. Here is what it does - Its meant to monitor a bunch of systems (reads in IPs one at a time from a flat file). For each IP, it fetches a set of web pages, parses them to extract certain numbers, compares them against defined thresholds and alerts if the metric falls outside the threshold range. The catch is for certain metrics, it requires the last 5 values that it observed so I store those in a flat file and every time a new value is retrieved from the web page, that along with the stored values are used to compare against the threshold. Basically, I am doing everything sequentially so 2 loops, one to read in the IP and the next to do the web page download, threshold check, etc. Every time a new IP is added or a new metric needs to be monitored, the time taken to loop back to a machine increases. I wanted to see if there was a way to improve this? Intuitively, I feel, because all historical values are stored in a single flat file, something like multi processing would not work since, a process would have that file locked. Any ideas?????

Thanks,
-p

prafulnama

View Public Profile for prafulnama

Find all posts by prafulnama

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with Flat Files Please!! BASH (New User)

Hello All, I am brand new to the UNIX world and so far and very intrigued and enjoy scripting. This is just a new language for me. I would really like assistance with the below request. Any help would be greatly appreciated! I want to create a flat file in Vi that has a header field and...

2. Programming

File - reading - Performance improvement

Hi All I am reading a huge file of size 2GB atleast. I am reading each line and cutting certain columns and writing it to another file. Here is the logic. int main() { string u_line; string Char_List; string u_file; int line_pos; string temp_form_u_file; ...

3. Shell Programming and Scripting

Any improvement possible in this script

Hi! Thank you for the help yesterday This is the finished product There is one more thing I would like to do to it but I’m not to certain On how to proceed I would like to log all output to a log in order to Be able to roll back This script is meant to be used in repairing a...

4. UNIX for Advanced & Expert Users

linux os improvement

can anyone help to share the knowledge on linux os improvement? 1) os account - use window AD authentication, such as ldap, but how to set /etc/passwd, where to put user home? 2) user account activity - how to log os user activity share the idea and what tools can do that...thx

5. Shell Programming and Scripting

Display-performance in terminal, bash or python?

Heyas I've been working on my project TUI (Text User Interface) for quite some time now, its a hobby project, so nothing i sit in front of 8hrs/day. Since the only 'real' programming language i knw is Visual Basic, based upon early steps with MS-Batch files. When i 'joined' linux 3 years ago,...

6. Shell Programming and Scripting

[BASH] Performance question - Script to STDOUT

Hello Coders Some time ago i was asking about python and bash performances, and i was told i could post the regarding code, and someone would kindly help to make it faster (if possible). If you have noted, i'm on the way to finalize, finish, stable TUI - Text(ual) User Interface. It is a...

7. Shell Programming and Scripting

Performance improvement in grep

Below script is used to search numeric data from around 400 files in a folder. I have 300 such folders. Need help in performance improvement in the script. Below Script searches 20 such folders ( 300 files in each folder) simultaneously. This increases cpu utilization upto 90% What changes...

8. Shell Programming and Scripting

Bash - array loop performance

Hi, another little question... "sn" is an array whose elements can vary from about 55,000 to about 150,000 elements. Each element consists of an integer between 0-255, eg: ${sn} contain the value: 103 . For a decrypt-procedure I need scroll all the elements 4 or 5 times. Here is an example of...

9. OS X (Apple)

Create a bash array from a flat file of whitespaces only.

Hi guys and gals... MacBook Pro. OSX 10.13.2, default bash terminal. I have a flat file 1920 bytes in size of whitespaces only. I need to put every single whitespace character into a bash array cell. Below are two methods that work, but both are seriously ugly. The first one requires that I...

10. Shell Programming and Scripting

Bash script search, improve performance with large files

Hello, For several of our scripts we are using awk to search patterns in files with data from other files. This works almost perfectly except that it takes ages to run on larger files. I am wondering if there is a way to speed up this process or have something else that is quicker with the...

LEARN ABOUT DEBIAN

xgetpointercontrol

XChangePointerControl(3)					  XLIB FUNCTIONS					  XChangePointerControl(3)

NAME

       XChangePointerControl, XGetPointerControl - control pointer

SYNTAX

       int XChangePointerControl(Display *display, Bool do_accel, Bool do_threshold, int accel_numerator, int accel_denominator, int threshold);

       int XGetPointerControl(Display *display, int *accel_numerator_return, int *accel_denominator_return, int *threshold_return);

ARGUMENTS

       accel_denominator
		 Specifies the denominator for the acceleration multiplier.

       accel_denominator_return
		 Returns the denominator for the acceleration multiplier.

       accel_numerator
		 Specifies the numerator for the acceleration multiplier.

       accel_numerator_return
		 Returns the numerator for the acceleration multiplier.

       display	 Specifies the connection to the X server.

       do_accel  Specifies a Boolean value that controls whether the values for the accel_numerator or accel_denominator are used.

       do_threshold
		 Specifies a Boolean value that controls whether the value for the threshold is used.

       threshold Specifies the acceleration threshold.

       threshold_return
		 Returns the acceleration threshold.

DESCRIPTION

       The XChangePointerControl function defines how the pointing device moves.  The acceleration, expressed as a fraction, is a multiplier for
       movement.  For example, specifying 3/1 means the pointer moves three times as fast as normal.  The fraction may be rounded arbitrarily by
       the X server.  Acceleration only takes effect if the pointer moves more than threshold pixels at once and only applies to the amount beyond
       the value in the threshold argument.  Setting a value to -1 restores the default.  The values of the do_accel and do_threshold arguments
       must be True for the pointer values to be set, or the parameters are unchanged.	Negative values (other than -1) generate a BadValue error,
       as does a zero value for the accel_denominator argument.

       XChangePointerControl can generate a BadValue error.

       The XGetPointerControl function returns the pointer's current acceleration multiplier and acceleration threshold.

DIAGNOSTICS

       BadValue  Some numeric value falls outside the range of values accepted by the request.	Unless a specific range is specified for an argu-
		 ment, the full range defined by the argument's type is accepted.  Any argument defined as a set of alternatives can generate this
		 error.

SEE ALSO

       Xlib - C Language X Interface

X Version 11							   libX11 1.5.0 					  XChangePointerControl(3)