Sponsored Content
Special Forums UNIX and Linux Applications Infrastructure Monitoring Possible performance improvement (Bash and flat file) Post 302420433 by dunkar70 on Tuesday 11th of May 2010 03:17:18 PM
Old 05-11-2010
You can also keep the historical data manageable by tailing the file. Log all values into a single file, such as history.log
At the beginning of the log file processing, execute:
Code:
tail history.log > fileToProcess.log

This will give you a smaller file from which to get your historical data. The size of your history.log file will not matter, your processing file will always contain the last 10 entries.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with Flat Files Please!! BASH (New User)

Hello All, I am brand new to the UNIX world and so far and very intrigued and enjoy scripting. This is just a new language for me. I would really like assistance with the below request. Any help would be greatly appreciated! I want to create a flat file in Vi that has a header field and... (0 Replies)
Discussion started by: cyberjax21
0 Replies

2. Programming

File - reading - Performance improvement

Hi All I am reading a huge file of size 2GB atleast. I am reading each line and cutting certain columns and writing it to another file. Here is the logic. int main() { string u_line; string Char_List; string u_file; int line_pos; string temp_form_u_file; ... (10 Replies)
Discussion started by: dhanamurthy
10 Replies

3. Shell Programming and Scripting

Any improvement possible in this script

Hi! Thank you for the help yesterday This is the finished product There is one more thing I would like to do to it but I’m not to certain On how to proceed I would like to log all output to a log in order to Be able to roll back This script is meant to be used in repairing a... (4 Replies)
Discussion started by: Ex-Capsa
4 Replies

4. UNIX for Advanced & Expert Users

linux os improvement

can anyone help to share the knowledge on linux os improvement? 1) os account - use window AD authentication, such as ldap, but how to set /etc/passwd, where to put user home? 2) user account activity - how to log os user activity share the idea and what tools can do that...thx (5 Replies)
Discussion started by: goodbid
5 Replies

5. Shell Programming and Scripting

Display-performance in terminal, bash or python?

Heyas I've been working on my project TUI (Text User Interface) for quite some time now, its a hobby project, so nothing i sit in front of 8hrs/day. Since the only 'real' programming language i knw is Visual Basic, based upon early steps with MS-Batch files. When i 'joined' linux 3 years ago,... (7 Replies)
Discussion started by: sea
7 Replies

6. Shell Programming and Scripting

[BASH] Performance question - Script to STDOUT

Hello Coders Some time ago i was asking about python and bash performances, and i was told i could post the regarding code, and someone would kindly help to make it faster (if possible). If you have noted, i'm on the way to finalize, finish, stable TUI - Text(ual) User Interface. It is a... (6 Replies)
Discussion started by: sea
6 Replies

7. Shell Programming and Scripting

Performance improvement in grep

Below script is used to search numeric data from around 400 files in a folder. I have 300 such folders. Need help in performance improvement in the script. Below Script searches 20 such folders ( 300 files in each folder) simultaneously. This increases cpu utilization upto 90% What changes... (3 Replies)
Discussion started by: vegasluxor
3 Replies

8. Shell Programming and Scripting

Bash - array loop performance

Hi, another little question... "sn" is an array whose elements can vary from about 55,000 to about 150,000 elements. Each element consists of an integer between 0-255, eg: ${sn} contain the value: 103 . For a decrypt-procedure I need scroll all the elements 4 or 5 times. Here is an example of... (15 Replies)
Discussion started by: math4
15 Replies

9. OS X (Apple)

Create a bash array from a flat file of whitespaces only.

Hi guys and gals... MacBook Pro. OSX 10.13.2, default bash terminal. I have a flat file 1920 bytes in size of whitespaces only. I need to put every single whitespace character into a bash array cell. Below are two methods that work, but both are seriously ugly. The first one requires that I... (7 Replies)
Discussion started by: wisecracker
7 Replies

10. Shell Programming and Scripting

Bash script search, improve performance with large files

Hello, For several of our scripts we are using awk to search patterns in files with data from other files. This works almost perfectly except that it takes ages to run on larger files. I am wondering if there is a way to speed up this process or have something else that is quicker with the... (15 Replies)
Discussion started by: SDohmen
15 Replies
Log::Any::Test(3pm)					User Contributed Perl Documentation				       Log::Any::Test(3pm)

NAME
Log::Any::Test -- Test what you're logging with Log::Any SYNOPSIS
use Test::More; use Log::Any::Test; # should appear before 'use Log::Any'! use Log::Any qw($log); # ... # call something that logs using Log::Any # ... # now test to make sure you logged the right things $log->contains_ok(qr/good log message/, "good message was logged"); $log->does_not_contain_ok(qr/unexpected log message/, "unexpected message was not logged"); $log->empty_ok("no more logs"); # or my $msgs = $log->msgs; cmp_deeply($msgs, [{message => 'msg1', level => 'debug'}, ...]); DESCRIPTION
"Log::Any::Test" is a simple module that allows you to test what has been logged with Log::Any. Most of its API and implementation have been taken from Log::Any::Dispatch. Using "Log::Any::Test" sends all subsequent Log::Any log messages to a single global in-memory buffer. It should be used before Log::Any. METHODS
The test_name is optional in the *_ok methods; a reasonable default will be provided. msgs () Returns the current contents of the global log buffer as an array reference, where each element is a hash containing a category, level, and message key. e.g. { category => 'Foo', level => 'error', message => 'this is an error' }, { category => 'Bar::Baz', level => 'debug', message => 'this is a debug' } contains_ok ($regex[, $test_name]) Tests that a message in the log buffer matches $regex. On success, the message is removed from the log buffer (but any other matches are left untouched). does_not_contain_ok ($regex[, $test_name]) Tests that no message in the log buffer matches $regex. empty_ok ([$test_name]) Tests that there is no log buffer left. On failure, the log buffer is cleared to limit further cascading failures. contains_only_ok ($regex[, $test_name]) Tests that there is a single message in the log buffer and it matches $regex. On success, the message is removed. clear () Clears the log buffer. SEE ALSO
Log::Any, Test::Log::Dispatch AUTHOR
Jonathan Swartz COPYRIGHT &; LICENSE Copyright (C) 2009 Jonathan Swartz, all rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.10.1 2009-12-08 Log::Any::Test(3pm)
All times are GMT -4. The time now is 06:56 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy