Sponsored Content
Operating Systems Linux Searching for gaps in huge (2.2G) log file? Post 95469 by deckard on Wednesday 11th of January 2006 12:52:43 PM
Old 01-11-2006
OK... grumble grumble...

Here's what I wound up doing:

Run a grep for the HH:MM:AM/PM string for each specific hour in 24 hours and pipe it to a file for just that hour. Then I have a better shot of trying to scroll through the logs to find any missing timestamps. Interesting to see our network usage starts going up at 9:00AM when our doors open, peaks in the afternoon when the public computers are all in use (200+ meg logs per hour between 12:00PM and 5:00PM) and then slowly drops off until we close at 9:00PM.
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

help searching log file with dates

Im tyring to create a script that will show me any lines in a file with todays date and yesterdays, the date format in the file is as follows ----- amqxfdcx.c : 728 -------------------------------------------------------- 07/12/05 09:53:20 AMQ6109: An internal WebSphere MQ error has... (3 Replies)
Discussion started by: csaunders
3 Replies

2. Shell Programming and Scripting

searching a log file and appending to a .txt file

I'm new to shell scripting and am writing a script to help me log the free memory and hd space on a server. As of now, the script just runs 'df -h' and appends the output to a file and then runs 'top' and appends the output to a log file. What I want to do, is have the script also search the... (3 Replies)
Discussion started by: enator45
3 Replies

3. Shell Programming and Scripting

Searching for a string in a log file with little movement

I have a script which tails a log file and if it finds certain strings in the data tailed it sends an email, basically like this: tail -f logfile > tmp.file & sleep 10 kill $! STRING=$(grep -c "string" tmp.file) && echo $STRING | mailx -s "Warning.." admin@123.com When the string is... (10 Replies)
Discussion started by: Moxy
10 Replies

4. Shell Programming and Scripting

Searching for Gaps in Time

I am very new to shell scripting. We use C-Shell here and I know the issues that surround it. I hope a solution can be created using awk, sed, etc... instead of having to write a program. I have an input file that is sorted by date and time in ascending order ... (2 Replies)
Discussion started by: jclanc8
2 Replies

5. Shell Programming and Scripting

Searching for Log / Bad file and Reading and writing to a flat file

Need to develop a unix shell script for the below requirement and I need your assistance: 1) search for file.log and file.bad file in a directory and read them 2) pull out "Load_Start_Time", "Data_File_Name", "Error_Type" from log file 4) concatinate each row from bad file as... (3 Replies)
Discussion started by: mlpathir
3 Replies

6. Shell Programming and Scripting

Sorting and moving file sequence with gaps

Hello, I have lots of sequentially numbered files which make up an image sequence. I'm trying to do two things with it: #1: Find gaps in the sequence and move each range of sequencial files into their own subfolder. #2: Designate a starting point (file) and move every 24th file into... (4 Replies)
Discussion started by: ex_H
4 Replies

7. Shell Programming and Scripting

searching a file with a specified text without using conventional file searching commands

without using conventional file searching commands like find etc, is it possible to locate a file if i just know that the file that i'm searching for contains a particular text like "Hello world" or something? (5 Replies)
Discussion started by: arindamlive
5 Replies

8. Shell Programming and Scripting

Getting info from a huge log file

Hello everyone. I am having problem with parsing a data from the huge log file. the log file is an application log with around 5 Gb in size and it rotates every midnight. Now if the application encountered such issue, it sends an email with a specific info but without further details. So I... (13 Replies)
Discussion started by: cwiggler
13 Replies

9. UNIX for Dummies Questions & Answers

Gaps and frequencies

I have this infile: >GHL8OVD01BNNCA Freq 10 TAGATGTGCCCGTGGGTTTCCCGTCAACACCGGATAGT-GCAGCA-TA >GHL8OVD01CMQVT Freq 1 TTGATGTCGTGGGTTTCCCGTCAACACCGGCAAATAGT-GCAGCA-TA >GHL8OVD01CMQVT Freq 1 TTGATGTGCCAGTTTCCCGTCTAGCAGCACTACCAGGACCTTCGC-TA >GHL8OVD01CMQVW Freq 1... (1 Reply)
Discussion started by: Xterra
1 Replies
rotatelogs(8)						      System Manager's Manual						     rotatelogs(8)

NAME
rotatelogs - rotate Apache logs without having to kill the server SYNOPSIS
rotatelogs logfile rotationtime [offset] DESCRIPTION
rotatelogs is a simple program for use in conjunction with Apache's piped logfile feature, which can be used like this: CustomLog "|bin/rotatelogs /var/logs/logfile 86400" common This creates the files /var/logs/logfile.nnnn where nnnn is the system time at which the log nominally starts (this time will always be a multiple of the rotation time, so you can synchronize cron scripts with it). At the end of each rotation time (here after 24 hours) a new log is started. OPTIONS
logfile The path plus basename of the logfile. If logfile includes any Otherwise, the suffix .nnnnnnnnnn is automatically added and is the time in seconds. Both formats compute the start time from the beginning of the current period. rotationtime The rotation time in seconds. offset The number of minutes offset from UTC. If omitted, zero is assumed and UTC is used. For example, to use local time in the zone UTC -5 hours, specify a value of -300 for this argument. PORTABILITY
The following logfile format string substitutions should be supported by all strftime(3) implementations, see the strftime(3) man page for library-specific extensions. %A full weekday name (localized) %a 3-character weekday name (localized) %B full month name (localized) %b 3-character month name (localized) %c date and time (localized) %d 2-digit day of month %H 2-digit hour (24 hour clock) %I 2-digit hour (12 hour clock) %j 3-digit day of year %M 2-digit minute %m 2-digit month %p am/pm of 12 hour clock (localized) %S 2-digit second %U 2-digit week of year (Sunday first day of week) %W 2-digit week of year (Monday first day of week) %w 1-digit weekday (Sunday first day of week) %X time (localized) %x date (localized) %Y 4-digit year %y 2-digit year %Z time zone name %% literal `%' SEE ALSO
httpd(8) March 2001 rotatelogs(8)
All times are GMT -4. The time now is 03:16 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy