Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Best approach for a 10 min extract out of several log files with timestamped records Post 89764 by Perderabo on Tuesday 15th of November 2005 04:49:46 PM
Old 11-15-2005
With only 10 minutes to look for, you could put all 10 timestamps in one grep statement. To reverse the lines, prepend line numbers, sort in reverse order on the line numbers, and remove the line numbers. This script uses datacalc which you can find on this site.

Code:
#! /usr/bin/ksh

date "+%Y %m %d %H %M" | read year month day hour minute
pattern=""
count=10
while((count)) ; do
        now="${year}-${month}-${day} ${hour}:${minute}"
        ((count=count-1))
        if ((minute)) ; then
                ((minute=minute-1))
        else
                minute=59
                if ((hour)) ; then
                        ((hour=hour-1))
                else
                        hour=23
                        datecalc -a $year $month $day - 1 | read year month day
                fi
        fi
        [[ -n $pattern ]] && pattern="${pattern}|"
        pattern="${pattern}${now}"
done
pattern=$(echo $pattern | tr "|" "\n")

grep -F "$pattern" | nl -ba | sort -nr | cut -f2-
exit 0

 

10 More Discussions You Might Find Interesting

1. AIX

need to extract info from log files

hi guys i need to extract information from log files generated by an application. log file has the following lines for each process.. ---------------------------------------------- Fri Aug 03 12:06:43 WST 2007 INFO: Running project PROJECT1 Fri Aug 03 12:06:43 WST 2007 INFO: Source Files... (7 Replies)
Discussion started by: kirantalla
7 Replies

2. Shell Programming and Scripting

Need to Extract Data From 94000 records

i have a input file which does not have a delimiter All i Need to do is to identify a line and extract the data from it and run the loop again and need to ensure that it was not extracted earlier Input file ------------ abcd 12345 egfhijk ip 192.168.0.1 CNN.com abcd 12345 egfhijk ip... (12 Replies)
Discussion started by: vasimm
12 Replies

3. Shell Programming and Scripting

Compare Records between to files and extract it

I am not an expert in awk, SED, etc... but I really hope there is a way to do this, because I don't want to have to right a program. I am using C shell. FILE 1 FILE 2 H0000000 H0000000 MA1 MA1 CA1DDDDDD CA1AAAAAA MA2 ... (2 Replies)
Discussion started by: jclanc8
2 Replies

4. Shell Programming and Scripting

Creating timestamped directories/files in expect

Hi, I am looking for the syntax to provide a timestamped log_file in expect. Basically I want the equivalent of: /outputs/`date +%d%m%y`/session`date +%H%M` But scripted in expect so it can be handed over to the logfile function. I have tried playing around with the timestamp... (1 Reply)
Discussion started by: krypton
1 Replies

5. Shell Programming and Scripting

Compare two files with different number of records and output only the Extra records from file1

Hi Freinds , I have 2 files . File 1 |nag|HYd|1|Che |esw|Gun|2|hyd |pra|bhe|3|hyd |omu|hei|4|bnsj |uer|oeri|5|uery File 2 |nag|HYd|1|Che |esw|Gun|2|hyd |uer|oi|3|uery output : (9 Replies)
Discussion started by: i150371485
9 Replies

6. Shell Programming and Scripting

Extract records from list

Hi Gents, I have a file 1 like this 1 1000 20 2 2000 30 3 1000 40 5 1000 50 And I have other file 1 like 2 1 I would like to get from the file 1 the complete line which are in file 2, the key to compare is the column 2 then output should be. 2 2000 30. I was trying to get it... (5 Replies)
Discussion started by: jiam912
5 Replies

7. Shell Programming and Scripting

Can you extract (remove) lines from log files?

I use "MineOS" (a linux distro with python scripts and web ui included for managing a Minecraft Server). The author of the scripts is currently having a problem with the Minecraft server log file being spammed with certain entries. He's working on clearing up the spam. But in the meantime, I'm... (8 Replies)
Discussion started by: nbsparks
8 Replies

8. Shell Programming and Scripting

Extract lines with min value, using two field separators.

I have a file with two ID columns followed by five columns of counts in fraction form. I'd like to print lines that have a count of at least 4 (so at least 4 in the numerator, e.g. 4/17) in at least one of the five columns. Input file: comp51820_c1_seq1 693 0/29 0/50 0/69 0/36 0/31... (6 Replies)
Discussion started by: pathunkathunk
6 Replies

9. Shell Programming and Scripting

How to extract part of string from all log files.?

Hi All, Let us say we have 5 log files, extract the data from all log files and save the output in a file. home/log/first.log home/log/second.log home/log/third.log home/log/four.log home/log/five.log I want to extract the following text from the log files and save the output in a file.... (7 Replies)
Discussion started by: ROCK_PLSQL
7 Replies

10. UNIX for Beginners Questions & Answers

Extract lines if string found from last 30 min only

Hi guys, Appreciate your help as I am stuck with searching the logs for last 30 minutes from the current time. Current time is time when you execute the script and it will search for <string> through the logs for last 30 minutes only and if <string> found then print those lines only. The... (18 Replies)
Discussion started by: rockstar
18 Replies
FAILLOG(8)						    System Management Commands							FAILLOG(8)

NAME
faillog - display faillog records or set login failure limits SYNOPSIS
faillog [options] DESCRIPTION
faillog displays the contents of the failure log database (/var/log/faillog). It can also set the failure counters and limits. When faillog is run without arguments, it only displays the faillog records of the users who had a login failure. OPTIONS
The options which apply to the faillog command are: -a, --all Display (or act on) faillog records for all users having an entry in the faillog database. The range of users can be restricted with the -u option. In display mode, this is still restricted to existing users but forces the display of the faillog entries even if they are empty. With the -l, -m, -r, -t options, the users' records are changed, even if the user does not exist on the system. This is useful to reset records of users that have been deleted or to set a policy in advance for a range of users. -h, --help Display help message and exit. -l, --lock-secs SEC Lock account for SEC seconds after failed login. Write access to /var/log/faillog is required for this option. -m, --maximum MAX Set the maximum number of login failures after the account is disabled to MAX. Selecting a MAX value of 0 has the effect of not placing a limit on the number of failed logins. The maximum failure count should always be 0 for root to prevent a denial of services attack against the system. Write access to /var/log/faillog is required for this option. -r, --reset Reset the counters of login failures. Write access to /var/log/faillog is required for this option. -R, --root CHROOT_DIR Apply changes in the CHROOT_DIR directory and use the configuration files from the CHROOT_DIR directory. -t, --time DAYS Display faillog records more recent than DAYS. -u, --user LOGIN|RANGE Display faillog record or maintains failure counters and limits (if used with -l, -m or -r options) only for the specified user(s). The users can be specified by a login name, a numerical user ID, or a RANGE of users. This RANGE of users can be specified with a min and max values (UID_MIN-UID_MAX), a max value (-UID_MAX), or a min value (UID_MIN-). When none of the -l, -m, or -r options are used, faillog displays the faillog record of the specified user(s). CAVEATS
faillog only prints out users with no successful login since the last failure. To print out a user who has had a successful login since their last failure, you must explicitly request the user with the -u flag, or print out all users with the -a flag. FILES
/var/log/faillog Failure logging file. SEE ALSO
login(1), faillog(5). shadow-utils 4.1.5.1 05/25/2012 FAILLOG(8)
All times are GMT -4. The time now is 09:28 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy