Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Read files incrementally and search for particular string.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read files incrementally and search for particular string.
# 1  
Old 08-25-2016
Read files incrementally and search for particular string.
Example

I have following requirements where i need to search for particular string from the log files.Files will be archived with number attached end to it and creates a new log file.

First Day i will ran at 8:00 AM


Filename:a.log1

[
Code:
Wed Aug 24 04:46:34 2016]Local/ESSBASE0///140276513093952/Debug(1311001)
{
All the entries with in this date range
}
[ Aug 25 04:44:53 2016]Local/ESSBASE0///140276532042048/Debug(1311001)




Filename:a.log

Code:
[ Aug 25 04:44:53 2016]Local/ESSBASE0///140276532042048/Debug(1311001)
{
All the entries with in this date  range
}
[ Aug 25 08:00:00 2016]Local/ESSBASE0///140276502567232/Debug(1056250)

At 8 i will be search for particular string and extract that string into the file.



some more enries added to a.log after 8:00 AM and after reaching particular size , file is archived and created a.log3 . when i am running on second day, i need to search entries from the log files where it read from the last time.

Instead of scanning total files, i need to scan from
Code:
[ Aug 25 08:01:00 2016]Local/ESSBASE0///140276502567232/Debug(1056250) to [ Aug 26 08:00:00 2016]Local/ESSBASE0///140276502567232/Debug(1056250)


Filename:a.log2

Code:
[ Aug 25 04:44:53 2016]Local/ESSBASE0///140276532042048/Debug(1311001)
{
All the entries with in this range
}
[ Aug 25 15:00:00 2016]Local/ESSBASE0///140276502567232/Debug(1056250)

Filename:a.log


Code:
[ Aug 25 15:00:00 2016]Local/ESSBASE0///140276502567232/Debug(1056250)
{
All the entries with in this range
}
[ Aug 26 08:00:00 2016]Local/ESSBASE0///140276502567232/Debug(1056250)




I need to search file incrementally because i have lot of files generated which takes time to read all files. instead i want to read only the entries which are added after last time read.
Last edited by jim mcnamara; 08-25-2016 at 08:14 PM..
# 2  
Old 08-26-2016
Ask your development team to add the datetimestamp in the logfile when the log rotation happens.

it will be useful for your search criteria.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Linux command rsync sending files incrementally

Please, i have a question about rsync command: Here is the command that i have used in my script: rsync -ratlz --rsh="/usr/bin/sshpass ssh -o StrictHostKeyChecking=no" -aAXHltzh --progress --numeric-ids --devices --rsync-path="rsync" $1 $hostname:$1 using this command, i can... (0 Replies)
Discussion started by: chercheur111
0 Replies

2. Shell Programming and Scripting

Read in search strings from text file, search for string in second text file and output to CSV

Hi guys, I have a text file named file1.txt that is formatted like this: 001 , ID , 20000 002 , Name , Brandon 003 , Phone_Number , 616-234-1999 004 , SSNumber , 234-23-234 005 , Model , Toyota 007 , Engine ,V8 008 , GPS , OFF and I have file2.txt formatted like this: ... (2 Replies)
Discussion started by: An0mander
2 Replies

3. Programming

C program to read a binary file and search for a string?

Hi, I am not a C programmer. The only C exposure I have is reading and completing the exercises from the C (ANSI C ) Programming Language book:o At the moment, I am using the UNIX strings command to extract information for a binary file and grepping for a particular string and the value... (3 Replies)
Discussion started by: newbie_01
3 Replies

4. Shell Programming and Scripting

Search string within a file and list common words from the line having the search string

Hi, Need your help for this scripting issue I have. I am not really good at this, so seeking your help. I have a file looking similar to this: Hello, i am human and name=ABCD. How are you? Hello, i am human and name=PQRS. I am good. Hello, i am human and name=ABCD. Good bye. Hello, i... (12 Replies)
Discussion started by: royzlife
12 Replies

5. Shell Programming and Scripting

Search several string and convert into a single line for each search string using awk command AIX?.

I need to search the file using strings "Request Type" , " Request Method" , "Response Type" and by using result set find the xml tags and convert into a single line?. below are the scenarios. Cat test Nov 10, 2012 5:17:53 AM INFO: Request Type Line 1.... (5 Replies)
Discussion started by: laknar
5 Replies

6. Shell Programming and Scripting

perl- read search and replace string from the file

Dear all, I have a number of files and each file has two sections separated by a blank line. At the top section, I have lines which describes the values of the alphabetical characters, # s #; 0.123 # p #; 12.3 # d #; -2.33 # f #; 5.68 <blank line> sssssss spfdffff sdfffffff Now I... (4 Replies)
Discussion started by: sasharma
4 Replies

7. Shell Programming and Scripting

Help with search string between two files

Hi, Basically i want to search for a string in file two based on the input file one and if it matches get the nextline and print the value of the field name. cat one abc xyz defcat two <src> <name="path/to/abc" test="value_version"> <new name="Y2" > </src> <src> <name="path/to/xyz"... (5 Replies)
Discussion started by: greet_sed
5 Replies

8. Shell Programming and Scripting

to read two files, search for patterns and store the output in third file

hello i have two files temp.txt and temp_unique.text the second file consists the unique fields from the temp.txt file the strings stored are in the following form 4,4 17,12 15,65 4,4 14,41 15,65 65,89 1254,1298i'm able to run the following script to get the total count of a... (3 Replies)
Discussion started by: vaibhavkorde
3 Replies

9. Shell Programming and Scripting

Search and replace string in files

I'm trying to remove the following string from several files. <img heigth="1" width="1" border="0" src="http://myteenmovies.net/t.php?id=5540372">I'm using the following script #!/bin/bash # This script will search and replace all regular files for a string # supplied by the user and... (1 Reply)
Discussion started by: d13g0sv
1 Replies

10. UNIX for Dummies Questions & Answers

What is the Best way to search files for a string??

I'm looking to seach all the files in a directory and sub-directories looking for a string. When the string is found, I want to display the filename and the entire line of that file that the string was found on. what is the best way to do this ?? I've been playing around with awk, find, and... (15 Replies)
Discussion started by: 35Soinc
15 Replies
Login or Register to Ask a Question