Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Read files incrementally and search for particular string.
Top Forums Shell Programming and Scripting Read files incrementally and search for particular string. Post 302980280 by nareshnani211 on Thursday 25th of August 2016 04:44:03 PM
Old 08-25-2016
Read files incrementally and search for particular string.
Example

I have following requirements where i need to search for particular string from the log files.Files will be archived with number attached end to it and creates a new log file.

First Day i will ran at 8:00 AM


Filename:a.log1

[
Code:
Wed Aug 24 04:46:34 2016]Local/ESSBASE0///140276513093952/Debug(1311001)
{
All the entries with in this date range
}
[ Aug 25 04:44:53 2016]Local/ESSBASE0///140276532042048/Debug(1311001)




Filename:a.log

Code:
[ Aug 25 04:44:53 2016]Local/ESSBASE0///140276532042048/Debug(1311001)
{
All the entries with in this date  range
}
[ Aug 25 08:00:00 2016]Local/ESSBASE0///140276502567232/Debug(1056250)

At 8 i will be search for particular string and extract that string into the file.



some more enries added to a.log after 8:00 AM and after reaching particular size , file is archived and created a.log3 . when i am running on second day, i need to search entries from the log files where it read from the last time.

Instead of scanning total files, i need to scan from
Code:
[ Aug 25 08:01:00 2016]Local/ESSBASE0///140276502567232/Debug(1056250) to [ Aug 26 08:00:00 2016]Local/ESSBASE0///140276502567232/Debug(1056250)


Filename:a.log2

Code:
[ Aug 25 04:44:53 2016]Local/ESSBASE0///140276532042048/Debug(1311001)
{
All the entries with in this range
}
[ Aug 25 15:00:00 2016]Local/ESSBASE0///140276502567232/Debug(1056250)

Filename:a.log


Code:
[ Aug 25 15:00:00 2016]Local/ESSBASE0///140276502567232/Debug(1056250)
{
All the entries with in this range
}
[ Aug 26 08:00:00 2016]Local/ESSBASE0///140276502567232/Debug(1056250)




I need to search file incrementally because i have lot of files generated which takes time to read all files. instead i want to read only the entries which are added after last time read.
Last edited by jim mcnamara; 08-25-2016 at 08:14 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

What is the Best way to search files for a string??

I'm looking to seach all the files in a directory and sub-directories looking for a string. When the string is found, I want to display the filename and the entire line of that file that the string was found on. what is the best way to do this ?? I've been playing around with awk, find, and... (15 Replies)
Discussion started by: 35Soinc
15 Replies

2. Shell Programming and Scripting

Search and replace string in files

I'm trying to remove the following string from several files. <img heigth="1" width="1" border="0" src="http://myteenmovies.net/t.php?id=5540372">I'm using the following script #!/bin/bash # This script will search and replace all regular files for a string # supplied by the user and... (1 Reply)
Discussion started by: d13g0sv
1 Replies

3. Shell Programming and Scripting

to read two files, search for patterns and store the output in third file

hello i have two files temp.txt and temp_unique.text the second file consists the unique fields from the temp.txt file the strings stored are in the following form 4,4 17,12 15,65 4,4 14,41 15,65 65,89 1254,1298i'm able to run the following script to get the total count of a... (3 Replies)
Discussion started by: vaibhavkorde
3 Replies

4. Shell Programming and Scripting

Help with search string between two files

Hi, Basically i want to search for a string in file two based on the input file one and if it matches get the nextline and print the value of the field name. cat one abc xyz defcat two <src> <name="path/to/abc" test="value_version"> <new name="Y2" > </src> <src> <name="path/to/xyz"... (5 Replies)
Discussion started by: greet_sed
5 Replies

5. Shell Programming and Scripting

perl- read search and replace string from the file

Dear all, I have a number of files and each file has two sections separated by a blank line. At the top section, I have lines which describes the values of the alphabetical characters, # s #; 0.123 # p #; 12.3 # d #; -2.33 # f #; 5.68 <blank line> sssssss spfdffff sdfffffff Now I... (4 Replies)
Discussion started by: sasharma
4 Replies

6. Shell Programming and Scripting

Search several string and convert into a single line for each search string using awk command AIX?.

I need to search the file using strings "Request Type" , " Request Method" , "Response Type" and by using result set find the xml tags and convert into a single line?. below are the scenarios. Cat test Nov 10, 2012 5:17:53 AM INFO: Request Type Line 1.... (5 Replies)
Discussion started by: laknar
5 Replies

7. Shell Programming and Scripting

Search string within a file and list common words from the line having the search string

Hi, Need your help for this scripting issue I have. I am not really good at this, so seeking your help. I have a file looking similar to this: Hello, i am human and name=ABCD. How are you? Hello, i am human and name=PQRS. I am good. Hello, i am human and name=ABCD. Good bye. Hello, i... (12 Replies)
Discussion started by: royzlife
12 Replies

8. Programming

C program to read a binary file and search for a string?

Hi, I am not a C programmer. The only C exposure I have is reading and completing the exercises from the C (ANSI C ) Programming Language book:o At the moment, I am using the UNIX strings command to extract information for a binary file and grepping for a particular string and the value... (3 Replies)
Discussion started by: newbie_01
3 Replies

9. Shell Programming and Scripting

Read in search strings from text file, search for string in second text file and output to CSV

Hi guys, I have a text file named file1.txt that is formatted like this: 001 , ID , 20000 002 , Name , Brandon 003 , Phone_Number , 616-234-1999 004 , SSNumber , 234-23-234 005 , Model , Toyota 007 , Engine ,V8 008 , GPS , OFF and I have file2.txt formatted like this: ... (2 Replies)
Discussion started by: An0mander
2 Replies

10. Shell Programming and Scripting

Linux command rsync sending files incrementally

Please, i have a question about rsync command: Here is the command that i have used in my script: rsync -ratlz --rsh="/usr/bin/sshpass ssh -o StrictHostKeyChecking=no" -aAXHltzh --progress --numeric-ids --devices --rsync-path="rsync" $1 $hostname:$1 using this command, i can... (0 Replies)
Discussion started by: chercheur111
0 Replies

LEARN ABOUT FREEBSD

iprop-log

IPROP-LOG(8)						    BSD System Manager's Manual 					      IPROP-LOG(8)

NAME

     iprop-log -- maintain the iprop log file

SYNOPSIS

     iprop-log [--version] [-h | --help] command

     iprop-log truncate [-c file | --config-file=file] [-r string | --realm=string] [-h | --help]

     iprop-log dump [-c file | --config-file=file] [-r string | --realm=string] [-h | --help]

     iprop-log replay [--start-version=version-number] [--end-version=version-number] [-c file | --config-file=file] [-r string | --realm=string]
	       [-h | --help]

DESCRIPTION

     Supported options:

     --version

     -h, --help

     command can be one of the following:

     truncate

	       -c file, --config-file=file
		       configuration file

	       -r string, --realm=string
		       realm

	       Truncates the log. Sets the new logs version number for the to the last entry of the old log.  If the log is truncted by emptying
	       the file, the log will start over at the first version (0).

     dump

	       -c file, --config-file=file
		       configuration file

	       -r string, --realm=string
		       realm

	       Print out all entries in the log to standard output.

     replay

	       --start-version=version-number
		       start replay with this version

	       --end-version=version-number
		       end replay with this version

	       -c file, --config-file=file
		       configuration file

	       -r string, --realm=string
		       realm

	       Replay the changes from specified entries (or all if none is specified) in the transaction log to the database.

     last-version

	       -c file, --config-file=file
		       configuration file

	       -r string, --realm=string
		       realm

	       prints the version of the last log entry.

SEE ALSO

     iprop(8)

Heimdal 							 February 18, 2007							   Heimdal
All times are GMT -4. The time now is 10:08 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy