Sponsored Content
Top Forums UNIX for Dummies Questions & Answers How to scan only new lines added in file? Post 70420 by VENC22 on Thursday 28th of April 2005 02:57:18 PM
Old 04-28-2005
Searching thru only new lines add

Here is the trick I think (Moderators please advise).

Use diff.
Suppose u r having file1 and file2. file2 is new and u want to search for occurence of LOTUS and red in new lines added in file2.
U can do this:

diff file1 file2 >> diff12
egrep 'LOTUS|red' diff12

This will work for single file.
If u have many files...U will have to right a shell script..
It will hae somewhat following structure.. (I have written algorithm so it won't be syntactically correct)

DO WHILE END OF NUMBER OF FILES.
integer i
string name
name = 'difffile' & string(i) & 'diff' --> if i = 4 then name = difffile4.diff
diff folder1/file1 folder2/file2 >> file2/name.diff
LOOP

this will give u a set of files with diferences in set of log files with only new lines.
(I am assuming that existing entries in a log file won't get updated or changed. E.g. if file1 contains line 1 as 'REDANDGREEN' then file 2 will contain line1 as REDANDGREEN and not REDANDORANGE. If the existing entries are going to get updated then u may need to process the files with differences again to keep only new line differences. Try out DIFF and try to investigate more.)
U can then fire a grep on these files with differences.

Let me know ur comments..
Thank
VEN
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

File Scan

Hi everyone , i m working on Sun solaris and i have a file "smsapp.cur" which has information like this paragraph given below , there are millions of such paragraphs From:923212802736 To:923222326807 logMessage: 07-04-08 17:34:29 Getting message topup from code page default in language English... (2 Replies)
Discussion started by: Dastard
2 Replies

2. Shell Programming and Scripting

File Inventory Scan

Is there such a script out there that will Analyze a folder and subfolders, getting file type utilization and file aging. A CSV file will be created with file information File,Size,Date Created,Last Modified,Last Accessed,Extension,File Type,Owner A Summary report text file will also be created... (4 Replies)
Discussion started by: seacros
4 Replies

3. Shell Programming and Scripting

Perl script to scan back lines

Hi Perl gurus, I have this file to scan through. Sample lines below: 2008031A, USERNAME, 12345, give ABC, take XYZ, transaction submitted 2008031B, USERNAME, 12346, waiting for processing 2008031C, USERNAME, 12347, Retrieving response 2008031D, USERNAME, 12348, This is not a valid dealing... (3 Replies)
Discussion started by: gholdbhurg
3 Replies

4. Shell Programming and Scripting

Get the latest added part of a text file?

Hi, I want to get the latest added part of a text file sent over the network to for analysis. Is there a tool to use to keep track of which part of a text file that has already been analysed so only the "new" lines will be sent and handled? I have checked a few tools but I still donīt know how to... (3 Replies)
Discussion started by: pcrs
3 Replies

5. Shell Programming and Scripting

Scan a log file

hi guys, i am very new to scripting & lookin for a shell script/perl script which would scan another file with the keyword "no change" & take a count of the same. Let me know if any further details are required. (3 Replies)
Discussion started by: nhanda
3 Replies

6. Shell Programming and Scripting

how to check if a new line has been added to a file?

Have come up with the following but it doesn't seem to work.. Is there some other command i could use to get this to work? OUTPATH=/home/out PARMFILE=$OUTPATH/jobcount_test.txt LOG=$OUTPATH/job_count_monthlymail_log.txt HLOG=$OUTPATH/job_count_monthlymail_hlog.txt # echo " started at... (2 Replies)
Discussion started by: Jazmania
2 Replies

7. Shell Programming and Scripting

I want Trailer to be added into the text file.

Hi folks, I want Trailer to be added into the txt file the format is below. flatfile-> abc.txt count of the file is 500 records. I want the trailer in this format: TRAILER|500 (pipe delimeter). Please suggest the comands ASAP. Rgds Ann (5 Replies)
Discussion started by: Haque123
5 Replies

8. Shell Programming and Scripting

Log File Scan

I need to read the last line of a log file and save it, sleep for X minutes and read the last line again. If the line is the same, exit 1, otherwise sleep for X minutes until the last line contains 'Status: Process completed'. Can anyone offer advice here? Thanks. (2 Replies)
Discussion started by: mode09
2 Replies

9. Shell Programming and Scripting

Scan a file in realtime and execute certain commands on encountering 5 consecutive identical lines

Mysql log has something like below: I need a bash shell script that will do the following: 1) The script will scan the mysql.log file constantly in real time (something like tail -F mysql.log) 2) If it encounters 5 consecutive identical lines then it would invoke some commands (say... (4 Replies)
Discussion started by: proactiveaditya
4 Replies

10. Shell Programming and Scripting

Error after I added several lines

I have added several lines to my code let cnt=0 if then if then cnt=`expr ${cnt} + 1` sftp_name=EOMMSRPT1 echo "EOMsftpname = ${sftp_name}" >> ${LOGF} ... (2 Replies)
Discussion started by: digioleg54
2 Replies
ns_sched(3aolserver)					    AOLserver Built-In Commands 				      ns_sched(3aolserver)

__________________________________________________________________________________________________________________________________________________

NAME
ns_after, ns_cancel, ns_pause, ns_resume, ns_schedule_daily, ns_schedule_proc, ns_schedule_weekly, ns_unschedule_proc - commands SYNOPSIS
ns_after seconds {script | procname ?args?} ns_cancel id ns_pause id ns_resume id ns_schedule_daily ?-thread? ?-once? hour minute {script | procname ?args?} ns_schedule_proc ?-thread? ?-once? interval {script | procname ?args?} ns_schedule_weekly ?-thread? ?-once? day hour minute {script | procname ?args?} ns_unschedule_proc id _________________________________________________________________ DESCRIPTION
ns_after run the specified script or procedure after the specified number of seconds ns_after returns an id which can be used with the ns_pause, ns_cancel and ns_resume apis. ns_cancel stops the scheduled running of the id returned by an ns_after returns 1 if unscheduled 0 if the script of procedure couldn't be unscheduled ns_pause pauses the scheduled running of the id returned by an ns_after returns 1 if paused, 0 if the script of procedure couldn't be paused ns_resume resumes the scheduled running of the id returned by an ns_after returns 1 if resumed, 0 if the script of procedure couldn't be resumed ns_schedule_daily ns_schedule_daily runs the specified Tcl script or procedure (procname) once a day at the time specified by hour and minute. The hour can be from 0 to 23, and the minute can be from 0 to 59. Specify -thread if you want a thread created to run the procedure. This will allow the scheduler to continue with other scheduled procedures. Specifying -thread is appropriate in situations where the script will not return immediately, such as when the script performs network activity. Specify -once if you want the script to run only one time. The default is that the script will be re-scheduled after each time it is run. ns_schedule_daily returns an id number for the scheduled procedure that is needed to stop the scheduled procedure with ns_unsched- ule_proc. ns_schedule_proc ns_schedule_proc runs the specified Tcl script or procedure (procname) at an interval specified by interval. The interval is the number of seconds between runs of the script. Specify -thread if you want a thread created to run the procedure. This will allow the scheduler to continue with other scheduled procedures. Specifying -thread is appropriate in situations where the script will not return immediately, such as when the script performs network activity. Specify -once if you want the script to run only one time. The default is that the script will be re-scheduled after each time it is run. ns_schedule_proc returns an id number for the scheduled procedure that is needed to stop the scheduled procedure with ns_unsched- ule_proc. ns_schedule_weekly ns_schedule_weekly runs the specified Tcl script or procedure (procname) once a week on the day specified by day and the time speci- fied by hour and minute. The day can be from 0 to 6, where 0 represents Sunday. The hour can be from 0 to 23, and the minute can be from 0 to 59. Specify -thread if you want a thread created to run the procedure. This will allow the scheduler to continue with other scheduled procedures. Specifying -thread is appropriate in situations where the script will not return immediately, such as when the script performs network activity. Specify -once if you want the script to run only one time. The default is that the script will be re-scheduled after each time it is run. ns_schedule_weekly returns an id number for the scheduled procedure that is needed to stop the scheduled procedure with ns_unsched- ule_proc. ns_unschedule_proc id ns_unschedule_proc stops a scheduled procedure from executing anymore. The scheduled procedure to be stopped is identified by its id, which was returned by the ns_schedule* function that was used to schedule the procedure. EXAMPLES
ns_after ns_cancel ns_pause ns_resume This example illustrates a web interface used to manage jobs. Depending on the action provided a job can be created, cancelled, paused or resumed. set action [ns_queryget action] set job [ns_queryget job] switch $action { create { set job [ns_after 10 [ns_queryget script]] ns_puts "Job created with id: $job" } cancel { if {[ns_cancel $job]} { ns_puts "Job $job cancelled" } else { ns_puts "Job $job not cancelled" } } pause { if {[ns_pause $job]} { ns_puts "Job $job paused" } else { ns_puts "Job $job not paused } } resume { if {[ns_resume $job]} { ns_puts "Job $job resumed" } else { ns_puts "Job $job couldn't be resumed" } } default { ns_puts "Invalid action $action" } } ns_schedule_daily This example defines a script called rolllog that uses ns_accesslog to roll the access log to a file with an extension containing the current date. The ns_schedule_daily function is used to execute the rolllog script on a daily basis. # Script to roll and rcp log file to host "grinder" proc rolllog {} { set suffix [ns_strftime "%y-%m-%d"] set new [ns_accesslog file].$suffix ns_accesslog roll $new exec rcp $new grinder:/logs/[file tail $new] } # Schedule "rolllog" to run at 3:30 am each morning ns_schedule_daily -thread 3 30 rolllog ns_schedule_proc proc dosomething blah { ns_log Notice "proc with arg '$blah'" } ns_schedule_proc 10 dosomething $arg1 SEE ALSO
KEYWORDS
schedule pause resume unschedule cancel after AOLserver 4.0 ns_sched(3aolserver)
All times are GMT -4. The time now is 01:56 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy