Sponsored Content
Top Forums UNIX for Dummies Questions & Answers How to scan only new lines added in file? Post 70370 by RTM on Thursday 28th of April 2005 08:26:10 AM
Old 04-28-2005
You can find out how many lines are in the log file with wc -l. Use a second wc -l on the file to find out how many more lines were added - then just tail the difference. Save your last wc -l into a separate file for use on the next run

Run this command to start it all off (change the location and name of file to suit your needs)
wc -l logfile > /tmp/logfile.linecount

Get the number of lines from the linecount file (use cut or awk to get only the first field). Use that number and a new wc -l of the log file to find out how many new lines. Do your processing against only the new lines utilizing the tail command (see the man page). Save the new number of lines to your linecount file for the next run.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

File Scan

Hi everyone , i m working on Sun solaris and i have a file "smsapp.cur" which has information like this paragraph given below , there are millions of such paragraphs From:923212802736 To:923222326807 logMessage: 07-04-08 17:34:29 Getting message topup from code page default in language English... (2 Replies)
Discussion started by: Dastard
2 Replies

2. Shell Programming and Scripting

File Inventory Scan

Is there such a script out there that will Analyze a folder and subfolders, getting file type utilization and file aging. A CSV file will be created with file information File,Size,Date Created,Last Modified,Last Accessed,Extension,File Type,Owner A Summary report text file will also be created... (4 Replies)
Discussion started by: seacros
4 Replies

3. Shell Programming and Scripting

Perl script to scan back lines

Hi Perl gurus, I have this file to scan through. Sample lines below: 2008031A, USERNAME, 12345, give ABC, take XYZ, transaction submitted 2008031B, USERNAME, 12346, waiting for processing 2008031C, USERNAME, 12347, Retrieving response 2008031D, USERNAME, 12348, This is not a valid dealing... (3 Replies)
Discussion started by: gholdbhurg
3 Replies

4. Shell Programming and Scripting

Get the latest added part of a text file?

Hi, I want to get the latest added part of a text file sent over the network to for analysis. Is there a tool to use to keep track of which part of a text file that has already been analysed so only the "new" lines will be sent and handled? I have checked a few tools but I still donīt know how to... (3 Replies)
Discussion started by: pcrs
3 Replies

5. Shell Programming and Scripting

Scan a log file

hi guys, i am very new to scripting & lookin for a shell script/perl script which would scan another file with the keyword "no change" & take a count of the same. Let me know if any further details are required. (3 Replies)
Discussion started by: nhanda
3 Replies

6. Shell Programming and Scripting

how to check if a new line has been added to a file?

Have come up with the following but it doesn't seem to work.. Is there some other command i could use to get this to work? OUTPATH=/home/out PARMFILE=$OUTPATH/jobcount_test.txt LOG=$OUTPATH/job_count_monthlymail_log.txt HLOG=$OUTPATH/job_count_monthlymail_hlog.txt # echo " started at... (2 Replies)
Discussion started by: Jazmania
2 Replies

7. Shell Programming and Scripting

I want Trailer to be added into the text file.

Hi folks, I want Trailer to be added into the txt file the format is below. flatfile-> abc.txt count of the file is 500 records. I want the trailer in this format: TRAILER|500 (pipe delimeter). Please suggest the comands ASAP. Rgds Ann (5 Replies)
Discussion started by: Haque123
5 Replies

8. Shell Programming and Scripting

Log File Scan

I need to read the last line of a log file and save it, sleep for X minutes and read the last line again. If the line is the same, exit 1, otherwise sleep for X minutes until the last line contains 'Status: Process completed'. Can anyone offer advice here? Thanks. (2 Replies)
Discussion started by: mode09
2 Replies

9. Shell Programming and Scripting

Scan a file in realtime and execute certain commands on encountering 5 consecutive identical lines

Mysql log has something like below: I need a bash shell script that will do the following: 1) The script will scan the mysql.log file constantly in real time (something like tail -F mysql.log) 2) If it encounters 5 consecutive identical lines then it would invoke some commands (say... (4 Replies)
Discussion started by: proactiveaditya
4 Replies

10. Shell Programming and Scripting

Error after I added several lines

I have added several lines to my code let cnt=0 if then if then cnt=`expr ${cnt} + 1` sftp_name=EOMMSRPT1 echo "EOMsftpname = ${sftp_name}" >> ${LOGF} ... (2 Replies)
Discussion started by: digioleg54
2 Replies
ns_sched(3aolserver)					    AOLserver Built-In Commands 				      ns_sched(3aolserver)

__________________________________________________________________________________________________________________________________________________

NAME
ns_after, ns_cancel, ns_pause, ns_resume, ns_schedule_daily, ns_schedule_proc, ns_schedule_weekly, ns_unschedule_proc - commands SYNOPSIS
ns_after seconds {script | procname ?args?} ns_cancel id ns_pause id ns_resume id ns_schedule_daily ?-thread? ?-once? hour minute {script | procname ?args?} ns_schedule_proc ?-thread? ?-once? interval {script | procname ?args?} ns_schedule_weekly ?-thread? ?-once? day hour minute {script | procname ?args?} ns_unschedule_proc id _________________________________________________________________ DESCRIPTION
ns_after run the specified script or procedure after the specified number of seconds ns_after returns an id which can be used with the ns_pause, ns_cancel and ns_resume apis. ns_cancel stops the scheduled running of the id returned by an ns_after returns 1 if unscheduled 0 if the script of procedure couldn't be unscheduled ns_pause pauses the scheduled running of the id returned by an ns_after returns 1 if paused, 0 if the script of procedure couldn't be paused ns_resume resumes the scheduled running of the id returned by an ns_after returns 1 if resumed, 0 if the script of procedure couldn't be resumed ns_schedule_daily ns_schedule_daily runs the specified Tcl script or procedure (procname) once a day at the time specified by hour and minute. The hour can be from 0 to 23, and the minute can be from 0 to 59. Specify -thread if you want a thread created to run the procedure. This will allow the scheduler to continue with other scheduled procedures. Specifying -thread is appropriate in situations where the script will not return immediately, such as when the script performs network activity. Specify -once if you want the script to run only one time. The default is that the script will be re-scheduled after each time it is run. ns_schedule_daily returns an id number for the scheduled procedure that is needed to stop the scheduled procedure with ns_unsched- ule_proc. ns_schedule_proc ns_schedule_proc runs the specified Tcl script or procedure (procname) at an interval specified by interval. The interval is the number of seconds between runs of the script. Specify -thread if you want a thread created to run the procedure. This will allow the scheduler to continue with other scheduled procedures. Specifying -thread is appropriate in situations where the script will not return immediately, such as when the script performs network activity. Specify -once if you want the script to run only one time. The default is that the script will be re-scheduled after each time it is run. ns_schedule_proc returns an id number for the scheduled procedure that is needed to stop the scheduled procedure with ns_unsched- ule_proc. ns_schedule_weekly ns_schedule_weekly runs the specified Tcl script or procedure (procname) once a week on the day specified by day and the time speci- fied by hour and minute. The day can be from 0 to 6, where 0 represents Sunday. The hour can be from 0 to 23, and the minute can be from 0 to 59. Specify -thread if you want a thread created to run the procedure. This will allow the scheduler to continue with other scheduled procedures. Specifying -thread is appropriate in situations where the script will not return immediately, such as when the script performs network activity. Specify -once if you want the script to run only one time. The default is that the script will be re-scheduled after each time it is run. ns_schedule_weekly returns an id number for the scheduled procedure that is needed to stop the scheduled procedure with ns_unsched- ule_proc. ns_unschedule_proc id ns_unschedule_proc stops a scheduled procedure from executing anymore. The scheduled procedure to be stopped is identified by its id, which was returned by the ns_schedule* function that was used to schedule the procedure. EXAMPLES
ns_after ns_cancel ns_pause ns_resume This example illustrates a web interface used to manage jobs. Depending on the action provided a job can be created, cancelled, paused or resumed. set action [ns_queryget action] set job [ns_queryget job] switch $action { create { set job [ns_after 10 [ns_queryget script]] ns_puts "Job created with id: $job" } cancel { if {[ns_cancel $job]} { ns_puts "Job $job cancelled" } else { ns_puts "Job $job not cancelled" } } pause { if {[ns_pause $job]} { ns_puts "Job $job paused" } else { ns_puts "Job $job not paused } } resume { if {[ns_resume $job]} { ns_puts "Job $job resumed" } else { ns_puts "Job $job couldn't be resumed" } } default { ns_puts "Invalid action $action" } } ns_schedule_daily This example defines a script called rolllog that uses ns_accesslog to roll the access log to a file with an extension containing the current date. The ns_schedule_daily function is used to execute the rolllog script on a daily basis. # Script to roll and rcp log file to host "grinder" proc rolllog {} { set suffix [ns_strftime "%y-%m-%d"] set new [ns_accesslog file].$suffix ns_accesslog roll $new exec rcp $new grinder:/logs/[file tail $new] } # Schedule "rolllog" to run at 3:30 am each morning ns_schedule_daily -thread 3 30 rolllog ns_schedule_proc proc dosomething blah { ns_log Notice "proc with arg '$blah'" } ns_schedule_proc 10 dosomething $arg1 SEE ALSO
KEYWORDS
schedule pause resume unschedule cancel after AOLserver 4.0 ns_sched(3aolserver)
All times are GMT -4. The time now is 10:10 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy