Read from last point Post: 302976658

Login or Register to Ask a Question and Join Our Community

Sponsored Content

Top Forums Shell Programming and Scripting Read from last point Post 302976658 by Don Cragun on Saturday 2nd of July 2016 03:31:03 PM

Old

07-02-2016

Registered User

You can save the last timestamp processed by your awk script and reload that value in your shell script before calling awk the next time and then have awk skip over all lines with a timestamp that is less than or equal to the saved timestamp. For example:

Code:

#!/bin/ksh
IAm=${0##*/}
TSF="$HOME/.LastTimeStamp"

if [ ! -f $TSF ]
then	printf '%s: Time stamp file (%s) not found.\n\tProcessing all timestamps.\n' \
	    "$IAm" "$TSF" >&2
	ts=0
else	read ts < "$TSF"
fi
awk -v TS="$ts" -v TSF="$TSF" '
$1 + 0 <= TS {
	# Skip previous processed timestamps.
	next
}
{	# Do what every you want to do with the new timestamp...
	# Save last completed time stamp.
	print $1 > TSF
	close(TSF)
}' input_file...

Obviously, you should also verify that the read was successful, but this might work as a starting point for what you need.

This User Gave Thanks to Don Cragun For This Post:

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Fibre connection Point to Point SUN

Anyone know of a guide or instructions for Solaris I got to configure a SBUS HBA to talk to a tape robot. I have done this on a switch but not point to point. just going HBA >>>>> TAPE Fibre simple two nodes Kie

2. UNIX for Advanced & Expert Users

read() wont allow me to read files larger than 2 gig (on a 64bit)

Hi the following c-code utilizing the 'read()' man 2 read method cant read in files larger that 2gig. Hi I've found a strange problem on ubuntu64bit, that limits the data you are allowed to allocate on a 64bit platform using the c function 'read()' The following program wont allow to allocate...

3. Shell Programming and Scripting

Read Embedded Newline characters with read (builtin) in KSH93

Hi Guys, Happy New Year to you all! I have a requirement to read an embedded new-line using KSH's read builtin. Here is what I am trying to do: run_sql "select guestid, address, email from guest" | while read id addr email do ## Biz logic goes here done I can take care of any...

4. UNIX for Dummies Questions & Answers

When reading a csv file, counter to read 20 lines and wait for minute then read next 20 till end

Hello All, i am a newbie and need some help when reading a csv file in a bourne shell script. I want to read 10 lines, then wait for a minute and then do a reading of another 10 lines and so on in the same way. I want to do this till the end of file. Any inputs are appreciated ...

5. Shell Programming and Scripting

Help with Bash piped while-read and a read user input at the same time

Hi I am new to writing script and want to use a Bash Piped while-read and read from user input. if something happens on server.log then do while loop or if something happend on user input then do while loop. Pseudocode something like: tail -n 3 -f server.log | while read serverline || read...

6. Shell Programming and Scripting

How to perform a hexdump using dd from start point to end point?

hi, I would like to ask or is it possible to dump a hex using dd from starting point to end point just like the "xxd -s 512 -l 512 <bin file>" I know the redirect hexdump -C but i can't figure it out the combination options of dd. Hope someone can share their knowledge.. Thanks in...

7. Linux

Make directory used as mount point read-only

For my backup , I mount and external hard disk to /mnt/mybackup and then I do an rsync to /mnt/mybackup If for some reason the rsync fails, I want to prevent it from writing data on the server hard disk itself since the external hard disk will no longer be mounted on it. I want /mnt/mybackup...

8. Shell Programming and Scripting

[Bash] Read History function & Read Arrowkeys

Hi. How can I create a history function? (By "read" command or so) & How can I configure a read command so that the arrow keys are not displayed so funny? (^[[A) Thanks in advance.

9. Shell Programming and Scripting

Read from last point

Hi again, first thanks for all your suggestions. This forum it is very useful. I have a question. Is it possible to read from the last line a file was closed. For example, imagine that i've got a file with a LOT of timestamps : 1467387616.868717770 1467387616.874189609...

10. UNIX for Beginners Questions & Answers

How to create a new mount point with 600GB and add 350 GBexisting mount point? IN AIX

How to create a new mount point with 600GB and add 350 GBexisting mount point Best if there step that i can follow or execute before i mount or add diskspace IN AIX Thanks

LEARN ABOUT HPUX

rc.config

rc.config(4)						     Kernel Interfaces Manual						      rc.config(4)

NAME

       rc.config, rc.config.d - files containing system configuration information

SYNOPSIS

DESCRIPTION

       The  system  configuration  used  at  startup  is  contained in files within the directory The file sources all of the files within and and
       exports their contents to the environment.

   /etc/rc.config
       The file is a script that sources all of the scripts, and also sources To read the  configuration  definitions,	only  this  file  need	be
       sourced.   This	file  is  sourced by whenever it is run, such as when the command is run to transition between run states.  Each file that
       exists in is sourced, without regard to which startup scripts are to be executed.

   /etc/rc.config.d
       The configuration information is structured as a directory of files, rather than as a single file containing the  same  information.   This
       allows  developers  to  create and manage their own configuration files here, without the complications of shared ownership and access of a
       common file.

   /etc/rc.config.d/* Files
       This is where files containing configuration variable assignments are located.

       Configuration scripts must be written to be read by the POSIX shell, and not the Bourne shell, or In some cases, these files must  also	be
       read  and  possibly modified by control scripts or the sam program.  See sd(4) and sam(1M).  For this reason, each variable definition must
       appear on a separate line, with the syntax:

       No trailing comments may appear on a variable definition line.  Comment statements must be on separate lines, with the comment character in
       column one.  This example shows the required syntax for configuration files:

       Configuration variables may be declared as array parameters when describing multiple instances of the variable configuration.  For example,
       a system may contain two network interfaces, each having a unique IP address and subnet mask (see ifconfig(1M)).  An example of such a dec-
       laration is as follows:

       Note  that  there  must	be  no requirements on the order of the files sourced.	This means configuration files must not refer to variables
       defined in other configuration files, since there is no guarantee that the variable being referenced is currently  defined.   There  is	no
       protection  against  environment variable namespace collision in these configuration files.  Programmers must take care to avoid such prob-
       lems.

   /etc/TIMEZONE
       The file contains the definition of the environment variable.  This file is required by POSIX.  It is sourced by at the same time the files
       are sourced.

SEE ALSO

       rc(1M).

																      rc.config(4)