Get the latest added part of a text file?

Login or Register for Dates, Times and to Reply

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Get the latest added part of a text file?
# 1  
Get the latest added part of a text file?

I want to get the latest added part of a text file sent over the network to for analysis. Is there a tool to use to keep track of which part of a text file that has already been analysed so only the "new" lines will be sent and handled? I have checked a few tools but I still don´t know how to solve this. Rsync seems to be able to copy the latest added content of a file in an "incremental way" but I don´t know how to take care of this "incremental content" in a script.
Do you have any ideas?
# 2  
i dnon't know if there is a tool to do what you want. but how about this workaround?

after every analyse make a copy of the file. before every new analyse look for the "diff" and only analyse these differences... this can be done very easy with some lines of shell script code!

# 3  
Hammer & Screwdriver What about this?

If the data is appended to the file, then perhaps...
(the following is a rough coding; there are issues with spacing and field layouts known)

analyze data as normal
wc -l mydatafile >mydatafile.last

next time to analyze
newcnt=$(wc -l mydatafile)
lastcnt=$(cat mydatafile.last)
newlines=$(echo $newcnt - $lastcnt | bc)

now, a tail command could provide you with the last lines to review
# 4  
Thanks for your help. I will probably use the diff command to get the latest content of the textfile.
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #8
Difficulty: Easy
macOS is based on the Unix operating system.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Sending mail to multiple recipient added in a text file

I am trying to find a code that can help me mail to a list of recipients which are in a text file. Sample code $cat recipient.txt I want a mailx step that can read contents of recipient.txt and mail to all the users. I don't want to use mails... (1 Reply)
Discussion started by: Gurkamal83
1 Replies

2. Windows & DOS: Issues & Discussions

Renaming part of a text file ?

I have several files that are named like this "DotP_D14 - Verknüpfung" They all have the " - Verknüpfung" in common. I'd like to rename all of them to get rid of that last part. Is this possible with DOS on windows ? (1 Reply)
Discussion started by: pasc
1 Replies

3. Shell Programming and Scripting

To display the selected part in text file of unix

0400903071220312 20120322 20:21 1TRANTELSTRAFLEXCAB22032012CMP201323930000812201108875802100A003485363 12122011AUS 182644 000C2 8122011 0000 000 1TRANTELSTRAFLEXCAB22032012CMP201323930000812201108875802100A003485363 12122011AUS ... (6 Replies)
Discussion started by: rammm
6 Replies

4. Shell Programming and Scripting

extract part of text file

I need to extract the following lines from this text and put it in different files. From Thu Jun 10 21:15:46 2010 Return-Path: <> X-Original-To: xxx@localhost Delivered-To:xxxx@localhost Received: from ubuntu (localhost ) by ubuntu (Postfix) with ESMTP... (11 Replies)
Discussion started by: waxo
11 Replies

5. Shell Programming and Scripting

Remove part of a column in a text file

I have a text file with delimiter "|" and sometimes the zipcode is in "5th" column or "6th" column. I want to scan the file and remove the "-2323" from the zipcode which has zip+4 digits From blah|blah|foo|bar||blah|945523-232|USA blah|blah|foo|bar||blah|foo|94555-2323|USA To... (8 Replies)
Discussion started by: gubbu
8 Replies

6. Shell Programming and Scripting

extracting part of a text file

Hi guys So I have a very large log file where each event is logged along with the time that it occurred. So for e.g. The contents of the file look like: ... 12:00:07 event 0 happened. 12:01:01 event 1 happened. 12:01:05 event 2 happened. 12:01:30 event 3 happened. 12:02:01 event 4... (10 Replies)
Discussion started by: alinaqvi90
10 Replies

7. Shell Programming and Scripting

Moving a part of the text in a file

*************** #some other text ***************** *************** #some other text ***************** address1= address2= address3= I have a file where i need to push all the text starting from address1 till end of file to, below . Can anyone of you... (6 Replies)
Discussion started by: srikanthgoodboy
6 Replies

8. UNIX for Dummies Questions & Answers

Replacing part of a text file with user input.

Ok, I am brand new to UNIX and I am trying to learn a cross between basic script and database use. I had got some ideas off the net on simple ideas for learning UNIX. I am working on creating a simple phone book program that allows myself to enter our employees from work into a phone book text... (0 Replies)
Discussion started by: georgefurbee
0 Replies

9. Shell Programming and Scripting

I want Trailer to be added into the text file.

Hi folks, I want Trailer to be added into the txt file the format is below. flatfile-> abc.txt count of the file is 500 records. I want the trailer in this format: TRAILER|500 (pipe delimeter). Please suggest the comands ASAP. Rgds Ann (5 Replies)
Discussion started by: Haque123
5 Replies

10. Shell Programming and Scripting

Moving part of Text in a file

Hi, I have this text in a file where I need to move part of the text.... <Relation1 OriginatingObjectID="Holding_1" RelatedObjectID="Party_1" id="Relation_1"> <OriginatingObjectType tc="4">Holding</OriginatingObjectType> <RelatedObjectType tc="6">Party</RelatedObjectType>... (4 Replies)
Discussion started by: mgirinath
4 Replies

Featured Tech Videos