Get the latest added part of a text file?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Get the latest added part of a text file?
# 1  
Old 10-23-2008
Get the latest added part of a text file?

Hi,
I want to get the latest added part of a text file sent over the network to for analysis. Is there a tool to use to keep track of which part of a text file that has already been analysed so only the "new" lines will be sent and handled? I have checked a few tools but I still don´t know how to solve this. Rsync seems to be able to copy the latest added content of a file in an "incremental way" but I don´t know how to take care of this "incremental content" in a script.
Do you have any ideas?
# 2  
Old 10-23-2008
i dnon't know if there is a tool to do what you want. but how about this workaround?

after every analyse make a copy of the file. before every new analyse look for the "diff" and only analyse these differences... this can be done very easy with some lines of shell script code!

hth,
DN2
# 3  
Old 10-23-2008
Hammer & Screwdriver What about this?

If the data is appended to the file, then perhaps...
(the following is a rough coding; there are issues with spacing and field layouts known)

analyze data as normal
wc -l mydatafile >mydatafile.last

next time to analyze
newcnt=$(wc -l mydatafile)
lastcnt=$(cat mydatafile.last)
newlines=$(echo $newcnt - $lastcnt | bc)

now, a tail command could provide you with the last lines to review
# 4  
Old 10-27-2008
Thanks for your help. I will probably use the diff command to get the latest content of the textfile.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Sending mail to multiple recipient added in a text file

I am trying to find a code that can help me mail to a list of recipients which are in a text file. Sample code $cat recipient.txt me@test.com me1@test.com me2@test.com I want a mailx step that can read contents of recipient.txt and mail to all the users. I don't want to use mails... (1 Reply)
Discussion started by: Gurkamal83
1 Replies

2. Windows & DOS: Issues & Discussions

Renaming part of a text file ?

I have several files that are named like this "DotP_D14 - Verknüpfung" They all have the " - Verknüpfung" in common. I'd like to rename all of them to get rid of that last part. Is this possible with DOS on windows ? (1 Reply)
Discussion started by: pasc
1 Replies

3. Shell Programming and Scripting

To display the selected part in text file of unix

0400903071220312 20120322 20:21 1TRANTELSTRAFLEXCAB22032012CMP201323930000812201108875802100A003485363 12122011AUS 182644 000C2 8122011 0000 000 1TRANTELSTRAFLEXCAB22032012CMP201323930000812201108875802100A003485363 12122011AUS ... (6 Replies)
Discussion started by: rammm
6 Replies

4. Shell Programming and Scripting

extract part of text file

I need to extract the following lines from this text and put it in different files. From xxxx@gmail.com Thu Jun 10 21:15:46 2010 Return-Path: <xxxxx@gmail.com> X-Original-To: xxx@localhost Delivered-To:xxxx@localhost Received: from ubuntu (localhost ) by ubuntu (Postfix) with ESMTP... (11 Replies)
Discussion started by: waxo
11 Replies

5. Shell Programming and Scripting

Remove part of a column in a text file

I have a text file with delimiter "|" and sometimes the zipcode is in "5th" column or "6th" column. I want to scan the file and remove the "-2323" from the zipcode which has zip+4 digits From blah|blah|foo|bar||blah|945523-232|USA blah|blah|foo|bar||blah|foo|94555-2323|USA To... (8 Replies)
Discussion started by: gubbu
8 Replies

6. Shell Programming and Scripting

extracting part of a text file

Hi guys So I have a very large log file where each event is logged along with the time that it occurred. So for e.g. The contents of the file look like: ... 12:00:07 event 0 happened. 12:01:01 event 1 happened. 12:01:05 event 2 happened. 12:01:30 event 3 happened. 12:02:01 event 4... (10 Replies)
Discussion started by: alinaqvi90
10 Replies

7. Shell Programming and Scripting

Moving a part of the text in a file

*************** #some other text ***************** *************** #some other text ***************** address1=1.1.1.1 address2=2.2.2.2 address3=3.3.3.3 I have a file where i need to push all the text starting from address1 till end of file to, below . Can anyone of you... (6 Replies)
Discussion started by: srikanthgoodboy
6 Replies

8. UNIX for Dummies Questions & Answers

Replacing part of a text file with user input.

Ok, I am brand new to UNIX and I am trying to learn a cross between basic script and database use. I had got some ideas off the net on simple ideas for learning UNIX. I am working on creating a simple phone book program that allows myself to enter our employees from work into a phone book text... (0 Replies)
Discussion started by: georgefurbee
0 Replies

9. Shell Programming and Scripting

I want Trailer to be added into the text file.

Hi folks, I want Trailer to be added into the txt file the format is below. flatfile-> abc.txt count of the file is 500 records. I want the trailer in this format: TRAILER|500 (pipe delimeter). Please suggest the comands ASAP. Rgds Ann (5 Replies)
Discussion started by: Haque123
5 Replies

10. Shell Programming and Scripting

Moving part of Text in a file

Hi, I have this text in a file where I need to move part of the text.... <Relation1 OriginatingObjectID="Holding_1" RelatedObjectID="Party_1" id="Relation_1"> <OriginatingObjectType tc="4">Holding</OriginatingObjectType> <RelatedObjectType tc="6">Party</RelatedObjectType>... (4 Replies)
Discussion started by: mgirinath
4 Replies
Login or Register to Ask a Question