Shell Programming and Scripting

Hello everbody.

I'm trying to write a ksh script to find errors inside a logfile. But the main problem I found is that the "Logfile.txt" will got new lines very often. The software generates about 1000 lines per hour. (yes, ~24000 lines/day)

"Logfile.txt" (with just very few lines)

Code:

# LOG:[00:57:12 30.9.2011] queued:/FolderA/ProcessA
# LOG:[00:57:12 30.9.2011] queued:/FolderB/ProcessB
# LOG:[01:00:21 1.10.2011] submitted:/FolderA/ProcessA
# LOG:[01:59:48 1.10.2011] submitted:/FolderB/ProcessB
# LOG:[02:00:08 1.10.2011] complete:/FolderA/ProcessA
# LOG:[02:59:08 1.10.2011] aborted:/FolderB/ProcessB

I need to:
- Read every line ( get date, time and the status )

Code:

date="${year}${month}${day}${hour}"
eg: 2012020216

- If the $date is greater than in the previous line ($old_date ) then ...
Get all lines with the $old_date in it ... and copy it to new files like:

Code:

Folder_${year}${month}${day}/log_${hour}.txt

- Then delete all the lines (in the "Logfile.txt"), I just moved to the new files.
So the "Logfile.txt" doesn't get 1GB+ anymore.

The problem is that the "Logfile.txt" will get new lines very often and I don't know how to delete from it without losing data.

Thanks in advance.
Killua.

( sorry for my terrible english )