Need to process files created an hour ago


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need to process files created an hour ago
# 1  
Old 06-14-2005
Question Need to process files created an hour ago

Hello all,

I would like to ask for an advice on how to deal with the following scenario.

Every now and then, our ERP system creates an interface text file with the following file format - XORD????.DLD where ???? is a sequence number. We can have 1 or more XORD files created in an hour. Now, I would like to create a script that is run every hour to check XORD* created for the last hour and then verify its contents. How do I approach this using scripting (ksh). The first thing that comes into my mind is to save the result of `ls -lt XORD` to a text file and then sort it but not sure how to compare the time stamp with the current time. Thank you very much

Sample listing of files, say if I run my script now, I wont get anything......

-rw-rw-rw- 1 amcgraw dba 5609 Jun 14 07:17 XORD9306.DLD
-rw-rw-rw- 1 root sys 172859 Jun 14 00:22 XORD9305.DLD
-rw-rw-rw- 1 ldowning dba 7287 Jun 13 17:24 XORD9304.DLD
-rw-rw-rw- 1 root sys 39985 Jun 13 17:05 XORD9303.DLD
-rw-rw-rw- 1 swelch dba 1427 Jun 13 16:20 XORD9302.DLD
-rw-rw-rw- 1 athomas dba 1207 Jun 13 16:09 XORD9301.DLD
-rw-rw-rw- 1 shicks dba 73610 Jun 13 16:07 XORD9300.DLD
-rw-rw-rw- 1 kscott dba 9860 Jun 13 15:52 XORD9299.DLD
-rw-rw-rw- 1 jrobinso dba 2404 Jun 13 15:28 XORD9298.DLD
-rw-rw-rw- 1 dburke dba 2436 Jun 13 14:48 XORD9297.DLD
-rw-rw-rw- 1 srainwat dba 1936 Jun 13 14:26 XORD9296.DLD
-rw-rw-rw- 1 kscott dba 8829 Jun 13 14:18 XORD9295.DLD
-rw-rw-rw- 1 mgwaltne dba 2532 Jun 13 13:57 XORD9294.DLD
-rw-rw-rw- 1 nthackre dba 2754 Jun 13 13:47 XORD9293.DLD
-rw-rw-rw- 1 dburke dba 2186 Jun 13 13:29 XORD9292.DLD
-rw-rw-rw- 1 shicks dba 6543 Jun 13 13:27 XORD9291.DLD
-rw-rw-rw- 1 dburke dba 62840 Jun 13 12:34 XORD9290.DLD
-rw-rw-rw- 1 shicks dba 75595 Jun 13 11:55 XORD9289.DLD

Joseph
# 2  
Old 06-14-2005
Hi Joseph

Can the file name change once you have processed it ? if so, just rename the file to a name that has a extension of the time inclusive of second it was run.

J
# 3  
Old 06-14-2005
Thanks J.

The file name cannot change after processing that is why i have to depend on the `ls` information. Smilie
# 4  
Old 06-15-2005
Any help is much appreciated...please. Smilie
# 5  
Old 06-15-2005
On each run `touch timestamp_file`
Then you can `find -newer timestamp_file`

You haven't given enough info but here is some
bash to process each new file seperately,
assuming all the files are in 1 directory:

Code:
#!/bin/sh

WORK_DIR=/var/reports
TSF=/tmp/batch_process.timestamp

[ ! -e $TSF ] && touch -r $WORK_DIR $TSF

find $WORK_DIR -newer $TSF -type f -maxdepth 1 |
while read REPORT; do
    echo processing $REPORT
done

touch $TSF

# 6  
Old 06-15-2005
Thanks!

The files are only created in one directory and all files starts with XORD. These files are created anytime within the hour and every top of the hour, I would like to check all XORD files created the last hour on the time my script run (thru cron).

Say if my script runs at 3:00PM, I would like to get the list of XORD files created between 2:00-2:59PM and then put it in a variable so I can check each file if the contents are correct.

I tried using the find command using option -ctime, atime and mtime but this is by the day and sometimes the results is not consistent (or maybe I am not using it or understand it correctly...)

THanks!

Joseph
# 7  
Old 06-15-2005
The timestamp option should work, and be more general.
The hourly period is just determined/configured in cron.
If you do want to have problematic edge cases where
you may miss files or process files twice, then use the following:

find $WORK_DIR -mmin -60 -type f -name "XORD*" -maxdepth 1
Login or Register to Ask a Question

Previous Thread | Next Thread

7 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to find the files created within one hour in Solaris?

Hi Gurus, I want to find the file created within one hour in solaris. I have tried below command, but it is no lucky. $find . -mtime -1/24, -name "abc*" above command give me the file name which created two hours ago find . -cmin -60, -name "abc*" above command I got error as... (4 Replies)
Discussion started by: ken6503
4 Replies

2. Shell Programming and Scripting

Check how many minutes ago the last file created

Hi , I need help in getting how many minutes ago the last file, matching some pattern in file name, was created in a folder. Thanks in advance. (8 Replies)
Discussion started by: Anupam_Halder
8 Replies

3. UNIX for Dummies Questions & Answers

To create alist of files created in the last 1 hour.

Hi, I need to create a script whcih ill run ever hour and will check all the files which are created in that hour for an error string. Need help in finding all the files which were created in the last 1 hour. Thanks in Advance. Asheesh (4 Replies)
Discussion started by: Asheesh
4 Replies

4. Solaris

Find any spike in CPU usage a hour ago

Folks, I need to find out from the system logs ( or elsewhere ) if the CPU utilization reached say 90% an hour ago from now. Is there a way out. (2 Replies)
Discussion started by: kavera
2 Replies

5. UNIX for Dummies Questions & Answers

file was created before 15 days ago.

How can I get difference date between today and 15 days ago and all filename is was created before 15 days ago? It has to be korn shell script. Thanks. (1 Reply)
Discussion started by: YoungBlood
1 Replies

6. Shell Programming and Scripting

command unix to list all files created since n month ago

Hello, I want to list all files that were created since 3 month ago. it exist a unix command to do it ? thank you (8 Replies)
Discussion started by: yacsil
8 Replies

7. UNIX for Dummies Questions & Answers

Deleting files created before two days ago

Dear All: I want to build a shell that delete files created two or more days ago ... I think it could be built using a special command with ls or grep, I'd apreciate any help from you guys I have a lot of log files from november, december, january and this tool will help me a lot The files... (3 Replies)
Discussion started by: josecollantes
3 Replies
Login or Register to Ask a Question

Featured Tech Videos