Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Looking out for the file-watcher

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Looking out for the file-watcher
# 1  
Old 01-31-2013
Question Looking out for the file-watcher
Hi friends,

I do not have much thoughts so need any help on the below issue:

I need to create shell script that will find the files & throw an error through job (autosys) when file not found.
Daily we use to receive 3 files from a system.

Obstacles:

1) All 3 files names are same.
2) Timestamp is same.
3) Even the received time may be same
Thats why got stuck.
Script which I have created:

Code:
first_time_stamp=`ls -l/apps/rdp/prd/prd1/interfaces/auto/input/archive/*tri003_global*20130131* | awk '{print $8}' | sed -n '1p'`
sec_time_stamp=`ls -l /apps/rdp/prd/prd1/interfaces/auto/input/archive/*tri003_global*20130131* | awk '{print $8}' | sed -n '2p'`
third_time_stamp=`ls -l /apps/rdp/prd/prd1/interfaces/auto/input/archive/*tri003_global*20130131* | awk '{print $8}' | sed -n '3p'`

if[ $first_time_stamp -eq $sec_time_stamp ]; then
echo "Two files received at the same time"
elif [ $first_time_stamp -gt $sec_time_stamp && $third_time_stamp -gt $sec_time_stamp ]; then
echo "Three files reveived & time stamp's are different"
else
echo files are missing"
fi

My inner feeling is that the above script is not good to solve my pupose.

Script should be give this:

Code:
[first file not arrived]; then
contact upstream for the first file
[second file not arrived] then
contact upstream for the second file
[third file not arrived] then
contact upstream for the third file
else
echo "All looks good & all 3 files arrived"

Sample file name:

Code:
-rw-rw-r-- 1 32598 wmadmin 157335 Jan 31 13:10 tri003_global_201301310810_erd.txt-20130131131318
-rw-rw-r-- 1 32598 wmadmin 157334 Jan 31 15:31 tri003_global_201301311030_erd.txt-20130131153255
-rw-rw-r-- 1 32598 wmadmin 157368 Jan 31 18:24 tri003_global_201301311324_erd.txt-20130131182819

Really appreciate if someone can help me out here.

Thanks very much
Last edited by DukeNuke2; 01-31-2013 at 06:34 PM.. Reason: Code tags
# 2  
Old 01-31-2013
Why don't you go by the time-stamp in the file name itself instead of checking the received time-stamp to identify the sequence of files received?
Code:
tri003_global_201301310810_erd.txt-20130131131318
tri003_global_201301311030_erd.txt-20130131153255
tri003_global_201301311324_erd.txt-20130131182819

There is no way you can have more than one file with the same name, if a file with same name is pushed from upstream it will overwrite the existing file.
# 3  
Old 01-31-2013
Question
Quote:
Originally Posted by bipinajith
Why don't you go by the time-stamp in the file name itself instead of checking the received time-stamp to identify the sequence of files received?
Code:
tri003_global_201301310810_erd.txt-20130131131318
tri003_global_201301311030_erd.txt-20130131153255
tri003_global_201301311324_erd.txt-20130131182819

There is no way you can have more than one file with the same name, if a file with same name is pushed from upstream it will overwrite the existing file.
Thanks for the reply mate but overwrite is not the case here as you already know that some of the part of the timestamp is diff

ex: tri003_global_20130131<first_part>_erd.txt-20130131<Sec_part>

first & sec part i cant control its a system generated, If i would have been aware of the generation of first & sec part then I never asked this question Smilie
hope you got my point but if you have some solution with generating the seq no then plz share it will be helpful
# 4  
Old 01-31-2013
Use -t option with ls command to sort by modification time, newest first.

Use -r option to reverse order while sorting:
Code:
ls -ltr

Now you will know the sequence of file received.
# 5  
Old 02-01-2013
This is in AutoSys? Why don't you just create a box job with 3 file watcher jobs who have dependencies on each other? Unsure if you have the few job types available in 11.0 or the many in 11.3, but it can handle simple things like new file, different file size, etc. Something like

SomeBox
SomeBox.00.EmptyDir - command, empties directory
Somebox.05.BoxStatus - No deps, just runs to see when things finish/status (optional)
SomeBox.10.FileWatch1 - timeout at X time, or at box level if more appropriate
Somebox.20.FileWatch2 - ""
Somebox.30.FileWatch3 - ""

D'no, lots of options - additionally I'm not sure what was bipinajith's simple ls solution. Assuming you have event manager component from workload manager alerts will autofire off based on whatever rules you defined (Otherwise, add a job whose condition is status failure or terminated to email).
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help needed for Sleep (File watcher)

Hi All, I have a requirement, where I need to check for file. If file is not available then wait for 5 minutes and if it still not available, fail the process. It should check every one minute. If File available in between, should start remaining process. Now my problem is, I have both part... (0 Replies)
Discussion started by: ace_friends22
0 Replies

2. Shell Programming and Scripting

File watcher script

if test -n "$(find/data/path/ 'filename.txst' -print-quit) then echo "file found" exit 0 else echo "file not found" exit 46 fi So I basically looking to understand the if test -n "$(find/data/path/ 'filename.txst' -print-quit) line. Pls help to elaborate what this command... (6 Replies)
Discussion started by: Nsharma3006
6 Replies

3. UNIX for Dummies Questions & Answers

Autosys file watcher

Hi friends, Instead of creating a script autosys 11.3 has a feature for filewatcher. Can anyone please share a link where ican create a file watcher job (more like manual). i need to create a filewatcher job where the job should search for a file for every 5 mins and send a mail whether the... (1 Reply)
Discussion started by: robertbrown624
1 Replies

4. Shell Programming and Scripting

Help needed with File Watcher script

Hi all, I am writing a script to do the following: 1) Check if there are any new <csv> files (in abc directory) to process 2) If there is new file, then perform some operation on that file. 3) Move it to a different location(after step2 is successful) 4) Do further processing on the file ... (2 Replies)
Discussion started by: stunnerz_84
2 Replies

5. Shell Programming and Scripting

File Watcher Help !!

Hi Experts I will have be having 3 types of files in directory file1_p0_date file1_p1_date file1_p2_date As soon as it sees any of the files it needs to kick off another process and also would need the file name For this I am creating a file watcher script which will look for file1* My... (2 Replies)
Discussion started by: westernmichnov
2 Replies

6. UNIX for Advanced & Expert Users

Need Script for File Watcher

Hi, I need a script for "FILE WATCHER" I have in a remote FTP location server named MAIN _104 EveryDay mrng 8 i hav to search couple of folders(COL, ATT, TRA) in that FTP server (MAIN_104), and if any files are present in that folders i hav to bring back to my computer and store it anywhere... (4 Replies)
Discussion started by: nani1984
4 Replies

7. Shell Programming and Scripting

File Watcher using autosys

Hi, Would like to kick off a script based on the file arrival using autosys but this has to monitor every second instead of a minute. Is there a better way to implement the monitor apart from cronjob and only need to use autosys in UNIX. Regards, Rajasilpa (0 Replies)
Discussion started by: rajasilpa
0 Replies

8. Shell Programming and Scripting

File Watcher

Hi Please help me in this I want to execute a shell script abc.ksh. But I only want it to execute if file XYZ is not present. If file XYZ is present than I want to unix to sleep for 5 Sec and than agaian check for XYX existence. if it sleeps for more than 30 seconds ( 6 time )I want it to... (3 Replies)
Discussion started by: er_zeeshan05
3 Replies

9. UNIX for Advanced & Expert Users

File Watcher Anamolous Behaviour

My file watcher is scheduled to look for a File ABC.XYZ. Ideally when it find this file present in the directory it is looking in should end :b:, but this file watcher is wasting 6-9 minutes each day :eek: i am unable to understand why this file watcher is behaving in freakish manner...:confused: (1 Reply)
Discussion started by: sameerspice
1 Replies

10. Shell Programming and Scripting

file watcher job

Hello, I am trying to create a file watcher script to check for a file until certain time and throw a message. can some one please help me if there is script handy ?? Thanks, Sateesh (1 Reply)
Discussion started by: kotasateesh
1 Replies
Login or Register to Ask a Question