Script to check file sequence


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to check file sequence
# 1  
Old 04-16-2009
Script to check file sequence

Hi everyone,

I need help in creating a script that would check if the file sequence is in order in a particular directory. These are log files that are generated throughout the day.

Example of the file name will be, ABC01_YYMMDDHHMM###### (ABC01_0904161829000001)

Sometimes the file generated will skip and I need to determine which one.

Could someone please help me.

Thank you all very much.
# 2  
Old 04-16-2009
i assume the last 6 digits is the running sequence and the max it can go is 999999.
make a base
Code:
for i in {1..99999}; do printf "%.6d\n" $i; done > base.txt

doing the check
Code:
ls ABC_* | sed "s/.*\(......\)$/\1/"|sort -n > check.txt

use diff
Code:
diff base.txt check.txt

# 3  
Old 04-22-2009
Hi ghostdog74,

Thank you for your suggestion.

But when I ran your code

for i in {1..99999}; do printf "%.6d\n" $i; done > base.txt
it returned the following error

printf: {1..99999}: invalid number

The other 2 codes were fine though.

The other thing I found out is that only the last 4 digits are running numbers as in the example below.

ABC01_YYMMDDHHMMSS#### (ABC01_0904161829290001)

I currently do have a simple script that lists the file in the directory for the day, I was hoping that I could integrate the sequence check script into this file.

Please advice me if that is possible. My script file is below

while true
do
ll /data/data01/ARCHIVE/ABC | grep ABC01_* | grep "`date '+%b %e'`" | awk '{print $5, $6, $7, $8, $9}'
echo
date
sleep 60
done

My script out is;

510149 Apr 22 12:01 ABC01_0904221153032890
508721 Apr 22 12:01 ABC01_0904221154042891
509632 Apr 22 12:01 ABC01_0904221155052892
508150 Apr 22 12:01 ABC01_0904221156082893
508451 Apr 22 12:01 ABC01_0904221157092894
509378 Apr 22 12:01 ABC01_0904221158072895
509437 Apr 22 12:01 ABC01_0904221159072896
508824 Apr 22 12:01 ABC01_0904221200012897
508270 Apr 22 12:01 ABC01_0904221200592898

Wed Apr 22 12:18:47 MYT 2009

Maybe the script can check the files for correct sequence and display a message like "File missing from sequence ABC01_090416182929####" when there is a missing file.

Thank you for your input, really appreciate it.
# 4  
Old 04-22-2009
Which OS you are using?

Please try :

Code:
for each in `seq 1 9999`
do
[[ -f "ABC01_090416182929$each" ]] || echo "File ABC01_090416182929$each not exist";
done

# 5  
Old 04-22-2009
Hi,

Check the below code

infile is the input file containing the list of ABC01_* files
this is can be done by adding the code in the below script

Code:
cd <dirname>
ls -l | awk '{print $9}'

outfile is the file containing the missing sequence.

I ran the script and o/p is shown below

Code:
 
# more test
#!/bin/sh
set -x
flag=0
j=0
if [ ! -f outfile ]
then
touch outfile
else
rm outfile
fi
while read line
do
        i=`echo $line | cut -c19-`
        if [ $i -ne $j -a $flag -ne 0 ]
        then
        echo "file ending with the sequence $j is missing" >> outfile
        fi
        j=`expr $i + 1`
        if [ $flag -eq 0 ];then
        :
        fi
        flag=`expr $flag + 1`
        echo $flag
done < infile
# more infile
ABC01_0904221153032890
ABC01_0904221154042892
ABC01_0904221154042893
ABC01_0904221154042895
ABC01_0904221154042896
# echo
# more outfile
file ending with the sequence 2891 is missing
file ending with the sequence 2894 is missing

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

To check the missing file based on sequence number.

Hi All, I have a requirement that i need to list only the missing sequences with a unix script. For Example: Input: FILE_001.txt FILE_002.txt FILE_005.txt FILE_006.txt FILE_008.txt FILE_009.txt FILE_010.txt FILE_014.txt Output: FILE_003.txt FILE_004.txt FILE_007.txt FILE_011.txt... (5 Replies)
Discussion started by: Arun1992
5 Replies

2. Shell Programming and Scripting

How to check missing sequence?

I want to listed files every hours and check the missing sequence my file format is CV.020220131430.txt CV.020220131440.txt CV.020220131450.txt CV.ddmmyyhhm.txt how to check if i have missing files in sequence .. thanks (3 Replies)
Discussion started by: before4
3 Replies

3. Shell Programming and Scripting

find common entries and match the number with long sequence and cut that sequence in output

Hi all, I have a file like this ID 3BP5L_HUMAN Reviewed; 393 AA. AC Q7L8J4; Q96FI5; Q9BQH8; Q9C0E3; DT 05-FEB-2008, integrated into UniProtKB/Swiss-Prot. DT 05-JUL-2004, sequence version 1. DT 05-SEP-2012, entry version 71. FT COILED 59 140 ... (1 Reply)
Discussion started by: manigrover
1 Replies

4. Shell Programming and Scripting

Check Sequence

* Expiry DATE: * Address1: Address2: Address3: Address4: Address5: * PO_ref_number: aadad HolderId_1: HolderId_2: HolderId_3: HolderId_4: * adad: 00000 ada: 00000 adad: RANDOM adad: RANDOM ****************************** (4 Replies)
Discussion started by: arunshankar.c
4 Replies

5. Shell Programming and Scripting

Script to check for the file existence, if file exists it should echo the no of modified days

Hi, I am looking for a shell script with the following. 1. It should check whether a particular file exists in a location #!/bin/sh if ; then echo "xxx.txt File Exists" else echo "File Not Found" fi 2. If file exists, it should check for the modified date and run a command... (2 Replies)
Discussion started by: karthikeyan_mac
2 Replies

6. OS X (Apple)

Check file sequence script not working. Need help.

I have a script to check a file sequence for missing frames. It used to work in true Unix but now on OSX 10.6.4 (bash) it reports every frame is missing. Hopefully someone here can help. #!/bin/csh -f if ($#argv < 1) then echo echo "Usage: checkseq <name> " echo... (3 Replies)
Discussion started by: scribling
3 Replies

7. Shell Programming and Scripting

Script check for file, alert if not there, and continue checking until file arrives

All, Is there a way to keep checking for a file over and over again in the same script for an interval of time? Ie If { mail -user continue checking until file arrives file arrives tasks exit I don't want the script to run each time and email the user each time a file... (4 Replies)
Discussion started by: markdjones82
4 Replies

8. Shell Programming and Scripting

help - script can check jump sequence?

Hi, if I have a lot of files like (AABBCC0010, AABBCC0011, AABBCC0012....AABBCC1000), can I write a small script to check any sequence number jump and show me the result which sequence number? #The first sequence may start from 0010 or 0101... :confused: Thank you!! (10 Replies)
Discussion started by: happyv
10 Replies

9. Shell Programming and Scripting

Check File Exists and compare to previous day file script

We have data files that are ftp'd every morning to a SUN server. The file names are exactly the same except for that each has the date included in its name. I have to write script to do 2 things: STEP 1) Verify that the file arrived in morning. STEP 2) Compare the file size of the current... (3 Replies)
Discussion started by: rbknisely
3 Replies

10. UNIX for Dummies Questions & Answers

Script to check for a file, check for 2hrs. then quit

I wish to seach a Dir for a specific file, once the file is found i will perform additional logic. If the file is not found within two hours, i would like to exit. Logically, I'm looking for the best way to approach this Thanks for any assistance in advance. Note: I'm using a C shell and... (2 Replies)
Discussion started by: mmarsh
2 Replies
Login or Register to Ask a Question

Featured Tech Videos