To check the missing file based on sequence number.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting To check the missing file based on sequence number.
# 1  
Old 03-13-2016
To check the missing file based on sequence number.

Hi All,

I have a requirement that i need to list only the missing sequences with a unix script.

For Example:
Input:
Code:
FILE_001.txt
FILE_002.txt
FILE_005.txt
FILE_006.txt
FILE_008.txt
FILE_009.txt
FILE_010.txt
FILE_014.txt

Output:
Code:
FILE_003.txt
FILE_004.txt
FILE_007.txt
FILE_011.txt
FILE_012.txt
FILE_013.txt

(or)
Code:
003
004
007
011
012
013

If anyone can help me out, it will be great.

Regards,
Arun

Last edited by Franklin52; 03-13-2016 at 01:33 PM.. Reason: Please use code tags
# 2  
Old 03-13-2016
You may find hints for the solution by searching these fora:
How to takes the missing files in ascending order
or
How to take the missing sequence Number?
# 3  
Old 04-22-2016
List unsequenced files

Hi, the following works in ksh.

Assumptions are that the format of the file name is file_{num}.txt and that the file numbers are all 3 digits.

Code:
#!/bin/ksh

#Do a file listing and count and set variables

ls FILE* > answers.txt
tot_line=`wc -l answers.txt | awk '{print $1}'`
typeset -Z3 file_missing
current_line=1
next_line=2

#loop through list of files getting the file number

while [ $current_line -lt $tot_line ]
do
current_file=`awk 'NR=='$current_line'' answers.txt | sed 's/\(.*_\)\(.*[0-9]\)\
(.*$\)/\2/'`
next_file=`awk 'NR=='$next_line'' answers.txt | sed 's/\(.*_\)\(.*[0-9]\)\(.*$\)
/\2/'`

#find out if files are sequential

sequence=`expr $next_file - $current_file`

#If files not sequential start loop to print list of unsequential files

if [ $sequence -ne 1 ]
then
curr_file=$current_file
while [ $sequence -gt 1 ]
do

file_missing=`expr $curr_file + 1`
curr_file=`expr $curr_file + 1`

echo "FILE_$file_missing.txt is missing"
sequence=`expr $sequence - 1`
done

fi

current_line=`expr $current_line + 1`
next_line=`expr $next_line + 1`

done

output

FILE_003.txt is missing
FILE_004.txt is missing
FILE_007.txt is missing
FILE_011.txt is missing
FILE_012.txt is missing
FILE_013.txt is missing
# 4  
Old 04-22-2016
bash
Code:
for i in {1..14}
do
seq=`printf "%03d" $i`
if [ ! -f "FILE_${seq}.txt" ]
then
echo "FILE_${i}.txt"
fi
done

# 5  
Old 04-22-2016
You might also try the following ksh script that just uses shell built-ins (so it runs a little faster than andy391791's script). And, unlike andy391791's script, it also adds some error checking code and will work with any shell that performs basic parameter expansions required by the POSIX standards instead of just working with a Korn shell. And, unlike looney's bash script, it will report missing files from the lowest numbered existing file with a name matching FILE_[0-9][0-9][0-9].txt to the highest numbered existing file with a name matching that pattern instead of just looking for missing files in the range 001 through 014.
Code:
#!/bin/ksh
first=1
missing=0

nextfile() {
	seq=$((seq + 1))
	next=$(printf 'FILE_%03d.txt' "$seq")
}

for i in FILE_[0-9][0-9][0-9].txt
do	if [ "$first" = 1 ]
	then	if [ ! -f "$i" ]
		then	printf 'No files matching pattern found.\n' >&2
			exit 1
		fi
		seq=${i#FILE_}
		seq=${seq#0}	# Avoid problems with some shells treating
		seq=${seq#0}	# numbers with leading zeros as octal.
		seq=${seq%.txt}
		first=
		continue
	fi
	nextfile
	while [ "$next" != "$i" ]
	do	printf '%s is missing\n' "$next"
		missing=$((missing + 1))
		nextfile
	done
done
if [ "$missing" = 0 ]
then	printf 'No missing files detected.\n'
fi

# 6  
Old 04-22-2016
Another one with awk
Code:
ls | awk '
function xmatch() {
  if ( match($0,/[0-9]+/) ) {
    x=substr($0,RSTART,RLENGTH)+0
  } else {
    print ">>"$0
    x=0
  }
}
n {
  xmatch(); if (x) {
    for ( ; ++n<x; ) { printf "%0*d\n", RLENGTH, n }
  }
  next
}
{
  xmatch(); n=x
}
'


Last edited by MadeInGermany; 04-22-2016 at 04:38 PM.. Reason: indention
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Check/print missing number in a consecutive range and remove duplicate numbers

Hi, In an ideal scenario, I will have a listing of db transaction log that gets copied to a DR site and if I have them all, they will be numbered consecutively like below. 1_79811_01234567.arc 1_79812_01234567.arc 1_79813_01234567.arc 1_79814_01234567.arc 1_79815_01234567.arc... (3 Replies)
Discussion started by: newbie_01
3 Replies

2. Shell Programming and Scripting

Need Help in adding sequence number to a file

Hi All , I have a file which contains data(comma separated) in below format : 500,Sourav ,kolkata ,8745775020,700091 505,ram,delhi ,9875645874,600032 510 ,madhu ,mumbai ,5698756430 ,500042 515 ,ramesh ,blore ,8769045601 ,400092 I want to add unique sequence number at the start of each... (7 Replies)
Discussion started by: STCET22
7 Replies

3. Shell Programming and Scripting

How to find a missing file sequence using shell scripting?

Hey guys, I want the below files to be processed with the help of BASH so that i will be able to find the missing file names : PP01674520141228X.gz PP01674620141228X.gz PP01674820141228X.gz PP01674920141228X.gz PP01675420141228X.gz PP01675520141228X.gz PP01676020141228X.gz . . . .... (4 Replies)
Discussion started by: TANUJ
4 Replies

4. Shell Programming and Scripting

Identifying Missing File Sequence

Hi, I have a file which contains few columns and the first column has the file names, and I would like to identify the missing file sequence number form the file and would copy to another file. My files has data in below format. APKRISPSIN320131231201319_0983,1,54,125,... (5 Replies)
Discussion started by: rramkrishnas
5 Replies

5. Shell Programming and Scripting

How to check missing sequence?

I want to listed files every hours and check the missing sequence my file format is CV.020220131430.txt CV.020220131440.txt CV.020220131450.txt CV.ddmmyyhhm.txt how to check if i have missing files in sequence .. thanks (3 Replies)
Discussion started by: before4
3 Replies

6. Shell Programming and Scripting

find common entries and match the number with long sequence and cut that sequence in output

Hi all, I have a file like this ID 3BP5L_HUMAN Reviewed; 393 AA. AC Q7L8J4; Q96FI5; Q9BQH8; Q9C0E3; DT 05-FEB-2008, integrated into UniProtKB/Swiss-Prot. DT 05-JUL-2004, sequence version 1. DT 05-SEP-2012, entry version 71. FT COILED 59 140 ... (1 Reply)
Discussion started by: manigrover
1 Replies

7. Shell Programming and Scripting

How to take the missing sequence Number?

Am using unix aix KSH... I have the files called MMRR0106.DAT MMRR0206.DAT MMRR0406.DAT MMRR0506.DAT MMRR0806.DAT .... ... MMRR3006.DAT MMRR0207.DAT These files are in one dircetory /venky ? I want the output like this ? Missing files are : MMRR0306.DAT MMRR0606.DAT... (7 Replies)
Discussion started by: Venkatesh1
7 Replies

8. Shell Programming and Scripting

Perl : print the sequence number without missing number

Dear Perl users, I need your help to solve my problem below. I want to print the sequence number without missing number within the range. E.g. my sequence number : 1 2 3 4 5 6 7 8 11 12 13 14 my desired output: 1 -8 , 11-14 my code below but still problem with the result: 1 - 14 1 -... (2 Replies)
Discussion started by: mandai
2 Replies

9. Shell Programming and Scripting

join based on line number when one file is missing lines

I have a file that contains 87 lines, each with a set of coordinates (x & y). This file looks like: 1 200.3 -0.3 2 201.7 -0.32 ... 87 200.2 -0.314 I have another file which contains data that was taken at certain of these 87 positions. i.e.: 37 125 42 175 86 142 where the first... (1 Reply)
Discussion started by: jackiev
1 Replies

10. Shell Programming and Scripting

Script to check file sequence

Hi everyone, I need help in creating a script that would check if the file sequence is in order in a particular directory. These are log files that are generated throughout the day. Example of the file name will be, ABC01_YYMMDDHHMM###### (ABC01_0904161829000001) Sometimes the file... (4 Replies)
Discussion started by: kumaran21
4 Replies
Login or Register to Ask a Question