How to find list of missing files based on the file format?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to find list of missing files based on the file format?
# 8  
Old 05-19-2017
Hi Don,

Pleasde find the details as below.

What operating system (including release number) are you using?
Code:
Linux dev.voda.mp.com 2.6.18-400.1.1.el5 #1 SMP Sun Dec 14 06:01:17 EST 2014 x86_64 x86_64 x86_64 GNU/Linux

What shell (including version number) are you using?
Code:
ksh

Do you have a ksh (version 93u+ or later) that I can use instead of whatever shell you're using to provide an example?
Code:
version         sh (AT&T Research) 93u+ 2010-06-21

Please help me to convert to the user file format to date format string as below.

Code:
adv_voda_%Y-%m-%d-??????_????-??-??-??????_????-??-??-??????.txt

Thanks in advance

Last edited by nalu; 05-19-2017 at 04:37 AM..
# 9  
Old 05-19-2017
The following script is a quick and dirty demonstration producing results similar to what your script seemed to be trying to do. It takes the number of days to examine and the file pattern as command line arguments and prints the date in %Y-%m-%d format for dates that had no file matching the given pattern. It only uses ksh built-ins (without invoking awk, date, expr, head, sed, or wc). The missing dates are printed to standard output from this script since the redirection to missingfiles.txt is commented out.
Code:
#!/bin/ksh
IAm=${0##*/}

check_mode=1
count=0

if [[ $# -ne 2 ]]
then	printf 'Usage: %s interval filename_format\n' "$IAm" >&2
	exit 1
fi

interval=$1
file_format=$2

date_pattern=${file_format//YYYY/%Y}
date_pattern=${date_pattern//YY/%y}
if [[ "$date_pattern" == "$file_format" ]]
then	printf '%s: Invalid date format: No "YYYY" or "YY" found:\n\t"%s"\n' \
	    "$IAm" "$file_format" >&2
	exit 2
fi
date_pattern=${date_pattern//MM/%m}
date_pattern=${date_pattern//DD/%d}
date_pattern=${date_pattern//HH/%H}
date_pattern=${date_pattern//II/%M}
date_pattern=${date_pattern//SS/%S}

printf '%s: Processing interval:%d & date_pattern:\n\t"%s"\nfrom file_format:\n\t"%s"\n' \
    "$IAm" "$interval" "$date_pattern" "$file_format"

if [[ $check_mode -eq 1 ]]
then	while [[ $interval -ne 0 ]]
	do	file_pattern=$(printf "%($date_pattern)T\n" "$interval day ago")
		for path in $file_pattern
		do	if [[ -f $path ]]
			then	count=1
				break
			fi
		done
		if [[ $count -eq 0 ]]
		then	printf '%(%Y-%m-%d)T\n' "$interval day ago"
		else	count=0
		fi
		interval=$((interval - 1))
	done # >missingfiles.txt
fi

In a directory containing the files:
Code:
total 32
-rw-r--r--  1 dwc  staff     0 May 18 10:36 2017-05-10.txt
-rw-r--r--  1 dwc  staff     0 May 18 10:36 2017-05-11
-rw-r--r--  1 dwc  staff     0 May 18 10:36 2017-05-12
-rw-r--r--  1 dwc  staff     0 May 18 10:36 adb_voda_2017-05-10.txt
-rw-r--r--  1 dwc  staff     0 May 18 10:36 adb_voda_2017-05-11-150325_2017-05-10-112227_2017-05-13-112227.txt
-rw-r--r--  1 dwc  staff     0 May 18 10:36 adb_voda_2017-05-12-150325_2017-05-12-112227_2017-05-11-112227
-rw-r--r--  1 dwc  staff     0 May 18 10:36 adb_voda_2017-05-14-150325_2017-05-11-112227_2017-05-10-112227.txt
-rw-r--r--  1 dwc  staff     0 May 18 10:36 adb_voda_2017-05-16_2017-04-30_2017-05-01.txt
-rw-r--r--  1 dwc  staff     0 May 18 10:36 adb_voda_20170510.txt
-rw-r--r--  1 dwc  staff     0 May 18 10:36 asr_spir_2017-05-10-150325_2017-05-10-112227_2017-05-13-112227.txt
-rwxr-xr-x  1 dwc  staff   704 May 19 04:04 driver
-rw-r--r--  1 dwc  staff  6471 May 18 10:40 problem
-rwxr-xr-x  1 dwc  staff  1108 May 19 03:46 tester

where the above script is named tester, the command:
Code:
./tester 10 "adb_voda_YYYY-MM-DD-??????_????-??-??-??????_????-??-??-??????.txt"

when run on May 19, 2017 produces the output:
Code:
tester: Processing interval:10 & date_pattern:
	"adb_voda_%Y-%m-%d-??????_????-??-??-??????_????-??-??-??????.txt"
from file_format:
	"adb_voda_YYYY-MM-DD-??????_????-??-??-??????_????-??-??-??????.txt"
2017-05-09
2017-05-10
2017-05-12
2017-05-13
2017-05-15
2017-05-16
2017-05-17
2017-05-18

and the command:
Code:
./tester 15 "YYYY-MM-DD"

produces the output:
Code:
tester: Processing interval:15 & date_pattern:
	"%Y-%m-%d"
from file_format:
	"YYYY-MM-DD"
2017-05-04
2017-05-05
2017-05-06
2017-05-07
2017-05-08
2017-05-09
2017-05-10
2017-05-13
2017-05-14
2017-05-15
2017-05-16
2017-05-17
2017-05-18

and the command:
Code:
./tester 10 "adb_voda_????-??-??-??????_????-??-??-??????_YYYY-MM-DD-??????.txt"

produces the output:
Code:
tester: Processing interval:10 & date_pattern:
	"adb_voda_????-??-??-??????_????-??-??-??????_%Y-%m-%d-??????.txt"
from file_format:
	"adb_voda_????-??-??-??????_????-??-??-??????_YYYY-MM-DD-??????.txt"
2017-05-09
2017-05-11
2017-05-12
2017-05-14
2017-05-15
2017-05-16
2017-05-17
2017-05-18

Hopefully, you can modify this to get something that works for you.
# 10  
Old 05-24-2017
Hi Don,

Thanks a lot.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

List the files after sorting based on file content

Hi, I have two pipe separated files as below: head -3 file1.txt "HD"|"Nov 11 2016 4:08AM"|"0000000018" "DT"|"240350264"|"56432" "DT"|"240350264"|"56432" head -3 file2.txt "HD"|"Nov 15 2016 2:18AM"|"0000000019" "DT"|"240350264"|"56432" "DT"|"240350264"|"56432" I want to list the... (6 Replies)
Discussion started by: Prasannag87
6 Replies

2. Shell Programming and Scripting

Find list of files missing read & execute permission

Hi, I'm writing a post-upgrade script and I want to find which files don't have read and execute to everyone. I can run a find . ! -perm, but then I have to use a list of the possible permissions (777,775, 755 etc). Is there a more elegant solution? Thanks (2 Replies)
Discussion started by: Catullus
2 Replies

3. Shell Programming and Scripting

I have this list of files . Now I will have to pick the latest file based on some condition

3679 Jul 21 23:59 belk_rpo_error_**po9324892**_07212014.log 0 Jul 22 23:59 belk_rpo_error_**po9324892**_07222014.log 3679 Jul 23 23:59 belk_rpo_error_**po9324892**_07232014.log 22 Jul 22 06:30 belk_rpo_error_**po9324267**_07012014.log 0 Jul 20 05:50... (5 Replies)
Discussion started by: LoneRanger
5 Replies

4. Shell Programming and Scripting

How to list the files based on the modification time using the find command?

Hi All, I need to list the files based modification time of the files from a directory, I cannot use "ls -t" as there are lot of files, which "ls" command cannot handle. New files will land there daily. So iam looking for an alternative through "find"command. All suggestions are welcomed. ... (6 Replies)
Discussion started by: Kesavan
6 Replies

5. Shell Programming and Scripting

Find files older than X with a weird file format

I have an issue with a korn shell script that I am writing. The script parses through a configuration file which lists a heap of path/directories for some files which need to be FTP'd. Now the script needs to check whether there are any files which have not been processed and are X minutes old. ... (2 Replies)
Discussion started by: MickAAA
2 Replies

6. Shell Programming and Scripting

Find missing files from a list

counter=0; while read line; do ] && let counter=counter+1; done < input_file.txt echo $counter The above code is reading a file line by line and checking whether the filenames mentioned in the file exist or not . At present the o/p is value of counter I want to echo out the name of... (5 Replies)
Discussion started by: ultimatix
5 Replies

7. Shell Programming and Scripting

KSH: Opening Files based on a file list

I'd like to grep files for key words using korn shell, and compile the actual contents (not just file name) of those files that contain a combination of those grepped key words into one repository file for reference. However, I'm stuck at the combining part. Here's what I have thus far: egrep... (5 Replies)
Discussion started by: drumminfool91
5 Replies

8. Shell Programming and Scripting

Bash snippet to find files based on a text file?

Evening all. I'm having a terrible time with a script I've been working on for a few days now... Say I have a text file named top10song.tm2, with the following in it: kernkraft 400 Imagine i kissed a girl Thriller animals hallelujah paint it black psychosocial Oi to the world... (14 Replies)
Discussion started by: DJ Charlie
14 Replies

9. Shell Programming and Scripting

create diffrent files based on other file and parameters list

I would like ot create shell script/ bash to create diffrent files based on a file and parameters list. Here is the detail example: I have a textfile and four static parameter files (having ‘?'). mainfile.txt has below records (this count may be more than 50) A200001 A200101 B200001... (9 Replies)
Discussion started by: raghav525
9 Replies

10. Shell Programming and Scripting

Help with find command and list in a long format each found file

The purpose of those comands are to find the newest file in a directory acvrdind to system date, and it has to be recursively found in each directory. The problem is that i want to list in a long format every found file, but the commands i use produce unexpected results ,so the output lists in a... (5 Replies)
Discussion started by: alexcol
5 Replies
Login or Register to Ask a Question