Hi Experts,
I have written the below script to check the missing files based on the date in the file name from current date to in a given interval of days.
In the file names we have dates along with some name. ex:jera_sit_2017-04-25-150325.txt. The below script is working fine if we have only one date in the filename.
It's not working if we have more than one date in the file name for example: jera_sit_2017-04-28-150325_2017-04-29-112227.txt My script is considering as two files one is for 2017-04-28 and 2017-04-29.
Since the logic is ls -l ${path}/*${start}* It should consider file is existed only for the last date in the file name i.e. 2017-04-29.
Could you please help me with the logic.
Code:
#!/bin/ksh
file_path=/bin/daily/dtefiles
file_format=XXXX_XXX_YYYY-MM-DD-HHIISS.TXT
check_mode=1
back_days_hr_min_hr_min=5
#checking before . extension
YearFormat=$(echo $file_format | sed 's/X//g;s/x//g;s/_*$//g;s/^_*//g' | awk 'BEGIN{FS=".";}{for (i = 1; i <= NF; i++){if ( $i ~ /YY/ ){print $i;}}}' | sed 's/X//g;s/x//g;s/_*$//g;s/^_*//g' | head -1)
YearFormat_count=$(echo $YearFormat | wc -l)
echo "YearFormat-$YearFormat"
if [[ $YearFormat_count -lt 1 ]]; then
echo "enter proper year format"
exit 0
fi
#exit is missing here
month_count=0;y_count=0;d_count=0;hour_count=0;min_count=0;sec_count=0
for (( i=0; i<${#YearFormat}; i++ )); do
year=$(echo "${YearFormat:$i:1}")
if [[ $year == 'Y' ]];then
y_count=`expr $y_count + 1`
fi
if [[ $year == 'M' ]];then
month_count=`expr $month_count + 1`
fi
if [[ $year == 'D' ]];then
d_count=`expr $d_count + 1`
fi
if [[ $year == 'H' ]];then
hour_count=`expr $hour_count + 1`
fi
if [[ $year == 'I' ]];then
min_count=`expr $min_count + 1`
fi
if [[ $year == 'S' ]];then
sec_count=`expr $sec_count + 1`
fi
done
if [[ $month_count -ne 0 && $month_count -ne 2 ]] ; then
echo "enter correct month format"
exit 0
fi
if [[ $d_count -ne 0 && $d_count -ne 2 ]]; then
echo "enter correct date format"
exit 0
fi
if [[ $hour_count -ne 0 && $hour_count -ne 2 ]]; then
echo "enter correct hour format"
exit 0
fi
if [[ $min_count -ne 0 && $min_count -ne 2 ]]; then
echo "enter correct min format"
exit 0
fi
if [[ $sec_count -ne 0 && $sec_count -ne 2 ]]; then
echo "enter corrrect sec format"
exit 0
fi
#Remove the duplicates
cleandateFormat=$(echo $YearFormat | tr -s 'A-Z') #YmD_HMS
echo $cleandateFormat
if [[ $y_count -eq 2 ]]; then
cleandateFormat=$(echo $cleandateFormat | sed 's/Y/y/g;s/M/m/g;s/D/d/g;s/I/M/g')
elif [[ $y_count -eq 4 ]]; then
cleandateFormat=$(echo $cleandateFormat | sed 's/M/m/g;s/D/d/g;s/I/M/g')
else
echo "enter correct year format"
exit 0
fi
#maintain format including special character as seperator
finalFormat=""
for (( i=0; i<${#cleandateFormat}; i++ )); do
f_year=$(echo "${cleandateFormat:$i:1}")
echo "char : $f_year"
if [[ $f_year == [a-zA-Z] ]];then
temp=$(echo $f_year | sed 's/^/%/g')
finalFormat="$finalFormat$temp"
echo "IF : $finalFormat"
else
finalFormat="$finalFormat$f_year"
fi
done
#checking missing files
if [[ $check_mode -eq 1 ]]
then
missing_count=0
day_count=0
while [[ $back_days_hr_min_hr_min -ne 0 ]]; do
finalFormat=$(echo "$finalFormat" | sed -r 's/[HMS]+//g;s/%*$//g;s/-*$//g;s/_*$//g')
start=`date +${finalFormat} -d "$back_days_hr_min_hr_min day ago"`
IFS=$','
for path in ${file_path}; do
count=$(ls -l ${path}/*${start}* 2>/dev/null | wc -l)
if [[ $count -gt 0 ]]
then
break
fi
done
unset IFS
if [[ $count -eq 0 ]]
then
missing_count=`expr $missing_count + 1`
file_name="${start}"
printf "$file_name\n"
fi
back_days_hr_min_hr_min=`expr $back_days_hr_min_hr_min - 1`
done >missingfiles.txt
fi
Thanks in advance.
---------- Post updated at 05:26 AM ---------- Previous update was at 02:18 AM ----------
Hi All,
Could you please help me.
Thanks in advance.
---------- Post updated 05-04-17 at 12:20 AM ---------- Previous update was 05-03-17 at 05:26 AM ----------
Hi All,
Could any body please help me.
Thanks
Moderator's Comments:
Bumping up posts or double posting is not permitted in these forums.
Please read the rules, which you agreed to when you registered, if you have not already done so.
You may receive an infraction for this. If so, don't worry, just try to follow the rules more carefully. The infraction will expire in the near future
If after more than a day nobody can answer your question, you should consider rephrasing it to provide more clarity and understanding and help people help you.
Please become accustomed to provide decent context info of your problem.
It is always helpful to support a request with system info like OS and shell, related environment (variables, options), preferred tools, and adequate (representative) sample input and desired output data and the logics connecting the two, to avoid ambiguities and keep people from guessing.
Hi Rudic,
I have explained the requirement with expected result.
My requirement:
I want to find the missing files based on the date in the file name from current date to in a given interval of days.
Code:
Case1 : Want to find missing files in 10 days with one date in the file name.
For example:
I have the below files in the directory /bin/daily/dtefiles
jera_sit_2017-04-24-150325.txt
jera_sit_2017-04-25-150325.txt
jera_sit_2017-04-26-141232.txt
jera_sit_2017-04-29-122344.txt
jera_sit_2017-05-02-122344.txt
jera_sit_2017-05-03-122344.txt
In this case the missing files are for the dates 2017-04-27,2017-04-28,2017-04-30,2017-05-01.
The script I posted is working fine for case1.
Code:
Case2:Want to find missing files in 10 days with more than one date in the file name.
For example:
I have the below files in the directory /bin/daily/msn_files
jera_msn_2017-04-28-150325_2017-04-29-112227_2017-04-29-112227.txt
jera_msn_2017-04-24-150325_2017-04-24-112227_2017-04-25-112227.txt
jera_msn_2017-04-24-150325_2017-04-26-112227_2017-04-26-112227.txt
jera_msn_2017-04-25-150325_2017-04-26-112227_2017-04-27-112227.txt
jera_msn_2017-04-30-150325_2017-04-30-112227_2017-05-01-112227.txt
In the above files names we have more than one date. It should consider only last date in the file name.
It should not consider other dates.
In this case the missing files are for the dates 2017-04-24,2017-04-28,2017-04-30,2017-05-02.
My script is not working for case2 .
Code:
For case 1 Parameter values are.
file_path=/bin/daily/dtefiles
file_format=XXXX_XXX_YYYY-MM-DD-HHIISS.TXT
check_mode=1
back_days_hr_min_hr_min=10
For case 2 Parameter values are.
file_path=/bin/daily/dtefiles
file_format=XXXX_XXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_YYYY-MM-DD-HHIISS.TXT
check_mode=1
back_days_hr_min_hr_min=10
Hi Rudic,
Thanks for your script. However this is not serve my purpose.
I Want to find missing files in 10 days with more than one date in the file name.
I have the below files in the directory /bin/daily/msn_files
In the above files names we have more than one date. It should consider only last date in the file name.
It should not consider other dates.
In this case the missing files are for the dates
Did you grasp what was done in post#4? The "only last date in the file name" for ALL files was presented to you for further processing. e.g finding missing files as you do in your script.
Hi Rudic,
In my script start will give date based on the parameter value of back_days_hr_min_hr_min.
For example if back_days_hr_min_hr_min=10
then start would be from yesterday 10 days back i.e.2017-04-28 then it will loop with one incremental date.
Code:
start will be 2017-04-28,2017-04-29,2017-04-30 up to 2017-05-07.
In my script I am checking for file is existed or not with the dates giving by start parameter value.
Code:
ls -l ${path}/*${start}*
since in the file name we have three dates in the file name jera_msn_2017-04-28-150325_2017-04-29-112227_2017-04-29-112227.txt.
Script will check
Code:
ls -l *2017-04-28*
ls -l *2017-04-29*
Hence its showing file is existed for 2017-04-28 and 2017-04-29.
But it should only last date and say file is existed for 2017-04-29.
Please help me.
Thanks in advance.
Hi,
I have a file which contains few columns and the first column has the file names, and I would like to identify the missing file sequence number form the file and would copy to another file. My files has data in below format.
APKRISPSIN320131231201319_0983,1,54,125,... (5 Replies)
I have the following method to identify missing primary keys in a MySQL database schema:
USE information_schema;
SELECT xx.table_name
FROM (SELECT table_name, COUNT(*) FROM columns WHERE table_schema = @myDB GROUP BY table_name, column_key) xx
GROUP BY xx.table_name
HAVING COUNT(*) = 1;
I... (0 Replies)
I have an HP-UX server that runs a script each night. The script connects to an SFTP server and downloads all xml files (if any are present) from a certain folder, and then deletes the files from the SFTP server. So sometimes it will download a new file, sometimes it will download 2 or 3 new... (4 Replies)
Hi Am Using Unix Ksh ...
I have a Table called date
select * from date ;
Date
01/02/2013
06/02/2013
I need the output as
Missing Date
01/02/2013
02/02/2013
03/02/2013
04/02/2013
05/02/2013
06/02/2013 (2 Replies)
Hi,
A script is running for multiple databases so data is also being populated for multiple DBs in a.txt file.
I need to rename this file once all the data is populated. Kindly suggest me How can I check once file is populated completely before renaming?
Thanks in advance. (3 Replies)
Hi All,
I am trying to print the dates that falls between 2 date variables into a file. Here is the example.
$BUS_DATE =20120616
$SUB_DATE=20120613
Output to file abc.txt should be : 20120613,20120614,120120615,20120616
Can you pls help me accomplish this in LINUX.
Thanks... (5 Replies)
Hello,
I am interested in finding and identifying suffixes for Indian names through an awk script or a perl program. Suffixes normally are found at the end of a word as is shown in the sample given below.
What I need is a perl script which will identify suffixes of a defined lenght to be given in... (4 Replies)
Hi all,
My objective is to find out the charset using which a file is encoded. (The OS is SunOs)
I have set NLS_LANG to AR8MSWIN1256 and spooled the file.
When viewed the file using vi, I saw the following
\307\341\321\355\307\326
I then inserted the line containing these codes in a... (3 Replies)
I have a huge file with 20 fileds in each record and each field is seperated by "|". If i want to get all the reocrds that have 18th or for that matter any filed as null how can i do it? Please let me know (3 Replies)
05-04-2017
