Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Count files between multiple directories

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Count files between multiple directories
# 1  
Old 06-27-2014
Count files between multiple directories
Hi All,

Everyday we will receive 33 files in our source directory(/prd/pk) with the current date. Once our jobs are completed all the 33 files immediately will be moved to backup folder (/prd/pk/backup).

Now, I need to check between source file directory (/prd/pdk) and backup file directory based on date to ensure that all files are received for current date.

If not, I need to display how may files are missed for current date.

Kindly help me how to achive this.

HTML Code:
Ex:
 
Source File Directory:
/prd/pdw
Fileone_20140627
FileTwo_20140627
FileThree_20140627
 
Backup File Directory:
/prd/pdw/Backup
 
Fileone_20140626
FileTwo_20140626
FileThree_20140626
FileFour_20140626
FileFive_20140626
 
Expected Output:
 
FileFour_20140626
FileFive_20140626
# 2  
Old 06-27-2014
RedHat
if you are on a linux system this will definetly work in bash, for unix you will need to test in ksh

Code:
#!/bin/bash

Date=`date "+%Y%m%d"`

for fl in /prd/pdw/Backup/*${Date}*
do
    file=`basename $fl`;
    if [ ! -e "/prd/pdw/${file}" ]
    then
        echo "$file"
    fi
done

# 3  
Old 06-27-2014
It seems more like this would be quicker and give the answer required:-
Code:
#!/bin/ksh
Date=`date "+%Y%m%d"`
Req_count=33

cd /prd/pdw
count=`ls -1d *${Date}*|wc -l`

if [ $count -ne $Req_count ]
then
   ((Shortfall=$Req_count-$count))
   echo "You have only got $count files.  You are $Shortfall short."
   exit 99
fi

Set the value in red as applicable if your needs change.

Finding no files will give you an error message to standard error from the ls and the count will be zero. You could remove the message by changing that line to have:-
Code:
count=`ls -1d *${Date}* 2>/dev/null|wc -l`


I hope that this helps,
Robin
# 4  
Old 06-28-2014
Hello,

I created these files for testing purpose,

Code:
kibou@laptop:~$ ls dirToday/
file1_2014-06-28  file2_2014-06-28
kibou@laptop:~$ ls dirBackup/
file1_2014-06-27  file2_2014-06-27  file3_2014-06-27  file4_2014-06-27

So I've tried something like this,

Code:
kibou@laptop:~$ ls dirToday/ | awk -F"_" '{print $1}' > today.lst
kibou@laptop:~$ ls dirBackup/ | grep -f today.lst 
kibou@laptop:~$ echo "* Missing files *"; ls dirBackup/ | grep -vf today.lst 
* Missing files *
file3_2014-06-27
file4_2014-06-27

Last edited by Kibou; 06-28-2014 at 06:10 AM.. Reason: Small clarification
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compress Files in Multiple Directories

I would like to compress the files in multiple directories. For some reason, it only compress the first directory (/Sanbox/logs1) but not the rest of the other directories ("/Sanbox/logs2" "/Sanbox/logs3" "/Sanbox/logs4" ). Any help would be appreciated. Here's my code: #!/bin/bash... (1 Reply)
Discussion started by: Loc
1 Replies

2. Shell Programming and Scripting

List directories and count files inside

I'm trying to make a script that will list all directories under a selection as well as the number of files in each. I cannot get it to work under a symbolic link. The file structure is: XXX_20131127_001 dir01 (sym link) 2404x912 file.0000.xxx to ... (10 Replies)
Discussion started by: scribling
10 Replies

3. Shell Programming and Scripting

Script to count number of files in directories

Hi All! I would like to have a script that will count the number of files at the top of the hour of soome directories and mail the results to me. I was thinking on : a=`/directory/subdirectory/ | wc -l` echo "/directory/subdirectory :$a" b=`/another_dir/subdir/ | wc -l` echo... (12 Replies)
Discussion started by: fretagi
12 Replies

4. Shell Programming and Scripting

perform 3 awk commands to multiple files in multiple directories

Hi, I have a directory /home/datasets/ which contains a bunch (720) of subdirectories called hour_1/ hour_2/ etc..etc.. in each of these there is a single text file called (hour_1.txt in hour_1/ , hour_2.txt for hour_2/ etc..etc..) and i would like to do some text processing in them. Each of... (20 Replies)
Discussion started by: amarn
20 Replies

5. UNIX for Dummies Questions & Answers

Count of files and directories

Hi I have a jfs2 filesystem named /software in my aix 5.3 box. Please note that there are a lot of subdirectories under /software? I need to know the count of how many total files and directories are present under that mount point /software ? For example if by some commands we find that there... (3 Replies)
Discussion started by: samsungsamsung
3 Replies

6. Shell Programming and Scripting

FTP multiple files from multiple directories

I have multiple files that starts as TRADE_LOG spread across multiple folders in the given structure.. ./dir1/1/TRADE_LOG*.gz ./dir2/10/TRADE_LOG*.gz ./dir11/12/TRADE_LOG*.gz ./dir12/13/TRADE_LOG*.gz when I do ftp uisng mput from the "." dir I am getting the below given error mput... (1 Reply)
Discussion started by: prasperl
1 Replies

7. Shell Programming and Scripting

Count of files in directories

Hi, I have a requirement to find out the count of files in directories. I can do this very well by goind to each directory and then ls -lrt | wc -l. But I need to do it for hundreds of directories/sub-directories. I tried with this - for i in `ls -F | grep '/$'`; do `echo "$i"`| ls -lrt... (2 Replies)
Discussion started by: unx100
2 Replies

8. Shell Programming and Scripting

AWK Script - Count Files In Directories

Hey, I'm very new to AWK and am trying to write a script that counts the number of files in all subdirectories. So, basically, my root has many subdirectories, and each subdirectory has many files. How can I get the total count? I haven't been able to figure out how to loop through the... (1 Reply)
Discussion started by: beefeater267
1 Replies

9. Solaris

Get Count of all files in all the directories

Hello, I am looking for a way to get the TOTAL COUNT of the files present in all directories(sub directories) under the root directory..The files can be of any type viz. txt, doc, html, wav, jpeg etc. If it has an extension, it has to be counted.. I want to run the script from the root directory.... (6 Replies)
Discussion started by: oniondosa
6 Replies

10. UNIX for Dummies Questions & Answers

Search for files in multiple directories

I want to search for a file pattern in more than one directory. How I need to do that? Here is the scenario: I am having a directory structure like the following: /log ...../20051001 ..........ftp_server_20051001.log ..........ftp_down_server.log ..........ftp_up_server.log... (7 Replies)
Discussion started by: ravikirankethe
7 Replies
Login or Register to Ask a Question