Count the number of files copied from source to destination location


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Count the number of files copied from source to destination location
# 1  
Old 07-23-2015
Count the number of files copied from source to destination location

Hi Guys,

how to count number of files successfully copied while coping files from source to destination path

Code:
ex:10 files from source to target location copying
if 8 files copied successfully then 
echo successfully copied=8

failure=2 files

if two files get error to coping files from source to destination directories
then print cause of error file
ex 
if directory not exist 

path not exist in target location when comparing with source path files and destination path target

# 2  
Old 07-23-2015
There's umpteen ways to skin this cat. Any attempt from your side?
# 3  
Old 07-23-2015
hi tried my side...
here my code is
Code:
xm=$(ls -tp | grep $*.xml | head -1)
gawk -F"[\"<>]" ' BEGIN {id=m=mc=ct=" "} /\/Filename/ {id=$3} /\/DivisionCode/{m=$3} /\/ProductCode/ {mc=$3} /\/ContentType/ {ct=$3} /\/sequence/
{print id"\t"m"\t"mc"\t"ct;id=m=mc=ct=" "}' OFS=,  $xm > XmlDataNew.txt
IFS=$'\n'
while read -r line; do
fn=$(echo $line |awk -F "\t" '{print $1}')
dc=$(echo $line |awk -F "\t" '{print $2}')
pc=$(echo $line |awk -F "\t" '{print $3}')
ct=$(echo $line |awk -F "\t" '{print $4}')
src_dir="/home/oracle/arun/IRMS-CM"
dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct"
if [ -d $dest_dir ]; then
for FILE in `ls $src_dir{$fn,*/$fn}`
   do
      cp /home/oracle/arun/IRMS-CM/{$fn,*/$fn} /home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct 
      ((Succ_Cnt++))	  
   done
else
echo "$dest_dir Directory Does not Exist"	
fi
done <<XmlDataNew.txt

here files coping from source to destination location we need to print successfully copied with in number...success copied=10
if failure occurred while coping files from source to destination
ex :failure=5
# 4  
Old 07-23-2015
Well, on first sight there's quite some opportunities in your code sample.
- why use grep (btw, the regex $*.xml is somewhat dubious)? ls -t | head -1 should give the most recent .xml file.
- why define IFS=$'\n'? The file that you read has , as the field separator (OFS=,), or <TAB>, being explicitly printed.
- why not read fn dc pc ct REST from the input file and drop the four assignment lines? You'd need to define IFS corrrectly, then.
- why not define src_dir outside the while loop as it doesn't contain variables?
- why the for loop as the loop variable FILE isn't used anywhere inside the loop?
- why not use the -n (no-clobber) option to cp, as copying multiple files with identical file name (from different source dirs) to one single destination dir will overwrite the target files.
- why increment the Succ_Cnt without knowing if the cp was a success?
- why not create the dest_dir if it doesn't exist?
- why use a here document ( << redirection operator), which will fail, instead redirect stdin ( < operator)?

---------- Post updated at 12:26 ---------- Previous update was at 12:15 ----------

You may want to give this a try, although untested and needing some polishing:
Code:
gawk -F"[\"<>]" ' BEGIN {id=m=mc=ct=" "} /\/Filename/ {id=$3} /\/DivisionCode/{m=$3} /\/ProductCode/ {mc=$3} /\/ContentType/ {ct=$3} /\/sequence/
{print id"\t"m"\t"mc"\t"ct;id=m=mc=ct=" "}' OFS=,  $(ls -t *.xml | head -1) > XmlDataNew.txt
src_dir="/home/oracle/arun/IRMS-CM"    
while read -r fn dc pc ct REST
  do dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct"
     if [ ! -d "$dest_dir" ]
       then echo "$dest_dir Directory Does not Exist; creating ..."     
            mkdir "$dest_dir"
     fi

     for FILE in $src_dir{$fn,*/$fn}     
       do if [ ! -e "$dest_dir/$FILE" ]
            then cp -n "$FILE" "$dest_dir" && ((Succ_Cnt++)) || ((Err_Cnt++))
            else ((Err_Cnt++))
          fi
       done
  done <XmlDataNew.txt

# 5  
Old 07-23-2015
hi rudic
actually my intention....i explained my necessity...here step by step

step1. reading latest xml file from source and print the values in text.file
sample text.file out put
Code:
filename         divisioncode      product code      contenttype
aldorzum.doc      us                    vimzim           template
alozyme.doc       ca                                          standard
aser.doc           sa                   encloser         template

step2;;reading text.file and copy the files from source to destination path
step3;check directory exist or not exist in target location
step4;if directory exist then copy the files from source to destination then count how many files copied according to text.file column $1 wise line by line
if 10 files copied out of 15
then print sccessfully copied =10
step5;if directory not exist (no need create directory)
just print how many files not copied successfully if 5 files failure to copy
then print failure =5
cause of failure; following destination folder or directory path not exist will print in to another text2.file

Code:
example i need to print text2.file 
 successfully copied=10
failure copied =5
cause of failure 
/home/oracle/arun/livelink/irms-cm/us/vimizim/product directory does not exist
/home/oracle/arun/livelink/irms-cm/us/vimizim/enclosure directory does not exist
/home/oracle/arun/livelink/irms-cm/us/aldurazyme/product directory does not exist
/home/oracle/arun/livelink/irms-cm/us/ /template directory does not exist
/home/oracle/arun/livelink/irms-cm/us/naglazyme/product directory does not exist

step6;finally text2.txt file will attached in mail and send notification to me
using mailx or mail command with "subject status"
if all files copied success then subject status else status error
if copied 14 files out of 15 then we need to get error status
subject: irms-cm document import into livelink mm/dd/yyyy (status = ______)
Code:
if all files copied success
subject: irms-cm document import into livelink 23/07/2015 (status = success)
else
subject: irms-cm document import into livelink 23/07/2015 (status = error)
with attached file text2.txt
with require info

step7; after review the attached file we will create destination folders manually

Last edited by sravanreddy; 07-23-2015 at 08:01 AM..
# 6  
Old 07-23-2015
Hello sravanreddy,

It's a request.

i- Please use code tags for commands/codes/inputs you are using in your posts as per forum rules.
ii- Go through the forum rules in following link for a better understanding please.
https://www.unix.com/misc.php?do=cfrules
iii- Please do not use capital letters in posts it may be taken as offense. So request you to not to repeat this.

We all are here to help and learn from each other. We should respect each other as each one has their own identity and working/learning style.


Thanks,
R. Singh

Last edited by RavinderSingh13; 07-23-2015 at 08:36 AM..
# 7  
Old 07-23-2015
thank you...RavinderSingh13

actually i'm new to this forum i will follow your instructions
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Open ports from source to destination

Is there a way to find out all the ports open between source IP & destination IP in any way ? (12 Replies)
Discussion started by: UnknownGuy
12 Replies

2. UNIX for Dummies Questions & Answers

How to move gz files from one source directory to destination directory?

Hi All, Daily i am doing the house keeping in one of my server and manually moving the files which were older than 90 days and moving to destination folder. using the find command . Could you please assist me how to put the automation using the shell script . ... (11 Replies)
Discussion started by: venkat918
11 Replies

3. Shell Programming and Scripting

Error files count while coping files from source to destination locaton as well count success full

hi All, Any one answer my requirement. I have source location src_dir="/home/oracle/arun/IRMS-CM" My Target location dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct" my source text files check with below example.text file content $fn "\t" $dc "\t" $pc "\t" ... (3 Replies)
Discussion started by: sravanreddy
3 Replies

4. UNIX for Dummies Questions & Answers

How to count number files successfully copied from source to target location?

Hi Guys, how to count number of files successfully copied while coping files from source to destination path ex:10 files from source to target location copying if 8 files copied successfully then echo successfully copied=8 failure=2 files if two files get error to coping files... (2 Replies)
Discussion started by: sravanreddy
2 Replies

5. Post Here to Contact Site Administrators and Moderators

How to count successfully copy files source to target location with check directory in Linux?

Hi guys...please any one help me .... how to copy files from source to target location if 5 files copied successfully out of 10 files then implement success=10 and if remaining 5 files not copied successfully then count error=5 how to implement this condition with in loop i need code linux... (0 Replies)
Discussion started by: sravanreddy
0 Replies

6. Shell Programming and Scripting

How to count number of files in directory and write to new file with number of files and their name?

Hi! I just want to count number of files in a directory, and write to new text file, with number of files and their name output should look like this,, assume that below one is a new file created by script Number of files in directory = 25 1. a.txt 2. abc.txt 3. asd.dat... (20 Replies)
Discussion started by: Akshay Hegde
20 Replies

7. Linux

rpmbuild, how to specify a different source and destination path for files

I'd like to specify a different build and deployment path for files, by default the same path is used for both build and install, I wasn't able to find a way to make these different. With Solaris pkgadd, one can specify different paths in prototype, so I would assume something like that is possible... (0 Replies)
Discussion started by: tiburblium
0 Replies

8. Shell Programming and Scripting

Move all files from source to destination directory based on the filename

Move all files starting with a specific name to different directory. This shell script program should have three parameters File Name Source Directory Destination Directory User should be able to enter ‘AB_CD*' in file name parameter. In this case all the files starting with AB_CD will... (1 Reply)
Discussion started by: chetancrsp18
1 Replies

9. Shell Programming and Scripting

Moving extremely large number of files to destination

Hello Techies, m here with a small issue. Have read out all the relavent threads on unix.com but it was not so good sol. for me. It's a simple & known issue. I want to move lots of files to dest folder. Actually I want to pick up 1 year older files, but that is even taking lots of... (16 Replies)
Discussion started by: kedar.mehta
16 Replies

10. Shell Programming and Scripting

Count total unique destination for source

Hi, need help how to count unique destination for the source ip. The file is contains 4 number of fields. Example of the file is here src ip dest ip #of flows total bytes 192.168.6.0 88.0.33.2 12 128 192.168.6.0 88.0.33.2 1 168 192.168.6.0 ... (5 Replies)
Discussion started by: new_buddy
5 Replies
Login or Register to Ask a Question