Count the number of files copied from source to destination location


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Count the number of files copied from source to destination location
# 15  
Old 07-31-2015
Sorry, I don't get the logics. Please describe every single step, provide more examples, and take more care to your orthography. Run this
Code:
src_dir="/home/oracle/arun/IRMS-CM"
while read fn dc pc ct REST
  do ls "$src_dir"/{$fn,*/$fn}
     dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct"
     echo $dest_dir
     ls -d "$dest_dir"
  done <example.txt

(or example.text?) and post the result.
# 16  
Old 07-31-2015
above code not correct manner...

Code:
while coping files from source path of root folder files to destination path of root folders
if target folder exist then print error count file 
if files copied successfully then count successfully copied

note must check once files present folder or with in folder of r0ot structure not count two times if target folder not exist or file not exist from source location count only once
one file if not copied successfully

Any body help i have urgent requirement
# 17  
Old 07-31-2015
I was not saying that code snippet was a solution to your problem. It is meant to find out what your problem really is. If you don't give more details, I won't be in a position to help.
# 18  
Old 07-31-2015
check my code here complete code posted here
Code:
#cnt_file="/home/oracle/arun/IRMS-CM/Control_File.txt"
#while [ -f cnt_file ];
#do
xm=$(ls -tp | grep $*.xml | head -1)
gawk -F"[\"<>]" ' BEGIN {fn=dc=pc=ct=" "} /\/Filename/ {fn=$3} /\/DivisionCode/{dc=$3} /\/ProductCode/ {pc=$3} /\/ContentType/ {ct=$3}
 /\/sequence/ {print fn"\t"dc"\t"pc"\t"ct;fn=dc=pc=ct=" "}' OFS=, $xm >XmlDataNew.txt
cp /dev/null Test4.txt
IFS=$'\n'
while read -r line; do
fn=$(echo $line |awk -F "\t" '{print $1}')
dc=$(echo $line |awk -F "\t" '{print $2}')
pc=$(echo $line |awk -F "\t" '{print $3}')
ct=$(echo $line |awk -F "\t" '{print $4}')
src_dir="/home/oracle/arun/IRMS-CM"
dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct"
if [ -d $dest_dir ]; then
for FILE in $src_dir/{$fn,*/$fn}
      do
if [ -f $FILE ];then
    cp $FILE $dest_dir && ((Succ_Cnt++)) || ((Err_Cnt1++))
	echo
fi    
done
else
echo " $dest_dir  directory Does Not exists!" >> Test4.txt
Err_Cnt=$(wc -l<Test4.txt)
fi
done< XmlDataNew.txt   
err=$[$Err_Cnt1+$Err_Cnt]
echo $err
if $err=0
then 
echo "Status = Success.
     $Succ_Cnt files added/updated." | mailx -s "IRMS-CM Document Import into LiveLink $(date +'%m/%d/%Y') (Status = Success)" xxxxxxx@gmail.com
else
echo "Status = Error.
      $Succ_Cnt files added/updated.
      $err are failed to copy" | mailx -s "IRMS-CM Document Import into LiveLink $(date +'%m/%d/%Y') (Status = Error)"  -a "/home/oracle/arun/IRMS-CM/Test4.txt"   xxxxxxx@gmail.com 
  fi

above code developed with my spurce...
here problem is
while coping files from source to destination path if $pc folder not exist then check $ct folder exist or not exist and copy the related file to $ct folder.
second problem is
Code:
if files copied successfully  count number files copied successfully
& if any files not copied count number of files not copied successfully

# 19  
Old 08-01-2015
Several of us are trying to help you in this thread, but when we ask questions that will help us help you you show us a different copy of your code instead of answering the questions. We haven't seen your input files. We haven't seen the filesystem hierarchy you're using or the files you're trying to copy and whether or not the destination directory already contains files with those names. You have two different counts of successful and failed copies and have refused to show us why this matters in the output you want to produce. You have not explained why non-existent destination directories should be treated as failed copies instead of creating the missing directories.

I am guessing that you are using some type of Linux system (since you're using gawk instead of awk), but, of course, gawk can be installed on non-Linux systems). And, you haven't told us what shell you're using.

You are using $* in your code, but you have never given us any indication of what operands you are passing to your script when you invoke it. (So there is no way for us to understand what you're trying to do nor for us to guess at how your code is going to behave.)

Please help us help you!

Please answer the following questions. Without answers to these questions, I don't think we're going to be able to help you get the results you want.
  1. What operating system are you using?
  2. What shell are you using?
  3. What is the name of your shell script?
  4. What arguments do you pass to your shell script when you invoke it?
  5. What output do you get from your shell script?
  6. What output do you want to get from your shell script?
  7. Have you tried tracing your script as you run it? If not, why not? Examining the trace output from a shell script is usually an easy way to see where things are going wrong and identify areas that need to be fixed! Add set -xv to your script and show us the tracing output you get when you run your script.
  8. After running your script what are the contents of the file XmlDataNew.txt?
  9. After running your script what are the contents of the file Test4.txt?
  10. What are the contents of the .xml file that you processed with this invocation of your shell script?
  11. Assuming that you are using a recent version of bash or ksh as your shell; what output do you get if you run the script RudiC provided in post #15 in this thread?
# 20  
Old 08-01-2015
Thanks for Replay Don Cragun,

My answers for your questions;
1. Operating system windows 7 and installed virtual machine with linux version on operating system, then we used putty terminal for developing shell script.
2.Gawk command used instead awk command beacause putty terminal take gawk command

My requirement is ...following steps step by step...
Code:
1.Explore more on how to read an xml file using linux shell script.  
2. Read and split the content of xml file for source and target by file and location.
3. store the values in text file below manner
$fn      "\t"   $dc     "\t"     $pc     "\t"  $ct
file1.doc       US             SANZA         ENCLOSRE
FILE2.DOC    CA                                SUPER
FILE3.DOC    DA              SLIDE          SAVE
FILE4.DOC    SA                                 DOC
file5.doc       IZ                STANDS       SRAV
file6.doc        US                                SRAVAS
then source file are in folder and with in folders some files from source location
copy the files to target location as root folders
ex;
file1.doc copy to enclosure folder
as well as 
file2.doc copy Super folder
file5.doc copy to srav folder(if folder srav not exist at target location just print /targetpath/srav not exist print in another textfile all the failed files info)
as well count if files copied to destination path ,count successfully copied files & failed files count and print failed file reason in another text file 
ex;if files 6 from source
successfully copied =4
failed to copy=2
failed cause print in test4.txt file
/destination/srav/ (if folder folder not exist at destination location
/sourcepath/file4.doc (if file not exist at source location )
cause of above reasons files not copied and failed to copy
print failed files=2 with number count and cause failure issue in TEST4.txt file
and if files coped successfully just print number of files copied successfully only no need to print path loactions

no need to create destination path...after getting test4.txt file we will create folders manually

source path folders ex;/sourcepath/folders
US(FOLDER)
|
|---SANZA(FOLDER)
| |
| -------file1.doc
|
--------file6.doc
so files are from source location info

Below script is working almost but problem is
while error count counted one files check with folder and with in folder shows
if one file not exist source location it shows failed count=2(instead one)
i need to print success count and failed count
if all files coped successfully count success=6
and faild cunt=0
Code:
#cnt_file="/home/oracle/arun/IRMS-CM/Control_File.txt"
#while [ -f cnt_file ];
#do
xm=$(ls -tp | grep $*.xml | head -1)
gawk -F"[\"<>]" ' BEGIN {fn=dc=pc=ct=" "} /\/Filename/ {fn=$3} /\/DivisionCode/{dc=$3} /\/ProductCode/ {pc=$3} /\/ContentType/ {ct=$3}
 /\/sequence/ {print fn"\t"dc"\t"pc"\t"ct;fn=dc=pc=ct=" "}' OFS=, $xm >XmlDataNew.txt
cp /dev/null Test4.txt
IFS=$'\n'
while read -r line; do
fn=$(echo $line |awk -F "\t" '{print $1}')
dc=$(echo $line |awk -F "\t" '{print $2}')
pc=$(echo $line |awk -F "\t" '{print $3}')
ct=$(echo $line |awk -F "\t" '{print $4}')
src_dir="/home/oracle/arun/IRMS-CM"
dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct"
if [ -d $dest_dir ]; then
for FILE in $src_dir/{$fn,*/$fn}
      do
if [ -f $FILE ];then
    cp $FILE $dest_dir && ((Succ_Cnt++)) || ((Err_Cnt1++))
fi    
done
else
echo " $dest_dir  directory Does Not exists!" >> Test4.txt
Err_Cnt=$(wc -l<Test4.txt)
fi
done< XmlDataNew.txt   
err=$[$Err_Cnt1+$Err_Cnt]
echo $err
if $err=0
then 
echo "Status = Success.
     $Succ_Cnt files added/updated." | mailx -s "IRMS-CM Document Import into LiveLink $(date +'%m/%d/%Y') (Status = Success)" arunasaagi81@gmail.com
else
echo "Status = Error.
      $Succ_Cnt files added/updated.
      $err are failed to copy" | mailx -s "IRMS-CM Document Import into LiveLink $(date +'%m/%d/%Y') (Status = Error)"  -a "/home/oracle/arun/IRMS-CM/Test4.txt"   arunasaagi81@gmail.com 
  fi

as well as one more script have
Code:
#while [ -f cnt_file ];
#do
xm=$(ls -tp | grep $*.xml | head -1)
gawk -F"[\"<>]" ' BEGIN {fn=dc=pc=ct=" "} /\/Filename/ {fn=$3} /\/DivisionCode/{dc=$3} /\/ProductCode/ {pc=$3} /\/ContentType/ {ct=$3}
/\/sequence/ {print fn"\t"dc"\t"pc"\t"ct;fn=dc=pc=ct=" "}' OFS=, $xm >XmlDataNew.txt
cp /dev/null Test4.txt
cp /dev/null Test.txt
IFS=$'\n'
while read -r line; do
fn=$(echo $line |awk -F "\t" '{print $1}')
dc=$(echo $line |awk -F "\t" '{print $2}')
pc=$(echo $line |awk -F "\t" '{print $3}')
ct=$(echo $line |awk -F "\t" '{print $4}')
src_dir="/home/oracle/arun/IRMS-CM"
dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc"
for FILE in $src_dir/{$fn,*/$fn}
do
if [ -d $dest_dir ]; then
if [ -d $dest_dir/$pc ]; then
  if [ -d $dest_dir/$pc/$ct ]; then   
  if [ -f $FILE ];then
     cp $FILE $dest_dir/$pc/$ct && ((Succ_Cnt++)) || ((Err_Cnt++))
fi    
else
echo "$dest_dir/$pc/$ct not exists"
fi
 elif [ -d $dest_dir/$ct ];then
 cp $FILE $dest_dir/$ct && ((Succ_Cnt1++))
fi
else
echo "$dest_dir/$pc/$ct not exists"
fi
done
done< XmlDataNew.txt 
Succ=$[$Succ_Cnt1+$Succ_Cnt]
echo "Succ=$Succ"
echo $Err_Cnt
if $Err_Cnt=
then 
echo "Status = Success.
    $Succ files added/updated." | mailx -s "IRMS-CM Document Import into LiveLink $(date +'%m/%d/%Y') (Status = Success)" xxxxxx@gmail.com
else
echo "Status = Error.
     $Succ files added/updated.
     $Err_Cnt are failed to copy" | mailx -s "IRMS-CM Document Import into LiveLink $(date +'%m/%d/%Y') (Status = Error)"  -a "/home/oracle/arun/IRMS-CM/Test4.txt"   xxxxxxxxxx@gmail.com 
 fi


plese help me...

count of files success count and failed count
# 21  
Old 08-01-2015
In post #19 in this thread I asked you to answer 11 questions so we might be able to help you complete your task. You answered question #1 and ignored the other 10 questions.

Please answer questions 2 through 11 from post #19 or this thread will be closed!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Open ports from source to destination

Is there a way to find out all the ports open between source IP & destination IP in any way ? (12 Replies)
Discussion started by: UnknownGuy
12 Replies

2. UNIX for Dummies Questions & Answers

How to move gz files from one source directory to destination directory?

Hi All, Daily i am doing the house keeping in one of my server and manually moving the files which were older than 90 days and moving to destination folder. using the find command . Could you please assist me how to put the automation using the shell script . ... (11 Replies)
Discussion started by: venkat918
11 Replies

3. Shell Programming and Scripting

Error files count while coping files from source to destination locaton as well count success full

hi All, Any one answer my requirement. I have source location src_dir="/home/oracle/arun/IRMS-CM" My Target location dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct" my source text files check with below example.text file content $fn "\t" $dc "\t" $pc "\t" ... (3 Replies)
Discussion started by: sravanreddy
3 Replies

4. UNIX for Dummies Questions & Answers

How to count number files successfully copied from source to target location?

Hi Guys, how to count number of files successfully copied while coping files from source to destination path ex:10 files from source to target location copying if 8 files copied successfully then echo successfully copied=8 failure=2 files if two files get error to coping files... (2 Replies)
Discussion started by: sravanreddy
2 Replies

5. Post Here to Contact Site Administrators and Moderators

How to count successfully copy files source to target location with check directory in Linux?

Hi guys...please any one help me .... how to copy files from source to target location if 5 files copied successfully out of 10 files then implement success=10 and if remaining 5 files not copied successfully then count error=5 how to implement this condition with in loop i need code linux... (0 Replies)
Discussion started by: sravanreddy
0 Replies

6. Shell Programming and Scripting

How to count number of files in directory and write to new file with number of files and their name?

Hi! I just want to count number of files in a directory, and write to new text file, with number of files and their name output should look like this,, assume that below one is a new file created by script Number of files in directory = 25 1. a.txt 2. abc.txt 3. asd.dat... (20 Replies)
Discussion started by: Akshay Hegde
20 Replies

7. Linux

rpmbuild, how to specify a different source and destination path for files

I'd like to specify a different build and deployment path for files, by default the same path is used for both build and install, I wasn't able to find a way to make these different. With Solaris pkgadd, one can specify different paths in prototype, so I would assume something like that is possible... (0 Replies)
Discussion started by: tiburblium
0 Replies

8. Shell Programming and Scripting

Move all files from source to destination directory based on the filename

Move all files starting with a specific name to different directory. This shell script program should have three parameters File Name Source Directory Destination Directory User should be able to enter ‘AB_CD*' in file name parameter. In this case all the files starting with AB_CD will... (1 Reply)
Discussion started by: chetancrsp18
1 Replies

9. Shell Programming and Scripting

Moving extremely large number of files to destination

Hello Techies, m here with a small issue. Have read out all the relavent threads on unix.com but it was not so good sol. for me. It's a simple & known issue. I want to move lots of files to dest folder. Actually I want to pick up 1 year older files, but that is even taking lots of... (16 Replies)
Discussion started by: kedar.mehta
16 Replies

10. Shell Programming and Scripting

Count total unique destination for source

Hi, need help how to count unique destination for the source ip. The file is contains 4 number of fields. Example of the file is here src ip dest ip #of flows total bytes 192.168.6.0 88.0.33.2 12 128 192.168.6.0 88.0.33.2 1 168 192.168.6.0 ... (5 Replies)
Discussion started by: new_buddy
5 Replies
Login or Register to Ask a Question