Count the number of files copied from source to destination location


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Count the number of files copied from source to destination location
# 22  
Old 08-03-2015
1.What operating system are you using?
ans; we used On windows server 2008 servoce pack 2008 service pack 2 , using oracle virtual machine installed Linux operating system with putty terminal.
2.What shell are you using?
ans;linux bash shell script
3.What is the name of your shell script?
ans; shellscript.sh
4.What arguments do you pass to your shell script when you invoke it?
ans; My requirement is copy the files from sftp server to livelink server
5.What output do you get from your shell script?
ans; Using my shell script i was processed successfully mail notification with error atached test4.txt file

6.What output do you want to get from your shell script?
ans;
I want my output is while coping files from source to destination location
if files copied successfully count number of files copied =....
and if any files not copies if any error count number of error counts=...
as well as cause of error issue also print in test4.txt file

Code:
if target folder not exist 
/arun/livelink/ca/vimzim/standard/   folder not exist
              /$dc/$pc/$ct
/arun/livelink/ca/alodro/standard/  folder not exist
/arun/livelink/da/template folder not exist (don't print like this manner if product code not exist in xmldatanew.txt file  "/arun/livelinl/da/ /template") 
/arun/irms-cm/file6.doc    ""if file not exist source location""

7.Have you tried tracing your script as you run it? If not, why not? Examining the trace output from a shell script is usually an easy way to see where things are going wrong and identify areas that need to be fixed! Add set -xv to your script and show us the tracing output you get when you run your script.
ans; i don't know how to tracing set -xv
8.After running your script what are the contents of the file XmlDataNew.txt ?
ans; while reading latest xml file source location
xml content below manner
Code:
XML:
<sequence>
<Filename>VIMIZIMDOSE_10007.docx</Filename>
<DivisionCode>US</DivisionCode>
<ContentType>Standard</ContentType>
<ProductCode>VIMIZIM</ProductCode>
</sequence>
<sequence>
<Filename>VIMIZIMDOSE_10026.Doc</Filename>
<DivisionCode>CA</DivisionCode>
<ContentType>Standard</ContentType>
<ProductCode>VIMIZIM</ProductCode>
</sequence>
<sequence>
<Filename>Opening_10003.Doc</Filename>
<DivisionCode>US</DivisionCode>
<ContentType>Template</ContentType>
<ProductCode />
</sequence>

while reading latest xml file values will stored 
Xmldatanew.txt 
$fn                    $dc   $pc       $ct
Opening_10003.Doc	    US		       Template
Closing_10004.Doc	    US		       Template
VIMIZIMDOSE_10007.docx	US	VIMIZIM	   Standard
VIMIZIMDOSE_10026.Doc	CA	VIMIZIM	   Standard
ALDURAZYME_FAQ_10028.docx	US	ALDURAZYME	Standard

9.After running your script what are the contents of the file Test4.txt ?
Test4.txt file is error file issues stored in text file attached mail notification refer 6th question answer

10What are the contents of the .xml file that you processed with this invocation of your shell script?

ans;Latest .xml file xml file content refer 8th question answer
11.Assuming that you are using a recent version of bash or ksh as your shell; what output do you get if you run the script RudiC provided in post #15 in this thread?
ans;ya i'm used bash script, RudiC code not executed need to check once That code.


Please above are my answer please help ..
while coping files from source as SFTPserver location to Livelink server
check with source directory files with $fn and destination folders
then copy if folders exist and count number of files copied
and if any file not copied to error tehn count number files error to copy and cause of failure issue print on Test4.txt file
then aatached file to mail notification
after review test4.txt file will create folders manuaaly no need to create folders automatically with shell script.


If any clarification ask once....help me solve to issue..
# 23  
Old 08-04-2015
Quote:
Originally Posted by sravanreddy
... ... ...
4.What arguments do you pass to your shell script when you invoke it?
and; My requirement is copy the files from sftp server to livelink server
That doesn't answer the question. Your script is using command line arguments. Please show us the exact command line used to invoke your script!
Quote:
6.What output do you want to get from your shell script?
ans;
I want my output is while coping files from source to destination location
if files copied successfully count number of files copied =....
and if any files not copies if any error count number of error counts=...
as well as cause of error issue also print in test4.txt file

Code:
if target folder not exist 
/arun/livelink/ca/vimzim/standard/   folder not exist
              /$dc/$pc/$ct
/arun/livelink/ca/alodro/standard/  folder not exist
/arun/livelink/da/template folder not exist (don't print like this manner if product code not exist in xmldatanew.txt file  "/arun/livelinl/da/ /template") 
/arun/irms-cm/file6.doc    ""if file not exist source location""

7.Have you tried tracing your script as you run it? If not, why not? Examining the trace output from a shell script is usually an easy way to see where things are going wrong and identify areas that need to be fixed! Add set -xv to your script and show us the tracing output you get when you run your script.
and; i don't know how to tracing set -xv
Change the line in your script that currently says:
Code:
xm=$(ls -tp | grep $*.xml | head -1)

to:
Code:
set -xv
xm=$(ls -tp | grep $*.xml | head -1)

and then rerun your script and show us the exact output produced by your script (not the mail it sends; the output it writes to standard output and the output it writes to standard error output).
Quote:
8.After running your script what are the contents of the file XmlDataNew.txt ?
ans; while reading latest xml file source location
xml content below manner
Code:
XML:
<sequence>
<Filename>VIMIZIMDOSE_10007.docx</Filename>
<DivisionCode>US</DivisionCode>
<ContentType>Standard</ContentType>
<ProductCode>VIMIZIM</ProductCode>
</sequence>
<sequence>
<Filename>VIMIZIMDOSE_10026.Doc</Filename>
<DivisionCode>CA</DivisionCode>
<ContentType>Standard</ContentType>
<ProductCode>VIMIZIM</ProductCode>
</sequence>
<sequence>
<Filename>Opening_10003.Doc</Filename>
<DivisionCode>US</DivisionCode>
<ContentType>Template</ContentType>
<ProductCode />
</sequence>

while reading latest xml file values will stored 
Xmldatanew.txt 
$fn                    $dc   $pc       $ct
Opening_10003.Doc	    US		       Template
Closing_10004.Doc	    US		       Template
VIMIZIMDOSE_10007.docx	US	VIMIZIM	   Standard
VIMIZIMDOSE_10026.Doc	CA	VIMIZIM	   Standard
ALDURAZYME_FAQ_10028.docx	US	ALDURAZYME	Standard

The code you showed us would not produce the above Xmldatanew.txt (it uses tabs in the output, not a combination of tabs and spaces; and it would produce three line of output plus the header; not five).
Quote:
9.After running your script what are the contents of the file Test4.txt ?
Test4.txt file is error file issues stored in text file attached mail notification refer 6th question answer
The text you showed us in the response to #6 above could not possibly be produced by either of the latest scripts you showed us in post #20 in this thread. (Maybe you just added comments that were somehow supposed to make it easier for us to understand the contents of the file; but what we need is to actually see the exact contents of the files produced when you ran your script. And, that is not what you gave us.)
Quote:
... ... ...
11.Assuming that you are using a recent version of bash or ksh as your shell; what output do you get if you run the script RudiC provided in post #15 in this thread?
ans;ya i'm used bash script, RudiC code not executed need to check once That code.


Please above are my answer please help ..
while coping files from source as SFTPserver location to Livelink server
check with source directory files with $fn and destination folders
then copy if folders exist and count number of files copied
and if any file not copied to error tehn count number files error to copy and cause of failure issue print on Test4.txt file
then aatached file to mail notification
after review test4.txt file will create folders manuaaly no need to create folders automatically with shell script.


If any clarification ask once....help me solve to issue..
It would be simpler to have your code create missing directories than to have it go through the additional error handling cases you currently have in your code. But if you want longer code that requires more manual intervention, you are obviously free to choose to do it that way.

The last if statement in both of the scripts you showed us in post #20 has a syntax error that would produce a diagnostic message about unknown commands and would, therefore, always execute code in the else side of the if no matter to what strings $err or $Err_Cnt expand.

We need to see the output produced by your script: The actual output written to standard output. The actual output written to standard error output (including the set -xv trace output). The actual output written to Test4.txt. And, the actual output written to XmlDataNew.txt.
# 24  
Old 08-04-2015
I think there's an understanding/applying issue between:
Quote:
Originally Posted by sravanreddy
how to count number of files successfully copied while coping files from source to destination path
And the retrieval of the file-list/stream.

Since you dont seem to apply any of the actual code suggestions, i'll share a solution only with functions you are supposed to write on your own - since you seem to prefer this way.

The CAP'd WORDs are functions you have to write, with the exception of $TABLEFILE which is the file WRITE_TABLE has written to.
Code:
function COPY() { .... ; } # Does the path check, copies the file
function WRITE_FILE() { ... ; } # Writes just the required info to a file

WRITE_TABLE	
while read fn dc pc ct
do 	COPY "$fn" "/arun/livelink/$dc/$pc/$ct" && \
		cnt_good=$(($cnt_good + 1))|| \
		cnt_err=$(($cnt_err + 1))
done<"$TABLEFILE"

echo "Successfully copied: $cnt_good files, while $cnt_err failed."

It COULD be so simple.

hth & have a good day
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Open ports from source to destination

Is there a way to find out all the ports open between source IP & destination IP in any way ? (12 Replies)
Discussion started by: UnknownGuy
12 Replies

2. UNIX for Dummies Questions & Answers

How to move gz files from one source directory to destination directory?

Hi All, Daily i am doing the house keeping in one of my server and manually moving the files which were older than 90 days and moving to destination folder. using the find command . Could you please assist me how to put the automation using the shell script . ... (11 Replies)
Discussion started by: venkat918
11 Replies

3. Shell Programming and Scripting

Error files count while coping files from source to destination locaton as well count success full

hi All, Any one answer my requirement. I have source location src_dir="/home/oracle/arun/IRMS-CM" My Target location dest_dir="/home/oracle/arun/LiveLink/IRMS-CM/$dc/$pc/$ct" my source text files check with below example.text file content $fn "\t" $dc "\t" $pc "\t" ... (3 Replies)
Discussion started by: sravanreddy
3 Replies

4. UNIX for Dummies Questions & Answers

How to count number files successfully copied from source to target location?

Hi Guys, how to count number of files successfully copied while coping files from source to destination path ex:10 files from source to target location copying if 8 files copied successfully then echo successfully copied=8 failure=2 files if two files get error to coping files... (2 Replies)
Discussion started by: sravanreddy
2 Replies

5. Post Here to Contact Site Administrators and Moderators

How to count successfully copy files source to target location with check directory in Linux?

Hi guys...please any one help me .... how to copy files from source to target location if 5 files copied successfully out of 10 files then implement success=10 and if remaining 5 files not copied successfully then count error=5 how to implement this condition with in loop i need code linux... (0 Replies)
Discussion started by: sravanreddy
0 Replies

6. Shell Programming and Scripting

How to count number of files in directory and write to new file with number of files and their name?

Hi! I just want to count number of files in a directory, and write to new text file, with number of files and their name output should look like this,, assume that below one is a new file created by script Number of files in directory = 25 1. a.txt 2. abc.txt 3. asd.dat... (20 Replies)
Discussion started by: Akshay Hegde
20 Replies

7. Linux

rpmbuild, how to specify a different source and destination path for files

I'd like to specify a different build and deployment path for files, by default the same path is used for both build and install, I wasn't able to find a way to make these different. With Solaris pkgadd, one can specify different paths in prototype, so I would assume something like that is possible... (0 Replies)
Discussion started by: tiburblium
0 Replies

8. Shell Programming and Scripting

Move all files from source to destination directory based on the filename

Move all files starting with a specific name to different directory. This shell script program should have three parameters File Name Source Directory Destination Directory User should be able to enter ‘AB_CD*' in file name parameter. In this case all the files starting with AB_CD will... (1 Reply)
Discussion started by: chetancrsp18
1 Replies

9. Shell Programming and Scripting

Moving extremely large number of files to destination

Hello Techies, m here with a small issue. Have read out all the relavent threads on unix.com but it was not so good sol. for me. It's a simple & known issue. I want to move lots of files to dest folder. Actually I want to pick up 1 year older files, but that is even taking lots of... (16 Replies)
Discussion started by: kedar.mehta
16 Replies

10. Shell Programming and Scripting

Count total unique destination for source

Hi, need help how to count unique destination for the source ip. The file is contains 4 number of fields. Example of the file is here src ip dest ip #of flows total bytes 192.168.6.0 88.0.33.2 12 128 192.168.6.0 88.0.33.2 1 168 192.168.6.0 ... (5 Replies)
Discussion started by: new_buddy
5 Replies
Login or Register to Ask a Question