Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Partial file pulling

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Partial file pulling
# 1  
Old 11-10-2014
Partial file pulling
I am connecting to another server through sftp. I am running one batch script to pull file from another server. sometimes i am receiving partial files. I am using below commands in batch script.

ls -ltr new.txt
mget new.txt
bye


The file is of 1 MB only.In most of the cases , i received partial file of 200kB only. Can anyone help me on this issue.
Last edited by jim mcnamara; 11-10-2014 at 08:42 AM..
# 2  
Old 11-10-2014
What help do you need? I'm afraid we can't help with shaky lines or wires.
You could use the exit code of sftp or its
Quote:
-v Raise logging level. This option is also passed to ssh.
option to check if the transfer succeeded.
# 3  
Old 11-10-2014
Are you sure the file is closed (fully written) when you download it? If the remote system is writing files constantly and you grab one that is only partially written, you would see this problem. Then when you check back on the system, the file write has completed and you see that you got less than the full file because you started copying too early.

RudiC covered the other obvious possible problem.
Last edited by jim mcnamara; 11-10-2014 at 08:47 AM..
# 4  
Old 11-13-2014
Yes the file is closed and fully written.

I am using below command to bring the file
sftp -b bringFile {Servername}

bringFile contains below commands
Code:
cd {path}
ls -l new.txt
ls -l new_1.txt
mget new.txt
mget new_1.txt
bye

I am getting below log message. It is not executing the second mget command at all.
Code:
sftp> cd /path/
sftp> ls -l new.txt
-rw-------    0 0        0          745335 Nov 13 00:57 new.txt
sftp> ls -l new_1.txt
-rw-------    0 0        0              28 Nov 13 00:30 new_1.txt
sftp> mget new.txt
Fetching c:/path/new.txt to new.txt

At the end i am checking the size of both files
Size of the pulled file 131072 bytes
Size of the Source File 745335 bytes
The Pulled file size and the Source File size are different.
Last edited by Franklin52; 11-13-2014 at 07:35 AM.. Reason: Please use code tags
# 5  
Old 11-13-2014
Your script is obviously incomplete since you don't show it saving a logfile anywhere but you say a logfile is saved. Are you trapping sftp's stderr or all stderr?

Your script should check the return code of sftp. It can probably tell if the download was interrupted.

You should also check if something else is killing your script. sftp would not catch that.

Code:
trap "echo 'Script was killed for unknown reason' >&2" EXIT
# Trap all errors, from script and sftp, into 'sftplog' file
exec 2>sftplog

TRY=0
MAXTRIES=3

while ! sftp -b bringFile {Servername}
then
        let TRY=TRY+1
        if [[ "$TRY" -gt "$MAXTRIES" ]]
        then
                echo "SFTP download failed, giving up after $MAXTRIES tries" >&2
                exit 1
        fi

        echo "SFTP download had a problem, try $TRY/$MAXTRIES" >&2
fi

Last edited by Corona688; 11-13-2014 at 11:51 AM..
# 6  
Old 11-17-2014
Thanks for your help.

I am getting the exit code as 255. What might be the reason??

Exit code 0 - successful
Exit code 137 - Session killed
# 7  
Old 11-17-2014
man ssh:
Quote:
EXIT STATUS
ssh exits with the exit status of the remote command or with 255 if an error occurred.
What about the - several times - requested log file?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Pulling Data, Then Moving to the Next File

I'm scanning a list of emails- I need to pull 2 pieces of data, then move to the next file: Sender's Email Address Email Date I need these to be outputted into a single column- separated by a ",". Like this: Email1's Address, Email1's Date Stamp Email2's Address, Email2's Date Stamp... (4 Replies)
Discussion started by: sudo
4 Replies

2. UNIX for Dummies Questions & Answers

Pulling Parms from Config File

Hello all, I'm working on a general script for something at work. I'm an up-and-comer backup for a Shell Scripter this company has had for 35 years lol. Anyway, I have a config file I'm trying to pull Variables from as the Config File is used for multiple scripts. Does the below make sense and... (7 Replies)
Discussion started by: phunk
7 Replies

3. Shell Programming and Scripting

AWK - Print partial line/partial field

Hello, this is probably a simple request but I've been toying with it for a while. I have a large list of devices and commands that were run with a script, now I have lines such as: a-router-hostname-C#show ver I want to print everything up to (and excluding) the # and everything after it... (3 Replies)
Discussion started by: ippy98
3 Replies

4. Shell Programming and Scripting

pulling different fields from a csv file

Hi, I have a requirment where I need to pull different columns from a .csv file. Here is the sample of the csv file. account,item,flag1,flag2,flag3,flag4,flag5,......feed,tran I will be have a config.txt file which will have the following information. item,flag5,flag10,feed,tran... (2 Replies)
Discussion started by: akdevula
2 Replies

5. Shell Programming and Scripting

Pulling Ip's from log and redirecting to a file

Hi all, I am fairly new to scripting, but I do try and script as much as possible but the more advanced stuff does tend to boggle my mind a bit. I am at a bit of a loss with this one. I get entries in my DNS logs, like the below: I want to extract only the IP address, without the hashes... (5 Replies)
Discussion started by: codenjanod
5 Replies

6. UNIX for Dummies Questions & Answers

Pulling a file off a backup tape

I have AIX 5.1 This may sound like a really dumb question but I have never done this before. I would like to pull a file off a backup tape and put back on the AIX is this as simple as as doing a mount /dev/rmt1 then the file name that is on the tape /dump/rpt/xxxxxx Do I just copy it... (14 Replies)
Discussion started by: rocker40
14 Replies

7. Shell Programming and Scripting

Pulling data and following lines from file

I saw a few posts close to what i want to do, but they didn't look like they would work exactly.. or I need to think out of the box on this. I have a file that I keep server stats in for my own performance analysis. this file has the output from many commands in it (uptime, vmstats, ps, swap... (2 Replies)
Discussion started by: MizzGail
2 Replies

8. Shell Programming and Scripting

pulling a column from a file in ksh

I would like to pull a column from a file and place it in a variable: The file would look like this: N.Korea gibberish garbage S.Korea gibberish garbage USA gibberish garbage Iraq gibberish garbage Canada gibberish garbage and items in the first... (8 Replies)
Discussion started by: dangral
8 Replies

9. UNIX for Dummies Questions & Answers

pulling the following line from a file

I have return files from a process that has then original input record followed on the next line by a response record..either AA,........... for accepted or EE,.......... for errored. i.e 11,new,123 AA,accepted 12,exist,443 EE,rejected 13,old,223 AA,accepted I want to write a small... (4 Replies)
Discussion started by: peter.herlihy
4 Replies

10. UNIX for Advanced & Expert Users

Pulling out fields from a file

Hi, I have a file that contains 1400 lines similar to the one shown below: NAME=sara, TOWN=southampton, POSTCODE=SO18777, EMAIL=sara@hotmail.com, PASSWORD=asjdflkjds etc etc (note: this is one line). Each line has the same fields, but on each line they are in a different order. Eg. the line... (2 Replies)
Discussion started by: Saz
2 Replies
Login or Register to Ask a Question