Need to get specified number of files from remote server


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need to get specified number of files from remote server
# 1  
Old 06-23-2009
Question Need to get specified number of files from remote server

Hi All,

I need to get a specified number of files from a remote server. The remote server has a huge number of files to the tune of more than 200000. If I use scp or rsync to transfer files using the wildcard "*", am getting the error "arg list too long"

So, I would like to know if i can get a specified number of files from the remote server

Thanks
Uma
# 2  
Old 06-23-2009
why don't you login to the remote server, make a tar-zip of the required files and then scp the single file?
# 3  
Old 06-23-2009
This should be done in a shell script which maybe called in a cron

---------- Post updated at 11:35 AM ---------- Previous update was at 11:26 AM ----------

I should be able to get say 1000 files in a single transfer and should run this in a loop until all files are transfered
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Copying files to remote server

I am trying to copy code to remote server, doing something like this However it is copying one file and than the script exits i=0; j=0 while read fn; do dir=${fn%/*} # Gets directory path fnm=${fn##*/} # Gets filename excl. path rgx_nwk="s/${nwk}/${nwk}.sac/g" ... (2 Replies)
Discussion started by: kristinu
2 Replies

2. Shell Programming and Scripting

Script connect to remote server, not find files and exit only from remote server, but not from scrip

I have a script, which connecting to remote server and first checks, if the files are there by timestamp. If not I want the script exit without error. Below is a code TARFILE=${NAME}.tar TARGZFILE=${NAME}.tar.gz ssh ${DESTSERVNAME} 'cd /export/home/iciprod/download/let/monthly;... (3 Replies)
Discussion started by: digioleg54
3 Replies

3. Shell Programming and Scripting

Sudo connect to a remote server and execute scripts in remote server

Hello Every one!! I am trying to write a shell script which will connect to a remote server and execute scripts which are at a certain path in the remote server. Before this I am using a sudo command to change the user. The place where I am stuck is, I am able to connect to the... (6 Replies)
Discussion started by: masubram
6 Replies

4. Shell Programming and Scripting

Do I require remote login access to a windows server to transfer files from a UNIX server

Hi All I need to transfer a file from a UNIX server to a windows server. I saw that it is possible to do this using scp command by looking at the forum listed below: ... (2 Replies)
Discussion started by: vx04
2 Replies

5. Solaris

Script to get files from remote server to local server through sftp without prompting for password

Hi, I am trying to automate the process of fetching files from remote server to local server through sftp. I have the username and password for the remote solaris server. But I need to give password manually everytime i run the script. Can anyone help me in automating the script such that it... (3 Replies)
Discussion started by: ssk250
3 Replies

6. Shell Programming and Scripting

Count number of files in remote server

Conneect remote server and then count no of files in the directory sftp ksmsft1@laixu110.jkea.jlroot.net << EOF cd /opt/jls/ ls quit EOF When I try above command it doesnt work , we have ssh configured and hence with below command I can connect without asking for password sftp... (6 Replies)
Discussion started by: lalitpct
6 Replies

7. Shell Programming and Scripting

copy files from remote server (B) to target server (A)?

Hi All, what is the comand to log off the remote server? I have 2 servers A, B. I need to find all files older than 7 days on server B and copy over to server A. My logic is: login the remote server: ================= ssh hostB cd /data/test find . -mtime -7 -ls | awk '{print... (4 Replies)
Discussion started by: Beginer0705
4 Replies

8. AIX

listing files on remote server

I am writing a script where in i have to log into a remote machine and check for necessary file by typing (ls -ltr *200505) (this gets all 05month of 2008 yr files) and if files are found get them to the local machine. If not found print a message saying no files on local machine. When i was... (3 Replies)
Discussion started by: vasuarjula
3 Replies

9. UNIX for Advanced & Expert Users

delete files on a remote server

Hi Guys, I am currently working on a script to find all the files that have not been accessed for the past 2 years. This, i guess has been discussed n number of times in this forum. Now, my requirement is to find all the files in the remote windows server. I have it mounted in unix. I... (1 Reply)
Discussion started by: bond_bhai
1 Replies

10. Shell Programming and Scripting

FTP multiple files from remote server to local server

Hi, I am facing a weired problem in my FTP script. I want to transfer multiple files from remote server to local server everyday, using mget * in my script. I also, want to send an email for successful or failed FTP. My script works for file transfer, but it don't send any mail. There is... (2 Replies)
Discussion started by: berlin_germany
2 Replies
Login or Register to Ask a Question