Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Rsync and Move files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Rsync and Move files
# 1  
Old 04-24-2018
Display Rsync and Move files
I Have a requirement where i have to sync two directories one on source location server A and other on destination location server B
as i do not have ssh access from server A----------->B I am doing rsync from server B,
The Requirement is as follows
  1. Two directories on the source and destination have to be synced
  2. After that they have to be moved to different directory on the source server i.e server A
  3. In step 1 only files having extension .tif have to be synced and the files other than .tif have to be synced with timestamp

Code:
#!/bin/bash
sourcef="root@192.168.83.8:/Somedirectory/*.*"
datelog=`date +%Y-%m-%d:%H:%M:%S`
destination="/mnt/xyz/"
/usr/bin/sshpass -p 'password' /usr/bin/rsync  --include="*.tif" --exclude="*" -avzh  $sourcef $destination --log-file=/var/log/rsync.log

This is incomplete what is needed is the the files other than .tif have to be timestamp and synced to destination later the it has to be moved to a folder in source server A.
The tif files does not necessarily end with *.tif it can be like *.tif.sometext
Last edited by rbatte1; 05-02-2018 at 09:46 AM.. Reason: Formatted numbered list with LIST=1 tags
# 2  
Old 04-25-2018
With sshpass you are breaking the security protocols that ssh and similar tools require. This is never a good thing.

You say that you don't have ssh access, but you know a password. Do you mean that the account does not allow you to get to a shell prompt if you use ssh? Can you not ask the administrator of the server you connect to? They should be able to enable access with SSH-key authentication.

Another alternative might be to set the variable RSYNC_PASSWORD or use the --password-file option. Would they not be better? Is that better than sledge-hammering your way on?






Robin
# 3  
Old 04-25-2018
I can change that but the main thing here is the script which i need to work on to make the work done i can ssh server B from from A password less
Code:
#!/bin/bash
sourcef="/Somedirectory/*.*"
datelog=`date +%Y-%m-%d:%H:%M:%S`
destination="root@192.168.83.7/mnt/xyz/"
 /usr/bin/rsync  --include="*.tif" --exclude="*" -avzh  $sourcef $destination --log-file=/var/log/rsync.log

Need the below required work to be done

The Requirement is as follows
1. Two directories on the source and destination have to be synced
2. After that they have to be moved to different directory on the source server i.e server A
3.In step 1 only files having extension .tif have to be synced and the files other than .tif have to be synced with timestamp

The tif files does not necessarily end with *.tif it can be like *.tif.sometext
Moderator's Comments:
Mod Comment Please use CODE tags (not ICODE tags) when displaying full-line and multi-line sample input, output, and code segments.
Last edited by Don Cragun; 04-25-2018 at 01:28 PM.. Reason: Change ICODE tags to CODE tags, add ICODE tags.
# 4  
Old 04-25-2018
Can you explain in more detail what you want to happen when you say
Quote:
files other than .tif have to be synced with timestamp
.

rsync supports two different methods to determine if files have changed and need to be transfered. By default a "quick check" is done using the files size and timestamp, if --checksum option is set then a 128-bit checksum is calculated for each file that has a matching size.

Are you after a full checksum comparison on *.tif* files and filesize + timestamp for everything else?
# 5  
Old 05-01-2018
Code:
#bin/bash -x
source1="/home/u02/Oracle"
dest1="/home/u02/Oracle/LOG"
chmod 777  $source1/*.tif
chmod 777  $source1/*.txt
/usr/bin/scp  $source1/*.tif  192.168.83.7:/mnt/dbfs/
 if [ $? -eq 0 ]
        then
         echo " files were copied with extension tif on `date`" >> /var/log/croncopy.txt
        fi
/usr/bin/scp  $source1/*.txt  10.83.83.7:/mnt/dbfs/
 if [ $? -eq 0 ]
        then
         echo " files were copied with extension txt on `date`" >> /var/log/croncopy.txt
        fi
/usr/bin/scp  $source1/*.TIF  10.83.83.7:/mnt/dbfs/
 if [ $? -eq 0 ]
        then
         echo " files were copied with extension TIF on `date`" >> /var/log/croncopy.txt
        fi

exit

---------- Post updated at 02:52 AM ---------- Previous update was at 02:51 AM ----------

Code:
#!/bin/bash
source1="/home/u02/Oracle"
dest1="/home/u02/Oracle/LOG"
dirdatelog=`date +%Y%m%d%H%M%S`
mkdir $dest1/$dirdatelog
movedest="$dest1/$dirdatelog/"
mv $source1/*.tif $movedest > /dev/null 2>>/var/log/errormove.txt
mv $source1/*.txt $movedest  > /dev/null 2>>/var/log/errormove.txt
mv $source1/*.TIF $movedest  > /dev/null 2>>/var/log/errormove.txt
exit

---------- Post updated at 02:53 AM ---------- Previous update was at 02:52 AM ----------

The two above scripts will copy initially and later move the files to the destination folder
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Move files with a certain suffix based on how many files are in another folder

Hello, First time poster. I am looking for a way to script or program the process of moving files from one folder to another, automatically, based on the count of files in the destination folder. I was thinking a shell script would work, but am open to the suggestions of the experts... (6 Replies)
Discussion started by: comtech
6 Replies

2. UNIX for Beginners Questions & Answers

Move directory with rsync and delete from source

I need a rsync command which will exclude certain files and directories from source and copy the rest. I got this command working, sudo rsync -avzh --exclude 'bin' --exclude 'braintree' --exclude 'colinmollenhour' --exclude 'composer' --exclude 'doctrine' --exclude 'fabpot' --exclude... (2 Replies)
Discussion started by: Siddheshk
2 Replies

3. Shell Programming and Scripting

SBATCH trinity for multiple files and rename/move the output files

Hey guys, I have wrote the following script to apply a module named "trinity" on my files. (it takes two input files and spit a trinity.fasta as output) #!/bin/bash -l #SBATCH -p node #SBATCH -A <projectID> #SBATCH -n 16 #SBATCH -t 7-00:00:00 #SBATCH --mem=128GB #SBATCH --mail-type=ALL... (1 Reply)
Discussion started by: @man
1 Replies

4. Shell Programming and Scripting

Rsync move with symlinks

Hi, I use rsync to move from source to target, but there are cases that I need to exclude: Suppose in my file system, I have a soft link ~/data -> /media/volgrp/data. Under data folder, there is a file hello.txt. After moving command "rsync --remove-source-files -aH --force ~/data/... (3 Replies)
Discussion started by: huangyingw
3 Replies

5. Shell Programming and Scripting

Move all files except sys date (today) files in Solaris 10

I want to move all files from one directory to another directory excluding today (sysdate files) on daily basis. file name is in pattern file_2013031801, file_2013031802 etc (2 Replies)
Discussion started by: khattak
2 Replies

6. Shell Programming and Scripting

Recursively move directories along with files/specific files

I would like to transfer all files ending with .log from /tmp and to /tmp/archive (using find ) The directory structure looks like :- /tmp a.log b.log c.log /abcd d.log e.log When I tried the following command , it movies all the log files... (8 Replies)
Discussion started by: frintocf
8 Replies

7. UNIX for Dummies Questions & Answers

Move same files and issue ls -al command on remaining files

I know I can use an ls -l junk1 command to get a listing of all files in the directory junk1, but I was wondering how I'd go about going through the files in junk1 in a for-in loop and issuing the ls -l command on them one by one. This is what I have so far: for file in $(ls -a $1) do ls... (1 Reply)
Discussion started by: Trinimini
1 Replies

8. Shell Programming and Scripting

How to check files and move the results to differents files?

Hi, I am a newbie to shell scripting. here is my objective: 1)The shell program should take 2 parameters - ie-> DestinationFolder, WebFolder 2)Destination folder contains few files that has to has be verified and deleted. 3)WebFolder is a folder containing a list of master files 4)It... (1 Reply)
Discussion started by: sandhyagupta
1 Replies

9. Shell Programming and Scripting

Bash rsync but move not delete

There might be an easy solution to my problem, or maybe not, but here it is. I am trying to rsync a Volume with 1.5 terabytes of data and send it via fibre channel to a raid array. Now normally when I rsync it scans through everything and, before copying anything, deletes any files that have... (1 Reply)
Discussion started by: Movomito
1 Replies

10. UNIX for Dummies Questions & Answers

rsync, which files where moved?

Hello, I am using rsync to make sure that my folder "local" mirrors the remote directory "remote". When a file is copied from "remote" to "local", I need to apply a bash script to it. What would be a neat way to do that? Thanks ps: is there a way to edit the title of the thread (I am a bit... (5 Replies)
Discussion started by: JCR
5 Replies
Login or Register to Ask a Question