Visit Our UNIX and Linux User Community


Multiple site / multiple transfers


 
Thread Tools Search this Thread
Operating Systems Linux Multiple site / multiple transfers
# 1  
Old 07-21-2009
Multiple site / multiple transfers

I need to revise a transfer process. I have 11 "remote" servers all transfering 1 transfer file nightly. Now, this need to be more "automatic" in that the transfers need to "catch" all missed transfers up to the current date.

What can anyone suggest?
# 2  
Old 07-21-2009
You didn't say how you are transferring the files.

You could delete files after they are transferred, thus leaving only those that were not.
# 3  
Old 07-24-2009
11 servers each create a single transfer file of all "file" changes done to the local file that day. Earily morning transfers are done of the Internet and each server site receive the daily transfer. Then, "our" custom application read the transfer file(s) and process the update on the "other" server local copy. The transfer file is a standard text file.

Now, we need to figure out a method of letting the transfer detect the missed "daily" transfer and to get from the missed day.

No "central" location for all the transfer files would be considered, unless a very good valid reason.
# 4  
Old 07-24-2009
Sorry, but could you explain it again? Do you have 11 servers that create 1 file, which then has to transferred to an additional server for processing? Or do you have one server that generates files for each of the 11 servers which then have to be transferred to them? Or do you have 11 servers that exchange those files with each other?

Also, how do you transfer these files now?
# 5  
Old 07-24-2009
If what you are saying is that you are backing up changed files on the 11 servers to a backup server, you can just use rsync to backup the changes and if it missed anything it will catchup the next "sync".

But maybe I also dont fully understand what you are trying to do.
# 6  
Old 07-27-2009
11 server each create 1 transfer file, containing all file changes done during the day.

11 servers each connect to the other servers so as to receive this transfer file and process the update to the "local" copy of the "other" branch files.

And my client has "refused" any recommendation for a centralized backup server.
# 7  
Old 07-27-2009
Then rsync is probably the best bet for you. Create the files in a special directory, say /var/tmp/send. Loop through a list of all 11 servers and rsync to it, skipping if it's the current server. If a file is transferred correctly, remove it, otherwise rsync will take care of it on the next pass
Create a second directory, say /var/tmp/received, to which rsync sends the files, and from which they are then processed.
In order to avoid having rsync run in daemon mode, set up ssh on each server, which public key authentication, so that rsync can use that for transfer.

Previous Thread | Next Thread
Test Your Knowledge in Computers #20
Difficulty: Medium
India's first Super Computer, the PARAM 8000, was installed in 1991.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to apply the update statement in multiple servers on multiple dbs at a time .?

Hi , Can any please help the below requirement on all multiple servers and multiple dbs. update configuration set value='yes' ;1) the above statement apply on 31 Databases at a time on different Ip address eg : 10.104.1.12 (unix ip address ) the above ip box contains 4 db's eg : db... (2 Replies)
Discussion started by: venkat918
2 Replies

2. Shell Programming and Scripting

sed parser behaving strange on replacing multiple words in multiple files

I have 4000 files like $cat clus_grp_seq10_g.phy 18 1002 anig_OJJ65951_1 ATGGTTTCGCAGCGTGATAGAGAATTGTTTAGGGATGATATTCGCTCGCGAGGAACGAAGCTCAATGCTGCCGAGCGCGAGAGTCTGCTAAGGCCATATCTGCCAGATCCGTCTGACCTTCCACGCAGGCCACTTCAGCGGCGCAAGAAGGTTCCTCG aver_OOF92921_1 ... (1 Reply)
Discussion started by: sammy777888
1 Replies

3. Shell Programming and Scripting

Ssh to multiple hosts and then run multiple for loops under remote session

Hello, I am trying to login to multiple servers and i have to run multiple loops to gather some details..Could you please help me out. I am specifically facing issues while running for loops. I have to run multiple for loops in else condition. but the below code is giving errors in for... (2 Replies)
Discussion started by: mohit_vardhani
2 Replies

4. Shell Programming and Scripting

Removing carriage returns from multiple lines in multiple files of different number of columns

Hello Gurus, I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file. 1|ABC DEF|100|10 2|PQ RS T|200|20 3| UVWXYZ|300|30 4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies

5. Shell Programming and Scripting

Checking File record equal to multiple of 70 or nearest number to multiple of 70

Hello, I have a file with below content - Example 3 6 69 139 210 345 395 418 490 492 I would like the result as - Multiple of 70 or nearest number in the file less than the multiple of 70 69 139 (5 Replies)
Discussion started by: Mannu2525
5 Replies

6. Shell Programming and Scripting

Assigning multiple column's value from Oracle query to multiple variables in UNIX

Hi All, I need to read values of 10 columns from oracle query and assign the same to 10 unix variables. The query will return only one record(row). I tried to append all these columns using a delimiter(;) in the select query and assign the same to a single variable(V) in unix. I thought I... (3 Replies)
Discussion started by: hkrishnan91
3 Replies

7. Shell Programming and Scripting

Reading multiple values from multiple lines and columns and setting them to unique variables.

Hello, I would like to ask for help with csh script. An example of an input in .txt file is below, the number of lines varies from file to file and I have 2 or 3 columns with values. I would like to read all the values (probably one by one) and set them to independent unique variables that... (7 Replies)
Discussion started by: FMMOLA
7 Replies

8. Shell Programming and Scripting

Create Multiple UNIX Files for Multiple SQL Rows output

Dear All, I am trying to write a Unix Script which fires a sql query. The output of the sql query gives multiple rows. Each row should be saved in a separate Unix File. The number of rows of sql output can be variable. I am able save all the rows in one file but in separate files. Any... (14 Replies)
Discussion started by: Rahul_Bhasin
14 Replies

9. Shell Programming and Scripting

Grep from multiple patterns multiple file multiple output

Hi, I want to grep multiple patterns from multiple files and save to multiple outputs. As of now its outputting all to the same file when I use this command. Input : 108 files to check for 390 patterns to check for. output I need to 108 files with the searched patterns. Xargs -I {} grep... (3 Replies)
Discussion started by: Diya123
3 Replies

10. UNIX for Dummies Questions & Answers

grep command to find multiple strings in multiple lines in a file.

I want to search files (basically .cc files) in /xx folder and subfolders. Those files (*.cc files) must contain #include "header.h" AND x() function. I am writing it another way to make it clear, I wanna list of *.cc files that have 'header.h' & 'x()'. They must have two strings, header.h... (2 Replies)
Discussion started by: ritikaSharma
2 Replies

Featured Tech Videos