Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

Need to synchronize filesystems with huge size

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Need to synchronize filesystems with huge size
# 1  
Old 04-16-2008
Need to synchronize filesystems with huge size
Hello,

I am using IXOS 3rd party utility to maintain invoices (images) in my client's SAP system. We have a DR (Disaster Recovery) setup & for that I need to synchronize my /ixos filesystem (with size more than 50 GB) between two servers say x182 & x050.

Initially I thought to tar the directory then rcp it & then un'tar it at destination but this method is giving me problems related to storage of the tar file & the time it will take it to rcp the huge tar file.

I tried RDIST, but that also takes lot of time (around 5-6 hours).
Can anybody suggest me any other solution to this problem?

Regards,
Vishal
# 2  
Old 04-16-2008
10GB per hour sounds half decent. How fast is the connection? If the majority of the files are not changing after you have copied them once, rsync would sound like the ideal solution.
# 3  
Old 04-16-2008
hi
Hi Era,

How differrent is RSYNC from RDIST?

- Vishal
# 4  
Old 04-16-2008
Perhaps you should visit rsync - Wikipedia, the free encyclopedia
# 5  
Old 04-21-2008
If its a 50GB SAP system I assume that most of that size is an Oracle database?

If thats the case then Oracle must not be active when you make a DR copy of it.

Oracle has lots of files open and keeps making changes. If you are making copies of it over a several hour window then your copy is likely to be junk (especially given SAP likes updating Oracle).

I think IXOS is an archiving solution isnt it? If the IXOS archive is on a unix filesystem and is not an open database then you can copy those files whenever you like. The files in an IXOS archive are almost never going to change so using a utitility like rsync is a possibility. Rsync will only copy the stuff that has changed since the last time it ran.

To answer your actual question. Both rdist and rsync will only copy the files that have changed. Rsync is considered to be better in most circles and comes with the ability to only copy the blocks that have changed in files rather than copy whole files (there are cavaets on using that functionality though). Rsync has more functionality in general.

If you ran rdist to an empty directory on your DR machine then it would have copied the entire filesystem. Try running it a second time when there is already a full copy in place on the remote machine, and it should only copy across and changes since the last time it ran. Use rsync, not rdist though Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. HP-UX

Performance issue with 'grep' command for huge file size

I have 2 files; one file (say, details.txt) contains the details of employees and another file (say, emp.txt) has some selected employee names. I am extracting employee details from details.txt by using emp.txt and the corresponding code is: while read line do emp_name=`echo $line` grep -e... (7 Replies)
Discussion started by: arb_1984
7 Replies

2. Shell Programming and Scripting

Optimised way for search & replace a value on one line in a very huge file (File Size is 24 GB).

Hi Experts, I had to edit (a particular value) in header line of a very huge file so for that i wanted to search & replace a particular value on a file which was of 24 GB in Size. I managed to do it but it took long time to complete. Can anyone please tell me how can we do it in a optimised... (7 Replies)
Discussion started by: manishkomar007
7 Replies

3. Shell Programming and Scripting

FTP a huge Size file

Dear All, Good Evening!! I have a requirement to ftp a 220GB backup file to a remote backup server. I wrote a script for this purpose. But it takes more than 8 hours to transfer this file. Is there any other method to do it in less time??? Thanks in Advance!!! ---------- Post updated... (5 Replies)
Discussion started by: Naga06
5 Replies

4. Shell Programming and Scripting

Implement in one line sed or awk having no delimiter and file size is huge

I have file which contains around 5000 lines. The lines are fixed legth but having no delimiter.Each line line contains nearly 3000 characters. I want to delete the lines a> if it starts with 1 and if 576th postion is a digit i,e 0-9 or b> if it starts with 0 or 9(i,e header and footer) ... (4 Replies)
Discussion started by: millan
4 Replies

5. AIX

/var/adm/wtmp - few entries & huge size.

Hi all. I have a strange case on one of my AIX boxes. /var/adm/wtmp on server01 is ~ 400MB large but it only has ~1200 lines. For example on server02 there are ~85000 lines and the file is ~158MB large. I check lines through 'last | wc -l'. But when I check line directly with 'wc -l... (2 Replies)
Discussion started by: robroy
2 Replies

6. Shell Programming and Scripting

Synchronize Files-Help

Hi, I have two servers1&2, one is not in the network. Cant communicate from it to other servers. The second one can communicate to above mentioned server. I am trying a script which synchronizes files between server 1 an 2? server1: cant communicate to any other servers server2: can... (4 Replies)
Discussion started by: Tuxidow
4 Replies

7. Programming

need a way to synchronize processes

I am writing a program in C for my networking class, so I am relatively new to this. To begin, I have 7 processes that need do send messages to every other one, and every one of them needs to receive the messages sent by others. I am using fork() to create 6 more processes. The message... (1 Reply)
Discussion started by: inabramova
1 Replies

8. Shell Programming and Scripting

how to synchronize different dirs

I have 4 directory Dir1 file1 file2 file3 file4 Dir2 file3 file5 file6 file8 Dir3 file1 file2 file6 file9 file10 Dir4 file3 file6 file12 file15 and all the 4 dirs are having couple of files. Few of the files are common to other directory/ies and few... (1 Reply)
Discussion started by: reldb
1 Replies

9. Programming

synchronize as in java

Hi, I am trying to implement the synchronize feature of java using C. I am using a semaphore for the same. I have a wrapper called "synch" to which I pass the function pointer(any_fn). This pointer points to the function (my_fn) which needs to be synchronized. However to create the semaphore I... (8 Replies)
Discussion started by: linuxpenguin
8 Replies

10. UNIX for Advanced & Expert Users

Conversion between different block size filesystems

Hi, this is for the first time I am using this forum. I have one question: When i transfer one file/directory from a Unix system,having file block size of 1024 bytes, to another Unix system ,having file block size of 512 bytes what difference can it cause in terms of file size? When i... (2 Replies)
Discussion started by: pankschawla
2 Replies
Login or Register to Ask a Question