advice on managing files between two systems


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting advice on managing files between two systems
# 1  
Old 05-22-2008
advice on managing files between two systems

I am looking for some guidance to see if this is the most efficient solution.

I have a piece of software that updates a directory every night at 2am. The directory contains xml files so at 2am any new files are written, any modified files are modified and any deleted files are removed from the directory.

I need to send new and modified files to a different system. I also need to send a list of deleted files.

I had thought about using rsynch but the system I am feeding doesn't want to upload all files every day as it is a drain. That is why they want new/amended files only and a list of deleted files.

Here's my plan;
- directory on main server (let's call is 'a') is refreshed at 2.00am
- i can run rscnch on 'a' and a new folder 'b' at 4.00am
- between 2.00am and 4.00am folder 'a' and 'b' will be different.
- at 2.30am i can run a chron job on directory 'a' to send the new/amended files to the other system.
- at 3.00am i can run a script to cmpare the difference between 'a' and 'b' and output a list of files that are in 'b' but not 'a'. this can then be sent to the other system.

It seems a bit long winded but I am not aware of another way to do this. Does anyone have any suggestions for a better approach?

Thanks
# 2  
Old 05-22-2008
I don't know , if I got this correct, but you need to send the modified files and list of deleted files to another machine.

The update/operation on files takes place at 2.00 AM in the morning.Suppose this is script1.

Suppose this operation takes place till 4.00 AM. You can run a script , say script2 which would list the deleted files and the modified files by script1.

Script2 can use find command with atime(or some other) option to find the modified files.These files can be ftp'ied to other machine , say host2.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Find all log files under all file systems older than 2 days and zip them

Hi All, Problem Statement:Find all log files under all file systems older than 2 days and zip them. Find all zip files older than 3days and remove them. Also this has to be set under cron. I have a concerns here find . -mtime +2 -iname "*.log" -exec gzip {} Not sure if this will work as... (4 Replies)
Discussion started by: saurabh.mishra
4 Replies

2. Shell Programming and Scripting

Advice on monitoring gziped files

so, every 5 minutes, i monitor some data files based on their line numbers. so if i checked now and there are 100 lines in the file, i will scan the 100 lines for specific errors. if i check again 5 minutes later and there are 150 lines in the file, i will monitor the file from line 100 to... (2 Replies)
Discussion started by: SkySmart
2 Replies

3. Red Hat

Advice regarding filesystems handling large number of files

Hi All, I have a CentOS operating system installed. I work with really huge number of files which are not only huge in number but some of them really huge in size. Minimum number of files could be 1 million to 2 million in one directory itself. Some of the files are even several Gigabytes in... (2 Replies)
Discussion started by: shoaibjameel123
2 Replies

4. SCO

Need advice: Copying large CSV report files off SCO system

I have a SCO Unix server from 1999 running SCO 5.0.5 and some ancient accounting software called Real World A report writer program on the system is used to generate CSV files from accounting that we write with DOSCOPY commands to 3.5" floppies In the next 60 days we will be decommissioning... (11 Replies)
Discussion started by: magnetman
11 Replies

5. UNIX for Dummies Questions & Answers

move files between file systems with privileges, time stamp

Hi I have to move files between file systems but files in new file system must have the same attributes as in old one (privileges, time stamp etc). Which tool is best : - ufsdump / ufsrestore - tar - cpio - pax - dd - mv Or maybe there is sth else, you suggest to use. Thx for help (5 Replies)
Discussion started by: presul
5 Replies

6. Solaris

Could not able to delete the files through find command need expert advice

Hi Gurus I am facing a problem, there is a folder called /a where there are lots of files which are occupying space anything between 30 GB to 100 GB as I am not able to check the space occupied by that folder through "du -sh /a" command as I don't see any output after more than 1 hour of... (4 Replies)
Discussion started by: amity
4 Replies

7. HP-UX

[Solved] Deleting a all core files present in file systems ?

Hi All IN HPUX 11 How to delete an unwanted "core" file with a single command which is being generated in different locations of the system the command should be able to free up the space occupied by all "core" file which is present in different folders and filesytems in a system ... (5 Replies)
Discussion started by: sidharthmellam
5 Replies

8. UNIX for Advanced & Expert Users

Working with remote Unix files systems from Windows

I am currently using Samba to access remote Unix file systems from Windows. However, it is slow, and I presume insecure in the sense that file contents are transmitted unencrypted. I also wonder if passwords are transmitted in plain text in this protocol or not. For these reasons I am looking for... (2 Replies)
Discussion started by: Bilge
2 Replies

9. UNIX for Dummies Questions & Answers

Possible to use a Java app with dll files on Unix-systems

Hi... I have build a program for the Velleman K8000 interface card, in java, which works just fine in windows, but now I want to use the program on a Unix-System. Is it possible to somehow convert the dll file to a format that Unix supports, or do I have to find another way? Dll-file:... (3 Replies)
Discussion started by: Scorp-D
3 Replies

10. Filesystems, Disks and Memory

Number of disks, space available, files systems ?

Hello, When I am logged on a Unix machine, I would like get a maximum of informations related to the number of disks used, the space available, the description of the different files systems. I'm using Unix OSF1 V4.0. Thanks in advance for your help. Geoffrey (2 Replies)
Discussion started by: geogeo
2 Replies
Login or Register to Ask a Question