Sponsored Content
Top Forums UNIX for Beginners Questions & Answers rsync a file as it gets created and after 3 minutes old Post 303033824 by malaika on Friday 12th of April 2019 06:49:00 AM
Old 04-12-2019
replye to RudiC
Quote:
Do you know when the "last file" has been created? Are you able to identify it? NO
You seem to know the creating program / script. Are you able to modify it? If you can modify it to work that will be great.

Quote:
Do you REALLY have to rsync before the last files has arrived? Can't you wait until they're complete? The process doesn't seem to take that long.
So the sample i showed you is just for splitting a 73 G file. In actual fact the file i have to split is 3 TB. It will take over 5 hours just to split all the files. It will take another 15 hours just to rsync to complete sending all the files to a remote location which about 1200 miles away. If I ship the 3TB+ as a whole it will take about 2 days to rsync the bulk file over 1200 miles between hostA and hostB.

The reason behind not waiting for all the files to complete the splitting, is because i want to save time.
Once i receive all the files on the remote server , i will have to worry about total time to restore. If i go through the old school method of shipping 3TB at once to the remote site and restoring at the remote site it takes me about 3 days depending the distance between server A and server B.
If i have to wait until all the files are split before i rsync each file , then I have to endure the time it takes for all the files to split, which depending on the size of the file could be humongous.
The smart thing, is to split, dont wait for the next file, rsync the first spit file , wait till the nest split file to complete and rynsc and so forth until the last file is split. Hopefull by ussing a "nohup" , i can rsynch each file using different sessions.

Last edited by RavinderSingh13; 04-12-2019 at 02:37 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How can i search a file which has been created or modified in last five minutes

Hi Can some one please help me How can i search a file which has been created or modified in last five minutes I have used the command find . -mmin -5 and it does not work i get an error -mmin is bad option Please help Much regards Tarun (2 Replies)
Discussion started by: tarundeepdhawan
2 Replies

2. Shell Programming and Scripting

find files created within 30 minutes

find . -name *.txt -mmin -30 This is working in Redhat but not in Solaris.. What is the equivalent option in Solaris? (1 Reply)
Discussion started by: tene
1 Replies

3. Shell Programming and Scripting

Ftp get files created in last 30 minutes

Is it possible in an ftp script to get remote files based on whether they have been created in the last 30 minutes? (5 Replies)
Discussion started by: gefa
5 Replies

4. Shell Programming and Scripting

Check how many minutes ago the last file created

Hi , I need help in getting how many minutes ago the last file, matching some pattern in file name, was created in a folder. Thanks in advance. (8 Replies)
Discussion started by: Anupam_Halder
8 Replies

5. Shell Programming and Scripting

How to send a file in UNIX through email which is created only 15 minutes before the current time?

I wanted to send an email to the client whenever there is failed record created in a /feed/HR-76/failed folder after processing of feed file. I can find out with the help of below script that what is the new file created but that file didn't make just 15 minutes before. ... (1 Reply)
Discussion started by: puneetkhullar
1 Replies

6. Shell Programming and Scripting

Rsync Error: rsync: link_stat failed: No such file or directory (2)

I wish to copy all the files & folder under /web/Transfer_Files/data/ on mymac1 (Linux) to remote server mybank.intra.com (Solaris 10) /tmp/ location I am using Ansible tool synchronize module which triggers the unix rsync command as below:rsync --delay-updates -F --compress --archive --rsh=ssh... (2 Replies)
Discussion started by: mohtashims
2 Replies

7. UNIX for Beginners Questions & Answers

How to convert days hours minutes seconds to minutes?

Hi, please help with below time conversion to minutes. one column values: 2 minutes 16 seconds 420 msec 43 seconds 750 msec 0 days 3 hours 29 minutes 58 seconds 480 msec 11 seconds 150 msec I need output in minutes(total elapsed time in minutes) (2 Replies)
Discussion started by: ramu.badugula
2 Replies

8. Shell Programming and Scripting

Grep a log file for the last 5 minutes of contents every 5 minutes

Hi all, System Ubuntu 16.04.3 LTS i have the following log INFO 2019-02-07 15:13:31,099 module.py:700] default: "POST /join/8550614e-3e94-4fa5-9ab2-135eefa69c1b HTTP/1.0" 500 2042 INFO 2019-02-07 15:13:31,569 module.py:700] default: "POST /join/6cb9c452-dcb1-45f3-bcca-e33f5d450105... (15 Replies)
Discussion started by: charli1
15 Replies

9. Shell Programming and Scripting

Check file creation Time minutes and if file older then 5 minutes execute some stuff

Hello all, Info: System RedHat 7.5 I need to create a script that based on the creation time, if the file is older then 5 minutes then execute some stuff, if not exit. I thought to get the creation time and minutes like this. CreationTime=$(stat -c %y /tmp/test.log | awk -F" " '{ print... (3 Replies)
Discussion started by: charli1
3 Replies

10. UNIX for Beginners Questions & Answers

Find if create time of last created file in a directory is older than 5 minutes

A process xyz is running and creating file1, file2, file3, .... filen. how do i know if the process has stopped and createtime of the last file (filen) is older than 5 minutes? OS is AIX (3 Replies)
Discussion started by: malaika
3 Replies
vm-install-jobs(8)					      System Manager's Manual						vm-install-jobs(8)

NAME
vm-install-jobs - queries and manages VM creation jobs that were started by vm-install SYNOPSIS
vm-install-jobs [options ...] DESCRIPTION
VMs may be created by vm-install interactively, or as backgrounded jobs. The vm-install-jobs command manages backgrounded VM creation jobs. OPTIONS
--cancel job-id Cancels the specified job. If the VM is still running, it is stopped. This command may return before the VM is completely stopped or the vm-install process has completely exited. The job is not implicitly deleted, so that the cancelation progress may be observed. --delete job-id Deletes the specified job. The job must be in one of the FINISHED, FAILED, or CANCELED states. The VM's configuration is deleted, as are any disks that were created during the job. --detach job-id Detaches the created VM from the specified job. The job must be in the FINISHED state. The job is deleted, but the VM's configura- tion and its disks are left intact. --list Lists all known VM creation jobs, by ID. One job ID is output per line. --status job-id Outputs the status of the specified job. JOB STATES
Every job is always in one of the following states: 0: SETTINGS 1: PREP_INSTALL 2: INSTALL 3: POST_INSTALL 4: RUNTIME_CONF 5: RUNTIME 6: FINISHED 7: FAILED 8: CANCELED The normal progression is from state 0 to state 6. States 6, 7, and 8 are terminal states. JOB STATUS
Querying the job status will print (to stdout) a number of key/value pairs. The possible keys and their meanings are: exit The exit code of the VM creation job. This key is only present once the job is in one of the states FINISHED, FAILED, or CANCELED. To interpret the meaning of the exit code, see vm-install(8) id The job ID. Always present. inotify A filename to monitor (via inotify(7) ) for state changes. The recommended inotify mask to use is IN_CLOSE_WRITE. Always present. log The location of the log file. Always present. pid The process ID of the VM creation job. This key is only present if the VM creation process is still running. The PID is used internally to cancel the job. state The job's current state. See JOB STATES. Always present. uuid The VM's UUID. This key is only present once all of the VM's settings have been gathered. vnc The last known location to view the VM's console via VNC. Note that the presence of this key does not guarantee that the VM is cur- rently running. When no authentication is necessary, the URL will be ip::port. If authentication is necessary, the URL will be username:password@ip::port. DIAGNOSTICS
A zero exit code indicates success; non-zero indicates failure. Note that the exit code of this tool indicates the success of the job management command. It does not represent the success or failure of the job itself. SEE ALSO
To interpret the exit code of a job or of this command, see vm-install(8) To monitor state changes of jobs, see inotify(7) To report problems with this software or its documentation, visit http://bugzilla.novell.com 2007-02-01 vm-install-jobs(8)
All times are GMT -4. The time now is 05:16 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy