07-26-2010
Can I just add something to this...
I think rsync is the path you have to look down... however, in dealing with Splunk recently for our centralized log management I have found using rsync in a cron is a very very silly idea, especially every minute.
Problem is if you have 2GB of files lets say, every minute rsync connects reads that entire 2GB image from disk and checksums it, if it comes back clear it disconnects else it syncs itself.
Problem is, you are now performing 2GB or reads every single minute.
In our case using splunk recently, 10GB every 5 minutes, which was able to DOS our SAN fairly quickly. (as a result we moved to syslog).
Anyway, thing is, you mentioned you have (some method) running that when something is changed, you are alerted and then you copy over the files. Is this a scripted alert or manual alert (someone emails you)?
If it's scripted, then you can just add an rsync to run rather than alert you.
If it's manual, then make a script to trigger the rsync.
Or perhaps give your users access to a function calling the rsync, ie. a perl cgi page or similar, with a sync it button.
JM2C.
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
Is there a way to monitor FTP and AFP bandwidth on a per user basis? AFP = AppleFilesharingProtocol, and by monitor, I don't mean graphs, I mean getting raw byte counts I can use in my scripts any way I choose. I am attempting to use mod_watch to monitor virtual domain bandwidth usage, but I really... (0 Replies)
Discussion started by: l008com
0 Replies
2. UNIX for Dummies Questions & Answers
I have an ftp process using vpn, my box is receiving data hit and miss from the host system. Any tool that can be used within unix (ksh) to monitor that ftp process? (2 Replies)
Discussion started by: arkhewit
2 Replies
3. Shell Programming and Scripting
Hello,
A couple of times per week, i receive emails notifications when files are available for processing.
Currently i read these eamails with a java program and store the attachement on my C: drive and would now like to generate a PC script to send this file name up to UNIX-Solaris and... (3 Replies)
Discussion started by: bobk544
3 Replies
4. UNIX for Advanced & Expert Users
Hi,
We have some clients who will place huge files in to one of the remote server.
And the shell script written in our local server to retrieve client files (using FTP) placed on one of the remote server of ours by clients.
My question Is there any FTP command/script to check from my local... (1 Reply)
Discussion started by: nmsrao
1 Replies
5. Shell Programming and Scripting
Hi,
I have a ftp script which first gets all the file names and echo's the latest file.
I'm using another ftp command sets to get the file name given by first ftp. The problem is the parameter is not accepted by second ftp. The error message i'm getting is
> Rename Temp File calloc:ICMP... (5 Replies)
Discussion started by: ammu
5 Replies
6. Shell Programming and Scripting
Hi all,
I'm using the following script to automated ftp files to 1 ftp servers
host=192.168.0.1
/usr/bin/ftp -vi >> $bkplog 2>&1 <<ftp
open $host
bin
cd ${directory}
put $files
quit
ftp
and the .netrc file contain
machine 192.168.0.1
login abc... (4 Replies)
Discussion started by: varu0612
4 Replies
7. AIX
how to connect to ftp server in passive mode?
ftp server.abc
and how can i see ftp settings, doesn't exist some ftpd.conf
there is some other file where i check the options and configurations of ftp server?
Thanks (3 Replies)
Discussion started by: prpkrk
3 Replies
8. HP-UX
Hello All,
I am trying to connect to ftp server and get the files. Also i need to rename the file in other ftp dir.
rename method is not allowing me to rename the file in other dir. When i tried copy command by using net::FTP:FILE then perl says it is not installed.
Can some body help me to... (2 Replies)
Discussion started by: krsnadasa
2 Replies
9. Shell Programming and Scripting
Hi,
Can anyone help me to start a monitoring script in perl of a FTP server. (2 Replies)
Discussion started by: phanik143
2 Replies
10. Solaris
OS: Solaris 9
Configuration /etc/syslog.conf
daemon.debug
/etc/inetd.conf
ftp stream tcp6 nowait root /usr/sbin/in.ftpd in.ftpd -A -l -d
Found the ftp.log only generate those entries from other servers/hosts.
Can we trace on all ftp entries either from/to the server?
... (6 Replies)
Discussion started by: KhawHL
6 Replies
LEARN ABOUT CENTOS
ocf_heartbeat_rsyncd
OCF_HEARTBEAT_RSYNCD(7) OCF resource agents OCF_HEARTBEAT_RSYNCD(7)
NAME
ocf_heartbeat_rsyncd - Manages an rsync daemon
SYNOPSIS
rsyncd [start | stop | monitor | meta-data | validate-all]
DESCRIPTION
This script manages rsync daemon
SUPPORTED PARAMETERS
binpath
The rsync binary path. For example, "/usr/bin/rsync"
(optional, string, default "rsync")
conffile
The rsync daemon configuration file name with full path. For example, "/etc/rsyncd.conf"
(optional, string, default "/etc/rsyncd.conf")
bwlimit
This option allows you to specify a maximum transfer rate in kilobytes per second. This option is most effective when using rsync with
large files (several megabytes and up). Due to the nature of rsync transfers, blocks of data are sent, then if rsync determines the
transfer was too fast, it will wait before sending the next data block. The result is an average transfer rate equaling the specified
limit. A value of zero specifies no limit.
(optional, string, no default)
SUPPORTED ACTIONS
This resource agent supports the following actions (operations):
start
Starts the resource. Suggested minimum timeout: 20s.
stop
Stops the resource. Suggested minimum timeout: 20s.
monitor
Performs a detailed status check. Suggested minimum timeout: 20s. Suggested interval: 60s.
validate-all
Performs a validation of the resource configuration. Suggested minimum timeout: 20s.
meta-data
Retrieves resource agent metadata (internal use only). Suggested minimum timeout: 5s.
EXAMPLE
The following is an example configuration for a rsyncd resource using the crm(8) shell:
primitive p_rsyncd ocf:heartbeat:rsyncd
op monitor depth="0" timeout="20s" interval="60s"
SEE ALSO
http://www.linux-ha.org/wiki/rsyncd_(resource_agent)
AUTHOR
Linux-HA contributors (see the resource agent source for information about individual authors)
resource-agents UNKNOWN 06/09/2014 OCF_HEARTBEAT_RSYNCD(7)