Sponsored Content
Top Forums Shell Programming and Scripting Script to archive logs and sftp to another archive server Post 303012094 by Arjun Goswami on Tuesday 30th of January 2018 12:38:16 AM
Old 01-30-2018
Script to archive logs and sftp to another archive server

Requirement:

Under fuse application we have placeholders called containers;



Every container has their logs under:

Code:
<container1>/data/log/fuse.log
<container1>/data/log/fuse.log.1
<container1>/data/log/fuse.log.XX
   
<container2>/data/log/fuse.log
<container2>/data/log/fuse.log.1
<container2>/data/log/fuse.log.XX




Our requirement is to have the logs moved from the server to another archiver server using SFTP after compressing it.


We have 9 servers and each of them will get their logs stored in the archiver server in different paths.

Lets say,

1st node has 3 containers
The archiver script will make a directory in the archiver server and store the compressed files as below.

Code:
/<app>/Prod/Cluster1/<Node1>/<Container1>/<compressed logs>
/<app>/Prod/Cluster1/<Node1>/<Container2>/<compressed logs>
/<app>/Prod/Cluster1/<Node1>/<Container3>/<compressed logs>

Similarly,

2nd node has 3 containers
Code:
/<app>/Prod/Cluster1/<Node2>/<Container1>/<compressed logs>
/<app>/Prod/Cluster1/<Node2>/<Container2>/<compressed logs>
/<app>/Prod/Cluster1/<Node2>/<Container3>/<compressed logs>

Any suggestion on this would be of great help Smilie

---------- Post updated at 11:08 AM ---------- Previous update was at 10:41 AM ----------

We had this implemented using perl and ftp but now we need to implement shell script with sftp
Smilie

Last edited by Scott; 01-30-2018 at 03:33 AM.. Reason: Code tags
 

9 More Discussions You Might Find Interesting

1. Solaris

Archive data accross solaris server

Can anyone tell me how to safely archive data across the network instead of tape in Solaris. Thanks Remi (1 Reply)
Discussion started by: Remi
1 Replies

2. UNIX for Advanced & Expert Users

how to archive logs older than 5 days & then delete them?

My code is tar -cvf logs.tar `find /usr/openv/logs/512*.log -mtime +2` && find *.log* -mtime +2 -exec rm {} \; this gives me output as: tar: Missing filenames:confused: (1 Reply)
Discussion started by: timus1980
1 Replies

3. Shell Programming and Scripting

simple script to alert of archive logs filling

Hi all. I am not a DBA. But I do have responsibility for making sure the archive logs dont fill up and cause the database. This happend the other day while I was absent (sick) and I got a good ticking off for it. Needless to say I dont want this happen! Could anyone lend a hand to a... (8 Replies)
Discussion started by: Incremental
8 Replies

4. Shell Programming and Scripting

Extracting from archive | compressing to new archive

Hi there, I have one huge archive (it's a system image). I need sometime to create smaller archives with only one or two file from my big archive. So I'm looking for a command that extracts files from an archive and pipe them to another one. I tried the following : tar -xzOf oldarchive.tgz... (5 Replies)
Discussion started by: chebarbudo
5 Replies

5. Shell Programming and Scripting

Archive log Pull script from one server to another server

Hi all, Iam looking out for a shell script which pulls archive from Server A to Server B. And at the same time, we dont want the archives which are already pulled in Server B to be removed from Server A. Please help me on this, I have been trying on this for a quiet few time. ... (3 Replies)
Discussion started by: vivi.raghav
3 Replies

6. Shell Programming and Scripting

Connect to Server B from Server A and Archive the file

Hi I need to connect to a ServerB from Server A eg: Server A : 172.20.273.51, un: xxx, pwd: xxx Server B: 172.20.273.51, un:yyy, pwd: yyy Need to copy the files(with name starts with IME*) from /grid/pc/IME* to /grid/pc/archive What will be the command or script to... (3 Replies)
Discussion started by: vsmeruga
3 Replies

7. UNIX for Dummies Questions & Answers

Archive files that was already sftp'd/process

Hi, I need to modify my archive script to archive only files which is already sftp to the windows. We encounter some issue which the file is already archiving though it is not yet sftp'd to the Windows. Any answers.?Thanks! (1 Reply)
Discussion started by: sonja
1 Replies

8. Solaris

archive logs mount point space check script

I have the below shell script which is checking /archlog mount point space on cappire(solaris 10) server. When the space usage is above 80% it should e-mail. When i tested this script it is working as expected. -------------------------------------------------------------------------... (0 Replies)
Discussion started by: dreams5617
0 Replies

9. Shell Programming and Scripting

SFTP file and archive only if its succedded

Hi All, I have a requirement where I have to archive the file as soon as I sFTP the file to remote location below is the code I am going with ls $FILE_NAME | while read FNAME do sftp -v -oport=nn $FTP_USER@$FTP_HOST <<EOF 2>&1 | tee -a ${LOG_FILE_NAME} cd ${TGT_FOLDER_NAME} lcd... (4 Replies)
Discussion started by: Pratik4891
4 Replies
DOVEADM-LOG(1)							      Dovecot							    DOVEADM-LOG(1)

NAME
doveadm-log - Locate, test or reopen Dovecot's log files SYNOPSIS
doveadm [-Dv] log errors doveadm [-Dv] log find [directory] doveadm [-Dv] log reopen doveadm [-Dv] log test DESCRIPTION
The doveadm log commands are used to locate and reopen the log files of dovecot(1). It's also possible to test the configured targets of the *log_path settings. OPTIONS
Global doveadm(1) options: -D Enables verbosity and debug messages. -v Enables verbosity, including progress counter. COMMANDS
log errors doveadm log errors The log errors command is used to show the last - up to 1,000 - errors and warnings. If no output is generated, no errors have occurred since the last start. log find doveadm log find [directory] The log find command is used to show the location of the log files, to which dovecot(1) sends its log messages. If dovecot(1) logs its messages through syslogd(8) and doveadm(1) could not find any log files, you can specify the directory where your syslogd writes its log files. log reopen doveadm log reopen This command causes doveadm to reopen all log files, configured in the log_path, info_log_path and debug_log_path settings. These settings are configured in /etc/dovecot/conf.d/10-logging.conf. This is for example useful after manually rotating the log files. log test doveadm log test This command causes doveadm to write the message "This is Dovecot's priority log (timestamp)" to the configured log files. The used prior- ities are: debug, info, warning, error and fatal. EXAMPLE
This example shows how to locate the log files used by dovecot(1). doveadm log find Looking for log files from /var/log Debug: /var/log/dovecot.debug Info: /var/log/mail.log Warning: /var/log/mail.log Error: /var/log/mail.log Fatal: /var/log/mail.log REPORTING BUGS
Report bugs, including doveconf -n output, to the Dovecot Mailing List <dovecot@dovecot.org>. Information about reporting bugs is avail- able at: http://dovecot.org/bugreport.html SEE ALSO
doveadm(1) Dovecot v2.1 2012-02-22 DOVEADM-LOG(1)
All times are GMT -4. The time now is 02:59 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy