Sponsored Content
Top Forums Shell Programming and Scripting Script to archive logs and sftp to another archive server Post 303012094 by Arjun Goswami on Tuesday 30th of January 2018 12:38:16 AM
Old 01-30-2018
Script to archive logs and sftp to another archive server

Requirement:

Under fuse application we have placeholders called containers;



Every container has their logs under:

Code:
<container1>/data/log/fuse.log
<container1>/data/log/fuse.log.1
<container1>/data/log/fuse.log.XX
   
<container2>/data/log/fuse.log
<container2>/data/log/fuse.log.1
<container2>/data/log/fuse.log.XX




Our requirement is to have the logs moved from the server to another archiver server using SFTP after compressing it.


We have 9 servers and each of them will get their logs stored in the archiver server in different paths.

Lets say,

1st node has 3 containers
The archiver script will make a directory in the archiver server and store the compressed files as below.

Code:
/<app>/Prod/Cluster1/<Node1>/<Container1>/<compressed logs>
/<app>/Prod/Cluster1/<Node1>/<Container2>/<compressed logs>
/<app>/Prod/Cluster1/<Node1>/<Container3>/<compressed logs>

Similarly,

2nd node has 3 containers
Code:
/<app>/Prod/Cluster1/<Node2>/<Container1>/<compressed logs>
/<app>/Prod/Cluster1/<Node2>/<Container2>/<compressed logs>
/<app>/Prod/Cluster1/<Node2>/<Container3>/<compressed logs>

Any suggestion on this would be of great help Smilie

---------- Post updated at 11:08 AM ---------- Previous update was at 10:41 AM ----------

We had this implemented using perl and ftp but now we need to implement shell script with sftp
Smilie

Last edited by Scott; 01-30-2018 at 03:33 AM.. Reason: Code tags
 

9 More Discussions You Might Find Interesting

1. Solaris

Archive data accross solaris server

Can anyone tell me how to safely archive data across the network instead of tape in Solaris. Thanks Remi (1 Reply)
Discussion started by: Remi
1 Replies

2. UNIX for Advanced & Expert Users

how to archive logs older than 5 days & then delete them?

My code is tar -cvf logs.tar `find /usr/openv/logs/512*.log -mtime +2` && find *.log* -mtime +2 -exec rm {} \; this gives me output as: tar: Missing filenames:confused: (1 Reply)
Discussion started by: timus1980
1 Replies

3. Shell Programming and Scripting

simple script to alert of archive logs filling

Hi all. I am not a DBA. But I do have responsibility for making sure the archive logs dont fill up and cause the database. This happend the other day while I was absent (sick) and I got a good ticking off for it. Needless to say I dont want this happen! Could anyone lend a hand to a... (8 Replies)
Discussion started by: Incremental
8 Replies

4. Shell Programming and Scripting

Extracting from archive | compressing to new archive

Hi there, I have one huge archive (it's a system image). I need sometime to create smaller archives with only one or two file from my big archive. So I'm looking for a command that extracts files from an archive and pipe them to another one. I tried the following : tar -xzOf oldarchive.tgz... (5 Replies)
Discussion started by: chebarbudo
5 Replies

5. Shell Programming and Scripting

Archive log Pull script from one server to another server

Hi all, Iam looking out for a shell script which pulls archive from Server A to Server B. And at the same time, we dont want the archives which are already pulled in Server B to be removed from Server A. Please help me on this, I have been trying on this for a quiet few time. ... (3 Replies)
Discussion started by: vivi.raghav
3 Replies

6. Shell Programming and Scripting

Connect to Server B from Server A and Archive the file

Hi I need to connect to a ServerB from Server A eg: Server A : 172.20.273.51, un: xxx, pwd: xxx Server B: 172.20.273.51, un:yyy, pwd: yyy Need to copy the files(with name starts with IME*) from /grid/pc/IME* to /grid/pc/archive What will be the command or script to... (3 Replies)
Discussion started by: vsmeruga
3 Replies

7. UNIX for Dummies Questions & Answers

Archive files that was already sftp'd/process

Hi, I need to modify my archive script to archive only files which is already sftp to the windows. We encounter some issue which the file is already archiving though it is not yet sftp'd to the Windows. Any answers.?Thanks! (1 Reply)
Discussion started by: sonja
1 Replies

8. Solaris

archive logs mount point space check script

I have the below shell script which is checking /archlog mount point space on cappire(solaris 10) server. When the space usage is above 80% it should e-mail. When i tested this script it is working as expected. -------------------------------------------------------------------------... (0 Replies)
Discussion started by: dreams5617
0 Replies

9. Shell Programming and Scripting

SFTP file and archive only if its succedded

Hi All, I have a requirement where I have to archive the file as soon as I sFTP the file to remote location below is the code I am going with ls $FILE_NAME | while read FNAME do sftp -v -oport=nn $FTP_USER@$FTP_HOST <<EOF 2>&1 | tee -a ${LOG_FILE_NAME} cd ${TGT_FOLDER_NAME} lcd... (4 Replies)
Discussion started by: Pratik4891
4 Replies
nislog(1M)                                                System Administration Commands                                                nislog(1M)

NAME
nislog - display the contents of the NIS+ transaction log SYNOPSIS
/usr/sbin/nislog [-h num | -t num] [-v] [directory...] DESCRIPTION
nislog displays the contents of the NIS+ server transaction log on the standard output. This command can be used to track changes in the namespace. The /var/nis/trans.log file contains the transaction log maintained by the NIS+ server. When updates occur, they are logged to this file and then propagated to replicas as log transactions. When the log is checkpointed, updates that have been propagated to the replicas are removed. The nislog command can only be run on an NIS+ server by superuser. It displays the log entries for that server only. If directory is not specified, the entire log is searched. Otherwise, only those logs entries that correspond to the specified directories are displayed. OPTIONS
-h num Display num transactions from the ``head'' of the log. If the numeric parameter is 0, only the log header is displayed. -t num Display num transactions from the ``tail'' of the log. If the numeric parameter is 0, only the log header is displayed. -v Verbose mode. FILES
/var/nis/trans.log transaction log ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWnisu | +-----------------------------+-----------------------------+ SEE ALSO
nis+(1), rpc.nisd(1M), nisfiles(4), attributes(5) NOTES
NIS+ might not be supported in future releases of the SolarisTM Operating Environment. Tools to aid the migration from NIS+ to LDAP are available in the Solaris 9 operating environment. For more information, visit http://www.sun.com/directory/nisplus/transition.html. SunOS 5.10 12 Dec 2001 nislog(1M)
All times are GMT -4. The time now is 04:37 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy