Sponsored Content
Top Forums Shell Programming and Scripting Script to archive logs and sftp to another archive server Post 303012094 by Arjun Goswami on Tuesday 30th of January 2018 12:38:16 AM
Old 01-30-2018
Script to archive logs and sftp to another archive server

Requirement:

Under fuse application we have placeholders called containers;



Every container has their logs under:

Code:
<container1>/data/log/fuse.log
<container1>/data/log/fuse.log.1
<container1>/data/log/fuse.log.XX
   
<container2>/data/log/fuse.log
<container2>/data/log/fuse.log.1
<container2>/data/log/fuse.log.XX




Our requirement is to have the logs moved from the server to another archiver server using SFTP after compressing it.


We have 9 servers and each of them will get their logs stored in the archiver server in different paths.

Lets say,

1st node has 3 containers
The archiver script will make a directory in the archiver server and store the compressed files as below.

Code:
/<app>/Prod/Cluster1/<Node1>/<Container1>/<compressed logs>
/<app>/Prod/Cluster1/<Node1>/<Container2>/<compressed logs>
/<app>/Prod/Cluster1/<Node1>/<Container3>/<compressed logs>

Similarly,

2nd node has 3 containers
Code:
/<app>/Prod/Cluster1/<Node2>/<Container1>/<compressed logs>
/<app>/Prod/Cluster1/<Node2>/<Container2>/<compressed logs>
/<app>/Prod/Cluster1/<Node2>/<Container3>/<compressed logs>

Any suggestion on this would be of great help Smilie

---------- Post updated at 11:08 AM ---------- Previous update was at 10:41 AM ----------

We had this implemented using perl and ftp but now we need to implement shell script with sftp
Smilie

Last edited by Scott; 01-30-2018 at 03:33 AM.. Reason: Code tags
 

9 More Discussions You Might Find Interesting

1. Solaris

Archive data accross solaris server

Can anyone tell me how to safely archive data across the network instead of tape in Solaris. Thanks Remi (1 Reply)
Discussion started by: Remi
1 Replies

2. UNIX for Advanced & Expert Users

how to archive logs older than 5 days & then delete them?

My code is tar -cvf logs.tar `find /usr/openv/logs/512*.log -mtime +2` && find *.log* -mtime +2 -exec rm {} \; this gives me output as: tar: Missing filenames:confused: (1 Reply)
Discussion started by: timus1980
1 Replies

3. Shell Programming and Scripting

simple script to alert of archive logs filling

Hi all. I am not a DBA. But I do have responsibility for making sure the archive logs dont fill up and cause the database. This happend the other day while I was absent (sick) and I got a good ticking off for it. Needless to say I dont want this happen! Could anyone lend a hand to a... (8 Replies)
Discussion started by: Incremental
8 Replies

4. Shell Programming and Scripting

Extracting from archive | compressing to new archive

Hi there, I have one huge archive (it's a system image). I need sometime to create smaller archives with only one or two file from my big archive. So I'm looking for a command that extracts files from an archive and pipe them to another one. I tried the following : tar -xzOf oldarchive.tgz... (5 Replies)
Discussion started by: chebarbudo
5 Replies

5. Shell Programming and Scripting

Archive log Pull script from one server to another server

Hi all, Iam looking out for a shell script which pulls archive from Server A to Server B. And at the same time, we dont want the archives which are already pulled in Server B to be removed from Server A. Please help me on this, I have been trying on this for a quiet few time. ... (3 Replies)
Discussion started by: vivi.raghav
3 Replies

6. Shell Programming and Scripting

Connect to Server B from Server A and Archive the file

Hi I need to connect to a ServerB from Server A eg: Server A : 172.20.273.51, un: xxx, pwd: xxx Server B: 172.20.273.51, un:yyy, pwd: yyy Need to copy the files(with name starts with IME*) from /grid/pc/IME* to /grid/pc/archive What will be the command or script to... (3 Replies)
Discussion started by: vsmeruga
3 Replies

7. UNIX for Dummies Questions & Answers

Archive files that was already sftp'd/process

Hi, I need to modify my archive script to archive only files which is already sftp to the windows. We encounter some issue which the file is already archiving though it is not yet sftp'd to the Windows. Any answers.?Thanks! (1 Reply)
Discussion started by: sonja
1 Replies

8. Solaris

archive logs mount point space check script

I have the below shell script which is checking /archlog mount point space on cappire(solaris 10) server. When the space usage is above 80% it should e-mail. When i tested this script it is working as expected. -------------------------------------------------------------------------... (0 Replies)
Discussion started by: dreams5617
0 Replies

9. Shell Programming and Scripting

SFTP file and archive only if its succedded

Hi All, I have a requirement where I have to archive the file as soon as I sFTP the file to remote location below is the code I am going with ls $FILE_NAME | while read FNAME do sftp -v -oport=nn $FTP_USER@$FTP_HOST <<EOF 2>&1 | tee -a ${LOG_FILE_NAME} cd ${TGT_FOLDER_NAME} lcd... (4 Replies)
Discussion started by: Pratik4891
4 Replies
fuse-utils(1)							     Emulators							     fuse-utils(1)

NAME
fuse-utils - various ZX Spectrum related utilities DESCRIPTION
The Fuse (Free Unix Spectrum Emulator) utilities are a small suite of utilities relating to some of the file formats used by ZX Spectrum emulators. The currently available utilities are: audio2tape Convert an audio file into a tape format. createhdf Create a blank IDE hard disk image in .hdf format. listbasic Extract the BASIC program from a ZX Spectrum file. profile2map Convert Fuse profiler output into Z80-style map format. raw2hdf Create an .hdf hard disk image from a raw data file. rzxcheck Verify the `competition mode' signature in an RZX input recording file. rzxdump Display the contents of an RZX file. rzxtool Make simple modifications to an RZX file. scl2trd Convert .scl disk images into .trd format. snap2tzx Convert a snapshot into a .tzx tape image. snapconv Convert between various snapshot formats. tape2wav Convert a tape file into .wav audio format. tapeconv Convert between various tape formats. tzxlist Display the contents of a .tzx tape image. SEE ALSO
audio2tape(1), createhdf(1), fuse(1), libspectrum(3), listbasic(1), raw2hdf(1), rzxcheck(1), rzxdump(1), rzxtool(1), scl2trd(1), snap2tzx(1), snapconv(1), tape2wav(1), tapeconv(1), tzxlist(1), xspect(1), xzx(1) The comp.sys.sinclair Spectrum FAQ, at http://www.worldofspectrum.org/faq/index.html. AUTHOR
Philip Kendall (philip-fuse@shadowmagic.org.uk) and others; see the individual man pages for more details. Version 1.0.0 16th December, 2010 fuse-utils(1)
All times are GMT -4. The time now is 05:51 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy