gather, modify stats, email to self


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting gather, modify stats, email to self
# 1  
Old 01-08-2011
Question gather, modify stats, email to self

I have a range of log files from rsync on an Ubuntu 10.04 server. At the bottom of these files, I have "--stats" -- a rsync switch -- which produces interesting information/statistics.

Steps the script will take:
  • take the the lines that contain the words "sent, total size, bytes"
  • put these into a text file
  • format the text e.g. "MiB transferred: ####MiB"
  • pipe the text file to mailx with the subject "rsync stats yyyy/mm/dd"
  • daily cron job
I'm a novice to this sort of work. I don't know what the best tools to use would be. Perhaps I could tail the information to the text file?, and then use a tool to format the text nicely. I don't know if grep would suffice to grab those bottom 12 lines or so, or if I should dive into the sed/awk world? Maybe even a shell/perl/python script? If you could point me into a good direction with suggestions, that'd be great

Last edited by ScannerDarkly; 01-08-2011 at 08:13 AM.. Reason: typo
# 2  
Old 01-08-2011
Post the output of the last 12 lines of a logfile (tail -12 logfile) and the desired formatted output.


For sending a mail could have a you read of:

How do I send email? - The UNIX and Linux Forums

and for cron jobs:

https://www.unix.com/answers-frequent...n-crontab.html
# 3  
Old 01-08-2011
Quote:
Originally Posted by Franklin52
Post the output of the last 12 lines of a logfile (tail -12 logfile) and the desired formatted output.
If I wanted the last 12 lines, that would suffice. There are a lot of log files that differ, however. So I would need to get lines via keywords; line numbers wouldn't be as effective.


Rsync and the respective logging is what I have already set up as a cron job. I'm modifying arbitrary text into a file, formatting that text, then piping that file into mailx with a subject. The latter I am fine with -- I'm asking for advice on how to certain gather text from files that are timestamped in the format yyyymmdd, and then format that text into something easier to read. Sorry if I was unclear! Smilie
# 4  
Old 01-08-2011
Quote:
Originally Posted by ScannerDarkly
If I wanted the last 12 lines, that would suffice. There are a lot of log files that differ, however. So I would need to get lines via keywords; line numbers wouldn't be as effective.


Rsync and the respective logging is what I have already set up as a cron job. I'm modifying arbitrary text into a file, formatting that text, then piping that file into mailx with a subject. The latter I am fine with -- I'm asking for advice on how to certain gather text from files that are timestamped in the format yyyymmdd, and then format that text into something easier to read. Sorry if I was unclear! Smilie
To format the lines you could use awk, something like:
Code:
awk '
$0 ~ d { 
  # Format the line here
  # ....
  # and print the line:
  print
}' d="20110108" logfile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Create a shell script to gather user account information and displays the result to administrator

I want to create a shell script to gather user account information and displays the result to administrator. I have created a script but its showing all the information when i search for username like: amit@mx:~$ ./uinfo.sh amit Username : amit User Info ... (2 Replies)
Discussion started by: amit1986
2 Replies

2. Shell Programming and Scripting

Script to Gather data from logs and export to a CSV file

Greetings, After a few hours of trial and error, I decide to ask for some help. I am new to AWK and shell script, so please don't laugh :p I made the below script, to gather data from some logs and have the output into a CSV file : #!/bin/sh #Script to collect Errors ... (9 Replies)
Discussion started by: Yagami_Sama
9 Replies

3. Shell Programming and Scripting

Execute Oracle gather stats via shell script

Hi , I am trying to automate a gather stats in shell script #!/usr/bin/ksh export ORACLE_HOME=/orcl/app/oracle/product/11.2.0.1/db_1 export PATH="$PATH:$ORACLE_HOME/bin" export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$ORACLE_HOME/lib32" export TNS_ADMIN=/opt/netprobe/config... (1 Reply)
Discussion started by: neil.k
1 Replies

4. UNIX for Dummies Questions & Answers

How can I use my code to gather information from a file in a completely different directory?

I need my code to compare two different files that are in two completely different directories, How can I do this? So for example, my code will look at file1 which is in my home directory, and compare the files with those from file2 that is in /abc/adf/adr/afc/adf/file2... does that make sense? (1 Reply)
Discussion started by: castrojc
1 Replies

5. AIX

how to gather HMC information

does somebody know how to gather HMC information? It looks like there is some tool can gather HMC configuration to a html file and make them as a xx.tar.gz file? (2 Replies)
Discussion started by: rainbow_bean
2 Replies

6. AIX

Using NIM to gather system information

Hi, Need help if its possible to use NIM server to gather information. Basically, we need to gather firmware version and oslevel for environment wide servers (these servers are connected thru nim). I understand you can use NIM script resource to trigger the script but don't know if possible to... (3 Replies)
Discussion started by: depam
3 Replies

7. AIX

IO Stats

Aix 5.3 I am trying to view the IO stats. I do the sar 5 5 but that is the WIO and si different than the IO stats right? I am just blanking on this. I know there is a command that I used to run that brings up a whole bunch of live stats that run live such as mem and so on just can't rememeber... (4 Replies)
Discussion started by: rocker40
4 Replies

8. Shell Programming and Scripting

FTP script to gather logs.

Hi, I am currently working on a FTP script which would ftp to extranet servers from intranet server and gather logs based on timestamps in archive logs and by using a wildcard character in the present logs. I have the following in place AAA="Extranet Server 1:Instance Extranet Server... (2 Replies)
Discussion started by: openspark
2 Replies

9. Shell Programming and Scripting

Gather File permission during scripting on unix as numbers.

Hi, I have a script with following file permission on box. -rwxr-xr-x 1 root system 15347 Aug 14 15:08 b_reboot.ksh Without calculating or watching at -rwxr-xr-x (permission's) of this above mentioned file. I would like to get the file permission assigned to a file. Basically... (7 Replies)
Discussion started by: ajilesh
7 Replies

10. Shell Programming and Scripting

script to gather weblogic jvm heap size stats

Hello, has anyone written something that will monitor/gather weblogic heap info ? I need to gather size, high/low stats to a file that I can upload to a speadsheet thanks for your help! (2 Replies)
Discussion started by: galenw
2 Replies
Login or Register to Ask a Question