Sponsored Content
Top Forums Shell Programming and Scripting Reading/Compressing of log file Post 302345097 by chompy on Tuesday 18th of August 2009 11:51:27 AM
Old 08-18-2009
Depending on the nature of these logfiles, and your craftiness, you might be able to use a tool like logwatch to do the parsing for you.

20-30gb log file?!?!?! What happened to syslog?
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Log File Writing and Reading

Hi all, I have the following shell script code which tries to sftp and writes the log into the log file. TestConnection () { echo 'Connection to ' $DESTUSERNAME@$DESTHOSTNAME $SETDEBUG if ]; then rm $SCRIPT ; fi touch $SCRIPT echo "cd" $REMOTEDIR >> $SCRIPT echo "quit" >>... (10 Replies)
Discussion started by: valluvan
10 Replies

2. UNIX for Dummies Questions & Answers

Compressing of log files

Hello All My first post in the forum. :) I've this huge log files of size 20GB-30 GB in my unix server. I want to analyse the log file for some error messages. But because of the enormity of the size of these files i'm not able to grep/search the pattern in the file . Also, tried to gzip the... (1 Reply)
Discussion started by: sgbhat
1 Replies

3. Shell Programming and Scripting

Searching for Log / Bad file and Reading and writing to a flat file

Need to develop a unix shell script for the below requirement and I need your assistance: 1) search for file.log and file.bad file in a directory and read them 2) pull out "Load_Start_Time", "Data_File_Name", "Error_Type" from log file 4) concatinate each row from bad file as... (3 Replies)
Discussion started by: mlpathir
3 Replies

4. Shell Programming and Scripting

Is there any way to find the compressed size of a file without compressing it in linux

i need to backup a directory from one partition to another and and compress that directory after backing up, so i need to predict the compressed size of the directory with out actually compressing it, to check whether the space is available in the destination partition to accommodate the zipped... (2 Replies)
Discussion started by: Kesavan
2 Replies

5. Shell Programming and Scripting

Compressing previous log automatically

I want to create a script that will zip the previous log. Example. abc.log.2012.12.02 abc.log.2012.12.01.gzip abc.log If today is 2012.12.03 , my current log is abc.log and my previous date is 2012.12.02, i want abc.log.2012.12.02 to compress everytime I run the script. I can... (5 Replies)
Discussion started by: kaibiganmi
5 Replies

6. Shell Programming and Scripting

Changing the file name while compressing

Hi, Is there any way to change the file name while compressing ? using Compress and gzip and tar ? Say, I have a file foo.txt - so I have to compress this file and the resultant file name is foo.txt_20130113.gz or foo.txt_20130113.Z This to be done while performing the compression... (2 Replies)
Discussion started by: karumudi7
2 Replies

7. Shell Programming and Scripting

Compressing old files as zip file through script

I have below files in foler one/archive> one. txt 6/21/2013 two txt 7/23/2013 three.txt 6/20/2013 I wanted to move all the old files (>30 days) compressing single .zip file into one/archive/ as below two txt 7/23/2013 oldfiles.zip 6/21/2013 Please provide... (6 Replies)
Discussion started by: Ganesh L
6 Replies

8. Shell Programming and Scripting

Reading line by line from live log file using while loop and considering only those lines start from

Hi, I want to read a live log file line by line and considering those line which start from time stamp; Below code I am using, which read line but throws an exception when comparing line that does not contain error code tail -F /logs/COMMON-ERROR.log | while read myline; do... (2 Replies)
Discussion started by: ketanraut
2 Replies

9. Shell Programming and Scripting

Reading a log file

Hi, I'm trying to write a script to go through few folders read some log files and make a list from the data. the log files are in few steps each step having a final energy but I need to read the last final energy coming after the keyword "hurray". I have so far accomplished a code like... (16 Replies)
Discussion started by: raymondg
16 Replies

10. Homework & Coursework Questions

Trouble with Shell Script Compressing file

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: You will create a shell script that performs the following action: Use a parameter to pass a file that... (5 Replies)
Discussion started by: Luvs2drnk
5 Replies
LOGWATCH(8)							   User Manuals 						       LOGWATCH(8)

NAME
logwatch - system log analyzer and reporter SYNOPSIS
logwatch [--detail level ] [--logfile log-file-group ] [--service service-name ] [--mailto address ] [--archives] [--range range ] [--debug level ] [--filename file-name ] [--logdir directory ] [--hostname hostname ] [--hostformat host based options ] [--output output-type ] [--format report format ] [--encode encoding to use ] [--numeric] [--version] [--help|--usage] DESCRIPTION
Logwatch is a customizable, pluggable log-monitoring system. It will go through your logs for a given period of time and make a report in the areas that you wish with the detail that you wish. Logwatch is being used for Linux and many types of UNIX. OPTIONS
--detail level This is the detail level of the report. level can be a positive integer, or high, med, low, which correspond to the integers 10, 5, and 0, repectively. --logfile log-file-group This will force Logwatch to process only the set of logfiles defined by log-file-group (i.e. messages, xferlog, ...). Logwatch will therefore process all services that use those logfiles. This option can be specified more than once to specify multiple logfile- groups. --service service-name This will force Logwatch to process only the service specified in service-name (i.e. login, pam, identd, ...). Logwatch will there- fore also process any log-file-groups necessary to process these services. This option can be specified more than once to specify multiple services to process. A useful service-name is All which will process all services (and logfile-groups) for which you have filters installed. --mailto address Mail the results to the email address or user specified in address. This option overrides the --print option. --range range You can specify a date-range to process. Common ranges are Yesterday, Today, All, and Help. Additional options are listed when invoked with the Help parameter. --archives Each log-file-group has basic logfiles (i.e. /var/log/messages) as well as archives (i.e. /var/log/messages.? or /var/log/mes- sages.?.gz). When used with "--range all", this option will make Logwatch search through the archives in addition to the regular logfiles. For other values of --range, Logwatch will search the appropriate archived logs. --debug level For debugging purposes. level can range from 0 to 100. This will really clutter up your output. You probably don't want to use this. --filename file-name Save the output to file-name instead of displaying or mailing it. --logdir directory Look in directory for log subdirectories or log files instead of the default directory. --hostname hostname Use hostname for the reports instead of this system's hostname. In addition, if HostLimit is set in the logwatch.conf configuration file (see MORE INFORMATION, below), then only logs from this hostname will be processed (where appropriate). --numeric Inhibits additional name lookups, displaying IP addresses numerically. --usage Displays usage information --help same as --usage. FILES
/usr/share/logwatch/ This directory contains all the perl executables and configuration files shipped with the logwatch distribution. /etc/logwatch This directory contains local configuration files that override the default configuration. See MORE INFORMATION below for more information. EXAMPLES
logwatch --service ftpd-xferlog --range all --detail high --print --archives This will print out all FTP transfers that are stored in all current and archived xferlogs. logwatch --service pam_pwdb --range yesterday --detail high --print This will print out login information for the previous day... MORE INFORMATION
The directory /usr/share/doc/logwatch-* contains several files with additional documentation: HOWTO-Customize-LogWatch Documents the directory structure of Logwatch configuration and executable files, and describes how to customize Logwatch by over- riding these default files. LICENSE Describes the License under which Logwatch is distributed. Additional clauses may be specified in individual files. README Describes how to install, where to find it, mailing lists, and other useful information. AUTHOR
Kirk Bauer <kirk@kaybee.org> http://www.kaybee.org/~kirk ftp://ftp.kaybee.org/pub/redhat/RPMS Linux October 2005 LOGWATCH(8)
All times are GMT -4. The time now is 02:14 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy