Splitting log files

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,...

2. Shell Programming and Scripting

Splitting files into 100 files with field value

I want a script to split my file upon the last field (15) As file A,b,c,.......,01 C,v,n,.......,02 C,r,v,........,01 F,s,a,........,03 X,y,d,........,99 To make output 01.txt A,b,c,.......,01 C,r,v,........,01 02.txt C,v,n,.......,02

3. Shell Programming and Scripting

Splitting files

Hello all I have a file which has around 80 million records, I want to split it to 12 equal files, I tried using the split command but it is allowing me to split according to number of lines or by size. Is there a way i can split the file into 12 files without worrying about the number of lines...

4. UNIX for Dummies Questions & Answers

Splitting Files Help

Hi Gurus, Lets say i have a file with some 30 records... How can i split that file into 3 files Also it shud be dynamic in the sense.. I wouldnt mind if file 1 has 15, file 2 has 10 and file 3 has 5.... Please help.. Thanks

5. Shell Programming and Scripting

Splitting file into 2 files ?

Hi extending to one of my previous posted query .... I am using nawk -v invar1="$aa" '{print > ("ABS\_"((/\|/)?"A\_":"B\_")invar1"\_NETWORKID.txt")}' spfile.txt to get 2 different files based on split condition i.e. "|" Similar to invar1 variable in nawk I also need one more variable...

6. Shell Programming and Scripting

Splitting files from one file

Hi, I have an input file like: 111 abcdefgh asdfghjk dfghjkl 222 aaaaaaa bbbbbb 333 djfhfgjktitjhgfkg 444 djdhfjkhfjkghjkfg hsbfjksdbhjkgherjklg fjkhfjklsahjgh fkrjkgnj I want to read this input file and make separate output files with the header as numric value like "111"...

7. UNIX for Dummies Questions & Answers

splitting the files

Hi, I have some files with 2 million odd records which i need to split into chunks of 0.5 millions. I have the file sorted with a key column in order. The same key value can appear as 4 or 5 records in the file. Hence after splitting we are checking whether all the key values are present in the...

8. Shell Programming and Scripting

Splitting input files into multiple files through AWK command

Hi, I needs to split *.txt files from single directory depends on the some mutltiple input values. i have wrote the code like below for file in *.txt do grep -i -h "value1|value2" $file > $file; done. My requirment is more input values needs to be given in grep; let us say 50...

9. UNIX for Advanced & Expert Users

splitting the files

Hi, How can i split the big file by the lines?. For eg. I wanna split the file from the line 140 to 1700.

10. Shell Programming and Scripting

splitting the files

Hi, I'am using HP-UX.I have a input file which has 102 drop statements in it.I'am using csplit to split the files.The upper limit is 99 only.I'am using the -n 102 option.It says "suffix size not vaild".Any suggestions how to do it using csplit? Thanx in advance, sounder.

LEARN ABOUT DEBIAN

lr_vendor_cron

LR_VENDOR_CRON.IN(1)					  LogReport's Lire Documentation				      LR_VENDOR_CRON.IN(1)

NAME

       lr_vendor_cron - generate Lire reports from all known log files on the system

SYNOPSIS

       lr_vendor_cron daily|weekly|monthly

DESCRIPTION

       lr_vendor_cron is designed to be invoked by cron.  On systems supporting /etc/cron.d/, this script can be invoked by creating
       /etc/cron.d/lire, which should read e.g.:

	02 5 * * * root /usr/sbin/lr_vendor_cron daily
	22 5 * * 0 root /usr/sbin/lr_vendor_cron weekly
	42 5 1 * * root /usr/sbin/lr_vendor_cron monthly

       lr_vendor_cron reads files /etc/sysconfig/lire.d/*.cfg and /etc/default/lire.d/*.cfg.  For each log file, such a file should exist.  The
       files should set the following shell variables:

       daemon
	   the name of a binary which generated the specific log.  This is used to test wether the *.cfg-file is suitable for the current system.
	   If you'd like to generate a log for which no daemon is present on the current system, just use a placeholder like "daemon=/bin/true".

       service
	   a Lire service, e.g. sendmail,

       logfile
	   e.g.  /var/log/mail.log.1.gz,

       filter
	   e.g. 'gzip -dc' or 'zcat | grep sendmail', and

       rotateperiod
	   one of the supported lr_vendor_cron arguments.

       extraopts
	   optional: extra options passed to lr_log2mail(1).  Set this to e.g. '--template www_local'.

       After parsing the configuration files, lr_vendor_cron invokes lr_log2mail(1) on the `filter'ed log file. Reading and filtering the log is
       done as root, lr_log2mail is run as user lire, using su(1).  Output is fed to logger(1).

SECURITY

       lr_vendor_cron is supposed to be run as root.  However, please note that this script is distributed WITHOUT ANY WARRANTY.  If you don't
       like to run such a script as root, make sure your lire user has read access to all log files you'd like to get processed, and adapt this
       script.	We supply this script mainly for people building packages.  On many Unix and GNU/Linux distributions, it's hard to enforce log
       file readability by the lire user.  This script is mainly a way to work around this.

ENVIRONMENT VARIABLES

       lr_vendor_cron honours DEBUG, CONFDIR, PRIORITY (for logger(1) call), and LIREUSER.  Beware!  PRIORITY as used here is independent of the
       LR_FACILITY as set in /etc/lire/defaults.  This script does not use any of the settings in /etc/lire/defaults.

VERSION

       $Id: lr_vendor_cron.in,v 1.14 2006/07/23 13:16:33 vanbaal Exp $

COPYRIGHT

       Copyright (C) 2002, 2003 Stichting LogReport Foundation LogReport@LogReport.org Copyright (C) 2005 Joost van Baal

       This program is part of Lire.

       Lire is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free
       Software Foundation; either version 2 of the License, or (at your option) any later version.

       This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.

       You should have received a copy of the GNU General Public License along with this program (see COPYING); if not, check with
       http://www.gnu.org/copyleft/gpl.html.

AUTHORS

       Francis J. Lacoste <flacoste@logreport.org>, Joost van Baal <joostvb@logreport.org>

Lire 2.1.1							    2006-07-23						      LR_VENDOR_CRON.IN(1)

UNIX for Dummies Questions & Answers