Sponsored Content
Full Discussion: Splitting log files
Top Forums UNIX for Dummies Questions & Answers Splitting log files Post 302968339 by Stacked on Tuesday 8th of March 2016 12:08:20 PM
Old 03-08-2016
Splitting log files

I am trying to split my IRSSI logs into weekly and monthly .log files. My log format looks like this:

Code:
--- Day changed Fri Mar 04 2016
00:11 <Jack> Test
--- Day changed Sat Mar 05 2016
00:11 <Jack> Test 
--- Day changed Sun Mar 06 2016
15:20 <Jack> Test

The script I have been playing with that I am stuck on to split the logs looks like this:
Code:
grep -m 1 -n --regexp="^--- Day changed `date --date="$2" +"%a %b %d %Y"`$" $1 | cut -d ":" -f 1 | xargs -I '{}' tail --lines=+{} $1

$1 = The location of the .log file to split.
$2 = A date string, I use "1 day ago" as an example.

When I run this script it will create my #channel-weekly.log file but it is empty, it isn't splitting the x number of days I tell it to.

All help appreciated, thanks!

Also, thanks to all of the community members on this forum, it has helped me a lot in the past!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

splitting the files

Hi, I'am using HP-UX.I have a input file which has 102 drop statements in it.I'am using csplit to split the files.The upper limit is 99 only.I'am using the -n 102 option.It says "suffix size not vaild".Any suggestions how to do it using csplit? Thanx in advance, sounder. (1 Reply)
Discussion started by: sounder123
1 Replies

2. UNIX for Advanced & Expert Users

splitting the files

Hi, How can i split the big file by the lines?. For eg. I wanna split the file from the line 140 to 1700. (9 Replies)
Discussion started by: sharif
9 Replies

3. Shell Programming and Scripting

Splitting input files into multiple files through AWK command

Hi, I needs to split *.txt files from single directory depends on the some mutltiple input values. i have wrote the code like below for file in *.txt do grep -i -h "value1|value2" $file > $file; done. My requirment is more input values needs to be given in grep; let us say 50... (3 Replies)
Discussion started by: arund_01
3 Replies

4. UNIX for Dummies Questions & Answers

splitting the files

Hi, I have some files with 2 million odd records which i need to split into chunks of 0.5 millions. I have the file sorted with a key column in order. The same key value can appear as 4 or 5 records in the file. Hence after splitting we are checking whether all the key values are present in the... (5 Replies)
Discussion started by: dnat
5 Replies

5. Shell Programming and Scripting

Splitting files from one file

Hi, I have an input file like: 111 abcdefgh asdfghjk dfghjkl 222 aaaaaaa bbbbbb 333 djfhfgjktitjhgfkg 444 djdhfjkhfjkghjkfg hsbfjksdbhjkgherjklg fjkhfjklsahjgh fkrjkgnj I want to read this input file and make separate output files with the header as numric value like "111"... (9 Replies)
Discussion started by: saltysumi
9 Replies

6. Shell Programming and Scripting

Splitting file into 2 files ?

Hi extending to one of my previous posted query .... I am using nawk -v invar1="$aa" '{print > ("ABS\_"((/\|/)?"A\_":"B\_")invar1"\_NETWORKID.txt")}' spfile.txt to get 2 different files based on split condition i.e. "|" Similar to invar1 variable in nawk I also need one more variable... (18 Replies)
Discussion started by: shekharjchandra
18 Replies

7. UNIX for Dummies Questions & Answers

Splitting Files Help

Hi Gurus, Lets say i have a file with some 30 records... How can i split that file into 3 files Also it shud be dynamic in the sense.. I wouldnt mind if file 1 has 15, file 2 has 10 and file 3 has 5.... Please help.. Thanks (6 Replies)
Discussion started by: saggiboy10
6 Replies

8. Shell Programming and Scripting

Splitting files

Hello all I have a file which has around 80 million records, I want to split it to 12 equal files, I tried using the split command but it is allowing me to split according to number of lines or by size. Is there a way i can split the file into 12 files without worrying about the number of lines... (7 Replies)
Discussion started by: Sri3001
7 Replies

9. Shell Programming and Scripting

Splitting files into 100 files with field value

I want a script to split my file upon the last field (15) As file A,b,c,.......,01 C,v,n,.......,02 C,r,v,........,01 F,s,a,........,03 X,y,d,........,99 To make output 01.txt A,b,c,.......,01 C,r,v,........,01 02.txt C,v,n,.......,02 (12 Replies)
Discussion started by: teefa
12 Replies

10. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies
PERIODIC(8)						    BSD System Manager's Manual 					       PERIODIC(8)

NAME
periodic -- run periodic system functions SYNOPSIS
periodic directory ... DESCRIPTION
The periodic utility is intended to be called by launchd(8) to execute shell scripts located in the specified directory. One or more of the following arguments must be specified: daily Perform the standard daily periodic executable run. This usually occurs early in the morning (local time). weekly Perform the standard weekly periodic executable run. This usually occurs very early on Saturday mornings. monthly Perform the standard monthly periodic executable run. This usually occurs on the first day of the month. path An arbitrary directory containing a set of executables to be run. If an argument is an absolute directory name it is used as is, otherwise it is searched for under /etc/periodic and any other directories specified by the local_periodic setting in periodic.conf(5) (see below). The periodic utility will run each executable file in the directory or directories specified. If a file does not have the executable bit set, it is silently ignored. Each script is required to exit with one of the following values: 0 The script has produced nothing notable in its output. The <basedir>_show_success variable controls the masking of this output. 1 The script has produced some notable information in its output. The <basedir>_show_info variable controls the masking of this output. 2 The script has produced some warnings due to invalid configuration settings. The <basedir>_show_badconfig variable controls the mask- ing of this output. >2 The script has produced output that must not be masked. If the relevant variable (where <basedir> is the base directory in which the script resides) is set to ``NO'' in periodic.conf, periodic will mask the script output. If the variable is not set to either ``YES'' or ``NO'', it will be given a default value as described in periodic.conf(5). All remaining script output is delivered based on the value of the <basedir>_output setting. If this is set to a path name (beginning with a '/' character), output is simply logged to that file. newsyslog(8) knows about the files /var/log/daily.log, /var/log/weekly.log and /var/log/monthly.log, and if they exist, it will rotate them at the appropriate times. These are therefore good values if you wish to log periodic output. If the <basedir>_output value does not begin with a '/' and is not empty, it is assumed to contain a list of email addresses, and the output is mailed to them. If <basedir>_show_empty_output is set to ``NO'', then no mail will be sent if the output was empty. If <basedir>_output is not set or is empty, output is sent to standard output. ENVIRONMENT
The periodic utility sets the PATH environment to include all standard system directories, but no additional directories, such as /usr/local/bin. If executables are added which depend upon other path components, each executable must be responsible for configuring its own appropriate environment. FILES
/System/Library/LaunchDaemons/com.apple.periodic-*.plist the periodic utility is typically called via these launchd(8) jobs /etc/periodic the top level directory containing daily, weekly, and monthly subdirectories which contain standard system peri- odic executables /etc/defaults/periodic.conf the periodic.conf system registry contains variables that control the behaviour of periodic and the standard daily, weekly, and monthly scripts /etc/periodic.conf this file contains local overrides for the default periodic configuration EXIT STATUS
Exit status is 0 on success and 1 if the command fails. EXAMPLES
The /etc/defaults/periodic.conf system registry will typically have a local_periodic variable reading: local_periodic="/usr/local/etc/periodic" To log periodic output instead of receiving it as email, add the following lines to /etc/periodic.conf: daily_output=/var/log/daily.log weekly_output=/var/log/weekly.log monthly_output=/var/log/monthly.log To only see important information from daily periodic jobs, add the following lines to /etc/periodic.conf: daily_show_success=NO daily_show_info=NO daily_show_badconfig=NO DIAGNOSTICS
The command may fail for one of the following reasons: usage: periodic <directory of files to execute> No directory path argument was passed to periodic to specify where the script fragments reside. <directory> not found Self explanatory. SEE ALSO
sh(1), periodic.conf(5), launchd(8), newsyslog(8) HISTORY
The periodic utility first appeared in FreeBSD 3.0. AUTHORS
Paul Traina <pst@FreeBSD.org> Brian Somers <brian@Awfulhak.org> BUGS
Since one specifies information about a directory using shell variables containing the string, <basedir>, <basedir> must only contain charac- ters that are valid within a sh(1) variable name, alphanumerics and underscores, and the first character may not be numeric. BSD
August 30, 2007 BSD
All times are GMT -4. The time now is 01:58 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy