Logrotate and Compressing only yesterdays files


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Logrotate and Compressing only yesterdays files
# 1  
Old 08-15-2019
Logrotate and Compressing only yesterdays files

Hello,

I have a syslog server at home and am currently experiencing an issue where my logs will rotate and compress however it will rotate and compress yesterdays file and the newly created log file for the current day. When it does this however it will also create another new file for today and as a result my compressed files look like this:

Code:
4.8M Aug 12 23:59 router.2019.08.12.log.1.gz--> this will be the encrypted log file from the 11th of August
5.3K Aug 12 00:02 router.2019.08.12.log.2.gz--> this will be the encrypted log file from the 12th of August Morning
37M Aug 12 00:02 router.2019.08.12.log --> this will be the new current day log file

Here is a copy of my syslog-ng.conf:

Code:
/var/log/router/router.*.log
{
        rotate 2
        daily
        missingok
        notifempty
        postrotate
                invoke-rc.d syslog-ng reload > /dev/null
        endscript
}

Here is the section in my syslog-ng.conf file that is relevant to the logs above

Code:
# Router ###########
# Add Filter to add our router
filter router { host( "192.168.XX.1" ); };
# Add destination file where logs will be stored
#destination df_router { file("/home/pi/files/router.log"); };
log { source ( s_net ); filter( f_router ); destination ( df_router ); };
destination df_router {
file("/var/log/router/router.${YEAR}.${MONTH}.${DAY}.log"
template-escape(no));
};

Here is my cron job that is running:

Code:
02 00 * * * sudo logrotate --force /etc/logrotate.d/syslog-ng

My question is this, what do I do in order to make sure that only yesterdays file gets compressed at 2 minutes beyond midnight instead of both current and yesterday files. If there is any other information required don't hesistate to ask.

Last edited by MyUserName7000; 08-15-2019 at 01:32 PM..
# 2  
Old 08-15-2019
Sorry, in your post, you mean "compressed file" and not "encrypted file" for your .gz files right?
# 3  
Old 08-15-2019
Yes sorry I just updated it. For some reason I am always defaulting to encrypted.
# 4  
Old 08-15-2019
Seems to me from youf post that the current day logfile is not compressed.

This is standard for logrotate, to compress the logs from the prior prior when logs are rotated.
# 5  
Old 08-15-2019
Hi Neo,

What will happen is logrotate will create a new file for today when we hit midnight as it should.
However when the compression runs it compresses both the newly created .log file for the new day and the .log file from yesterday.
It will still however end up creating a second log file for the new day that starts where the initial one left off so in this case 2 minutes after midnight.
In the morning I end up with two compressed .log files and a none compressed .log file.
Here is a screenshot from windows perpsective. The compressed file at the top would be files from the 11th and the one at the bottom would be the first 2 minutes after midnight for the 12th.
What isn't shown is the third actual log file that would be there (I can't show current because I tried something that didn't work out).
Also apologies for the massive thumbnail.
Logrotate and Compressing only yesterdays files-routerpng
# 6  
Old 08-15-2019
Most distros have a default crontab entry for logrotate that runs all rule files in /etc/logrotate.d/
(Once a day. Check all the cron files /etc/crontab /etc/cron.d/* /etc/cron.daily/* for the default logrotate!)
With your dedicated crontab entry the rule file /etc/logrotate.d/syslog-ng will run twice.
# 7  
Old 08-15-2019
Hello,

There is a default logrotate in /etc/daily which looks like this

Code:
#!/bin/sh

test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate /etc/logrotate.conf

I looked in the logrotate.conf file and have this and just now uncommented compress and create.

Code:
# see "man logrotate" for details
# rotate log files weekly
daily

# keep 4 weeks worth of backlogs
rotate 2

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

I removed my entries in crontab that were forcing the rotation to run using the syslog-ng.conf file.
I shall see what happens tomorrow/tonight and will let you know. Thanks to you both for the help in the mean time!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Logrotate - I am not able to rotate files using logrotate

I have written script which is working in Home directory perfectly and also compressing log files and rotating correctly. But, when i try to run script for /var/log/ i am able to get compressed log files but not able to get rotation of compressed log files. Please suggest. I am using below command... (5 Replies)
Discussion started by: VSom007
5 Replies

2. Shell Programming and Scripting

Compressing files

I need help to do a script that will compress a file that's bigger than 5000 octets and won't overwrite the previous compress file. lets say I have mylogfile.log and I would compress it I would become mylogfile. 1. log and if I would compress again mylogfile.log it would be mylogfile. 2.... (8 Replies)
Discussion started by: Froob
8 Replies

3. Shell Programming and Scripting

How to list todays and yesterdays .rej files from a directory?

I am trying to display todays and yesterdays .rej files from a directory. ls -lrt *.rej | grep 'Aug 12' ; ls -lrt *.rej | grep 'Aug 13' Which is working as above. But i want take 'Aug 12' and 'Aug 13' from a variable and the command should work everyday. I am able to get todays files by... (9 Replies)
Discussion started by: GopalKrishnaP
9 Replies

4. Shell Programming and Scripting

What is the code for compressing files using pkzip ?

hi everyone , Can someone provide me a shell program to compress and decompress files using gzip , i dont know anything in shell programming , this code is a part of my project. So can someone help with me the code ? (2 Replies)
Discussion started by: mig23
2 Replies

5. Shell Programming and Scripting

Filtering the yesterdays date from log files via script.

hi All, I have this sample text file - access.log: Jan 18 21:34:29 root 209.151.232.70 Jan 18 21:34:40 root 209.151.232.70 Jan 18 21:34:43 root 209.151.232.70 Jan 18 21:34:56 root 209.151.232.70 Jan 18 21:35:10 root 209.151.232.70 Jan 18 21:35:23 root 209.151.232.70 Jan 18 21:36:04 root... (2 Replies)
Discussion started by: linuxgeek
2 Replies

6. UNIX for Dummies Questions & Answers

Compressing of log files

Hello All My first post in the forum. :) I've this huge log files of size 20GB-30 GB in my unix server. I want to analyse the log file for some error messages. But because of the enormity of the size of these files i'm not able to grep/search the pattern in the file . Also, tried to gzip the... (1 Reply)
Discussion started by: sgbhat
1 Replies

7. Shell Programming and Scripting

using find to get all of yesterdays files

i tried to use "find" to get all of yesterdays files but missed something in the 24 hours logic. can anybody help me with this one? i thought that -daystart -atime 1 was enough but i got more files (2 Replies)
Discussion started by: progressdll
2 Replies

8. UNIX for Dummies Questions & Answers

compressing two files in a script

Hi, i have written a script in unix which produces two files(.csv file) at the end. Now i want to add these to files in a zip file and send the zip file across the network by FTP. Problem is that i dunno how to make a single zip file containing the two files that have been created by the script.... (1 Reply)
Discussion started by: nimish
1 Replies

9. UNIX for Dummies Questions & Answers

yesterdays files

I am new to UNIX and I am trying to write a shell script. I want to be able to list all files that were created with yesterdays dates (APR 29 as an example) that are not 0 file size.Then in those files I want to look for the string 'Process Complete' and list all files that DONT have that string.... (8 Replies)
Discussion started by: tonydsam
8 Replies

10. UNIX for Dummies Questions & Answers

Compressing files

I have never used this command before. I need to use the "compress" command to compress all files located in the subdirectories under the following directories: /home/ftp/inbound/Fordin Please advise, I appreciate your help. Thanks, Syed (3 Replies)
Discussion started by: sh9492
3 Replies
Login or Register to Ask a Question