01-24-2009
SH script to split squid log by date
Hi,
I really need your help to make a script to split a large squid's log file into a multiple files, each of them containing the log entries for every logged date.
To achieve the result I planned to use the function "date" with the first log field as a parameter using this syntax:
date -d @(ten number squid timestamp field) +%F
to obtain the desired filename for splitted logs.
Example:
# date -d @1232163496 +%F
will return: 2009-01-16
The idea is to analize the first field of each line in the log file and append whole line to the corresponding file.
Any help would be apreciated,
I'm a newbie ¿Someone can help me?
Last edited by _MCRH_; 01-28-2009 at 05:02 PM..
Reason: Before close it
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I am trying to get the content of Apache access.log file for the current date for viewing purposes.
I can get it with the following sed command
sed -n '/09\/Oct\/2008/,/09\/Oct\/2008/p' access.log | less
now I want to enhance it such that it will automatically take current date instead of... (4 Replies)
Discussion started by: ysprathap
4 Replies
2. Shell Programming and Scripting
I am a newbie to scripting.
I need a korn shell script to copy log files of current day to archive folder and rename with current days date stamp.
I would really appreciate your help.
File structure is as follows. Everyday files get overwritten, so I need copy to a archive directory and... (3 Replies)
Discussion started by: mdncan
3 Replies
3. Shell Programming and Scripting
Hi frnds,
I have flat file as
.
Say :
output-file1.txt
Output-file2.txt (1 Reply)
Discussion started by: Gopal_Engg
1 Replies
4. Shell Programming and Scripting
hi All,
I have this sample text file - access.log:
Jan 18 21:34:29 root 209.151.232.70
Jan 18 21:34:40 root 209.151.232.70
Jan 18 21:34:43 root 209.151.232.70
Jan 18 21:34:56 root 209.151.232.70
Jan 18 21:35:10 root 209.151.232.70
Jan 18 21:35:23 root 209.151.232.70
Jan 18 21:36:04 root... (2 Replies)
Discussion started by: linuxgeek
2 Replies
5. Shell Programming and Scripting
Hi all,
I have logs(in a log file) with the following structure
20100916011501559;0.812;null;TRUE;;FALSE;0.812;0;0;;19
20100916011504762;0.015;null;TRUE;;FALSE;0;4|4;0.015;;4
20100916011504762;0;null;TRUE;;FALSE;0;0;0;;4
20100916011501731;3.343;null;TRUE;;FALSE;3.156;131|65;0.172;;11... (14 Replies)
Discussion started by: babai
14 Replies
6. IP Networking
Hello,
I have a pretty useless satellite link at home (far from any civilization), so I wanted to set up caching in order to speed things up. My Squid 2.6 runs "3128 transparent" and is set up quite well on a separate machine.
I also have my dd-wrt router to move all port 80 traffic through... (0 Replies)
Discussion started by: theWojtek
0 Replies
7. UNIX for Advanced & Expert Users
I searched and did not find a useful answer. Can someone please tell me the best practice to change the log directory from /var/log to /opt/squid? (4 Replies)
Discussion started by: glev2005
4 Replies
8. Shell Programming and Scripting
To delete log files content older than 30 days and append the lastest date log file date in the respective logs
I want to write a shell script that deletes all log files content older than 30 days and append the lastest log file date in the respective logs
This is my script
cd... (2 Replies)
Discussion started by: sreekumarhari
2 Replies
9. Shell Programming and Scripting
i need squid proxy log file daily auto generate by shell script (1 Reply)
Discussion started by: sanjeetkumar086
1 Replies
10. UNIX for Dummies Questions & Answers
I need to check 1 log file, which is logging:
2014-08-18T09:10:39+02:00 user: XXXXX START FEATURE
2014-08-18T09:10:39+02:00 user: XXXXX FINISH FEATURE
I first need to check that the START FEATURE starts and finish on the same time/date for the same user, which is different each time START... (2 Replies)
Discussion started by: TheBest43
2 Replies
LEARN ABOUT CENTOS
negotiate_kerberos_auth
negotiate_kerberos_auth(8) System Manager's Manual negotiate_kerberos_auth(8)
NAME
negotiate_kerberos_auth - Squid kerberos based authentication helper
Version 3.0.4sq
SYNOPSIS
negotiate_kerberos_auth [-h] [-d] [-i] [-r] [-s Service-Principal-Name]
DESCRIPTION
negotiate_kerberos_auth is an installed binary and allows Squid to authenticate users via the Negotiate protocol and Kerberos.
OPTIONS
-h Display the binary help and command line syntax info using stderr.
-d Write debug messages to stderr.
-i Write informational messages to stderr.
-r Remove realm from username before returning the username to squid.
-s Service-Principal-name
Provide Service Principal Name.
CONFIGURATION
This helper is intended to be used as an authentication helper in squid.conf.
auth_param negotiate program /path/to/negotiate_kerberos_auth
auth_param negotiate children 10
auth_param negotiate keep_alive on
NOTE: The following squid startup file modification may be required:
Add the following lines to the squid startup script to point squid to a keytab file which contains the HTTP/fqdn service principal for the
default Kerberos domain. The fqdn must be the proxy name set in IE or firefox. You can not use an IP address.
KRB5_KTNAME=/etc/squid/HTTP.keytab export KRB5_KTNAME
If you use a different Kerberos domain than the machine itself is in you can point squid to the seperate Kerberos config file by setting
the following environmnet variable in the startup script.
KRB5_CONFIG=/etc/krb5-squid.conf export KRB5_CONFIG
Kerberos can keep a replay cache to detect the reuse of Kerberos tickets (usually only possible in a 5 minute window) . If squid is under
high load with Negotiate(Kerberos) proxy authentication requests the replay cache checks can create high CPU load. If the environment does
not require high security the replay cache check can be disabled for MIT based Kerberos implementations by adding the following to the
startup script
KRB5RCACHETYPE=none export KRB5RCACHETYPE
If negotiate_kerberos_auth doesn't determine for some reason the right service principal you can provide it with -s HTTP/fqdn.
If you serve multiple Kerberos realms add a HTTP/fqdn@REALM service principal per realm to the HTTP.keytab file and use the -s
GSS_C_NO_NAME option with negotiate_kerberos_auth.
AUTHOR
This program was written by Markus Moeller <markus_moeller@compuserve.com>
This manual was written by Markus Moeller <markus_moeller@compuserve.com>
COPYRIGHT
This program and documentation is copyright to the authors named above.
Distributed under the GNU General Public License (GNU GPL) version 2 or later (GPLv2+).
QUESTIONS
Questions on the usage of this program can be sent to the Squid Users mailing list <squid-users@squid-cache.org>
REPORTING BUGS
Bug reports need to be made in English. See http://wiki.squid-cache.org/SquidFaq/BugReporting for details of what you need to include with
your bug report.
Report bugs or bug fixes using http://bugs.squid-cache.org/
Report serious security bugs to Squid Bugs <squid-bugs@squid-cache.org>
Report ideas for new improvements to the Squid Developers mailing list <squid-dev@squid-cache.org>
SEE ALSO
squid(8) ext_kerberos_ldap_group_acl(8)
RFC4559 - SPNEGO-based Kerberos and NTLM HTTP Authentication in Microsoft Windows,
RFC2478 - The Simple and Protected GSS-API Negotiation Mechanism,
RFC1964 - The Kerberos Version 5 GSS-API Mechanism,
The Squid FAQ wiki http://wiki.squid-cache.org/SquidFaq
The Squid Configuration Manual http://www.squid-cache.org/Doc/config/ http://wiki.squid-cache.org/ConfigExamples/Authenticate/Kerberos
negotiate_kerberos_auth(8)