make_combined_log.pl(1) General Commands Manual make_combined_log.pl(1)NAME
make_combined_log.pl - make combined logfile from SQL database
SYNOPSIS
make_combined_log.pl <days> <virtual host>
DESCRIPTION
This perl script extracts the httpd access data from a MySQL database and formats it properly for parsing by 3rd-party log analysis tools.
The script is intended to be run out by cron. Its commandline arguments tell it how many days' worth of access records to extract, and
which virtual_host you are interested in (because many people log several virthosts to one MySQL db.) This permits you to run it daily,
weekly, every 9 days -- whatever you decide.
NOTE
By "days" I mean "chunks of 24 hours prior to the moment this script is run." So if you run it at 4:34 p.m. on the 12th, it will go back
through 4:34 p.m. on the 11th.
KNOWN ISSUES
Because GET and POST are not discriminated in the MySQL log, we'll just assume that all requests are GETs. This should have negligible
effect on any analysis software. This could be remedied IF you stored the full HTTP request in your database instead of just the URI, but
that's going to cost you a LOT of space really quickly...
Because this is somewhat of a quick hack it doesn't do the most robust error checking in the world. Run it by hand to confirm your usage
before putting it in crontab.
AUTHOR
Edward Rudd <eddie@omegaware.com>
MAN PAGE CREATED BY
Michael A. Toth <lirul.lists@gmail.com> - based on comments of script
COMMENTS
This man page was written using xml2man (1) by the same author.
Manuals User make_combined_log.pl(1)
Check Out this Related Man Page
mysql_import_combined_log.pl(1) General Commands Manual mysql_import_combined_log.pl(1)NAME
mysql_import_combined_log.pl - Imports an Apache combined log into a MySQL database.
SYNOPSIS
mysql_import_combined_log.pl -d <database name> -t <table name> [-h <host name>] [-u <user name>] [-p <password>] [-f <filename>]
OPTIONS
--host|-h <host name>
The host to connect to. Default is localhost.
--database|-d <database name>
The database to use. Required.
--username|-u <username>
The user to connect as.
--password|-p <password>
The user's password.
--table|-t <table name>
The name of the table in which to insert data.
--logfile|-f <file name>
The file to read from. If not given, data is read from stdin.
--help|-?
Print out this help message.
--version
Print out the version of this software.
AUTHOR
Aaron Jenson
Updated to work under Perl 5.6.1 by Edward Rudd
MAN PAGE CREATED BY
Michael A. Toth <lirul.lists@gmail.com> - based on content of script
COMMENTS
This man page was written using xml2man (1) by the same author.
Manuals User mysql_import_combined_log.pl(1)
Okay, absolute newbie here...
I'm on a Mac trying to split an almost 2 Gig log file on a Unix box into manageable chunks for my web-based log analysis tool.
What do I need to do, what programs do I need to do it?
All and any help appreciated/needed :-)
Cheers (8 Replies)
Alright, I have being checking out various posting here trying to hack together something for a friend
Needed:
A script that can run in cron doing:
ping of several hosts and notifying via email when they are unavailable.
I am not going to post my non working messes (I am a basic... (11 Replies)
Hellooo
Salute to All.
I am a mini poor STUDENT from a Thirldworld country for learning abt UNIX,MINIX,LINUX etc.
I have not enough wisdom abt above topics.
But I want to walk,run & fly in UNIX,Linux platform.
But I studied in computer science degree...what i did'nt comple at all.
Though i... (8 Replies)
Morning,
I have a database log file that i need to scan thru to find all successful logins within the last 40 days,
What i have currently done is create a bash script to scan the last month and pipe out the results to a text file.
YEAR=`date '+%Y'`
MONTH=`date '+%m'`
grep 'Login... (12 Replies)
Hi All,
I got a strange problem here. I have a perl script which is fetching data from a database table and writing a file with that data.
If i run that script from linux command line, the file it creates is a normal ascii text file without any binary character in it.But... (9 Replies)
Hello,
I like to begin with :wall:.. literally... It has been 4 days and I have no idea how to fix it.
Environment - AIX 5.3
I wrote a script to call on ssh to log into another box via PKA to do something else.
If I run the script on the terminal, it works 100%. If the SAP customised... (11 Replies)
I'm still new to bash script , I have a log file and I want to extract the items within the last 5 days . and also within the last 10 hours
the log file is like this : it has 14000 items started from march 2002 to january 2003
awk '{print $4}' < *.log |uniq -c|sort -g|tail -10
but... (14 Replies)
How do i cleanup a log file data every 2 days, I was using below command in a script to remove log file every 2 days, but looks like its not working as log file date and time gets updated every 5 mins. Is there a way ?
find ./ \( -name . -o -prune \) -type f -mtime -2 -name... (9 Replies)
Hello all,
I recently wrote a simple script for the analysis jobs I do at work. I have to run multiple files through 5 different stages of an analysis program, the script simply runs all of the files through each stage automatically. My question is this: The computer I'm using has 12 cores, each... (8 Replies)
I have a file that should cover a days worth of stats, at the beginning of each 15 minute report I have a unique header that looks like the below example. The "0000" and "0015" will change in the header line to show which 15 Minute interval the report is covering and of course from day to day the... (7 Replies)
Hi All,
I have a script, I want to make sure the script should check whether the day is sunday, only then it should run, if it is run other days it should check and exit the script.
Kindly help.
Thanks in Advance !! (41 Replies)
Hello All,
I have a date with timestamp stored in a variable and another variable which stores no. of days.
Var1=2014-12-11 12:54:21
Var2=5
In my unix script, I have to add the no. of days stored in var2 with the var1 date.
Var3=var1+var2=2014-12-16 12:54:21
Can you please advise if... (8 Replies)
The script below works just fine.
Now i want to instead of just printing the last 7 days i want to print all the days last month. So if the script runs now, it will show all the 30 days in november.
I dont want any days to be shown in this month....
Im stuck, dont know how i will gett this to... (7 Replies)