Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Parsing out access.log with awk and grep

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parsing out access.log with awk and grep
# 8  
Old 04-25-2011
Quote:
Originally Posted by Franklin52
Have you tried this?
Code:
awk '/\.com/  || /\.org/ || /\.net/ {print $8}' logfile

It still prints out everything instead of lines ending in .com/ .org/, etc.
# 9  
Old 04-25-2011
Quote:
Originally Posted by druisgod
ugh. I tried, but because I'm a new user I'm not allowed to post links. The log file get's awk'ed out like this:
Did you try the solutions posted before your reply?
# 10  
Old 04-25-2011
Quote:
Originally Posted by Corona688
Did you try the solutions posted before your reply?
I did all but the perl repsonse. (I don't write perl and wasn't sure how to implement it inline). Unfortunately nothing worked. I just solved it though. It seems that the trailing / is the causing part of the issue, so I dropped it with sed and your prior responses worked! SmilieSmilie

awk '{print $8}' 1 | sed s'/.$//' | egrep "\.(com|org|net)/?$"

Thanks to all who helped!!! This was kicking my butt.

I've been admin'ing/scripting/building with Linux since 2000 and I've been avoiding any advanced regex like the plague. Guess thats my next training. BTW, anyone know of any good online tutorials for regex?
# 11  
Old 04-25-2011
Quote:
Originally Posted by druisgod
I did all but the perl repsonse. (I don't write perl and wasn't sure how to implement it inline). Unfortunately nothing worked. I just solved it though. It seems that the trailing / is the causing part of the issue, so I dropped it with sed and your prior responses worked! SmilieSmilie
My solution purposefully ignores the trailing / if present, that's what the /? is for. Something else must have been on the end too, maybe a space or tab?
Quote:
awk '{print $8}' 1 | sed s'/.$//' | egrep "\.(com|org|net)/?$"
This will delete any trailing char, be it / or m or t.

But that might be okay if .com always ends with / and you care about no other lines.
# 12  
Old 04-25-2011
Quote:
Originally Posted by Corona688
My solution purposefully ignores the trailing / if present, that's what the /? is for. Something else must have been on the end too, maybe a space or tab?
This will delete any trailing char, be it / or m or t.

But that might be okay if .com always ends with / and you care about no other lines.

Carp. You're right! Sorry about that, I must have messed it up somehow when I ran it. I seriously didn't think any of them were working. Thanks for the reply!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed / awk / grep to extract information from log

Hi all, I have a query that runs that outputs data in the following format - 01/09/12 11:43:40,ADMIN,4,77,Application Group Load: Name(TESTED) LoadId(5137-1-0-1XX-15343-15343) File(/dir/dir/File.T03.CI2.RYR.2012009.11433350806.ARD) InputSize(5344) OutputSize(1359) Rows(2) Time(1.9960)... (8 Replies)
Discussion started by: jeffs42885
8 Replies

2. Shell Programming and Scripting

awk script to find time difference between HTTP PUT and HTTP DELETE requests in access.log

Hi, I'm trying to write a script to determine the time gap between HTTP PUT and HTTP DELETE requests in the HTTP Servers access log. Normally client will do HTTP PUT to push content e.g. file_1.txt and 21 seconds later it will do HTTP DELETE, but sometimes the time varies causing some issues... (3 Replies)
Discussion started by: Juha
3 Replies

3. Shell Programming and Scripting

awk to grep log

hi expert, i Have log like : dn: EPC-Per=673,HYHGU objectClass: EPC-PerSubs ownerId: 0 groupId: 4001 shareTree: nodeName=HYHGU permissions: 15 EPC-Per: 673 dn:... (2 Replies)
Discussion started by: justbow
2 Replies

4. Shell Programming and Scripting

Issue with awk script parsing log file

Hello All, I am trying to parse a log file and i got this code from one of the good forum colleagues, However i realised later there is a problem with this awk script, being naive to awk world wanted to see if you guys can help me out. AWK script: awk '$1 ~ "^WRITER_" {p=1;next}... (18 Replies)
Discussion started by: Ariean
18 Replies

5. Shell Programming and Scripting

Need help with awk and access log

when i run this command: tail -200 /var/log/httpd/access_log | awk -F'' '{sub(/:*$/,"",$2);print $2}' i get: 13/Aug/2012:20:56 13/Aug/2012:20:56 13/Aug/2012:20:56 13/Aug/2012:20:56 13/Aug/2012:20:56 13/Aug/2012:20:56 13/Aug/2012:20:56 13/Aug/2012:20:56 13/Aug/2012:20:56... (3 Replies)
Discussion started by: SkySmart
3 Replies

6. Shell Programming and Scripting

Access log field - using awk to pull date/time

hey guys. the following line is a line taken from apache's access_log 10.10.10.10 - jdoe "GET /images/down.gif HTTP/1.1" 304 I'm concerned about the field that has the date and time in it. if assuming the delimiter in the file is a space, then the fourth field will always have the date... (5 Replies)
Discussion started by: SkySmart
5 Replies

7. UNIX for Dummies Questions & Answers

awk/grep or parsing in python code

Hello, I am writing a python code. The output of the python code needs a little bit of parsing. From the output of python code, which has a lot of redundant data, I need to cut only those words or numbers which end with &. for example: if the output is-- "This is an example of tgbn123& what i... (0 Replies)
Discussion started by: Screech_you
0 Replies

8. Shell Programming and Scripting

Router ping log extract data from it Awk/Sed/grep

Hi, I am new to this world.. Using expect i loging to router and checking ping response to my links. I need to genarate report using this output and that report contains only three file link name, packet loss, latency. my output of script is like below: -bash-3.00$ monmw/mwbkp... (2 Replies)
Discussion started by: jkmistry
2 Replies

9. UNIX for Advanced & Expert Users

WEB Server Log File Analysis using awk/sed/grep

I'm trying to find a way to show large page sizes (page size in K) from multiple web server log files. Essentially I want to show only rows from a file where a specific column is larger than some value. Has anyone ever done this type of log analysis? If so, a snippet of code would be very... (2 Replies)
Discussion started by: mike_cataldo@ad
2 Replies

10. UNIX for Dummies Questions & Answers

Constantly updating log files (tail -f? grep? awk?)

I have a log file which is continuously added to, called log.file. I'd like to monitor this file, and when certain lines are found, update some totals in another file. I've played around with tail -f, grep, and awk, but can't seem to hit the right note, so to speak. The lines I'm... (0 Replies)
Discussion started by: nortonloaf
0 Replies
Login or Register to Ask a Question