Sponsored Content
Full Discussion: parse log with sed
Top Forums Shell Programming and Scripting parse log with sed Post 302493286 by dba_frog on Wednesday 2nd of February 2011 10:03:41 AM
Old 02-02-2011
parse log with sed

I've been searching for an hour on how to parse a file like this:
Code:
10.200.5.83 - - [02/Feb/2011:08:50:31 -0500] "GET /portal/edits.js HTTP/1.1" 200 24324
10.200.5.83 - - [02/Feb/2011:08:50:31 -0500] "GET /portal/objects/PortalConfig.js HTTP/1.1" 200 12187
10.200.5.84 - - [02/Feb/2011:08:50:31 -0500] "GET /portal/objects/CommonDialog.js HTTP/1.1" 200 8283
10.200.5.84 - - [02/Feb/2011:08:50:31 -0500] "GET /portal/objects/ErrorObject.js HTTP/1.1" 200 8248
10.200.5.83 - - [02/Feb/2011:08:50:31 -0500] "GET /portal/objects/FilterObject.js HTTP/1.1" 200 15520
10.200.5.83 - - [02/Feb/2011:08:50:31 -0500] "GET /portal/objects/DME.js HTTP/1.1" 200 13425
10.200.5.83 - - [02/Feb/2011:08:50:31 -0500] "GET /portal/objects/HotkeysManager.js HTTP/1.1" 200 16475
10.200.5.83 - - [02/Feb/2011:08:50:31 -0500] "GET /portal/objects/ValueStorage.js HTTP/1.1" 200 1922

When complete I need:
Code:
10.200.5.83  02/Feb/2011
10.200.5.84  02/Feb/2011

So in the end I really need a distinct list of each IP rolled up by Day so that we can get an idea of traffic.

I've been getting hung up on this:
Code:
sed '/\:/s/[0-9]*//' $file > tmpfile ; mv tmpfile test2.log

it keeps stripping out the first octet of the ip, not the time portion.
This User Gave Thanks to dba_frog For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parse Line Using Sed

Hello All, I am new to using sed, and I need to extract from the string data after : delimeter. Can you help me please with the sed command? Here's the input: ipAddress: 10.20.10.11 ioIpAddressNodeB: 10.20.10.10 ioIpAddressNodeA: 10.20.10.9 ipAddress: 0.0.0.0 Expected Output:... (7 Replies)
Discussion started by: racbern
7 Replies

2. Shell Programming and Scripting

Parse String Using Sed

Hi, I am wondering if there's a simpler way to extract the second occurrence of a word enclosed in that matches my search criteria. Sample Input is as follows: Error installing feature - com.er.nms.cif.ist.NoMatchingUpgra Error installing feature -... (4 Replies)
Discussion started by: racbern
4 Replies

3. Shell Programming and Scripting

How to parse slash / in sed command..

Hi All, I am trying to replace paths with \ (as in NT) with / (as in Unix) in a script using sed. Following is the command I use: sed "s/e:\Kenny_test\csv_files/var/opt/ciw/data/outbound/g" EMEA4710.SQL > test the string to replace is e:\Kenny_test\csv_files the target string needs to... (3 Replies)
Discussion started by: Abhidey
3 Replies

4. Shell Programming and Scripting

sed parse small xml file

I have a tmp.xml file like: <?xml version="1.0" encoding="UTF-8"?> <Response> <Ip>193.143.121.198</Ip> <Status>OK</Status> <CountryCode>PL</CountryCode> <CountryName>Poland</CountryName> <RegionCode>82</RegionCode> <RegionName>Pomorskie</RegionName> <City>Gdansk</City> ... (9 Replies)
Discussion started by: unclecameron
9 Replies

5. Shell Programming and Scripting

sed or awk to parse this text

I am just beginning with sed and awk and understand that they are "per" line input. That is, they operate on each line individually, and output based on rules, etc. But I have multi-line text blocks that looks as follows, and wish to ONLY extract the text between the first hyphen (-) and the... (13 Replies)
Discussion started by: bulgin
13 Replies

6. Shell Programming and Scripting

sed to parse html

Hello, I have a html file like this : <html> ... ... ... <table> ....... ...... </table> <table name = "hi"> ...... ..... ... </table> <h1> Welcome </h1> ....... ...... </html> (11 Replies)
Discussion started by: prasanna1157
11 Replies

7. Shell Programming and Scripting

parse xm entry with awk/sed

Hi folks, I have XML files with the following sections (section occurs once per file) in them: <AuthorList CompleteYN="Y"> <Author ValidYN="Y"> <LastName>Bernal</LastName> <ForeName>Federico</ForeName> ... (3 Replies)
Discussion started by: euval
3 Replies

8. Shell Programming and Scripting

sed parse a lign into a new sql file

Hi everyone, another question while using sed. my sed statement should parse every line in a file and store all "i" variable item a a new file. any wrong arguments here? Thanks a million. task_name => name, object_type => 'TABLE', attr1 => 'TestR3', attr2 => '$i', for i... (4 Replies)
Discussion started by: sundaygeek
4 Replies

9. Shell Programming and Scripting

Sed to parse log file

Hi all, thanks for reading the post. I'm trying to parse hundreds of log files in a directory. One log file looks similar to below: Investigator : Jim_Foo Custodian : Jim_Foo-HDD1-FOO-1234 Export Path : N:\FOO-1234\Foo_Foo Compute MD5 : No File List Only: No Extensions Selected:... (4 Replies)
Discussion started by: chipperuga
4 Replies

10. Shell Programming and Scripting

sed script to parse logs issue

I have this script to parse some logs: #!/bin/bash id=$1 shift sed "/(id=$id)/,/^$/!d" "$@" Usage: ./script.sh 1234 logfile The logs have an empty line before the logged events/timestamps -- most of the time. And this is my issue, since when there is no empty line, it will catch things... (4 Replies)
Discussion started by: KidCactus
4 Replies
Mojo::Message::Response(3pm)				User Contributed Perl Documentation			      Mojo::Message::Response(3pm)

NAME
Mojo::Message::Response - HTTP 1.1 response container SYNOPSIS
use Mojo::Message::Response; # Parse my $res = Mojo::Message::Reponse->new; $res->parse("HTTP/1.0 200 OKx0ax0d"); $res->parse("Content-Length: 12x0ax0dx0ax0d"); $res->parse("Content-Type: text/plainx0ax0dx0ax0d"); $res->parse('Hello World!'); say $res->body; # Build my $res = Mojo::Message::Response->new; $res->code(200); $res->headers->content_type('text/plain'); $res->body('Hello World!'); say $res->to_string; DESCRIPTION
Mojo::Message::Response is a container for HTTP 1.1 responses as described in RFC 2616. EVENTS
Mojo::Message::Response inherits all events from Mojo::Message. ATTRIBUTES
Mojo::Message::Response inherits all attributes from Mojo::Message and implements the following new ones. "code" my $code = $res->code; $res = $res->code(200); HTTP response code. "message" my $message = $res->message; $res = $res->message('OK'); HTTP response message. METHODS
Mojo::Message::Response inherits all methods from Mojo::Message and implements the following new ones. "cookies" my $cookies = $res->cookies; $res = $res->cookies(Mojo::Cookie::Response->new); $res = $res->cookies({name => 'foo', value => 'bar'}); Access response cookies, usually Mojo::Cookie::Response objects. say $res->cookies->[1]->value; "default_message" my $message = $res->default_message; Generate default response message for code. "fix_headers" $res = $res->fix_headers; Make sure response has all required headers for the current HTTP version. "is_status_class" my $success = $res->is_status_class(200); Check response status class. SEE ALSO
Mojolicious, Mojolicious::Guides, <http://mojolicio.us>. perl v5.14.2 2012-09-05 Mojo::Message::Response(3pm)
All times are GMT -4. The time now is 08:58 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy