Sponsored Content
Full Discussion: Sed to parse log file
Top Forums Shell Programming and Scripting Sed to parse log file Post 302633121 by chipperuga on Tuesday 1st of May 2012 11:19:11 AM
Old 05-01-2012
Sed to parse log file

Hi all, thanks for reading the post.

I'm trying to parse hundreds of log files in a directory. One log file looks similar to below:

Code:
Investigator  : Jim_Foo
Custodian     : Jim_Foo-HDD1-FOO-1234
Export Path   : N:\FOO-1234\Foo_Foo
Compute MD5   : No
File List Only: No
Extensions Selected:
     DOC
     DOCX
     EML
     MSG
     OST
     PDF
     PPT
     PPTX
     PST
     XLS
     XLSX
     XLSM
     ZIP
     REP

Matched File : foo_file1.pdf
Match Type   : File Extension Match
File Number  : 0
File Size    : 640
MD5 Hash     : 'Compute MD5 Hash' not selected
MAC Times    : M=09/27/10 02:05:26PM  A=02/01/12 09:59:49AM  C=09/27/10 02:05:26PM  
Original Path: foopath\foopath\foopath\foobar\foo_file.pdf

Matched File : foo_file2.pdf
Match Type   : File Extension Match
File Number  : 0
File Size    : 123
MD5 Hash     : 'Compute MD5 Hash' not selected
MAC Times    : M=09/27/10 02:05:26PM  A=02/01/12 09:59:49AM  C=09/27/10 02:05:26PM  
Original Path: foopath\foopath\foopath\foobar\foo_file.pdf

I would like to search for strings with sed and put the results into a csv. I can search for the right patterns, but I'm having trouble printing it correctly.

Code:
$ sed -e '/Custodian/b' -e '/Matched File/b' -e '/File Size/b' -e '/Original Path/b' -e d *

Custodian : Jim_Foo-HDD1-FOO-1234
Export Path : N:\FOO-1234\Foo_Foo
Matched File : foo_file1.pdf
File Size : 640
Original Path: foopath\foopath\foopath\foobar\foo_file1.pdf
Matched File : foo_file2.pdf
File Size : 123
Original Path: foopath\foopath\foopath\foobar\foo_file.pdf[/CODE]

CSV needs to look something like this:
Code:
Custodian,Export Path,Matched File,File Size,Original Path
Custodian,Export Path,Matched File,File Size,Original Path

Code:
"Jim_Foo-HDD1-FOO-1234","N:\FOO-1234\Foo_Foo","foo_file1.pdf","640","foopath\foopath\foopath\foobar\foo_file1.pdf"
"Jim_Foo-HDD1-FOO-1234","N:\FOO-1234\Foo_Foo","foo_file2.pdf","123","foopath\foopath\foopath\foobar\foo_file.pdf"


Last edited by chipperuga; 05-01-2012 at 12:52 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

To parse through the file and print output using awk or sed script

suppose if u have a file like that Hen ABCCSGSGSGJJJJK 15 Cock ABCCSGGGSGIJJJL 15 * * * * * * : * * * . * * * : Hen CFCDFCSDFCDERTF 30 Cock CHCDFCSDHCDEGFI 30 * . * * * * * * * : * * :* : : . The output shud be where there is : and . It shud... (4 Replies)
Discussion started by: cdfd123
4 Replies

2. Shell Programming and Scripting

Parse out known messages from a log file

I am looking for a script to do the following. I have a large log file that contains hundreds of warnings, a lot of which can be ignored. The tool doesn't allow me to suppress it, so I like to parse it out from the log file and isolate just the new messages/warnings, based on an exception file. ... (12 Replies)
Discussion started by: cdn2008
12 Replies

3. Shell Programming and Scripting

sed command to parse Apache config file

Hi there, am trying to parse an Apache 'server' config file. A snippet of the config file is shown below: ..... ProxyPassReverse /foo http://foo.example.com/bar ..... ..... RewriteRule ^/(.*) http://www.example.com/$1 RewriteRule /redirect https://www.example1.com/$1 ........ (7 Replies)
Discussion started by: jy2k7ca
7 Replies

4. Shell Programming and Scripting

sed parse small xml file

I have a tmp.xml file like: <?xml version="1.0" encoding="UTF-8"?> <Response> <Ip>193.143.121.198</Ip> <Status>OK</Status> <CountryCode>PL</CountryCode> <CountryName>Poland</CountryName> <RegionCode>82</RegionCode> <RegionName>Pomorskie</RegionName> <City>Gdansk</City> ... (9 Replies)
Discussion started by: unclecameron
9 Replies

5. Shell Programming and Scripting

parse a log file and remember last line

Hi all: I'm working on a HPUX 11.23 system and I am needing to parse a tomcat-jakarta log file for memory use. Getting the desired data is easy, assuming the log file does not grow. This file grows constantly and I want to check it q 5 min. The next check will pick up from where it left off 5... (4 Replies)
Discussion started by: raggmopp
4 Replies

6. Shell Programming and Scripting

Parse the log file

./abc.sh started at Sun Oct 24 06:42:04 PDT 2010 Message: ======= Summary Report of NAME count ----------------------------------------------------------------- Below is the output of the SQL query :- NAME COUNT... (2 Replies)
Discussion started by: sandy1028
2 Replies

7. Shell Programming and Scripting

parse log with sed

I've been searching for an hour on how to parse a file like this: 10.200.5.83 - - "GET /portal/edits.js HTTP/1.1" 200 24324 10.200.5.83 - - "GET /portal/objects/PortalConfig.js HTTP/1.1" 200 12187 10.200.5.84 - - "GET /portal/objects/CommonDialog.js HTTP/1.1" 200 8283 10.200.5.84 - - "GET... (4 Replies)
Discussion started by: dba_frog
4 Replies

8. Shell Programming and Scripting

sed parse a lign into a new sql file

Hi everyone, another question while using sed. my sed statement should parse every line in a file and store all "i" variable item a a new file. any wrong arguments here? Thanks a million. task_name => name, object_type => 'TABLE', attr1 => 'TestR3', attr2 => '$i', for i... (4 Replies)
Discussion started by: sundaygeek
4 Replies

9. Shell Programming and Scripting

Parse A Log File

Hello All, Below is the excerpt from my Informatica log file which has 4 blocks of lines (starting with WRITER_1_*_1). Like these my log file will have multiple blocks of same pattern. WRITER_1_*_1> WRT_8161 TARGET BASED COMMIT POINT Thu May 08 09:33:21 2014... (13 Replies)
Discussion started by: Ariean
13 Replies

10. Shell Programming and Scripting

Parse log file to insert into database

I have a log file that's created daily by this command: sar -u 300 288 >> /var/log/usage/$(date "+%Y-%m-%d")_$(hostname)_cpu.log It that contains data like this: Linux 3.16.0-4-amd64 (myhostname) 08/15/2015 _x86_64_ (1 CPU) 11:34:17 PM CPU %user %nice ... (12 Replies)
Discussion started by: unplugme71
12 Replies
Path::Class(3pm)					User Contributed Perl Documentation					  Path::Class(3pm)

NAME
Path::Class - Cross-platform path specification manipulation VERSION
version 0.26 SYNOPSIS
use Path::Class; my $dir = dir('foo', 'bar'); # Path::Class::Dir object my $file = file('bob', 'file.txt'); # Path::Class::File object # Stringifies to 'foo/bar' on Unix, 'fooar' on Windows, etc. print "dir: $dir "; # Stringifies to 'bob/file.txt' on Unix, 'bobfile.txt' on Windows print "file: $file "; my $subdir = $dir->subdir('baz'); # foo/bar/baz my $parent = $subdir->parent; # foo/bar my $parent2 = $parent->parent; # foo my $dir2 = $file->dir; # bob # Work with foreign paths use Path::Class qw(foreign_file foreign_dir); my $file = foreign_file('Mac', ':foo:file.txt'); print $file->dir; # :foo: print $file->as_foreign('Win32'); # foofile.txt # Interact with the underlying filesystem: # $dir_handle is an IO::Dir object my $dir_handle = $dir->open or die "Can't read $dir: $!"; # $file_handle is an IO::File object my $file_handle = $file->open($mode) or die "Can't read $file: $!"; DESCRIPTION
"Path::Class" is a module for manipulation of file and directory specifications (strings describing their locations, like '/home/ken/foo.txt' or 'C:WindowsFoo.txt') in a cross-platform manner. It supports pretty much every platform Perl runs on, including Unix, Windows, Mac, VMS, Epoc, Cygwin, OS/2, and NetWare. The well-known module "File::Spec" also provides this service, but it's sort of awkward to use well, so people sometimes avoid it, or use it in a way that won't actually work properly on platforms significantly different than the ones they've tested their code on. In fact, "Path::Class" uses "File::Spec" internally, wrapping all the unsightly details so you can concentrate on your application code. Whereas "File::Spec" provides functions for some common path manipulations, "Path::Class" provides an object-oriented model of the world of path specifications and their underlying semantics. "File::Spec" doesn't create any objects, and its classes represent the different ways in which paths must be manipulated on various platforms (not a very intuitive concept). "Path::Class" creates objects representing files and directories, and provides methods that relate them to each other. For instance, the following "File::Spec" code: my $absolute = File::Spec->file_name_is_absolute( File::Spec->catfile( @dirs, $file ) ); can be written using "Path::Class" as my $absolute = Path::Class::File->new( @dirs, $file )->is_absolute; or even as my $absolute = file( @dirs, $file )->is_absolute; Similar readability improvements should happen all over the place when using "Path::Class". Using "Path::Class" can help solve real problems in your code too - for instance, how many people actually take the "volume" (like "C:" on Windows) into account when writing "File::Spec"-using code? I thought not. But if you use "Path::Class", your file and directory objects will know what volumes they refer to and do the right thing. The guts of the "Path::Class" code live in the "Path::Class::File" and "Path::Class::Dir" modules, so please see those modules' documentation for more details about how to use them. EXPORT The following functions are exported by default. file A synonym for "Path::Class::File->new". dir A synonym for "Path::Class::Dir->new". If you would like to prevent their export, you may explicitly pass an empty list to perl's "use", i.e. "use Path::Class ()". The following are exported only on demand. foreign_file A synonym for "Path::Class::File->new_foreign". foreign_dir A synonym for "Path::Class::Dir->new_foreign". Notes on Cross-Platform Compatibility Although it is much easier to write cross-platform-friendly code with this module than with "File::Spec", there are still some issues to be aware of. o On some platforms, notably VMS and some older versions of DOS (I think), all filenames must have an extension. Thus if you create a file called foo/bar and then ask for a list of files in the directory foo, you may find a file called bar. instead of the bar you were expecting. Thus it might be a good idea to use an extension in the first place. AUTHOR
Ken Williams, KWILLIAMS@cpan.org COPYRIGHT
Copyright (c) Ken Williams. All rights reserved. This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO
Path::Class::Dir, Path::Class::File, File::Spec perl v5.14.2 2012-06-15 Path::Class(3pm)
All times are GMT -4. The time now is 10:55 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy