Sponsored Content
Full Discussion: Parsing a large log
Top Forums Shell Programming and Scripting Parsing a large log Post 302199981 by ripat on Wednesday 28th of May 2008 07:39:39 AM
Old 05-28-2008
I personally doubt that grep would be more efficient that awk for large files. Please post sample input and ouput files.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Splitting a large log file

Okay, absolute newbie here... I'm on a Mac trying to split an almost 2 Gig log file on a Unix box into manageable chunks for my web-based log analysis tool. What do I need to do, what programs do I need to do it? All and any help appreciated/needed :-) Cheers (8 Replies)
Discussion started by: simmonet
8 Replies

2. Shell Programming and Scripting

Problem with parsing a large file

Hi All, Following is the sample file and following is the op desired that is the last entry of each unique first field is required. My solution is as follows However the original file has around a million entries and around a 100,000 uniques first fields, so this soln.... (6 Replies)
Discussion started by: gauravgoel
6 Replies

3. Shell Programming and Scripting

Cutting a large log file in to smaller ones

I have a very large (150 megs) IRC log file from 2000-2001 which I want to cut down to individual daily log files. I have a very basic knowledge of the cat, sed and grep commands. The log file is time stamped and each day in the large log file begins with a "Session Start" string like so: ... (11 Replies)
Discussion started by: MrTangent
11 Replies

4. Shell Programming and Scripting

parsing large CDR XML file

Dear Freind in the file attached how parse the data to be like a normal table :D (3 Replies)
Discussion started by: saifsafaa
3 Replies

5. Red Hat

Help for capturing a URL from a line from large log file

Can someone please help me how do I find a URL from lines of log file and write all the output to a new file? For e.g - Log file has similar entries, 39.155.67.5 - - "GET /abc/login?service=http://161.120.36.39/CORPHR/TMA2007/default.asp HTTP/1.1" 401 3218 54.155.63.9 - - "GET... (2 Replies)
Discussion started by: rockf1bull
2 Replies

6. Shell Programming and Scripting

Help needed for parsing large XML with awk.

My XML structure looks like: <?xml version="1.0" encoding="UTF-8"?> <SearchRepository> <SearchItems> <SearchItem> ... </SearchItem> <SearchItem> ... ... (1 Reply)
Discussion started by: jasonjustice
1 Replies

7. UNIX for Dummies Questions & Answers

I need to isolate a date in a large log file

I wrote head -n1 example.log I grab the first line of the log, but I need to isolate just the date, which is 08/May/2012:09:52:52. I also need to find the reverse of this, which would be tail... http://i.imgur.com/Lp1eBD0.png Thanks in advance (4 Replies)
Discussion started by: spookydll
4 Replies

8. Shell Programming and Scripting

Log parsing

I have a directory with daily logs that have records like this: Date: 04/17/13 Time: 09:29:15 IP: 123.123.123.123 URL: usr/local/file1 and I want to only count how many times each file was accessed (e.g. file1 in that example above), and I want to also look in all the logs in the current... (3 Replies)
Discussion started by: Jaymz
3 Replies

9. Shell Programming and Scripting

Parsing large files in Solaris 11

I have a 1.2G file that contains no newline characters. This is essentially a log file with each entry being exactly 78bits long. The basic format is /DATE/USER/MISC/. The single uniform thing about the file is that that the 8 character is always ":" I worked with smaller files of the same... (8 Replies)
Discussion started by: os2mac
8 Replies

10. Shell Programming and Scripting

Parsing a subset of data from a large matrix

I do have a large matrix of the following format and it is tab delimited ch-ab1-20 ch-bb2-23 ch-ab1-34 ch-ab1-24 er-cc1-45 bv-cc1-78 ch-ab1-20 0 2 3 4 5 6 ch-bb2-23 3 0 5 ... (6 Replies)
Discussion started by: Kanja
6 Replies
ZGREP(1)						      General Commands Manual							  ZGREP(1)

NAME
zgrep - search possibly compressed files for a regular expression SYNOPSIS
zgrep [ grep_options ] [ -e ] pattern filename... DESCRIPTION
Zgrep is used to invoke the grep on compress'ed or gzip'ed files. All options specified are passed directly to grep. If no file is speci- fied, then the standard input is decompressed if necessary and fed to grep. Otherwise the given files are uncompressed if necessary and fed to grep. If zgrep is invoked as zegrep or zfgrep then egrep or fgrep is used instead of grep. If the GREP environment variable is set, zgrep uses it as the grep program to be invoked. For example: for sh: GREP=fgrep zgrep string files for csh: (setenv GREP fgrep; zgrep string files) AUTHOR
Charles Levert (charles@comm.polymtl.ca) SEE ALSO
grep(1), egrep(1), fgrep(1), zdiff(1), zmore(1), znew(1), zforce(1), gzip(1), gzexe(1) ZGREP(1)
All times are GMT -4. The time now is 03:08 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy