Sponsored Content
Top Forums Shell Programming and Scripting grep'ing and sed'ing chunks in bash... need help on speeding up a log parser. Post 302309278 by devtakh on Tuesday 21st of April 2009 01:36:17 PM
Old 04-21-2009
try this -
$ sort -k 4 logfile | awk 'NR==1{prev=$4;txt="session."$4"."$6"."$7;printf("%s\n%s",txt,$0);getliine}{if (prev !~ $4){txt="session."$4"."$6"."$7;printf("%s\n%s",txt,$0);prev=$4}else {print $0;prev=$4}}'

will give something like this -

session.11F.Processing.
20090409 000127 - 11F - Processing20090409 000127 - 11F - Processing
20090409 000127 - 11F - Query 543 starting session
session.BD0.More.Processing
20090409 000122 - BD0 - More Processing20090409 000125 - BD0 - More Processing
20090409 000126 - BD0 - Order 123 shutting down
20090409 000122 - BD0 - Order 123 starting session
20090409 000122 - BD0 - Processing 1
20090409 000124 - BD0 - Processing 2
session.DD1.Cancel.345
20090409 000125 - DD1 - Cancel 345 starting session20090409 000125 - DD1 - Processing
20090409 000126 - DD1 - Processing 2
session.EF0.Order.234
20090409 000123 - EF0 - Order 234 starting session20090409 000124 - EF0 - Processing
20090409 000125 - EF0 - Processing


cheers,
Devaraj Takhellambam
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

grep'ing for text within a bunch of files...?

I have, say, a dozen files, and I want to grep for a string of text within them. I don't remember the exact syntax, but let me give it a shot and show you an idea here... find . -type f -exec grep thisword {} \; ...and there's a way to put more than one grep into the statement, so it will tell... (1 Reply)
Discussion started by: kitykity
1 Replies

2. Shell Programming and Scripting

grep'ing for specific directories, and using the output to move files

Hello, this is probably another really simple tasks for most of you gurus, however I am trying to make a script which takes an input, greps a specific file for that input, prints back to screen the results (which are directory names) and then be able to use the directory names to move files.... (1 Reply)
Discussion started by: JayC89
1 Replies

3. Shell Programming and Scripting

grep'ing a file until a certain message appears

Hello, I'm writing a script that will automate the launch of some services on my AIX machine. However, some services are dependent on the successful startup of others. When I start these services manually, I usually just check a log file until I see a message that confirms a successful... (3 Replies)
Discussion started by: pallak7
3 Replies

4. Shell Programming and Scripting

pipe'ing grep output to awk

This script is supposed to find out if tomcat is running or not. #!/bin/sh if netstat -a | grep `grep ${1}: /tomcat/bases | awk -F: '{print $3}'` > /dev/null then echo Tomcat for $1 running else echo Tomcat for $1 NOT running fi the /tomcat/bases is a file that... (2 Replies)
Discussion started by: ziggy25
2 Replies

5. Programming

Obfuscate'ing a.out ... ???

Hi all, I've search the forums regarding posts similar to this already but can't find the suitable response. Am actually looking for something very trivial I think. I just want to mask/obfuscate the a.out file and run it like a normal UNIX program. I've look at gpg and encryption but it requires... (4 Replies)
Discussion started by: newbie_01
4 Replies

6. Shell Programming and Scripting

grep'ing a variable that contains a metacharacter ($) with a while loop

This is driving me crazy, and I'm hoping someone can help me out with this. I'm trying to do a simple while loop to go through a log file. I'm pulling out all of the lines with a specific log line, getting an ID from that line, and once I have a list of IDs I want to loop back through the log and... (2 Replies)
Discussion started by: DeCoTwc
2 Replies

7. Shell Programming and Scripting

Severe performance issue while 'grep'ing on large volume of data

Background ------------- The Unix flavor can be any amongst Solaris, AIX, HP-UX and Linux. I have below 2 flat files. File-1 ------ Contains 50,000 rows with 2 fields in each row, separated by pipe. Row structure is like Object_Id|Object_Name, as following: 111|XXX 222|YYY 333|ZZZ ... (6 Replies)
Discussion started by: Souvik
6 Replies

8. Shell Programming and Scripting

grep'ing dot history file

Hi, I tried to grep ".sh_history" (DOTsh_history) file and did not return anything though I found the word in .sh _history file through vi editor in Linux. Then I tried to grep ".profile" to check if it is the prob with hidden files and I got results. Then I verified the same with my friend... (4 Replies)
Discussion started by: bobbygsk
4 Replies

9. Shell Programming and Scripting

Grep'ing information from a log file on SUN OS 5

Hi Guys, I'm trying to write an script that will be launched by a user. The script will look at a log file and check for alerts with the date (supplied by user) and a machine's hostname (also supplied by the user). I'm trying to get the output formatted just like the log file. The logfile looks... (5 Replies)
Discussion started by: illgetit
5 Replies

10. UNIX for Dummies Questions & Answers

grep'ing a variable - why not working

Hi all, Am writing a ksh script where I am looking for processes that has gone defunct and all of which has the same PPID PID is the variable that I need to match as this is the process ID of the processes that has gone defunct Am just curious how come the following DOES NOT work? ps... (6 Replies)
Discussion started by: newbie_01
6 Replies
drotmg.f(3)							      LAPACK							       drotmg.f(3)

NAME
drotmg.f - SYNOPSIS
Functions/Subroutines subroutine drotmg (DD1, DD2, DX1, DY1, DPARAM) DROTMG Function/Subroutine Documentation subroutine drotmg (double precisionDD1, double precisionDD2, double precisionDX1, double precisionDY1, double precision, dimension(5)DPARAM) DROTMG Purpose: CONSTRUCT THE MODIFIED GIVENS TRANSFORMATION MATRIX H WHICH ZEROS THE SECOND COMPONENT OF THE 2-VECTOR (DSQRT(DD1)*DX1,DSQRT(DD2)*> DY2)**T. WITH DPARAM(1)=DFLAG, H HAS ONE OF THE FOLLOWING FORMS.. DFLAG=-1.D0 DFLAG=0.D0 DFLAG=1.D0 DFLAG=-2.D0 (DH11 DH12) (1.D0 DH12) (DH11 1.D0) (1.D0 0.D0) H=( ) ( ) ( ) ( ) (DH21 DH22), (DH21 1.D0), (-1.D0 DH22), (0.D0 1.D0). LOCATIONS 2-4 OF DPARAM CONTAIN DH11, DH21, DH12, AND DH22 RESPECTIVELY. (VALUES OF 1.D0, -1.D0, OR 0.D0 IMPLIED BY THE VALUE OF DPARAM(1) ARE NOT STORED IN DPARAM.) THE VALUES OF GAMSQ AND RGAMSQ SET IN THE DATA STATEMENT MAY BE INEXACT. THIS IS OK AS THEY ARE ONLY USED FOR TESTING THE SIZE OF DD1 AND DD2. ALL ACTUAL SCALING OF DATA IS DONE USING GAM. Parameters: DD1 DD1 is DOUBLE PRECISION DD2 DD2 is DOUBLE PRECISION DX1 DX1 is DOUBLE PRECISION DY1 DY1 is DOUBLE PRECISION DPARAM DPARAM is DOUBLE PRECISION array, dimension 5 DPARAM(1)=DFLAG DPARAM(2)=DH11 DPARAM(3)=DH21 DPARAM(4)=DH12 DPARAM(5)=DH22 Author: Univ. of Tennessee Univ. of California Berkeley Univ. of Colorado Denver NAG Ltd. Date: November 2011 Definition at line 91 of file drotmg.f. Author Generated automatically by Doxygen for LAPACK from the source code. Version 3.4.2 Tue Sep 25 2012 drotmg.f(3)
All times are GMT -4. The time now is 09:02 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy