Sponsored Content
Top Forums Shell Programming and Scripting grep'ing and sed'ing chunks in bash... need help on speeding up a log parser. Post 302309278 by devtakh on Tuesday 21st of April 2009 01:36:17 PM
Old 04-21-2009
try this -
$ sort -k 4 logfile | awk 'NR==1{prev=$4;txt="session."$4"."$6"."$7;printf("%s\n%s",txt,$0);getliine}{if (prev !~ $4){txt="session."$4"."$6"."$7;printf("%s\n%s",txt,$0);prev=$4}else {print $0;prev=$4}}'

will give something like this -

session.11F.Processing.
20090409 000127 - 11F - Processing20090409 000127 - 11F - Processing
20090409 000127 - 11F - Query 543 starting session
session.BD0.More.Processing
20090409 000122 - BD0 - More Processing20090409 000125 - BD0 - More Processing
20090409 000126 - BD0 - Order 123 shutting down
20090409 000122 - BD0 - Order 123 starting session
20090409 000122 - BD0 - Processing 1
20090409 000124 - BD0 - Processing 2
session.DD1.Cancel.345
20090409 000125 - DD1 - Cancel 345 starting session20090409 000125 - DD1 - Processing
20090409 000126 - DD1 - Processing 2
session.EF0.Order.234
20090409 000123 - EF0 - Order 234 starting session20090409 000124 - EF0 - Processing
20090409 000125 - EF0 - Processing


cheers,
Devaraj Takhellambam
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

grep'ing for text within a bunch of files...?

I have, say, a dozen files, and I want to grep for a string of text within them. I don't remember the exact syntax, but let me give it a shot and show you an idea here... find . -type f -exec grep thisword {} \; ...and there's a way to put more than one grep into the statement, so it will tell... (1 Reply)
Discussion started by: kitykity
1 Replies

2. Shell Programming and Scripting

grep'ing for specific directories, and using the output to move files

Hello, this is probably another really simple tasks for most of you gurus, however I am trying to make a script which takes an input, greps a specific file for that input, prints back to screen the results (which are directory names) and then be able to use the directory names to move files.... (1 Reply)
Discussion started by: JayC89
1 Replies

3. Shell Programming and Scripting

grep'ing a file until a certain message appears

Hello, I'm writing a script that will automate the launch of some services on my AIX machine. However, some services are dependent on the successful startup of others. When I start these services manually, I usually just check a log file until I see a message that confirms a successful... (3 Replies)
Discussion started by: pallak7
3 Replies

4. Shell Programming and Scripting

pipe'ing grep output to awk

This script is supposed to find out if tomcat is running or not. #!/bin/sh if netstat -a | grep `grep ${1}: /tomcat/bases | awk -F: '{print $3}'` > /dev/null then echo Tomcat for $1 running else echo Tomcat for $1 NOT running fi the /tomcat/bases is a file that... (2 Replies)
Discussion started by: ziggy25
2 Replies

5. Programming

Obfuscate'ing a.out ... ???

Hi all, I've search the forums regarding posts similar to this already but can't find the suitable response. Am actually looking for something very trivial I think. I just want to mask/obfuscate the a.out file and run it like a normal UNIX program. I've look at gpg and encryption but it requires... (4 Replies)
Discussion started by: newbie_01
4 Replies

6. Shell Programming and Scripting

grep'ing a variable that contains a metacharacter ($) with a while loop

This is driving me crazy, and I'm hoping someone can help me out with this. I'm trying to do a simple while loop to go through a log file. I'm pulling out all of the lines with a specific log line, getting an ID from that line, and once I have a list of IDs I want to loop back through the log and... (2 Replies)
Discussion started by: DeCoTwc
2 Replies

7. Shell Programming and Scripting

Severe performance issue while 'grep'ing on large volume of data

Background ------------- The Unix flavor can be any amongst Solaris, AIX, HP-UX and Linux. I have below 2 flat files. File-1 ------ Contains 50,000 rows with 2 fields in each row, separated by pipe. Row structure is like Object_Id|Object_Name, as following: 111|XXX 222|YYY 333|ZZZ ... (6 Replies)
Discussion started by: Souvik
6 Replies

8. Shell Programming and Scripting

grep'ing dot history file

Hi, I tried to grep ".sh_history" (DOTsh_history) file and did not return anything though I found the word in .sh _history file through vi editor in Linux. Then I tried to grep ".profile" to check if it is the prob with hidden files and I got results. Then I verified the same with my friend... (4 Replies)
Discussion started by: bobbygsk
4 Replies

9. Shell Programming and Scripting

Grep'ing information from a log file on SUN OS 5

Hi Guys, I'm trying to write an script that will be launched by a user. The script will look at a log file and check for alerts with the date (supplied by user) and a machine's hostname (also supplied by the user). I'm trying to get the output formatted just like the log file. The logfile looks... (5 Replies)
Discussion started by: illgetit
5 Replies

10. UNIX for Dummies Questions & Answers

grep'ing a variable - why not working

Hi all, Am writing a ksh script where I am looking for processes that has gone defunct and all of which has the same PPID PID is the variable that I need to match as this is the process ID of the processes that has gone defunct Am just curious how come the following DOES NOT work? ps... (6 Replies)
Discussion started by: newbie_01
6 Replies
cmannotifyd(8)						     CMAN Notification Daemon						    cmannotifyd(8)

NAME
cmannotifyd - CMAN Notification Daemon SYNOPSIS
cmannotifyd [-f] [-d] DESCRIPTION
The cmannotifyd daemon talks to CMAN and provides a mechanism to notify external entities about cluster changes. CMAN dispatches 3 kind of notifications: - CMAN_REASON_TRY_SHUTDOWN when cman requests to all clients if it is allowed to shutdown. - CMAN_REASON_STATECHANGE when cman detects a node joining or leaving the cluster. - CMAN_REASON_CONFIG_UPDATE when a configuration change event has been detected/requested. These notifications are then dispatched to the shell script cman_notify in the environment variable CMAN_NOTIFICATION. cman_notify will then execute all the scripts in the configured notification directory (default: /etc/cluster/cman-notify.d) passing a very minimal set of envvars including, of course, the CMAN_NOTIFICATION= type. The execution order is set by the filename as shown by "LC_ALL=C ls -las". cmannotifyd logs are stored in the default log file (/var/log/cluster/cmannotifyd.log). cman_notify logs are stored in the default log file (/var/log/cluster/cman_notify.log). By default the output from the scripts executed by cman_notify is redirected to /dev/null. Users can either set CMAN_NOTIFICATION_DEBUG=1 in their environment or set proper debug configura- tion in cluster.conf to redirect scripts output to the cman_notify log file. NOTES
cmannotifyd does not block on cman_notify nor check the exit status of the script. Notifications are dispatched in the same order as they arrive, one by one. CMAN_REASON_TRY_SHUTDOWN is passed to scripts for information only, they can not influence cman's decsion about whether or not to shut down. CMAN_REASON_STATECHANGE also implies CMAN_NOTIFICATION_QUORUM exported in the environment. CMAN_NOTIFICATION_QUORUM will be set to 1 (when the node is part of a quorate cluster) or 0 (otherwise). A template for cman_notify scripts can be found in the doc/ directory. OPTIONS
-f Run in the foreground (do not fork / daemonize). -d Enable debug output. November 2008 cmannotifyd(8)
All times are GMT -4. The time now is 01:47 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy