Sponsored Content
Top Forums Shell Programming and Scripting Extracting data from large logs. Post 302126751 by Nick31 on Thursday 12th of July 2007 07:47:24 PM
Old 07-12-2007
Well, you could do it in perl by building an array of 11 strings, and push in your lines until your array is full. Then check the 8th string for your pattern. If it doesn't match, pop the first one out, push a new one in, check again. When it does match, print out your array. It's a brute force method, but it should work.

You could do this same concept in awk or sed using buffers.

You might also try grep -p if you can identify a way to divide your lines into paragraphs. For example, if every log entry starts with a line of "++++++" and the text you're searching for is the 8th line of the log entry, do:
grep -p"++++++" "searchstring" logfile

That would be the easiest method.
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk Help for extracting report from logs

Hi I have a log file, with lines of following kind. ------------------------ 2009-05-15 07:49:42,574 INFO - SqlMapObjectDataDao - select - selectObject - 2 ms 2009-05-15 07:49:42,575 INFO - SqlMapUserDao - select - getUserSystemAdminSuperGroup - 0 ms 2009-05-15 07:49:42,576 INFO -... (3 Replies)
Discussion started by: jitendriya.dash
3 Replies

2. UNIX for Dummies Questions & Answers

restrict data from getting written to Logs

$SYBASE/bin/isql -U $DB_USERID -S $DB_SERVER << ! >> $OUTFILE `echo $DB_PASSWD` use $db go Print " The processing" go ! # Extract data to file echo $DB_PASSWD | $SYBASE/bin/bcp $WRK_DB..open out $CONV_DIR/open".csv -t\, -c -U $DB_USERID -S $DB_SERVER -b 1000 | tail -3 I am able to... (0 Replies)
Discussion started by: w020637
0 Replies

3. Shell Programming and Scripting

Extracting a portion of data from a very large tab delimited text file

Hi All I wanted to know how to effectively delete some columns in a large tab delimited file. I have a file that contains 5 columns and almost 100,000 rows 3456 f g t t 3456 g h 456 f h 4567 f g h z 345 f g 567 h j k lThis is a very large data file and tab delimited. I need... (2 Replies)
Discussion started by: Lucky Ali
2 Replies

4. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

5. UNIX for Dummies Questions & Answers

help in extracting logs in readable format

hello everyone. newbie here in unix. I am trying to extract the logs of a certain job and would like to output it in a readable format, see below the CAT part: cat /var/opt/ctma/ctm/sysout/idwesct_sh30_eng_r6_cdcs_sh.LOG_05l0du_000* | egrep -i 'orderid:|file_name=' | sed... (1 Reply)
Discussion started by: eanne_may
1 Replies

6. UNIX for Dummies Questions & Answers

Extracting a block of text from a large file using variables?

Hi UNIX Members, I've been tasked with performing the following: Extract a block of data in column form #This data changes each time, therefore automating future procedures Please Note the following: line = reading a line from a file_list that leads to the data The filename is called... (16 Replies)
Discussion started by: Klor
16 Replies

7. Shell Programming and Scripting

Extracting logs using gunzip awk and gzip

Hi All I am trying to use a hard coded script into shell scripting but I am unable to . Kindly find the Script below along with the command Please help gunzip -c FilePath/FileName_*.gz | awk '$0 > "" && $0 < ""'|\ gzip >> FilePath/Outputfile.log.gz I Am trying to use this... (9 Replies)
Discussion started by: pulkitbahl
9 Replies

8. UNIX for Beginners Questions & Answers

Transpose large data in UNIX

Hi I have the following sample of data: my full data dimention is 900,000* 1119 rs987435 C G 1 1 1 0 2 rs345783 C G 0 0 1 0 0 rs955894 G T 1 1 2 2 1 rs6088791 ... (7 Replies)
Discussion started by: marwah
7 Replies

9. Programming

C++ help in large data set

Hi All, We are trying to replace a 3rdparty where we don't know how they handled the reader part here. The query below is getting 197 * 2038017 row in the table. In the below code we are trying to run the query and execute in the DB part and fetch and read the record. That is where it is... (1 Reply)
Discussion started by: arunkumar_mca
1 Replies
cmannotifyd(8)						     CMAN Notification Daemon						    cmannotifyd(8)

NAME
cmannotifyd - CMAN Notification Daemon SYNOPSIS
cmannotifyd [-f] [-d] DESCRIPTION
The cmannotifyd daemon talks to CMAN and provides a mechanism to notify external entities about cluster changes. CMAN dispatches 3 kind of notifications: - CMAN_REASON_TRY_SHUTDOWN when cman requests to all clients if it is allowed to shutdown. - CMAN_REASON_STATECHANGE when cman detects a node joining or leaving the cluster. - CMAN_REASON_CONFIG_UPDATE when a configuration change event has been detected/requested. These notifications are then dispatched to the shell script cman_notify in the environment variable CMAN_NOTIFICATION. cman_notify will then execute all the scripts in the configured notification directory (default: /etc/cluster/cman-notify.d) passing a very minimal set of envvars including, of course, the CMAN_NOTIFICATION= type. The execution order is set by the filename as shown by "LC_ALL=C ls -las". cmannotifyd logs are stored in the default log file (/var/log/cluster/cmannotifyd.log). cman_notify logs are stored in the default log file (/var/log/cluster/cman_notify.log). By default the output from the scripts executed by cman_notify is redirected to /dev/null. Users can either set CMAN_NOTIFICATION_DEBUG=1 in their environment or set proper debug configura- tion in cluster.conf to redirect scripts output to the cman_notify log file. NOTES
cmannotifyd does not block on cman_notify nor check the exit status of the script. Notifications are dispatched in the same order as they arrive, one by one. CMAN_REASON_TRY_SHUTDOWN is passed to scripts for information only, they can not influence cman's decsion about whether or not to shut down. CMAN_REASON_STATECHANGE also implies CMAN_NOTIFICATION_QUORUM exported in the environment. CMAN_NOTIFICATION_QUORUM will be set to 1 (when the node is part of a quorate cluster) or 0 (otherwise). A template for cman_notify scripts can be found in the doc/ directory. OPTIONS
-f Run in the foreground (do not fork / daemonize). -d Enable debug output. November 2008 cmannotifyd(8)
All times are GMT -4. The time now is 03:43 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy