Dynamic Grep


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Dynamic Grep
# 1  
Old 04-11-2011
Dynamic Grep

Experts,

Hi there.

Feel like kicking myself today as I wish I could have paid more attention to learning while I was in college. If only wishes were horses ....

Anyway, Im parsing the contents of a log file.

Now what I need to do here is search for a pattern and then Pick the top 3 occurrences of lets say each one of the TEXT"s for example "PROVISIONED, AVAILABLE, EARL SIGNO and then list the corresponding 10 occurrences of it.

So, here is what Im doing

Code:
cat $LOGFILE | grep 'Deleting Text' |  cut -d \" -f2 | more

… but this is only giving me text as below.
Code:
PROVISIONED
LOGGED NOR
EARL SIGNO
TURNED OFF
AVAILABLE



What Im looking for is as follows -
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Code:
Deleting Text on line 124     (ESN: 665367B8760830) because the number wasnt  "PROVISIONED" and is not found in the database
Deleting Text on line  2474 (ESN: 607765B8577434) because the number wasnt  "PROVISIONED" and is not found in the database
Deleting Text on line  2474 (ESN: 877765B8343429) because the number wasnt  "PROVISIONED" and is not found in the database

Deleting Text on line 47764 (ESN: 3214567B8765430) because a number wasnt  "AVAILABLE" and is not found in the database
Deleting Text on line 12400 (ESN: 7683567Z8776055) because a number wasnt  "AVAILABLE" and is not found in the database
Deleting Text on line  12073 (ESN: 3213468Z4735412) because a number wasnt  "AVAILABLE" and is not found in the database

Deleting Text on line 47764 (ESN: 5754567B8765430) because a number wasnt  "TURNED ON" and is not found in the database
Deleting Text on line  12400 (ESN: 0334567B8765430) because a number wasnt  "TURNED ON"  and is not found in the database
Deleting Text on line  12073 (ESN: 8213467B8765430) because a number wasnt  "TURNED ON"  and is not found in the database

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Please advise !

regards,
Lee

---------- Post updated at 09:42 PM ---------- Previous update was at 08:20 PM ----------

I guess, I did not make myself clear enough here.

Anyway, here is how far I could push this ...

Code:
cat $LOGFILE | grep -i 'Deleting Text' | cut -d'(' -f2 |  sed -e 's/because a number wasnt //g' | sed -e 's/ because the number wasnt //g' | sed -e 's/and is not found in the database//g'

Code:
ESN: 607765B8577434) "PROVISIONED" 
ESN: 877765B8343429) "PROVISIONED" 
ESN: 3214567B8765430) "AVAILABLE" 
ESN: 7683567Z8776055) "AVAILABLE" 
ESN: 3213468Z4735412) "AVAILABLE" 
ESN: 5754567B8765430) "TURNED ON" 
ESN: 0334567B8765430) "TURNED ON" 
ESN: 8213467B8765430) "TURNED ON"

Now, from the above lines, how do I get the following -

Code:
PROVISIONED - 607765B8577434
PROVISIONED - 877765B8343429

AVAILABLE - 3214567B8765430
AVAILABLE - 7683567Z8776055
AVAILABLE - 3213468Z4735412

TURNED ON - 5754567B8765430
TURNED ON - 0334567B8765430
TURNED ON - 8213467B8765430

Please advise.

regards,
Lee.

Last edited by Franklin52; 04-12-2011 at 03:37 AM.. Reason: Please use code tags
# 2  
Old 04-11-2011
Code:
sed -n 's/Deleting Text .*ESN: \(.*\)).*"\(.*\)".*/\2 - \1/p' $LOGFILE


Last edited by Chubler_XL; 04-12-2011 at 01:12 AM..
This User Gave Thanks to Chubler_XL For This Post:
# 3  
Old 04-12-2011
Code:
awk -F \" '/Deleting/{split($1,a,"[) ]");print $2 " - " a[7];next}1' infile

This User Gave Thanks to rdcwayx For This Post:
# 4  
Old 04-12-2011
Rdcwayx & Chubler_XL,

Thank you for your help. I appreciate it & what you wrote in here works the way you expected it & let me rather say this, the way I explained it here. Sorry my bad. Please bear with me here.

What I wanted was something like a random sample of data for each type of error message along with 3 examples of ESN related to that specific error message. When I say Error Type I mean, each specific error type that you see here, namely, PROVISIONED, AVAILABLE, TURNED ON, LOGGED NOR & EARL SIGNO.

Like I said, we have over 10,000 types of error messages and I cant write a grep on all of the 10,000 different Error Message Types and then pipe it to tail -3.

I hope I have explained myself clear enough this time around.

Please help.

best regards,
Lee.


RAW DATA -
Code:
PROVISIONED - 607765B8577434
PROVISIONED - 877765B8343429

AVAILABLE - 3214567B8765430
AVAILABLE - 7683567Z8776055
AVAILABLE - 3213468Z4735412

TURNED ON - 5754567B8765430
TURNED ON - 0334567B8765430
TURNED ON - 8213467B8765430
TURNED ON - 8283277B2992311

LOGGED NOR - 0334567B8765112
LOGGED NOR - 3772372B3345677 
LOGGED NOR - 0817272B4873848
LOGGED NOR - 8388311B1200192
LOGGED NOR - 3723288B8788211

EARL SIGNO - 0334567B8348880
EARL SIGNO - 1226676B1210002
EARL SIGNO - 9923883B8212121
EARL SIGNO - 0232233B1255122
EARL SIGNO - 2377288B1162666

EXPECTED OUTPUT -
Code:
PROVISIONED - 607765B8577434
PROVISIONED - 877765B8343429

AVAILABLE - 3214567B8765430
AVAILABLE - 7683567Z8776055
AVAILABLE - 3213468Z4735412

TURNED ON - 5754567B8765430
TURNED ON - 0334567B8765430
TURNED ON - 8213467B8765430

LOGGED NOR - 0334567B8765112
LOGGED NOR - 3772372B3345677 
LOGGED NOR - 0817272B4873848

EARL SIGNO - 9923883B8212121
EARL SIGNO - 0232233B1255122
EARL SIGNO - 2377288B1162666


Last edited by Franklin52; 04-12-2011 at 03:31 AM.. Reason: Please use code tags
# 5  
Old 04-12-2011
Head 3 preserving blank lines ...
Code:
awk '!$1 || d[$1]++ < 3 {print}' <filename>

Head 3 losing blank lines ...
Code:
awk '$1 && d[$1]++ < 3 {print}' <filename>

Tail 3 losing blank lines is a little more involved (note that the order of output sections is unspecified) ...
Code:
awk '$1 { d[$1]=d[$1] " " $NF }
     END {
        for (e in d)
        {
            n=split(d[e], a)
            for (i=(n<4 ? 1 : n-2); i<=n; i++) print e " - " a[i]
        }
     }' <filename>

Regards,
Mark.
This User Gave Thanks to cambridge For This Post:
# 6  
Old 04-12-2011
Mark,

That worked like a charm.

Many a "THANKS" for the help.

regards,
Lee.

---------- Post updated at 03:31 AM ---------- Previous update was at 03:26 AM ----------

Mark,

Can you please decipher this for me ?

What is this "d[$1]++" doing ?

what is the purpose of using d before incrementing ?

Once again, thanks a million.

regards,
Lee.
# 7  
Old 04-14-2011
Code:
d[$1]++: 

set $1 in array d, and count it.

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Inconsistent `ps -eaf -o args | grep -i sfs_pcard_load_file.ksh | grep -v grep | wc -l`

i have this line of code that looks for the same file if it is currently running and returns the count. `ps -eaf -o args | grep -i sfs_pcard_load_file.ksh | grep -v grep | wc -l` basically it is assigned to a variable ISRUNNING=`ps -eaf -o args | grep -i sfs_pcard_load_file.ksh |... (6 Replies)
Discussion started by: wtolentino
6 Replies

2. Shell Programming and Scripting

dynamic thread in grep

hi name$count = some dynamic value now i want to use dat dynamic value in grep so how m i supposed to do dat i tried `grep $(name$count) source` , it is not working ...... what will be after grep so dat i will be... (4 Replies)
Discussion started by: Gl@)!aTor
4 Replies

3. Shell Programming and Scripting

dynamic string searching for grep

hi my code is something like count=0 echo "oracle TABLESPACE NAME nd TARGET" while do count=`expr $count + 1` (1) tts_space_name$count=`echo $tts | cut -d "," -f$count` (2) target$count=grep $(tts_space_name$count)... (2 Replies)
Discussion started by: Gl@)!aTor
2 Replies

4. Shell Programming and Scripting

Help with Dynamic variable

I need some variable help TEMP1=Jane TEMP2=Sue X=1 eval USER=TEMP${X} echo $USER This gives output USER1 I would like to get Jane I have tried eval USER='TEMP${X}' eval USER="TEMP${X}" eval USER=`TEMP${X}` (3 Replies)
Discussion started by: Jotne
3 Replies

5. UNIX for Advanced & Expert Users

Dynamic Tunnel

Hi All, Anyone here already done similar to the queries below ? Is there is a way to have a setup that would allow an SA to easily hook any prod server (external ip) to an internal ip without network reconfigurations ? Your comments here will be much appreciated. Thanks (4 Replies)
Discussion started by: linuxgeek
4 Replies

6. UNIX for Advanced & Expert Users

Sql dynamic table / dynamic inserts

I have a file that reads File (X.txt) Contents of record 1: rdrDESTINATION_ADDRESS (String) "91 971502573813" rdrDESTINATION_IMSI (String) "000000000000000" rdrORIGINATING_ADDRESS (String) "d0 movies" rdrORIGINATING_IMSI (String) "000000000000000" rdrTRAFFIC_EVENT_TIME... (0 Replies)
Discussion started by: magedfawzy
0 Replies

7. Shell Programming and Scripting

MEM=`ps v $PPID| grep -i db2 | grep -v grep| awk '{ if ( $7 ~ " " ) { print 0 } else

Hi Guys, I need to set the value of $7 to zero in case $7 is NULL. I've tried the below command but doesn't work. Any ideas. thanks guys. MEM=`ps v $PPID| grep -i db2 | grep -v grep| awk '{ if ( $7 ~ " " ) { print 0 } else { print $7}}' ` Harby. (4 Replies)
Discussion started by: hariza
4 Replies

8. Shell Programming and Scripting

Dynamic Menu Help

I'm working on a menu to read folders in as menu selections then CD to the selected folder and display the contained files as menu selections for execution. I'm using the following to read in the file list but I get lost after that. I only read in files that begin with CAPs. The problem is... (3 Replies)
Discussion started by: ScottKe
3 Replies

9. OS X (Apple)

Dynamic DNS

I have a need to regularly add all macs in our domain to dns. Ideally it would work like Wintel machines. Transparently and automatically. What are the tools, scripts, roadblocks to doing so? I'm not talking about DynDNS type of service here. This is the internal dns for where I work. (0 Replies)
Discussion started by: [MA]Flying_Meat
0 Replies

10. Programming

why to use dynamic cast in c++

class base { public: virtual void disp() {cout<<"from base\n";} }; class derv : public base { public: void disp() {cout<<"from der\n";} }; int main() { base *b=new derv; b->disp(); derv *d; d=dynamic_cast<derv*>(b); d->disp(); return(0); (0 Replies)
Discussion started by: sarwan
0 Replies
Login or Register to Ask a Question