Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Search and filter by TAG

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Search and filter by TAG
# 1  
Old 01-27-2011
Search and filter by TAG
Hello all,
searching on a text file (log file) is quite simple:
Code:
grep -i texttosearch filename | grep something

What I'm trying to do is filter the result by TAG and remove the double entries.

Like if the log file contains the following text (fields are separated by commas):
Code:
20101024_201000:Counter,RESPONSE_FAIL,NODE,ApplicationAccessGroup.ServerGroup.Server.41700,1,47,0;
20101024_201000:Counter,RESPONSE_OK,NODE,ApplicationAccessGroup.Server.41880,1,15,0;
20101024_201000:Counter,RESPONSE_FAIL,TOTAL,Total,25459;
20101024_201000:Counter,RESPONSE_FAIL,TOTAL,Total,1;
20101025_215000:Counter,RESPONSE_FAIL,TOTAL,Total,15459;

Now, time to filtering
Code:
20101024_201000:Counter,RESPONSE_OK,NODE,ApplicationAccessGroup.Server.41880,1,15,0;
20101025_215000:Counter,RESPONSE_FAIL,TOTAL,Total,15459;

So group by TAG (the bold/red one) and show only the last one.....

Is it possible to do in a simple way? Or it's so hard to do?

Hope my goal is clear! Smilie
# 2  
Old 01-27-2011
Code:
$ awk -F, '{A[$2]=$0} END{for(i in A) print A[i]}' infile
20101025_215000:Counter,RESPONSE_FAIL,TOTAL,Total,15459;
20101024_201000:Counter,RESPONSE_OK,NODE,ApplicationAccessGroup.Server.41880,1,15,0;

This User Gave Thanks to Chubler_XL For This Post:
Lord Spectre
# 3  
Old 01-27-2011
omg, that's awesome! All in one row!Smilie
Thanks Chubler, you are a Master, and I'm so newbie! Smilie

And, what about if I want to apply the same filter using one string like "ApplicationAccessGroup"? (so no based on positioning but filtering and group by text?



# 4  
Old 01-27-2011
I'm so 100% sure what you want here. I assume you want to filter by a text string, but still group by field 1?

Code:
$ awk -F, '/ApplicationAccessGroup/ {A[$2]=$0} END{for(i in A) print A[i]}' infile
20101024_201000:Counter,RESPONSE_OK,NODE,ApplicationAccessGroup.Server.41880,1,15,0;

Last edited by Chubler_XL; 01-27-2011 at 06:27 PM.. Reason: Typo
This User Gave Thanks to Chubler_XL For This Post:
Lord Spectre
# 5  
Old 01-27-2011
You're right! I was not clear!
I would like to filter and group ONLY by a text string (ie "ApplicationAccessGroup") and not by string positioning:

So, based on the same log from my previous post, the output should be:

Code:
20101024_201000:Counter,RESPONSE_OK,NODE,ApplicationAccessGroup.Server.41880,1,15,0;

What I don't understand is why you put:
Code:
{A[$2]=$0}

Does it mean it refer always to the second field (RESPONSE_OK)?
# 6  
Old 01-27-2011
So if I'm understanding this requirement correctly it could be written as:
The last record that contains "ApplicationAccessGroup"
Code:
grep "ApplicationAccessGroup" infile | tail -1

This User Gave Thanks to Chubler_XL For This Post:
Lord Spectre
# 7  
Old 01-28-2011
Yep! It was too simple! Smilie

May I ask you the last combination? Smilie Last question!!!!! Sorry, this time is very hard! Smilie
What about if I want to filter by TAG but I have this log:

Code:
INFO 2011.01.27 20:58:11.00  com.log.SocketQueryListener$ServerListener createSocketConnection
  PROT_SOCKET_OPEN   Accepted socket connection for port 5018

INFO 2011.01.27 20:58:15.242  com.log.SocketConnection doRun
  SOCKET_CLOSE_OK   Closed socket   node SocketQueryListener

INFO 2011.01.27 20:59:40.285  com.nbprotocol.connection.ServerSocketConnection onOpen
  PROT_SOCKET_OPEN   Open socket ServerSocketConnection : /192.168.1.27:56420//192.168.1.29:5000

INFO 2011.01.27 20:59:40.489  com.access.protocol.handler.ServerHandler 1234.1291717116.transitionState
  PROT_SOCKET_OPEN   CP connection 1234.1291717116 has changed state to Init

INFO 2011.01.27 20:59:40.489  java.lang.String EventDispatcher.logEvent
  PROT_STATE_CHANGE   Protocol handler  : connecting [NetworkAccessGroup.ClientGroup.Client.1234] changed {4}

INFO 2011.01.27 20:59:40.690  java.lang.String EventDispatcher.logEvent
  PROT_STATE_CHANGE   Protocol handler  : connectConfirmState [NetworkAccessGroup.ClientGroup.Client.1234] changed {4}

Well, I would like to filter and group like my first post, but this time I have no commas as reference and each log records is on two lines! Smilie
So the result should be:

Code:
INFO 2011.01.27 20:59:40.489  com.access.protocol.handler.ServerHandler 1234.1291717116.transitionState
  PROT_SOCKET_OPEN   CP connection 1234.1291717116 has changed state to Init

INFO 2011.01.27 20:58:15.242  com.log.SocketConnection doRun
  SOCKET_CLOSE_OK   Closed socket   node SocketQueryListener

INFO 2011.01.27 20:59:40.690  java.lang.String EventDispatcher.logEvent
  PROT_STATE_CHANGE   Protocol handler  : connectConfirmState [NetworkAccessGroup.ClientGroup.Client.1234] changed {4}
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed search and replace after xml tag

Hi All, I'm new to sed. In following XML file <interface type='direct'> <mac address='52:54:00:86:ce:f6'/> <source dev='eno1' mode='bridge'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> ... (8 Replies)
Discussion started by: varunrapelly
8 Replies

2. Shell Programming and Scripting

Search for a tag and display a message if not found.

Hi All, I am working with a XML file. Below is part for the file. <Emp:Profile> <Emp:Description>Admin</Emp:Description> <Emp:Id>12347</Emp:Id> </Emp:Profile> <Emp:Profile> ... (7 Replies)
Discussion started by: Girish19
7 Replies

3. Shell Programming and Scripting

To search for a particular tag in xml and collate all similar tag values and display them count

I want to basically do the below thing. Suppose there is a tag called object1. I want to display an output for all similar tag values under heading of Object 1 and the count of the xmls. Please help File: <xml><object1>house</object1><object2>child</object2>... (9 Replies)
Discussion started by: srkmish
9 Replies

4. Shell Programming and Scripting

Search for a html tag and print the entire tag

I want to print from <fruits> to </fruits> tag which have <fruit> as mango. Also i want both <fruits> and </fruits> in output. Please help eg. <fruits> <fruit id="111">mango<fruit> . another 20 lines . </fruits> (3 Replies)
Discussion started by: Ashik409
3 Replies

5. Shell Programming and Scripting

Need an efficient way to search for a tag in an xml file having millions of rows

Hi, I have an XML file with around 1 billion rows in it and i am trying to find the number of times a particular tag occurs in it. The solution i am using works but takes a lot of time (~1 hr) .Please help me with an efficient way to do this. Lets say the input file is <Root> ... (13 Replies)
Discussion started by: Sheel
13 Replies

6. Shell Programming and Scripting

Multiple Tag Search and Printing

Scenario: The following text belongs to a .doc file File: check1.asmFunction: MonksTag: NoTag: 001Tag: YesTag: 002File: check2.asmFunction: Perl MonksTag: YesTag: 003Tag: NoTag: 004File: check3.asmFunction: ExpertsTag: NoTag: 005Tag: NoTag: 006Function: Perl ExpertsTag: NoTag: 007Tag: YesTag: 008... (1 Reply)
Discussion started by: rajkrishna89
1 Replies

7. Shell Programming and Scripting

search pattern and mark/tag

Hi All, I have to search for patterns from a pattern file in a file and mark the matching lines. Input File: Student1 60 30 Student2 71 91 Student3 88 98 Pattern file: Student1 Fail Student2 Pass Student2 Pass Desired output: Student1 60 30 Fail Student2 71 91 Pass (5 Replies)
Discussion started by: saint2006
5 Replies

8. UNIX for Dummies Questions & Answers

Search and filter between two files

Hello, I have two files in this form that consist of three columns, a name (L*contig*), the length (length=**) and the sequence LT_file.txt LTcontig1 length=13 acccatgctttta LTcontig5 length=8 ggattacc LTcontig8 length=20 ccattgaccgtacctgatcg LTcontig23 length=5 accta and... (5 Replies)
Discussion started by: FelipeAd
5 Replies

9. Shell Programming and Scripting

Search for particular tag and arrange as coordinates

Hi I have a file whose sample contents are shown here, 1.2.3.4->2.4.2.4 a(10) b(20) c(30) 1.2.3.4->2.9.2.4 a(10) c(20) 2.3.4.3->3.6.3.2 b(40) d(50) c(20) 2.3.4.3->3.9.0.2 a(40) e(50) c(20) 1.2.3.4->3.4.2.4 a(10) c(30) 6.2.3.4->2.4.2.5 c(10) . . . . Here I need to search... (5 Replies)
Discussion started by: AKD
5 Replies

10. UNIX for Dummies Questions & Answers

ldap search filter

Hi, I am trying to do an ldapsearch with a filter that checks the uid and the userpassword: $filter= "(&(uid=$user) (userpassword=$password)"; $objs = $ldap->search( base => $basedn, filter => "($filter)"); i based it on this example i found on CPAN: $mesg = $ldap->search( ... (2 Replies)
Discussion started by: tine
2 Replies
Login or Register to Ask a Question