Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

extract based on pattern

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting extract based on pattern
# 1  
Old 12-12-2008
extract based on pattern
I have a mail log file and I want to extract some lines belonging to one domain. For example

Input File:
Code:
Dec 12 03:15:28 postfix/smtpd[550]: 3F481EB0295: client=unknown[x.x.x.x], sasl_method=PLAIN, sasl_username=abcd@xyz.com
Dec 12 03:22:08 postfix/smtpd[1428]: 60B56EE001D: client=5ad9b9ba.com[x.x.x.x], sasl_method=LOGIN, sasl_username=efgh@abc.com
Dec 12 postfix/smtpd[1428]: 4314FEE0019: client=5ad9b9ba.bb.com[x.x.x.x], sasl_method=LOGIN, sasl_username=xyz@xyz.com

and so on

I want to extract those lines which contains sasl_username=anything@xyz.com. There can be 1000 lines which contains @xyz.com, I want all the lines. Before @xyz.com it can be anything. Hope I am clear.

I am trying with grep and awk commands but could not able to do so. Please suggect any idea
# 2  
Old 12-12-2008
With grep it could be as simple as:
Code:
grep xyz.com logfile

What have you tried so far? man grep or search this forum for grep, similar questions have been asked so many times I guess.
# 3  
Old 12-12-2008
If you want to extract all lines which contain @xyz.com then

cat <input file> | grep @xyz.com

Cheers,
Thiru
# 4  
Old 12-12-2008
First thanx for the reply. I think I was not clear in my question. There will be n number of lines besides
Code:
Dec 12 03:15:28 postfix/smtpd[550]: 3F481EB0295: client=unknown[x.x.x.x], sasl_method=PLAIN, sasl_username=abcd@xyz.com
Dec 12 03:22:08 postfix/smtpd[1428]: 60B56EE001D: client=5ad9b9ba.com[x.x.x.x], sasl_method=LOGIN, sasl_username=efgh@abc.com
Dec 12 postfix/smtpd[1428]: 4314FEE0019: client=5ad9b9ba.bb.com[x.x.x.x], sasl_method=LOGIN, sasl_username=xyz@xyz.com

which also contains
Code:
blah blah from=<abc@xyz.com> blah blah
blah blah from=<def@xyz.com> .........
or it can be like
to=<abc@xyz.com>
to=<def@xyz.com>

But my requirement is I have to extract those lines which have string "sasl_username=anything@xyz.com" from the logs. I hope this time I am clear
# 5  
Old 12-12-2008
I found the solution of my own question...
Code:
awk '/sasl_username=/ && /xyz.com/' maillog.txt

This line contains queue id in the 6th field. I am trying to have that value like
Code:
awk '/sasl_username=/ && /xyz.com/' maillog.txt | awk 'printf {$6}'

I know this is wrong technique. Can anybody let me know, btw I am searching this forum for my answer.
# 6  
Old 12-12-2008
Quote:
Originally Posted by Bijayant Kumar
I found the solution of my own question...
Code:
awk '/sasl_username=/ && /xyz.com/' maillog.txt

This line contains queue id in the 6th field. I am trying to have that value like
Code:
awk '/sasl_username=/ && /xyz.com/' maillog.txt | awk 'printf {$6}'

I know this is wrong technique. Can anybody let me know, btw I am searching this forum for my answer.

if you want to get the 6th field and using the SPACE as the delimeter..

Code:
awk '/sasl_username=/ && /xyz.com/ {print $6}' maillog.txt

from your example above that would give you
Code:
client=unknown[x.x.x.x],
sasl_method=LOGIN,

strange! Is that what you want?
# 7  
Old 12-12-2008
Solved
Thanks. My requirement is half solved from your all help. I will return soon with my next queries of second stage related to my program.

Forum rocks !!!!!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract whole word preceding a specific character pattern with first occurence of the pattern

Hello. Here is a file contents : declare -Ax NEW_FORCE_IGNORE_ARRAY=(="§" ="§" ="§" ="§" ="§" .................. ="§"Here is a pattern =I want to extract 'NEW_FORCE_IGNORE_ARRAY' which is the whole word before the first occurrence of pattern '=' Is there a better solution than mine :... (3 Replies)
Discussion started by: jcdole
3 Replies

2. UNIX for Beginners Questions & Answers

Extract some characters from lines based on pattern

Hi All, i would like to get some help regarding extracting certain characters from a line grepped. blahblah{1:F01IRVTUS30XXXX0000000001}{2:I103IRVTDEF0XXXXN}{4:blah blahblah{1:F01IRVTUS30XXXX0000000001}{2:I103IRVTDEF0XXXXN}{4:blah... (10 Replies)
Discussion started by: mad man
10 Replies

3. UNIX for Beginners Questions & Answers

Extract file name based on the pattern

Hello All, I have multiple files in a hadoop /tmp/cloudera directory. Filename are as follows ABC_DATA_BAD5A_RO_F_20161104.CSV ABC_DATA_BAD6C_VR_F_20161202.CSV ABC_DATA_BAD7A_TR_F_20162104.CSV ABC_DATA_BAD2A_BR_F_20161803.CSV ABC_DATA_BAD3T_KT_F_20160106.CSV I just need filenames... (6 Replies)
Discussion started by: prajaktaraut
6 Replies

4. Shell Programming and Scripting

Extract date from files based on file pattern

I want to extract dates from the files and i have different types of files with pattern. I have list file with the patterns and want to date extract based on it in a sh script Files in the directory : file1_20160101.txt file2_20160101_abc.txt filexyz20160101.txt list file with... (2 Replies)
Discussion started by: lijjumathew
2 Replies

5. Shell Programming and Scripting

Splitting textfile based on pattern and name new file after pattern

Hi there, I am pretty new to those things, so I couldn't figure out how to solve this, and if it is actually that easy. just found that awk could help:(. so i have a textfile with strings and numbers (originally copy pasted from word, therefore some empty cells) in the following structure: SC... (9 Replies)
Discussion started by: luja
9 Replies

6. UNIX for Dummies Questions & Answers

Find next line based on pattern, if it is similar pattern skip it

Hi, I am able to get next line if it is matching a particular pattern. But i need a way to skip if next line also matches same pattern.. For example: No Records No Records Records found got it Records found Now i want to find 'Records found' after 'No Records' pattern matches.. ... (5 Replies)
Discussion started by: nagpa531
5 Replies

7. Shell Programming and Scripting

Search for a pattern,extract value(s) from next line, extract lines having those extracted value(s)

I have hundreds of files to process. In each file I need to look for a pattern then extract value(s) from next line and then search for value(s) selected from point (2) in the same file at a specific position. HEADER ELECTRON TRANSPORT 18-MAR-98 1A7V TITLE CYTOCHROME... (7 Replies)
Discussion started by: AshwaniSharma09
7 Replies

8. Shell Programming and Scripting

Print a pattern between the xml tags based on a search pattern

Hi all, I am trying to extract the values ( text between the xml tags) based on the Order Number. here is the sample input <?xml version="1.0" encoding="UTF-8"?> <NJCustomer> <Header> <MessageIdentifier>Y504173382</MessageIdentifier> ... (13 Replies)
Discussion started by: oky
13 Replies

9. Shell Programming and Scripting

sed: Find start of pattern and extract text to end of line, including the pattern

This is my first post, please be nice. I have tried to google and read different tutorials. The task at hand is: Input file input.txt (example) abc123defhij-E-1234jslo 456ujs-W-abXjklp From this file the task is to grep the -E- and -W- strings that are unique and write a new file... (5 Replies)
Discussion started by: TestTomas
5 Replies

10. Shell Programming and Scripting

pattern extract

Hi I have a pattern like : SYSTEM_NAME-232-S7-200810060949.LOG Here I need to extract system name and the timestamp and also the numeric number after "-S" i.e 7 here . I am not very sure of whether I should use sed / awk for this ?:confused: Thanks, Priya. (6 Replies)
Discussion started by: priyam
6 Replies
Login or Register to Ask a Question