Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Extracting strings from a log file.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extracting strings from a log file.
# 1  
Old 09-05-2011
Extracting strings from a log file.
I'm new to all this and I've been fiddling with this problem for HOURS and feel silly that I can't work it out!

I have a .log file that VERY long and looks like this:
Code:
2011-08-31 10:03:34      SUESTART AG Amndmnt Client WebRequest DNU [1-7661VJ]SUEEND Sequence: 600, 
2011-08-31 10:03:34      SUESTART AG Amndmnt Client WebRequest DNU [1-7661VJ]SUEEND Person Candidates: 
2011-08-31 10:03:34      SUESTART AG CRM SR - Email/SR Email Surveys [1-6NUS6A]SUEEND Sequence: 671, 
2011-08-31 10:03:34      SUESTART AG CRM SR - Email/SR Email Surveys [1-6NUS6A]SUEEND Person 
2011-08-31 10:03:34      SUESTART AG CRM SR - ServiceFailure/Complaint/SurveyResponse [1-67MD2A]SUEEND Sequence: 
31 10:03:34      SUESTART AG CRM SR -  ServiceFailure/Complaint/SurveyResponse [1-67MD2A]SUEEND Person  Candidates: From Rule, Organization Candidates: From Rule, Non-Exclusive

Note that this is not exactly the file - I've manipulated it a bit, but you'll get the idea. I've managed to amend the file to put in SUESTART and SUEEND to demarcate the data that I'm interested in. I'd like to output a file that only returns the text inbetween SUESTART and SUEEND that looks like this:
Code:
AG Amndmnt Client WebRequest DNU [1-7661VJ]
AG Amndmnt Client WebRequest DNU [1-7661VJ]
AG CRM SR - Email/SR Email Surveys [1-6NUS6A]
AG CRM SR - Email/SR Email Surveys [1-6NUS6A]
AG CRM SR - ServiceFailure/Complaint/SurveyResponse [1-67MD2A]
AG CRM SR -  ServiceFailure/Complaint/SurveyResponse [1-67MD2A]

This is the closest I've got:

Code:
sed -e 's/.*SUESTART //g' -e 's/SUEEND.*$//g' < rules.log > output.log

But this doesn't work because it only returns the first line and then just trims off the rest of the entire log file:
Code:
AG Amndmnt Client WebRequest DNU [1-7661VJ]

Any help will be much appreciated!
Thank you.
Last edited by Scott; 09-05-2011 at 10:03 AM.. Reason: Please use code tags
# 2  
Old 09-05-2011
Code:
$ sed 's/.*SUESTART\(.*\)SUEEND.*/\1/' file
 AG Amndmnt Client WebRequest DNU [1-7661VJ]
 AG Amndmnt Client WebRequest DNU [1-7661VJ]
 AG CRM SR - Email/SR Email Surveys [1-6NUS6A]
 AG CRM SR - Email/SR Email Surveys [1-6NUS6A]
 AG CRM SR - ServiceFailure/Complaint/SurveyResponse [1-67MD2A]
 AG CRM SR - ServiceFailure/Complaint/SurveyResponse [1-67MD2A]

Guru.
# 3  
Old 09-05-2011
A Perl solution using lookahead and look behind
Code:
 perl -e   'while(<>){print "$1\n" if /(?<=SUESTART)(.+)(?=SUEEND)/;}' ~/src/Perl/data.tmp

# 4  
Old 09-05-2011
Code:
perl -nle '/SUESTART (.*)SUEEND/ && print $1' file

# 5  
Old 09-05-2011
PHP
Wow - what a quick response. I've tried all 3 recommendations:


1. sed 's/.*SUESTART\(.*\)SUEEND.*/\1/' rules.log > output.log

returns just the first line:
Code:
AG SR Assignment - Unit Trust - Bulk - Private Client [1-6NUS86]


2. perl -e 'while(<>){print "$1\n" if /(?<=SUESTART)(.+)(?=SUEEND)/;}' rules.log > output.log

interesting output, but not what I need. It returns everything inbetween the first SUESTART and the last SUEEND:

Code:
AG Amndmnt Client WebRequest DNU [1-7661VJ]SUEEND Sequence: 600, 
2011-08-31 10:03:34      SUESTART AG Amndmnt Client WebRequest DNU [1-7661VJ]SUEEND Person Candidates: 
2011-08-31 10:03:34      SUESTART AG CRM SR - Email/SR Email Surveys [1-6NUS6A]SUEEND Sequence: 671, 
2011-08-31 10:03:34      SUESTART AG CRM SR - Email/SR Email Surveys [1-6NUS6A]SUEEND Person 
2011-08-31 10:03:34      SUESTART AG CRM SR - ServiceFailure/Complaint/SurveyResponse [1-67MD2A]SUEEND Sequence: 
31 10:03:34      SUESTART AG CRM SR -  ServiceFailure/Complaint/SurveyResponse [1-67MD2A]


3. perl -nle '/SUESTART (.*)SUEEND/ && print $1' rules.log > output.log

does the same as 2.



Any more suggestions?

Thanks so much!

---------- Post updated at 03:24 PM ---------- Previous update was at 03:22 PM ----------

just to add the icon to show that suggestion didn't work... thanks anyway!
Last edited by radoulov; 09-05-2011 at 10:30 AM.. Reason: Please use code tags!
# 6  
Old 09-05-2011
I think you should post the data that you are using this code on, without any modifications...
# 7  
Old 09-05-2011
attaching file
Hi

I attached the rules.txt file (I wasn't able to attach a .log file but the .txt works in the same way.

Thanks for any responses!
rules.txt (2.6 KB) 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash: extracting 2 strings from 1 line

Hi everyone. I am very new in bash scripting (and scripting at all). I've got lines like these: -rw-r--r-- 1 setub 1049089 27M mars 13 2017 arch_amiel_038g_f016r.tif -rw-r--r-- 1 setub 1049089 584K juin 9 2008 arch_amiel_composition.jpgI wish to extract 2 string types so that I can... (4 Replies)
Discussion started by: setub
4 Replies

2. UNIX for Beginners Questions & Answers

Extracting strings at various positions of text file

Hi Team - I hope everyone has been well! I export a file from one of our source systems that gives me more information than I need. The way the file outputs, I need to extract certain strings at different positions on the file and echo them to another file. I can do this in batch easily,... (2 Replies)
Discussion started by: SIMMS7400
2 Replies

3. Shell Programming and Scripting

Extracting text between two constant strings

Hi All, I have a file whose common patter is like this: .I 1 .U 87049087 .S Some text here too .M This is a text .T Some another text here .P Name of the book .W Some lines of more text. This text needs to be extracted. .A more text goes here too .I 2 (2 Replies)
Discussion started by: shoaibjameel123
2 Replies

4. Shell Programming and Scripting

Extracting text between two strings

Hi, I've looked at a few existing posts on this, but they don't seem to work for my inputs. I have a text file where I want to extract all the text between two strings, every time that occurs. Eg my input file is Anna said that she would fetch the bucket. Anna and Ben moved the bucket.... (9 Replies)
Discussion started by: JamesForeman
9 Replies

5. Shell Programming and Scripting

Extracting text between two strings, first instance only

There are a lot of ways to extract text from between two strings, but what if those strings occur multiple times and you only want the text from the first two strings? I can't seem to find anything to work here. I'm using sed to process the text after it's extracted, so I prefer a sed answer, but... (4 Replies)
Discussion started by: fubaya
4 Replies

6. Shell Programming and Scripting

extracting numbers from strings

Hello all, I am being dumb with this and I know there is a simple solution. I have a file with the follwing lines bc stuff (more)...............123 bc stuffagain (moretoo)............0 bc stuffyetagain (morehere)......34 failed L3 thing..............1 failed this... (2 Replies)
Discussion started by: gobi
2 Replies

7. Shell Programming and Scripting

extracting a set of strings from a text file

i have textfiles that contain a series of lines that look like this: string0 .................................................... column3a column4a string1**384y0439 ..................................... column3b column4b... (2 Replies)
Discussion started by: Deanne
2 Replies

8. Shell Programming and Scripting

Extracting the lines between 2 strings of a file

Hi, I have a sql file and i need to extract the table names used in the sql file using a unix script. If i can extract the lines between the keywords 'FROM' and 'WHERE' in the file, my job is done. can somebody tell me how to do this using a shell script. If u can just let me know, how to... (2 Replies)
Discussion started by: babloo
2 Replies

9. Shell Programming and Scripting

Help with extracting strings from a file

I want to collect the characters from 1-10 and 20-30 from each line of the file and take them in a file in the following format.Can someone help me with this : string1,string2 string1,string2 string1,string2 : : : : (7 Replies)
Discussion started by: cmsdelhi
7 Replies

10. UNIX for Dummies Questions & Answers

Extracting strings

Hi, How do I extract the bytes size string from the ls -l command. (1 Reply)
Discussion started by: hugow
1 Replies
Login or Register to Ask a Question