Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

sed multiple multi line blocks of text containing pattern

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sed multiple multi line blocks of text containing pattern
# 1  
Old 05-30-2013
sed multiple multi line blocks of text containing pattern
Hi,

I have a log file which has sessionids in it, each block in the log starts with a date entry, a block may be a single line or multiple lines. I need to sed (or awk) out the lines/blocks with that start with a date and include the session id.

The files are large at several Gb.

My example file is :
Code:
$> cat test
Date1;stuff;sessionid1;stuff;restofline1
Date2;stuff;sessionid1;stuff;restofline2
 Something1;stuff
Something2;stuff
Date3;stuff;sessionid2;stuff;restofline3
Date4;stuff;sessionid2;stuff;restofline4
Date5;stuff;sessionid1;stuff;restofline5
Date6;stuff;sessionid2;stuff;restofline6

my required output would be :
Code:
Date1;stuff;sessionid1;stuff;restofline1
Date2;stuff;sessionid1;stuff;restofline2
Something1;stuff
Something2;stuff
Date5;stuff;sessionid1;stuff;restofline5

I am totally unsure how to use the N command to process multiple lines between say regex "^Date.*;sessionid1;" and "^Date" but I think thats the direction I need to be going.

I'm using ksh on Solaris

Any help would be very much appreciated.
# 2  
Old 05-30-2013
Here is an awk code:
Code:
/usr/xpg4/bin/awk '(/^Date/ && /sessionid1/) || !/^Date/' file

# 3  
Old 05-30-2013
Yoda, your answer is not correct; it'll match !/^Date/ in the entire file!
It is necessary to have a kind of state; awk and a state variable is appropriate.
Code:
awk '/^Date/ {s=0} /^Date2;.*;sessionid1;/ {s=1} s' test

nawk or /usr/xpg4/bin/awk.
/bin/awk needs (this time with the OPs input)
Code:
awk '/^Date/ {s=0} /^Date.*;sessionid1;/ {s=1} s!=0' test

Last edited by MadeInGermany; 05-30-2013 at 03:58 PM..
This User Gave Thanks to MadeInGermany For This Post:
andyatit
# 4  
Old 05-31-2013
Thanks, thats great ! Very neat.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Modify text file if found multiple pattern match for every line.

Looking for help, i have input file like below and want to modify to expected output, if can without create additional file, hope can direct modify it. have 2 thing need do. 1st is adding a word (testplan generation off) after ! ! IPG: Tue Aug 07 14:31:17 2018 2nd is adding... (16 Replies)
Discussion started by: kttan
16 Replies

2. Shell Programming and Scripting

sed a multiple line pattern that has newlines followed by text and r

Here is the text that I was to run sed on. In this text I want to insert a semi colon ';' before 'select a13.STORE_TYPE STORE_TYPE,' and after 'from ZZMR00 pa11' Input text: insert into ZZMQ01 select pa11.STATE_NBR STATE_NBR, pa11.STORE_TYPE STORE_TYPE, ... (9 Replies)
Discussion started by: v_vineeta11
9 Replies

3. UNIX for Advanced & Expert Users

sed REGEX to print multiple occurrences of a pattern from a line

I have a line that I need to parse through and extract a pattern that occurs multiple times in it. Example line: getInfoCall: info received please proceed, getInfoCall: info received please proceed, getInfoCall: info received please proceed, getInfoCall: info received please proceed,... (4 Replies)
Discussion started by: Vidhyaprakash
4 Replies

4. Shell Programming and Scripting

sed - filter blocks between single delimiters matching a pattern

Hi! I have a file with the following format:CDR ... MSISDN=111 ... CDR ... MSISDN=xxx ... CDR ... MSISDN=xxx ... CDR ... MSISDN=111 (2 Replies)
Discussion started by: Flavius
2 Replies

5. Shell Programming and Scripting

Replacing lines matching a multi-line pattern (sed/perl/awk)

Dear Unix Forums, I am hoping you can help me with a pattern matching problem. What am I trying to do? I want to replace multiple lines of a text file (that match a multi-line pattern) with a single line of text. These patterns can span several lines and do not always have the same number of... (10 Replies)
Discussion started by: thefang
10 Replies

6. Shell Programming and Scripting

sed command to grep multiple pattern present in single line and delete that line

here is what i want to achieve.. i have a file with below contents cat fileName blah blah blah . .DROP this REJECT that . --sport 7800 -j REJECT --reject-with icmp-port-unreachable --dport 7800 -j REJECT --reject-with icmp-port-unreachable . . . more blah blah blah --dport 3306... (14 Replies)
Discussion started by: vivek d r
14 Replies

7. Homework & Coursework Questions

sed Multiple Pattern search and delete the line

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: I have file which has got the following content sam 123 LD 41 sam 234 kp sam LD 41 kam pu sam LD 61 Now... (1 Reply)
Discussion started by: muchyog
1 Replies

8. Shell Programming and Scripting

Sed replace using same pattern repeating multiple times in a line

Sed replace using same pattern repeating multiple times in a line I have text like below in a file: I am trying to replace the above line to following How can I acheive this? I am able to do it if the occurrence is for 1 time: But If I try like below I am getting like this: I have to... (4 Replies)
Discussion started by: sol_nov
4 Replies

9. Shell Programming and Scripting

Multi-line filtering based on multi-line pattern in a file

I have a file with data records separated by multiple equals signs, as below. ========== RECORD 1 ========== RECORD 2 DATA LINE ========== RECORD 3 ========== RECORD 4 DATA LINE ========== RECORD 5 DATA LINE ========== I need to filter out all data from this file where the... (2 Replies)
Discussion started by: Finja
2 Replies

10. Shell Programming and Scripting

sed: Find start of pattern and extract text to end of line, including the pattern

This is my first post, please be nice. I have tried to google and read different tutorials. The task at hand is: Input file input.txt (example) abc123defhij-E-1234jslo 456ujs-W-abXjklp From this file the task is to grep the -E- and -W- strings that are unique and write a new file... (5 Replies)
Discussion started by: TestTomas
5 Replies
Login or Register to Ask a Question