Capturing multi-line records containing known value?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Capturing multi-line records containing known value?
# 1  
Old 03-03-2010
Question Capturing multi-line records containing known value?

Some records in a file look like this, with any number of lines between start and end flags:

Code:
/Start
Some stuff
Banana 1
Some more stuff
End/

/Start
Some stuff
End/

/Start
Some stuff
Some more stuff
Banana 2
End/

...how would I process this file to find records containing the keyword "Banana", i.e. to output:

Code:
/Start
Some stuff
Banana 1
Some more stuff
End/

/Start
Some stuff
Some more stuff
Banana 2
End/

I can do it using two awk scripts, but am sure it is doable with one! TIA...
# 2  
Old 03-03-2010
In case you want to do it using a script:

Code:
#!/usr/bin/ksh

v_banana="false"
v_output=""

while read LINE
do
    if [ "$LINE" = "/Start" ]
    then
        v_output="${LINE}\n"

    elif [ "$LINE" = "End/" ]
    then
        v_output="${v_output}${LINE}\n"

        if [ "$v_banana" = "true" ]
        then
            echo "$v_output"
        fi
        v_output=""
        v_banana="false"
    else
        v_output="${v_output}${LINE}\n"

        if [[ $LINE = *Banana* ]]
        then
            v_banana="true"
        fi
    fi
done < infile

# 3  
Old 03-03-2010
Another one:
Code:
awk -v var="Banana" '
/Start/ { s="" }
$0 ~ var { print s; f=1 }
{ s=s?s "\n" $0:$0 }
f { print }
/End/ { f=0 }
' file

# 4  
Old 03-04-2010
MySQL

Thanks to you both, but Franklin52 - that is exactly what I was after, save for needing to use nawk instead of awk (presumably due to the size of my input file). Would you be so kind as to walk me through the script and explain how some of the less obvious lines work - specifically:

Code:
{ s=s?s "\n" $0:$0 }


Last edited by cs03dmj; 03-04-2010 at 08:00 AM..
# 5  
Old 03-04-2010
Quote:
Originally Posted by cs03dmj
Thanks to you both, but Franklin52 - that is exactly what I was after, save for needing to use nawk instead of awk (presumably due to the size of my input file). Would you be so kind as to walk me through the script and explain how some of the less obvious lines work - specifically:

Code:
{ s=s?s "\n" $0:$0 }

This is called a conditional expression and is similar to:

Code:
if(s) {
  s=s "\n" $0
}
else {
  s=$0
}

Have a read of this:

Conditional Exp - The GNU Awk User's Guide
# 6  
Old 03-04-2010
Hello, all:

It's not as flexible as Franklin52's parameterized AWK script, but here's a solution for the sed freaks amongst us:
Code:
sed -n '/^\/Start/,/^End\//{H;/^End\//{x;/Banana/p;};/^\/Start/h;}'

And another AWK option:
Code:
awk -v key=Banana '/^\/Start/,/^End\//{s=s $0 RS} /^End\//{printf "%s",index(s,key)?s:""; s=""}'

Cheers,
Alister
# 7  
Old 03-17-2010
Thanks to you all for this; I have subsequently learnt a lot and expanded my final script (should it help anyone in future), which allows multiple searches to be passed from the command line and is based upon records separated by one or more empty lines.

Last edited by cs03dmj; 03-17-2010 at 11:41 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Multi line log files to single line format

I want to read the log file which was generate from other command . And the output was having multi line in log files for job name and server name. But i need to make all the logs on one line Source file 07/15/2018 17:02:00 TRANSLOG_1700 Server0005_SQL ... (2 Replies)
Discussion started by: ranjancom2000
2 Replies

2. Shell Programming and Scripting

Help with reformat single-line multi-fasta into multi-line multi-fasta

Input File: >Seq1 ASDADAFASFASFADGSDGFSDFSDFSDFSDFSDFSDFSDFSDFSDFSDFSD >Seq2 SDASDAQEQWEQeqAdfaasd >Seq3 ASDSALGHIUDFJANCAGPATHLACJHPAUTYNJKG ...... Desired Output File >Seq1 ASDADAFASF ASFADGSDGF SDFSDFSDFS DFSDFSDFSD FSDFSDFSDF SD >Seq2 (4 Replies)
Discussion started by: patrick87
4 Replies

3. Shell Programming and Scripting

Multi-line filtering based on multi-line pattern in a file

I have a file with data records separated by multiple equals signs, as below. ========== RECORD 1 ========== RECORD 2 DATA LINE ========== RECORD 3 ========== RECORD 4 DATA LINE ========== RECORD 5 DATA LINE ========== I need to filter out all data from this file where the... (2 Replies)
Discussion started by: Finja
2 Replies

4. Shell Programming and Scripting

Joining multi-line output to a single line in a group

Hi, My Oracle query is returing below o/p ---------------------------------------------------------- Ins trnas value a lkp1 x a lkp1 y b lkp1 a b lkp2 x b lkp2 y ... (7 Replies)
Discussion started by: gvk25
7 Replies

5. Shell Programming and Scripting

Transpose multi-line records into a single row

Now that I've parsed out the data that I desire I'm left with variable length multi-line records that are field seperated by new lines (\n) and record seperated by a single empty line ("") At first I was considering doing something like this to append all of the record rows into a single row: ... (4 Replies)
Discussion started by: daveyabe
4 Replies

6. Shell Programming and Scripting

Capturing the invalid records to error file

HI, I have a source file which has the below data. Tableid,table.txt sourceid,1,2,3,4,5,6 targetid,1,2,3,4,5,6 Tableid,table sourceid,1,2,3,4,5,6 targetid,1,2,3,4,5,6 Tableid,table.txt sourceid,1,2,3,4,5,6 targetid,1,2,3,4,5,6 Tableid,table sourceid,1,2,3,4,5,6 targetid,1,2,3,4,5,6... (6 Replies)
Discussion started by: shruthidwh
6 Replies

7. UNIX for Dummies Questions & Answers

Alphabetical sort for multi line records contains in a single file

Hi all, I So, I've got a monster text document comprising a list of various company names and associated info just in a long list one after another. I need to sort them alphabetically by name... The text document looks like this: Company Name: the_first_company's_name_here Address:... (2 Replies)
Discussion started by: quee1763
2 Replies

8. UNIX for Dummies Questions & Answers

Grep specific records from a file of records that are separated by an empty line

Hi everyone. I am a newbie to Linux stuff. I have this kind of problem which couldn't solve alone. I have a text file with records separated by empty lines like this: ID: 20 Name: X Age: 19 ID: 21 Name: Z ID: 22 Email: xxx@yahoo.com Name: Y Age: 19 I want to grep records that... (4 Replies)
Discussion started by: Atrisa
4 Replies

9. Shell Programming and Scripting

AWK Multi-Line Records Numbering Problem

I have a set of files of multi-line records with the records separated by a blank line. I needed to add a record number to the front of each line followed by a colon and did the following: awk 'BEGIN {FS = "\n"; RS = ""}{for (i=1; i<=NF; i++)print NR,":",$i}' ~/Desktop/data98-1-25.txt >... (3 Replies)
Discussion started by: RacerX
3 Replies

10. Shell Programming and Scripting

AWK Multi-Line Records Processing

I am an Awk newbie and cannot wrap my brain around my problem: Given multi-line records of varying lengths separated by a blank line I need to skip the first two lines of every record and extract every-other line in each record unless the first line of the record has the word "(CONT)" in the... (10 Replies)
Discussion started by: RacerX
10 Replies
Login or Register to Ask a Question