Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

how to filter out some paragraphs in a file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting how to filter out some paragraphs in a file
# 1  
Old 06-07-2006
Tools how to filter out some paragraphs in a file
Hi,

I am trying to filter out those paragraphs that contains 'CONNECT', 'alter system switch logfile'. That means say the input file is :

-------------------------------------------------------
Wed Jun 7 00:32:31 2006
ACTION : 'CONNECT'
CLIENT USER: prdadm
CLIENT TERMINAL:

Wed Jun 7 00:32:31 2006
ACTION : 'alter system switch logfile'
CLIENT USER: prdadm
CLIENT TERMINAL:


Wed Jun 7 00:32:31 2006
ACTION : 'CONNECT'
CLIENT USER: prdadm
CLIENT TERMINAL:

Wed Jun 7 00:32:42 2006
ACTION : 'ALTER DATABASE CLOSE NORMAL'
CLIENT USER: prdadm
CLIENT TERMINAL:
----------------------------------------------------------------

I would like the output file to be :

----------------------------------------------------------------
Wed Jun 7 00:32:42 2006
ACTION : 'ALTER DATABASE CLOSE NORMAL'
CLIENT USER: prdadm
CLIENT TERMINAL:
----------------------------------------------------------------

Do you have any script to perform this, thank you very much Smilie .
# 2  
Old 06-08-2006
I'm assuming the horizontal lines aren't actually in the file, and are just used instead of
Code:
code tags

Code tags are better because they preserve spacing.

Here's a script:
Code:
#!/bin/sh

SHOULDPRINT=1

function do_flush
{
        if [[ ${SHOULDPRINT} -eq 1 && ! -z ${OUT} ]]
        then
                echo "${OUT}"
        fi
}

while read LINE
do
        if [[ -z ${LINE} ]]
        then
                do_flush
                OUT=""
                SHOULDPRINT=1
        elif    echo "${LINE}" | egrep -q "CONNECT|alter system switch logfile"
        then
                SHOULDPRINT=0
        else
                if [[ -z ${OUT} ]]
                then
                        OUT="
${LINE}"
                else
                        OUT="${OUT}
${LINE}"
                fi
        fi
done
do_flush

# 3  
Old 06-08-2006
An awk example of the same thing:
Code:
 nawk '
    $0 !~ /ACTION : / {
        last_line=$0
    }

    $0 ~ /^ACTION : / && $0 !~ /CONNECT|alter system/ {
        print last_line
        print $0
        getline ; print
        getline ; print
    }
' your_log_file

# 4  
Old 06-08-2006
nawk -f cn.awk your_log_file

cn.awk:
Code:
BEGIN {
  RS=FS=""
  PATexclude="(CONNECT)|(alter system switch logfile)"
}
$0 !~ PATexclude

# 5  
Old 06-08-2006
Tools
Quote:
Originally Posted by vgersh99
nawk -f cn.awk your_log_file

cn.awk:
Code:
BEGIN {
  RS=FS=""
  PATexclude="(CONNECT)|(alter system switch logfile)"
}
$0 !~ PATexclude

Nice! This works as long as the paragraphs have whitespace between them.
# 6  
Old 06-08-2006
Quote:
Originally Posted by tmarikle
Nice! This works as long as the paragraphs have whitespace between them.
yes, that was the assumption - 'paragraphs are separated by blank lines'!
# 7  
Old 06-09-2006
Code:
awk 'BEGIN{RS=""; ORS="\n\n"}
  !/CONNECT|alter system switch logfile/'

The lines separating the paragraphs must be empty.

If the in-between lines may contain spaces or tabs:

Code:
gawk 'BEGIN{RS="\n([ \t]*\n)+";ORS="\n\n"}
 !/CONNECT|alter system switch logfile/'
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract paragraphs and count them

Hi, I have a text with a number of paragraphs in them. My problem is I need to locate certain errors/warning and extract/count them. Problem is I do not know how many paras are there with that particular type of error/warning. I had thought that somehow if I could count the number of... (25 Replies)
Discussion started by: dsid
25 Replies

2. UNIX for Dummies Questions & Answers

Filter records in a huge text file from a filter text file

Hi Folks, I have a text file with lots of rows with duplicates in the first column, i want to filter out records based on filter columns in a different filter text file. bash scripting is what i need. Data.txt Name OrderID Quantity Sam 123 300 Jay 342 498 Kev 78 2500 Sam 420 50 Vic 10... (3 Replies)
Discussion started by: tech_frk
3 Replies

3. Shell Programming and Scripting

Need help with sorting in paragraphs

I am very new to shell scripting, current try to do a sorting of a text file in paragraphs with ksh script. example: File content: A1100001 line 1 = "testing" line 2 = something, line 3 = 100 D1200003 line 1 = "testing" line 2 = something, line 3 = 100 B1200003 line 1 =... (3 Replies)
Discussion started by: gavin_L
3 Replies

4. Shell Programming and Scripting

Split text into paragraphs

Hi all! I want to make a code to split sentences into paragraphs maybe 4-5 sentences into one <p>text</p> there are no new lines in the text string any ideas with AWK, SSH? Thank you! (5 Replies)
Discussion started by: sanantonio7777
5 Replies

5. Shell Programming and Scripting

Extract paragraphs under conditions

Hi all, I want to extract some paragraphs out of a file under certain conditions. - The paragraph must start with 'fmri' - The paragraph must contain the string 'restarter svc:/system/svc/restarter:default' My input is like that : fmri svc:/system/vxpbx:default state_time Wed... (4 Replies)
Discussion started by: Armoric
4 Replies

6. Shell Programming and Scripting

file separated into paragraphs or pages

hi, i have file, file is separated into parahgraphs by these line(----------). i want to find out logId = string : "AIALARM", in each parahgraph or page if found then i want to cut next five lines.... ... (3 Replies)
Discussion started by: dodasajan
3 Replies

7. Shell Programming and Scripting

splitting a large text file into paragraphs

Hello all, newbie here. I've searched the forum and found many "how to split a text file" topics but none that are what I'm looking for. I have a large text file (~15 MB) in size. It contains a variable number of "paragraphs" (for lack of a better word) that are each of variable length. A... (3 Replies)
Discussion started by: lupin..the..3rd
3 Replies

8. Shell Programming and Scripting

fetching paragraphs with SED

hi, i am a SED newbie and i need some help. i have a log file as shown below. and i want to search specific Error Code, and fetch the whole paragraph. ... ... ................. ....ErrCode... ................. ... ... ... ................. ....ErrCode... ... (4 Replies)
Discussion started by: ipat
4 Replies

9. Shell Programming and Scripting

how to sort paragraphs by date within a file

hi all i want help in sortng date in paragraphs within file , i want to ask as if there any option to sort a certain pattern of file not the rest of file.i.e the data of file become sorted with respect to date i have a log file as follows !! *A0628/081 /08-01-10/13 H... (1 Reply)
Discussion started by: nabmufti
1 Replies

10. Shell Programming and Scripting

how to extract paragraphs from file in BASH script followed by prefix ! , !! and !!!

I]hi all i am in confusion since last 2 days :( i posted thraed yesterday and some friends did help but still i couldnt get solution to my problem let it be very clear i have a long log file of alkatel switch and i have to seperate the minor major and critical alarms shown by ! , !! and !!!... (6 Replies)
Discussion started by: nabmufti
6 Replies
Login or Register to Ask a Question