Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to search multiple patterns and remove lines from a file?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to search multiple patterns and remove lines from a file?
# 1  
Old 07-31-2013
How to search multiple patterns and remove lines from a file?
Hi,

I have a file content as below.

Code:
Table  : PAYR
Displayed fields:  15 of  15  Fixed columns:                4  List width 0999
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| |HBKID  |HKTID  |RZAWE|CHECT        |LIFNR     |VBLNR     |ZALDT     |RWBTR             |PRIDT     |BANCD     |ZNME1                              |ZPSTL     |ZORT1                              |VOIDR|VOIDD     |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| |CHASD  |DISB   |C    |1000471443   |0010190595|2000047407|10/02/2006|            41.37-|10/02/2006|10/12/2006|SOUTHWESTERN BELL TELEPHONE CO L P |75263-0047|DALLAS                             |00   |00/00/0000|
| |CHASD  |DISB   |C    |1000471444   |0010190604|2000047408|10/02/2006|           193.54-|10/02/2006|10/12/2006|SOUTHWESTERN BELL TELEPHONE CO L P |75265-0661|DALLAS                             |00   |00/00/0000|
Table  : PAYR
Displayed fields:  15 of  15  Fixed columns:                4  List width 0999
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| |HBKID  |HKTID  |RZAWE|CHECT        |LIFNR     |VBLNR     |ZALDT     |RWBTR             |PRIDT     |BANCD     |ZNME1                              |ZPSTL     |ZORT1                              |VOIDR|VOIDD     |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Where ever the header lines (lines start with Table, Displayed fieleds,| |HBKID,--- ) present , I have to remove these lines from the file to process the file further steps. Pls help me.

Moderator's Comments:
Mod Comment Use code tags please, see PM.
Last edited by zaxxon; 07-31-2013 at 06:27 AM.. Reason: code tags
# 2  
Old 07-31-2013
Use Code Tags

Code:
awk '!/^Table|^Displayed|^----/' filename

You can add more separators using |^xxxxx
# 3  
Old 07-31-2013
Hello,

Could you please try the following code and let me know.



Code:
$ cat check_lines.ksh
while read line
do
value=`echo $line | grep -v "^| |HBKID"`
echo $value

done < "check_lines_data"


Output should be as follows.


Code:
$ ksh check_lines.ksh
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| |CHASD |DISB |C |1000471443 |0010190595|2000047407|10/02/2006| 41.37-|10/02/2006|10/12/2006|SOUTHWESTERN BELL TELEPHONE CO L P |75263-0047|DALLAS |00 |00/00/0000|
| |CHASD |DISB |C |1000471444 |0010190604|2000047408|10/02/2006| 193.54-|10/02/2006|10/12/2006|SOUTHWESTERN BELL TELEPHONE CO L P |75265-0661|DALLAS |00 |00/00/0000|
Table : PAYR
Displayed fields: 15 of 15 Fixed columns: 4 List width 0999
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------





Thanks,
R. Singh
# 4  
Old 07-31-2013
Hi, Thanks for the help Smilie
but this is writing the result to standard output. But I want the result to be in the same file , where we are removing the lines.
suppose we are removing the lines from file.txt, after removign the lines rest of the data I want to be in the same file.txt.

Can you pls help?
# 5  
Old 07-31-2013
Code:
awk '!/^Table|^Displayed|^----/' file.txt > temp.txt ; mv temp.txt file.txt

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Delete multiple lines between blank lines containing two patterns

Hi all, I'm looking for a way (sed or awk) to delete multiple lines between blank lines containing two patterns ex: user: alpha parameter_1 = 15 parameter_2 = 1 parameter_3 = 0 user: alpha parameter_1 = 15 parameter_2 = 1 parameter_3 = 0 user: alpha parameter_1 = 16... (3 Replies)
Discussion started by: ce9888
3 Replies

2. Shell Programming and Scripting

Search for patterns on different lines

im using the following code to search a log for entries on two different lines: awk 'BEGIN{count=0} /'"${firstpattern}"'/,/'"${secondpattern}"'/ { print; if ($0 ~ /'"${thirdpattern}"'/){count++}; } END { print count }' data.txt firstpattern="start error log" secondpattern="i am logging the... (1 Reply)
Discussion started by: SkySmart
1 Replies

3. Shell Programming and Scripting

reading lines from a file between two search patterns

Hi, I am new to shell scripting and is working on a script to extract lines from a log file between two time stamps using awk command. After some research I used following command: awk '/01 Oct 2011/{p=1} /10 Oct 2011/{p=0} p' test.log >> tmp.log This works fine. But now i want to... (3 Replies)
Discussion started by: davidtd
3 Replies

4. Shell Programming and Scripting

search multiple patterns

I have two lists in a file that look like a b b a e f c d f e d c I would like a final list a b c d e f I've tried multiple grep and awk but can't get it to work (8 Replies)
Discussion started by: godzilla07
8 Replies

5. Shell Programming and Scripting

Search multiple patterns in multiple files

Hi, I have to write one script that has to search a list of numbers in certain zipped files. For eg. one file file1.txt contains the numbers. File1.txt contains 5,00,000 numbers and I have to search each number in zipped files(The number of zipped files are around 1000 each file is 5 MB) I have... (10 Replies)
Discussion started by: vsachan
10 Replies

6. Shell Programming and Scripting

Remove multiple lines in the file

Hi, How to remove duplicate lines in the file. a ------ b c 24 23 a ---- c b (4 Replies)
Discussion started by: sandy1028
4 Replies

7. Shell Programming and Scripting

Search and Remove Lines within File

Hello, I've searched through the scripting section but could not find what I need. I need to search for empty sections within a file and remove them. Here is an example file: Title 123 A B C D E Title 098 Title 567 Z Y (4 Replies)
Discussion started by: leepet01
4 Replies

8. Shell Programming and Scripting

Find multiple patterns on multiple lines and concatenate output

I'm trying to parse COBOL code to combine variables into one string. I have two variable names that get literals moved into them and I'd like to use sed, awk, or similar to find these lines and combine the variables into the final component. These variable names are always VAR1 and VAR2. For... (8 Replies)
Discussion started by: wilg0005
8 Replies

9. Shell Programming and Scripting

Perl - How to search a text file with multiple patterns?

Good day, great gurus, I'm new to Perl, and programming in general. I'm trying to retrieve a column of data from my text file which spans a non-specific number of lines. So I did a regexp that will pick out the columns. However,my pattern would vary. I tried using a foreach loop unsuccessfully.... (2 Replies)
Discussion started by: Sp3ck
2 Replies

10. UNIX for Dummies Questions & Answers

How to parameterize multiple search patterns and generate a new file

I have one file: 123*100*abcd*10 123*101*abcd*-29*def 123*100*abcd*-10 123*102*abcd*-105*asd I would like to parameterize the search patterns in the following way so that the user could dynamically change the search pattern. *100* and *- (ie *minus) *102* and *- The output that is... (6 Replies)
Discussion started by: augustinep
6 Replies
Login or Register to Ask a Question