Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

delete two patterns and remove one pattern

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting delete two patterns and remove one pattern
# 1  
Old 04-07-2009
delete two patterns and remove one pattern
Friends,

I have .txt file with following format.
START
ABC|Prashant1|Patel1
ABC|Prashant2|Patel2
ABC|Prashant1|Patel1
ABC|Prashant2|Patel2
END

I would like to do:
1) Delete line with START
2) Delete line with END
3) Remove ABC|
4) Delete duplicate records

The following command works fine which deletes line with START and END
sed -e /^START/d -e /^END/d Filename.txt

How do I incorporate task 3 and 4?
NOTE: The file will have more than 500,000 thousand rows.

Thanks in advance for suggestion,
Prashant
# 2  
Old 04-07-2009
Quote:
Originally Posted by ppat7046
Friends,

I have .txt file with following format.
START
ABC|Prashant1|Patel1
ABC|Prashant2|Patel2
ABC|Prashant1|Patel1
ABC|Prashant2|Patel2
END



I would like to do:
1) Delete line with START
2) Delete line with END
3) Remove ABC|
4) Delete duplicate records

The following command works fine which deletes line with START and END
sed -e /^START/d -e /^END/d Filename.txt

How do I incorporate task 3 and 4?
NOTE: The file will have more than 500,000 thousand rows.

Thanks in advance for suggestion,
Prashant

Code:
nawk -F'|'  'NF==3{print $2,$3}' patel

# 3  
Old 04-07-2009
Quote:
Originally Posted by zenith
Code:
nawk -F'|'  'NF==3{print $2,$3}' patel

This code is not doing uniq + removes pipe delimiter

use this

Code:
grep "|" test | sed 's/^ABC|//g' | sort -u

# 4  
Old 04-07-2009
Quote:
Originally Posted by tostay2003
This code is not doing uniq + removes pipe delimiter

use this

Code:
grep "|" test | sed 's/^ABC|//g' | sort -u


Tostay2003:
What if the data doesnt start with ABC your logic fails

use this
nawk -F'|' 'NF==3{print $2,$3}' patel | sort -u
# 5  
Old 04-07-2009
Quote:
Originally Posted by zenith
Tostay2003:
What if the data doesnt start with ABC your logic fails

use this
nawk -F'|' 'NF==3{print $2,$3}' patel | sort -u
I assumed from the authors description that the first field remains common in the file

Quote:
START
ABC|Prashant1|Patel1
ABC|Prashant2|Patel2
ABC|Prashant1|Patel1
ABC|Prashant2|Patel2
END
...

3) Remove ABC|
Use this if you didnt mean that the first field woudl be same.
Code:
grep "|" test | cut -d'|' -f2,3 | sort -u

or with slight amendment to code written by zenith i.e. by adding OFS
# 6  
Old 04-07-2009
Code:
nawk -F'|' 'NF==3 && !a[$2,$3]++ {print $2,$3}' patel

# 7  
Old 04-08-2009
Thank you all for your reply.

Code:
nawk -F'|' 'NF==3 && !a[$2,$3]++ {print $2,$3}' patel

I had made small change in print statment because it was not printing | symbol.
Code:
nawk -F'|' 'NF==3 && !a[$2,$3]++ {print $2,"|",$3}' patel

However, it prints the space after and before | symbol.

Thanks,
Prashant
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to delete all lines before a particular pattern when the pattern is defined in a variable?

I have a file Line 1 a Line 22 Line 33 Line 1 b Line 22 Line 1 c Line 4 Line 5 I want to delete all lines before last occurrence of a line which contains something which is defined in a variable. Say a variable var contains 'Line 1', then I need the following in the output. ... (21 Replies)
Discussion started by: Soham
21 Replies

2. Shell Programming and Scripting

Delete patterns matching

Delete patterns matching OS version: RHEL 7.3 Shell : Bash I have a file like below (pattern.txt). I need to delete all lines starting with the following words (words separated by comma below) and ) character. LOGGING, NOCOMPRESS, TABLESPACE , PCTFREE, INITRANS, MAXTRANS, STORAGE,... (3 Replies)
Discussion started by: John K
3 Replies

3. Shell Programming and Scripting

Delete values between 2 patterns

Hi, How can i delete values between 2 patterns like below:- input.txt 192.1.1.2.22 blablabala 23.1.A.1.2 blablabalbl 5.4.1.1.12 blablaba i need to delete all values between starting from "." no 3 and second column. the output should be: 192.1.1 blablabala... (15 Replies)
Discussion started by: redse171
15 Replies

4. Shell Programming and Scripting

Delete lines and the first pattern between 2 matched patterns

Hi, i need help to delete all the lines between 2 matched patterns and the first pattern must be deleted too. sample as follows: inputfile.txt >kump_1 ........................... ........................... >start_0124 dgfhghgfh fgfdgfh fdgfdh >kump_2 ............................. (7 Replies)
Discussion started by: redse171
7 Replies

5. Shell Programming and Scripting

awk delete/remove rest of line on multiple search pattern

Need to remove rest of line after the equals sign on search pattern from the searchfile. Can anybody help. Couldn't find any similar example in the forum: infile: 64_1535: Delm. = 86 var, aaga 64_1535: Fran. = 57 ex. ccc 64_1639: Feb. = 26 (link). def 64_1817: mar. = 3/4. drz ... (7 Replies)
Discussion started by: sdf
7 Replies

6. Shell Programming and Scripting

sed pattern to delete lines containing a pattern, except the first occurance

Hello sed gurus. I am using ksh on Sun and have a file created by concatenating several other files. All files contain header rows. I just need to keep the first occurrence and remove all other header rows. header for file 1111 2222 3333 header for file 1111 2222 3333 header for file... (8 Replies)
Discussion started by: gary_w
8 Replies

7. Shell Programming and Scripting

Perl to remove words between patterns

Hi, I am using following code to remove words between start and end points. $mystring = "The start text always precedes the end of the end text."; if($mystring =~ s/start(.*)end/\0/) { print $1; print "\n"; print $mystring; } But this is writing special chars in place of... (1 Reply)
Discussion started by: sarbjit
1 Replies

8. Shell Programming and Scripting

delete lines between patterns

Hi, I've searched in this forum all day long but was not able to find enough codes to help me do a task. The only code that I can come up with is this: sed '/ /,/ /{//p;d;}' inputfile > outputfile I would like to sed/awk/grep a file for two patterns and then delete the lines between... (4 Replies)
Discussion started by: shamushamu
4 Replies

9. Shell Programming and Scripting

Delete lines between two patterns without deleting the second pattern

I want to delete lines like this sed '/FROM_HERE/,/TO_HERE/d' but I would like to *not* delete the second match, i.e. the TO_HERE line. How can I achieve this? Thank you! (1 Reply)
Discussion started by: Ilja
1 Replies

10. Shell Programming and Scripting

comment/delete a particular pattern starting from second line of the matching pattern

Hi, I have file 1.txt with following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433 ** ** ** In file 2.txt I have the following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433... (4 Replies)
Discussion started by: imas
4 Replies
Login or Register to Ask a Question