Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Replacing or removing a long list of pattern by using awk or sed

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Replacing or removing a long list of pattern by using awk or sed
# 1  
Old 11-19-2009
Replacing or removing a long list of pattern by using awk or sed
Input:
>abc|123456|def|EXIT|
>abc|203456|def|EXIT2|
>abc|234056|def|EXIT3|
>abc|340056|def|EXIT4|
>abc|456000|def|EXIT5|
.
.
.

Output:
def|EXIT|
def|EXIT2|
def|EXIT3|
def|EXIT4|
def|EXIT5|
.
.

My try code:
Code:
sed 's/>abc\|(\d+)\|//' input_file

Unfortunately, it can't work Smilie
Does anybody got any better idea?
# 2  
Old 11-19-2009
Try this one:

Code:
sed 's/.*\(def.*\)/\1/' file

# 3  
Old 11-19-2009
Python, if you have
Code:
#!/usr/bin/env python
for line in open("file"):
    print '|'.join(line.split("|")[2:]).strip()

output
Code:
# python script.py
def|EXIT|
def|EXIT2|
def|EXIT3|
def|EXIT4|
def|EXIT5|



---------- Post updated at 08:16 AM ---------- Previous update was at 08:16 AM ----------

Python, if you have
Code:
#!/usr/bin/env python
for line in open("file"):
    print '|'.join(line.split("|")[2:]).strip()

output
Code:
# python script.py
def|EXIT|
def|EXIT2|
def|EXIT3|
def|EXIT4|
def|EXIT5|

# 4  
Old 11-19-2009
Code:
nawk -F/abc\|[0-9][0-9][0-9][0-9][0-9][0-9]\|/ ' {print $2} ' infile

# 5  
Old 11-19-2009
Thanks a lot, Franklin52.
Your code is worked perfectly if it is only "def" after the ">abc|number|"
If my input is something like this:
>abc|123456|def|EXIT|
>abc|203456|def|EXIT2|
>abc|234056|def|EXIT3|
>abc|340056|acg|EXIT4|
>abc|456000|hta|EXIT5|

And my output is:
def|EXIT|
def|EXIT2|
def|EXIT3|
acg|EXIT4|
hta|EXIT5|

can I write the code like this:
Code:
cat file | sed 's/.*\(def.*\)/\1/' | sed 's/.*\(acg.*\)/\1/' | sed 's/.*\(hta.*\)/\1/'

Or you got any better suggestion?
# 6  
Old 11-19-2009
If the file you are getting is a fixed format , then you can try

cat file | cut -f 3- -d '|'
# 7  
Old 11-19-2009
what's the pattern of the file? do you want to retain whatever comes after the numbers? will there always be numbers?

assuming there are always numbers in the output and you want to retain everything after that:

with sed:
Code:
mo@mo-laptop:~/scripts$ echo "abc|123456|def|EXIT|" | sed 's/^.*[0-9].//g'
def|EXIT|
mo@mo-laptop:~/scripts$

with awk:
Code:
mo@mo-laptop:~/scripts$ echo "abc|123456|def|EXIT|" | awk 'BEGIN { FS="|" } { OFS="|" }{ print $3, $4}'
def|EXIT
mo@mo-laptop:~/scripts$
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed cannot execute [Argument list too long]

Hi All, This question has been asked many times, but my problem is slightly different. In my shell script i am connecting to oracle database and loading the results to .dat file. This .dat file is later used to create to .xls file Some times the size of .dat file becomes more than 120000... (8 Replies)
Discussion started by: galaxy_rocky
8 Replies

2. Shell Programming and Scripting

Replacing lines matching a multi-line pattern (sed/perl/awk)

Dear Unix Forums, I am hoping you can help me with a pattern matching problem. What am I trying to do? I want to replace multiple lines of a text file (that match a multi-line pattern) with a single line of text. These patterns can span several lines and do not always have the same number of... (10 Replies)
Discussion started by: thefang
10 Replies

3. UNIX for Advanced & Expert Users

Argument list too long w/ sed

Hi all, I am using GNU sed (named gsed under macports) in OSX. I have a directory with a series of files named pool_01.jpg through pool_78802.jpg. I am trying to use this command to rename the files to their checksum + extension. md5sum * | gsed -e 's/\(*\) \(.*\(\..*\)\)$/mv -v \2 \1\3/e' ... (3 Replies)
Discussion started by: openthomas
3 Replies

4. Solaris

Replacing a string in a long list of files

I have a script that needs to read a file with a long list of /path/filenames - replace the name of the server in each file - and write the file to the same path with a date extension. This is the script that I have so far #!/bin/ksh umask 022 LIST=`scripts.list` for i in $LIST do ... (2 Replies)
Discussion started by: bjdamon
2 Replies

5. UNIX for Dummies Questions & Answers

Argument list too long for Sed command

Hi guys Following command results in sed -i 's/#/\\#/g' /home/test/sqlstents* -bash: /bin/sed: Argument list too long Please help me solve it.. is there any other way i can do this?.. thanks (4 Replies)
Discussion started by: depakjan
4 Replies

6. Shell Programming and Scripting

Replacing part of a pattern in sed

Hi I have a piece of xml that has a pattern like this <int>159</int><int>30</int> I want to find this pattern but only substitute the second part of the pattern to {rid1}. Is that possible in sed ? Thanks. ---------- Post updated at 12:10 PM ---------- Previous update was at 12:01 PM... (11 Replies)
Discussion started by: vnn
11 Replies

7. Shell Programming and Scripting

sed - replacing on the right of a pattern and looking for exact word?

how would you get SED to do the following, say you have the following lines in a text file: user=tigger some text some text some text some text some text some text user=ted some text some text some text some text some text some text user=thekingofrockandroll you want to find any line... (15 Replies)
Discussion started by: rich@ardz
15 Replies

8. UNIX for Dummies Questions & Answers

sed problem replacing long strings

Hi all, I have a script which uses sed to replace one string with another. The problem is, the string to be matched, and its replacement are coming in as two command line arguments $1 and $2 $1 and $2 can be absolutely anything, but both should be treated purely as strings. My sed command... (1 Reply)
Discussion started by: mark007
1 Replies

9. Shell Programming and Scripting

Sed and replacing one occurence of pattern

I would like to use sed to replace one occurence of a pattern in a file. When I use the s/// command it replaces all occurences of the pattern in the file. Should I be using something other than sed? Thanks (6 Replies)
Discussion started by: ss9u
6 Replies

10. Shell Programming and Scripting

removing a line containing a pattern in sed

i need to use sed to remove an entire line containing a pattern stored in a variable say $var1 this var1 will be a URL and will therefore contain slashes any help would be greatly appreciated (1 Reply)
Discussion started by: Fire_Storm
1 Replies
Login or Register to Ask a Question