Remove string between two special characters


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove string between two special characters
# 1  
Old 09-06-2011
Remove string between two special characters

Hi All,

I have a variable like
Code:
AVAIL="\
BACK:bkpstg:testdb3.iad.expertcity.com:backtest|\
#AUTH:authstg:testdb3.iad.expertcity.com:authiapd|\
TEST:authstg:testdb3.iad.expertcity.com:authiapd|\
"

What I want to do here is that If a find # before any entry, remove the entire string between # and first | (including # and |). SO the resulting string would be like -
Code:
AVAIL="\
BACK:bkpstg:testdb3.iad.expertcity.com:backtest|\
TEST:authstg:testdb3.iad.expertcity.com:authiapd|\
"

With the help of this forum itself, I found the solution as below -
Code:
echo "${AVAIL}" | sed -e "s/#[^|]*|//g"

But I am not able to exactly understand how above is working. My initial understanding was that #[^|]*| deletes all entries encapsulated between # and first pipe except the ones encapsulated between two pipes but that doesn't seem to explain above transalation. I have played a good amount with sed but this one seems to be tough to understand.

Can anybody please help understand the functioning of above sed trick ?

Thanks

Last edited by Franklin52; 09-06-2011 at 03:04 AM.. Reason: Please use code tags for code and data samples, thank you
# 2  
Old 09-06-2011
Code:
$ echo $AVAIL | nawk -F"[#|]" '{if ($0~/#/) {print $1"|"$4}}'

# 3  
Old 09-06-2011
Computer

Thanks for the nawk trick.

Sorry. I was not much clear earlier. Its not necessary that every entry between pipes will contain only 4 fields seperated by colon (Smilie. We might have more 5 or 6 or any number of fields. So probably $4 won't be a good idea.

Moreover, nawk is not installed on our linux boxes and we don't want to install it extra when sed trick is solving the purpose. We would stick to sed.

Please help me understanding original sed trick I pasted as to how its working ?

Smilie
# 4  
Old 09-06-2011
Maybe this can help (| is special in perl regex and # in regex with \x option, so it's needed to escape them here):
Code:
echo "$AVAIL" | perl -pe 's/
\#       # start your sequence with #
[^|]*    # as many any but "|" chars in the sequence as possible
\|       # finish with |
//xg'

# 5  
Old 09-06-2011
Thanks for the great explaination Yazu.

But I am still confused. Say, $AVAIL is like below -

#AUTH:authstg:testdb3.iad.expertcity.com:authiapd|BACK:authstg:testdb3.iad.expertcity.com:authiapd|

According to your explaination, it will start searching from # and will search as many chars as possible till last pipe (or first pipe ?). If its searching till last pipe, the end result of
Code:
echo "${AVAIL}" | sed -e "s/#[^|]*|//g"

should be (since ^| instructing it to ignore pipe)

|

Please correct me if I wrong in understanding above.....
Smilie

Last edited by Franklin52; 09-06-2011 at 05:24 AM.. Reason: Please use code tags for code and data samples, thank you
# 6  
Old 09-06-2011
Any but "|"
Look at your string, find sharp, then look at all chars but "|" and then find "|" and change this sequence to nothing.
BTW, "Mastering regular expressions" is really a good book.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to remove special characters?

Hi Gurus, I have file which contains some unicode charachator like "ü". I want to replace it with some charactors. I searched in internet and got command sed "s/ü/-/g", but I don't know how to type ü in unix command line. Please help me for this one. Thanks in advance (7 Replies)
Discussion started by: ken6503
7 Replies

2. Shell Programming and Scripting

How to remove some special characters in a string?

Hi, I have string like this ="Lookup Procedure" But i want the output like this Lookup Procedure =," should be removed. Please suggest me the solution. Regards, Madhuri (2 Replies)
Discussion started by: srimadhuri
2 Replies

3. Shell Programming and Scripting

Remove the special characters from field

Hi, In source data few of columns are having special charates(like *) due to this i am not able to display the data into flat file.it's displaying the some of junk data into the flat file. source dataExample: Address1="XDERFTG * HYJUYTG" how to remove the special charates in a string (2 Replies)
Discussion started by: koti_rama
2 Replies

4. Shell Programming and Scripting

remove special characters

hello all I am writing a perl code and i wish to remove the special characters for text. I wish to remove all extended ascii characters. If the list of special characters is huge, how can i do this using substitute command s/specialcharacters/null/g I really want to code like... (3 Replies)
Discussion started by: vasuarjula
3 Replies

5. UNIX for Dummies Questions & Answers

Files with special characters - how to remove

Hi, I have a directory that has a file which contained special characters in the filename. Can someone please advise how to remove the file, preferably with a rm -i ? Thanks in advance. Listing is as below: {oracle}> ls -1b bplog.bkup.001 bplog.bkup.002 bplog.bkup.003 bplog.bkup.004... (1 Reply)
Discussion started by: newbie_01
1 Replies

6. UNIX for Dummies Questions & Answers

How to Remove Special Characters

Dear Members, We have a file which contains some special characters. I need to replace these special character by a new line character(\n). The Special character is \x85. I am not sure what this character means and how we can remove it. Any inputs are greatly appreciated. Thanks... (5 Replies)
Discussion started by: sandeep_1105
5 Replies

7. Shell Programming and Scripting

How to remove special characters from each line?

Hello, Is there a simpler way to remove special characters (color codes) from each lines in a log file? I use sed like in the example below but I think there should be a more simple way to achieve the same result: $ cat -vet file1 ^, , , , Maybe to convert the file somehow? ... (5 Replies)
Discussion started by: majormark
5 Replies

8. Shell Programming and Scripting

Remove special characters from string

Hi there, I'd like to write a script that removes any set of character from any string. The first argument would be the string, the second argument would be the characters to remove. For example: $ myscript "My name's Santiago. What's yours?" "atu" My nme's Snigo. Wh's yors? I wrote the... (11 Replies)
Discussion started by: chebarbudo
11 Replies

9. UNIX for Dummies Questions & Answers

Remove directory that has special Characters

Hi All, I have a script written that creates a new directory within the shell program and if a parameter isn't passed in, it creates a strange directory name by mistake. So I have a directory like "-_12" and I am unable to remove it. I tried removing it using double quote and many others. I have... (12 Replies)
Discussion started by: datherriault
12 Replies

10. UNIX for Dummies Questions & Answers

remove special and unicode characters

Hi, How do I remove the lines where special characters or Unicode characters appear? The following query does work but I wonder if there is a better way. cat test.txt | egrep -v '\)|#|,|&|-|\(|\\|\/|\.' The following lines show that my query is incomplete. Warning: The word "*Khan" is... (1 Reply)
Discussion started by: shantanuo
1 Replies
Login or Register to Ask a Question