How to remove special characters from each line?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to remove special characters from each line?
# 1  
Old 07-14-2009
How to remove special characters from each line?

Hello,

Is there a simpler way to remove special characters (color codes) from each lines in a log file?

I use sed like in the example below but I think there should be a more simple way to achieve the same result:

Code:
$ cat -vet file1
^[[36m20014:35:14.20232 90], [ 80|42^[[0m$
^[[34m200:35:14. 87], [ 80| ^[[0m$

$ cat -vet file1 | sed s/^\\^\\[\\[..m//g | sed s/\\^\\[\\[0m\\$//g
20014:35:14.20232 90], [ 80|42
200:35:14. 87], [ 80| 

The idea is to remove the begin-end pattern: "^[[<NR>m ..... ^[[<NR>m$"
This characters only show when using the option with cat, otherwise the text appears in color.

Maybe to convert the file somehow?

Thanks.
# 2  
Old 07-14-2009
You can pipe through tr -dc '[[:print:]]' if character classes are supported. Something similar will work with sed.

You can give this a try:
Code:
sed -i 's/[^[:print:]]/g'

# 3  
Old 07-15-2009
^ It does not work with "sed -i .." and not even with /usr/xpg4/bin/sed because I'm on a Solaris machine.
# 4  
Old 07-15-2009
If I understand you correct, you want to get rid of the ESCape char.
ie:- HEX value "1B"

Then try this.
tr -d '\x1B' < file1 > file2
# 5  
Old 07-15-2009
Try using perl:


Code:
open HNDL, "<$ARGV[0]" or die "Cannot open the file";
while(read HNDL, $ch,1)
        {
        if((ord($ch) >=32) && (ord($ch) <=127)) { print $ch; }
        }
close HNDL;

# 6  
Old 07-16-2009
@dennis.jacob, edidataguy :

Code:
I need to remove the begin-end pattern: "^[[<NR>m ..... ^[[<NR>m$" for each line in the file not the ESCape char.

Of course ^[[... are special formatting characters which do not show normally when using "more".
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove Special Characters Within Text

Hi, I have a "|" delimited file that is exported from a database. There is one column in the file which has description/comments entered by some application user. It has "Control-M" character and "New Line" character in between the text. Hence, when i export the data, this record with the new... (4 Replies)
Discussion started by: tarun.trehan
4 Replies

2. Shell Programming and Scripting

How to remove special characters?

Hi Gurus, I have file which contains some unicode charachator like "ü". I want to replace it with some charactors. I searched in internet and got command sed "s/ü/-/g", but I don't know how to type ü in unix command line. Please help me for this one. Thanks in advance (7 Replies)
Discussion started by: ken6503
7 Replies

3. Shell Programming and Scripting

How to remove some special characters in a string?

Hi, I have string like this ="Lookup Procedure" But i want the output like this Lookup Procedure =," should be removed. Please suggest me the solution. Regards, Madhuri (2 Replies)
Discussion started by: srimadhuri
2 Replies

4. Shell Programming and Scripting

Sed - remove special characters

Hi, I have a file with this line, it's always in the first line: I want to remove these special characters: ´╗┐ file1 ´╗┐\\bar\c$\test2\;3.348.118 Bytes;160 ;3 \\bar\c$\test\;35 Bytes;2 ;1 I want the same file to be only \\bar\c$\test2\;3.348.118 Bytes;160 ;3 \\bar\c$\test\;35... (4 Replies)
Discussion started by: nakaedu
4 Replies

5. Shell Programming and Scripting

Remove the special characters from field

Hi, In source data few of columns are having special charates(like *) due to this i am not able to display the data into flat file.it's displaying the some of junk data into the flat file. source dataExample: Address1="XDERFTG * HYJUYTG" how to remove the special charates in a string (2 Replies)
Discussion started by: koti_rama
2 Replies

6. Shell Programming and Scripting

Remove string between two special characters

Hi All, I have a variable like AVAIL="\ BACK:bkpstg:testdb3.iad.expertcity.com:backtest|\ #AUTH:authstg:testdb3.iad.expertcity.com:authiapd|\ TEST:authstg:testdb3.iad.expertcity.com:authiapd|\ " What I want to do here is that If a find # before any entry, remove the entire string... (5 Replies)
Discussion started by: engineermayur
5 Replies

7. Shell Programming and Scripting

remove special characters

hello all I am writing a perl code and i wish to remove the special characters for text. I wish to remove all extended ascii characters. If the list of special characters is huge, how can i do this using substitute command s/specialcharacters/null/g I really want to code like... (3 Replies)
Discussion started by: vasuarjula
3 Replies

8. UNIX for Dummies Questions & Answers

How to Remove Special Characters

Dear Members, We have a file which contains some special characters. I need to replace these special character by a new line character(\n). The Special character is \x85. I am not sure what this character means and how we can remove it. Any inputs are greatly appreciated. Thanks... (5 Replies)
Discussion started by: sandeep_1105
5 Replies

9. Shell Programming and Scripting

Remove special characters from string

Hi there, I'd like to write a script that removes any set of character from any string. The first argument would be the string, the second argument would be the characters to remove. For example: $ myscript "My name's Santiago. What's yours?" "atu" My nme's Snigo. Wh's yors? I wrote the... (11 Replies)
Discussion started by: chebarbudo
11 Replies

10. UNIX for Dummies Questions & Answers

Remove directory that has special Characters

Hi All, I have a script written that creates a new directory within the shell program and if a parameter isn't passed in, it creates a strange directory name by mistake. So I have a directory like "-_12" and I am unable to remove it. I tried removing it using double quote and many others. I have... (12 Replies)
Discussion started by: datherriault
12 Replies
Login or Register to Ask a Question