Remove special characters from text file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove special characters from text file
# 1  
Old 12-07-2009
Remove special characters from text file

Hi All,

i am trying to remove all special charecters().,/\~!@#%^$*&^_- and others from a tab delimited file.

I am using the following code.

Code:
while read LINE
do
echo $LINE | tr -d '=;:`"<>,./?!@#$%^&(){}[]'|tr -d "-"|tr -d "'" | tr -d "_" 
done < trial.txt > output.txt

Problem

1.The output file is not tab delimited. space is reduced. how do i correct it.
2.when I include ( -, _) in the first pattern it gives me error/weird results ( can you explain why.( let me know if you need output)

Thanks you
# 2  
Old 12-07-2009
Try:
Code:
tr -d '[:punct:]'

# 3  
Old 12-07-2009
If I interpret you correctly, you want to remove all characters except A-Z (any case) and 0-9, and preserve any whitespaces, right? If so:
Code:
perl -pe 's/[^A-Za-z0-9\s]//g' trial.txt > output.txt

# 4  
Old 12-07-2009
What about
Code:
tr -cd '[:alnum:]\ \t' < file > newfile

# 5  
Old 12-07-2009
Quote:
Originally Posted by kkb
Code:
while read LINE
do
echo $LINE | tr -d '=;:`"<>,./?!@#$%^&(){}[]'|tr -d "-"|tr -d "'" | tr -d "_" 
done < trial.txt > output.txt

1.The output file is not tab delimited. space is reduced. how do i correct it.
Code:
echo "$LINE" | tr ....

Quote:
2.when I include ( -, _) in the first pattern it gives me error/weird results ( can you explain why.( let me know if you need output)
Maybe it works now ?
# 6  
Old 12-07-2009
Quote:
Originally Posted by danmero
What about
Code:
tr -cd '[:alnum:]\ \t' < file > newfile


only one problem we need to use a newline character some where so that the output is in the same form as input. I was unable to fix it
# 7  
Old 12-07-2009
What about
Code:
tr -d '[:punct:]' < trial.txt > output.txt

Where does that differ from your desired result?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

To remove any invisible and special characters from the file(exclude @#!$*)

Hi Guys, My requirement is to remove any invisible and special characters from the file like control M(carriage return) and alt numerics and it should not replace @#!$% abc|xyz|acd¥£ó adc|123| 12áí Please help on this. Thanks Rakesh (1 Reply)
Discussion started by: rakeshp
1 Replies

2. UNIX for Beginners Questions & Answers

To remove any invisible and special characters from the file(exclude @!#$&*)

Hi Guys, My requirement is to remove any invisible and special characters from the file like control M(carriage return) and alt numerics and it should not replace @#!$% abc|xyz|acd¥£ó adc|123| 12áí Please help on this. Thanks Rakesh (1 Reply)
Discussion started by: rakeshp
1 Replies

3. UNIX for Dummies Questions & Answers

How to enter special characters in a text file using vi?

Hi, I need to create a test text file with the special characters \342\200\223 in it and to be able to use sed maybe to delete them I tried doing it using vi by pressing CTRL-V and then typing 342 but it does not work. After pressing CTRL-V and typing 342 it seems to just insert the numbers... (1 Reply)
Discussion started by: newbie_01
1 Replies

4. Shell Programming and Scripting

Remove Special Characters Within Text

Hi, I have a "|" delimited file that is exported from a database. There is one column in the file which has description/comments entered by some application user. It has "Control-M" character and "New Line" character in between the text. Hence, when i export the data, this record with the new... (4 Replies)
Discussion started by: tarun.trehan
4 Replies

5. UNIX for Advanced & Expert Users

What my puzzle file!, How to remove special characters ??

My application generate file but it have special characters in these file. I would like to clear special characters by vi editor and not use cat /dev/null > to_file I try to remove characters manually, but I'm can not! root@MyHost /tmp> ls -l puzzle.txt -rw-r--r-- 1 root system ... (5 Replies)
Discussion started by: arm_naja
5 Replies

6. Shell Programming and Scripting

remove the special characters and move the file into another server

(5 Replies)
Discussion started by: number10
5 Replies

7. Shell Programming and Scripting

Read file and remove special characters or strings

Hello all I am getting data like col1 | col2 | col3 asdafa | asdfasfa | asf*&^sgê 345./ |sdfasd23425^%^&^ | sdfsa23 êsfsfd | sf(* | sdfsasf My requirement is like I have to to read the file and remove all special characters and hex characters ranging form 00-1f from 1st column, remove %"'... (1 Reply)
Discussion started by: vasuarjula
1 Replies

8. Solaris

How to remove a directory or file with special characters in Solaris

I finally figured out how to remove a file or directory with special characters in the name. It's kind of rudimentary so I thought I would share it with everyone: find .inum -exec rm -rf {} \; (7 Replies)
Discussion started by: jastanle84
7 Replies

9. Shell Programming and Scripting

Adding text to file on certain lines with(special characters)

I need to add "new lines" of text with special characters, to specific lines in the file. There are 3 modifications needed. Been testing 2 here without success. #!/usr/bin/perl use FileHandle; $file=FileHandle->new; $FILENAME="/opt/etc/usr/file.txt"; $file->open ("<$FILENAME") or die... (13 Replies)
Discussion started by: A4ron4perl
13 Replies

10. Shell Programming and Scripting

remove special characters from text using PERL

Hi, I am stuck with a problem here. Suppose i have a variable which is assigned some string containing special charatcers. for eg: $a="abcdef^bbwk#kdbcd@"; I have to remove the special characters using Perl. The text is assigned to the variable implicitly. How to do it? (1 Reply)
Discussion started by: agarwal
1 Replies
Login or Register to Ask a Question