What my puzzle file!, How to remove special characters ??


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users What my puzzle file!, How to remove special characters ??
# 1  
Old 03-28-2013
What my puzzle file!, How to remove special characters ??

My application generate file but it have special characters in these file.
I would like to clear special characters by vi editor and not use
Code:
cat /dev/null > to_file

I try to remove characters manually, but I'm can not!

root@MyHost /tmp> ls -l puzzle.txt
Code:
-rw-r--r--    1 root     system            3 Mar 28 10:37 puzzle.txt
root@MyHost /tmp> 
root@MyHost /tmp> cat -v puzzle.txt
๏ปฟroot@MyHost /tmp>

-- After remove it by vi (I press x for 3 times because it have 3 char right?) --
Code:
root@MyHost /tmp> wc puzzle.txt
       1       0       1 puzzle.txt
root@MyHost /tmp> cat -v puzzle.txt

root@MyHost /tmp> ls -l puzzle.txt
-rw-r--r--    1 root     system            1 Mar 28 10:55 puzzle.txt
root@MyHost /tmp>

How I remove special chars???
you can try from attachement [NO Virus]

Last edited by vbe; 04-08-2013 at 09:32 AM..
# 2  
Old 03-28-2013
You could try to remove all special characters like this:
Code:
LANG=C tr -d '\200-\377' < puzzle.txt > puzzle.gone


Last edited by Scrutinizer; 04-05-2013 at 03:02 AM..
This User Gave Thanks to Scrutinizer For This Post:
# 3  
Old 03-28-2013
Quote:
Originally Posted by Scrutinizer
You could try to remove all special characters like this:
Code:
LANG=C tr -d '[\200-\377]' < puzzle.txt > puzzle.gone


Oh! Thank you Scrutinizer,

But It will be effect with non-english alphabet too.
# 4  
Old 03-28-2013
I couldn't download puzzle.txt so cannot be 100% sure. But the other proposed solution did NOT seem to work when I tried it on a small test file I made. I think this will get rid of all special characters, and adjust to your locale:
Code:
tr -cd '[:print:]\n' < puzzle.txt

# 5  
Old 04-05-2013
Quote:
Originally Posted by arm_naja
-- After remove it by vi (I press x for 3 times because it have 3 char right?) --
Code:
root@MyHost /tmp> wc puzzle.txt
       1       0       1 puzzle.txt
root@MyHost /tmp> cat -v puzzle.txt

root@MyHost /tmp> ls -l puzzle.txt
-rw-r--r--    1 root     system            1 Mar 28 10:55 puzzle.txt
root@MyHost /tmp>

How I remove special chars???
The short answer is: you can't The single char "ls" shows is the End-of-File-character (literally a "^D") and it isn't possible with "vi" to delete it. In fact "vi" will even append such a EOF char to a file if it was missing.

Generate a file with "touch", it will have 0 characters. Now open this in "vi", write some text, delete it completely (this way "vi" thinks you have changed the file, do NOT use the undo-function) and save the file. You will notice that it has also 1 character in it - the EOF char.

I hope this helps.

bakunin
# 6  
Old 04-05-2013
Quote:
Originally Posted by bakunin
The short answer is: you can't The single char "ls" shows is the End-of-File-character (literally a "^D") and it isn't possible with "vi" to delete it. In fact "vi" will even append such a EOF char to a file if it was missing.

Generate a file with "touch", it will have 0 characters. Now open this in "vi", write some text, delete it completely (this way "vi" thinks you have changed the file, do NOT use the undo-function) and save the file. You will notice that it has also 1 character in it - the EOF char.

I hope this helps.

bakunin
Sorry, but no. UNIX text files do not have an End-of-File character. Each line in a text file is terminated by a <newline> character. If you have a file open in vi and issue the commands:
Code:
:1,$d
:w file

(which deletes all lines in the file), the size of file will be 0 bytes.

If you have exactly one line in a file and you edit it with vi and delete all of the characters on the line by repeatedly executing the x command until the line is empty and then issue the command:
:w file
then the size of file will be 1 byte because you didn't delete the line, you just deleted the characters on the line preceding the terminating <newline> character.

The current line (including the terminating <newline> character can also be deleted in vi with the dd command.
These 2 Users Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

To remove any invisible and special characters from the file(exclude @#!$*)

Hi Guys, My requirement is to remove any invisible and special characters from the file like control M(carriage return) and alt numerics and it should not replace @#!$% abc|xyz|acd¥£ó adc|123| 12áí Please help on this. Thanks Rakesh (1 Reply)
Discussion started by: rakeshp
1 Replies

2. UNIX for Beginners Questions & Answers

To remove any invisible and special characters from the file(exclude @!#$&*)

Hi Guys, My requirement is to remove any invisible and special characters from the file like control M(carriage return) and alt numerics and it should not replace @#!$% abc|xyz|acd¥£ó adc|123| 12áí Please help on this. Thanks Rakesh (1 Reply)
Discussion started by: rakeshp
1 Replies

3. Shell Programming and Scripting

How to remove alphabets/special characters/space in the 5th field of a tab delimited file?

Thank you for 4 looking this post. We have a tab delimited file where we are facing problem in a lot of funny character. I have tried using awk but failed that is not working. In the 5th field ID which is supposed to be a integer only of that file, we are getting corrupted data as below. I... (12 Replies)
Discussion started by: Srithar
12 Replies

4. Shell Programming and Scripting

How to remove special characters?

Hi Gurus, I have file which contains some unicode charachator like "ü". I want to replace it with some charactors. I searched in internet and got command sed "s/ü/-/g", but I don't know how to type ü in unix command line. Please help me for this one. Thanks in advance (7 Replies)
Discussion started by: ken6503
7 Replies

5. Shell Programming and Scripting

remove the special characters and move the file into another server

(5 Replies)
Discussion started by: number10
5 Replies

6. Shell Programming and Scripting

Read file and remove special characters or strings

Hello all I am getting data like col1 | col2 | col3 asdafa | asdfasfa | asf*&^sgê 345./ |sdfasd23425^%^&^ | sdfsa23 êsfsfd | sf(* | sdfsasf My requirement is like I have to to read the file and remove all special characters and hex characters ranging form 00-1f from 1st column, remove %"'... (1 Reply)
Discussion started by: vasuarjula
1 Replies

7. Solaris

How to remove a directory or file with special characters in Solaris

I finally figured out how to remove a file or directory with special characters in the name. It's kind of rudimentary so I thought I would share it with everyone: find .inum -exec rm -rf {} \; (7 Replies)
Discussion started by: jastanle84
7 Replies

8. Shell Programming and Scripting

remove special characters

hello all I am writing a perl code and i wish to remove the special characters for text. I wish to remove all extended ascii characters. If the list of special characters is huge, how can i do this using substitute command s/specialcharacters/null/g I really want to code like... (3 Replies)
Discussion started by: vasuarjula
3 Replies

9. Shell Programming and Scripting

Remove special characters from text file

Hi All, i am trying to remove all special charecters().,/\~!@#%^$*&^_- and others from a tab delimited file. I am using the following code. while read LINE do echo $LINE | tr -d '=;:`"<>,./?!@#$%^&(){}'|tr -d "-"|tr -d "'" | tr -d "_" done < trial.txt > output.txt Problem ... (10 Replies)
Discussion started by: kkb
10 Replies

10. UNIX for Dummies Questions & Answers

How to Remove Special Characters

Dear Members, We have a file which contains some special characters. I need to replace these special character by a new line character(\n). The Special character is \x85. I am not sure what this character means and how we can remove it. Any inputs are greatly appreciated. Thanks... (5 Replies)
Discussion started by: sandeep_1105
5 Replies
Login or Register to Ask a Question