Hidden control characters in a Unix Text File!

Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Hidden control characters in a Unix Text File!
# 1  
Old 05-29-2002
Question Hidden control characters in a Unix Text File!

Can anyone seem to know how to find out whether a UNIX text file has 'hidden' control characters?

Can I view them using 'vi' by some command line options?

If there are control characters in a text file which are invisible/hidden.. then how do I get rid of them?

Your intelletual answers are greatly appreciated.

Thanks in anticipation.
# 2  
Old 05-29-2002
try the :set all in command mode in vi. It will list all the available options.

See what that tells you. I think that the :I will show (using the letter "I") newline and various special characters.

Hope this helps.
# 3  
Old 05-30-2002
vi does not work well with binary data. I suggest that you stay away from vi for this issue. To display the data you can use either "cat -v" or "od -c". Once you know what you're dealing with, "tr -d" can delete unwanted bytes.
# 4  
Old 05-30-2002
Surely VI is okay for text file though?

Using ":set list" in vi will tell you any tab's (^I) and show whitespace by giving you the eol character '$'

Don't know how good this is for other 'special characters' etc.
# 5  
Old 09-16-2008
Don't know if this helps, but python can give you a quick view of the control characters in the file.
python -c "f = open('filename.txt', 'r'); f.seek(0); file = f.readlines(); print file"
Just replace filename.txt with your filename.
# 6  
Old 09-30-2008
MySQL how to remove a special char and whitespace char in a file

Hi pals,

Could you guys clear my doubts?..

I want to remove all white space chars, ctrl key chars, arrow key chars, Esc key char and del key chars...

for ex..

Get the input value[ while typing use all arrow keys/del keys/esc key]
and store it one file. for example you can get this input value as Hello^[[AHow a^[[D]re you^[ "

but, I wanna the o/p will be

Hello How are you?

plz, solve my query guys

Thnx in advance
# 7  
Old 09-30-2008
How do you know the A before How is not part of the input?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to strip control-A characters from a column in a file

Hi All, I currently have flat file with 32 columns. The field delimiter is cntl-A ( \x01). The file has been extracted from an oracle table using a datastage job. However, in the 6th field, the data contains additional control -A characters which came as a part of the table data. I need... (12 Replies)
Discussion started by: harsha1238
12 Replies

2. Red Hat

Special control characters in file

Hi Guys, We receive some huge files on to Linux server. Source system use FTP mechanism to transfer these files on our server. Occasionally one record is getting corrupted while transfer, some control characters are injecting into the file. How to fix this issue ? please advice ? Sample... (2 Replies)
Discussion started by: srikanth38
2 Replies

3. Shell Programming and Scripting

How to view the control characters in a file?

Hello, How can I view control and special characters of a text file?. For example, space, tabs, new line chars etc. Can I use hexdump for it? Thanks (3 Replies)
Discussion started by: reddyr
3 Replies

4. UNIX for Dummies Questions & Answers

Control characters in UNIX

Hi, My files are showing some control characters in vi editor ^M ^@ and somtimes ^H I removed ^M with %s/^M//g command but how to represent ^@ and ^H e.g. for ^M it is hold ctrl then v and m.. Please help.. I am very new to unix.. (7 Replies)
Discussion started by: prabhat.diwaker
7 Replies

5. Shell Programming and Scripting

Request for advise on how to remove control characters in a UNIX file extracted from top command

Hi, Please excuse for posting new thread on control characters, I am facing some difficulties in removing the control character from a file extracted from top command, i am able to see control characters using more command and in vi mode, through cat control characters are not visible ... (8 Replies)
Discussion started by: karthikram
8 Replies

6. Shell Programming and Scripting

Removing "Hidden Characters" on a file

Hi - I'm having a problem with hidden characters on Linux. When I produced an output from Oracle database, there is a an extra "Hidden Character" included on the output. How can I remove that character? See below: The extra dollar sign is creating a new line on my .csv output file. I... (16 Replies)
Discussion started by: Jin_
16 Replies

7. Shell Programming and Scripting

display all possible control characters from .xml file in unix

Hi, I have a .xml file in unix. We are passing this file through a xml parser. But we are getting some control characters from input file and XML parser is failing for the control character in file.Now I am getting following error, Error at byte 243206625 of file filename_$.xml: Error... (1 Reply)
Discussion started by: fantushmayu
1 Replies

8. UNIX for Advanced & Expert Users

Hidden Characters

Hello all, I have two files, that I suspect may contain hidden characters (EG, three spaces instead of a tab). Does anyone know of any tool that can display this (I have tried using diff, but I'm not quite sure it would do the job) (6 Replies)
Discussion started by: Khoomfire
6 Replies

9. AIX

Randomly appearing control characters in text files

Hi, From some time, we have noticed that our ascii files have started corrupting due to the presence of some random control characters (^@, ^M, ^H, ^D). The characters appear randomly on any file after the process that creates the file finishes. If we rerun the process, the files re creates... (0 Replies)
Discussion started by: aakashahuja
0 Replies

10. Programming

Identifying and removing control characters in a file.

What is the best method to identify an remove control characters in a file. Would it be easier to do this in Unix or in C. (0 Replies)
Discussion started by: oracle8
0 Replies
Login or Register to Ask a Question