Removing the ^M control character


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Removing the ^M control character
# 1  
Old 07-21-2014
Removing the ^M control character

I've got a file where each line is separated by ^M characters. I want to be able to cat the file without those lines. When I cat the file now what I see are blank lines. However, the blank lines are actually ^M characters; when I open the file with vi they show up.

Code:
X38888
No
No
No
X38888

X39999
No
No
No
X39999

This is what it looks like in vi:
Code:
^M
X38888
No
No
No
X38888
^M
X39999
No
No
No
X39999

I have tried cat file |sed '/^M/d' to remove them and also perl commands. I was unable to remove them, or it appears that they did not remove, as the blank lines remained. I was also unable to remove the blank lines with cat file |sed '/^$/d' as well.

Does anyone have any suggestions on how to remove the blank line that the ^M appears to be producing?
# 2  
Old 07-21-2014
A Control-M character is the 'Carriage Return". Depending on what version of unix you are running, and what you are running on (server type, pc, etc...) this may have meaning. Or, it may be safe to remove.

\r = carriage return
\n = new line

Last edited by joeyg; 07-21-2014 at 01:39 PM.. Reason: added a couple of the codes
# 3  
Old 07-21-2014
I just want to remove it. When I remove it through vi the blank lines are gone and the character is gone too. However, I need to do this removal by sed or awk.
# 4  
Old 07-21-2014
have you tried
Code:
dos2unix filename filename

# 5  
Old 07-21-2014
The standard way is
Code:
tr -d '\r' < file.dos > file.ux

This User Gave Thanks to Scrutinizer For This Post:
# 6  
Old 07-21-2014
worked great. I found that after that, all I had to do was run a sed command to then remove the blank lines
# 7  
Old 07-21-2014
If you want to remove lines ended by a ^M character:

Code:
sed '/^M$/d' /pathto/infile > /path/to/outfile

In case you want to delete only the character itself, but not the line:

Code:
sed 's/^M//g' /pathto/infile > /path/to/outfile

Notice that "^M" is ONE character, not two! (You will see this when you pass over it with the cursor in "vi".) If you enter caret-M, nothing will happen at all. To enter the "^M" character press CTRL-V, then press the "ENTER" key (or "CTRL-M", which is the same).

I hope this helps.

bakunin
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help in removing control M and Line feed in output file.

Hi All, In my output file i am getting control m character and also the line feeds at different places and with different combinations, the content of the file is supposed to be in a single line but if there is a line feed in between then from there onwards it's going into new line. I tried... (7 Replies)
Discussion started by: Bipin Kumar
7 Replies

2. UNIX for Dummies Questions & Answers

[Solved] Removing control-m characters from shell script

Hi All, I need to remove control m character from a file. Steps which i am doing in shell script are: 1) We are comparing the header of the file to the database table header Here the file header has control-m characters. How do i remove it. Please help. Below are the steps i am using,... (12 Replies)
Discussion started by: abhi_123
12 Replies

3. Shell Programming and Scripting

Removing special characters - Control M

I have developed a small script to remove the Control M characters that get embedded when we move any file from Windows to Unix. For some reason, its not working in all scenarios. Some times I still see the ^M not being removed. Is there anything missing in the script: cd ${inputDir}... (7 Replies)
Discussion started by: vskr72
7 Replies

4. Shell Programming and Scripting

control M character in unix file

in a file we are getting control character in a file , is there any way that they can be removed once we have the file for eg. BEGIN-PROCEDURE INITIALIZE ^M LET #row_count = 0^M ^M ^M (2 Replies)
Discussion started by: lalitpct
2 Replies

5. UNIX for Dummies Questions & Answers

Need help removing last character of every line if certain character

I need help removing the last character of every line if it is a certain character. For example I need to get rid of a % character if it is in the last position. Input: aaa% %bbb ccc d%dd% Output should be: aaa %bbb ccc d%dd I tried this but it gets rid of all of the % characters.... (5 Replies)
Discussion started by: raptor25
5 Replies

6. Shell Programming and Scripting

unwanted control character in the input

Hi Shell Scripting Experts, I have a shell script running daily on a remote machine through ssh. To avoid the trouble of restarting the script when ssh disconnects, I use screen (a unix tool) and run the script within a screen session. What this script does is to ask the user to input y or n... (2 Replies)
Discussion started by: jeff_cen
2 Replies

7. Shell Programming and Scripting

Removing Control M character

Hi, I'm using MKS tool kit to execute KSH on windows and samba to move files to unix from windows. My script is appending header record for the data file. I'm using echo "$header" > $SambaFilename cat $windowsfile >> $SambaFilename But after execution of script ,The file in Unix... (2 Replies)
Discussion started by: ammu
2 Replies

8. UNIX for Dummies Questions & Answers

Control character in a file

Hi All, I am looking for a solution to capture any ASCII control character in a file ( where the ASCII control character is in decimal value from 0 to 31 and 127 ( Hex value from 00 to 1F and 7F ) ) by returning any affected lines. The intended good file should contain "ASCII printable... (5 Replies)
Discussion started by: cursive
5 Replies

9. Shell Programming and Scripting

Removing control-Ms (^M)

Anyone know how to remove ^M's from a text file using sed command? (6 Replies)
Discussion started by: djkane
6 Replies

10. Programming

Identifying and removing control characters in a file.

What is the best method to identify an remove control characters in a file. Would it be easier to do this in Unix or in C. (0 Replies)
Discussion started by: oracle8
0 Replies
Login or Register to Ask a Question

Featured Tech Videos