Issue with Removing Carriage Return (^M) in delimited file

sirahc · #1 (**permalink**) 03-19-2008

Hi - I tried to remove ^M in a delimited file using "tr -d "\r" and "sed 's/^M//g'", but it does not work quite well. While the ^M is removed, the format of the record is still cut in half, like
a,b, c
c,d,e

The delimited file is generated using sh script by outputing a SQL query result to tab delimited file. The ^M embedded in the record is captured from user input, I believe.

Any insight is appreciated.

joeyg · #2 (**permalink**) 03-19-2008

Or at least the first few lines of data?

try something like the following which will look at the first five lines:
(use your file in place of infile)

>head -5 infile | od -An -t oC -w10

Note, in octal
<CR> carriage return = 015
<LF> line feed = 012

sirahc · #3 (**permalink**) 03-19-2008

Hi joeyg,

Thanks for your reply. Unfortunately, this is sensitive data so I can't send out a dump. I did use the od command to pull the data and checked for \r (od -c). I see that there is carriage return (\r)+ new line control (\n) embedded in the record.

When I used tr -d "\r\n" , however it remove all line breaks with just "\n". Do you know how I can remove only the string "\r\n" without touching those with only "\n"?

Thanks,
sirahc

joeyg · #4 (**permalink**) 03-20-2008

How about something like:

Code:

infile="original.dat"
outfile="revised.dat"
while read zf
  do
    printf "$zf\n" >>$outfile
done < $infile

kanu_kanu · #5 (**permalink**) 03-20-2008

how abt dos2ux / dtox command ?

sirahc · #6 (**permalink**) 03-21-2008

Thanks for all your suggestions.

The printf/dos2unix commands did work to some extend but cause format problem to other columns. I eventually used SQL to remove ^M before outputting it to the file.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Carriage Return at end of file	bd_joy	Shell Programming and Scripting	14	10-20-2006 01:20 PM
Removing Carriage Return and or line feed from a file	tbone231	Shell Programming and Scripting	1	02-18-2005 04:37 PM
sed removing carriage return and newline	mored	Shell Programming and Scripting	2	05-06-2004 12:28 PM
Removing Carriage return to create one record	r1500	Shell Programming and Scripting	3	02-06-2004 03:45 PM
Removing carriage return characters from file	b1saini	UNIX for Dummies Questions & Answers	3	09-10-2003 09:41 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread:


	Powered by