Shell Programming and Scripting

Hi guys,

Hope you are all well.

This is a line of data from a csv file. I have used vi and set the 'set list' option to display the trailing $ character.

"01","Grocery","01006","eat Fish & Spreads"$

I have tried the following commands, but neither of them appear to be working?

1) tr -d "^M" < originalfile.csv > newfile.csv
2) tr -d "\015" < originalfile.csv > newfile.csv

....but when I vi & set list the new file, I am still seeing the trailing $ character in the file.

Can anyone please tell me where I am going wrong?

Thanks in advance,
Kev

Hope this helps.

I had a big issue with this while trying to get a flat file into Oracle. I tried using vi and sed, and ultimately came upon a solution using strings and tr.

Here is my shell script:

#make a backup
cp $1 $1.orig

#Strip out nulls.
tr -d ‘\0′ < $1.orig > $1.temp

#Strip out other unprintable characters
strings $1.temp > $1

#Remove temp file.
rm $1.temp

#Fix permissions. (You may not need this for your application)
chmod 744 $1

Have you misunderstood vi's "set list". What it does is print a "$" sign at the end of each line (and show tabs as ^I). The $ sign isn't in the file itself it is just displayed to make it easier for you to spot things like trailing spaces at the end of lines.

cheers

The "^M" things are from dos/windows files. dos carriage returns are "\n\r", unix "\n"

The extra character shows up as "^M".

Use dos2ux (or dos2unix depending on your flavor of unix) to convert windows text files.

Code:

dos2ux oldfile > newfile

in vi you can replace all ^M characters with command
:%s/^M//g

(Note that to type ^M in vi type Ctrl-V Enter)

I realise that this note may be offensive for vi users

Thanks for the swift response guys - much appreciated.

Thestevew was right - I was misunderstanding the vi 'set list' option, so thanks for putting me straight.

I eventually came up with a couple of possibilites - I could use the unix2dos command within a batch file to properly format my csv file.

I also found another thread on this forum with the following perl command...

Code:

perl -i -pe 's/$/\r/' myfile.csv

I'll probably use the perl command to avoid implementing a batch file. Plus, I don't have to output the results to a new file.

Out of interest, does anyone know of a sed command that would do the same as the above perl command?

Thanks again,
Kev

Code:

sed -i 's/$/\r/' myfile.csv

I don't think that all implementations of sed has this feature. I've used GNU sed 4.1.4