Problem with ^M in input file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Problem with ^M in input file
# 1  
Old 10-24-2005
Problem with ^M in input file

Greetings to all,

I have this problem comparing two file and extract the output from it.

"input.txt" file(Extracted from excel):
ABC
ABB
ABA

"compare.txt" file(Extracted from excel):
ABA 1
ABB 2
ABC 3
ABD 4
ABE 5
ABF 6
ABG 7

I would like to get output of
ABC 3
ABB 2
ABA 1

This is my script:
-----------------
while read record
do
grep "$record" compare.txt
done < input.txt > output.txt

The result of `wc -l output.txt` is zero.

I later `echo "try" > input.txt
and when I `vi input.txt` I found this:

ABC^M
ABB^M
ABA^M
try

Question:
Why is the ^M appearing and how to get rid of it?

I tried removing the ^M manually and my output is all NICE.

Please advice.

Thank you.

cheers,
-NoeL-
# 2  
Old 10-24-2005
The ^M appears in files that come over from Windows or DOS. This is because the line ending character in Unix is "\n", while in Windows and DOS, you have "\r\n". The "\n" from these is shown as the ^M.

For a nice explaination go here.

To get rid of these characters, you can use the dos2unix (Solaris) or dos2ux (HP) commands. Or these little perl statements:
Code:
# neither of the following perl statements have been tested by me
perl -pe 's/\015\012/\n/g' ##- dos to unix -- just pipe through it
perl -pe 's/\n/\015\012/g' ##- unix to dos -- just pipe through it

Or do what I do:
Code:
strings file_to_convert > converted_file; mv converted_file file_to_convert


Last edited by blowtorch; 10-24-2005 at 05:49 AM..
# 3  
Old 10-24-2005
Thanks blowtorch.

It works like charm.

cheers,
-NoeL-
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Homework & Coursework Questions

Removing punctuations from file input or standard input

Just started learning Unix and received my first assignment recently. We haven't learned many commands and honestly, I'm stumped. I'd like to receive assistance/guidance/hints. 1. The problem statement, all variables and given/known data: How do I write a shell script that takes in a file or... (4 Replies)
Discussion started by: fozilla
4 Replies

2. Shell Programming and Scripting

Read input files and merge them in given order and write them to input one param or one file

Dear Friends, I am looking for a shell script to merge input files into one file .. here is my idea: 1st paramter would be outfile file (all input files content) read all input files and merge them to input param 1 ex: if I pass 6 file names to the script then 1st file name as output file... (4 Replies)
Discussion started by: hyd1234
4 Replies

3. Shell Programming and Scripting

XML variable for input in same input file

Dear All , i stuck in one problem executing xml .. i have input xml as <COMMAND name="ARRANGEMENT.WRITE" timestamp="0" so="initial"> <SVLOBJECT> <LONG name="CSP_PMNT_ID" val="-1"/> <MONEY name="CSP_CEILING" amount="0.0" currency="AUD"/> ... (6 Replies)
Discussion started by: arvindng
6 Replies

4. Shell Programming and Scripting

Script to delete files with an input for directories and an input for path/file

Hello, I'm trying to figure out how best to approach this script, and I have very little experience, so I could use all the help I can get. :wall: I regularly need to delete files from many directories. A file with the same name may exist any number of times in different subdirectories.... (3 Replies)
Discussion started by: *ShadowCat*
3 Replies

5. Shell Programming and Scripting

Input and value problem

Hi, I need to convert given cents to dollars so I have a script with this this: A=$(echo "$1*0.01" | bc) echo $A B=$(echo "$2*0.01" | bc) echo $B C=$(echo "$3*0.01" | bc) echo $C D=$(echo "$4*0.01" | bc) echo $D E=$(echo "$5*0.01" | bc) echo $E F=$(echo "$6*0.01" | bc) echo... (2 Replies)
Discussion started by: Tr0cken
2 Replies

6. Shell Programming and Scripting

Exclude specific word from input file problem asking...

Hi, Below is my input file and desired output file: Input file: >header_N_1 ASFDGDGNDGEGWETWRYWTETWNETOWETWETWNETTETNWET . . Desired output file: >header_N_1 ASFDGDGDGEGWETWRYWTETWETOWETWETWETTETWET . . From the input file, I just hope to exclude the 'N' word from its content... (5 Replies)
Discussion started by: patrick87
5 Replies

7. Shell Programming and Scripting

Need script to take input from file, match on it in file 2 and input data

All, I am trying to figure out a script to run in windows that will allow me to match on First column in file1 to 8th Column in File2 then Insert file1 column2 to file2 column4 then create a new file. File1: 12345 Sam 12346 Bob 12347 Bill File2:... (1 Reply)
Discussion started by: darkoth
1 Replies

8. UNIX for Dummies Questions & Answers

Intermittent problem reading from an input file.

First of all thanks to all for the good post, and the great site. I'm a noob, but I've been able to learna a lot by checking past posts. I haven't been able to make sense of a problem that I've been working on for a while, hopefully someone can help me out. The script I wrote telnets into... (7 Replies)
Discussion started by: Wallygooo32
7 Replies

9. Shell Programming and Scripting

Reading specific contents from 1 input files and appending it to another input file

Hi guys, I am new to AWK and unix scripting. Please see below my problem and let me know if anyone you can help. I have 2 input files (example given below) Input file 2 is a standard file (it will not change) and we have to get the name (second column after comma) from it and append it... (5 Replies)
Discussion started by: sksahu
5 Replies

10. Shell Programming and Scripting

Problem taking input from file with for loop

I am trying to take input from a file and direct it into a bash script. This script is meant to be a foreach loop. I would like the script to process each item in the list one by one and direct the output to a file. # cat 1loop #!/bin/bash # this 2>&1 to redirect STDERR & STDOUT to file... (4 Replies)
Discussion started by: bash_in_my_head
4 Replies
Login or Register to Ask a Question