Getting rid of ^M


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Getting rid of ^M
# 1  
Old 12-13-2011
Getting rid of ^M

I have a text file with hundreds of 32-character hash codes in it, each terminated with a linefeed (/l, or ^M).
Code:
185ead08e45a5cbb51e9f7b0b384aaa2
57643e1a17252a9fc746d49c3da04168
60cba11d09221d52aaabb5db30f408a2
2b75ee6e5c2efc31b4ee9a190d09a4df

...... etc.

I want to create a file for each hash code, with the .txt extension.

I ran a test shell script called test1.sh to see if I could pull out each line in the file and create a file name from it:
Code:
#!/bin/sh
   while read line
   do
      echo $line.txt
  done < hashlist.txt
echo "done"

What I got was
Code:
$ ./test1.sh | less

  185ead08e45a5cbb51e9f7b0b384aaa2^M.txt
57643e1a17252a9fc746d49c3da04168^M.txt
60cba11d09221d52aaabb5db30f408a2^M.txt
2b75ee6e5c2efc31b4ee9a190d09a4df^M.txt
...... etc.

How can I create these file names without the ^M?
It seems if I remove the ^M and replace it with a carriage return (/r) then the while loop won't work.

Teledon

Last edited by vbe; 12-13-2011 at 01:32 PM.. Reason: code tags
# 2  
Old 12-13-2011
Convert the file first using dos2unix or dos2ux utility, if you can't find it (would be surprised though...) I will give you some alternatives (scripts?).
After conversion you should not have any trouble...

P.S.
Next time, use code tags for your code and data
https://www.unix.com/how-post-unix-li...code-tags.html
# 3  
Old 12-13-2011
Getting rid of ^M

You can use " perl -pi -e 's/\r//g' file_name " (dont use double quotes) to get rid of the control m characters.
# 4  
Old 12-13-2011
If you imported your file from MS win... world , you have the reasom of the presence of these char on end of line...(thoufh some clever ftp utilities do the conversion for you...)
# 5  
Old 12-13-2011
Or remove the extra characters with "tr" as you read the file.
Code:
#!/bin/sh
cat hashlist.txt | tr -d '\r' | while read line
do
      echo "${line}.txt"
done
echo "done"

As I'm sure you realise, the spurious carriage-return is Microsoft Text File format. How you transfer a file to a unix platform is important (e.g. use text mode ftp not binary mode ftp).

Last edited by methyl; 12-13-2011 at 01:52 PM.. Reason: tweak
This User Gave Thanks to methyl For This Post:
# 6  
Old 12-13-2011
tr is nice cpu friendly little tool to convert chars or delete chars.
Code:
#!/bin/sh
# \015 = CR
cat hashline.txt | tr -d '\015' | while read line
do
      echo $line.txt
done
echo "done"

# 7  
Old 12-13-2011
To remove the ^M characters at the end of all lines in vi, use the following command:

Code:
shift :%s/^V^M//g

Code:

This User Gave Thanks to bobby1015 For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. AIX

How to get rid of GPFS ?

Hi We are doing the migration of DMX3 disks to DMX4 disks using migratepv. We are not using GPFS but we have gpfs disks present in the server. Can anyone advise how to get rid of GPFS in both the servers cbspsrdb01 and cbspsrdb02. I will do migratepv for the other disks present in the servers... (2 Replies)
Discussion started by: newtoaixos
2 Replies

2. Linux

How to get rid of ^m

Hi all, I am new to unix....pls help me with this. I have a binary file generating output by passing arguments in bash.when i open the output file in VI i can see that ^m is included in between most of lines,as a result when i pass this file to my java application it dosent parse the data... (3 Replies)
Discussion started by: asheshrocky
3 Replies

3. UNIX for Dummies Questions & Answers

how to get rid of last _ in the files name?

ex: I have list of files in a folder. abc_def_geh_.txt abc_.txt abc_def_geh_12345_.txt ab134c_d345345ef_444geh_12345_.txt i need to rename all files to get rid of the _ before .txt result should look like this: abc_def_geh.txt abc.txt abc_def_geh_12345.txt... (2 Replies)
Discussion started by: lv99
2 Replies

4. Shell Programming and Scripting

Getting rid of whitespace

Hello I am working aon script, that tells me how many users or on the system when i run it. The script is #!/bin/bash w | cut -f 1 -d ' ' |sort -u | wc -l When ran it shows 16 users including myself and a line of white space. I was wondering what I need to add to remove my user... (2 Replies)
Discussion started by: mosdojaf
2 Replies

5. Shell Programming and Scripting

how to get rid of ' from variable?

hi, I want to get rid of ' from my variable value in ksh, and I also want to read the value on the right hand side of = into a variable thanks (10 Replies)
Discussion started by: JamesByars
10 Replies

6. UNIX for Dummies Questions & Answers

How to get rid of ^[[D

Hi All, Im selecting a large record from a table and putting it in a file in the unix box. The file has a hidden character "^[[D " present in it. Can any one help me in getting rid of the character Thanks in advance, (4 Replies)
Discussion started by: madhan@29
4 Replies

7. UNIX for Dummies Questions & Answers

how to get rid of ==>

ok the assignment question: That English paper you were writing on the works of Lewis Carroll is due in a few hours and you have forgeotten the name of the text file in which you has written a number of quotations to use in your paper. Luckily, you know that the file is somewhere in your... (1 Reply)
Discussion started by: mek86
1 Replies

8. Shell Programming and Scripting

How to get rid of the last period (.)

Hello, I'm able to trim down the log message to "192.168.1.0.", but can't get rid off the last period. Can any one help me on the syntax to get rid of the last period? Basically, I want "192.168.1.0" instead of "192.168.1.0." Thanks, (4 Replies)
Discussion started by: dannytrinh
4 Replies

9. Shell Programming and Scripting

How to get rid of last line

I have to process a data file in Ab Initio. This data file is pipe delimited. BUt the file may have a Disclaimer line at the end. So before picking it for processing, I need to check if this line is there I need to remove it. ANy suggestions. Thanks Shalu (1 Reply)
Discussion started by: shalua
1 Replies

10. Solaris

getting rid of ^H

Hello everybody I have a very annoying problem on my Solaris (Unix in general) servers. When I open a shell and press the backspace button, it results in a ^H character being printed on screen. I can resolve it by typing stty erase <backspace>, but does anyone know how I can prevent the... (3 Replies)
Discussion started by: soliberus
3 Replies
Login or Register to Ask a Question

Featured Tech Videos