Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Getting rid of ^M

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Getting rid of ^M
# 1  
Old 12-13-2011
Getting rid of ^M
I have a text file with hundreds of 32-character hash codes in it, each terminated with a linefeed (/l, or ^M).
Code:
185ead08e45a5cbb51e9f7b0b384aaa2
57643e1a17252a9fc746d49c3da04168
60cba11d09221d52aaabb5db30f408a2
2b75ee6e5c2efc31b4ee9a190d09a4df

...... etc.

I want to create a file for each hash code, with the .txt extension.

I ran a test shell script called test1.sh to see if I could pull out each line in the file and create a file name from it:
Code:
#!/bin/sh
   while read line
   do
      echo $line.txt
  done < hashlist.txt
echo "done"

What I got was
Code:
$ ./test1.sh | less

  185ead08e45a5cbb51e9f7b0b384aaa2^M.txt
57643e1a17252a9fc746d49c3da04168^M.txt
60cba11d09221d52aaabb5db30f408a2^M.txt
2b75ee6e5c2efc31b4ee9a190d09a4df^M.txt
...... etc.

How can I create these file names without the ^M?
It seems if I remove the ^M and replace it with a carriage return (/r) then the while loop won't work.

Teledon
Last edited by vbe; 12-13-2011 at 01:32 PM.. Reason: code tags
# 2  
Old 12-13-2011
Convert the file first using dos2unix or dos2ux utility, if you can't find it (would be surprised though...) I will give you some alternatives (scripts?).
After conversion you should not have any trouble...

P.S.
Next time, use code tags for your code and data
https://www.unix.com/how-post-unix-li...code-tags.html
# 3  
Old 12-13-2011
Getting rid of ^M
You can use " perl -pi -e 's/\r//g' file_name " (dont use double quotes) to get rid of the control m characters.
# 4  
Old 12-13-2011
If you imported your file from MS win... world , you have the reasom of the presence of these char on end of line...(thoufh some clever ftp utilities do the conversion for you...)
# 5  
Old 12-13-2011
Or remove the extra characters with "tr" as you read the file.
Code:
#!/bin/sh
cat hashlist.txt | tr -d '\r' | while read line
do
      echo "${line}.txt"
done
echo "done"

As I'm sure you realise, the spurious carriage-return is Microsoft Text File format. How you transfer a file to a unix platform is important (e.g. use text mode ftp not binary mode ftp).
Last edited by methyl; 12-13-2011 at 01:52 PM.. Reason: tweak
This User Gave Thanks to methyl For This Post:
teledon
# 6  
Old 12-13-2011
tr is nice cpu friendly little tool to convert chars or delete chars.
Code:
#!/bin/sh
# \015 = CR
cat hashline.txt | tr -d '\015' | while read line
do
      echo $line.txt
done
echo "done"

# 7  
Old 12-13-2011
To remove the ^M characters at the end of all lines in vi, use the following command:

Code:
shift :%s/^V^M//g

Code:















        

        

       



            
            
            
            


            

            
            




            
                
            

            

        



        


        


 


        

         



	

           

                  

	
                
				
					
This User Gave Thanks to bobby1015 For This Post:

				
            

			
			

        

		
 methyl

    





        

        


 

            
        




    

         

          
        

    

    

       

          
        

    









		








            

        


        
        



        



        
        

            


                
                    
                        

                        
                    
                    
                
            




        





Login or Register to Ask a Question






 
            
            

            

               
                Previous Thread
                |
                Next Thread
                
            

            
        

        

        
        
        
        
        

        

        



        

  

            


            

        

       

        

        

       
        

            




10 More Discussions You Might Find Interesting









1. AIX






 How to get rid of GPFS ?




Hi 
 
We are doing the migration of DMX3 disks to DMX4 disks using migratepv. We are not using GPFS but we have gpfs disks present in the server. Can anyone advise how to get rid of GPFS in both the servers cbspsrdb01 and cbspsrdb02. I will do migratepv for the other disks present in the servers... (2 Replies)





Discussion started by: newtoaixos



2 Replies








2. Linux






 How to get rid of ^m




Hi all, 
  
I am new to unix....pls help me with this. 
  
I have a binary file generating output by passing arguments in bash.when i open the output file in VI i can see that ^m is included in between most of lines,as a result when i pass this file to my java application it dosent parse the data... (3 Replies)





Discussion started by: asheshrocky



3 Replies








3. UNIX for Dummies Questions & Answers






 how to get rid of last _ in the files name?




ex: I have list of files in a folder.  
abc_def_geh_.txt 
abc_.txt 
abc_def_geh_12345_.txt 
ab134c_d345345ef_444geh_12345_.txt 
 
i need to rename all files to get rid of the _ before .txt 
 
result should look like this: 
abc_def_geh.txt 
abc.txt 
abc_def_geh_12345.txt... (2 Replies)





Discussion started by: lv99



2 Replies








4. Shell Programming and Scripting






 Getting rid of whitespace




Hello I am working aon script, that tells me how many users or on the system when i run it.   
  
The script is  
 #!/bin/bash 
w | cut -f 1 -d ' ' |sort -u | wc -l 
  
When ran it shows 16 users including myself and a line of white space. 
  
I was wondering what I need to add to remove my user... (2 Replies)





Discussion started by: mosdojaf



2 Replies








5. Shell Programming and Scripting






 how to get rid of ' from variable?




hi, 
 
I want to get rid of ' from my variable value in ksh, and I also want to read the value on the right hand side of = into a variable 
 
thanks (10 Replies)





Discussion started by: JamesByars



10 Replies








    

       

          
        

    

    

      

          
        

    






6. UNIX for Dummies Questions & Answers






 How to get rid of ^[[D




Hi All, 
  
    Im selecting a large record from a table and putting it in a file in the unix box. The file has a hidden character "^[[D " present in it. Can any one help me in getting rid of the character 
  
  
Thanks in advance, (4 Replies)





Discussion started by: madhan@29



4 Replies








7. UNIX for Dummies Questions & Answers






 how to get rid of ==>




ok the assignment question: 
That English paper you were writing on the works of Lewis Carroll is due in a few hours and you have forgeotten the name of the text file in which you has written a number of quotations to use in your paper. Luckily, you know that the file is somewhere in your... (1 Reply)





Discussion started by: mek86



1 Replies








8. Shell Programming and Scripting






 How to get rid of the last period (.)




Hello, 
I'm able to trim down the log message to "192.168.1.0.", but can't get rid off the last period. 
Can any one help me on the syntax to get rid of the last period? 
 
Basically, I want "192.168.1.0" instead of "192.168.1.0." 
 
Thanks, (4 Replies)





Discussion started by: dannytrinh



4 Replies








9. Shell Programming and Scripting






 How to get rid of last line




I have to process a data file in Ab Initio. This data file is pipe delimited. BUt the file may have a Disclaimer line at the end. So before picking it for processing, I need to check if this line is there I need to remove it. ANy suggestions. 
 
Thanks 
Shalu (1 Reply)





Discussion started by: shalua



1 Replies








10. Solaris






 getting rid of ^H




Hello everybody 
 
I have a very annoying problem on my Solaris (Unix in general) servers.  When I open a shell and press the backspace button, it results in a ^H character being printed on screen.  I can resolve it by typing stty erase <backspace>, but does anyone know how I can prevent the... (3 Replies)





Discussion started by: soliberus



3 Replies







        

       

        
        

            


                


                
                    

                        
                        

                            
                                
                            


                        

                    

                		
            

        

        


        




Login or Register to Ask a Question