How to replace special characters?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to replace special characters?
# 1  
Old 10-30-2012
How to replace special characters?

Hi Unix Guru,

I have an requirement for replace some specail characters in a file, my file came from mainframe.
please see below example:
when open it with vi
Code:
17896660|89059215|04/24/1998 00:00:00.000000| abc 123-453-1312^M<85>^M<85>|124557

if I run cat -v I got following:
Code:
17896660|89059215|04/24/1998 00:00:00.000000| abc 123-453-1312^MM-^E^MM-^E|124557

there are more than 10000 records

my question is how can I remove these special characters.

Thanks in advance

Smilie
ken002
# 2  
Old 10-30-2012
Are they at a fixed location in the line?
Eks:after number, but before |
# 3  
Old 10-30-2012
Quote:
Originally Posted by Jotne
Are they at a fixed location in the line?
Eks:after number, but before |
No, they are in fixed field but the field's content changed
ken002
# 4  
Old 10-30-2012
Try using the below command on your file and redirect the output to a new file and check if the special characters are removed:-

Code:
sed 's/'"$(printf '\015')"'//g' input_file > output_file

# 5  
Old 10-30-2012
Quote:
Originally Posted by bipinajith
Try using the below command on your file and redirect the output to a new file and check if the special characters are removed:-

Code:
sed 's/'"$(printf '\015')"'//g' input_file > output_file


thanks for you reply, I tried it. it removed one ^M. the file looks as following:
Code:
 
17896660|89059215|04/24/1998 00:00:00.000000| abc 100-453-1312M-^EM-^E|124557

using vi
Code:
17896660|89059215|04/24/1998 00:00:00.000000| abc 100-453-1312<85><85>|124557

anyidea?

Thanks in advance
ken002
# 6  
Old 10-30-2012
So using vi you are not seeing ^M characters anymore, is that correct? Also do you want to remove < & > symbols ?
# 7  
Old 10-30-2012
Quote:
Originally Posted by bipinajith
So using vi you are not seeing ^M characters anymore, is that correct? Also do you want to remove < & > symbols ?
Yes, I need to remove <85><85> because this is a invisable character.
ken002
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Replace Pattern with another that has Special Characters

Hello Team, Any help would be much appreciated for the below scenario: I have a sed command below where I am trying to replace the contents of 'old_pkey' variable with 'new_pkey' variable in a Soap request file (delete_request.txt). This works fine for regular string values, but this new_pkey... (8 Replies)
Discussion started by: ChicagoBlues
8 Replies

2. UNIX for Advanced & Expert Users

How to replace special characters?

Hi Team, I have data like this. |*|.5|*|0.2|*|A.B|*| Would like to add zero (0) before the decimal point where there is no zero as |*|0.5|*|0.2|*|A.B|*| How to replace |*|. with |*|0. I tried below command which didn't work echo '|*|.5|*|0.2|*|A.B|*' | sed... (4 Replies)
Discussion started by: Ravi.K
4 Replies

3. Shell Programming and Scripting

Help to replace the string with special characters

{"name":"alR_pl-ENVIRONMENT_192_168_211_123_sDK_PROVISION_7","description":"aLR_pl-ENVIRONMENT_192_168_211_123_sDK_PROVISION_7","json_class":"Chef::Role","default_attributes":{},"override_attributes":{"yoapp":{"jboss":"5.1.0","port":"2243","warname":"soap","datacenter":"alR","ip":"192.168.211.123","... (3 Replies)
Discussion started by: nikhil jain
3 Replies

4. Shell Programming and Scripting

Replace special characters with backslash and character

Hi, I have a string wherein i need to replace special characters with backslash and that character. Ex: If my string is a=qwerty123@!, then the new string should be a_new=qwerty123\@\!\, Thanks (3 Replies)
Discussion started by: temp_user
3 Replies

5. Shell Programming and Scripting

Replace special characters with Escape characters?

i need to replace the any special characters with escape characters like below. test!=123-> test\!\=123 !@#$%^&*()-= to be replaced by \!\@\#\$\%\^\&\*\(\)\-\= (8 Replies)
Discussion started by: laknar
8 Replies

6. Shell Programming and Scripting

Replace special characters

I have a line ending with special character and 0 The special character is the field separator for this line in VI mode the file will look like below, but while cat the special character wont display i know the hexa code for the special character ^_ is \x1f and ascii code is \0037, ... (0 Replies)
Discussion started by: ratheeshjulk
0 Replies

7. Solaris

How to replace special characters in vi?

Hi , I want to replace the special characters in the file. For eg: cat abc 1234/4455/acb 234/k/lll/ 234`fs`fd I want to replace / and ` with the letter a and the output should like below. How to achieve this. 1234a4455aacb 234akallla 234afsafd (2 Replies)
Discussion started by: rogerben
2 Replies

8. Shell Programming and Scripting

Using sed to replace special characters

Hi everyone I have file1 contains: '7832' ' 8765 6543 I want a sed command that will format as: '7832' , '8765' , '6543' I tried sed -e s/\'//g -e 's/^*//;s/*$//' file1 > file2 sed -e :a -e '$!N; s/\n/ /; ta' file2 which gives: 7832 8765 6543 I need some help to continue with... (5 Replies)
Discussion started by: nimo
5 Replies

9. UNIX for Dummies Questions & Answers

Find and replace special characters in a file

HI All I need a shell script ehich removes all special characters from file and converts the file to UTF-* format Specail characters to be removed must be configurable. strIllegal = @"?/><,:;""'{|\\+=-)(*&^%$#@!~`"; Please help me in getting this script as my scripting skilla are... (2 Replies)
Discussion started by: sujithchandra
2 Replies

10. UNIX for Dummies Questions & Answers

Replace Special characters in a file

Hi, I have a data like this in a file, 402003279034002000100147626030003300010000000000002000029000000 ær^M^\MÍW^H I need to replace those special char to some other char like # or $ Is there any ways to do it... I tried commands tr,sed and many but it was not able to replace because... (1 Reply)
Discussion started by: solai
1 Replies
Login or Register to Ask a Question