Unix/Linux Go Back    


Shell Programming and Scripting BSD, Linux, and UNIX shell scripting Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

How to replace special characters?

Shell Programming and Scripting


Closed    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 10-30-2012   -   Original Discussion by ken002
ken002 ken002 is offline
Users Awaiting Email Confirmation
 
Join Date: Nov 2010
Last Activity: 3 January 2017, 6:36 PM EST
Posts: 80
Thanks: 46
Thanked 1 Time in 1 Post
How to replace special characters?

Hi Unix Guru,

I have an requirement for replace some specail characters in a file, my file came from mainframe.
please see below example:
when open it with vi

Code:
17896660|89059215|04/24/1998 00:00:00.000000| abc 123-453-1312^M<85>^M<85>|124557

if I run cat -v I got following:

Code:
17896660|89059215|04/24/1998 00:00:00.000000| abc 123-453-1312^MM-^E^MM-^E|124557

there are more than 10000 records

my question is how can I remove these special characters.

Thanks in advance

Linux
Sponsored Links
    #2  
Old Unix and Linux 10-30-2012   -   Original Discussion by ken002
Jotne's Unix or Linux Image
Jotne Jotne is offline
Registered User
 
Join Date: Dec 2010
Last Activity: 20 September 2014, 2:08 AM EDT
Posts: 1,040
Thanks: 62
Thanked 213 Times in 203 Posts
Are they at a fixed location in the line?
Eks:after number, but before |
Sponsored Links
    #3  
Old Unix and Linux 10-30-2012   -   Original Discussion by ken002
ken002 ken002 is offline
Users Awaiting Email Confirmation
 
Join Date: Nov 2010
Last Activity: 3 January 2017, 6:36 PM EST
Posts: 80
Thanks: 46
Thanked 1 Time in 1 Post
Quote:
Originally Posted by Jotne View Post
Are they at a fixed location in the line?
Eks:after number, but before |
No, they are in fixed field but the field's content changed
    #4  
Old Unix and Linux 10-30-2012   -   Original Discussion by ken002
Yoda's Unix or Linux Image
Yoda Yoda is offline Forum Advisor  
Jedi Master
 
Join Date: Jan 2012
Last Activity: 19 November 2017, 8:21 PM EST
Location: Galactic Empire
Posts: 3,601
Thanks: 253
Thanked 1,315 Times in 1,232 Posts
Try using the below command on your file and redirect the output to a new file and check if the special characters are removed:-


Code:
sed 's/'"$(printf '\015')"'//g' input_file > output_file

Sponsored Links
    #5  
Old Unix and Linux 10-30-2012   -   Original Discussion by ken002
ken002 ken002 is offline
Users Awaiting Email Confirmation
 
Join Date: Nov 2010
Last Activity: 3 January 2017, 6:36 PM EST
Posts: 80
Thanks: 46
Thanked 1 Time in 1 Post
Quote:
Originally Posted by bipinajith View Post
Try using the below command on your file and redirect the output to a new file and check if the special characters are removed:-


Code:
sed 's/'"$(printf '\015')"'//g' input_file > output_file


thanks for you reply, I tried it. it removed one ^M. the file looks as following:

Code:
 
17896660|89059215|04/24/1998 00:00:00.000000| abc 100-453-1312M-^EM-^E|124557

using vi

Code:
17896660|89059215|04/24/1998 00:00:00.000000| abc 100-453-1312<85><85>|124557

anyidea?

Thanks in advance
Sponsored Links
    #6  
Old Unix and Linux 10-30-2012   -   Original Discussion by ken002
Yoda's Unix or Linux Image
Yoda Yoda is offline Forum Advisor  
Jedi Master
 
Join Date: Jan 2012
Last Activity: 19 November 2017, 8:21 PM EST
Location: Galactic Empire
Posts: 3,601
Thanks: 253
Thanked 1,315 Times in 1,232 Posts
So using vi you are not seeing ^M characters anymore, is that correct? Also do you want to remove < & > symbols ?
Sponsored Links
    #7  
Old Unix and Linux 10-30-2012   -   Original Discussion by ken002
ken002 ken002 is offline
Users Awaiting Email Confirmation
 
Join Date: Nov 2010
Last Activity: 3 January 2017, 6:36 PM EST
Posts: 80
Thanks: 46
Thanked 1 Time in 1 Post
Quote:
Originally Posted by bipinajith View Post
So using vi you are not seeing ^M characters anymore, is that correct? Also do you want to remove < & > symbols ?
Yes, I need to remove <85><85> because this is a invisable character.
Sponsored Links
Closed

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Replace special characters with Escape characters? laknar Shell Programming and Scripting 8 01-06-2012 12:40 AM
Replace special characters ratheeshjulk Shell Programming and Scripting 0 12-22-2011 09:18 PM
How to replace special characters in vi? rogerben Solaris 2 08-03-2011 06:53 AM
Replace Special characters in a file solai UNIX for Dummies Questions & Answers 1 07-13-2006 11:36 AM



All times are GMT -4. The time now is 10:13 PM.