Replace Junk chars (Sed)

Tags
shell scripts

 
Thread Tools Search this Thread
# 1  
Old 03-20-2007
Replace Junk chars (Sed)

I know this has been asked previously on this forum...But I think I have a different scenario to present.

I ahve a file tht looks like this (note:there are control Z and other chars tht are not visible on line with anme bowers)

Code:
BB7118450      	6004718		BIANCALANA =HEI         
BZ5842819      	1493750		ZACHAR=/                
AB1250620      	0300080		BOWERS               
MR0408256      	0458355		DYCUS/BOWERS/PA         
AB9381233      	6811741		BOWERS *6*              
BL6039588      	0184195		LIMERES JR. |           
AD1901253      	0458720		DURAZO - CELAYA         
BE4340698      	0609541		EL-HAJJAOUI(LUN         
AH5528267      	6786292		HARSANY, JR ++          
BD0762662      	4527818		D`SOUZA                 
AC6629577      	3059914		CANOVA`                 
AC9130991      	0485282		CARROLL ›16!            
AT8874679      	0490912		TAIT ›20!               
AH8136459      	0136813		HUNTER ›06! (16

This file has special chars and Control M/Z chars....I need to remove all these junk chars and replace with a space. I tried the following commands..

Code:
sed 's/[!@#\$%^&*()<>+|={}\/`-]//g' special.txt

The above worked except for the fact that the Control Z chars were not replaced.

I tried
Code:
tr -d '\032' < special.txt | sed 's/[!@#\$%^&*()<>+|={}\/`\"\^M\'026'-]//g' > t

but some additional chars were deleted...

Any help in getting the command right would be appreciated...
# 2  
Old 03-20-2007
Quote:
Originally Posted by alfredo123
I know this has been asked previously on this forum...But I think I have a different scenario to present.

I ahve a file tht looks like this (note:there are control Z and other chars tht are not visible on line with anme bowers)

Code:
BB7118450      	6004718		BIANCALANA =HEI         
BZ5842819      	1493750		ZACHAR=/                
AB1250620      	0300080		BOWERS               
MR0408256      	0458355		DYCUS/BOWERS/PA         
AB9381233      	6811741		BOWERS *6*              
BL6039588      	0184195		LIMERES JR. |           
AD1901253      	0458720		DURAZO - CELAYA         
BE4340698      	0609541		EL-HAJJAOUI(LUN         
AH5528267      	6786292		HARSANY, JR ++          
BD0762662      	4527818		D`SOUZA                 
AC6629577      	3059914		CANOVA`                 
AC9130991      	0485282		CARROLL ›16!            
AT8874679      	0490912		TAIT ›20!               
AH8136459      	0136813		HUNTER ›06! (16

This file has special chars and Control M/Z chars....I need to remove all these junk chars and replace with a space. I tried the following commands..

Code:
sed 's/[!@#\$%^&*()<>+|={}\/`-]//g' special.txt

The above worked except for the fact that the Control Z chars were not replaced.

I tried
Code:
tr -d '\032' < special.txt | sed 's/[!@#\$%^&*()<>+|={}\/`\"\^M\'026'-]//g' > t

but some additional chars were deleted...

Any help in getting the command right would be appreciated...
Try to use

tr -d "\015\032" < inputfile > outputfile

Then use your sed on the outputfile. Let me know if this works.
# 3  
Old 03-21-2007
HI,
CTRL M characters occur in files if the files are created in Windows and opened in Unix.
To remove the CTRL characters use the command
If u r using Linux
dos2unix <filename>
If u are using solaris
dos2unix <filename> <filename>

Thanks
Raghuram
# 4  
Old 02-27-2008
Hi, I also have the Control-Z problem. I tried the following code and it works
Code:
tr -d '\32' < test1.dat > test2.dat

However, I try to do the following and it doesn't work
Code:
var=32
tr -d '\{$var}' < test1.dat > test2.dat

I need to assign the octal number to a variable because we want the script to be able to remove other ascii char. Anyone get any ideas?
# 5  
Old 02-27-2008
Anyone knows how to fix the problem?

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

More UNIX and Linux Forum Topics You Might Find Helpful
How to replace and remove few junk characters from a specific field? snemuk14 UNIX for Dummies Questions & Answers 10 10-01-2014 12:32 PM
Grep or sed to search, replace/insert chars! graymj UNIX for Dummies Questions & Answers 7 04-14-2014 08:48 AM
Replace char between chars - help needed go0ogl3 Shell Programming and Scripting 10 07-07-2012 10:57 AM
How to erase junk Chars coming in only the first line subramanian2008 Shell Programming and Scripting 3 11-02-2011 01:47 AM
AWK/SED: handle max chars in a line unknown7 Shell Programming and Scripting 5 04-20-2011 05:25 PM
sed discard chars after last _ dips_ag Shell Programming and Scripting 14 05-13-2010 04:08 AM
Special chars in sed variable mutex Shell Programming and Scripting 2 12-03-2009 03:59 PM
sed - how to insert chars into a line tangentviper Shell Programming and Scripting 2 02-01-2009 03:19 AM
is is possible remove junk chars from the strings? balan_mca Shell Programming and Scripting 1 10-31-2008 08:26 PM
sed to extract first two uppercase chars in targeted lines str8danked Shell Programming and Scripting 5 06-10-2008 05:32 AM
Using SED to get n chars after given value from file Kally UNIX for Dummies Questions & Answers 12 12-19-2007 05:54 AM
replace chars, Jairaj Shell Programming and Scripting 7 02-28-2007 06:34 AM
Clense Junk Data File - Using Shell or awk or sed rimss Shell Programming and Scripting 1 06-07-2006 11:00 AM
replace ascii chars without loosing it. braindrain Shell Programming and Scripting 4 02-01-2006 10:02 AM
Extracting the last 3 chars from a string using sed 435 Gavea UNIX for Dummies Questions & Answers 7 11-27-2003 06:55 AM