Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Replace Junk chars (Sed)

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Replace Junk chars (Sed)
# 1  
Old 03-20-2007
Replace Junk chars (Sed)
I know this has been asked previously on this forum...But I think I have a different scenario to present.

I ahve a file tht looks like this (note:there are control Z and other chars tht are not visible on line with anme bowers)

Code:
BB7118450      	6004718		BIANCALANA =HEI         
BZ5842819      	1493750		ZACHAR=/                
AB1250620      	0300080		BOWERS               
MR0408256      	0458355		DYCUS/BOWERS/PA         
AB9381233      	6811741		BOWERS *6*              
BL6039588      	0184195		LIMERES JR. |           
AD1901253      	0458720		DURAZO - CELAYA         
BE4340698      	0609541		EL-HAJJAOUI(LUN         
AH5528267      	6786292		HARSANY, JR ++          
BD0762662      	4527818		D`SOUZA                 
AC6629577      	3059914		CANOVA`                 
AC9130991      	0485282		CARROLL ›16!            
AT8874679      	0490912		TAIT ›20!               
AH8136459      	0136813		HUNTER ›06! (16

This file has special chars and Control M/Z chars....I need to remove all these junk chars and replace with a space. I tried the following commands..

Code:
sed 's/[!@#\$%^&*()<>+|={}\/`-]//g' special.txt

The above worked except for the fact that the Control Z chars were not replaced.

I tried
Code:
tr -d '\032' < special.txt | sed 's/[!@#\$%^&*()<>+|={}\/`\"\^M\'026'-]//g' > t

but some additional chars were deleted...

Any help in getting the command right would be appreciated...
# 2  
Old 03-20-2007
Quote:
Originally Posted by alfredo123
I know this has been asked previously on this forum...But I think I have a different scenario to present.

I ahve a file tht looks like this (note:there are control Z and other chars tht are not visible on line with anme bowers)

Code:
BB7118450      	6004718		BIANCALANA =HEI         
BZ5842819      	1493750		ZACHAR=/                
AB1250620      	0300080		BOWERS               
MR0408256      	0458355		DYCUS/BOWERS/PA         
AB9381233      	6811741		BOWERS *6*              
BL6039588      	0184195		LIMERES JR. |           
AD1901253      	0458720		DURAZO - CELAYA         
BE4340698      	0609541		EL-HAJJAOUI(LUN         
AH5528267      	6786292		HARSANY, JR ++          
BD0762662      	4527818		D`SOUZA                 
AC6629577      	3059914		CANOVA`                 
AC9130991      	0485282		CARROLL ›16!            
AT8874679      	0490912		TAIT ›20!               
AH8136459      	0136813		HUNTER ›06! (16

This file has special chars and Control M/Z chars....I need to remove all these junk chars and replace with a space. I tried the following commands..

Code:
sed 's/[!@#\$%^&*()<>+|={}\/`-]//g' special.txt

The above worked except for the fact that the Control Z chars were not replaced.

I tried
Code:
tr -d '\032' < special.txt | sed 's/[!@#\$%^&*()<>+|={}\/`\"\^M\'026'-]//g' > t

but some additional chars were deleted...

Any help in getting the command right would be appreciated...
Try to use

tr -d "\015\032" < inputfile > outputfile

Then use your sed on the outputfile. Let me know if this works.
# 3  
Old 03-21-2007
HI,
CTRL M characters occur in files if the files are created in Windows and opened in Unix.
To remove the CTRL characters use the command
If u r using Linux
dos2unix <filename>
If u are using solaris
dos2unix <filename> <filename>

Thanks
Raghuram
# 4  
Old 02-27-2008
Hi, I also have the Control-Z problem. I tried the following code and it works
Code:
tr -d '\32' < test1.dat > test2.dat

However, I try to do the following and it doesn't work
Code:
var=32
tr -d '\{$var}' < test1.dat > test2.dat

I need to assign the octal number to a variable because we want the script to be able to remove other ascii char. Anyone get any ideas?
# 5  
Old 02-27-2008
Anyone knows how to fix the problem?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to replace and remove few junk characters from a specific field?

I would like to remove all characters starting with "%" and ending with ")" in the 4th field - please help!! 1412007819.864 /device/services/heartbeatxx 204 0.547%!i(int=0) 0.434 0.112 1412007819.866 /device/services/heartbeatxx 204 0.547%!i(int=1) 0.423 0.123... (10 Replies)
Discussion started by: snemuk14
10 Replies

2. UNIX for Dummies Questions & Answers

Grep or sed to search, replace/insert chars!

HI All Im trying to come up with an approach to finding a string, using a portion of that string to insert it on lines starting with the value "GOTO" appending to end of line after removing PT's ( See example below! ) EXAMPLE: 1. I would like to search for the line that starts with "TLAXIS/"... (7 Replies)
Discussion started by: graymj
7 Replies

3. Shell Programming and Scripting

Replace char between chars - help needed

Hello, I have a csv file with "^" as text delimiters and "|" as field delimiters. It's converted from a xls file. One record looks like this: ^Tablete Internet^|Archos|501838|^Tableta Internet ARCHOS 80 G9 ...| ... (more lines) ... "501|838"^|330.00|USD|sl|12|0|Link|^router wireless 150... (10 Replies)
Discussion started by: go0ogl3
10 Replies

4. Shell Programming and Scripting

How to erase junk Chars coming in only the first line

I have a file containing few thousands of lines. when I do cat on it , i find it having two special Chars at the start of first line alone as shown down here. ÿþHDR|20111024|01 If i delete this line and do a cat on file , the current first line is shown to have the same special Chars. ... (3 Replies)
Discussion started by: subramanian2008
3 Replies

5. Shell Programming and Scripting

sed discard chars after last _

Hi, I'd get fields like unix_linux_form_yyyyddmmhhmi.file.txt shell_programming_and_scripting.txt so on... and want them as below unix_linux_form shell_programming_and I could remove everything after a '.' as below echo $field | sed 's/\..*//' but how to remove... (14 Replies)
Discussion started by: dips_ag
14 Replies

6. Shell Programming and Scripting

is is possible remove junk chars from the strings?

Hello friendz, dfl;g435hkd.fg ..this is what I am getting. I want to print strings without junk chars. I want to exactly like this... dflg435hkd.fg ...need some specific operators also. for example, dot or comma should allow. plz help me out ya! ~Balan:confused: (1 Reply)
Discussion started by: balan_mca
1 Replies

7. UNIX for Dummies Questions & Answers

Using SED to get n chars after given value from file

Hello, my name is Marc, I'm a linux starter :) and I hope you specialists can help me solving this issue. I have a file containing a lot of data. Somewhere in this file, there's a string called "Faultdump", directly followed by 64 chars of HEX data. I need to get the HEX part. I accomplished... (12 Replies)
Discussion started by: Kally
12 Replies

8. Shell Programming and Scripting

replace chars,

:rolleyes: Hi, I want to replace the particular text in the middle of the line. Ex. In the line 40-50 wanted to replace the char as 'x' (7 Replies)
Discussion started by: Jairaj
7 Replies

9. Shell Programming and Scripting

Clense Junk Data File - Using Shell or awk or sed

Hello Shell Gurus i need help in solving this puzzle. We have a junk data file that needs to be fed into the database. Need to clense the data file thru shell script. I am not a expert and so need help with Here is what i need to do on the input file -Step -1 Replace all pipes ‘|' within... (1 Reply)
Discussion started by: rimss
1 Replies

10. Shell Programming and Scripting

replace ascii chars without loosing it.

Hi, Can some one tell, how to replace ascii non printable TAB from the while to something, then later on replace it back to TAB. Basciallz we do bulk data processing, our processin treats TAB as new field , So I thought we can replace it with something and later on revert it. TIA (4 Replies)
Discussion started by: braindrain
4 Replies
Login or Register to Ask a Question