Shell Programming and Scripting

Hi Techies,

I need a help in finding junk characters and remove them from a Datafile.

we have a file and it had crores of records like below

Code:

SGSN_MCC_MNC=01150

but sometime due to the issue with sending server we are getting some junk characters in the middle of data like below

Code:

SGSN_MCC_MNC=011;50
SGSN_MCC_MNC=011>60
SGSN_MCC_MNC=011/76
SGSN_MCC_MNC=011<89
SGSN_MCC_MNC=011"50
SGSN_MCC_MNC=011^50

we are not sure what kinda junk character would come in between the data.

can you please help me in getting rid of this junk character from this field?

i tried to write some script with "zgrep -c" but failed. please help.

Thanks a ton in Advance!!

With the help of tr you can delete all characters not in a set of specified characters:

Code:

tr -dc '[A-Za-z0-9_\12]' <input >output

Don't forget to include the newline (\12), or all lines will be join to one single line.

Thanks for the Reply, apparently it doesn't work as the file is .gz type.

Code:

awk 'gsub(/\;|\>|\<|\"|\^|\//,X,$0)' file

Then use

Code:

gzip -dc input.gz | tr -dc '[A-Za-z0-9_\12]' | gzip -c >output.gz

or

Code:

gzip -dc input.gz | tr -dc '[A-Za-z0-9_\12]' >output

if you want the output to be uncompressed.

Thanks but neither of the suggestion don't work

WHAT does not work? Post the (partial) output of the first part of the pipe; redirect it to a file and apply the tr command on that. Post that result as well.