Shell Programming and Scripting

Hi All

I have a whole block of pages that have come in from various sources, unfortunately the pages in many instances have blocks of corrupted text. What I'm trying to do is write a sed line that will just delete non alphanumeric characters if they're in a block of say three or four characters, i.e.

constipated would stay the same

con5tipated would stay the same

con%^|pated would stay the same

&*^%%pated would stay the same

^& would stay the same

&^*^ would get deleted

I was thinking along the lines of....

Code:

sed -r 's/^.*[[:punct:]]//g'

However this seems to delete anything with a punctuation character in the block even if they are valid alphanumerics.

I'm familiar with using /b for word blocks but unless I can get the core sed to work I'm stuck.

Could anyone possibly offer some pointers with an explanation of why my example doesn't work and there's does, that way it helps me learn.

Thanks in advance.

I am not sure if I understand what you would like to achieve, but how about this for starters:

Code:

sed 's/[[:punct:]]\{4,\}//g' infile

Maybe I am missing something but why should

Code:

&*^%%pated

not be changed but

Code:

&^*^

would be deleted?

Hi

Because there's fuzzy searching on keywords against the page to happen....

So you may be able to get a 'hit' on %&^%pated which will allow you to see manually if it's a match whereas ^*&^ (or any block of consecutive) non alpha characters are always a dud.

It's the results of the fuzzy search I'm really interested in so I don't want to delete too much data that may 'hit' but I do want to delete as much 'garbage' as possible to speed up search times.

Hope that makes it clearer?

Scrutinizer

Thanks for the code but it's not working the way I expected...for example

Code:

echo "&^^%% a" | sed  -r 's/[[:punct:]]\{4,\}//g'

outputs &^^%% a

whereas I expected it to output

a

Have I missed something?

I'm using Fedora 13

Hi, yes you should lose -r

Thanks that got it working.