UNIX for Advanced & Expert Users

Hi,

I have a file in unix with 15 columns.It consists special characters(#,$,^M,@,*,% etc)at the end of the each record.I want to remove these special characters.I used the following:

Code:

Sed -e 's[^a-zA-Z|0-9]/ /g;s/  */ /g'

. But It is removing special characters exists everywhere in the file(begining,middle and end of the file). But I want to remove the special characters appearing only at the end of the each record. I do not want to delete the special characters which are present in the begining or middle of the record. The Unix operating system is AIX.Please do the needful.

Moderator's Comments:

Please use CODE tags when displaying sample input, sample output, and code segments.

Thanks
Rakesh

Hi, rakeshp
Try this. Will only delete the last one

Code:

sed 's/[^\w]$//'

or. Will delete range of last characters

Code:

sed 's/[^\w]\+$//'

Hi Rakesh,
Please note that UNIX shells and utilities are very picky about spelling, capitalization, and quoting. The utility named sed will not be found if you spell it with a leading capital letter (as in Sed as shown in your sample code above) and the single-quote characters surrounding the last argument to sed must be the single-quote character that is found as the unshifted character on the key that has the double-quote character as the shifted character on a US keyboard. The pretty-printed characters in the Sed command you showed us above will cause a syntax error and a mismatched quotes error.

Also note that ^M is two characters (one that is special in some, but not all, circumstances and one that is seldom special). Some utilities will print control characters as a two-character sequence (with ^M representing the <carriage-return> character), but you can't search for control characters using these two-character sequences in a regular expression.

Also note that your specification is ambiguous. Are you trying to get rid of a single special character at the end of a line or are you trying to get rid of all adjacent special characters at the end of a line? Please give us a more precise definition of what you're trying to do and show us a small, representative sample input file (in CODE tags) that shows us what the input files you will be processing look like and show us a corresponding sample output file (in CODE tags) that shows us the output you hope to produce. Producing a list of special characters ending with "etc." leaves us guessing at what you mean by special. Is your definition of special anything that is not an alphanumeric character in your current locale? If it is, please confirm that this is what you mean. If not, please provide a definitive definition of the term special character that you are using.

Since you're converting all non-alphanumeric characters to <space>s, you seem to be indicating that <space> is to be treated as a special character. Does that mean that a single <space> or all <space>s at the end of a line should be deleted?

Your code seems to be trying to change all occurrences of two or more adjacent <space> characters to a single <space> in your output. You didn't say anything about doing that in the description of what you're trying to do. Is this a requirement or just a side effect of the way you're converting non-alphanumeric characters to <space> as an intermediate step in your algorithm?

Hi nezabudka,
Note that rakeshp is using AIX. The GNU sed \w escape sequence to match a "word" in a BRE is not supported by AIX (and is not allowed by the POSIX standards for any version of sed).

Hi nezabudka,

Thank you for the reply. sed 's/[^\w]$//'. This code is working good so far in AIX. Can we use the same code in Linux as well?

Thanks
Rakesh

--- Post updated at 02:05 PM ---

Hi Don Cragun,

Thank you for the reply and corrections in the code. I am trying to get rid of single special character and all adjacent special characters at the end of a line. I am not trying to convert any non-alphanumeric characters to <space>.I am only trying to get rid of single special/adjacent special characters appearing at the end of the each record only.

Thanks
Rakesh

You weren't too far off in your own attempt. Try, after correcting the already commented on syntax errors and anchoring the regex at the line end,

Code:

sed 's/[^a-zA-Z|0-9]$/ /; s/  */ /g' file

I'm surprised that \w is working for you in AIX, but I'm glad that nezabudka's suggestion is working for you. (Even if \w is recognized as special in a BRE, I would not have expected it to treat the vertical bar symbol (|) as a regular character. Note that the BRE in the substitute command you were using (after inserting the missing slash character /) was:

Code:

s/[^a-zA-Z|0-9]/ /g

would match any character that was not lowercase alphabetic, was not uppercase alphabetic, was not numeric, and was not a vertical bar character and change each of the matched characters to a <space>. So, I assumed you want the vertical bar character to be treated as a normal character. Is \w really treating the vertical bar character the way you want it to?)

Something that should work with any standard sed (assuming that your definition of a special character is anything that is not an alphanumeric character and is not a vertical bar symbol) would be:

Code:

sed 's/[^[:alnum:]|]*$//'

Hi RudiC,
I'm afraid that still won't work:Point #1 above leads to a syntax error. The other two points don't meet the (still poorly specified) requirements for this problem.

Staying closer to what was given originally, we could try:

Code:

sed 's/[^a-zA-Z|0-9]*$/ /g; s/  *$//' file

but that just uses two substitutions to do what I think I did in post #6 with a single substitution.

You're usually better with sed than I am. Am I missing something here?

~ Don