UNIX for Advanced & Expert Users

Thanks for pointing out the remaining syntax error - didn't test the command before posting. Corrected in my above post. For your other questions - I don't know what exactly the requestor is after, as the spec is unclear, so I just corrected (partly, haha) the obvious, taking for granted the rest of the original regex. Let's wait for a statement of the requestor, or, even better, a data sample.

You have specified pure ASCII characters but not said if your file(s) have had, say for example, utf-8 characters so...
...to expand on everyones' post so far, as 'sed' can be caught out with some UNICODE characters then:
IF you encounter UNICODE characters inside ANY part of your file this might work using Don's simplest version.
Longhand OSX 10.14.1, default bash terminal; BSD sed version unknown.
Note there are 6 UNICODE characters in the test string.

Code:

Last login: Tue Jan 22 21:16:39 on ttys000
AMIGA:amiga~> echo "abc\πœ†/o*&&^%&^?*s123HGFi(*&*Åå*&â" | iconv -c -f utf-8 -t ascii | sed 's/[^[:alnum:]|]*$//'
abc\/o*&&^%&^?*s123HGFi
AMIGA:amiga~> _

Hi Don Cragun/RudiC/nezabudka/All,

Now I have different requirement,. Totally i have 20 columns in each record. I need to remove ^M characters at the end of the each record and special characters in specific column like say 15th column in each record.

I have tried this

Code:

sed 's/[^\w]$//g' | awk 'BEGIN{FS="|";OFS="|"}{gsub("[^[:alnum:]]","",$15);print }'.

But this command is not working for me. Please let me know how we can write the command for this

Thanks
Rakesh

Bad idea to use sed and awk together. Try this

Code:

awk 'BEGIN {FS="|"; OFS="|"} {sub("[^[:alnum:]]$", ""); gsub("[^[:alnum:]]", "", $15)} 1'

I'm disappointed that you ignored my request for clarity in the description of your problem again and still won't show us a sample of what you're trying to do.

If we assume that when you say you say you need to "remove ^M characters at the end of each record", you really mean that you want to remove a single <carriage-return> character at the end of the record and that a <carriage-return> character exists at the end of every record you will process, then the first awk sub() call nezabudka suggested may meet that requirement. However, it might be safer to change:

Code:

sub("[^[:alnum:]]$", "");

in her suggestion to:

Code:

sub(/\r$/, "");

which will only remove a <carriage-return> character at the end of a record if there is one and won't take of chance of removing some other character if there is no <carriage-return>. If you mean that you want to remove all adjacent <circumflex> and <capital-latin-M> characters from the end of every record no matter how many of those characters appear at the end of each record, you need a completely different substitution:

Code:

sub(/[M^]*$/, "");

If we assume that by "special characters in specific column like say 15th column" you mean characters in the 15th field that are not alpha-numeric in the current locale, then the awk gsub() call nezabudka suggested will do what you want. But, of course, we really don't have any way to guess what characters you consider special since you haven't given us any definition for what you want that term to mean. And, we don't know if you really want to change the 15th field or some other field that contains data that is in some unspecified way similar to the data in the 15th field. If you mean that you wan't to specify an arbitrary field number to be processed by your awk or sed script, that of course would only be successfully met by using $15 only about 5 percent of the time in records containing twenty fields.

If nezabudka didn't correctly guess at what your requirements are, then PLEASE give us a clear English definition of what you are trying to do, show us a small representative sample input file, and show us the corresponding sample output file that contains the output you want to produce from that sample input file.

Thanks alot nezabudka. This code is working fine as of now.I will get back to you if there any more calrifications!

--- Post updated 01-30-19 at 03:04 AM ---

nezabudka/Don Cragun,

This code is working fine but it is removing space at the end of the record before the ^M, but i do not want that space to be removed.Please let me know if we can modify this code.

sample data

input

Code:

aa|bb|##$$abch^^$$|xy ^M

output

Code:

aa|bb|##$$abch^^$$|xy ^M

Thanks
Rakesh

Moderator's Comments:

Please use CODE tags as required by forum rules!

Sure that output is what you're after (with the ^M)?


Difficult to believe. The code removes ONE single non-alphanumeric character at EOL. ^M if it is there, space if it is not.
Include "space" in the set to be excluded, or try

Code:

awk 'BEGIN {FS="|"; OFS="|"} {sub("[[:cntrl:]]$", ""); gsub("[^[:alnum:]]", "", $15)} 1' file