UNIX for Advanced & Expert Users

if we only assume that the fifteenth field is the last at the same time

Code:

awk 'BEGIN {FS="|"; OFS="|"} {gsub("[^a-zA-Z0-9 ]", "", $FN)} 1' file

Moderator's Comments:

Please use CODE tags as required by forum rules!

--- Post updated at 09:19 ---

Code:

awk 'BEGIN {FS="|"; OFS="|"} {sub("[\r]$", ""); gsub("[^a-zA-Z_0-9 ]", "", $15)} 1' file

gsub("[^[:alnum:][:space:]]"

Of course it is being removed. You refuse to define what you believe are special characters and the code that has been supplied assumes that all non-alphanumeric characters except (in some cases) the <vertical-bar> character are special (and that includes <space>). And you refuse to tell us whether ^M represents the two characters ^ and M or represent a single <carriage-return> character.

If you keep telling us that our code isn't working without answering our questions, we'll continue to make bad guesses about what you really mean and we'll all continue to be frustrated.

What command (including the utility name and the options you gave it) did you use to display the sample input and output you showed us in post #13 in this thread?

How do you expect a line containing three field delimiters to have twenty fields? The data you showed us in post #13 can't possibly be related to the problem you're trying to solve in this thread.

Please show us some representative sample input data and then show us the output you are hoping to get from that sample input.

Please answer our questions and help us help you! If you continue to refuse to answer our questions, it is obvious that we won't be able to guess at what you're really trying to do and we are all just wasting our time trying to help you.

Hi Don,

Except space,all other non numeric characters are treated as special characters. ^M represents two characters.

Thanks
Rakesh

--- Post updated at 07:10 AM ---

Actual data contains 16fields only, but I have shown here only the sample data where I cannot show the actual data here.

--- Post updated at 07:12 AM ---

I understand that I should show the actual sample data, But i cannot do that here

I never asked you to show us actual data; I only asked you to show us representative sample data. You could easily have taken a couple of examples of real data and replaced every occurrence of a numeric character with a "9" and replaced every occurrence of a lowercase alphabetic character with an "x" and every occurrence of an uppercase alphabetic character with an "X" (except for things like <circumflex><capital-latin-M> where you should have left the "M" as it appears in your real data.

Why you would need to tell us that there are 20 fields and that you want to remove special characters from field 15 when there are only 15 fields in your actual data makes absolutely no sense to me. Why you would explicitly want us to remove the characters "^" and "M" from the 15th field (which is also the last field in your actual data) and then remove any characters that are not numeric or <space> characters from the same field makes no sense to me. Since "^" and "M" are both non-<space> and non-<digit> characters, removing what you are calling "special" characters from the 15th field will remove "^" and "M" from the end of the record without adding any special code to just remove those two characters.

If the above accurately describes your real data and what you are trying to do to it (i.e., remove all characters that are not <space> and are not numeric from the last field of each record where each record contains 15 fields), then all you need is something like:

Code:

awk 'BEGIN {FS="|"; OFS="|"} {gsub("[^ [:digit:]]", "", $15)} 1'

If you want to remove all characters that are not <space> and are not numeric from the 15th field in each record where each record contain 15 or more fields, and you also want to remove the two character sequence ^M from the end of each input record, then you need something more like:

Code:

awk 'BEGIN {FS="|"; OFS="|"} {sub("[^]M$", ""); gsub("[^ [:digit:]]", "", $15)} 1' file

Hi Don Cragun/RudiC/nezabudka/All,

As you suggested,Please see the sample data below replaced from actual data

Code:

XXXX99999999|x99999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999|9999999  999999|||X ^M
XXXX99999999|X99999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999|||X ^M
XXXX99999999|X99999999999|9999999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X ^M
XXXX99999999|X99999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X ^M
XXXX99999999|X9999999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X ^M

Thanks
Rakesh Puli

Thank you. That is a good start.

We now know that your input looks like what you described in post #1 in this thread and that it does not look like what you described on post #10 (where there were 20 fields and two of those fields were to be changed) and it does not look like what you described in post #13 (where there were 4 fields and absolutely nothing was supposed to be changed) and it does not like what you described in post #17 update 1 (where there were 16 fields).

In addition to the number of fields changing, your description of what characters are to be considered "special" seems to vary from post to post.

If what you want to do is:then the following command should do what you want:

Code:

sed 's/[^ [:digit:]]*$//' file

This will not be sufficient if any other field is to be modified and it will not be sufficient if any other character is not to be treated as special.

Since you still have not shown us what output you hope to produce from the above sample input, we have no way of knowing which of your many different statements about what changes are to be made is the correct set of requirements. The above code produces the output:

Code:

XXXX99999999|x99999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999|9999999  999999|||X 
XXXX99999999|X99999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999|||X 
XXXX99999999|X99999999999|9999999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X 
XXXX99999999|X99999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X 
XXXX99999999|X9999999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X

Note that the above output include a single <space> character at the end of each line.

Hi Don,

I am considering a character is special, if it not a alphanumeric and not space. In your code you considered only non numeric and not space special character. Please correct it to alphanumeric. I think i might mentioned it wrongly in the previous post. Sorry about that. I have only 15 fields as of now in each record. But I mentioned there were 20 fields in the each record in post#13. It is due to the requirement might change in future.

Thanks
Rakesh