Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

Need help to delete special characters exists only at the end of the each record in UNIX file?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Need help to delete special characters exists only at the end of the each record in UNIX file?
# 22  
Old 02-07-2019
Quote:
Originally Posted by rakeshp
Hi Don, ... In your code you considered only non numeric and not space special character. Please correct it to alphanumeric.

Hi rakeshp,


after 21 posts in this thread I think it is time that YOU start to take the responsibilty for YOUR problem, and don't request solution atop solution from people in here with repeated careless, insufficient, contradictory, and misleading specifications. Everything is at your fingertips:
- a plethora of sample scripts
- comments on how to improve
- man pages of the relevant tools
- internet searches


How about carefully rereading and understanding all posts, applying what you learned in them, and combining / modifying / adapting the code snippets given in this thread to a satisfying solution?
Last edited by RudiC; 02-07-2019 at 05:30 AM..
# 23  
Old 02-07-2019
Hi Rakesh,
You say that the letter M at the end of each input line is special and has to be removed. The letter M can't be removed if you say it is not special. We can't possibly meet your requirements if your requirements are inconsistent.

If you want to change your requirements to only remove sequences of characters that are not a space character and that are not alphanumeric characters from the end of each input line, the output produced from your sample input will be identical to the input because the last character is an alphanumeric character!

You had several suggestions earlier that, among many other things, removed all non-alphanumeric characters from the end of field 15 or from the end of the input record and you rejected all of them because they didn't get rid of trailing M.

I give up!
# 24  
Old 02-07-2019
HI Don,

^M This is a carriage return, usually when files are transfered from windows to unix this special character is appearing at the end of the each record.This is a Single character and It is special character. As per your previous comment, it is not alphabet 'M' and these are not two characters. It is only a single special character and it is a carriage return.

Thanks
Rakesh Puli

--- Post updated at 01:17 PM ---

Hi Don/nezabudka,

Code:

Code:
awk 'BEGIN {FS="|"; OFS="|"} {sub("[\r]$", ""); gsub("[^a-zA-Z_0-9 ]", "", $15)} 1' file

Code:
awk 'BEGIN {FS="|"; OFS="|"} {gsub("[^ [:digit:]]", "", $15)} 1

Code:
awk 'BEGIN {FS="|"; OFS="|"} {sub("[^]M$", ""); gsub("[^ [:digit:]]", "", $15)} 1' file

Code:
sed 's/[^ [:digit:]]*$//' file

The above codes didn't work for the below sample data. some extra special characters added at the end of the records by using alt keys.

alt157 will write ‘¥'
alt158 will write ‘₧'
alt162 will write ó

sample data:-

Code:
XXXX99999999|x99999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999|9999999  999999|||X ^M¥
XXXX99999999|X99999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999|||X ^M₧
XXXX99999999|X99999999999|9999999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X ^Mó
XXXX99999999|X99999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X ^M¥
XXXX99999999|X9999999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X ^Mó
XXXX99999999|X9999999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X ^Mó¥₧ó
XXXX99999999|X9999999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X ^Mó¥₧ó

some changes in the requirement

Exclude @#!$% and space from special characters. These special characters should not be replaced. remaining all other special characters and invisible characters need to be removed from the end of the record. Please do the needful

Thanks and Regards
Rakesh

Moderator's Comments:
Mod Comment edit by bakunin: are you so callous as to ignore all the requests to use CODE-tags or are you just pretending to be? 5 points infraction for blatant disregard of the code of conduct required here. That is the first infraction i give since probably 3 or 4 years. I am perhaps the most lenient of the whole moderation team but you see i can be callous too if i need to be.
Last edited by bakunin; 02-07-2019 at 03:02 PM..
# 25  
Old 02-07-2019
Since you may not be able to see the PMs here is my commentary i intended to send by PM:

Moderator's Comments:
Mod Comment Dear rakeshp,

you have been told over and over again to use CODE-tags when posting code, data or terminal output. You did not even try, you did in no way react or otherwise even acknowledge these requests. If you are not willing to play by the rules of our community we see no reason to help you in any way. This infraction will result in you being read-only for some time. Your posting privileges are removed for that time being.

Use this time to think about which relationship you want to have to our community. The way you behave right now is infuriating to most of us and that is not a healthy thing since you rely on us to solve the problems you are not able to solve yourself.

kind regards
bakunin
This User Gave Thanks to bakunin For This Post:
RudiC
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Delete special characters

My sed is not working on deleting the entire special characters and leaving what is necessary.grep connections_per a|sed -e 's/\<\!\-\-//g' INPUT: <!-- <connections_per_instance>1</connections_per_instance> --> <method>HALF</method> <!--... (10 Replies)
Discussion started by: kenshinhimura
10 Replies

2. Shell Programming and Scripting

UNIX Special Characters

Any time I do : ls *.txt > mytext.txt I get something like this in the output file: ^ Tue Jan 22 16:19:19 EST 2013 x86_64 x86_64 x86_64 GNU/Linux t1Fam_BrOv :~>alias | grep ls alias l.='ls -d .* --color=tty' alias lR='ls -R' alias la='ls -Al' alias lc='ls -ltcr' alias ldd='ls -ltr |... (5 Replies)
Discussion started by: genehunter
5 Replies

3. Shell Programming and Scripting

How to add trailer record at the end of the flat file in the unix ksh shell scripting?

Hi, How to add trailer record at the end of the flat file in the unix ksh shell scripting can you please let me know the procedure Regards Srikanth (3 Replies)
Discussion started by: srikanth_sagi
3 Replies

4. Shell Programming and Scripting

Need unix commands to delete records from one file if the same record present in another file...

Need unix commands to delete records from one file if the same record present in another file... just like join ... if the record present in both files.. delete from first file or delete the particular record and write the unmatched records to new file.. tried with grep and while... (6 Replies)
Discussion started by: msathees
6 Replies

5. Shell Programming and Scripting

Windows to UNIX FTP Special characters!

I have a file that has the name in one of the lines as MARíA MENDOZA in Windows. When this gets FTPed over to UNIX it appears as MAR�A MENDOZA. Is there anyway to overcome this? Its causing a issue because the file is Postional and fields are getting pushed by 2 digits.. Any help would be... (4 Replies)
Discussion started by: venky338
4 Replies

6. Shell Programming and Scripting

how to delete special characters from the file content

Hello Team, Any one suggest how to delte the below special character from a file which is having one column 10 rows of same below content. ---------------------------------------- Kosten|bersicht gemd_ ' =Welche Kosten kvnnen... (2 Replies)
Discussion started by: kanakaraju
2 Replies

7. Shell Programming and Scripting

sed delete pattern with special characters

Hi all, I have the following lines <b>A gtwrhwrthwr text hghthwrhtwrtw </b><font color='#06C'>; text text (text) <b>B gtwrhwrthwr text hghthwrhtwrtw </b><font color='#06C'>; text text (text) <b>J gtwrhwrthwr text hghthwrhtwrtw </b><font color='#06C'>; text text (text) and I would like to... (5 Replies)
Discussion started by: stinkefisch
5 Replies

8. UNIX for Dummies Questions & Answers

Advice on extracting special characters from a DB2 table to a file in the UNIX ENV

need some advice on the following situation. I have a DB2 table which has a varchar Column. This varchar column can have special characters like ©, ®, ™ . When I extract from this table to a sequential file for this varchar column I am only able to get © and ® . To Get the ™... (1 Reply)
Discussion started by: cosec
1 Replies

9. UNIX for Dummies Questions & Answers

How to delete a file with special characters

I don't now exactly how I did it, but I created a file named " -C " cexdi:/home1 $ls -lt total 1801336 -rw------- 1 cexdi ced-group 922275840 23 mars 10:03 -C How do I delete this file ? cexdi:/home1 $rm -C rm: invalid option -- C Syntax : rm filename ... Doesn't work...... (5 Replies)
Discussion started by: yveslagace
5 Replies

10. UNIX for Dummies Questions & Answers

Unix file does not display special characters

We have a unix file that contains special characters (ie. Ñ, °, É, ¿ , £ , ø ). When I try to read this file I get a codepage error and the characters are replaced by the # symbol. How do I keep the special characters from being read? Thanks. Ryan (3 Replies)
Discussion started by: Ryan2786
3 Replies
Login or Register to Ask a Question