Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Need help to delete special characters exists only at the end of the each record in UNIX file?
Top Forums UNIX for Advanced & Expert Users Need help to delete special characters exists only at the end of the each record in UNIX file? Post 303030272 by rakeshp on Thursday 7th of February 2019 01:17:20 PM
Old 02-07-2019
HI Don,

^M This is a carriage return, usually when files are transfered from windows to unix this special character is appearing at the end of the each record.This is a Single character and It is special character. As per your previous comment, it is not alphabet 'M' and these are not two characters. It is only a single special character and it is a carriage return.

Thanks
Rakesh Puli

--- Post updated at 01:17 PM ---

Hi Don/nezabudka,

Code:

Code:
awk 'BEGIN {FS="|"; OFS="|"} {sub("[\r]$", ""); gsub("[^a-zA-Z_0-9 ]", "", $15)} 1' file

Code:
awk 'BEGIN {FS="|"; OFS="|"} {gsub("[^ [:digit:]]", "", $15)} 1

Code:
awk 'BEGIN {FS="|"; OFS="|"} {sub("[^]M$", ""); gsub("[^ [:digit:]]", "", $15)} 1' file

Code:
sed 's/[^ [:digit:]]*$//' file

The above codes didn't work for the below sample data. some extra special characters added at the end of the records by using alt keys.

alt157 will write ‘¥'
alt158 will write ‘₧'
alt162 will write ó

sample data:-

Code:
XXXX99999999|x99999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999|9999999  999999|||X ^M¥
XXXX99999999|X99999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999|||X ^M₧
XXXX99999999|X99999999999|9999999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X ^Mó
XXXX99999999|X99999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X ^M¥
XXXX99999999|X9999999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X ^Mó
XXXX99999999|X9999999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X ^Mó¥₧ó
XXXX99999999|X9999999999999|999999999|X|X|99999999|99999999|X|99999999|99999999|99999999||||X ^Mó¥₧ó

some changes in the requirement

Exclude @#!$% and space from special characters. These special characters should not be replaced. remaining all other special characters and invisible characters need to be removed from the end of the record. Please do the needful

Thanks and Regards
Rakesh

Moderator's Comments:
Mod Comment edit by bakunin: are you so callous as to ignore all the requests to use CODE-tags or are you just pretending to be? 5 points infraction for blatant disregard of the code of conduct required here. That is the first infraction i give since probably 3 or 4 years. I am perhaps the most lenient of the whole moderation team but you see i can be callous too if i need to be.
Last edited by bakunin; 02-07-2019 at 03:02 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Unix file does not display special characters

We have a unix file that contains special characters (ie. Ñ, °, É, ¿ , £ , ø ). When I try to read this file I get a codepage error and the characters are replaced by the # symbol. How do I keep the special characters from being read? Thanks. Ryan (3 Replies)
Discussion started by: Ryan2786
3 Replies

2. UNIX for Dummies Questions & Answers

How to delete a file with special characters

I don't now exactly how I did it, but I created a file named " -C " cexdi:/home1 $ls -lt total 1801336 -rw------- 1 cexdi ced-group 922275840 23 mars 10:03 -C How do I delete this file ? cexdi:/home1 $rm -C rm: invalid option -- C Syntax : rm filename ... Doesn't work...... (5 Replies)
Discussion started by: yveslagace
5 Replies

3. UNIX for Dummies Questions & Answers

Advice on extracting special characters from a DB2 table to a file in the UNIX ENV

need some advice on the following situation. I have a DB2 table which has a varchar Column. This varchar column can have special characters like ©, ®, ™ . When I extract from this table to a sequential file for this varchar column I am only able to get © and ® . To Get the ™... (1 Reply)
Discussion started by: cosec
1 Replies

4. Shell Programming and Scripting

sed delete pattern with special characters

Hi all, I have the following lines <b>A gtwrhwrthwr text hghthwrhtwrtw </b><font color='#06C'>; text text (text) <b>B gtwrhwrthwr text hghthwrhtwrtw </b><font color='#06C'>; text text (text) <b>J gtwrhwrthwr text hghthwrhtwrtw </b><font color='#06C'>; text text (text) and I would like to... (5 Replies)
Discussion started by: stinkefisch
5 Replies

5. Shell Programming and Scripting

how to delete special characters from the file content

Hello Team, Any one suggest how to delte the below special character from a file which is having one column 10 rows of same below content. ---------------------------------------- Kosten|bersicht gemd_ ' =Welche Kosten kvnnen... (2 Replies)
Discussion started by: kanakaraju
2 Replies

6. Shell Programming and Scripting

Windows to UNIX FTP Special characters!

I have a file that has the name in one of the lines as MARíA MENDOZA in Windows. When this gets FTPed over to UNIX it appears as MAR�A MENDOZA. Is there anyway to overcome this? Its causing a issue because the file is Postional and fields are getting pushed by 2 digits.. Any help would be... (4 Replies)
Discussion started by: venky338
4 Replies

7. Shell Programming and Scripting

Need unix commands to delete records from one file if the same record present in another file...

Need unix commands to delete records from one file if the same record present in another file... just like join ... if the record present in both files.. delete from first file or delete the particular record and write the unmatched records to new file.. tried with grep and while... (6 Replies)
Discussion started by: msathees
6 Replies

8. Shell Programming and Scripting

How to add trailer record at the end of the flat file in the unix ksh shell scripting?

Hi, How to add trailer record at the end of the flat file in the unix ksh shell scripting can you please let me know the procedure Regards Srikanth (3 Replies)
Discussion started by: srikanth_sagi
3 Replies

9. Shell Programming and Scripting

UNIX Special Characters

Any time I do : ls *.txt > mytext.txt I get something like this in the output file: ^ Tue Jan 22 16:19:19 EST 2013 x86_64 x86_64 x86_64 GNU/Linux t1Fam_BrOv :~>alias | grep ls alias l.='ls -d .* --color=tty' alias lR='ls -R' alias la='ls -Al' alias lc='ls -ltcr' alias ldd='ls -ltr |... (5 Replies)
Discussion started by: genehunter
5 Replies

10. Shell Programming and Scripting

Delete special characters

My sed is not working on deleting the entire special characters and leaving what is necessary.grep connections_per a|sed -e 's/\<\!\-\-//g' INPUT: <!-- <connections_per_instance>1</connections_per_instance> --> <method>HALF</method> <!--... (10 Replies)
Discussion started by: kenshinhimura
10 Replies
All times are GMT -4. The time now is 06:55 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy