Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Need help to delete special characters exists only at the end of the each record in UNIX file?
Top Forums UNIX for Advanced & Expert Users Need help to delete special characters exists only at the end of the each record in UNIX file? Post 303029667 by Don Cragun on Wednesday 30th of January 2019 09:09:29 AM
Old 01-30-2019
Quote:
Originally Posted by rakeshp
Hi Don,

Except space,all other non numeric characters are treated as special characters. ^M represents two characters.

Thanks
Rakesh

--- Post updated at 07:10 AM ---

Actual data contains 16fields only, but I have shown here only the sample data where I cannot show the actual data here.

--- Post updated at 07:12 AM ---

I understand that I should show the actual sample data, But i cannot do that here
I never asked you to show us actual data; I only asked you to show us representative sample data. You could easily have taken a couple of examples of real data and replaced every occurrence of a numeric character with a "9" and replaced every occurrence of a lowercase alphabetic character with an "x" and every occurrence of an uppercase alphabetic character with an "X" (except for things like <circumflex><capital-latin-M> where you should have left the "M" as it appears in your real data.

Why you would need to tell us that there are 20 fields and that you want to remove special characters from field 15 when there are only 15 fields in your actual data makes absolutely no sense to me. Why you would explicitly want us to remove the characters "^" and "M" from the 15th field (which is also the last field in your actual data) and then remove any characters that are not numeric or <space> characters from the same field makes no sense to me. Since "^" and "M" are both non-<space> and non-<digit> characters, removing what you are calling "special" characters from the 15th field will remove "^" and "M" from the end of the record without adding any special code to just remove those two characters.

If the above accurately describes your real data and what you are trying to do to it (i.e., remove all characters that are not <space> and are not numeric from the last field of each record where each record contains 15 fields), then all you need is something like:
Code:
awk 'BEGIN {FS="|"; OFS="|"} {gsub("[^ [:digit:]]", "", $15)} 1'

If you want to remove all characters that are not <space> and are not numeric from the 15th field in each record where each record contain 15 or more fields, and you also want to remove the two character sequence ^M from the end of each input record, then you need something more like:
Code:
awk 'BEGIN {FS="|"; OFS="|"} {sub("[^]M$", ""); gsub("[^ [:digit:]]", "", $15)} 1' file

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Unix file does not display special characters

We have a unix file that contains special characters (ie. Ñ, °, É, ¿ , £ , ø ). When I try to read this file I get a codepage error and the characters are replaced by the # symbol. How do I keep the special characters from being read? Thanks. Ryan (3 Replies)
Discussion started by: Ryan2786
3 Replies

2. UNIX for Dummies Questions & Answers

How to delete a file with special characters

I don't now exactly how I did it, but I created a file named " -C " cexdi:/home1 $ls -lt total 1801336 -rw------- 1 cexdi ced-group 922275840 23 mars 10:03 -C How do I delete this file ? cexdi:/home1 $rm -C rm: invalid option -- C Syntax : rm filename ... Doesn't work...... (5 Replies)
Discussion started by: yveslagace
5 Replies

3. UNIX for Dummies Questions & Answers

Advice on extracting special characters from a DB2 table to a file in the UNIX ENV

need some advice on the following situation. I have a DB2 table which has a varchar Column. This varchar column can have special characters like ©, ®, ™ . When I extract from this table to a sequential file for this varchar column I am only able to get © and ® . To Get the ™... (1 Reply)
Discussion started by: cosec
1 Replies

4. Shell Programming and Scripting

sed delete pattern with special characters

Hi all, I have the following lines <b>A gtwrhwrthwr text hghthwrhtwrtw </b><font color='#06C'>; text text (text) <b>B gtwrhwrthwr text hghthwrhtwrtw </b><font color='#06C'>; text text (text) <b>J gtwrhwrthwr text hghthwrhtwrtw </b><font color='#06C'>; text text (text) and I would like to... (5 Replies)
Discussion started by: stinkefisch
5 Replies

5. Shell Programming and Scripting

how to delete special characters from the file content

Hello Team, Any one suggest how to delte the below special character from a file which is having one column 10 rows of same below content. ---------------------------------------- Kosten|bersicht gemd_ ' =Welche Kosten kvnnen... (2 Replies)
Discussion started by: kanakaraju
2 Replies

6. Shell Programming and Scripting

Windows to UNIX FTP Special characters!

I have a file that has the name in one of the lines as MARíA MENDOZA in Windows. When this gets FTPed over to UNIX it appears as MAR�A MENDOZA. Is there anyway to overcome this? Its causing a issue because the file is Postional and fields are getting pushed by 2 digits.. Any help would be... (4 Replies)
Discussion started by: venky338
4 Replies

7. Shell Programming and Scripting

Need unix commands to delete records from one file if the same record present in another file...

Need unix commands to delete records from one file if the same record present in another file... just like join ... if the record present in both files.. delete from first file or delete the particular record and write the unmatched records to new file.. tried with grep and while... (6 Replies)
Discussion started by: msathees
6 Replies

8. Shell Programming and Scripting

How to add trailer record at the end of the flat file in the unix ksh shell scripting?

Hi, How to add trailer record at the end of the flat file in the unix ksh shell scripting can you please let me know the procedure Regards Srikanth (3 Replies)
Discussion started by: srikanth_sagi
3 Replies

9. Shell Programming and Scripting

UNIX Special Characters

Any time I do : ls *.txt > mytext.txt I get something like this in the output file: ^ Tue Jan 22 16:19:19 EST 2013 x86_64 x86_64 x86_64 GNU/Linux t1Fam_BrOv :~>alias | grep ls alias l.='ls -d .* --color=tty' alias lR='ls -R' alias la='ls -Al' alias lc='ls -ltcr' alias ldd='ls -ltr |... (5 Replies)
Discussion started by: genehunter
5 Replies

10. Shell Programming and Scripting

Delete special characters

My sed is not working on deleting the entire special characters and leaving what is necessary.grep connections_per a|sed -e 's/\<\!\-\-//g' INPUT: <!-- <connections_per_instance>1</connections_per_instance> --> <method>HALF</method> <!--... (10 Replies)
Discussion started by: kenshinhimura
10 Replies

LEARN ABOUT OPENDARWIN

cap_mkdb

CAP_MKDB(1)						    BSD General Commands Manual 					       CAP_MKDB(1)

NAME

     cap_mkdb -- create capability database

SYNOPSIS

     cap_mkdb [-v] [-f outfile] file1 [file2 ...]

DESCRIPTION

     Cap_mkdb builds a hashed database out of the getcap(3) logical database constructed by the concatenation of the specified files .

     The database is named by the basename of the first file argument and the string ``.db''.  The getcap(3) routines can access the database in
     this form much more quickly than they can the original text file(s).

     The ``tc'' capabilities of the records are expanded before the record is stored into the database.

     The options as as follows:

     -f outfile
	     Specify a different database basename.

     -v      Print out the number of capability records in the database.

FORMAT

     Each record is stored in the database using two different types of keys.

     The first type is a key which consists of the first capability of the record (not including the trailing colon (``:'')) with a data field
     consisting of a special byte followed by the rest of the record.  The special byte is either a 0 or 1, where a 0 means that the record is
     okay, and a 1 means that there was a ``tc'' capability in the record that couldn't be expanded.

     The second type is a key which consists of one of the names from the first capability of the record with a data field consisting a special
     byte followed by the the first capability of the record.  The special byte is a 2.

     In normal operation names are looked up in the database, resulting in a key/data pair of the second type.	The data field of this key/data
     pair is used to look up a key/data pair of the first type which has the real data associated with the name.

RETURN VALUE

     The cap_mkdb utility exits 0 on success and >0 if an error occurs.

SEE ALSO

     dbopen(3), getcap(3), termcap(5)

BSD
								   June 2, 2019 							       BSD
All times are GMT -4. The time now is 10:46 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy