Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Need help to delete special characters exists only at the end of the each record in UNIX file?
Top Forums UNIX for Advanced & Expert Users Need help to delete special characters exists only at the end of the each record in UNIX file? Post 303029667 by Don Cragun on Wednesday 30th of January 2019 09:09:29 AM
Old 01-30-2019
Quote:
Originally Posted by rakeshp
Hi Don,

Except space,all other non numeric characters are treated as special characters. ^M represents two characters.

Thanks
Rakesh

--- Post updated at 07:10 AM ---

Actual data contains 16fields only, but I have shown here only the sample data where I cannot show the actual data here.

--- Post updated at 07:12 AM ---

I understand that I should show the actual sample data, But i cannot do that here
I never asked you to show us actual data; I only asked you to show us representative sample data. You could easily have taken a couple of examples of real data and replaced every occurrence of a numeric character with a "9" and replaced every occurrence of a lowercase alphabetic character with an "x" and every occurrence of an uppercase alphabetic character with an "X" (except for things like <circumflex><capital-latin-M> where you should have left the "M" as it appears in your real data.

Why you would need to tell us that there are 20 fields and that you want to remove special characters from field 15 when there are only 15 fields in your actual data makes absolutely no sense to me. Why you would explicitly want us to remove the characters "^" and "M" from the 15th field (which is also the last field in your actual data) and then remove any characters that are not numeric or <space> characters from the same field makes no sense to me. Since "^" and "M" are both non-<space> and non-<digit> characters, removing what you are calling "special" characters from the 15th field will remove "^" and "M" from the end of the record without adding any special code to just remove those two characters.

If the above accurately describes your real data and what you are trying to do to it (i.e., remove all characters that are not <space> and are not numeric from the last field of each record where each record contains 15 fields), then all you need is something like:
Code:
awk 'BEGIN {FS="|"; OFS="|"} {gsub("[^ [:digit:]]", "", $15)} 1'

If you want to remove all characters that are not <space> and are not numeric from the 15th field in each record where each record contain 15 or more fields, and you also want to remove the two character sequence ^M from the end of each input record, then you need something more like:
Code:
awk 'BEGIN {FS="|"; OFS="|"} {sub("[^]M$", ""); gsub("[^ [:digit:]]", "", $15)} 1' file

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Unix file does not display special characters

We have a unix file that contains special characters (ie. Ñ, °, É, ¿ , £ , ø ). When I try to read this file I get a codepage error and the characters are replaced by the # symbol. How do I keep the special characters from being read? Thanks. Ryan (3 Replies)
Discussion started by: Ryan2786
3 Replies

2. UNIX for Dummies Questions & Answers

How to delete a file with special characters

I don't now exactly how I did it, but I created a file named " -C " cexdi:/home1 $ls -lt total 1801336 -rw------- 1 cexdi ced-group 922275840 23 mars 10:03 -C How do I delete this file ? cexdi:/home1 $rm -C rm: invalid option -- C Syntax : rm filename ... Doesn't work...... (5 Replies)
Discussion started by: yveslagace
5 Replies

3. UNIX for Dummies Questions & Answers

Advice on extracting special characters from a DB2 table to a file in the UNIX ENV

need some advice on the following situation. I have a DB2 table which has a varchar Column. This varchar column can have special characters like ©, ®, ™ . When I extract from this table to a sequential file for this varchar column I am only able to get © and ® . To Get the ™... (1 Reply)
Discussion started by: cosec
1 Replies

4. Shell Programming and Scripting

sed delete pattern with special characters

Hi all, I have the following lines <b>A gtwrhwrthwr text hghthwrhtwrtw </b><font color='#06C'>; text text (text) <b>B gtwrhwrthwr text hghthwrhtwrtw </b><font color='#06C'>; text text (text) <b>J gtwrhwrthwr text hghthwrhtwrtw </b><font color='#06C'>; text text (text) and I would like to... (5 Replies)
Discussion started by: stinkefisch
5 Replies

5. Shell Programming and Scripting

how to delete special characters from the file content

Hello Team, Any one suggest how to delte the below special character from a file which is having one column 10 rows of same below content. ---------------------------------------- Kosten|bersicht gemd_ ' =Welche Kosten kvnnen... (2 Replies)
Discussion started by: kanakaraju
2 Replies

6. Shell Programming and Scripting

Windows to UNIX FTP Special characters!

I have a file that has the name in one of the lines as MARíA MENDOZA in Windows. When this gets FTPed over to UNIX it appears as MAR�A MENDOZA. Is there anyway to overcome this? Its causing a issue because the file is Postional and fields are getting pushed by 2 digits.. Any help would be... (4 Replies)
Discussion started by: venky338
4 Replies

7. Shell Programming and Scripting

Need unix commands to delete records from one file if the same record present in another file...

Need unix commands to delete records from one file if the same record present in another file... just like join ... if the record present in both files.. delete from first file or delete the particular record and write the unmatched records to new file.. tried with grep and while... (6 Replies)
Discussion started by: msathees
6 Replies

8. Shell Programming and Scripting

How to add trailer record at the end of the flat file in the unix ksh shell scripting?

Hi, How to add trailer record at the end of the flat file in the unix ksh shell scripting can you please let me know the procedure Regards Srikanth (3 Replies)
Discussion started by: srikanth_sagi
3 Replies

9. Shell Programming and Scripting

UNIX Special Characters

Any time I do : ls *.txt > mytext.txt I get something like this in the output file: ^ Tue Jan 22 16:19:19 EST 2013 x86_64 x86_64 x86_64 GNU/Linux t1Fam_BrOv :~>alias | grep ls alias l.='ls -d .* --color=tty' alias lR='ls -R' alias la='ls -Al' alias lc='ls -ltcr' alias ldd='ls -ltr |... (5 Replies)
Discussion started by: genehunter
5 Replies

10. Shell Programming and Scripting

Delete special characters

My sed is not working on deleting the entire special characters and leaving what is necessary.grep connections_per a|sed -e 's/\<\!\-\-//g' INPUT: <!-- <connections_per_instance>1</connections_per_instance> --> <method>HALF</method> <!--... (10 Replies)
Discussion started by: kenshinhimura
10 Replies

LEARN ABOUT DEBIAN

srec_fairchild

srec_fairchild(5)						File Formats Manual						 srec_fairchild(5)

NAME

       srec_fairchild - Fairchild Fairbug file format

DESCRIPTION

       The Fairchild Fairbug format has 8-byte records.  A file begins with an address record and ends with an end-of-file record.

       There are three record types in this file format.

       Address records are of the form

								       +--+------+
								       |S | nnnn |
       indicating the address for the following data records.	       +--+------+

       Data records are of the form

							       +--+------------------+---+
							       |X | ffffffffffffffff | c |
       Each  data  record  begins with an X and always contains+8-data-bytes.--The-ff+characters are hexadecimal byte values (8 bytes).  Each data
       byte is represented by 2 hexadecimal characters.  The c character is a hex digit being the the nibble-sum of the  data  bytes.	A  1-digit
       hexadecimal checksum follows the data in each data record.  The checksum represents, in hexadecimal notation, the sum of the binary equiva-
       lents of the 16 digits in the record; the half carry from the fourth bit is ignored.  The programmer  ignores  any  character  (except  for
       address	characters  and the asterisk character, which terminates the data transfer) between a checksum and the start character of the next
       data record. This space can be used for comments.

       The end-of-file record has the form

									   +--+
									   |* |
       The last record consists of an asterisk only, which indicates the end-of file.

   Size Multiplier
       In general, binary data will expand in sized by approximately 2.4 times when represented with this format.

EXAMPLE

       Here is an example Fairchild Fairbug file.  It contains the data "Hello, World[rq] to be loaded at address 0x1000.   Notice  how  the  last
       record is padded with 0xFF bytes.
	      S1000
	      X48656C6C6F2C2057C
	      X6F726C64210AFFFF3
	      *

COPYRIGHT

       srec_cat version 1.58
       Copyright (C) 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011 Peter Miller

       The srec_cat program comes with ABSOLUTELY NO WARRANTY; for details use the 'srec_cat -VERSion License' command.  This is free software and
       you are welcome to redistribute it under certain conditions; for details use the 'srec_cat -VERSion License' command.

AUTHOR

       Peter Miller   E-Mail:	pmiller@opensource.org.au
       //*		 WWW:	http://miller.emu.id.au/pmiller/

Reference Manual						      SRecord							 srec_fairchild(5)
All times are GMT -4. The time now is 01:14 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy