Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Replacing string/special characters using a 'conversion' table
Top Forums UNIX for Beginners Questions & Answers Replacing string/special characters using a 'conversion' table Post 302977271 by bakunin on Friday 15th of July 2016 10:42:04 AM
Old 07-15-2016
Quote:
Originally Posted by newbie_01
For example, I want to replace CTRL-I with an underscore, tabs with 3 spaces, CTRL-M with UNIX's newline etc.
You can simply use sed to that that, eventually embedded in a script (tabs and spaces written as <t> and <b> for clarity, use literal tabs/spaces when writing):

Code:
#! /bin/ksh
typeset fIn="$1"

if [ ! -r "$fIn" ] ; then
     print -u2 "File $fIn not readable or not existing".
     exit 1
fi
sed 's/^I/_/g
     s/<t>/<b><b><b>/g
     s/^M$//
     [....]' "$fIn"

exit 0

Use the script like:

Code:
/path/to/script /some/input.file > /some/output.file

As you enter the script in vi notice that you can enter any non-printing character pressing <CTRL>-<V> and then your character, i.e <CTRL><I> literally.

A word of caution about "^M" characters: look at my take on it. You probably don't want to change any "^M" but only thos at line ends. The one on line ends you don't want to change into anything, just remove them. They are probably being left over from a DOS<->UNIX file transfer and as DOS has two characters signifying the line change and UNIX has only one you simply remove them.

I hope this helps.

bakunin
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

replacing string with special character ???

the problem is while replacing the old string with new one with the help of SED i am unable to replace the special characters with new strings. how can i do that? i dont want the user to be given the trouble to write '\' before every special characters like * , . , \ , $ , &. sed... (4 Replies)
Discussion started by: imppayel
4 Replies

2. Shell Programming and Scripting

Replacing French special characters

Hi, I have tonnes of .txt files that are written in French. I need to replace the French special characters, however, with English equivalents (e.g. é -> e and ç -> c). I have tried this --- #!/bin/bash # Convert French characters to normal characters # Treat each of the files exec... (4 Replies)
Discussion started by: BlueberryPickle
4 Replies

3. UNIX for Dummies Questions & Answers

Advice on extracting special characters from a DB2 table to a file in the UNIX ENV

need some advice on the following situation. I have a DB2 table which has a varchar Column. This varchar column can have special characters like ©, ®, ™ . When I extract from this table to a sequential file for this varchar column I am only able to get © and ® . To Get the ™... (1 Reply)
Discussion started by: cosec
1 Replies

4. Shell Programming and Scripting

help on sed replacing special characters

Hello, I have a file with many lines with below format: \abc\\1234 jkl\\567 def\\345 \pqr\\567 \xyz\\234 Here, i need to do 2 things. 1. replace \\ with \ 2. remove starting \ so output to be as below: (11 Replies)
Discussion started by: prvnrk
11 Replies

5. Programming

C++ Special Characters in a String?

Hello. How can i put all of the special characters on my keyboard into a string in c++ ? I tried this but it doesn't work. string characters("~`!@#$%^&*()_-+=|\}]{ How can i accomplish this? Thanks in advance. (1 Reply)
Discussion started by: cbreiny
1 Replies

6. Shell Programming and Scripting

Replacing string with special characters in shell

Hi, I am trying to replace a string in shell but it is not working correctly. @xcom.file@ needs to be replaced with tb137 Plz help.Thx. Please use and tags when posting code, data or logs etc. to preserve formatting and enhance readability, thanks. (4 Replies)
Discussion started by: manish72
4 Replies

7. Shell Programming and Scripting

Need help in replacing special characters

I am writing a ksh script. I need to replace a set of characters in an xml file. FROM="ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÛÚÜÝßàáâãäåçèéêëìíîïðñòóôõö¿¶ø®"; TO="AAAAAAACEEEEIIIIDNOOOOOOUUUUYSaaaaaaceeeeiiiionooooo N R" I have used the code- sed 's/$FROM/$TO/g'<abc.xml But its not working. Can anyone tell me the code to do this? (3 Replies)
Discussion started by: saga20
3 Replies

8. Linux

File conversion and removing special characters from a file in Linux

I have a .CSV file when I check for the special characters in the file using the command cat -vet filename.csv, i get very lengthy lines with "^@", "^I^@" and "^@^M" characters in between each alphabet in all of the records. Using the code below file filename.csv I get the output as I have a... (2 Replies)
Discussion started by: dhruuv369
2 Replies

9. UNIX for Dummies Questions & Answers

Replacing valuses containig space and special characters

**Extremely sorry for the typos in heading Old:CAST ('${DEFAULT_HIGH_DATE}' AS DATE FORMAT 'YYYY-MM-DD') New :CAST(CAST('${G_DEFAULT_HIGH_DATE}' AS DATE FORMAT 'MM-DD-YYYY') as DATE FORMAT 'YYYY-MM-DD') Need to change old format as new format cat file1 CAST ('${DEFAULT_HIGH_DATE}' AS... (1 Reply)
Discussion started by: 100bees
1 Replies

10. UNIX for Beginners Questions & Answers

Any tip to replacing the special characters in a file

Hi, Please find attached a file that has special characters on it. It is a copy and paste from a Micro$oft file. I don't want to use strings as it remove all the 'indentations' / 'formatting' so I am replacing them with space instead. I am using the sed command below sed "s/$(printf... (1 Reply)
Discussion started by: newbie_01
1 Replies

LEARN ABOUT SUNOS

iconv_ko

iconv_ko(5)							File Formats Manual						       iconv_ko(5)

NAME

       iconv_ko - code set conversions in Korean (ko) locale

AVAILABILITY

       SUNWkleu

DESCRIPTION

       The following code set conversions are supported:

		       Code Set Conversions Supported
       Code	       Symbol	       TargetCode      Symbol

       Wansung	       ko_KR-euc       Johap	       ko_KR-johap92
       Wansung	       ko_KR-euc       Packed	       ko_KR-johap
       Wansung	       ko_KR-euc       N-Byte	       ko_KR-nbyte
       Wansung	       ko_KR-euc       ISO-2022-KR     ko_KR-iso2022-7
       Johap	       ko_KR-johap92   Wansung	       ko_KR-euc
       Packed	       ko_KR-johap     Wansung	       ko_KR-euc
       N-Byte	       ko_KR-nbyte     Wansung	       ko_KR-euc
       ISO-2022-KR     ko_KR-iso2022-7 Wansung	       ko_KR-euc

       The conversions are performed as like following descriptions.

Wansung to Johap and Johap to Wansung
       For  the  conversion  between  Wansung  which is in Korean EUC representation of KS C 5601-1992 and also known as Completion code and Johap
       which is described at KS C 5601-1992 Annex 3: Supplementary Code System (2 Byte Johap Code System) and also known as Combination code,  all
       characters  can	be  mapped  if	conversion is from Wansung to Johap.  However, from Johap to Wansung conversion, certain Hangul characters
       which are not in KS C 5601-1992 precompsed character set will not be able to mapped. Such characters will be mapped into underscore charac-
       ter  (octal  value  of 137) of KS C 5636.  In case of Hanja and special symbol characters, they will follow the mapping rules of Table 3 of
       Annex 3 of KS C 5601-1992.

Wansung to Packed and Packed to Wansung
       For the conversion between Wansung and Packed which is described at KS C 5601-1987 Annex 3: Supplementary Code System (2  Byte  Johap  Code
       System),  all  characters except Hanja and special symbol characters can be mapped if conversion is from Wansung to Packed.  Such Hanja and
       special symbol characters will be mapped into underscore character (octal value 137) of KS C 5636.  From Packed to Wansung conversion, cer-
       tain  Hangul  chracters will not be able to mapped since not every Hangul Packed code is representable in Wansung.  Such characters will be
       mapped into underscore character of KS C 5636.

Wansung to N-Byte and N-Byte to Wansung
       For the conversion between Wansung and N-Byte which is described at KS C 5601-1992 Annex 4: 7 Bit Hangul  Alphabet  codes,  all	characters
       except  Hanja and special symbol characters can be mapped.  Such Hanja and special symbol characters will be mapped into underscore charac-
       ter (octal value 137) of KS C 5636.

Wansung to ISO-2022-KR and ISO-2022-KR to Wansung
       For the conversion between Wansung and ISO-2022-KR which is described at RFC 1557: Korean Character Encoding  for  Internet  Messages,  all
       characters can be mapped.

FILES

       /usr/lib/iconv/ko_KR-euc%ko_KR-johap92.so
					       Shared object for Korean EUC to Johap code conversion
       /usr/lib/iconv/ko_KR-euc%ko_KR-johap.so Shared object for Korean EUC to Packed code conversion
       /usr/lib/iconv/ko_KR-euc%ko_KR-nbyte.so Shared object for Korean EUC to N-Byte code conversion
       /usr/lib/iconv/ko_KR-euc%ko_KR-iso2022-7.so
					       Shared object for Korean EUC to ISO-2022-KR code conversion
       /usr/lib/iconv/ko_KR-johap92%ko_KR-euc.so
					       Shared object for Johap to Korean EUC code conversion
       /usr/lib/iconv/ko_KR-johap%ko_KR-euc.so Shared object for Packed to Korean EUC code conversion
       /usr/lib/iconv/ko_KR-nbyte%ko_KR-euc.so Shared object for Johap to Korean EUC code conversion
       /usr/lib/iconv/ko_KR-iso2022-7%ko_KR-euc.so
					       Shared object for ISO-2022-KR to Korean EUC code conversion

SEE ALSO

       iconv(1), iconv(3), iconv(5)

								    10 Mar 1995 						       iconv_ko(5)
All times are GMT -4. The time now is 11:06 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy