Identifying invisible characters in Unix file Post: 70493

10 More Discussions You Might Find Interesting

1. Programming

Identifying and removing control characters in a file.

What is the best method to identify an remove control characters in a file. Would it be easier to do this in Unix or in C.

2. Shell Programming and Scripting

Limit of no of characters PER LINE in a unix file

Hi , Whats the limit of characters PER LINE in a unix file , allowed for editing..sort , cut , sed , awk etc

3. UNIX for Dummies Questions & Answers

Unix file does not display special characters

We have a unix file that contains special characters (ie. �, �, �, � , � , � ). When I try to read this file I get a codepage error and the characters are replaced by the # symbol. How do I keep the special characters from being read? Thanks. Ryan

4. Shell Programming and Scripting

Email a File from UNIX which has Japanese characters in it

Hi, I'm trying to email from UNIX, a file which has Japanese characters in it (i,e. in the contents -- not the filename). The file gets emailed, but the Japanese characters do not show up properly when I open the file on Windows in my Outlook mailbox. I searched a lot of forums but still...

5. Shell Programming and Scripting

Unix Script file to Append Characters before rows in file.

Hi Experts, I am working on HP-UX. I am new to shell scripting. I would like to have a shell script which will prefix: 1. "H|" before first row of my file and, 2. "T" for all other rows of the file. For Example - File before running the script 20100430|4123451810|218.50|TC 20100430 ...

6. UNIX for Dummies Questions & Answers

How to paste Kanji Characters into a Unix File

Hi, I am unable to copy Kanji characters into a unix file. They look like special characters when pasted into the Unix file. My objective is to copy these characters into a unix file and be able to print it and see the Kanji characters. Any help would be greatly appreciated. I am trying this...

7. Shell Programming and Scripting

Handling Invisible character in a file

Hi Experts, When i am trying to read a csv file ,i could find some invisible character in it. I tried to see those characters by following code od -c filename It is displaying 240 for those invisible character. can some one elobrate on this and provide solution remove those character from...

8. Shell Programming and Scripting

Replace first 3 characters in a unix file in all lines

Replace first 3 characters in a unix file (say replace "A&B" with "C&D") in all lines of the file. Need a sed or awk script to do this. Kindly help! -Kumar

9. UNIX for Beginners Questions & Answers

To remove any invisible and special characters from the file(exclude @!#$&*)

Hi Guys, My requirement is to remove any invisible and special characters from the file like control M(carriage return) and alt numerics and it should not replace @#!$% abc|xyz|acd�� adc|123| 12�� Please help on this. Thanks Rakesh

10. UNIX for Advanced & Expert Users

To remove any invisible and special characters from the file(exclude @#!$*)

LEARN ABOUT SUNOS

iconv_ko

iconv_ko(5)							File Formats Manual						       iconv_ko(5)

NAME

       iconv_ko - code set conversions in Korean (ko) locale

AVAILABILITY

       SUNWkleu

DESCRIPTION

       The following code set conversions are supported:

		       Code Set Conversions Supported
       Code	       Symbol	       TargetCode      Symbol

       Wansung	       ko_KR-euc       Johap	       ko_KR-johap92
       Wansung	       ko_KR-euc       Packed	       ko_KR-johap
       Wansung	       ko_KR-euc       N-Byte	       ko_KR-nbyte
       Wansung	       ko_KR-euc       ISO-2022-KR     ko_KR-iso2022-7
       Johap	       ko_KR-johap92   Wansung	       ko_KR-euc
       Packed	       ko_KR-johap     Wansung	       ko_KR-euc
       N-Byte	       ko_KR-nbyte     Wansung	       ko_KR-euc
       ISO-2022-KR     ko_KR-iso2022-7 Wansung	       ko_KR-euc

       The conversions are performed as like following descriptions.

Wansung to Johap and Johap to Wansung
       For  the  conversion  between  Wansung  which is in Korean EUC representation of KS C 5601-1992 and also known as Completion code and Johap
       which is described at KS C 5601-1992 Annex 3: Supplementary Code System (2 Byte Johap Code System) and also known as Combination code,  all
       characters  can	be  mapped  if	conversion is from Wansung to Johap.  However, from Johap to Wansung conversion, certain Hangul characters
       which are not in KS C 5601-1992 precompsed character set will not be able to mapped. Such characters will be mapped into underscore charac-
       ter  (octal  value  of 137) of KS C 5636.  In case of Hanja and special symbol characters, they will follow the mapping rules of Table 3 of
       Annex 3 of KS C 5601-1992.

Wansung to Packed and Packed to Wansung
       For the conversion between Wansung and Packed which is described at KS C 5601-1987 Annex 3: Supplementary Code System (2  Byte  Johap  Code
       System),  all  characters except Hanja and special symbol characters can be mapped if conversion is from Wansung to Packed.  Such Hanja and
       special symbol characters will be mapped into underscore character (octal value 137) of KS C 5636.  From Packed to Wansung conversion, cer-
       tain  Hangul  chracters will not be able to mapped since not every Hangul Packed code is representable in Wansung.  Such characters will be
       mapped into underscore character of KS C 5636.

Wansung to N-Byte and N-Byte to Wansung
       For the conversion between Wansung and N-Byte which is described at KS C 5601-1992 Annex 4: 7 Bit Hangul  Alphabet  codes,  all	characters
       except  Hanja and special symbol characters can be mapped.  Such Hanja and special symbol characters will be mapped into underscore charac-
       ter (octal value 137) of KS C 5636.

Wansung to ISO-2022-KR and ISO-2022-KR to Wansung
       For the conversion between Wansung and ISO-2022-KR which is described at RFC 1557: Korean Character Encoding  for  Internet  Messages,  all
       characters can be mapped.

FILES

       /usr/lib/iconv/ko_KR-euc%ko_KR-johap92.so
					       Shared object for Korean EUC to Johap code conversion
       /usr/lib/iconv/ko_KR-euc%ko_KR-johap.so Shared object for Korean EUC to Packed code conversion
       /usr/lib/iconv/ko_KR-euc%ko_KR-nbyte.so Shared object for Korean EUC to N-Byte code conversion
       /usr/lib/iconv/ko_KR-euc%ko_KR-iso2022-7.so
					       Shared object for Korean EUC to ISO-2022-KR code conversion
       /usr/lib/iconv/ko_KR-johap92%ko_KR-euc.so
					       Shared object for Johap to Korean EUC code conversion
       /usr/lib/iconv/ko_KR-johap%ko_KR-euc.so Shared object for Packed to Korean EUC code conversion
       /usr/lib/iconv/ko_KR-nbyte%ko_KR-euc.so Shared object for Johap to Korean EUC code conversion
       /usr/lib/iconv/ko_KR-iso2022-7%ko_KR-euc.so
					       Shared object for ISO-2022-KR to Korean EUC code conversion

SEE ALSO

       iconv(1), iconv(3), iconv(5)

								    10 Mar 1995 						       iconv_ko(5)